From webhook-mailer at python.org Sat Jul 1 05:54:35 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 01 Jul 2023 09:54:35 -0000 Subject: [Python-checkins] build(deps-dev): bump mypy from 1.3.0 to 1.4.1 in /Tools/clinic (#106305) Message-ID: https://github.com/python/cpython/commit/c336b4c14fea17763ee0d3acbb8e9155beb617e0 commit: c336b4c14fea17763ee0d3acbb8e9155beb617e0 branch: main author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> committer: AlexWaygood date: 2023-07-01T09:54:32Z summary: build(deps-dev): bump mypy from 1.3.0 to 1.4.1 in /Tools/clinic (#106305) Bumps [mypy](https://github.com/python/mypy) from 1.3.0 to 1.4.1. - [Commits](https://github.com/python/mypy/compare/v1.3.0...v1.4.1) --- updated-dependencies: - dependency-name: mypy dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> files: M Tools/clinic/requirements-dev.txt diff --git a/Tools/clinic/requirements-dev.txt b/Tools/clinic/requirements-dev.txt index 154934003c31c..e9529f3527e95 100644 --- a/Tools/clinic/requirements-dev.txt +++ b/Tools/clinic/requirements-dev.txt @@ -1,2 +1,2 @@ # Requirements file for external linters and checks we run on Tools/clinic/ in CI -mypy==1.3.0 +mypy==1.4.1 From webhook-mailer at python.org Sat Jul 1 06:24:01 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 01 Jul 2023 10:24:01 -0000 Subject: [Python-checkins] build(deps): bump mheap/github-action-required-labels from 4 to 5 (#106306) Message-ID: https://github.com/python/cpython/commit/d3abc9b5165eee7dbe24f8f081c0739e6ad3ef3b commit: d3abc9b5165eee7dbe24f8f081c0739e6ad3ef3b branch: main author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> committer: hugovk date: 2023-07-01T13:23:57+03:00 summary: build(deps): bump mheap/github-action-required-labels from 4 to 5 (#106306) Bumps [mheap/github-action-required-labels](https://github.com/mheap/github-action-required-labels) from 4 to 5. - [Release notes](https://github.com/mheap/github-action-required-labels/releases) - [Commits](https://github.com/mheap/github-action-required-labels/compare/v4...v5) --- updated-dependencies: - dependency-name: mheap/github-action-required-labels dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> files: M .github/workflows/require-pr-label.yml diff --git a/.github/workflows/require-pr-label.yml b/.github/workflows/require-pr-label.yml index 88aaea039f04f..9327b43ae0271 100644 --- a/.github/workflows/require-pr-label.yml +++ b/.github/workflows/require-pr-label.yml @@ -15,7 +15,7 @@ jobs: timeout-minutes: 10 steps: - - uses: mheap/github-action-required-labels at v4 + - uses: mheap/github-action-required-labels at v5 with: mode: exactly count: 0 From webhook-mailer at python.org Sat Jul 1 06:28:10 2023 From: webhook-mailer at python.org (iritkatriel) Date: Sat, 01 Jul 2023 10:28:10 -0000 Subject: [Python-checkins] gh-106149: move unconditional jump direction resolution from optimizer to assembler (#106291) Message-ID: https://github.com/python/cpython/commit/200f2554114f3d40684af0969fef6af875cb1462 commit: 200f2554114f3d40684af0969fef6af875cb1462 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-01T11:28:07+01:00 summary: gh-106149: move unconditional jump direction resolution from optimizer to assembler (#106291) files: M Python/assemble.c M Python/flowgraph.c M Python/opcode_metadata.h M Tools/cases_generator/generate_cases.py diff --git a/Python/assemble.c b/Python/assemble.c index 5d566a381b398..ff7bca22286cd 100644 --- a/Python/assemble.c +++ b/Python/assemble.c @@ -674,11 +674,45 @@ resolve_jump_offsets(instr_sequence *instrs) return SUCCESS; } +static int +resolve_unconditional_jumps(instr_sequence *instrs) +{ + /* Resolve directions of unconditional jumps */ + + for (int i = 0; i < instrs->s_used; i++) { + instruction *instr = &instrs->s_instrs[i]; + bool is_forward = (instr->i_oparg > i); + switch(instr->i_opcode) { + case JUMP: + assert(SAME_OPCODE_METADATA(JUMP, JUMP_FORWARD)); + assert(SAME_OPCODE_METADATA(JUMP, JUMP_BACKWARD)); + instr->i_opcode = is_forward ? JUMP_FORWARD : JUMP_BACKWARD; + break; + case JUMP_NO_INTERRUPT: + assert(SAME_OPCODE_METADATA(JUMP_NO_INTERRUPT, JUMP_FORWARD)); + assert(SAME_OPCODE_METADATA(JUMP_NO_INTERRUPT, JUMP_BACKWARD_NO_INTERRUPT)); + instr->i_opcode = is_forward ? + JUMP_FORWARD : JUMP_BACKWARD_NO_INTERRUPT; + break; + default: + if (OPCODE_HAS_JUMP(instr->i_opcode) && + IS_PSEUDO_INSTR(instr->i_opcode)) { + Py_UNREACHABLE(); + } + } + } + return SUCCESS; +} + PyCodeObject * _PyAssemble_MakeCodeObject(_PyCompile_CodeUnitMetadata *umd, PyObject *const_cache, PyObject *consts, int maxdepth, instr_sequence *instrs, int nlocalsplus, int code_flags, PyObject *filename) { + + if (resolve_unconditional_jumps(instrs) < 0) { + return NULL; + } if (resolve_jump_offsets(instrs) < 0) { return NULL; } diff --git a/Python/flowgraph.c b/Python/flowgraph.c index 429109b74beab..213c993bb863a 100644 --- a/Python/flowgraph.c +++ b/Python/flowgraph.c @@ -393,24 +393,17 @@ no_redundant_jumps(cfg_builder *g) { static int normalize_jumps_in_block(cfg_builder *g, basicblock *b) { cfg_instr *last = _PyCfg_BasicblockLastInstr(b); - if (last == NULL || !is_jump(last)) { + if (last == NULL || !is_jump(last) || + IS_UNCONDITIONAL_JUMP_OPCODE(last->i_opcode)) { return SUCCESS; } assert(!IS_ASSEMBLER_OPCODE(last->i_opcode)); + bool is_forward = last->i_target->b_visited == 0; - switch(last->i_opcode) { - case JUMP: - assert(SAME_OPCODE_METADATA(JUMP, JUMP_FORWARD)); - assert(SAME_OPCODE_METADATA(JUMP, JUMP_BACKWARD)); - last->i_opcode = is_forward ? JUMP_FORWARD : JUMP_BACKWARD; - return SUCCESS; - case JUMP_NO_INTERRUPT: - assert(SAME_OPCODE_METADATA(JUMP_NO_INTERRUPT, JUMP_FORWARD)); - assert(SAME_OPCODE_METADATA(JUMP_NO_INTERRUPT, JUMP_BACKWARD_NO_INTERRUPT)); - last->i_opcode = is_forward ? - JUMP_FORWARD : JUMP_BACKWARD_NO_INTERRUPT; - return SUCCESS; + if (is_forward) { + return SUCCESS; } + int reversed_opcode = 0; switch(last->i_opcode) { case POP_JUMP_IF_NOT_NONE: @@ -426,9 +419,6 @@ normalize_jumps_in_block(cfg_builder *g, basicblock *b) { reversed_opcode = POP_JUMP_IF_FALSE; break; } - if (is_forward) { - return SUCCESS; - } /* transform 'conditional jump T' to * 'reversed_jump b_next' followed by 'jump_backwards T' */ diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index 5c7de77deba2d..6a42775e09120 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -4,7 +4,7 @@ // Do not edit! -#define IS_PSEUDO_INSTR(OP) \ +#define IS_PSEUDO_INSTR(OP) ( \ ((OP) == LOAD_CLOSURE) || \ ((OP) == STORE_FAST_MAYBE_NULL) || \ ((OP) == LOAD_SUPER_METHOD) || \ @@ -17,7 +17,7 @@ ((OP) == SETUP_CLEANUP) || \ ((OP) == SETUP_WITH) || \ ((OP) == POP_BLOCK) || \ - 0 + 0) #define EXIT_TRACE 300 #define SET_IP 301 diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 2332d12c6c0c6..38be98034e0d8 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -1310,10 +1310,10 @@ def write_metadata(self) -> None: def write_pseudo_instrs(self) -> None: """Write the IS_PSEUDO_INSTR macro""" - self.out.emit("\n\n#define IS_PSEUDO_INSTR(OP) \\") + self.out.emit("\n\n#define IS_PSEUDO_INSTR(OP) ( \\") for op in self.pseudos: self.out.emit(f" ((OP) == {op}) || \\") - self.out.emit(f" 0") + self.out.emit(f" 0)") def write_uop_items(self, make_text: typing.Callable[[str, int], str]) -> None: """Write '#define XXX NNN' for each uop""" From webhook-mailer at python.org Sat Jul 1 07:24:37 2023 From: webhook-mailer at python.org (barneygale) Date: Sat, 01 Jul 2023 11:24:37 -0000 Subject: [Python-checkins] GH-89812: Make symlink support configurable in pathlib tests. (GH-106060) Message-ID: https://github.com/python/cpython/commit/3fd99b5a974314075424744747899b2acf18dadd commit: 3fd99b5a974314075424744747899b2acf18dadd branch: main author: Barney Gale committer: barneygale date: 2023-07-01T12:24:34+01:00 summary: GH-89812: Make symlink support configurable in pathlib tests. (GH-106060) Adjust the pathlib tests to add a new `PathTest.can_symlink` class attribute, which allows us to enable or disable symlink support in tests. A (near-)future commit will add an `AbstractPath` class; its tests will hard-code the value to `True` or `False` depending on a stub subclass's capabilities. files: M Lib/test/test_pathlib.py diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index 9d3db7bfee088..a87f6c4cdc3da 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1573,6 +1573,7 @@ class PathTest(unittest.TestCase): """Tests for the FS-accessing functionalities of the Path classes.""" cls = pathlib.Path + can_symlink = os_helper.can_symlink() # (BASE) # | @@ -1616,7 +1617,7 @@ def cleanup(): with open(join('dirC', 'dirD', 'fileD'), 'wb') as f: f.write(b"this is file D\n") os.chmod(join('dirE'), 0) - if os_helper.can_symlink(): + if self.can_symlink: # Relative symlinks. os.symlink('fileA', join('linkA')) os.symlink('non-existing', join('brokenLink')) @@ -1672,7 +1673,7 @@ def test_exists(self): self.assertIs(True, (p / 'dirA').exists()) self.assertIs(True, (p / 'fileA').exists()) self.assertIs(False, (p / 'fileA' / 'bah').exists()) - if os_helper.can_symlink(): + if self.can_symlink: self.assertIs(True, (p / 'linkA').exists()) self.assertIs(True, (p / 'linkB').exists()) self.assertIs(True, (p / 'linkB' / 'fileB').exists()) @@ -1739,12 +1740,13 @@ def test_iterdir(self): it = p.iterdir() paths = set(it) expected = ['dirA', 'dirB', 'dirC', 'dirE', 'fileA'] - if os_helper.can_symlink(): + if self.can_symlink: expected += ['linkA', 'linkB', 'brokenLink', 'brokenLinkLoop'] self.assertEqual(paths, { P(BASE, q) for q in expected }) - @os_helper.skip_unless_symlink def test_iterdir_symlink(self): + if not self.can_symlink: + self.skipTest("symlinks required") # __iter__ on a symlink to a directory. P = self.cls p = P(BASE, 'linkB') @@ -1772,23 +1774,23 @@ def _check(glob, expected): _check(it, ["fileA"]) _check(p.glob("fileB"), []) _check(p.glob("dir*/file*"), ["dirB/fileB", "dirC/fileC"]) - if not os_helper.can_symlink(): + if not self.can_symlink: _check(p.glob("*A"), ['dirA', 'fileA']) else: _check(p.glob("*A"), ['dirA', 'fileA', 'linkA']) - if not os_helper.can_symlink(): + if not self.can_symlink: _check(p.glob("*B/*"), ['dirB/fileB']) else: _check(p.glob("*B/*"), ['dirB/fileB', 'dirB/linkD', 'linkB/fileB', 'linkB/linkD']) - if not os_helper.can_symlink(): + if not self.can_symlink: _check(p.glob("*/fileB"), ['dirB/fileB']) else: _check(p.glob("*/fileB"), ['dirB/fileB', 'linkB/fileB']) - if os_helper.can_symlink(): + if self.can_symlink: _check(p.glob("brokenLink"), ['brokenLink']) - if not os_helper.can_symlink(): + if not self.can_symlink: _check(p.glob("*/"), ["dirA", "dirB", "dirC", "dirE"]) else: _check(p.glob("*/"), ["dirA", "dirB", "dirC", "dirE", "linkB"]) @@ -1810,8 +1812,9 @@ def _check(path, pattern, case_sensitive, expected): _check(path, "dirb/file*", True, []) _check(path, "dirb/file*", False, ["dirB/fileB"]) - @os_helper.skip_unless_symlink def test_glob_follow_symlinks_common(self): + if not self.can_symlink: + self.skipTest("symlinks required") def _check(path, glob, expected): actual = {path for path in path.glob(glob, follow_symlinks=True) if "linkD" not in path.parent.parts} # exclude symlink loop. @@ -1835,8 +1838,9 @@ def _check(path, glob, expected): _check(p, "dir*/*/../dirD/**/", ["dirC/dirD/../dirD"]) _check(p, "*/dirD/**/", ["dirC/dirD"]) - @os_helper.skip_unless_symlink def test_glob_no_follow_symlinks_common(self): + if not self.can_symlink: + self.skipTest("symlinks required") def _check(path, glob, expected): actual = {path for path in path.glob(glob, follow_symlinks=False)} self.assertEqual(actual, { P(BASE, q) for q in expected }) @@ -1868,14 +1872,14 @@ def _check(glob, expected): _check(p.rglob("fileB"), ["dirB/fileB"]) _check(p.rglob("**/fileB"), ["dirB/fileB"]) _check(p.rglob("*/fileA"), []) - if not os_helper.can_symlink(): + if not self.can_symlink: _check(p.rglob("*/fileB"), ["dirB/fileB"]) else: _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", "linkB/fileB", "dirA/linkC/fileB"]) _check(p.rglob("file*"), ["fileA", "dirB/fileB", "dirC/fileC", "dirC/dirD/fileD"]) - if not os_helper.can_symlink(): + if not self.can_symlink: _check(p.rglob("*/"), [ "dirA", "dirB", "dirC", "dirC/dirD", "dirE", ]) @@ -1900,8 +1904,9 @@ def _check(glob, expected): _check(p.rglob("*.txt"), ["dirC/novel.txt"]) _check(p.rglob("*.*"), ["dirC/novel.txt"]) - @os_helper.skip_unless_symlink def test_rglob_follow_symlinks_common(self): + if not self.can_symlink: + self.skipTest("symlinks required") def _check(path, glob, expected): actual = {path for path in path.rglob(glob, follow_symlinks=True) if 'linkD' not in path.parent.parts} # exclude symlink loop. @@ -1929,8 +1934,9 @@ def _check(path, glob, expected): _check(p, "*.txt", ["dirC/novel.txt"]) _check(p, "*.*", ["dirC/novel.txt"]) - @os_helper.skip_unless_symlink def test_rglob_no_follow_symlinks_common(self): + if not self.can_symlink: + self.skipTest("symlinks required") def _check(path, glob, expected): actual = {path for path in path.rglob(glob, follow_symlinks=False)} self.assertEqual(actual, { P(BASE, q) for q in expected }) @@ -1954,9 +1960,10 @@ def _check(path, glob, expected): _check(p, "*.txt", ["dirC/novel.txt"]) _check(p, "*.*", ["dirC/novel.txt"]) - @os_helper.skip_unless_symlink def test_rglob_symlink_loop(self): # Don't get fooled by symlink loops (Issue #26012). + if not self.can_symlink: + self.skipTest("symlinks required") P = self.cls p = P(BASE) given = set(p.rglob('*')) @@ -2003,9 +2010,10 @@ def test_glob_dotdot(self): self.assertEqual(set(p.glob("xyzzy/..")), set()) self.assertEqual(set(p.glob("/".join([".."] * 50))), { P(BASE, *[".."] * 50)}) - @os_helper.skip_unless_symlink def test_glob_permissions(self): # See bpo-38894 + if not self.can_symlink: + self.skipTest("symlinks required") P = self.cls base = P(BASE) / 'permissions' base.mkdir() @@ -2023,9 +2031,10 @@ def test_glob_permissions(self): self.assertEqual(len(set(base.glob("*/fileC"))), 50) self.assertEqual(len(set(base.glob("*/file*"))), 50) - @os_helper.skip_unless_symlink def test_glob_long_symlink(self): # See gh-87695 + if not self.can_symlink: + self.skipTest("symlinks required") base = self.cls(BASE) / 'long_symlink' base.mkdir() bad_link = base / 'bad_link' @@ -2043,8 +2052,9 @@ def test_glob_above_recursion_limit(self): with set_recursion_limit(recursion_limit): list(base.glob('**')) - @os_helper.skip_unless_symlink def test_readlink(self): + if not self.can_symlink: + self.skipTest("symlinks required") P = self.cls(BASE) self.assertEqual((P / 'linkA').readlink(), self.cls('fileA')) self.assertEqual((P / 'brokenLink').readlink(), @@ -2067,8 +2077,9 @@ def _check_resolve(self, p, expected, strict=True): # This can be used to check both relative and absolute resolutions. _check_resolve_relative = _check_resolve_absolute = _check_resolve - @os_helper.skip_unless_symlink def test_resolve_common(self): + if not self.can_symlink: + self.skipTest("symlinks required") P = self.cls p = P(BASE, 'foo') with self.assertRaises(OSError) as cm: @@ -2128,9 +2139,10 @@ def test_resolve_common(self): # resolves to 'dirB/..' first before resolving to parent of dirB. self._check_resolve_relative(p, P(BASE, 'foo', 'in', 'spam'), False) - @os_helper.skip_unless_symlink def test_resolve_dot(self): # See http://web.archive.org/web/20200623062557/https://bitbucket.org/pitrou/pathlib/issues/9/ + if not self.can_symlink: + self.skipTest("symlinks required") p = self.cls(BASE) p.joinpath('0').symlink_to('.', target_is_directory=True) p.joinpath('1').symlink_to(os.path.join('0', '0'), target_is_directory=True) @@ -2152,8 +2164,9 @@ def test_stat(self): self.addCleanup(p.chmod, st.st_mode) self.assertNotEqual(p.stat(), st) - @os_helper.skip_unless_symlink def test_stat_no_follow_symlinks(self): + if not self.can_symlink: + self.skipTest("symlinks required") p = self.cls(BASE) / 'linkA' st = p.stat() self.assertNotEqual(st, p.stat(follow_symlinks=False)) @@ -2163,8 +2176,9 @@ def test_stat_no_follow_symlinks_nosymlink(self): st = p.stat() self.assertEqual(st, p.stat(follow_symlinks=False)) - @os_helper.skip_unless_symlink def test_lstat(self): + if not self.can_symlink: + self.skipTest("symlinks required") p = self.cls(BASE)/ 'linkA' st = p.stat() self.assertNotEqual(st, p.lstat()) @@ -2180,7 +2194,7 @@ def test_is_dir(self): self.assertFalse((P / 'fileA').is_dir()) self.assertFalse((P / 'non-existing').is_dir()) self.assertFalse((P / 'fileA' / 'bah').is_dir()) - if os_helper.can_symlink(): + if self.can_symlink: self.assertFalse((P / 'linkA').is_dir()) self.assertTrue((P / 'linkB').is_dir()) self.assertFalse((P/ 'brokenLink').is_dir()) @@ -2193,7 +2207,7 @@ def test_is_dir_no_follow_symlinks(self): self.assertFalse((P / 'fileA').is_dir(follow_symlinks=False)) self.assertFalse((P / 'non-existing').is_dir(follow_symlinks=False)) self.assertFalse((P / 'fileA' / 'bah').is_dir(follow_symlinks=False)) - if os_helper.can_symlink(): + if self.can_symlink: self.assertFalse((P / 'linkA').is_dir(follow_symlinks=False)) self.assertFalse((P / 'linkB').is_dir(follow_symlinks=False)) self.assertFalse((P/ 'brokenLink').is_dir(follow_symlinks=False)) @@ -2206,7 +2220,7 @@ def test_is_file(self): self.assertFalse((P / 'dirA').is_file()) self.assertFalse((P / 'non-existing').is_file()) self.assertFalse((P / 'fileA' / 'bah').is_file()) - if os_helper.can_symlink(): + if self.can_symlink: self.assertTrue((P / 'linkA').is_file()) self.assertFalse((P / 'linkB').is_file()) self.assertFalse((P/ 'brokenLink').is_file()) @@ -2219,7 +2233,7 @@ def test_is_file_no_follow_symlinks(self): self.assertFalse((P / 'dirA').is_file(follow_symlinks=False)) self.assertFalse((P / 'non-existing').is_file(follow_symlinks=False)) self.assertFalse((P / 'fileA' / 'bah').is_file(follow_symlinks=False)) - if os_helper.can_symlink(): + if self.can_symlink: self.assertFalse((P / 'linkA').is_file(follow_symlinks=False)) self.assertFalse((P / 'linkB').is_file(follow_symlinks=False)) self.assertFalse((P/ 'brokenLink').is_file(follow_symlinks=False)) @@ -2237,7 +2251,7 @@ def test_is_mount(self): self.assertFalse((P / 'non-existing').is_mount()) self.assertFalse((P / 'fileA' / 'bah').is_mount()) self.assertTrue(R.is_mount()) - if os_helper.can_symlink(): + if self.can_symlink: self.assertFalse((P / 'linkA').is_mount()) self.assertIs((R / '\udfff').is_mount(), False) @@ -2247,13 +2261,13 @@ def test_is_symlink(self): self.assertFalse((P / 'dirA').is_symlink()) self.assertFalse((P / 'non-existing').is_symlink()) self.assertFalse((P / 'fileA' / 'bah').is_symlink()) - if os_helper.can_symlink(): + if self.can_symlink: self.assertTrue((P / 'linkA').is_symlink()) self.assertTrue((P / 'linkB').is_symlink()) self.assertTrue((P/ 'brokenLink').is_symlink()) self.assertIs((P / 'fileA\udfff').is_file(), False) self.assertIs((P / 'fileA\x00').is_file(), False) - if os_helper.can_symlink(): + if self.can_symlink: self.assertIs((P / 'linkA\udfff').is_file(), False) self.assertIs((P / 'linkA\x00').is_file(), False) @@ -2310,6 +2324,9 @@ def test_parts_interning(self): self.assertIs(p.parts[2], q.parts[3]) def _check_complex_symlinks(self, link0_target): + if not self.can_symlink: + self.skipTest("symlinks required") + # Test solving a non-looping chain of symlinks (issue #19887). P = self.cls(BASE) P.joinpath('link1').symlink_to(os.path.join('link0', 'link0'), target_is_directory=True) @@ -2350,15 +2367,12 @@ def _check_complex_symlinks(self, link0_target): finally: os.chdir(old_path) - @os_helper.skip_unless_symlink def test_complex_symlinks_absolute(self): self._check_complex_symlinks(BASE) - @os_helper.skip_unless_symlink def test_complex_symlinks_relative(self): self._check_complex_symlinks('.') - @os_helper.skip_unless_symlink def test_complex_symlinks_relative_dot_dot(self): self._check_complex_symlinks(os.path.join('dirA', '..')) @@ -2462,7 +2476,7 @@ def with_segments(self, *pathsegments): self.assertEqual(42, p.with_segments('~').expanduser().session_id) self.assertEqual(42, (p / 'fileA').rename(p / 'fileB').session_id) self.assertEqual(42, (p / 'fileB').replace(p / 'fileA').session_id) - if os_helper.can_symlink(): + if self.can_symlink: self.assertEqual(42, (p / 'linkA').readlink().session_id) for path in p.iterdir(): self.assertEqual(42, path.session_id) @@ -2783,8 +2797,9 @@ def my_mkdir(path, mode=0o777): self.assertNotIn(str(p12), concurrently_created) self.assertTrue(p.exists()) - @os_helper.skip_unless_symlink def test_symlink_to(self): + if not self.can_symlink: + self.skipTest("symlinks required") P = self.cls(BASE) target = P / 'fileA' # Symlinking a path target. @@ -3167,8 +3182,9 @@ def test_touch_mode(self): st = os.stat(join('masked_new_file')) self.assertEqual(stat.S_IMODE(st.st_mode), 0o750) - @os_helper.skip_unless_symlink def test_resolve_loop(self): + if not self.can_symlink: + self.skipTest("symlinks required") # Loops with relative symlinks. os.symlink('linkX/inside', join('linkX')) self._check_symlink_loop(BASE, 'linkX') From webhook-mailer at python.org Sat Jul 1 07:58:34 2023 From: webhook-mailer at python.org (barneygale) Date: Sat, 01 Jul 2023 11:58:34 -0000 Subject: [Python-checkins] GH-89812: Test that `pathlib.Path.is_junction()` returns false (GH-106062) Message-ID: https://github.com/python/cpython/commit/6e01055e15c252185b118956d545bfad03ae11ed commit: 6e01055e15c252185b118956d545bfad03ae11ed branch: main author: Barney Gale committer: barneygale date: 2023-07-01T12:58:30+01:00 summary: GH-89812: Test that `pathlib.Path.is_junction()` returns false (GH-106062) Slightly expand the test coverage of `is_junction()`. The existing test only checks that `os.path.isjunction()` is called under-the-hood. files: M Lib/test/test_pathlib.py diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index a87f6c4cdc3da..441a7bcd4d419 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -2271,6 +2271,15 @@ def test_is_symlink(self): self.assertIs((P / 'linkA\udfff').is_file(), False) self.assertIs((P / 'linkA\x00').is_file(), False) + def test_is_junction_false(self): + P = self.cls(BASE) + self.assertFalse((P / 'fileA').is_junction()) + self.assertFalse((P / 'dirA').is_junction()) + self.assertFalse((P / 'non-existing').is_junction()) + self.assertFalse((P / 'fileA' / 'bah').is_junction()) + self.assertFalse((P / 'fileA\udfff').is_junction()) + self.assertFalse((P / 'fileA\x00').is_junction()) + def test_is_fifo_false(self): P = self.cls(BASE) self.assertFalse((P / 'fileA').is_fifo()) From webhook-mailer at python.org Sat Jul 1 08:29:05 2023 From: webhook-mailer at python.org (barneygale) Date: Sat, 01 Jul 2023 12:29:05 -0000 Subject: [Python-checkins] GH-89812: Miscellaneous pathlib test improvements (GH-106063) Message-ID: https://github.com/python/cpython/commit/cbc33e4aede10f4d5799d9f7aa9ec5b91414f65b commit: cbc33e4aede10f4d5799d9f7aa9ec5b91414f65b branch: main author: Barney Gale committer: barneygale date: 2023-07-01T12:29:02Z summary: GH-89812: Miscellaneous pathlib test improvements (GH-106063) - Split out dedicated test for unbuffered `open()` - Split out dedicated test for `is_mount()` at the filesystem root - Avoid `os.stat()` when checking that empty paths point to '.' - Remove unnecessary `rmtree()` call - Remove unused `assertSame()` method files: M Lib/test/test_pathlib.py diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index 441a7bcd4d419..5be5a77721baa 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1628,11 +1628,6 @@ def cleanup(): # Broken symlink (pointing to itself). os.symlink('brokenLinkLoop', join('brokenLinkLoop')) - def assertSame(self, path_a, path_b): - self.assertTrue(os.path.samefile(str(path_a), str(path_b)), - "%r and %r don't point to the same file" % - (path_a, path_b)) - def assertFileNotFound(self, func, *args, **kwargs): with self.assertRaises(FileNotFoundError) as cm: func(*args, **kwargs) @@ -1664,7 +1659,7 @@ def test_samefile(self): def test_empty_path(self): # The empty path points to '.' p = self.cls('') - self.assertEqual(p.stat(), os.stat('.')) + self.assertEqual(str(p), '.') def test_exists(self): P = self.cls @@ -1693,9 +1688,6 @@ def test_open_common(self): with (p / 'fileA').open('rb') as f: self.assertIsInstance(f, io.BufferedIOBase) self.assertEqual(f.read().strip(), b"this is file A") - with (p / 'fileA').open('rb', buffering=0) as f: - self.assertIsInstance(f, io.RawIOBase) - self.assertEqual(f.read().strip(), b"this is file A") def test_read_write_bytes(self): p = self.cls(BASE) @@ -2017,7 +2009,6 @@ def test_glob_permissions(self): P = self.cls base = P(BASE) / 'permissions' base.mkdir() - self.addCleanup(os_helper.rmtree, base) for i in range(100): link = base / f"link{i}" @@ -2045,8 +2036,8 @@ def test_glob_above_recursion_limit(self): recursion_limit = 50 # directory_depth > recursion_limit directory_depth = recursion_limit + 10 - base = pathlib.Path(os_helper.TESTFN, 'deep') - path = pathlib.Path(base, *(['d'] * directory_depth)) + base = self.cls(BASE, 'deep') + path = self.cls(base, *(['d'] * directory_depth)) path.mkdir(parents=True) with set_recursion_limit(recursion_limit): @@ -2242,18 +2233,12 @@ def test_is_file_no_follow_symlinks(self): def test_is_mount(self): P = self.cls(BASE) - if os.name == 'nt': - R = self.cls('c:\\') - else: - R = self.cls('/') self.assertFalse((P / 'fileA').is_mount()) self.assertFalse((P / 'dirA').is_mount()) self.assertFalse((P / 'non-existing').is_mount()) self.assertFalse((P / 'fileA' / 'bah').is_mount()) - self.assertTrue(R.is_mount()) if self.can_symlink: self.assertFalse((P / 'linkA').is_mount()) - self.assertIs((R / '\udfff').is_mount(), False) def test_is_symlink(self): P = self.cls(BASE) @@ -2496,6 +2481,12 @@ def with_segments(self, *pathsegments): for dirpath, dirnames, filenames in p.walk(): self.assertEqual(42, dirpath.session_id) + def test_open_unbuffered(self): + p = self.cls(BASE) + with (p / 'fileA').open('rb', buffering=0) as f: + self.assertIsInstance(f, io.RawIOBase) + self.assertEqual(f.read().strip(), b"this is file A") + def test_resolve_nonexist_relative_issue38671(self): p = self.cls('non', 'exist') @@ -2896,6 +2887,14 @@ def test_is_char_device_true(self): self.assertIs(self.cls('/dev/null\udfff').is_char_device(), False) self.assertIs(self.cls('/dev/null\x00').is_char_device(), False) + def test_is_mount_root(self): + if os.name == 'nt': + R = self.cls('c:\\') + else: + R = self.cls('/') + self.assertTrue(R.is_mount()) + self.assertFalse((R / '\udfff').is_mount()) + def test_passing_kwargs_deprecated(self): with self.assertWarns(DeprecationWarning): self.cls(foo="bar") From webhook-mailer at python.org Sat Jul 1 08:33:32 2023 From: webhook-mailer at python.org (barneygale) Date: Sat, 01 Jul 2023 12:33:32 -0000 Subject: [Python-checkins] GH-89812: Improve test for `pathlib.Path.stat()` (GH-106064) Message-ID: https://github.com/python/cpython/commit/c2622a0d82078a7ab17597707b4dd42446a043ae commit: c2622a0d82078a7ab17597707b4dd42446a043ae branch: main author: Barney Gale committer: barneygale date: 2023-07-01T13:33:29+01:00 summary: GH-89812: Improve test for `pathlib.Path.stat()` (GH-106064) Make assertions about the `st_mode`, `st_ino` and `st_dev` attributes of the stat results from two files and a directory, rather than checking if `chmod()` affects `st_mode` (which is already tested elsewhere). files: M Lib/test/test_pathlib.py diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index 5be5a77721baa..464a835212d47 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -2145,15 +2145,25 @@ def test_resolve_dot(self): # Non-strict self.assertEqual(r.resolve(strict=False), p / '3' / '4') - @os_helper.skip_unless_working_chmod def test_stat(self): - p = self.cls(BASE) / 'fileA' - st = p.stat() - self.assertEqual(p.stat(), st) - # Change file mode by flipping write bit. - p.chmod(st.st_mode ^ 0o222) - self.addCleanup(p.chmod, st.st_mode) - self.assertNotEqual(p.stat(), st) + statA = self.cls(BASE).joinpath('fileA').stat() + statB = self.cls(BASE).joinpath('dirB', 'fileB').stat() + statC = self.cls(BASE).joinpath('dirC').stat() + # st_mode: files are the same, directory differs. + self.assertIsInstance(statA.st_mode, int) + self.assertEqual(statA.st_mode, statB.st_mode) + self.assertNotEqual(statA.st_mode, statC.st_mode) + self.assertNotEqual(statB.st_mode, statC.st_mode) + # st_ino: all different, + self.assertIsInstance(statA.st_ino, int) + self.assertNotEqual(statA.st_ino, statB.st_ino) + self.assertNotEqual(statA.st_ino, statC.st_ino) + self.assertNotEqual(statB.st_ino, statC.st_ino) + # st_dev: all the same. + self.assertIsInstance(statA.st_dev, int) + self.assertEqual(statA.st_dev, statB.st_dev) + self.assertEqual(statA.st_dev, statC.st_dev) + # other attributes not used by pathlib. def test_stat_no_follow_symlinks(self): if not self.can_symlink: From webhook-mailer at python.org Sat Jul 1 15:47:17 2023 From: webhook-mailer at python.org (hauntsaninja) Date: Sat, 01 Jul 2023 19:47:17 -0000 Subject: [Python-checkins] Fix duplicate word typos in comments (#106225) Message-ID: https://github.com/python/cpython/commit/822db860eada721742f878653d7ac9364ed8df59 commit: 822db860eada721742f878653d7ac9364ed8df59 branch: main author: Md Sadman Chowdhury <61442059+SpicyCatGames at users.noreply.github.com> committer: hauntsaninja <12621235+hauntsaninja at users.noreply.github.com> date: 2023-07-01T12:47:14-07:00 summary: Fix duplicate word typos in comments (#106225) files: M Include/cpython/object.h M Objects/frameobject.c M Python/initconfig.c diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 71d268fae1a6d..d681435c84545 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -231,7 +231,7 @@ struct _typeobject { }; /* This struct is used by the specializer - * It should should be treated as an opaque blob + * It should be treated as an opaque blob * by code other than the specializer and interpreter. */ struct _specialization_cache { // In order to avoid bloating the bytecode with lots of inline caches, the diff --git a/Objects/frameobject.c b/Objects/frameobject.c index 98f4a0378bf76..0158d72d6b4f9 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -1186,7 +1186,7 @@ frame_get_var(_PyInterpreterFrame *frame, PyCodeObject *co, int i, // (likely) MAKE_CELL must have executed already. value = PyCell_GET(value); } - // (likely) Otherwise it it is an arg (kind & CO_FAST_LOCAL), + // (likely) Otherwise it is an arg (kind & CO_FAST_LOCAL), // with the initial value set when the frame was created... // (unlikely) ...or it was set to some initial value by // an earlier call to PyFrame_LocalsToFast(). diff --git a/Python/initconfig.c b/Python/initconfig.c index 1dcefd478eefe..147cb370412cd 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -538,7 +538,7 @@ _Py_SetArgcArgv(Py_ssize_t argc, wchar_t * const *argv) _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc); // XXX _PyRuntime.orig_argv only gets cleared by Py_Main(), - // so it it currently leaks for embedders. + // so it currently leaks for embedders. res = _PyWideStringList_Copy(&_PyRuntime.orig_argv, &argv_list); PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &old_alloc); From webhook-mailer at python.org Sat Jul 1 18:27:22 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 01 Jul 2023 22:27:22 -0000 Subject: [Python-checkins] gh-106316: Remove pytime.h header file (#106317) Message-ID: https://github.com/python/cpython/commit/46d77610fc77088bceac720a13d9f2df3a50f29e commit: 46d77610fc77088bceac720a13d9f2df3a50f29e branch: main author: Victor Stinner committer: vstinner date: 2023-07-01T22:27:18Z summary: gh-106316: Remove pytime.h header file (#106317) Remove the "cpython/pytime.h" header file: it only contained private functions. Move functions to the internal pycore_time.h header file. Move tests from _testcapi to _testinternalcapi. Rename also test methods to have the same name than tested C functions. No longer export these functions: * _PyTime_Add() * _PyTime_As100Nanoseconds() * _PyTime_FromMicrosecondsClamp() * _PyTime_FromTimespec() * _PyTime_FromTimeval() * _PyTime_GetPerfCounterWithInfo() * _PyTime_MulDiv() files: A Misc/NEWS.d/next/C API/2023-07-01-21-23-33.gh-issue-106316.hp2Ijw.rst D Include/cpython/pytime.h M .github/CODEOWNERS M Doc/whatsnew/3.13.rst M Include/Python.h M Include/internal/pycore_import.h M Include/internal/pycore_time.h M Lib/test/test_time.py M Makefile.pre.in M Modules/Setup.stdlib.in M Modules/_queuemodule.c M Modules/_testcapi/parts.h M Modules/_testcapi/pytime.c M Modules/_testcapimodule.c M Modules/_testinternalcapi.c M Modules/_testsinglephase.c M Modules/selectmodule.c M Modules/socketmodule.h M PCbuild/_testcapi.vcxproj M PCbuild/_testcapi.vcxproj.filters M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters M Python/pytime.c M Tools/build/stable_abi.py M Tools/c-analyzer/c_parser/preprocessor/gcc.py diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 26efa5083672e..c044e9b8f88c5 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -79,7 +79,7 @@ Doc/library/time.rst @pganssle @abalkin Lib/test/test_time.py @pganssle @abalkin Modules/timemodule.c @pganssle @abalkin Python/pytime.c @pganssle @abalkin -Include/pytime.h @pganssle @abalkin +Include/internal/pycore_time.h @pganssle @abalkin # Email and related **/*mail* @python/email-team diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 9696dd4ff0b70..628945f7d5804 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -602,3 +602,6 @@ Removed use ``PyObject_Vectorcall()`` which is available since Python 3.8 (:pep:`590`). (Contributed by Victor Stinner in :gh:`106023`.) + +* Remove ``cpython/pytime.h`` header file: it only contained private functions. + (Contributed by Victor Stinner in :gh:`106316`.) diff --git a/Include/Python.h b/Include/Python.h index 45ba2ec12f2ad..183d07cc539b4 100644 --- a/Include/Python.h +++ b/Include/Python.h @@ -83,7 +83,6 @@ #include "weakrefobject.h" #include "structseq.h" #include "cpython/picklebufobject.h" -#include "cpython/pytime.h" #include "codecs.h" #include "pyerrors.h" #include "pythread.h" diff --git a/Include/cpython/pytime.h b/Include/cpython/pytime.h deleted file mode 100644 index 16d88d191e9e2..0000000000000 --- a/Include/cpython/pytime.h +++ /dev/null @@ -1,331 +0,0 @@ -// The _PyTime_t API is written to use timestamp and timeout values stored in -// various formats and to read clocks. -// -// The _PyTime_t type is an integer to support directly common arithmetic -// operations like t1 + t2. -// -// The _PyTime_t API supports a resolution of 1 nanosecond. The _PyTime_t type -// is signed to support negative timestamps. The supported range is around -// [-292.3 years; +292.3 years]. Using the Unix epoch (January 1st, 1970), the -// supported date range is around [1677-09-21; 2262-04-11]. -// -// Formats: -// -// * seconds -// * seconds as a floating pointer number (C double) -// * milliseconds (10^-3 seconds) -// * microseconds (10^-6 seconds) -// * 100 nanoseconds (10^-7 seconds) -// * nanoseconds (10^-9 seconds) -// * timeval structure, 1 microsecond resolution (10^-6 seconds) -// * timespec structure, 1 nanosecond resolution (10^-9 seconds) -// -// Integer overflows are detected and raise OverflowError. Conversion to a -// resolution worse than 1 nanosecond is rounded correctly with the requested -// rounding mode. There are 4 rounding modes: floor (towards -inf), ceiling -// (towards +inf), half even and up (away from zero). -// -// Some functions clamp the result in the range [_PyTime_MIN; _PyTime_MAX], so -// the caller doesn't have to handle errors and doesn't need to hold the GIL. -// For example, _PyTime_Add(t1, t2) computes t1+t2 and clamp the result on -// overflow. -// -// Clocks: -// -// * System clock -// * Monotonic clock -// * Performance counter -// -// Operations like (t * k / q) with integers are implemented in a way to reduce -// the risk of integer overflow. Such operation is used to convert a clock -// value expressed in ticks with a frequency to _PyTime_t, like -// QueryPerformanceCounter() with QueryPerformanceFrequency(). - -#ifndef Py_LIMITED_API -#ifndef Py_PYTIME_H -#define Py_PYTIME_H - -/************************************************************************** -Symbols and macros to supply platform-independent interfaces to time related -functions and constants -**************************************************************************/ -#ifdef __cplusplus -extern "C" { -#endif - -#ifdef __clang__ -struct timeval; -#endif - -/* _PyTime_t: Python timestamp with subsecond precision. It can be used to - store a duration, and so indirectly a date (related to another date, like - UNIX epoch). */ -typedef int64_t _PyTime_t; -// _PyTime_MIN nanoseconds is around -292.3 years -#define _PyTime_MIN INT64_MIN -// _PyTime_MAX nanoseconds is around +292.3 years -#define _PyTime_MAX INT64_MAX -#define _SIZEOF_PYTIME_T 8 - -typedef enum { - /* Round towards minus infinity (-inf). - For example, used to read a clock. */ - _PyTime_ROUND_FLOOR=0, - /* Round towards infinity (+inf). - For example, used for timeout to wait "at least" N seconds. */ - _PyTime_ROUND_CEILING=1, - /* Round to nearest with ties going to nearest even integer. - For example, used to round from a Python float. */ - _PyTime_ROUND_HALF_EVEN=2, - /* Round away from zero - For example, used for timeout. _PyTime_ROUND_CEILING rounds - -1e-9 to 0 milliseconds which causes bpo-31786 issue. - _PyTime_ROUND_UP rounds -1e-9 to -1 millisecond which keeps - the timeout sign as expected. select.poll(timeout) must block - for negative values." */ - _PyTime_ROUND_UP=3, - /* _PyTime_ROUND_TIMEOUT (an alias for _PyTime_ROUND_UP) should be - used for timeouts. */ - _PyTime_ROUND_TIMEOUT = _PyTime_ROUND_UP -} _PyTime_round_t; - - -/* Convert a time_t to a PyLong. */ -PyAPI_FUNC(PyObject *) _PyLong_FromTime_t( - time_t sec); - -/* Convert a PyLong to a time_t. */ -PyAPI_FUNC(time_t) _PyLong_AsTime_t( - PyObject *obj); - -/* Convert a number of seconds, int or float, to time_t. */ -PyAPI_FUNC(int) _PyTime_ObjectToTime_t( - PyObject *obj, - time_t *sec, - _PyTime_round_t); - -/* Convert a number of seconds, int or float, to a timeval structure. - usec is in the range [0; 999999] and rounded towards zero. - For example, -1.2 is converted to (-2, 800000). */ -PyAPI_FUNC(int) _PyTime_ObjectToTimeval( - PyObject *obj, - time_t *sec, - long *usec, - _PyTime_round_t); - -/* Convert a number of seconds, int or float, to a timespec structure. - nsec is in the range [0; 999999999] and rounded towards zero. - For example, -1.2 is converted to (-2, 800000000). */ -PyAPI_FUNC(int) _PyTime_ObjectToTimespec( - PyObject *obj, - time_t *sec, - long *nsec, - _PyTime_round_t); - - -/* Create a timestamp from a number of seconds. */ -PyAPI_FUNC(_PyTime_t) _PyTime_FromSeconds(int seconds); - -/* Macro to create a timestamp from a number of seconds, no integer overflow. - Only use the macro for small values, prefer _PyTime_FromSeconds(). */ -#define _PYTIME_FROMSECONDS(seconds) \ - ((_PyTime_t)(seconds) * (1000 * 1000 * 1000)) - -/* Create a timestamp from a number of nanoseconds. */ -PyAPI_FUNC(_PyTime_t) _PyTime_FromNanoseconds(_PyTime_t ns); - -/* Create a timestamp from a number of microseconds. - * Clamp to [_PyTime_MIN; _PyTime_MAX] on overflow. */ -PyAPI_FUNC(_PyTime_t) _PyTime_FromMicrosecondsClamp(_PyTime_t us); - -/* Create a timestamp from nanoseconds (Python int). */ -PyAPI_FUNC(int) _PyTime_FromNanosecondsObject(_PyTime_t *t, - PyObject *obj); - -/* Convert a number of seconds (Python float or int) to a timestamp. - Raise an exception and return -1 on error, return 0 on success. */ -PyAPI_FUNC(int) _PyTime_FromSecondsObject(_PyTime_t *t, - PyObject *obj, - _PyTime_round_t round); - -/* Convert a number of milliseconds (Python float or int, 10^-3) to a timestamp. - Raise an exception and return -1 on error, return 0 on success. */ -PyAPI_FUNC(int) _PyTime_FromMillisecondsObject(_PyTime_t *t, - PyObject *obj, - _PyTime_round_t round); - -/* Convert a timestamp to a number of seconds as a C double. */ -PyAPI_FUNC(double) _PyTime_AsSecondsDouble(_PyTime_t t); - -/* Convert timestamp to a number of milliseconds (10^-3 seconds). */ -PyAPI_FUNC(_PyTime_t) _PyTime_AsMilliseconds(_PyTime_t t, - _PyTime_round_t round); - -/* Convert timestamp to a number of microseconds (10^-6 seconds). */ -PyAPI_FUNC(_PyTime_t) _PyTime_AsMicroseconds(_PyTime_t t, - _PyTime_round_t round); - -/* Convert timestamp to a number of nanoseconds (10^-9 seconds). */ -PyAPI_FUNC(_PyTime_t) _PyTime_AsNanoseconds(_PyTime_t t); - -#ifdef MS_WINDOWS -// Convert timestamp to a number of 100 nanoseconds (10^-7 seconds). -PyAPI_FUNC(_PyTime_t) _PyTime_As100Nanoseconds(_PyTime_t t, - _PyTime_round_t round); -#endif - -/* Convert timestamp to a number of nanoseconds (10^-9 seconds) as a Python int - object. */ -PyAPI_FUNC(PyObject *) _PyTime_AsNanosecondsObject(_PyTime_t t); - -#ifndef MS_WINDOWS -/* Create a timestamp from a timeval structure. - Raise an exception and return -1 on overflow, return 0 on success. */ -PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv); -#endif - -/* Convert a timestamp to a timeval structure (microsecond resolution). - tv_usec is always positive. - Raise an exception and return -1 if the conversion overflowed, - return 0 on success. */ -PyAPI_FUNC(int) _PyTime_AsTimeval(_PyTime_t t, - struct timeval *tv, - _PyTime_round_t round); - -/* Similar to _PyTime_AsTimeval() but don't raise an exception on overflow. - On overflow, clamp tv_sec to _PyTime_t min/max. */ -PyAPI_FUNC(void) _PyTime_AsTimeval_clamp(_PyTime_t t, - struct timeval *tv, - _PyTime_round_t round); - -/* Convert a timestamp to a number of seconds (secs) and microseconds (us). - us is always positive. This function is similar to _PyTime_AsTimeval() - except that secs is always a time_t type, whereas the timeval structure - uses a C long for tv_sec on Windows. - Raise an exception and return -1 if the conversion overflowed, - return 0 on success. */ -PyAPI_FUNC(int) _PyTime_AsTimevalTime_t( - _PyTime_t t, - time_t *secs, - int *us, - _PyTime_round_t round); - -#if defined(HAVE_CLOCK_GETTIME) || defined(HAVE_KQUEUE) -/* Create a timestamp from a timespec structure. - Raise an exception and return -1 on overflow, return 0 on success. */ -PyAPI_FUNC(int) _PyTime_FromTimespec(_PyTime_t *tp, struct timespec *ts); - -/* Convert a timestamp to a timespec structure (nanosecond resolution). - tv_nsec is always positive. - Raise an exception and return -1 on error, return 0 on success. */ -PyAPI_FUNC(int) _PyTime_AsTimespec(_PyTime_t t, struct timespec *ts); - -/* Similar to _PyTime_AsTimespec() but don't raise an exception on overflow. - On overflow, clamp tv_sec to _PyTime_t min/max. */ -PyAPI_FUNC(void) _PyTime_AsTimespec_clamp(_PyTime_t t, struct timespec *ts); -#endif - - -// Compute t1 + t2. Clamp to [_PyTime_MIN; _PyTime_MAX] on overflow. -PyAPI_FUNC(_PyTime_t) _PyTime_Add(_PyTime_t t1, _PyTime_t t2); - -/* Compute ticks * mul / div. - Clamp to [_PyTime_MIN; _PyTime_MAX] on overflow. - The caller must ensure that ((div - 1) * mul) cannot overflow. */ -PyAPI_FUNC(_PyTime_t) _PyTime_MulDiv(_PyTime_t ticks, - _PyTime_t mul, - _PyTime_t div); - -/* Structure used by time.get_clock_info() */ -typedef struct { - const char *implementation; - int monotonic; - int adjustable; - double resolution; -} _Py_clock_info_t; - -/* Get the current time from the system clock. - - If the internal clock fails, silently ignore the error and return 0. - On integer overflow, silently ignore the overflow and clamp the clock to - [_PyTime_MIN; _PyTime_MAX]. - - Use _PyTime_GetSystemClockWithInfo() to check for failure. */ -PyAPI_FUNC(_PyTime_t) _PyTime_GetSystemClock(void); - -/* Get the current time from the system clock. - * On success, set *t and *info (if not NULL), and return 0. - * On error, raise an exception and return -1. - */ -PyAPI_FUNC(int) _PyTime_GetSystemClockWithInfo( - _PyTime_t *t, - _Py_clock_info_t *info); - -/* Get the time of a monotonic clock, i.e. a clock that cannot go backwards. - The clock is not affected by system clock updates. The reference point of - the returned value is undefined, so that only the difference between the - results of consecutive calls is valid. - - If the internal clock fails, silently ignore the error and return 0. - On integer overflow, silently ignore the overflow and clamp the clock to - [_PyTime_MIN; _PyTime_MAX]. - - Use _PyTime_GetMonotonicClockWithInfo() to check for failure. */ -PyAPI_FUNC(_PyTime_t) _PyTime_GetMonotonicClock(void); - -/* Get the time of a monotonic clock, i.e. a clock that cannot go backwards. - The clock is not affected by system clock updates. The reference point of - the returned value is undefined, so that only the difference between the - results of consecutive calls is valid. - - Fill info (if set) with information of the function used to get the time. - - Return 0 on success, raise an exception and return -1 on error. */ -PyAPI_FUNC(int) _PyTime_GetMonotonicClockWithInfo( - _PyTime_t *t, - _Py_clock_info_t *info); - - -/* Converts a timestamp to the Gregorian time, using the local time zone. - Return 0 on success, raise an exception and return -1 on error. */ -PyAPI_FUNC(int) _PyTime_localtime(time_t t, struct tm *tm); - -/* Converts a timestamp to the Gregorian time, assuming UTC. - Return 0 on success, raise an exception and return -1 on error. */ -PyAPI_FUNC(int) _PyTime_gmtime(time_t t, struct tm *tm); - -/* Get the performance counter: clock with the highest available resolution to - measure a short duration. - - If the internal clock fails, silently ignore the error and return 0. - On integer overflow, silently ignore the overflow and clamp the clock to - [_PyTime_MIN; _PyTime_MAX]. - - Use _PyTime_GetPerfCounterWithInfo() to check for failure. */ -PyAPI_FUNC(_PyTime_t) _PyTime_GetPerfCounter(void); - -/* Get the performance counter: clock with the highest available resolution to - measure a short duration. - - Fill info (if set) with information of the function used to get the time. - - Return 0 on success, raise an exception and return -1 on error. */ -PyAPI_FUNC(int) _PyTime_GetPerfCounterWithInfo( - _PyTime_t *t, - _Py_clock_info_t *info); - - -// Create a deadline. -// Pseudo code: _PyTime_GetMonotonicClock() + timeout. -PyAPI_FUNC(_PyTime_t) _PyDeadline_Init(_PyTime_t timeout); - -// Get remaining time from a deadline. -// Pseudo code: deadline - _PyTime_GetMonotonicClock(). -PyAPI_FUNC(_PyTime_t) _PyDeadline_Get(_PyTime_t deadline); - -#ifdef __cplusplus -} -#endif - -#endif /* Py_PYTIME_H */ -#endif /* Py_LIMITED_API */ diff --git a/Include/internal/pycore_import.h b/Include/internal/pycore_import.h index 0a9f24efbdb90..ee93f7d99d915 100644 --- a/Include/internal/pycore_import.h +++ b/Include/internal/pycore_import.h @@ -5,6 +5,8 @@ extern "C" { #endif +#include "pycore_time.h" // _PyTime_t + struct _import_runtime_state { /* The builtin modules (defined in config.c). */ diff --git a/Include/internal/pycore_time.h b/Include/internal/pycore_time.h index 949170c449379..3d394e8d36a13 100644 --- a/Include/internal/pycore_time.h +++ b/Include/internal/pycore_time.h @@ -1,3 +1,46 @@ +// The _PyTime_t API is written to use timestamp and timeout values stored in +// various formats and to read clocks. +// +// The _PyTime_t type is an integer to support directly common arithmetic +// operations like t1 + t2. +// +// The _PyTime_t API supports a resolution of 1 nanosecond. The _PyTime_t type +// is signed to support negative timestamps. The supported range is around +// [-292.3 years; +292.3 years]. Using the Unix epoch (January 1st, 1970), the +// supported date range is around [1677-09-21; 2262-04-11]. +// +// Formats: +// +// * seconds +// * seconds as a floating pointer number (C double) +// * milliseconds (10^-3 seconds) +// * microseconds (10^-6 seconds) +// * 100 nanoseconds (10^-7 seconds) +// * nanoseconds (10^-9 seconds) +// * timeval structure, 1 microsecond resolution (10^-6 seconds) +// * timespec structure, 1 nanosecond resolution (10^-9 seconds) +// +// Integer overflows are detected and raise OverflowError. Conversion to a +// resolution worse than 1 nanosecond is rounded correctly with the requested +// rounding mode. There are 4 rounding modes: floor (towards -inf), ceiling +// (towards +inf), half even and up (away from zero). +// +// Some functions clamp the result in the range [_PyTime_MIN; _PyTime_MAX], so +// the caller doesn't have to handle errors and doesn't need to hold the GIL. +// For example, _PyTime_Add(t1, t2) computes t1+t2 and clamp the result on +// overflow. +// +// Clocks: +// +// * System clock +// * Monotonic clock +// * Performance counter +// +// Operations like (t * k / q) with integers are implemented in a way to reduce +// the risk of integer overflow. Such operation is used to convert a clock +// value expressed in ticks with a frequency to _PyTime_t, like +// QueryPerformanceCounter() with QueryPerformanceFrequency(). + #ifndef Py_INTERNAL_TIME_H #define Py_INTERNAL_TIME_H #ifdef __cplusplus @@ -19,6 +62,275 @@ struct _time_runtime_state { }; +#ifdef __clang__ +struct timeval; +#endif + +/* _PyTime_t: Python timestamp with subsecond precision. It can be used to + store a duration, and so indirectly a date (related to another date, like + UNIX epoch). */ +typedef int64_t _PyTime_t; +// _PyTime_MIN nanoseconds is around -292.3 years +#define _PyTime_MIN INT64_MIN +// _PyTime_MAX nanoseconds is around +292.3 years +#define _PyTime_MAX INT64_MAX +#define _SIZEOF_PYTIME_T 8 + +typedef enum { + /* Round towards minus infinity (-inf). + For example, used to read a clock. */ + _PyTime_ROUND_FLOOR=0, + /* Round towards infinity (+inf). + For example, used for timeout to wait "at least" N seconds. */ + _PyTime_ROUND_CEILING=1, + /* Round to nearest with ties going to nearest even integer. + For example, used to round from a Python float. */ + _PyTime_ROUND_HALF_EVEN=2, + /* Round away from zero + For example, used for timeout. _PyTime_ROUND_CEILING rounds + -1e-9 to 0 milliseconds which causes bpo-31786 issue. + _PyTime_ROUND_UP rounds -1e-9 to -1 millisecond which keeps + the timeout sign as expected. select.poll(timeout) must block + for negative values." */ + _PyTime_ROUND_UP=3, + /* _PyTime_ROUND_TIMEOUT (an alias for _PyTime_ROUND_UP) should be + used for timeouts. */ + _PyTime_ROUND_TIMEOUT = _PyTime_ROUND_UP +} _PyTime_round_t; + + +/* Convert a time_t to a PyLong. */ +PyAPI_FUNC(PyObject*) _PyLong_FromTime_t(time_t sec); + +/* Convert a PyLong to a time_t. */ +PyAPI_FUNC(time_t) _PyLong_AsTime_t(PyObject *obj); + +/* Convert a number of seconds, int or float, to time_t. */ +PyAPI_FUNC(int) _PyTime_ObjectToTime_t( + PyObject *obj, + time_t *sec, + _PyTime_round_t); + +/* Convert a number of seconds, int or float, to a timeval structure. + usec is in the range [0; 999999] and rounded towards zero. + For example, -1.2 is converted to (-2, 800000). */ +PyAPI_FUNC(int) _PyTime_ObjectToTimeval( + PyObject *obj, + time_t *sec, + long *usec, + _PyTime_round_t); + +/* Convert a number of seconds, int or float, to a timespec structure. + nsec is in the range [0; 999999999] and rounded towards zero. + For example, -1.2 is converted to (-2, 800000000). */ +PyAPI_FUNC(int) _PyTime_ObjectToTimespec( + PyObject *obj, + time_t *sec, + long *nsec, + _PyTime_round_t); + + +/* Create a timestamp from a number of seconds. */ +PyAPI_FUNC(_PyTime_t) _PyTime_FromSeconds(int seconds); + +/* Macro to create a timestamp from a number of seconds, no integer overflow. + Only use the macro for small values, prefer _PyTime_FromSeconds(). */ +#define _PYTIME_FROMSECONDS(seconds) \ + ((_PyTime_t)(seconds) * (1000 * 1000 * 1000)) + +/* Create a timestamp from a number of nanoseconds. */ +PyAPI_FUNC(_PyTime_t) _PyTime_FromNanoseconds(_PyTime_t ns); + +/* Create a timestamp from a number of microseconds. + * Clamp to [_PyTime_MIN; _PyTime_MAX] on overflow. */ +extern _PyTime_t _PyTime_FromMicrosecondsClamp(_PyTime_t us); + +/* Create a timestamp from nanoseconds (Python int). */ +PyAPI_FUNC(int) _PyTime_FromNanosecondsObject(_PyTime_t *t, + PyObject *obj); + +/* Convert a number of seconds (Python float or int) to a timestamp. + Raise an exception and return -1 on error, return 0 on success. */ +PyAPI_FUNC(int) _PyTime_FromSecondsObject(_PyTime_t *t, + PyObject *obj, + _PyTime_round_t round); + +/* Convert a number of milliseconds (Python float or int, 10^-3) to a timestamp. + Raise an exception and return -1 on error, return 0 on success. */ +PyAPI_FUNC(int) _PyTime_FromMillisecondsObject(_PyTime_t *t, + PyObject *obj, + _PyTime_round_t round); + +/* Convert a timestamp to a number of seconds as a C double. */ +PyAPI_FUNC(double) _PyTime_AsSecondsDouble(_PyTime_t t); + +/* Convert timestamp to a number of milliseconds (10^-3 seconds). */ +PyAPI_FUNC(_PyTime_t) _PyTime_AsMilliseconds(_PyTime_t t, + _PyTime_round_t round); + +/* Convert timestamp to a number of microseconds (10^-6 seconds). */ +PyAPI_FUNC(_PyTime_t) _PyTime_AsMicroseconds(_PyTime_t t, + _PyTime_round_t round); + +/* Convert timestamp to a number of nanoseconds (10^-9 seconds). */ +PyAPI_FUNC(_PyTime_t) _PyTime_AsNanoseconds(_PyTime_t t); + +#ifdef MS_WINDOWS +// Convert timestamp to a number of 100 nanoseconds (10^-7 seconds). +extern _PyTime_t _PyTime_As100Nanoseconds(_PyTime_t t, + _PyTime_round_t round); +#endif + +/* Convert timestamp to a number of nanoseconds (10^-9 seconds) as a Python int + object. */ +PyAPI_FUNC(PyObject*) _PyTime_AsNanosecondsObject(_PyTime_t t); + +#ifndef MS_WINDOWS +/* Create a timestamp from a timeval structure. + Raise an exception and return -1 on overflow, return 0 on success. */ +extern int _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv); +#endif + +/* Convert a timestamp to a timeval structure (microsecond resolution). + tv_usec is always positive. + Raise an exception and return -1 if the conversion overflowed, + return 0 on success. */ +PyAPI_FUNC(int) _PyTime_AsTimeval(_PyTime_t t, + struct timeval *tv, + _PyTime_round_t round); + +/* Similar to _PyTime_AsTimeval() but don't raise an exception on overflow. + On overflow, clamp tv_sec to _PyTime_t min/max. */ +PyAPI_FUNC(void) _PyTime_AsTimeval_clamp(_PyTime_t t, + struct timeval *tv, + _PyTime_round_t round); + +/* Convert a timestamp to a number of seconds (secs) and microseconds (us). + us is always positive. This function is similar to _PyTime_AsTimeval() + except that secs is always a time_t type, whereas the timeval structure + uses a C long for tv_sec on Windows. + Raise an exception and return -1 if the conversion overflowed, + return 0 on success. */ +PyAPI_FUNC(int) _PyTime_AsTimevalTime_t( + _PyTime_t t, + time_t *secs, + int *us, + _PyTime_round_t round); + +#if defined(HAVE_CLOCK_GETTIME) || defined(HAVE_KQUEUE) +/* Create a timestamp from a timespec structure. + Raise an exception and return -1 on overflow, return 0 on success. */ +extern int _PyTime_FromTimespec(_PyTime_t *tp, struct timespec *ts); + +/* Convert a timestamp to a timespec structure (nanosecond resolution). + tv_nsec is always positive. + Raise an exception and return -1 on error, return 0 on success. */ +PyAPI_FUNC(int) _PyTime_AsTimespec(_PyTime_t t, struct timespec *ts); + +/* Similar to _PyTime_AsTimespec() but don't raise an exception on overflow. + On overflow, clamp tv_sec to _PyTime_t min/max. */ +PyAPI_FUNC(void) _PyTime_AsTimespec_clamp(_PyTime_t t, struct timespec *ts); +#endif + + +// Compute t1 + t2. Clamp to [_PyTime_MIN; _PyTime_MAX] on overflow. +extern _PyTime_t _PyTime_Add(_PyTime_t t1, _PyTime_t t2); + +/* Compute ticks * mul / div. + Clamp to [_PyTime_MIN; _PyTime_MAX] on overflow. + The caller must ensure that ((div - 1) * mul) cannot overflow. */ +extern _PyTime_t _PyTime_MulDiv(_PyTime_t ticks, + _PyTime_t mul, + _PyTime_t div); + +/* Structure used by time.get_clock_info() */ +typedef struct { + const char *implementation; + int monotonic; + int adjustable; + double resolution; +} _Py_clock_info_t; + +/* Get the current time from the system clock. + + If the internal clock fails, silently ignore the error and return 0. + On integer overflow, silently ignore the overflow and clamp the clock to + [_PyTime_MIN; _PyTime_MAX]. + + Use _PyTime_GetSystemClockWithInfo() to check for failure. */ +PyAPI_FUNC(_PyTime_t) _PyTime_GetSystemClock(void); + +/* Get the current time from the system clock. + * On success, set *t and *info (if not NULL), and return 0. + * On error, raise an exception and return -1. + */ +PyAPI_FUNC(int) _PyTime_GetSystemClockWithInfo( + _PyTime_t *t, + _Py_clock_info_t *info); + +/* Get the time of a monotonic clock, i.e. a clock that cannot go backwards. + The clock is not affected by system clock updates. The reference point of + the returned value is undefined, so that only the difference between the + results of consecutive calls is valid. + + If the internal clock fails, silently ignore the error and return 0. + On integer overflow, silently ignore the overflow and clamp the clock to + [_PyTime_MIN; _PyTime_MAX]. + + Use _PyTime_GetMonotonicClockWithInfo() to check for failure. */ +PyAPI_FUNC(_PyTime_t) _PyTime_GetMonotonicClock(void); + +/* Get the time of a monotonic clock, i.e. a clock that cannot go backwards. + The clock is not affected by system clock updates. The reference point of + the returned value is undefined, so that only the difference between the + results of consecutive calls is valid. + + Fill info (if set) with information of the function used to get the time. + + Return 0 on success, raise an exception and return -1 on error. */ +PyAPI_FUNC(int) _PyTime_GetMonotonicClockWithInfo( + _PyTime_t *t, + _Py_clock_info_t *info); + + +/* Converts a timestamp to the Gregorian time, using the local time zone. + Return 0 on success, raise an exception and return -1 on error. */ +PyAPI_FUNC(int) _PyTime_localtime(time_t t, struct tm *tm); + +/* Converts a timestamp to the Gregorian time, assuming UTC. + Return 0 on success, raise an exception and return -1 on error. */ +PyAPI_FUNC(int) _PyTime_gmtime(time_t t, struct tm *tm); + +/* Get the performance counter: clock with the highest available resolution to + measure a short duration. + + If the internal clock fails, silently ignore the error and return 0. + On integer overflow, silently ignore the overflow and clamp the clock to + [_PyTime_MIN; _PyTime_MAX]. + + Use _PyTime_GetPerfCounterWithInfo() to check for failure. */ +PyAPI_FUNC(_PyTime_t) _PyTime_GetPerfCounter(void); + +/* Get the performance counter: clock with the highest available resolution to + measure a short duration. + + Fill info (if set) with information of the function used to get the time. + + Return 0 on success, raise an exception and return -1 on error. */ +extern int _PyTime_GetPerfCounterWithInfo( + _PyTime_t *t, + _Py_clock_info_t *info); + + +// Create a deadline. +// Pseudo code: _PyTime_GetMonotonicClock() + timeout. +PyAPI_FUNC(_PyTime_t) _PyDeadline_Init(_PyTime_t timeout); + +// Get remaining time from a deadline. +// Pseudo code: deadline - _PyTime_GetMonotonicClock(). +PyAPI_FUNC(_PyTime_t) _PyDeadline_Get(_PyTime_t deadline); + + #ifdef __cplusplus } #endif diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py index 02cc3f43a66a6..3b5640abdb6b8 100644 --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -14,6 +14,10 @@ import _testcapi except ImportError: _testcapi = None +try: + import _testinternalcapi +except ImportError: + _testinternalcapi = None from test.support import skip_if_buggy_ucrt_strfptime @@ -761,7 +765,8 @@ def test_short_times(self): self.assertIs(lt.tm_zone, None) - at unittest.skipIf(_testcapi is None, 'need the _testcapi module') + at unittest.skipIf(_testcapi is None, 'need the _testinternalcapi module') + at unittest.skipIf(_testinternalcapi is None, 'need the _testinternalcapi module') class CPyTimeTestCase: """ Base class to test the C _PyTime_t API. @@ -769,7 +774,7 @@ class CPyTimeTestCase: OVERFLOW_SECONDS = None def setUp(self): - from _testcapi import SIZEOF_TIME_T + from _testinternalcapi import SIZEOF_TIME_T bits = SIZEOF_TIME_T * 8 - 1 self.time_t_min = -2 ** bits self.time_t_max = 2 ** bits - 1 @@ -897,39 +902,39 @@ class TestCPyTime(CPyTimeTestCase, unittest.TestCase): OVERFLOW_SECONDS = math.ceil((2**63 + 1) / SEC_TO_NS) def test_FromSeconds(self): - from _testcapi import PyTime_FromSeconds + from _testinternalcapi import _PyTime_FromSeconds - # PyTime_FromSeconds() expects a C int, reject values out of range + # _PyTime_FromSeconds() expects a C int, reject values out of range def c_int_filter(secs): return (_testcapi.INT_MIN <= secs <= _testcapi.INT_MAX) - self.check_int_rounding(lambda secs, rnd: PyTime_FromSeconds(secs), + self.check_int_rounding(lambda secs, rnd: _PyTime_FromSeconds(secs), lambda secs: secs * SEC_TO_NS, value_filter=c_int_filter) # test nan for time_rnd, _ in ROUNDING_MODES: with self.assertRaises(TypeError): - PyTime_FromSeconds(float('nan')) + _PyTime_FromSeconds(float('nan')) def test_FromSecondsObject(self): - from _testcapi import PyTime_FromSecondsObject + from _testinternalcapi import _PyTime_FromSecondsObject self.check_int_rounding( - PyTime_FromSecondsObject, + _PyTime_FromSecondsObject, lambda secs: secs * SEC_TO_NS) self.check_float_rounding( - PyTime_FromSecondsObject, + _PyTime_FromSecondsObject, lambda ns: self.decimal_round(ns * SEC_TO_NS)) # test nan for time_rnd, _ in ROUNDING_MODES: with self.assertRaises(ValueError): - PyTime_FromSecondsObject(float('nan'), time_rnd) + _PyTime_FromSecondsObject(float('nan'), time_rnd) def test_AsSecondsDouble(self): - from _testcapi import PyTime_AsSecondsDouble + from _testinternalcapi import _PyTime_AsSecondsDouble def float_converter(ns): if abs(ns) % SEC_TO_NS == 0: @@ -937,14 +942,14 @@ def float_converter(ns): else: return float(ns) / SEC_TO_NS - self.check_int_rounding(lambda ns, rnd: PyTime_AsSecondsDouble(ns), + self.check_int_rounding(lambda ns, rnd: _PyTime_AsSecondsDouble(ns), float_converter, NS_TO_SEC) # test nan for time_rnd, _ in ROUNDING_MODES: with self.assertRaises(TypeError): - PyTime_AsSecondsDouble(float('nan')) + _PyTime_AsSecondsDouble(float('nan')) def create_decimal_converter(self, denominator): denom = decimal.Decimal(denominator) @@ -956,7 +961,7 @@ def converter(value): return converter def test_AsTimeval(self): - from _testcapi import PyTime_AsTimeval + from _testinternalcapi import _PyTime_AsTimeval us_converter = self.create_decimal_converter(US_TO_NS) @@ -973,28 +978,28 @@ def seconds_filter(secs): else: seconds_filter = self.time_t_filter - self.check_int_rounding(PyTime_AsTimeval, + self.check_int_rounding(_PyTime_AsTimeval, timeval_converter, NS_TO_SEC, value_filter=seconds_filter) - @unittest.skipUnless(hasattr(_testcapi, 'PyTime_AsTimespec'), - 'need _testcapi.PyTime_AsTimespec') + @unittest.skipUnless(hasattr(_testinternalcapi, '_PyTime_AsTimespec'), + 'need _testinternalcapi._PyTime_AsTimespec') def test_AsTimespec(self): - from _testcapi import PyTime_AsTimespec + from _testinternalcapi import _PyTime_AsTimespec def timespec_converter(ns): return divmod(ns, SEC_TO_NS) - self.check_int_rounding(lambda ns, rnd: PyTime_AsTimespec(ns), + self.check_int_rounding(lambda ns, rnd: _PyTime_AsTimespec(ns), timespec_converter, NS_TO_SEC, value_filter=self.time_t_filter) - @unittest.skipUnless(hasattr(_testcapi, 'PyTime_AsTimeval_clamp'), - 'need _testcapi.PyTime_AsTimeval_clamp') + @unittest.skipUnless(hasattr(_testinternalcapi, '_PyTime_AsTimeval_clamp'), + 'need _testinternalcapi._PyTime_AsTimeval_clamp') def test_AsTimeval_clamp(self): - from _testcapi import PyTime_AsTimeval_clamp + from _testinternalcapi import _PyTime_AsTimeval_clamp if sys.platform == 'win32': from _testcapi import LONG_MIN, LONG_MAX @@ -1005,7 +1010,7 @@ def test_AsTimeval_clamp(self): tv_sec_min = self.time_t_min for t in (_PyTime_MIN, _PyTime_MAX): - ts = PyTime_AsTimeval_clamp(t, _PyTime.ROUND_CEILING) + ts = _PyTime_AsTimeval_clamp(t, _PyTime.ROUND_CEILING) with decimal.localcontext() as context: context.rounding = decimal.ROUND_CEILING us = self.decimal_round(decimal.Decimal(t) / US_TO_NS) @@ -1018,13 +1023,13 @@ def test_AsTimeval_clamp(self): tv_usec = 0 self.assertEqual(ts, (tv_sec, tv_usec)) - @unittest.skipUnless(hasattr(_testcapi, 'PyTime_AsTimespec_clamp'), - 'need _testcapi.PyTime_AsTimespec_clamp') + @unittest.skipUnless(hasattr(_testinternalcapi, '_PyTime_AsTimespec_clamp'), + 'need _testinternalcapi._PyTime_AsTimespec_clamp') def test_AsTimespec_clamp(self): - from _testcapi import PyTime_AsTimespec_clamp + from _testinternalcapi import _PyTime_AsTimespec_clamp for t in (_PyTime_MIN, _PyTime_MAX): - ts = PyTime_AsTimespec_clamp(t) + ts = _PyTime_AsTimespec_clamp(t) tv_sec, tv_nsec = divmod(t, NS_TO_SEC) if self.time_t_max < tv_sec: tv_sec = self.time_t_max @@ -1035,16 +1040,16 @@ def test_AsTimespec_clamp(self): self.assertEqual(ts, (tv_sec, tv_nsec)) def test_AsMilliseconds(self): - from _testcapi import PyTime_AsMilliseconds + from _testinternalcapi import _PyTime_AsMilliseconds - self.check_int_rounding(PyTime_AsMilliseconds, + self.check_int_rounding(_PyTime_AsMilliseconds, self.create_decimal_converter(MS_TO_NS), NS_TO_SEC) def test_AsMicroseconds(self): - from _testcapi import PyTime_AsMicroseconds + from _testinternalcapi import _PyTime_AsMicroseconds - self.check_int_rounding(PyTime_AsMicroseconds, + self.check_int_rounding(_PyTime_AsMicroseconds, self.create_decimal_converter(US_TO_NS), NS_TO_SEC) @@ -1058,13 +1063,13 @@ class TestOldPyTime(CPyTimeTestCase, unittest.TestCase): OVERFLOW_SECONDS = 2 ** 64 def test_object_to_time_t(self): - from _testcapi import pytime_object_to_time_t + from _testinternalcapi import _PyTime_ObjectToTime_t - self.check_int_rounding(pytime_object_to_time_t, + self.check_int_rounding(_PyTime_ObjectToTime_t, lambda secs: secs, value_filter=self.time_t_filter) - self.check_float_rounding(pytime_object_to_time_t, + self.check_float_rounding(_PyTime_ObjectToTime_t, self.decimal_round, value_filter=self.time_t_filter) @@ -1084,36 +1089,36 @@ def converter(secs): return converter def test_object_to_timeval(self): - from _testcapi import pytime_object_to_timeval + from _testinternalcapi import _PyTime_ObjectToTimeval - self.check_int_rounding(pytime_object_to_timeval, + self.check_int_rounding(_PyTime_ObjectToTimeval, lambda secs: (secs, 0), value_filter=self.time_t_filter) - self.check_float_rounding(pytime_object_to_timeval, + self.check_float_rounding(_PyTime_ObjectToTimeval, self.create_converter(SEC_TO_US), value_filter=self.time_t_filter) # test nan for time_rnd, _ in ROUNDING_MODES: with self.assertRaises(ValueError): - pytime_object_to_timeval(float('nan'), time_rnd) + _PyTime_ObjectToTimeval(float('nan'), time_rnd) def test_object_to_timespec(self): - from _testcapi import pytime_object_to_timespec + from _testinternalcapi import _PyTime_ObjectToTimespec - self.check_int_rounding(pytime_object_to_timespec, + self.check_int_rounding(_PyTime_ObjectToTimespec, lambda secs: (secs, 0), value_filter=self.time_t_filter) - self.check_float_rounding(pytime_object_to_timespec, + self.check_float_rounding(_PyTime_ObjectToTimespec, self.create_converter(SEC_TO_NS), value_filter=self.time_t_filter) # test nan for time_rnd, _ in ROUNDING_MODES: with self.assertRaises(ValueError): - pytime_object_to_timespec(float('nan'), time_rnd) + _PyTime_ObjectToTimespec(float('nan'), time_rnd) @unittest.skipUnless(sys.platform == "darwin", "test weak linking on macOS") class TestTimeWeaklinking(unittest.TestCase): diff --git a/Makefile.pre.in b/Makefile.pre.in index c1b512b94765d..54d0516ad4423 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1710,7 +1710,6 @@ PYTHON_HEADERS= \ $(srcdir)/Include/cpython/pystate.h \ $(srcdir)/Include/cpython/pythonrun.h \ $(srcdir)/Include/cpython/pythread.h \ - $(srcdir)/Include/cpython/pytime.h \ $(srcdir)/Include/cpython/setobject.h \ $(srcdir)/Include/cpython/sysmodule.h \ $(srcdir)/Include/cpython/traceback.h \ diff --git a/Misc/NEWS.d/next/C API/2023-07-01-21-23-33.gh-issue-106316.hp2Ijw.rst b/Misc/NEWS.d/next/C API/2023-07-01-21-23-33.gh-issue-106316.hp2Ijw.rst new file mode 100644 index 0000000000000..733954eb8614f --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-01-21-23-33.gh-issue-106316.hp2Ijw.rst @@ -0,0 +1,2 @@ +Remove ``cpython/pytime.h`` header file: it only contained private +functions. Patch by Victor Stinner. diff --git a/Modules/Setup.stdlib.in b/Modules/Setup.stdlib.in index c228cd5642e75..11a022e3d2044 100644 --- a/Modules/Setup.stdlib.in +++ b/Modules/Setup.stdlib.in @@ -159,7 +159,7 @@ @MODULE__XXTESTFUZZ_TRUE at _xxtestfuzz _xxtestfuzz/_xxtestfuzz.c _xxtestfuzz/fuzzer.c @MODULE__TESTBUFFER_TRUE at _testbuffer _testbuffer.c @MODULE__TESTINTERNALCAPI_TRUE at _testinternalcapi _testinternalcapi.c - at MODULE__TESTCAPI_TRUE@_testcapi _testcapimodule.c _testcapi/vectorcall.c _testcapi/vectorcall_limited.c _testcapi/heaptype.c _testcapi/unicode.c _testcapi/getargs.c _testcapi/pytime.c _testcapi/datetime.c _testcapi/docstring.c _testcapi/mem.c _testcapi/watchers.c _testcapi/long.c _testcapi/float.c _testcapi/structmember.c _testcapi/exceptions.c _testcapi/code.c _testcapi/buffer.c _testcapi/pyos.c _testcapi/immortal.c _testcapi/heaptype_relative.c _testcapi/gc.c + at MODULE__TESTCAPI_TRUE@_testcapi _testcapimodule.c _testcapi/vectorcall.c _testcapi/vectorcall_limited.c _testcapi/heaptype.c _testcapi/unicode.c _testcapi/getargs.c _testcapi/datetime.c _testcapi/docstring.c _testcapi/mem.c _testcapi/watchers.c _testcapi/long.c _testcapi/float.c _testcapi/structmember.c _testcapi/exceptions.c _testcapi/code.c _testcapi/buffer.c _testcapi/pyos.c _testcapi/immortal.c _testcapi/heaptype_relative.c _testcapi/gc.c @MODULE__TESTCLINIC_TRUE at _testclinic _testclinic.c # Some testing modules MUST be built as shared libraries. diff --git a/Modules/_queuemodule.c b/Modules/_queuemodule.c index db5be842b8a35..69cc05135c2a7 100644 --- a/Modules/_queuemodule.c +++ b/Modules/_queuemodule.c @@ -4,6 +4,7 @@ #include "Python.h" #include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_time.h" // _PyTime_t #include "structmember.h" // PyMemberDef #include // offsetof() diff --git a/Modules/_testcapi/parts.h b/Modules/_testcapi/parts.h index d1991ac6b464f..aaec0a6191694 100644 --- a/Modules/_testcapi/parts.h +++ b/Modules/_testcapi/parts.h @@ -28,7 +28,6 @@ int _PyTestCapi_Init_Vectorcall(PyObject *module); int _PyTestCapi_Init_Heaptype(PyObject *module); int _PyTestCapi_Init_Unicode(PyObject *module); int _PyTestCapi_Init_GetArgs(PyObject *module); -int _PyTestCapi_Init_PyTime(PyObject *module); int _PyTestCapi_Init_DateTime(PyObject *module); int _PyTestCapi_Init_Docstring(PyObject *module); int _PyTestCapi_Init_Mem(PyObject *module); diff --git a/Modules/_testcapi/pytime.c b/Modules/_testcapi/pytime.c index 7422bafc30193..e69de29bb2d1d 100644 --- a/Modules/_testcapi/pytime.c +++ b/Modules/_testcapi/pytime.c @@ -1,274 +0,0 @@ -#include "parts.h" - -#ifdef MS_WINDOWS -# include // struct timeval -#endif - -static PyObject * -test_pytime_fromseconds(PyObject *self, PyObject *args) -{ - int seconds; - if (!PyArg_ParseTuple(args, "i", &seconds)) { - return NULL; - } - _PyTime_t ts = _PyTime_FromSeconds(seconds); - return _PyTime_AsNanosecondsObject(ts); -} - -static int -check_time_rounding(int round) -{ - if (round != _PyTime_ROUND_FLOOR - && round != _PyTime_ROUND_CEILING - && round != _PyTime_ROUND_HALF_EVEN - && round != _PyTime_ROUND_UP) - { - PyErr_SetString(PyExc_ValueError, "invalid rounding"); - return -1; - } - return 0; -} - -static PyObject * -test_pytime_fromsecondsobject(PyObject *self, PyObject *args) -{ - PyObject *obj; - int round; - if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { - return NULL; - } - if (check_time_rounding(round) < 0) { - return NULL; - } - _PyTime_t ts; - if (_PyTime_FromSecondsObject(&ts, obj, round) == -1) { - return NULL; - } - return _PyTime_AsNanosecondsObject(ts); -} - -static PyObject * -test_pytime_assecondsdouble(PyObject *self, PyObject *args) -{ - PyObject *obj; - if (!PyArg_ParseTuple(args, "O", &obj)) { - return NULL; - } - _PyTime_t ts; - if (_PyTime_FromNanosecondsObject(&ts, obj) < 0) { - return NULL; - } - double d = _PyTime_AsSecondsDouble(ts); - return PyFloat_FromDouble(d); -} - -static PyObject * -test_PyTime_AsTimeval(PyObject *self, PyObject *args) -{ - PyObject *obj; - int round; - if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { - return NULL; - } - if (check_time_rounding(round) < 0) { - return NULL; - } - _PyTime_t t; - if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { - return NULL; - } - struct timeval tv; - if (_PyTime_AsTimeval(t, &tv, round) < 0) { - return NULL; - } - - PyObject *seconds = PyLong_FromLongLong(tv.tv_sec); - if (seconds == NULL) { - return NULL; - } - return Py_BuildValue("Nl", seconds, (long)tv.tv_usec); -} - -static PyObject * -test_PyTime_AsTimeval_clamp(PyObject *self, PyObject *args) -{ - PyObject *obj; - int round; - if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { - return NULL; - } - if (check_time_rounding(round) < 0) { - return NULL; - } - _PyTime_t t; - if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { - return NULL; - } - struct timeval tv; - _PyTime_AsTimeval_clamp(t, &tv, round); - - PyObject *seconds = PyLong_FromLongLong(tv.tv_sec); - if (seconds == NULL) { - return NULL; - } - return Py_BuildValue("Nl", seconds, (long)tv.tv_usec); -} - -#ifdef HAVE_CLOCK_GETTIME -static PyObject * -test_PyTime_AsTimespec(PyObject *self, PyObject *args) -{ - PyObject *obj; - if (!PyArg_ParseTuple(args, "O", &obj)) { - return NULL; - } - _PyTime_t t; - if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { - return NULL; - } - struct timespec ts; - if (_PyTime_AsTimespec(t, &ts) == -1) { - return NULL; - } - return Py_BuildValue("Nl", _PyLong_FromTime_t(ts.tv_sec), ts.tv_nsec); -} - -static PyObject * -test_PyTime_AsTimespec_clamp(PyObject *self, PyObject *args) -{ - PyObject *obj; - if (!PyArg_ParseTuple(args, "O", &obj)) { - return NULL; - } - _PyTime_t t; - if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { - return NULL; - } - struct timespec ts; - _PyTime_AsTimespec_clamp(t, &ts); - return Py_BuildValue("Nl", _PyLong_FromTime_t(ts.tv_sec), ts.tv_nsec); -} -#endif - -static PyObject * -test_PyTime_AsMilliseconds(PyObject *self, PyObject *args) -{ - PyObject *obj; - int round; - if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { - return NULL; - } - _PyTime_t t; - if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { - return NULL; - } - if (check_time_rounding(round) < 0) { - return NULL; - } - _PyTime_t ms = _PyTime_AsMilliseconds(t, round); - _PyTime_t ns = _PyTime_FromNanoseconds(ms); - return _PyTime_AsNanosecondsObject(ns); -} - -static PyObject * -test_PyTime_AsMicroseconds(PyObject *self, PyObject *args) -{ - PyObject *obj; - int round; - if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { - return NULL; - } - _PyTime_t t; - if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { - return NULL; - } - if (check_time_rounding(round) < 0) { - return NULL; - } - _PyTime_t us = _PyTime_AsMicroseconds(t, round); - _PyTime_t ns = _PyTime_FromNanoseconds(us); - return _PyTime_AsNanosecondsObject(ns); -} - -static PyObject * -test_pytime_object_to_time_t(PyObject *self, PyObject *args) -{ - PyObject *obj; - time_t sec; - int round; - if (!PyArg_ParseTuple(args, "Oi:pytime_object_to_time_t", &obj, &round)) { - return NULL; - } - if (check_time_rounding(round) < 0) { - return NULL; - } - if (_PyTime_ObjectToTime_t(obj, &sec, round) == -1) { - return NULL; - } - return _PyLong_FromTime_t(sec); -} - -static PyObject * -test_pytime_object_to_timeval(PyObject *self, PyObject *args) -{ - PyObject *obj; - time_t sec; - long usec; - int round; - if (!PyArg_ParseTuple(args, "Oi:pytime_object_to_timeval", &obj, &round)) { - return NULL; - } - if (check_time_rounding(round) < 0) { - return NULL; - } - if (_PyTime_ObjectToTimeval(obj, &sec, &usec, round) == -1) { - return NULL; - } - return Py_BuildValue("Nl", _PyLong_FromTime_t(sec), usec); -} - -static PyObject * -test_pytime_object_to_timespec(PyObject *self, PyObject *args) -{ - PyObject *obj; - time_t sec; - long nsec; - int round; - if (!PyArg_ParseTuple(args, "Oi:pytime_object_to_timespec", &obj, &round)) { - return NULL; - } - if (check_time_rounding(round) < 0) { - return NULL; - } - if (_PyTime_ObjectToTimespec(obj, &sec, &nsec, round) == -1) { - return NULL; - } - return Py_BuildValue("Nl", _PyLong_FromTime_t(sec), nsec); -} - -static PyMethodDef test_methods[] = { - {"PyTime_AsMicroseconds", test_PyTime_AsMicroseconds, METH_VARARGS}, - {"PyTime_AsMilliseconds", test_PyTime_AsMilliseconds, METH_VARARGS}, - {"PyTime_AsSecondsDouble", test_pytime_assecondsdouble, METH_VARARGS}, -#ifdef HAVE_CLOCK_GETTIME - {"PyTime_AsTimespec", test_PyTime_AsTimespec, METH_VARARGS}, - {"PyTime_AsTimespec_clamp", test_PyTime_AsTimespec_clamp, METH_VARARGS}, -#endif - {"PyTime_AsTimeval", test_PyTime_AsTimeval, METH_VARARGS}, - {"PyTime_AsTimeval_clamp", test_PyTime_AsTimeval_clamp, METH_VARARGS}, - {"PyTime_FromSeconds", test_pytime_fromseconds, METH_VARARGS}, - {"PyTime_FromSecondsObject", test_pytime_fromsecondsobject, METH_VARARGS}, - {"pytime_object_to_time_t", test_pytime_object_to_time_t, METH_VARARGS}, - {"pytime_object_to_timespec", test_pytime_object_to_timespec, METH_VARARGS}, - {"pytime_object_to_timeval", test_pytime_object_to_timeval, METH_VARARGS}, - {NULL}, -}; - -int -_PyTestCapi_Init_PyTime(PyObject *mod) -{ - if (PyModule_AddFunctions(mod, test_methods) < 0) { - return -1; - } - return 0; -} diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index dc8acec6c7692..ec2e64f2f46fc 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -4216,9 +4216,6 @@ PyInit__testcapi(void) if (_PyTestCapi_Init_GetArgs(m) < 0) { return NULL; } - if (_PyTestCapi_Init_PyTime(m) < 0) { - return NULL; - } if (_PyTestCapi_Init_DateTime(m) < 0) { return NULL; } diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 3c0c2adaf6f84..c598d7edef8ae 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -31,6 +31,10 @@ #include "clinic/_testinternalcapi.c.h" +#ifdef MS_WINDOWS +# include // struct timeval +#endif + #define MODULE_NAME "_testinternalcapi" @@ -969,6 +973,249 @@ pending_identify(PyObject *self, PyObject *args) } +static PyObject * +test_pytime_fromseconds(PyObject *self, PyObject *args) +{ + int seconds; + if (!PyArg_ParseTuple(args, "i", &seconds)) { + return NULL; + } + _PyTime_t ts = _PyTime_FromSeconds(seconds); + return _PyTime_AsNanosecondsObject(ts); +} + +static int +check_time_rounding(int round) +{ + if (round != _PyTime_ROUND_FLOOR + && round != _PyTime_ROUND_CEILING + && round != _PyTime_ROUND_HALF_EVEN + && round != _PyTime_ROUND_UP) + { + PyErr_SetString(PyExc_ValueError, "invalid rounding"); + return -1; + } + return 0; +} + +static PyObject * +test_pytime_fromsecondsobject(PyObject *self, PyObject *args) +{ + PyObject *obj; + int round; + if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { + return NULL; + } + if (check_time_rounding(round) < 0) { + return NULL; + } + _PyTime_t ts; + if (_PyTime_FromSecondsObject(&ts, obj, round) == -1) { + return NULL; + } + return _PyTime_AsNanosecondsObject(ts); +} + +static PyObject * +test_pytime_assecondsdouble(PyObject *self, PyObject *args) +{ + PyObject *obj; + if (!PyArg_ParseTuple(args, "O", &obj)) { + return NULL; + } + _PyTime_t ts; + if (_PyTime_FromNanosecondsObject(&ts, obj) < 0) { + return NULL; + } + double d = _PyTime_AsSecondsDouble(ts); + return PyFloat_FromDouble(d); +} + +static PyObject * +test_PyTime_AsTimeval(PyObject *self, PyObject *args) +{ + PyObject *obj; + int round; + if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { + return NULL; + } + if (check_time_rounding(round) < 0) { + return NULL; + } + _PyTime_t t; + if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { + return NULL; + } + struct timeval tv; + if (_PyTime_AsTimeval(t, &tv, round) < 0) { + return NULL; + } + + PyObject *seconds = PyLong_FromLongLong(tv.tv_sec); + if (seconds == NULL) { + return NULL; + } + return Py_BuildValue("Nl", seconds, (long)tv.tv_usec); +} + +static PyObject * +test_PyTime_AsTimeval_clamp(PyObject *self, PyObject *args) +{ + PyObject *obj; + int round; + if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { + return NULL; + } + if (check_time_rounding(round) < 0) { + return NULL; + } + _PyTime_t t; + if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { + return NULL; + } + struct timeval tv; + _PyTime_AsTimeval_clamp(t, &tv, round); + + PyObject *seconds = PyLong_FromLongLong(tv.tv_sec); + if (seconds == NULL) { + return NULL; + } + return Py_BuildValue("Nl", seconds, (long)tv.tv_usec); +} + +#ifdef HAVE_CLOCK_GETTIME +static PyObject * +test_PyTime_AsTimespec(PyObject *self, PyObject *args) +{ + PyObject *obj; + if (!PyArg_ParseTuple(args, "O", &obj)) { + return NULL; + } + _PyTime_t t; + if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { + return NULL; + } + struct timespec ts; + if (_PyTime_AsTimespec(t, &ts) == -1) { + return NULL; + } + return Py_BuildValue("Nl", _PyLong_FromTime_t(ts.tv_sec), ts.tv_nsec); +} + +static PyObject * +test_PyTime_AsTimespec_clamp(PyObject *self, PyObject *args) +{ + PyObject *obj; + if (!PyArg_ParseTuple(args, "O", &obj)) { + return NULL; + } + _PyTime_t t; + if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { + return NULL; + } + struct timespec ts; + _PyTime_AsTimespec_clamp(t, &ts); + return Py_BuildValue("Nl", _PyLong_FromTime_t(ts.tv_sec), ts.tv_nsec); +} +#endif + +static PyObject * +test_PyTime_AsMilliseconds(PyObject *self, PyObject *args) +{ + PyObject *obj; + int round; + if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { + return NULL; + } + _PyTime_t t; + if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { + return NULL; + } + if (check_time_rounding(round) < 0) { + return NULL; + } + _PyTime_t ms = _PyTime_AsMilliseconds(t, round); + _PyTime_t ns = _PyTime_FromNanoseconds(ms); + return _PyTime_AsNanosecondsObject(ns); +} + +static PyObject * +test_PyTime_AsMicroseconds(PyObject *self, PyObject *args) +{ + PyObject *obj; + int round; + if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { + return NULL; + } + _PyTime_t t; + if (_PyTime_FromNanosecondsObject(&t, obj) < 0) { + return NULL; + } + if (check_time_rounding(round) < 0) { + return NULL; + } + _PyTime_t us = _PyTime_AsMicroseconds(t, round); + _PyTime_t ns = _PyTime_FromNanoseconds(us); + return _PyTime_AsNanosecondsObject(ns); +} + +static PyObject * +test_pytime_object_to_time_t(PyObject *self, PyObject *args) +{ + PyObject *obj; + time_t sec; + int round; + if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { + return NULL; + } + if (check_time_rounding(round) < 0) { + return NULL; + } + if (_PyTime_ObjectToTime_t(obj, &sec, round) == -1) { + return NULL; + } + return _PyLong_FromTime_t(sec); +} + +static PyObject * +test_pytime_object_to_timeval(PyObject *self, PyObject *args) +{ + PyObject *obj; + time_t sec; + long usec; + int round; + if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { + return NULL; + } + if (check_time_rounding(round) < 0) { + return NULL; + } + if (_PyTime_ObjectToTimeval(obj, &sec, &usec, round) == -1) { + return NULL; + } + return Py_BuildValue("Nl", _PyLong_FromTime_t(sec), usec); +} + +static PyObject * +test_pytime_object_to_timespec(PyObject *self, PyObject *args) +{ + PyObject *obj; + time_t sec; + long nsec; + int round; + if (!PyArg_ParseTuple(args, "Oi", &obj, &round)) { + return NULL; + } + if (check_time_rounding(round) < 0) { + return NULL; + } + if (_PyTime_ObjectToTimespec(obj, &sec, &nsec, round) == -1) { + return NULL; + } + return Py_BuildValue("Nl", _PyLong_FromTime_t(sec), nsec); +} + + static PyMethodDef module_functions[] = { {"get_configs", get_configs, METH_NOARGS}, {"get_recursion_depth", get_recursion_depth, METH_NOARGS}, @@ -1005,6 +1252,20 @@ static PyMethodDef module_functions[] = { METH_VARARGS | METH_KEYWORDS}, // {"pending_fd_identify", pending_fd_identify, METH_VARARGS, NULL}, {"pending_identify", pending_identify, METH_VARARGS, NULL}, + {"_PyTime_AsMicroseconds", test_PyTime_AsMicroseconds, METH_VARARGS}, + {"_PyTime_AsMilliseconds", test_PyTime_AsMilliseconds, METH_VARARGS}, + {"_PyTime_AsSecondsDouble", test_pytime_assecondsdouble, METH_VARARGS}, +#ifdef HAVE_CLOCK_GETTIME + {"_PyTime_AsTimespec", test_PyTime_AsTimespec, METH_VARARGS}, + {"_PyTime_AsTimespec_clamp", test_PyTime_AsTimespec_clamp, METH_VARARGS}, +#endif + {"_PyTime_AsTimeval", test_PyTime_AsTimeval, METH_VARARGS}, + {"_PyTime_AsTimeval_clamp", test_PyTime_AsTimeval_clamp, METH_VARARGS}, + {"_PyTime_FromSeconds", test_pytime_fromseconds, METH_VARARGS}, + {"_PyTime_FromSecondsObject", test_pytime_fromsecondsobject, METH_VARARGS}, + {"_PyTime_ObjectToTime_t", test_pytime_object_to_time_t, METH_VARARGS}, + {"_PyTime_ObjectToTimespec", test_pytime_object_to_timespec, METH_VARARGS}, + {"_PyTime_ObjectToTimeval", test_pytime_object_to_timeval, METH_VARARGS}, {NULL, NULL} /* sentinel */ }; @@ -1019,6 +1280,11 @@ module_exec(PyObject *module) return 1; } + if (PyModule_AddObject(module, "SIZEOF_TIME_T", + PyLong_FromSsize_t(sizeof(time_t))) < 0) { + return 1; + } + return 0; } diff --git a/Modules/_testsinglephase.c b/Modules/_testsinglephase.c index dca7abff89146..922b41005e841 100644 --- a/Modules/_testsinglephase.c +++ b/Modules/_testsinglephase.c @@ -8,6 +8,7 @@ //#include #include "Python.h" #include "pycore_namespace.h" // _PyNamespace_New() +#include "pycore_time.h" // _PyTime_t typedef struct { diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c index 9a4943c9eb2f7..7ab0804ad2723 100644 --- a/Modules/selectmodule.c +++ b/Modules/selectmodule.c @@ -14,6 +14,7 @@ #include "Python.h" #include "pycore_fileutils.h" // _Py_set_inheritable() +#include "pycore_time.h" // _PyTime_t #include "structmember.h" // PyMemberDef #ifdef HAVE_SYS_DEVPOLL_H diff --git a/Modules/socketmodule.h b/Modules/socketmodule.h index f5ca00450ee92..663ae3d6e0dd6 100644 --- a/Modules/socketmodule.h +++ b/Modules/socketmodule.h @@ -1,5 +1,7 @@ /* Socket module header file */ +#include "pycore_time.h" // _PyTime_t + /* Includes needed for the sockaddr_* symbols below */ #ifndef MS_WINDOWS #ifdef __VMS diff --git a/PCbuild/_testcapi.vcxproj b/PCbuild/_testcapi.vcxproj index 3db9426d1a25f..de17d74c52e56 100644 --- a/PCbuild/_testcapi.vcxproj +++ b/PCbuild/_testcapi.vcxproj @@ -100,7 +100,6 @@ - @@ -131,4 +130,4 @@ - \ No newline at end of file + diff --git a/PCbuild/_testcapi.vcxproj.filters b/PCbuild/_testcapi.vcxproj.filters index 8df4874659fa1..637f7178d39d0 100644 --- a/PCbuild/_testcapi.vcxproj.filters +++ b/PCbuild/_testcapi.vcxproj.filters @@ -30,9 +30,6 @@ Source Files - - Source Files - Source Files @@ -75,4 +72,4 @@ Resource Files - \ No newline at end of file + diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index c99bc905c89e2..58abd871376a0 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -176,7 +176,6 @@ - diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 53e89457f72a3..4d7b5ccf9a8a9 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -447,9 +447,6 @@ Include\cpython - - Include\cpython - Include\cpython diff --git a/Python/pytime.c b/Python/pytime.c index d36d4417dabb2..49cd5f4e8ea61 100644 --- a/Python/pytime.c +++ b/Python/pytime.c @@ -1,4 +1,5 @@ #include "Python.h" +#include "pycore_time.h" // _PyTime_t #ifdef MS_WINDOWS # include // struct timeval #endif diff --git a/Tools/build/stable_abi.py b/Tools/build/stable_abi.py index 4cd1cd953d0d2..7cba788ff3357 100644 --- a/Tools/build/stable_abi.py +++ b/Tools/build/stable_abi.py @@ -40,7 +40,6 @@ "longintrepr.h", "parsetok.h", "pyatomic.h", - "pytime.h", "token.h", "ucnhash.h", } diff --git a/Tools/c-analyzer/c_parser/preprocessor/gcc.py b/Tools/c-analyzer/c_parser/preprocessor/gcc.py index 147615707a3cb..0929f7111eac9 100644 --- a/Tools/c-analyzer/c_parser/preprocessor/gcc.py +++ b/Tools/c-analyzer/c_parser/preprocessor/gcc.py @@ -60,6 +60,13 @@ def preprocess(filename, if not cwd or not os.path.isabs(cwd): cwd = os.path.abspath(cwd or '.') filename = _normpath(filename, cwd) + + postargs = POST_ARGS + if os.path.basename(filename) == 'socketmodule.h': + # Modules/socketmodule.h uses pycore_time.h which needs Py_BUILD_CORE. + # Usually it's defined by the C file which includes it. + postargs += ('-DPy_BUILD_CORE=1',) + text = _common.preprocess( TOOL, filename, @@ -67,7 +74,7 @@ def preprocess(filename, includes=includes, macros=macros, #preargs=PRE_ARGS, - postargs=POST_ARGS, + postargs=postargs, executable=['gcc'], compiler='unix', cwd=cwd, From webhook-mailer at python.org Sat Jul 1 18:46:10 2023 From: webhook-mailer at python.org (terryjreedy) Date: Sat, 01 Jul 2023 22:46:10 -0000 Subject: [Python-checkins] gh-102541: Fix Helper.help("mod") for non-existent mod (#105934) Message-ID: https://github.com/python/cpython/commit/0530f4f64629ff97f3feb7524da0833b9535e8b6 commit: 0530f4f64629ff97f3feb7524da0833b9535e8b6 branch: main author: Kirill Podoprigora committer: terryjreedy date: 2023-07-01T18:46:06-04:00 summary: gh-102541: Fix Helper.help("mod") for non-existent mod (#105934) If the output arg to Helper() is a stream rather than the default None, which means 'page to stdout', the ImportError from pydoc.resolve is currently not caught in pydoc.doc. The same error is caught when output is None. --------- Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst M Lib/pydoc.py M Lib/test/test_pydoc.py diff --git a/Lib/pydoc.py b/Lib/pydoc.py index d69369d4196ea..185f09e603df2 100755 --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -1790,7 +1790,11 @@ def doc(thing, title='Python Library Documentation: %s', forceload=0, raise print(exc) else: - output.write(render_doc(thing, title, forceload, plaintext)) + try: + s = render_doc(thing, title, forceload, plaintext) + except ImportError as exc: + s = str(exc) + output.write(s) def writedoc(thing, forceload=0): """Write HTML documentation to a file in the current directory.""" diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py index cefc71cb5a7f5..115ffd3c29d4d 100644 --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -631,6 +631,13 @@ def test_builtin_on_metaclasses(self): # Testing that the subclasses section does not appear self.assertNotIn('Built-in subclasses', text) + def test_fail_help_output_redirect(self): + with StringIO() as buf: + helper = pydoc.Helper(output=buf) + helper.help("abd") + expected = missing_pattern % "abd" + self.assertEqual(expected, buf.getvalue().strip().replace('\n', os.linesep)) + @unittest.skipIf(hasattr(sys, 'gettrace') and sys.gettrace(), 'trace function introduces __locals__ unexpectedly') @requires_docstrings diff --git a/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst b/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst new file mode 100644 index 0000000000000..efaf5db10f3e1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst @@ -0,0 +1 @@ +Make pydoc.doc catch bad module ImportError when output stream is not None. From webhook-mailer at python.org Sat Jul 1 19:15:48 2023 From: webhook-mailer at python.org (terryjreedy) Date: Sat, 01 Jul 2023 23:15:48 -0000 Subject: [Python-checkins] [3.11] gh-102541: Fix Helper.help("mod") for non-existent mod (GH-105934) (#106323) Message-ID: https://github.com/python/cpython/commit/9e335860188229a7985fe9a8b72ee62c684aa9eb commit: 9e335860188229a7985fe9a8b72ee62c684aa9eb branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-01T23:15:44Z summary: [3.11] gh-102541: Fix Helper.help("mod") for non-existent mod (GH-105934) (#106323) gh-102541: Fix Helper.help("mod") for non-existent mod (GH-105934) If the output arg to Helper() is a stream rather than the default None, which means 'page to stdout', the ImportError from pydoc.resolve is currently not caught in pydoc.doc. The same error is caught when output is None. --------- (cherry picked from commit 0530f4f64629ff97f3feb7524da0833b9535e8b6) Co-authored-by: Kirill Podoprigora Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst M Lib/pydoc.py M Lib/test/test_pydoc.py diff --git a/Lib/pydoc.py b/Lib/pydoc.py index 151c7e877ba5a..5e191bc1c4b2a 100755 --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -1788,7 +1788,11 @@ def doc(thing, title='Python Library Documentation: %s', forceload=0, raise print(exc) else: - output.write(render_doc(thing, title, forceload, plaintext)) + try: + s = render_doc(thing, title, forceload, plaintext) + except ImportError as exc: + s = str(exc) + output.write(s) def writedoc(thing, forceload=0): """Write HTML documentation to a file in the current directory.""" diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py index 89faf0d9141f9..71be6d510addd 100644 --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -631,6 +631,13 @@ def test_builtin_on_metaclasses(self): # Testing that the subclasses section does not appear self.assertNotIn('Built-in subclasses', text) + def test_fail_help_output_redirect(self): + with StringIO() as buf: + helper = pydoc.Helper(output=buf) + helper.help("abd") + expected = missing_pattern % "abd" + self.assertEqual(expected, buf.getvalue().strip().replace('\n', os.linesep)) + @unittest.skipIf(hasattr(sys, 'gettrace') and sys.gettrace(), 'trace function introduces __locals__ unexpectedly') @requires_docstrings diff --git a/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst b/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst new file mode 100644 index 0000000000000..efaf5db10f3e1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst @@ -0,0 +1 @@ +Make pydoc.doc catch bad module ImportError when output stream is not None. From webhook-mailer at python.org Sat Jul 1 19:16:17 2023 From: webhook-mailer at python.org (terryjreedy) Date: Sat, 01 Jul 2023 23:16:17 -0000 Subject: [Python-checkins] [3.12] gh-102541: Fix Helper.help("mod") for non-existent mod (GH-105934) (#106322) Message-ID: https://github.com/python/cpython/commit/730c873efd9aecbfa2e788faf021a3523ad6a65e commit: 730c873efd9aecbfa2e788faf021a3523ad6a65e branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-01T23:16:14Z summary: [3.12] gh-102541: Fix Helper.help("mod") for non-existent mod (GH-105934) (#106322) gh-102541: Fix Helper.help("mod") for non-existent mod (GH-105934) If the output arg to Helper() is a stream rather than the default None, which means 'page to stdout', the ImportError from pydoc.resolve is currently not caught in pydoc.doc. The same error is caught when output is None. --------- (cherry picked from commit 0530f4f64629ff97f3feb7524da0833b9535e8b6) Co-authored-by: Kirill Podoprigora Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst M Lib/pydoc.py M Lib/test/test_pydoc.py diff --git a/Lib/pydoc.py b/Lib/pydoc.py index d69369d4196ea..185f09e603df2 100755 --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -1790,7 +1790,11 @@ def doc(thing, title='Python Library Documentation: %s', forceload=0, raise print(exc) else: - output.write(render_doc(thing, title, forceload, plaintext)) + try: + s = render_doc(thing, title, forceload, plaintext) + except ImportError as exc: + s = str(exc) + output.write(s) def writedoc(thing, forceload=0): """Write HTML documentation to a file in the current directory.""" diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py index cefc71cb5a7f5..115ffd3c29d4d 100644 --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -631,6 +631,13 @@ def test_builtin_on_metaclasses(self): # Testing that the subclasses section does not appear self.assertNotIn('Built-in subclasses', text) + def test_fail_help_output_redirect(self): + with StringIO() as buf: + helper = pydoc.Helper(output=buf) + helper.help("abd") + expected = missing_pattern % "abd" + self.assertEqual(expected, buf.getvalue().strip().replace('\n', os.linesep)) + @unittest.skipIf(hasattr(sys, 'gettrace') and sys.gettrace(), 'trace function introduces __locals__ unexpectedly') @requires_docstrings diff --git a/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst b/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst new file mode 100644 index 0000000000000..efaf5db10f3e1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst @@ -0,0 +1 @@ +Make pydoc.doc catch bad module ImportError when output stream is not None. From webhook-mailer at python.org Sat Jul 1 19:44:10 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 01 Jul 2023 23:44:10 -0000 Subject: [Python-checkins] gh-106320: Remove _PyInterpreterState_Get() alias (#106321) Message-ID: https://github.com/python/cpython/commit/18b1fdebe0cd5e601aa341227c13ec9d89bdf32c commit: 18b1fdebe0cd5e601aa341227c13ec9d89bdf32c branch: main author: Victor Stinner committer: vstinner date: 2023-07-01T23:44:07Z summary: gh-106320: Remove _PyInterpreterState_Get() alias (#106321) Replace calls to the (removed) slow _PyInterpreterState_Get() with fast inlined _PyInterpreterState_GET() function. files: A Misc/NEWS.d/next/C API/2023-07-02-00-00-20.gh-issue-106320.tZWcvG.rst M Doc/whatsnew/3.13.rst M Include/cpython/pystate.h M Include/internal/pycore_pystate.h M Modules/_threadmodule.c M Objects/typeobject.c M Objects/unicodeobject.c M Python/ceval_gil.c M Python/hamt.c M Python/import.c M Python/instrumentation.c diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 628945f7d5804..8f737c5935f67 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -605,3 +605,10 @@ Removed * Remove ``cpython/pytime.h`` header file: it only contained private functions. (Contributed by Victor Stinner in :gh:`106316`.) + +* Remove ``_PyInterpreterState_Get()`` alias to + :c:func:`PyInterpreterState_Get()` which was kept for backward compatibility + with Python 3.8. The `pythoncapi-compat project + `__ can be used to get + :c:func:`PyInterpreterState_Get()` on Python 3.8 and older. + (Contributed by Victor Stinner in :gh:`106320`.) diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index f33c72d4cf4d2..7d9e41bc2dd7a 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -261,9 +261,6 @@ struct _ts { /* other API */ -// Alias for backward compatibility with Python 3.8 -#define _PyInterpreterState_Get PyInterpreterState_Get - /* An alias for the internal _PyThreadState_New(), kept for stable ABI compatibility. */ PyAPI_FUNC(PyThreadState *) _PyThreadState_Prealloc(PyInterpreterState *); @@ -295,7 +292,7 @@ PyAPI_FUNC(int) PyGILState_Check(void); This function doesn't check for error. Return NULL before _PyGILState_Init() is called and after _PyGILState_Fini() is called. - See also _PyInterpreterState_Get() and _PyInterpreterState_GET(). */ + See also PyInterpreterState_Get() and _PyInterpreterState_GET(). */ PyAPI_FUNC(PyInterpreterState *) _PyGILState_GetInterpreterStateUnsafe(void); /* The implementation of sys._current_frames() Returns a dict mapping diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h index 43652c4405ec1..63fc6b2129ce2 100644 --- a/Include/internal/pycore_pystate.h +++ b/Include/internal/pycore_pystate.h @@ -108,7 +108,7 @@ _Py_EnsureFuncTstateNotNULL(const char *func, PyThreadState *tstate) The caller must hold the GIL. - See also _PyInterpreterState_Get() + See also PyInterpreterState_Get() and _PyGILState_GetInterpreterStateUnsafe(). */ static inline PyInterpreterState* _PyInterpreterState_GET(void) { PyThreadState *tstate = _PyThreadState_GET(); diff --git a/Misc/NEWS.d/next/C API/2023-07-02-00-00-20.gh-issue-106320.tZWcvG.rst b/Misc/NEWS.d/next/C API/2023-07-02-00-00-20.gh-issue-106320.tZWcvG.rst new file mode 100644 index 0000000000000..145d2ce7b0ceb --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-02-00-00-20.gh-issue-106320.tZWcvG.rst @@ -0,0 +1,3 @@ +Remove ``_PyInterpreterState_Get()`` alias to +:c:func:`PyInterpreterState_Get()` which was kept for backward compatibility +with Python 3.8. Patch by Victor Stinner. diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index 7f8b3cbbfcec7..f570b4e700715 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -1104,7 +1104,7 @@ thread_run(void *boot_raw) static PyObject * thread_daemon_threads_allowed(PyObject *module, PyObject *Py_UNUSED(ignored)) { - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); if (interp->feature_flags & Py_RTFLAGS_DAEMON_THREADS) { Py_RETURN_TRUE; } diff --git a/Objects/typeobject.c b/Objects/typeobject.c index e67945db9af33..3d3a63a75bd2f 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6362,7 +6362,7 @@ object___reduce_ex___impl(PyObject *self, int protocol) /*[clinic end generated code: output=2e157766f6b50094 input=f326b43fb8a4c5ff]*/ { #define objreduce \ - (_Py_INTERP_CACHED_OBJECT(_PyInterpreterState_Get(), objreduce)) + (_Py_INTERP_CACHED_OBJECT(_PyInterpreterState_GET(), objreduce)) PyObject *reduce, *res; if (objreduce == NULL) { @@ -9688,7 +9688,7 @@ resolve_slotdups(PyTypeObject *type, PyObject *name) /* XXX Maybe this could be optimized more -- but is it worth it? */ /* pname and ptrs act as a little cache */ - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); #define pname _Py_INTERP_CACHED_OBJECT(interp, type_slots_pname) #define ptrs _Py_INTERP_CACHED_OBJECT(interp, type_slots_ptrs) pytype_slotdef *p, **pp; diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 4434bf19d43b7..74def5ada5134 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -5831,7 +5831,7 @@ _PyUnicode_DecodeUnicodeEscapeInternal(const char *s, PyObject *errorHandler = NULL; PyObject *exc = NULL; _PyUnicode_Name_CAPI *ucnhash_capi; - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); // so we can remember if we've seen an invalid escape char or not *first_invalid_escape = NULL; diff --git a/Python/ceval_gil.c b/Python/ceval_gil.c index bb1279f46cf9f..c6b1f9ef689ee 100644 --- a/Python/ceval_gil.c +++ b/Python/ceval_gil.c @@ -491,7 +491,7 @@ take_gil(PyThreadState *tstate) void _PyEval_SetSwitchInterval(unsigned long microseconds) { - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); struct _gil_runtime_state *gil = interp->ceval.gil; assert(gil != NULL); gil->interval = microseconds; @@ -499,7 +499,7 @@ void _PyEval_SetSwitchInterval(unsigned long microseconds) unsigned long _PyEval_GetSwitchInterval(void) { - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); struct _gil_runtime_state *gil = interp->ceval.gil; assert(gil != NULL); return gil->interval; diff --git a/Python/hamt.c b/Python/hamt.c index 8cb94641bef25..c78b5a7fab94f 100644 --- a/Python/hamt.c +++ b/Python/hamt.c @@ -2425,7 +2425,7 @@ hamt_alloc(void) } #define _empty_hamt \ - (&_Py_INTERP_SINGLETON(_PyInterpreterState_Get(), hamt_empty)) + (&_Py_INTERP_SINGLETON(_PyInterpreterState_GET(), hamt_empty)) PyHamtObject * _PyHamt_New(void) diff --git a/Python/import.c b/Python/import.c index 324fe3812bdd4..fcc0346e7f15a 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1123,7 +1123,7 @@ check_multi_interp_extensions(PyInterpreterState *interp) int _PyImport_CheckSubinterpIncompatibleExtensionAllowed(const char *name) { - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); if (check_multi_interp_extensions(interp)) { assert(!_Py_IsMainInterpreter(interp)); PyErr_Format(PyExc_ImportError, diff --git a/Python/instrumentation.c b/Python/instrumentation.c index 86d014ededc8e..3253a0ea13033 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -1219,7 +1219,7 @@ _Py_call_instrumentation_instruction(PyThreadState *tstate, _PyInterpreterFrame* PyObject * _PyMonitoring_RegisterCallback(int tool_id, int event_id, PyObject *obj) { - PyInterpreterState *is = _PyInterpreterState_Get(); + PyInterpreterState *is = _PyInterpreterState_GET(); assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); assert(0 <= event_id && event_id < PY_MONITORING_EVENTS); PyObject *callback = is->monitoring_callables[tool_id][event_id]; @@ -1678,7 +1678,7 @@ int _PyMonitoring_SetEvents(int tool_id, _PyMonitoringEventSet events) { assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); assert(events < (1 << PY_MONITORING_UNGROUPED_EVENTS)); if (check_tool(interp, tool_id)) { return -1; @@ -1696,7 +1696,7 @@ int _PyMonitoring_SetLocalEvents(PyCodeObject *code, int tool_id, _PyMonitoringEventSet events) { assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); assert(events < (1 << PY_MONITORING_UNGROUPED_EVENTS)); if (check_tool(interp, tool_id)) { return -1; @@ -1758,7 +1758,7 @@ monitoring_use_tool_id_impl(PyObject *module, int tool_id, PyObject *name) PyErr_SetString(PyExc_ValueError, "tool name must be a str"); return NULL; } - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); if (interp->monitoring_tool_names[tool_id] != NULL) { PyErr_Format(PyExc_ValueError, "tool %d is already in use", tool_id); return NULL; @@ -1782,7 +1782,7 @@ monitoring_free_tool_id_impl(PyObject *module, int tool_id) if (check_valid_tool(tool_id)) { return NULL; } - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); Py_CLEAR(interp->monitoring_tool_names[tool_id]); Py_RETURN_NONE; } @@ -1804,7 +1804,7 @@ monitoring_get_tool_impl(PyObject *module, int tool_id) if (check_valid_tool(tool_id)) { return NULL; } - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); PyObject *name = interp->monitoring_tool_names[tool_id]; if (name == NULL) { Py_RETURN_NONE; @@ -1865,7 +1865,7 @@ monitoring_get_events_impl(PyObject *module, int tool_id) if (check_valid_tool(tool_id)) { return -1; } - _Py_Monitors *m = &_PyInterpreterState_Get()->monitors; + _Py_Monitors *m = &_PyInterpreterState_GET()->monitors; _PyMonitoringEventSet event_set = get_events(m, tool_id); return event_set; } @@ -1990,7 +1990,7 @@ monitoring_restart_events_impl(PyObject *module) * last restart version > instrumented version for all code objects * last restart version < current version */ - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); interp->last_restart_version = interp->monitoring_version + 1; interp->monitoring_version = interp->last_restart_version + 1; if (instrument_all_executing_code_objects(interp)) { @@ -2038,7 +2038,7 @@ static PyObject * monitoring__all_events_impl(PyObject *module) /*[clinic end generated code: output=6b7581e2dbb690f6 input=62ee9672c17b7f0e]*/ { - PyInterpreterState *interp = _PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); PyObject *res = PyDict_New(); if (res == NULL) { return NULL; From webhook-mailer at python.org Sat Jul 1 20:49:21 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 02 Jul 2023 00:49:21 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyTraceMalloc C API functions (#106324) Message-ID: https://github.com/python/cpython/commit/feb51f3a6443d7c0148e2e7be2ed58b4c69fa265 commit: feb51f3a6443d7c0148e2e7be2ed58b4c69fa265 branch: main author: Victor Stinner committer: vstinner date: 2023-07-02T00:49:18Z summary: gh-106320: Remove private _PyTraceMalloc C API functions (#106324) * Remove private _PyTraceMalloc C API functions: move them to the internal C API. * Don't export most of these functions anymore, but still export _PyTraceMalloc_GetTraceback() used by tests. * Rename Include/tracemalloc.h to Include/cpython/tracemalloc.h files: A Include/cpython/tracemalloc.h D Include/tracemalloc.h M Include/Python.h M Include/internal/pycore_tracemalloc.h M Lib/test/test_tracemalloc.py M Makefile.pre.in M Modules/_testcapi/mem.c M Modules/_testinternalcapi.c M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters diff --git a/Include/Python.h b/Include/Python.h index 183d07cc539b4..07f6c202a7f12 100644 --- a/Include/Python.h +++ b/Include/Python.h @@ -103,7 +103,7 @@ #include "pystrcmp.h" #include "fileutils.h" #include "cpython/pyfpe.h" -#include "tracemalloc.h" +#include "cpython/tracemalloc.h" #include "cpython/optimizer.h" #endif /* !Py_PYTHON_H */ diff --git a/Include/cpython/tracemalloc.h b/Include/cpython/tracemalloc.h new file mode 100644 index 0000000000000..61a16ea9a9f3e --- /dev/null +++ b/Include/cpython/tracemalloc.h @@ -0,0 +1,26 @@ +#ifndef Py_LIMITED_API +#ifndef Py_TRACEMALLOC_H +#define Py_TRACEMALLOC_H + +/* Track an allocated memory block in the tracemalloc module. + Return 0 on success, return -1 on error (failed to allocate memory to store + the trace). + + Return -2 if tracemalloc is disabled. + + If memory block is already tracked, update the existing trace. */ +PyAPI_FUNC(int) PyTraceMalloc_Track( + unsigned int domain, + uintptr_t ptr, + size_t size); + +/* Untrack an allocated memory block in the tracemalloc module. + Do nothing if the block was not tracked. + + Return -2 if tracemalloc is disabled, otherwise return 0. */ +PyAPI_FUNC(int) PyTraceMalloc_Untrack( + unsigned int domain, + uintptr_t ptr); + +#endif // !Py_TRACEMALLOC_H +#endif // !Py_LIMITED_API diff --git a/Include/internal/pycore_tracemalloc.h b/Include/internal/pycore_tracemalloc.h index d086adc61c319..cfc4d1fe43999 100644 --- a/Include/internal/pycore_tracemalloc.h +++ b/Include/internal/pycore_tracemalloc.h @@ -117,6 +117,51 @@ struct _tracemalloc_runtime_state { } +/* Get the traceback where a memory block was allocated. + + Return a tuple of (filename: str, lineno: int) tuples. + + Return None if the tracemalloc module is disabled or if the memory block + is not tracked by tracemalloc. + + Raise an exception and return NULL on error. */ +PyAPI_FUNC(PyObject*) _PyTraceMalloc_GetTraceback( + unsigned int domain, + uintptr_t ptr); + +/* Return non-zero if tracemalloc is tracing */ +extern int _PyTraceMalloc_IsTracing(void); + +/* Clear the tracemalloc traces */ +extern void _PyTraceMalloc_ClearTraces(void); + +/* Clear the tracemalloc traces */ +extern PyObject* _PyTraceMalloc_GetTraces(void); + +/* Clear tracemalloc traceback for an object */ +extern PyObject* _PyTraceMalloc_GetObjectTraceback(PyObject *obj); + +/* Initialize tracemalloc */ +extern int _PyTraceMalloc_Init(void); + +/* Start tracemalloc */ +extern int _PyTraceMalloc_Start(int max_nframe); + +/* Stop tracemalloc */ +extern void _PyTraceMalloc_Stop(void); + +/* Get the tracemalloc traceback limit */ +extern int _PyTraceMalloc_GetTracebackLimit(void); + +/* Get the memory usage of tracemalloc in bytes */ +extern size_t _PyTraceMalloc_GetMemory(void); + +/* Get the current size and peak size of traced memory blocks as a 2-tuple */ +extern PyObject* _PyTraceMalloc_GetTracedMemory(void); + +/* Set the peak size of traced memory blocks to the current size */ +extern void _PyTraceMalloc_ResetPeak(void); + #ifdef __cplusplus } #endif diff --git a/Include/tracemalloc.h b/Include/tracemalloc.h deleted file mode 100644 index 580027a8e365e..0000000000000 --- a/Include/tracemalloc.h +++ /dev/null @@ -1,72 +0,0 @@ -#ifndef Py_TRACEMALLOC_H -#define Py_TRACEMALLOC_H - -#ifndef Py_LIMITED_API -/* Track an allocated memory block in the tracemalloc module. - Return 0 on success, return -1 on error (failed to allocate memory to store - the trace). - - Return -2 if tracemalloc is disabled. - - If memory block is already tracked, update the existing trace. */ -PyAPI_FUNC(int) PyTraceMalloc_Track( - unsigned int domain, - uintptr_t ptr, - size_t size); - -/* Untrack an allocated memory block in the tracemalloc module. - Do nothing if the block was not tracked. - - Return -2 if tracemalloc is disabled, otherwise return 0. */ -PyAPI_FUNC(int) PyTraceMalloc_Untrack( - unsigned int domain, - uintptr_t ptr); - -/* Get the traceback where a memory block was allocated. - - Return a tuple of (filename: str, lineno: int) tuples. - - Return None if the tracemalloc module is disabled or if the memory block - is not tracked by tracemalloc. - - Raise an exception and return NULL on error. */ -PyAPI_FUNC(PyObject*) _PyTraceMalloc_GetTraceback( - unsigned int domain, - uintptr_t ptr); - -/* Return non-zero if tracemalloc is tracing */ -PyAPI_FUNC(int) _PyTraceMalloc_IsTracing(void); - -/* Clear the tracemalloc traces */ -PyAPI_FUNC(void) _PyTraceMalloc_ClearTraces(void); - -/* Clear the tracemalloc traces */ -PyAPI_FUNC(PyObject *) _PyTraceMalloc_GetTraces(void); - -/* Clear tracemalloc traceback for an object */ -PyAPI_FUNC(PyObject *) _PyTraceMalloc_GetObjectTraceback(PyObject *obj); - -/* Initialize tracemalloc */ -PyAPI_FUNC(int) _PyTraceMalloc_Init(void); - -/* Start tracemalloc */ -PyAPI_FUNC(int) _PyTraceMalloc_Start(int max_nframe); - -/* Stop tracemalloc */ -PyAPI_FUNC(void) _PyTraceMalloc_Stop(void); - -/* Get the tracemalloc traceback limit */ -PyAPI_FUNC(int) _PyTraceMalloc_GetTracebackLimit(void); - -/* Get the memory usage of tracemalloc in bytes */ -PyAPI_FUNC(size_t) _PyTraceMalloc_GetMemory(void); - -/* Get the current size and peak size of traced memory blocks as a 2-tuple */ -PyAPI_FUNC(PyObject *) _PyTraceMalloc_GetTracedMemory(void); - -/* Set the peak size of traced memory blocks to the current size */ -PyAPI_FUNC(void) _PyTraceMalloc_ResetPeak(void); - -#endif - -#endif /* !Py_TRACEMALLOC_H */ diff --git a/Lib/test/test_tracemalloc.py b/Lib/test/test_tracemalloc.py index 94bcee302fe73..4af4ca3b97723 100644 --- a/Lib/test/test_tracemalloc.py +++ b/Lib/test/test_tracemalloc.py @@ -11,8 +11,10 @@ try: import _testcapi + import _testinternalcapi except ImportError: _testcapi = None + _testinternalcapi = None EMPTY_STRING_SIZE = sys.getsizeof(b'') @@ -1008,7 +1010,7 @@ def tearDown(self): tracemalloc.stop() def get_traceback(self): - frames = _testcapi.tracemalloc_get_traceback(self.domain, self.ptr) + frames = _testinternalcapi._PyTraceMalloc_GetTraceback(self.domain, self.ptr) if frames is not None: return tracemalloc.Traceback(frames) else: diff --git a/Makefile.pre.in b/Makefile.pre.in index 54d0516ad4423..e788e590dcbb4 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1658,7 +1658,6 @@ PYTHON_HEADERS= \ $(srcdir)/Include/structseq.h \ $(srcdir)/Include/sysmodule.h \ $(srcdir)/Include/traceback.h \ - $(srcdir)/Include/tracemalloc.h \ $(srcdir)/Include/tupleobject.h \ $(srcdir)/Include/unicodeobject.h \ $(srcdir)/Include/warnings.h \ @@ -1713,6 +1712,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/cpython/setobject.h \ $(srcdir)/Include/cpython/sysmodule.h \ $(srcdir)/Include/cpython/traceback.h \ + $(srcdir)/Include/cpython/tracemalloc.h \ $(srcdir)/Include/cpython/tupleobject.h \ $(srcdir)/Include/cpython/unicodeobject.h \ $(srcdir)/Include/cpython/warnings.h \ diff --git a/Modules/_testcapi/mem.c b/Modules/_testcapi/mem.c index af32e9668dda2..16bda66af554a 100644 --- a/Modules/_testcapi/mem.c +++ b/Modules/_testcapi/mem.c @@ -655,23 +655,6 @@ tracemalloc_untrack(PyObject *self, PyObject *args) Py_RETURN_NONE; } -static PyObject * -tracemalloc_get_traceback(PyObject *self, PyObject *args) -{ - unsigned int domain; - PyObject *ptr_obj; - - if (!PyArg_ParseTuple(args, "IO", &domain, &ptr_obj)) { - return NULL; - } - void *ptr = PyLong_AsVoidPtr(ptr_obj); - if (PyErr_Occurred()) { - return NULL; - } - - return _PyTraceMalloc_GetTraceback(domain, (uintptr_t)ptr); -} - static PyMethodDef test_methods[] = { {"check_pyobject_forbidden_bytes_is_freed", check_pyobject_forbidden_bytes_is_freed, METH_NOARGS}, @@ -697,7 +680,6 @@ static PyMethodDef test_methods[] = { // Tracemalloc tests {"tracemalloc_track", tracemalloc_track, METH_VARARGS}, {"tracemalloc_untrack", tracemalloc_untrack, METH_VARARGS}, - {"tracemalloc_get_traceback", tracemalloc_get_traceback, METH_VARARGS}, {NULL}, }; diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index c598d7edef8ae..f6ae389ea0567 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -1216,6 +1216,24 @@ test_pytime_object_to_timespec(PyObject *self, PyObject *args) } +static PyObject * +tracemalloc_get_traceback(PyObject *self, PyObject *args) +{ + unsigned int domain; + PyObject *ptr_obj; + + if (!PyArg_ParseTuple(args, "IO", &domain, &ptr_obj)) { + return NULL; + } + void *ptr = PyLong_AsVoidPtr(ptr_obj); + if (PyErr_Occurred()) { + return NULL; + } + + return _PyTraceMalloc_GetTraceback(domain, (uintptr_t)ptr); +} + + static PyMethodDef module_functions[] = { {"get_configs", get_configs, METH_NOARGS}, {"get_recursion_depth", get_recursion_depth, METH_NOARGS}, @@ -1250,7 +1268,6 @@ static PyMethodDef module_functions[] = { {"get_uop_optimizer", get_uop_optimizer, METH_NOARGS, NULL}, {"pending_threadfunc", _PyCFunction_CAST(pending_threadfunc), METH_VARARGS | METH_KEYWORDS}, -// {"pending_fd_identify", pending_fd_identify, METH_VARARGS, NULL}, {"pending_identify", pending_identify, METH_VARARGS, NULL}, {"_PyTime_AsMicroseconds", test_PyTime_AsMicroseconds, METH_VARARGS}, {"_PyTime_AsMilliseconds", test_PyTime_AsMilliseconds, METH_VARARGS}, @@ -1266,6 +1283,7 @@ static PyMethodDef module_functions[] = { {"_PyTime_ObjectToTime_t", test_pytime_object_to_time_t, METH_VARARGS}, {"_PyTime_ObjectToTimespec", test_pytime_object_to_timespec, METH_VARARGS}, {"_PyTime_ObjectToTimeval", test_pytime_object_to_timeval, METH_VARARGS}, + {"_PyTraceMalloc_GetTraceback", tracemalloc_get_traceback, METH_VARARGS}, {NULL, NULL} /* sentinel */ }; diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 58abd871376a0..436381c3ea5db 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -179,6 +179,7 @@ + @@ -320,7 +321,6 @@ - diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 4d7b5ccf9a8a9..4d9acf4893872 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -219,9 +219,6 @@ Include - - Include - Include @@ -453,6 +450,9 @@ Include\cpython + + Include + Include\cpython From webhook-mailer at python.org Sat Jul 1 21:39:41 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 02 Jul 2023 01:39:41 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyInterpreterState functions (#106325) Message-ID: https://github.com/python/cpython/commit/8571b271e7d16fe87d669a2e1e50f5ae3732bb31 commit: 8571b271e7d16fe87d669a2e1e50f5ae3732bb31 branch: main author: Victor Stinner committer: vstinner date: 2023-07-02T01:39:38Z summary: gh-106320: Remove private _PyInterpreterState functions (#106325) Remove private _PyThreadState and _PyInterpreterState C API functions: move them to the internal C API (pycore_pystate.h and pycore_interp.h). Don't export most of these functions anymore, but still export functions used by tests. Remove _PyThreadState_Prealloc() and _PyThreadState_Init() from the C API, but keep it in the stable API. files: M Include/cpython/pystate.h M Include/internal/pycore_interp.h M Include/internal/pycore_pystate.h M Modules/_testcapimodule.c M Modules/_testinternalcapi.c M Modules/_xxsubinterpretersmodule.c M Python/compile.c M Python/frozenmain.c M Python/pystate.c diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index 7d9e41bc2dd7a..e08bcdaf19762 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -40,8 +40,6 @@ PyAPI_FUNC(int) _PyInterpreterState_HasFeature(PyInterpreterState *interp, PyAPI_FUNC(int) _PyInterpreterState_RequiresIDRef(PyInterpreterState *); PyAPI_FUNC(void) _PyInterpreterState_RequireIDRef(PyInterpreterState *, int); -PyAPI_FUNC(PyObject *) _PyInterpreterState_GetMainModule(PyInterpreterState *); - /* State unique per thread */ @@ -261,16 +259,10 @@ struct _ts { /* other API */ -/* An alias for the internal _PyThreadState_New(), - kept for stable ABI compatibility. */ -PyAPI_FUNC(PyThreadState *) _PyThreadState_Prealloc(PyInterpreterState *); - /* Similar to PyThreadState_Get(), but don't issue a fatal error * if it is NULL. */ PyAPI_FUNC(PyThreadState *) _PyThreadState_UncheckedGet(void); -PyAPI_FUNC(PyObject *) _PyThreadState_GetDict(PyThreadState *tstate); - // Disable tracing and profiling. PyAPI_FUNC(void) PyThreadState_EnterTracing(PyThreadState *tstate); @@ -295,16 +287,6 @@ PyAPI_FUNC(int) PyGILState_Check(void); See also PyInterpreterState_Get() and _PyInterpreterState_GET(). */ PyAPI_FUNC(PyInterpreterState *) _PyGILState_GetInterpreterStateUnsafe(void); -/* The implementation of sys._current_frames() Returns a dict mapping - thread id to that thread's current frame. -*/ -PyAPI_FUNC(PyObject *) _PyThread_CurrentFrames(void); - -/* The implementation of sys._current_exceptions() Returns a dict mapping - thread id to that thread's current exception. -*/ -PyAPI_FUNC(PyObject *) _PyThread_CurrentExceptions(void); - /* Routines for advanced debuggers, requested by David Beazley. Don't use unless you know what you are doing! */ PyAPI_FUNC(PyInterpreterState *) PyInterpreterState_Main(void); @@ -324,45 +306,6 @@ PyAPI_FUNC(void) _PyInterpreterState_SetEvalFrameFunc( PyInterpreterState *interp, _PyFrameEvalFunction eval_frame); -PyAPI_FUNC(const PyConfig*) _PyInterpreterState_GetConfig(PyInterpreterState *interp); - -/* Get a copy of the current interpreter configuration. - - Return 0 on success. Raise an exception and return -1 on error. - - The caller must initialize 'config', using PyConfig_InitPythonConfig() - for example. - - Python must be preinitialized to call this method. - The caller must hold the GIL. - - Once done with the configuration, PyConfig_Clear() must be called to clear - it. */ -PyAPI_FUNC(int) _PyInterpreterState_GetConfigCopy( - struct PyConfig *config); - -/* Set the configuration of the current interpreter. - - This function should be called during or just after the Python - initialization. - - Update the sys module with the new configuration. If the sys module was - modified directly after the Python initialization, these changes are lost. - - Some configuration like faulthandler or warnoptions can be updated in the - configuration, but don't reconfigure Python (don't enable/disable - faulthandler and don't reconfigure warnings filters). - - Return 0 on success. Raise an exception and return -1 on error. - - The configuration should come from _PyInterpreterState_GetConfigCopy(). */ -PyAPI_FUNC(int) _PyInterpreterState_SetConfig( - const struct PyConfig *config); - -// Get the configuration of the current interpreter. -// The caller must hold the GIL. -PyAPI_FUNC(const PyConfig*) _Py_GetConfig(void); - /* cross-interpreter data */ diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index 466ae6fbbdc4e..a1f00df12d6ac 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -238,6 +238,43 @@ PyAPI_FUNC(int) _PyInterpreterState_IDInitref(PyInterpreterState *); PyAPI_FUNC(int) _PyInterpreterState_IDIncref(PyInterpreterState *); PyAPI_FUNC(void) _PyInterpreterState_IDDecref(PyInterpreterState *); +PyAPI_FUNC(PyObject*) _PyInterpreterState_GetMainModule(PyInterpreterState *); + +extern const PyConfig* _PyInterpreterState_GetConfig(PyInterpreterState *interp); + +/* Get a copy of the current interpreter configuration. + + Return 0 on success. Raise an exception and return -1 on error. + + The caller must initialize 'config', using PyConfig_InitPythonConfig() + for example. + + Python must be preinitialized to call this method. + The caller must hold the GIL. + + Once done with the configuration, PyConfig_Clear() must be called to clear + it. */ +PyAPI_FUNC(int) _PyInterpreterState_GetConfigCopy( + struct PyConfig *config); + +/* Set the configuration of the current interpreter. + + This function should be called during or just after the Python + initialization. + + Update the sys module with the new configuration. If the sys module was + modified directly after the Python initialization, these changes are lost. + + Some configuration like faulthandler or warnoptions can be updated in the + configuration, but don't reconfigure Python (don't enable/disable + faulthandler and don't reconfigure warnings filters). + + Return 0 on success. Raise an exception and return -1 on error. + + The configuration should come from _PyInterpreterState_GetConfigCopy(). */ +PyAPI_FUNC(int) _PyInterpreterState_SetConfig( + const struct PyConfig *config); + #ifdef __cplusplus } #endif diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h index 63fc6b2129ce2..0659084194d29 100644 --- a/Include/internal/pycore_pystate.h +++ b/Include/internal/pycore_pystate.h @@ -123,9 +123,6 @@ static inline PyInterpreterState* _PyInterpreterState_GET(void) { PyAPI_FUNC(PyThreadState *) _PyThreadState_New(PyInterpreterState *interp); PyAPI_FUNC(void) _PyThreadState_Bind(PyThreadState *tstate); -// We keep this around exclusively for stable ABI compatibility. -PyAPI_FUNC(void) _PyThreadState_Init( - PyThreadState *tstate); PyAPI_FUNC(void) _PyThreadState_DeleteExcept(PyThreadState *tstate); extern void _PyThreadState_InitDetached(PyThreadState *, PyInterpreterState *); @@ -133,6 +130,18 @@ extern void _PyThreadState_ClearDetached(PyThreadState *); extern void _PyThreadState_BindDetached(PyThreadState *); extern void _PyThreadState_UnbindDetached(PyThreadState *); +PyAPI_FUNC(PyObject*) _PyThreadState_GetDict(PyThreadState *tstate); + +/* The implementation of sys._current_frames() Returns a dict mapping + thread id to that thread's current frame. +*/ +extern PyObject* _PyThread_CurrentFrames(void); + +/* The implementation of sys._current_exceptions() Returns a dict mapping + thread id to that thread's current exception. +*/ +extern PyObject* _PyThread_CurrentExceptions(void); + /* Other */ @@ -161,6 +170,10 @@ PyAPI_FUNC(int) _PyOS_InterruptOccurred(PyThreadState *tstate); #define HEAD_UNLOCK(runtime) \ PyThread_release_lock((runtime)->interpreters.mutex) +// Get the configuration of the current interpreter. +// The caller must hold the GIL. +PyAPI_FUNC(const PyConfig*) _Py_GetConfig(void); + #ifdef __cplusplus } diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index ec2e64f2f46fc..398450d804f5c 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -2714,11 +2714,6 @@ test_tstate_capi(PyObject *self, PyObject *Py_UNUSED(args)) assert(PyDict_Check(dict)); // dict is a borrowed reference - // private _PyThreadState_GetDict() - PyObject *dict2 = _PyThreadState_GetDict(tstate); - assert(dict2 == dict); - // dict2 is a borrowed reference - // PyThreadState_GetInterpreter() PyInterpreterState *interp = PyThreadState_GetInterpreter(tstate); assert(interp != NULL); diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index f6ae389ea0567..2e0609dcae5b7 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -1234,6 +1234,27 @@ tracemalloc_get_traceback(PyObject *self, PyObject *args) } +// Test PyThreadState C API +static PyObject * +test_tstate_capi(PyObject *self, PyObject *Py_UNUSED(args)) +{ + // PyThreadState_Get() + PyThreadState *tstate = PyThreadState_Get(); + assert(tstate != NULL); + + // test _PyThreadState_GetDict() + PyObject *dict = PyThreadState_GetDict(); + assert(dict != NULL); + // dict is a borrowed reference + + PyObject *dict2 = _PyThreadState_GetDict(tstate); + assert(dict2 == dict); + // dict2 is a borrowed reference + + Py_RETURN_NONE; +} + + static PyMethodDef module_functions[] = { {"get_configs", get_configs, METH_NOARGS}, {"get_recursion_depth", get_recursion_depth, METH_NOARGS}, @@ -1284,6 +1305,7 @@ static PyMethodDef module_functions[] = { {"_PyTime_ObjectToTimespec", test_pytime_object_to_timespec, METH_VARARGS}, {"_PyTime_ObjectToTimeval", test_pytime_object_to_timeval, METH_VARARGS}, {"_PyTraceMalloc_GetTraceback", tracemalloc_get_traceback, METH_VARARGS}, + {"test_tstate_capi", test_tstate_capi, METH_NOARGS, NULL}, {NULL, NULL} /* sentinel */ }; diff --git a/Modules/_xxsubinterpretersmodule.c b/Modules/_xxsubinterpretersmodule.c index 4801f37d6f6c5..40dea170fd1f8 100644 --- a/Modules/_xxsubinterpretersmodule.c +++ b/Modules/_xxsubinterpretersmodule.c @@ -1,8 +1,12 @@ - /* interpreters module */ /* low-level access to interpreter primitives */ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + #include "Python.h" +#include "pycore_interp.h" // _PyInterpreterState_GetMainModule() #include "interpreteridobject.h" diff --git a/Python/compile.c b/Python/compile.c index d83bf0855ec25..29ea2742fad0c 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -33,6 +33,7 @@ #include "pycore_compile.h" #include "pycore_intrinsics.h" #include "pycore_long.h" // _PyLong_GetZero() +#include "pycore_pystate.h" // _Py_GetConfig() #include "pycore_symtable.h" // PySTEntryObject, _PyFuture_FromAST() #include "opcode_metadata.h" // _PyOpcode_opcode_metadata, _PyOpcode_num_popped/pushed diff --git a/Python/frozenmain.c b/Python/frozenmain.c index f8be165f7671d..767f9804903a9 100644 --- a/Python/frozenmain.c +++ b/Python/frozenmain.c @@ -1,7 +1,8 @@ /* Python interpreter main program for frozen scripts */ #include "Python.h" -#include "pycore_runtime.h" // _PyRuntime_Initialize() +#include "pycore_pystate.h" // _Py_GetConfig() +#include "pycore_runtime.h" // _PyRuntime_Initialize() #include #ifdef MS_WINDOWS diff --git a/Python/pystate.c b/Python/pystate.c index 20b02ef22109e..50ce1d0660210 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -1166,7 +1166,7 @@ PyInterpreterState_GetDict(PyInterpreterState *interp) The GIL must be held. */ -PyInterpreterState * +PyInterpreterState* PyInterpreterState_Get(void) { PyThreadState *tstate = current_fast_get(&_PyRuntime); @@ -1408,7 +1408,7 @@ _PyThreadState_New(PyInterpreterState *interp) } // We keep this for stable ABI compabibility. -PyThreadState * +PyAPI_FUNC(PyThreadState*) _PyThreadState_Prealloc(PyInterpreterState *interp) { return _PyThreadState_New(interp); @@ -1416,7 +1416,7 @@ _PyThreadState_Prealloc(PyInterpreterState *interp) // We keep this around for (accidental) stable ABI compatibility. // Realistically, no extensions are using it. -void +PyAPI_FUNC(void) _PyThreadState_Init(PyThreadState *tstate) { Py_FatalError("_PyThreadState_Init() is for internal use only"); From webhook-mailer at python.org Sun Jul 2 02:07:50 2023 From: webhook-mailer at python.org (methane) Date: Sun, 02 Jul 2023 06:07:50 -0000 Subject: [Python-checkins] gh-104922: remove PY_SSIZE_T_CLEAN (#106315) Message-ID: https://github.com/python/cpython/commit/d5bd32fb48ef8db2586b09d951514d75437b6195 commit: d5bd32fb48ef8db2586b09d951514d75437b6195 branch: main author: Inada Naoki committer: methane date: 2023-07-02T15:07:46+09:00 summary: gh-104922: remove PY_SSIZE_T_CLEAN (#106315) files: M Modules/_bisectmodule.c M Modules/_bz2module.c M Modules/_codecsmodule.c M Modules/_ctypes/_ctypes.c M Modules/_cursesmodule.c M Modules/_dbmmodule.c M Modules/_elementtree.c M Modules/_gdbmmodule.c M Modules/_hashopenssl.c M Modules/_io/_iomodule.c M Modules/_io/bufferedio.c M Modules/_io/fileio.c M Modules/_io/iobase.c M Modules/_io/stringio.c M Modules/_io/textio.c M Modules/_io/winconsoleio.c M Modules/_localemodule.c M Modules/_lzmamodule.c M Modules/_multiprocessing/multiprocessing.h M Modules/_multiprocessing/posixshmem.c M Modules/_sqlite/connection.h M Modules/_sqlite/cursor.h M Modules/_sqlite/microprotocols.h M Modules/_sqlite/module.h M Modules/_sqlite/row.h M Modules/_sqlite/statement.h M Modules/_sqlite/util.h M Modules/_sre/sre.c M Modules/_ssl.c M Modules/_stat.c M Modules/_struct.c M Modules/_testbuffer.c M Modules/_testcapi/float.c M Modules/_testcapi/getargs.c M Modules/_testcapi/structmember.c M Modules/_testcapi/unicode.c M Modules/_testcapimodule.c M Modules/_testclinic.c M Modules/_testinternalcapi.c M Modules/_tkinter.c M Modules/_uuidmodule.c M Modules/_xxtestfuzz/_xxtestfuzz.c M Modules/arraymodule.c M Modules/binascii.c M Modules/cjkcodecs/cjkcodecs.h M Modules/cjkcodecs/multibytecodec.c M Modules/fcntlmodule.c M Modules/itertoolsmodule.c M Modules/mmapmodule.c M Modules/posixmodule.c M Modules/socketmodule.c M Modules/unicodedata.c M Modules/zlibmodule.c M Objects/bytearrayobject.c M Objects/bytes_methods.c M Objects/bytesobject.c M Objects/exceptions.c M Objects/fileobject.c M Objects/picklebufobject.c M Objects/unicodeobject.c M PC/winreg.c M Parser/pegen.h M Parser/tokenizer.c M Python/marshal.c diff --git a/Modules/_bisectmodule.c b/Modules/_bisectmodule.c index 60f4dc69dc05d..9e0fd336419b4 100644 --- a/Modules/_bisectmodule.c +++ b/Modules/_bisectmodule.c @@ -7,7 +7,6 @@ Converted to C by Dmitry Vasiliev (dima at hlabs.spb.ru). # define Py_BUILD_CORE_MODULE 1 #endif -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_call.h" // _PyObject_CallMethod() diff --git a/Modules/_bz2module.c b/Modules/_bz2module.c index 97bd44b4ac969..eeefe6034998c 100644 --- a/Modules/_bz2module.c +++ b/Modules/_bz2module.c @@ -1,7 +1,5 @@ /* _bz2 - Low-level Python interface to libbzip2. */ -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "structmember.h" // PyMemberDef diff --git a/Modules/_codecsmodule.c b/Modules/_codecsmodule.c index 4dfd134f1cb11..c31c1b6d6f2bb 100644 --- a/Modules/_codecsmodule.c +++ b/Modules/_codecsmodule.c @@ -30,7 +30,6 @@ Copyright (c) Corporation for National Research Initiatives. ------------------------------------------------------------------------ */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_codecs.h" // _PyCodec_Lookup() diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index 840d0df85e9c8..a9d8a2b9cb4ad 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -101,7 +101,6 @@ bytes(cdata) #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif -#define PY_SSIZE_T_CLEAN #include "Python.h" // windows.h must be included before pycore internal headers diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c index 5691a419a32f8..1f5afa6fcd898 100644 --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -104,8 +104,6 @@ static const char PyCursesVersion[] = "2.2"; # define Py_BUILD_CORE_MODULE 1 #endif -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_structseq.h" // _PyStructSequence_NewType() diff --git a/Modules/_dbmmodule.c b/Modules/_dbmmodule.c index 9908174c94c45..5be444d53e8da 100644 --- a/Modules/_dbmmodule.c +++ b/Modules/_dbmmodule.c @@ -2,7 +2,6 @@ /* DBM module using dictionary interface */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 2a0eac4d2f808..48280690a707a 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -11,8 +11,6 @@ *-------------------------------------------------------------------- */ -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "structmember.h" // PyMemberDef #include "expat.h" diff --git a/Modules/_gdbmmodule.c b/Modules/_gdbmmodule.c index 4dbb5741b2ede..bedbdc081425c 100644 --- a/Modules/_gdbmmodule.c +++ b/Modules/_gdbmmodule.c @@ -3,7 +3,6 @@ /* Author: Anthony Baxter, after dbmmodule.c */ /* Doc strings: Mitch Chapman */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "gdbm.h" diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c index 259db809b8664..011cb765ed82e 100644 --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -22,8 +22,6 @@ # define Py_BUILD_CORE_MODULE 1 #endif -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "pycore_hashtable.h" #include "hashlib.h" diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c index 7b06c1bee5a83..1a7920eebe0b6 100644 --- a/Modules/_io/_iomodule.c +++ b/Modules/_io/_iomodule.c @@ -7,7 +7,6 @@ Mostly written by Amaury Forgeot d'Arc */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "_iomodule.h" #include "pycore_pystate.h" // _PyInterpreterState_GET() diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c index f30d54a5e11b0..25376f8204283 100644 --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -7,7 +7,6 @@ Written by Amaury Forgeot d'Arc and Antoine Pitrou */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_object.h" diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c index 005c9ffe56f71..1a5b61301dec5 100644 --- a/Modules/_io/fileio.c +++ b/Modules/_io/fileio.c @@ -1,6 +1,5 @@ /* Author: Daniel Stutzbach */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_fileutils.h" // _Py_BEGIN_SUPPRESS_IPH #include "pycore_object.h" // _PyObject_GC_UNTRACK() diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c index 5cd679c68b128..729a708a3fc40 100644 --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -8,7 +8,6 @@ */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_call.h" // _PyObject_CallMethod() #include "pycore_long.h" // _PyLong_GetOne() diff --git a/Modules/_io/stringio.c b/Modules/_io/stringio.c index d4028451754ce..1960002d405ed 100644 --- a/Modules/_io/stringio.c +++ b/Modules/_io/stringio.c @@ -1,4 +1,3 @@ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include // offsetof() #include "pycore_object.h" diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c index c8e6792657cd5..f704875280da3 100644 --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -6,7 +6,6 @@ Written by Amaury Forgeot d'Arc and Antoine Pitrou */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_call.h" // _PyObject_CallMethod() #include "pycore_codecs.h" // _PyCodecInfo_GetIncrementalDecoder() diff --git a/Modules/_io/winconsoleio.c b/Modules/_io/winconsoleio.c index 15f3053957da6..452b12c138fa8 100644 --- a/Modules/_io/winconsoleio.c +++ b/Modules/_io/winconsoleio.c @@ -6,7 +6,6 @@ Written by Steve Dower */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_fileutils.h" // _Py_BEGIN_SUPPRESS_IPH #include "pycore_object.h" // _PyObject_GC_UNTRACK() diff --git a/Modules/_localemodule.c b/Modules/_localemodule.c index 1ada7305117bb..970530facd01b 100644 --- a/Modules/_localemodule.c +++ b/Modules/_localemodule.c @@ -9,7 +9,6 @@ This software comes with no warranty. Use at your own risk. ******************************************************************/ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_fileutils.h" diff --git a/Modules/_lzmamodule.c b/Modules/_lzmamodule.c index e34fbad230d51..ba0987ee0abca 100644 --- a/Modules/_lzmamodule.c +++ b/Modules/_lzmamodule.c @@ -5,8 +5,6 @@ */ -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "structmember.h" // PyMemberDef diff --git a/Modules/_multiprocessing/multiprocessing.h b/Modules/_multiprocessing/multiprocessing.h index dfc2a8e0799a6..47257fd5d9fb2 100644 --- a/Modules/_multiprocessing/multiprocessing.h +++ b/Modules/_multiprocessing/multiprocessing.h @@ -1,8 +1,6 @@ #ifndef MULTIPROCESSING_H #define MULTIPROCESSING_H -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "structmember.h" #include "pythread.h" diff --git a/Modules/_multiprocessing/posixshmem.c b/Modules/_multiprocessing/posixshmem.c index 88c93fe313785..debef3267f77d 100644 --- a/Modules/_multiprocessing/posixshmem.c +++ b/Modules/_multiprocessing/posixshmem.c @@ -2,8 +2,6 @@ posixshmem - A Python extension that provides shm_open() and shm_unlink() */ -#define PY_SSIZE_T_CLEAN - #include // for shm_open() and shm_unlink() diff --git a/Modules/_sqlite/connection.h b/Modules/_sqlite/connection.h index 1df92065a587a..7a748ee3ea0c5 100644 --- a/Modules/_sqlite/connection.h +++ b/Modules/_sqlite/connection.h @@ -23,7 +23,6 @@ #ifndef PYSQLITE_CONNECTION_H #define PYSQLITE_CONNECTION_H -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pythread.h" #include "structmember.h" diff --git a/Modules/_sqlite/cursor.h b/Modules/_sqlite/cursor.h index 0bcdddc3e2959..42f817af7c54a 100644 --- a/Modules/_sqlite/cursor.h +++ b/Modules/_sqlite/cursor.h @@ -23,7 +23,6 @@ #ifndef PYSQLITE_CURSOR_H #define PYSQLITE_CURSOR_H -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "statement.h" diff --git a/Modules/_sqlite/microprotocols.h b/Modules/_sqlite/microprotocols.h index 6bde9d01f4529..8a8c33525ee53 100644 --- a/Modules/_sqlite/microprotocols.h +++ b/Modules/_sqlite/microprotocols.h @@ -26,7 +26,6 @@ #ifndef PSYCOPG_MICROPROTOCOLS_H #define PSYCOPG_MICROPROTOCOLS_H 1 -#define PY_SSIZE_T_CLEAN #include /** exported functions **/ diff --git a/Modules/_sqlite/module.h b/Modules/_sqlite/module.h index daa22091d38ad..a4ca45cf6326a 100644 --- a/Modules/_sqlite/module.h +++ b/Modules/_sqlite/module.h @@ -23,7 +23,6 @@ #ifndef PYSQLITE_MODULE_H #define PYSQLITE_MODULE_H -#define PY_SSIZE_T_CLEAN #include "Python.h" #define LEGACY_TRANSACTION_CONTROL -1 diff --git a/Modules/_sqlite/row.h b/Modules/_sqlite/row.h index b51909817584b..d42b781e49317 100644 --- a/Modules/_sqlite/row.h +++ b/Modules/_sqlite/row.h @@ -23,7 +23,6 @@ #ifndef PYSQLITE_ROW_H #define PYSQLITE_ROW_H -#define PY_SSIZE_T_CLEAN #include "Python.h" typedef struct _Row diff --git a/Modules/_sqlite/statement.h b/Modules/_sqlite/statement.h index 11a6464b1a1c2..b18f170ebb070 100644 --- a/Modules/_sqlite/statement.h +++ b/Modules/_sqlite/statement.h @@ -23,7 +23,6 @@ #ifndef PYSQLITE_STATEMENT_H #define PYSQLITE_STATEMENT_H -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "connection.h" diff --git a/Modules/_sqlite/util.h b/Modules/_sqlite/util.h index a22bcd82d2a05..68b1a8cb67ace 100644 --- a/Modules/_sqlite/util.h +++ b/Modules/_sqlite/util.h @@ -23,7 +23,6 @@ #ifndef PYSQLITE_UTIL_H #define PYSQLITE_UTIL_H -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pythread.h" #include "sqlite3.h" diff --git a/Modules/_sre/sre.c b/Modules/_sre/sre.c index 328e4be2fb5e4..3f11916f9e172 100644 --- a/Modules/_sre/sre.c +++ b/Modules/_sre/sre.c @@ -38,8 +38,6 @@ static const char copyright[] = " SRE 2.2.2 Copyright (c) 1997-2002 by Secret Labs AB "; -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_moduleobject.h" // _PyModule_GetState() diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 4254fde0f5190..df1496925f678 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -25,8 +25,6 @@ #endif #define OPENSSL_NO_DEPRECATED 1 -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "pycore_weakref.h" // _PyWeakref_GET_REF() diff --git a/Modules/_stat.c b/Modules/_stat.c index 4218799103b59..9747d848cbacb 100644 --- a/Modules/_stat.c +++ b/Modules/_stat.c @@ -11,7 +11,6 @@ * */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #ifdef __cplusplus diff --git a/Modules/_struct.c b/Modules/_struct.c index 4f9478bd98095..0a6f076aac0c5 100644 --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -7,8 +7,6 @@ # define Py_BUILD_CORE_MODULE 1 #endif -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "pycore_moduleobject.h" // _PyModule_GetState() #include "structmember.h" // PyMemberDef diff --git a/Modules/_testbuffer.c b/Modules/_testbuffer.c index 63ed4dc6ca80e..5101834cfe138 100644 --- a/Modules/_testbuffer.c +++ b/Modules/_testbuffer.c @@ -1,9 +1,6 @@ /* C Extension module to test all aspects of PEP-3118. Written by Stefan Krah. */ - -#define PY_SSIZE_T_CLEAN - #include "Python.h" diff --git a/Modules/_testcapi/float.c b/Modules/_testcapi/float.c index 33cbda83a81af..cff53fb950fcc 100644 --- a/Modules/_testcapi/float.c +++ b/Modules/_testcapi/float.c @@ -1,5 +1,3 @@ -#define PY_SSIZE_T_CLEAN - #include "parts.h" #include "clinic/float.c.h" diff --git a/Modules/_testcapi/getargs.c b/Modules/_testcapi/getargs.c index a473c41f60af3..10a1c1dd05253 100644 --- a/Modules/_testcapi/getargs.c +++ b/Modules/_testcapi/getargs.c @@ -3,8 +3,6 @@ * APIs that parse and build arguments. */ -#define PY_SSIZE_T_CLEAN - #include "parts.h" static PyObject * diff --git a/Modules/_testcapi/structmember.c b/Modules/_testcapi/structmember.c index 0fb872a4328d6..8522dc962efa4 100644 --- a/Modules/_testcapi/structmember.c +++ b/Modules/_testcapi/structmember.c @@ -1,4 +1,3 @@ -#define PY_SSIZE_T_CLEAN #include "parts.h" #include // for offsetof() diff --git a/Modules/_testcapi/unicode.c b/Modules/_testcapi/unicode.c index 73929eaffc676..367d2af1413e5 100644 --- a/Modules/_testcapi/unicode.c +++ b/Modules/_testcapi/unicode.c @@ -1,6 +1,5 @@ #include // ptrdiff_t -#define PY_SSIZE_T_CLEAN #include "parts.h" static struct PyModuleDef *_testcapimodule = NULL; // set at initialization diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 398450d804f5c..ce1131743eb2a 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -17,8 +17,6 @@ /* Always enable assertions */ #undef NDEBUG -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "frameobject.h" // PyFrame_New #include "marshal.h" // PyMarshal_WriteLongToFile diff --git a/Modules/_testclinic.c b/Modules/_testclinic.c index 6ff55a2755cf5..26cdb4371ca24 100644 --- a/Modules/_testclinic.c +++ b/Modules/_testclinic.c @@ -5,8 +5,6 @@ /* Always enable assertions */ #undef NDEBUG -#define PY_SSIZE_T_CLEAN - #include "Python.h" diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 2e0609dcae5b7..971b8efb5fd9a 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -9,8 +9,6 @@ /* Always enable assertions */ #undef NDEBUG -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "frameobject.h" #include "interpreteridobject.h" // _PyInterpreterID_LookUp() diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c index 97e5b2f738aa2..76af803bd6eef 100644 --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -21,7 +21,6 @@ Copyright (C) 1994 Steen Lumholt. */ -#define PY_SSIZE_T_CLEAN #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif diff --git a/Modules/_uuidmodule.c b/Modules/_uuidmodule.c index ed3b2fedfd4d8..2f5be1c5144d8 100644 --- a/Modules/_uuidmodule.c +++ b/Modules/_uuidmodule.c @@ -3,8 +3,6 @@ * DCE compatible Universally Unique Identifier library. */ -#define PY_SSIZE_T_CLEAN - #include "Python.h" #if defined(HAVE_UUID_H) // AIX, FreeBSD, libuuid with pkgconf diff --git a/Modules/_xxtestfuzz/_xxtestfuzz.c b/Modules/_xxtestfuzz/_xxtestfuzz.c index e0694de6719df..a2dbabce71ed6 100644 --- a/Modules/_xxtestfuzz/_xxtestfuzz.c +++ b/Modules/_xxtestfuzz/_xxtestfuzz.c @@ -1,4 +1,3 @@ -#define PY_SSIZE_T_CLEAN #include #include #include diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c index 15f7766a8dd28..f43a23467976b 100644 --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -7,7 +7,6 @@ # define Py_BUILD_CORE_MODULE 1 #endif -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_call.h" // _PyObject_CallMethod() #include "pycore_moduleobject.h" // _PyModule_GetState() diff --git a/Modules/binascii.c b/Modules/binascii.c index 356947d43e43a..cf9328795c2bc 100644 --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -57,8 +57,6 @@ # define Py_BUILD_CORE_MODULE 1 #endif -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "pycore_long.h" // _PyLong_DigitValue #include "pycore_strhex.h" // _Py_strhex_bytes_with_sep() diff --git a/Modules/cjkcodecs/cjkcodecs.h b/Modules/cjkcodecs/cjkcodecs.h index 36bc7024df9ac..48cdcfb3ad308 100644 --- a/Modules/cjkcodecs/cjkcodecs.h +++ b/Modules/cjkcodecs/cjkcodecs.h @@ -7,7 +7,6 @@ #ifndef _CJKCODECS_H_ #define _CJKCODECS_H_ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "multibytecodec.h" diff --git a/Modules/cjkcodecs/multibytecodec.c b/Modules/cjkcodecs/multibytecodec.c index 1070a751ffe63..cf437d09f1fe8 100644 --- a/Modules/cjkcodecs/multibytecodec.c +++ b/Modules/cjkcodecs/multibytecodec.c @@ -4,7 +4,6 @@ * Written by Hye-Shik Chang */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "structmember.h" // PyMemberDef #include "multibytecodec.h" diff --git a/Modules/fcntlmodule.c b/Modules/fcntlmodule.c index 6ca0b62bc5dca..e530621fd269a 100644 --- a/Modules/fcntlmodule.c +++ b/Modules/fcntlmodule.c @@ -1,8 +1,5 @@ - /* fcntl module */ -#define PY_SSIZE_T_CLEAN - #include "Python.h" #ifdef HAVE_SYS_FILE_H diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index ae63bae79d5d0..13ff253f560b0 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -1,4 +1,3 @@ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_long.h" // _PyLong_GetZero() diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index 6bde9939eaa2c..fef27123a3282 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -22,7 +22,6 @@ # define Py_BUILD_CORE_MODULE 1 #endif -#define PY_SSIZE_T_CLEAN #include #include "pycore_bytesobject.h" // _PyBytes_Find() #include "pycore_fileutils.h" // _Py_stat_struct diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index d73886f14cb9e..b9f42476e6b82 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -7,8 +7,6 @@ of the compiler used. Different compilers define their own feature test macro, e.g. '_MSC_VER'. */ -#define PY_SSIZE_T_CLEAN - #include "Python.h" #ifdef __VXWORKS__ diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 3add80233522b..1d3f34b857a1d 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -105,7 +105,6 @@ Local naming conventions: # pragma weak inet_aton #endif -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_fileutils.h" // _Py_set_inheritable() #include "pycore_moduleobject.h" // _PyModule_GetState diff --git a/Modules/unicodedata.c b/Modules/unicodedata.c index 7be4d83c6f070..b6e50528a23c8 100644 --- a/Modules/unicodedata.c +++ b/Modules/unicodedata.c @@ -16,8 +16,6 @@ # define Py_BUILD_CORE_MODULE 1 #endif -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "pycore_ucnhash.h" // _PyUnicode_Name_CAPI #include "structmember.h" // PyMemberDef diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c index 534d065765f0f..c0f6b96f51bab 100644 --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -3,8 +3,6 @@ /* Windows users: read Python's PCbuild\readme.txt */ -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "structmember.h" // PyMemberDef #include "zlib.h" diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c index 944ec65b64f6a..18a24a369a64c 100644 --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -1,6 +1,5 @@ /* PyByteArray (bytearray) implementation */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_bytes_methods.h" diff --git a/Objects/bytes_methods.c b/Objects/bytes_methods.c index 33aa9c3db6e80..c1bc6383df30c 100644 --- a/Objects/bytes_methods.c +++ b/Objects/bytes_methods.c @@ -1,4 +1,3 @@ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_bytes_methods.h" diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c index bf54ec1d51d20..477bc4d31e812 100644 --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1,7 +1,5 @@ /* bytes object implementation */ -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_bytesobject.h" // _PyBytes_Find(), _PyBytes_Repeat() diff --git a/Objects/exceptions.c b/Objects/exceptions.c index 2bfa07ea3121f..85cf2cca16516 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -4,7 +4,6 @@ * Thanks go to Tim Peters and Michael Hudson for debugging. */ -#define PY_SSIZE_T_CLEAN #include #include #include "pycore_abstract.h" // _PyObject_RealIsSubclass() diff --git a/Objects/fileobject.c b/Objects/fileobject.c index e99e155f2b8c9..6d980a1b37916 100644 --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -1,6 +1,5 @@ /* File object implementation (what's left of it -- see io.py) */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_runtime.h" // _PyRuntime diff --git a/Objects/picklebufobject.c b/Objects/picklebufobject.c index aaa852cfbb05b..ca83a0a0806ce 100644 --- a/Objects/picklebufobject.c +++ b/Objects/picklebufobject.c @@ -1,6 +1,5 @@ /* PickleBuffer object implementation */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 74def5ada5134..79402714f23b9 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -38,7 +38,6 @@ OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_atomic_funcs.h" // _Py_atomic_size_get() diff --git a/PC/winreg.c b/PC/winreg.c index 279d48f792b96..aa2055c7e619d 100644 --- a/PC/winreg.c +++ b/PC/winreg.c @@ -12,7 +12,6 @@ */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_object.h" // _PyObject_Init() #include "pycore_moduleobject.h" diff --git a/Parser/pegen.h b/Parser/pegen.h index fe13d10e6b83e..5f29285951e81 100644 --- a/Parser/pegen.h +++ b/Parser/pegen.h @@ -1,7 +1,6 @@ #ifndef PEGEN_H #define PEGEN_H -#define PY_SSIZE_T_CLEAN #include #include #include diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 6bdf371a7adf0..f19198600fa01 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1,7 +1,6 @@ /* Tokenizer implementation */ -#define PY_SSIZE_T_CLEAN #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() diff --git a/Python/marshal.c b/Python/marshal.c index 7cfc7cc00306f..517220a4463cf 100644 --- a/Python/marshal.c +++ b/Python/marshal.c @@ -6,8 +6,6 @@ Version 3 of this protocol properly supports circular links and sharing. */ -#define PY_SSIZE_T_CLEAN - #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_code.h" // _PyCode_New() From webhook-mailer at python.org Sun Jul 2 10:15:31 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 02 Jul 2023 14:15:31 -0000 Subject: [Python-checkins] gh-106217: Truncate the issue body size of `new-bugs-announce-notifier` (#106329) Message-ID: https://github.com/python/cpython/commit/5950e7dbfcd9307c7e74184a1586ef99f9f35c3b commit: 5950e7dbfcd9307c7e74184a1586ef99f9f35c3b branch: main author: Nikita Sobolev committer: hugovk date: 2023-07-02T17:15:28+03:00 summary: gh-106217: Truncate the issue body size of `new-bugs-announce-notifier` (#106329) Co-authored-by: Hugo van Kemenade files: M .github/workflows/new-bugs-announce-notifier.yml diff --git a/.github/workflows/new-bugs-announce-notifier.yml b/.github/workflows/new-bugs-announce-notifier.yml index 73806c5d6d58a..e3572db670693 100644 --- a/.github/workflows/new-bugs-announce-notifier.yml +++ b/.github/workflows/new-bugs-announce-notifier.yml @@ -41,7 +41,10 @@ jobs: url : issue.data.html_url, labels : issue.data.labels.map(label => { return label.name }).join(", "), assignee : issue.data.assignees.map(assignee => { return assignee.login }), - body : issue.data.body + // We need to truncate the body size, because the max size for + // the whole payload is 16kb. We want to be safe and assume that + // body can take up to ~8kb of space. + body : issue.data.body.substring(8000) }; const data = { From webhook-mailer at python.org Sun Jul 2 11:54:38 2023 From: webhook-mailer at python.org (methane) Date: Sun, 02 Jul 2023 15:54:38 -0000 Subject: [Python-checkins] gh-104922: Doc: add note about PY_SSIZE_T_CLEAN (#106314) Message-ID: https://github.com/python/cpython/commit/20b7c79b9d67a761aaf818d3b92498ea0b0d80d9 commit: 20b7c79b9d67a761aaf818d3b92498ea0b0d80d9 branch: main author: Inada Naoki committer: methane date: 2023-07-03T00:54:35+09:00 summary: gh-104922: Doc: add note about PY_SSIZE_T_CLEAN (#106314) Add note about PY_SSIZE_T_CLEAN in extending and embedding document. files: M Doc/extending/embedding.rst M Doc/extending/extending.rst diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst index 1470029b80477..bd1abe36cbb80 100644 --- a/Doc/extending/embedding.rst +++ b/Doc/extending/embedding.rst @@ -87,6 +87,13 @@ perform some operation on a file. :: Py_ExitStatusException(status); } +.. note:: + + ``#define PY_SSIZE_T_CLEAN`` was used to indicate that ``Py_ssize_t`` should be + used in some APIs instead of ``int``. + It is not necessary since Python 3.13, but we keep it here for backward compatibility. + See :ref:`arg-parsing-string-and-buffers` for a description of this macro. + Setting :c:member:`PyConfig.program_name` should be called before :c:func:`Py_InitializeFromConfig` to inform the interpreter about paths to Python run-time libraries. Next, the Python interpreter is initialized with diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index ef93848840861..7d08bb9f6b8dd 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -69,8 +69,10 @@ the module and a copyright notice if you like). headers on some systems, you *must* include :file:`Python.h` before any standard headers are included. - It is recommended to always define ``PY_SSIZE_T_CLEAN`` before including - ``Python.h``. See :ref:`arg-parsing-string-and-buffers` for a description of this macro. + ``#define PY_SSIZE_T_CLEAN`` was used to indicate that ``Py_ssize_t`` should be + used in some APIs instead of ``int``. + It is not necessary since Python 3.13, but we keep it here for backward compatibility. + See :ref:`arg-parsing-string-and-buffers` for a description of this macro. All user-visible symbols defined by :file:`Python.h` have a prefix of ``Py`` or ``PY``, except those defined in standard header files. For convenience, and From webhook-mailer at python.org Sun Jul 2 12:11:48 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 02 Jul 2023 16:11:48 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyInterpreterState functions (#106335) Message-ID: https://github.com/python/cpython/commit/9a51a419619bb9dd1075d683708c57803c5d48c7 commit: 9a51a419619bb9dd1075d683708c57803c5d48c7 branch: main author: Victor Stinner committer: vstinner date: 2023-07-02T18:11:45+02:00 summary: gh-106320: Remove private _PyInterpreterState functions (#106335) Remove private _PyThreadState and _PyInterpreterState C API functions: move them to the internal C API (pycore_pystate.h and pycore_interp.h). Don't export most of these functions anymore, but still export functions used by tests. Remove _PyThreadState_Prealloc() and _PyThreadState_Init() from the C API, but keep it in the stable API. files: From webhook-mailer at python.org Sun Jul 2 12:37:41 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 02 Jul 2023 16:37:41 -0000 Subject: [Python-checkins] gh-106320: Use _PyInterpreterState_GET() (#106336) Message-ID: https://github.com/python/cpython/commit/bc7eb1708452da59c22782c487ae7f05f1788970 commit: bc7eb1708452da59c22782c487ae7f05f1788970 branch: main author: Victor Stinner committer: vstinner date: 2023-07-02T16:37:37Z summary: gh-106320: Use _PyInterpreterState_GET() (#106336) Replace PyInterpreterState_Get() with inlined _PyInterpreterState_GET(). files: M Modules/_asynciomodule.c M Modules/_io/bufferedio.c M Modules/_posixsubprocess.c M Modules/_testinternalcapi.c M Modules/_typingmodule.c M Modules/_winapi.c M Objects/typevarobject.c M Python/codecs.c M Python/instrumentation.c M Python/optimizer.c M Python/pystate.c diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 5b28f2dd28a22..05f94ef9ed281 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -527,7 +527,7 @@ future_init(FutureObj *fut, PyObject *loop) if (is_true < 0) { return -1; } - if (is_true && !_Py_IsInterpreterFinalizing(PyInterpreterState_Get())) { + if (is_true && !_Py_IsInterpreterFinalizing(_PyInterpreterState_GET())) { /* Only try to capture the traceback if the interpreter is not being finalized. The original motivation to add a `_Py_IsFinalizing()` call was to prevent SIGSEGV when a Future is created in a __del__ diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c index 25376f8204283..9c6a9cddba225 100644 --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -292,7 +292,7 @@ _enter_buffered_busy(buffered *self) "reentrant call inside %R", self); return 0; } - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); relax_locking = _Py_IsInterpreterFinalizing(interp); Py_BEGIN_ALLOW_THREADS if (!relax_locking) diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index 52e1d2faf3e9f..6caa4b8852911 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -1025,7 +1025,7 @@ subprocess_fork_exec_impl(PyObject *module, PyObject *process_args, int *c_fds_to_keep = NULL; Py_ssize_t fds_to_keep_len = PyTuple_GET_SIZE(py_fds_to_keep); - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); if ((preexec_fn != Py_None) && interp->finalizing) { PyErr_SetString(PyExc_RuntimeError, "preexec_fn not supported at interpreter shutdown"); diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 971b8efb5fd9a..4875ee7bed168 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -559,7 +559,7 @@ static PyObject * set_eval_frame_default(PyObject *self, PyObject *Py_UNUSED(args)) { module_state *state = get_module_state(self); - _PyInterpreterState_SetEvalFrameFunc(PyInterpreterState_Get(), _PyEval_EvalFrameDefault); + _PyInterpreterState_SetEvalFrameFunc(_PyInterpreterState_GET(), _PyEval_EvalFrameDefault); Py_CLEAR(state->record_list); Py_RETURN_NONE; } @@ -587,7 +587,7 @@ set_eval_frame_record(PyObject *self, PyObject *list) return NULL; } Py_XSETREF(state->record_list, Py_NewRef(list)); - _PyInterpreterState_SetEvalFrameFunc(PyInterpreterState_Get(), record_eval); + _PyInterpreterState_SetEvalFrameFunc(_PyInterpreterState_GET(), record_eval); Py_RETURN_NONE; } @@ -883,7 +883,7 @@ pending_threadfunc(PyObject *self, PyObject *args, PyObject *kwargs) { return NULL; } - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); /* create the reference for the callbackwhile we hold the lock */ Py_INCREF(callable); diff --git a/Modules/_typingmodule.c b/Modules/_typingmodule.c index 39a124a26adf3..59d3a80a9305d 100644 --- a/Modules/_typingmodule.c +++ b/Modules/_typingmodule.c @@ -5,8 +5,9 @@ #endif #include "Python.h" -#include "internal/pycore_interp.h" -#include "internal/pycore_typevarobject.h" +#include "pycore_interp.h" +#include "pycore_pystate.h" // _PyInterpreterState_GET() +#include "pycore_typevarobject.h" #include "clinic/_typingmodule.c.h" /*[clinic input] @@ -44,7 +45,7 @@ PyDoc_STRVAR(typing_doc, static int _typing_exec(PyObject *m) { - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); #define EXPORT_TYPE(name, typename) \ if (PyModule_AddObjectRef(m, name, \ diff --git a/Modules/_winapi.c b/Modules/_winapi.c index a7e6bb582fc64..d4291f557b6a6 100644 --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -36,6 +36,7 @@ #include "Python.h" #include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_pystate.h" // _PyInterpreterState_GET #include "structmember.h" // PyMemberDef @@ -133,7 +134,7 @@ overlapped_dealloc(OverlappedObject *self) { /* The operation is no longer pending -- nothing to do. */ } - else if (_Py_IsInterpreterFinalizing(PyInterpreterState_Get())) + else if (_Py_IsInterpreterFinalizing(_PyInterpreterState_GET())) { /* The operation is still pending -- give a warning. This will probably only happen on Windows XP. */ diff --git a/Objects/typevarobject.c b/Objects/typevarobject.c index 406a6eb76e3a8..0b44f84b6f01d 100644 --- a/Objects/typevarobject.c +++ b/Objects/typevarobject.c @@ -1,6 +1,6 @@ // TypeVar, TypeVarTuple, and ParamSpec #include "Python.h" -#include "pycore_object.h" // _PyObject_GC_TRACK/UNTRACK +#include "pycore_object.h" // _PyObject_GC_TRACK/UNTRACK #include "pycore_typevarobject.h" #include "pycore_unionobject.h" // _Py_union_type_or #include "structmember.h" @@ -144,7 +144,7 @@ static int contains_typevartuple(PyTupleObject *params) { Py_ssize_t n = PyTuple_GET_SIZE(params); - PyTypeObject *tp = PyInterpreterState_Get()->cached_objects.typevartuple_type; + PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.typevartuple_type; for (Py_ssize_t i = 0; i < n; i++) { PyObject *param = PyTuple_GET_ITEM(params, i); if (Py_IS_TYPE(param, tp)) { @@ -165,7 +165,7 @@ unpack_typevartuples(PyObject *params) if (new_params == NULL) { return NULL; } - PyTypeObject *tp = PyInterpreterState_Get()->cached_objects.typevartuple_type; + PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.typevartuple_type; for (Py_ssize_t i = 0; i < n; i++) { PyObject *param = PyTuple_GET_ITEM(params, i); if (Py_IS_TYPE(param, tp)) { @@ -291,7 +291,7 @@ typevar_alloc(PyObject *name, PyObject *bound, PyObject *evaluate_bound, bool covariant, bool contravariant, bool infer_variance, PyObject *module) { - PyTypeObject *tp = PyInterpreterState_Get()->cached_objects.typevar_type; + PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.typevar_type; assert(tp != NULL); typevarobject *tv = PyObject_GC_New(typevarobject, tp); if (tv == NULL) { @@ -576,7 +576,7 @@ paramspecargs_repr(PyObject *self) { paramspecattrobject *psa = (paramspecattrobject *)self; - PyTypeObject *tp = PyInterpreterState_Get()->cached_objects.paramspec_type; + PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.paramspec_type; if (Py_IS_TYPE(psa->__origin__, tp)) { return PyUnicode_FromFormat("%U.args", ((paramspecobject *)psa->__origin__)->name); @@ -656,7 +656,7 @@ paramspeckwargs_repr(PyObject *self) { paramspecattrobject *psk = (paramspecattrobject *)self; - PyTypeObject *tp = PyInterpreterState_Get()->cached_objects.paramspec_type; + PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.paramspec_type; if (Py_IS_TYPE(psk->__origin__, tp)) { return PyUnicode_FromFormat("%U.kwargs", ((paramspecobject *)psk->__origin__)->name); @@ -789,14 +789,14 @@ static PyMemberDef paramspec_members[] = { static PyObject * paramspec_args(PyObject *self, void *unused) { - PyTypeObject *tp = PyInterpreterState_Get()->cached_objects.paramspecargs_type; + PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.paramspecargs_type; return (PyObject *)paramspecattr_new(tp, self); } static PyObject * paramspec_kwargs(PyObject *self, void *unused) { - PyTypeObject *tp = PyInterpreterState_Get()->cached_objects.paramspeckwargs_type; + PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.paramspeckwargs_type; return (PyObject *)paramspecattr_new(tp, self); } @@ -810,7 +810,7 @@ static paramspecobject * paramspec_alloc(PyObject *name, PyObject *bound, bool covariant, bool contravariant, bool infer_variance, PyObject *module) { - PyTypeObject *tp = PyInterpreterState_Get()->cached_objects.paramspec_type; + PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.paramspec_type; paramspecobject *ps = PyObject_GC_New(paramspecobject, tp); if (ps == NULL) { return NULL; @@ -1059,7 +1059,7 @@ static PyMemberDef typevartuple_members[] = { static typevartupleobject * typevartuple_alloc(PyObject *name, PyObject *module) { - PyTypeObject *tp = PyInterpreterState_Get()->cached_objects.typevartuple_type; + PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.typevartuple_type; typevartupleobject *tvt = PyObject_GC_New(typevartupleobject, tp); if (tvt == NULL) { return NULL; @@ -1598,7 +1598,7 @@ _Py_subscript_generic(PyThreadState* unused, PyObject *params) { params = unpack_typevartuples(params); - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); if (interp->cached_objects.generic_type == NULL) { PyErr_SetString(PyExc_SystemError, "Cannot find Generic type"); return NULL; diff --git a/Python/codecs.c b/Python/codecs.c index 1983f56ba204c..f9f23005debb1 100644 --- a/Python/codecs.c +++ b/Python/codecs.c @@ -11,7 +11,7 @@ Copyright (c) Corporation for National Research Initiatives. #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_interp.h" // PyInterpreterState.codec_search_path -#include "pycore_pyerrors.h" // _PyErr_FormatNote() +#include "pycore_pyerrors.h" // _PyErr_FormatNote() #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_ucnhash.h" // _PyUnicode_Name_CAPI #include @@ -55,7 +55,7 @@ int PyCodec_Register(PyObject *search_function) int PyCodec_Unregister(PyObject *search_function) { - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); PyObject *codec_search_path = interp->codec_search_path; /* Do nothing if codec_search_path is not created yet or was cleared. */ if (codec_search_path == NULL) { diff --git a/Python/instrumentation.c b/Python/instrumentation.c index 3253a0ea13033..03d7d2f215af7 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -1,5 +1,3 @@ - - #include "Python.h" #include "pycore_call.h" #include "pycore_frame.h" @@ -9,7 +7,7 @@ #include "pycore_object.h" #include "pycore_opcode.h" #include "pycore_pyerrors.h" -#include "pycore_pystate.h" +#include "pycore_pystate.h" // _PyInterpreterState_GET() /* Uncomment this to dump debugging output when assertions fail */ // #define INSTRUMENT_DEBUG 1 @@ -390,7 +388,7 @@ dump_instrumentation_data(PyCodeObject *code, int star, FILE*out) fprintf(out, "NULL\n"); return; } - dump_monitors("Global", PyInterpreterState_Get()->monitors, out); + dump_monitors("Global", _PyInterpreterState_GET()->monitors, out); dump_monitors("Code", data->local_monitors, out); dump_monitors("Active", data->active_monitors, out); int code_len = (int)Py_SIZE(code); @@ -449,7 +447,7 @@ sanity_check_instrumentation(PyCodeObject *code) if (data == NULL) { return; } - _Py_Monitors active_monitors = PyInterpreterState_Get()->monitors; + _Py_Monitors active_monitors = _PyInterpreterState_GET()->monitors; if (code->_co_monitoring) { _Py_Monitors local_monitors = code->_co_monitoring->local_monitors; active_monitors = monitors_or(active_monitors, local_monitors); @@ -740,7 +738,7 @@ remove_tools(PyCodeObject * code, int offset, int event, int tools) static bool tools_is_subset_for_event(PyCodeObject * code, int event, int tools) { - int global_tools = PyInterpreterState_Get()->monitors.tools[event]; + int global_tools = _PyInterpreterState_GET()->monitors.tools[event]; int local_tools = code->_co_monitoring->local_monitors.tools[event]; return tools == ((global_tools | local_tools) & tools); } diff --git a/Python/optimizer.c b/Python/optimizer.c index 9d77ab4ff879b..b00825ade16e0 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -1,10 +1,9 @@ - #include "Python.h" #include "opcode.h" #include "pycore_interp.h" #include "pycore_opcode.h" #include "opcode_metadata.h" -#include "pycore_pystate.h" +#include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_uops.h" #include "cpython/optimizer.h" #include @@ -125,7 +124,7 @@ _PyOptimizerObject _PyOptimizer_Default = { _PyOptimizerObject * PyUnstable_GetOptimizer(void) { - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); if (interp->optimizer == &_PyOptimizer_Default) { return NULL; } @@ -138,7 +137,7 @@ PyUnstable_GetOptimizer(void) void PyUnstable_SetOptimizer(_PyOptimizerObject *optimizer) { - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); if (optimizer == NULL) { optimizer = &_PyOptimizer_Default; } @@ -155,7 +154,7 @@ _PyOptimizer_BackEdge(_PyInterpreterFrame *frame, _Py_CODEUNIT *src, _Py_CODEUNI { PyCodeObject *code = (PyCodeObject *)frame->f_executable; assert(PyCode_Check(code)); - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); if (!has_space_for_executor(code, src)) { goto jump_to_destination; } diff --git a/Python/pystate.c b/Python/pystate.c index 50ce1d0660210..a9b404bd5c93e 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -2835,7 +2835,7 @@ _PyInterpreterState_GetConfig(PyInterpreterState *interp) int _PyInterpreterState_GetConfigCopy(PyConfig *config) { - PyInterpreterState *interp = PyInterpreterState_Get(); + PyInterpreterState *interp = _PyInterpreterState_GET(); PyStatus status = _PyConfig_Copy(config, &interp->config); if (PyStatus_Exception(status)) { From webhook-mailer at python.org Sun Jul 2 12:50:43 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sun, 02 Jul 2023 16:50:43 -0000 Subject: [Python-checkins] gh-106078: Move DecimalException to _decimal state (#106301) Message-ID: https://github.com/python/cpython/commit/dbefa88b27ee1f124f782363b139aee3f1ccf590 commit: dbefa88b27ee1f124f782363b139aee3f1ccf590 branch: main author: Charlie Zhao committer: erlend-aasland date: 2023-07-02T16:50:40Z summary: gh-106078: Move DecimalException to _decimal state (#106301) files: M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/globals-to-fix.tsv diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index b7cb19515b300..da62372500342 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -46,6 +46,9 @@ typedef struct { PyTypeObject *PyDec_Type; PyTypeObject *PyDecSignalDict_Type; PyTypeObject *DecimalTuple; + + /* Top level Exception; inherits from ArithmeticError */ + PyObject *DecimalException; } decimal_state; static decimal_state global_state; @@ -164,9 +167,6 @@ typedef struct { PyObject *ex; /* corresponding exception */ } DecCondMap; -/* Top level Exception; inherits from ArithmeticError */ -static PyObject *DecimalException = NULL; - /* Exceptions that correspond to IEEE signals */ #define SUBNORMAL 5 #define INEXACT 6 @@ -5902,10 +5902,10 @@ PyInit__decimal(void) CHECK_INT(PyModule_AddType(m, state->DecimalTuple)); /* Create top level exception */ - ASSIGN_PTR(DecimalException, PyErr_NewException( + ASSIGN_PTR(state->DecimalException, PyErr_NewException( "decimal.DecimalException", PyExc_ArithmeticError, NULL)); - CHECK_INT(PyModule_AddObject(m, "DecimalException", Py_NewRef(DecimalException))); + CHECK_INT(PyModule_AddType(m, (PyTypeObject *)state->DecimalException)); /* Create signal tuple */ ASSIGN_PTR(SignalTuple, PyTuple_New(SIGNAL_MAP_LEN)); @@ -5918,10 +5918,11 @@ PyInit__decimal(void) switch (cm->flag) { case MPD_Float_operation: - base = PyTuple_Pack(2, DecimalException, PyExc_TypeError); + base = PyTuple_Pack(2, state->DecimalException, PyExc_TypeError); break; case MPD_Division_by_zero: - base = PyTuple_Pack(2, DecimalException, PyExc_ZeroDivisionError); + base = PyTuple_Pack(2, state->DecimalException, + PyExc_ZeroDivisionError); break; case MPD_Overflow: base = PyTuple_Pack(2, signal_map[INEXACT].ex, @@ -5933,7 +5934,7 @@ PyInit__decimal(void) signal_map[SUBNORMAL].ex); break; default: - base = PyTuple_Pack(1, DecimalException); + base = PyTuple_Pack(1, state->DecimalException); break; } diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 1131edff265ee..8fdc54df2b072 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -393,7 +393,6 @@ Modules/xxlimited_35.c - Xxo_Type - ## exception types Modules/_ctypes/_ctypes.c - PyExc_ArgError - Modules/_cursesmodule.c - PyCursesError - -Modules/_decimal/_decimal.c - DecimalException - Modules/_tkinter.c - Tkinter_TclError - Modules/xxlimited_35.c - ErrorObject - Modules/xxmodule.c - ErrorObject - From webhook-mailer at python.org Sun Jul 2 17:20:02 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 02 Jul 2023 21:20:02 -0000 Subject: [Python-checkins] gh-106320: Add pycore_complexobject.h header file (#106339) Message-ID: https://github.com/python/cpython/commit/c38c66687f148991031914f12ebd5c42d6e26b5b commit: c38c66687f148991031914f12ebd5c42d6e26b5b branch: main author: Victor Stinner committer: vstinner date: 2023-07-02T21:19:59Z summary: gh-106320: Add pycore_complexobject.h header file (#106339) Add internal pycore_complexobject.h header file. Move _Py_c_xxx() functions and _PyComplex_FormatAdvancedWriter() function to this new header file. files: A Include/internal/pycore_complexobject.h M Include/cpython/complexobject.h M Makefile.pre.in M Modules/cmathmodule.c M Objects/complexobject.c M Objects/stringlib/unicode_format.h M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters diff --git a/Include/cpython/complexobject.h b/Include/cpython/complexobject.h index b7d7283ae8896..b524ec42c2437 100644 --- a/Include/cpython/complexobject.h +++ b/Include/cpython/complexobject.h @@ -7,16 +7,6 @@ typedef struct { double imag; } Py_complex; -/* Operations on complex numbers from complexmodule.c */ - -PyAPI_FUNC(Py_complex) _Py_c_sum(Py_complex, Py_complex); -PyAPI_FUNC(Py_complex) _Py_c_diff(Py_complex, Py_complex); -PyAPI_FUNC(Py_complex) _Py_c_neg(Py_complex); -PyAPI_FUNC(Py_complex) _Py_c_prod(Py_complex, Py_complex); -PyAPI_FUNC(Py_complex) _Py_c_quot(Py_complex, Py_complex); -PyAPI_FUNC(Py_complex) _Py_c_pow(Py_complex, Py_complex); -PyAPI_FUNC(double) _Py_c_abs(Py_complex); - /* Complex object interface */ /* @@ -31,14 +21,3 @@ typedef struct { PyAPI_FUNC(PyObject *) PyComplex_FromCComplex(Py_complex); PyAPI_FUNC(Py_complex) PyComplex_AsCComplex(PyObject *op); - -#ifdef Py_BUILD_CORE -/* Format the object based on the format_spec, as defined in PEP 3101 - (Advanced String Formatting). */ -extern int _PyComplex_FormatAdvancedWriter( - _PyUnicodeWriter *writer, - PyObject *obj, - PyObject *format_spec, - Py_ssize_t start, - Py_ssize_t end); -#endif // Py_BUILD_CORE diff --git a/Include/internal/pycore_complexobject.h b/Include/internal/pycore_complexobject.h new file mode 100644 index 0000000000000..fb344b7bb79bd --- /dev/null +++ b/Include/internal/pycore_complexobject.h @@ -0,0 +1,33 @@ +#ifndef Py_INTERNAL_COMPLEXOBJECT_H +#define Py_INTERNAL_COMPLEXOBJECT_H +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef Py_BUILD_CORE +# error "this header requires Py_BUILD_CORE define" +#endif + +/* Operations on complex numbers from complexmodule.c */ + +PyAPI_FUNC(Py_complex) _Py_c_sum(Py_complex, Py_complex); +PyAPI_FUNC(Py_complex) _Py_c_diff(Py_complex, Py_complex); +PyAPI_FUNC(Py_complex) _Py_c_neg(Py_complex); +PyAPI_FUNC(Py_complex) _Py_c_prod(Py_complex, Py_complex); +PyAPI_FUNC(Py_complex) _Py_c_quot(Py_complex, Py_complex); +PyAPI_FUNC(Py_complex) _Py_c_pow(Py_complex, Py_complex); +PyAPI_FUNC(double) _Py_c_abs(Py_complex); + +/* Format the object based on the format_spec, as defined in PEP 3101 + (Advanced String Formatting). */ +extern int _PyComplex_FormatAdvancedWriter( + _PyUnicodeWriter *writer, + PyObject *obj, + PyObject *format_spec, + Py_ssize_t start, + Py_ssize_t end); + +#ifdef __cplusplus +} +#endif +#endif // !Py_INTERNAL_COMPLEXOBJECT_H diff --git a/Makefile.pre.in b/Makefile.pre.in index e788e590dcbb4..7560d17e3796f 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1734,6 +1734,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/internal/pycore_code.h \ $(srcdir)/Include/internal/pycore_codecs.h \ $(srcdir)/Include/internal/pycore_compile.h \ + $(srcdir)/Include/internal/pycore_complexobject.h \ $(srcdir)/Include/internal/pycore_condvar.h \ $(srcdir)/Include/internal/pycore_context.h \ $(srcdir)/Include/internal/pycore_dict.h \ diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c index 1a31bdc824bb0..db8b321e72e8c 100644 --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -7,6 +7,7 @@ #endif #include "Python.h" +#include "pycore_complexobject.h" // _Py_c_neg() #include "pycore_pymath.h" // _PY_SHORT_FLOAT_REPR /* we need DBL_MAX, DBL_MIN, DBL_EPSILON, DBL_MANT_DIG and FLT_RADIX from float.h. We assume that FLT_RADIX is either 2 or 16. */ diff --git a/Objects/complexobject.c b/Objects/complexobject.c index aee03ddfb07ae..12968a63cd6fd 100644 --- a/Objects/complexobject.c +++ b/Objects/complexobject.c @@ -7,6 +7,7 @@ #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() +#include "pycore_complexobject.h" // _PyComplex_FormatAdvancedWriter() #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_object.h" // _PyObject_Init() #include "pycore_pymath.h" // _Py_ADJUST_ERANGE2() diff --git a/Objects/stringlib/unicode_format.h b/Objects/stringlib/unicode_format.h index f4ba0a92776a9..91c71ab5736c2 100644 --- a/Objects/stringlib/unicode_format.h +++ b/Objects/stringlib/unicode_format.h @@ -2,6 +2,7 @@ unicode_format.h -- implementation of str.format(). */ +#include "pycore_complexobject.h" // _PyComplex_FormatAdvancedWriter() #include "pycore_floatobject.h" // _PyFloat_FormatAdvancedWriter() /************************************************************************/ diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 436381c3ea5db..79ce2d3d14017 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -212,6 +212,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 4d9acf4893872..d47a22909e1e3 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -543,6 +543,9 @@ Include\internal + + Include\internal + Include\internal From webhook-mailer at python.org Sun Jul 2 17:57:02 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 02 Jul 2023 21:57:02 -0000 Subject: [Python-checkins] gh-106320: _testcapi avoids private _PyUnicode_EqualToASCIIString() (#106341) Message-ID: https://github.com/python/cpython/commit/4e3aa7cd312759922556ee96aa42033c375027e2 commit: 4e3aa7cd312759922556ee96aa42033c375027e2 branch: main author: Victor Stinner committer: vstinner date: 2023-07-02T21:56:58Z summary: gh-106320: _testcapi avoids private _PyUnicode_EqualToASCIIString() (#106341) Replace private _PyUnicode_EqualToASCIIString() with public PyUnicode_CompareWithASCIIString(). files: M Modules/_testcapi/unicode.c diff --git a/Modules/_testcapi/unicode.c b/Modules/_testcapi/unicode.c index 367d2af1413e5..9c2760c3f763a 100644 --- a/Modules/_testcapi/unicode.c +++ b/Modules/_testcapi/unicode.c @@ -1100,7 +1100,7 @@ test_string_from_format(PyObject *self, PyObject *Py_UNUSED(ignored)) } \ else if (result == NULL) \ return NULL; \ - else if (!_PyUnicode_EqualToASCIIString(result, EXPECTED)) { \ + else if (PyUnicode_CompareWithASCIIString(result, EXPECTED) != 0) { \ PyErr_Format(PyExc_AssertionError, \ "test_string_from_format: failed at \"%s\" " \ "expected \"%s\" got \"%s\"", \ From webhook-mailer at python.org Sun Jul 2 19:42:42 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sun, 02 Jul 2023 23:42:42 -0000 Subject: [Python-checkins] gh-104050: Add more type hints to Argument Clinic DSLParser() (#106343) Message-ID: https://github.com/python/cpython/commit/569887aa9da0da2eb2e2ad7d6a7496290f223ab0 commit: 569887aa9da0da2eb2e2ad7d6a7496290f223ab0 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-02T23:42:38Z summary: gh-104050: Add more type hints to Argument Clinic DSLParser() (#106343) Annotate the following method signatures: - state_dsl_start() - state_parameter_docstring_start() - state_parameters_start() Inverting ignore_line() logic, add type hints (including type guard) to it, and rename to valid_line(). files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 43b7a9d9ac4aa..27d3d57286888 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -29,7 +29,15 @@ from collections.abc import Callable from types import FunctionType, NoneType -from typing import Any, Final, NamedTuple, NoReturn, Literal, overload +from typing import ( + Any, + Final, + Literal, + NamedTuple, + NoReturn, + TypeGuard, + overload, +) # TODO: # @@ -4471,17 +4479,20 @@ def parse(self, block: Block) -> None: block.output = self.saved_output @staticmethod - def ignore_line(line): + def valid_line(line: str | None) -> TypeGuard[str]: + if line is None: + return False + # ignore comment-only lines if line.lstrip().startswith('#'): - return True + return False # Ignore empty lines too # (but not in docstring sections!) if not line.strip(): - return True + return False - return False + return True @staticmethod def calculate_indent(line: str) -> int: @@ -4497,9 +4508,9 @@ def next( if line is not None: self.state(line) - def state_dsl_start(self, line): + def state_dsl_start(self, line: str | None) -> None: # self.block = self.ClinicOutputBlock(self) - if self.ignore_line(line): + if not self.valid_line(line): return # is it a directive? @@ -4716,8 +4727,8 @@ def state_modulename_name(self, line): ps_start, ps_left_square_before, ps_group_before, ps_required, \ ps_optional, ps_group_after, ps_right_square_after = range(7) - def state_parameters_start(self, line): - if self.ignore_line(line): + def state_parameters_start(self, line: str) -> None: + if not self.valid_line(line): return # if this line is not indented, we have no parameters @@ -4742,7 +4753,7 @@ def state_parameter(self, line): line = self.parameter_continuation + ' ' + line.lstrip() self.parameter_continuation = '' - if self.ignore_line(line): + if not self.valid_line(line): return assert self.indent.depth == 2 @@ -5075,7 +5086,7 @@ def parse_special_symbol(self, symbol): fail("Function " + self.function.name + " mixes keyword-only and positional-only parameters, which is unsupported.") p.kind = inspect.Parameter.POSITIONAL_ONLY - def state_parameter_docstring_start(self, line): + def state_parameter_docstring_start(self, line: str) -> None: self.parameter_docstring_indent = len(self.indent.margin) assert self.indent.depth == 3 return self.next(self.state_parameter_docstring, line) From webhook-mailer at python.org Sun Jul 2 23:22:46 2023 From: webhook-mailer at python.org (JelleZijlstra) Date: Mon, 03 Jul 2023 03:22:46 -0000 Subject: [Python-checkins] Replace the esoteric term 'datum' when describing dict comprehensions (#106119) Message-ID: https://github.com/python/cpython/commit/987b712b4aeeece336eed24fcc87a950a756c3e2 commit: 987b712b4aeeece336eed24fcc87a950a756c3e2 branch: main author: Ned Batchelder committer: JelleZijlstra date: 2023-07-02T20:22:40-07:00 summary: Replace the esoteric term 'datum' when describing dict comprehensions (#106119) files: M Doc/reference/expressions.rst M Doc/reference/simple_stmts.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 0c700f908d687..ce1c9a59d5835 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -298,27 +298,27 @@ Dictionary displays .. index:: pair: dictionary; display pair: dictionary; comprehensions - key, datum, key/datum pair + key, value, key/value pair pair: object; dictionary single: {} (curly brackets); dictionary expression single: : (colon); in dictionary expressions single: , (comma); in dictionary displays -A dictionary display is a possibly empty series of key/datum pairs enclosed in -curly braces: +A dictionary display is a possibly empty series of dict items (key/value pairs) +enclosed in curly braces: .. productionlist:: python-grammar - dict_display: "{" [`key_datum_list` | `dict_comprehension`] "}" - key_datum_list: `key_datum` ("," `key_datum`)* [","] - key_datum: `expression` ":" `expression` | "**" `or_expr` + dict_display: "{" [`dict_item_list` | `dict_comprehension`] "}" + dict_item_list: `dict_item` ("," `dict_item`)* [","] + dict_item: `expression` ":" `expression` | "**" `or_expr` dict_comprehension: `expression` ":" `expression` `comp_for` A dictionary display yields a new dictionary object. -If a comma-separated sequence of key/datum pairs is given, they are evaluated +If a comma-separated sequence of dict items is given, they are evaluated from left to right to define the entries of the dictionary: each key object is -used as a key into the dictionary to store the corresponding datum. This means -that you can specify the same key multiple times in the key/datum list, and the +used as a key into the dictionary to store the corresponding value. This means +that you can specify the same key multiple times in the dict item list, and the final dictionary's value for that key will be the last one given. .. index:: @@ -328,7 +328,7 @@ final dictionary's value for that key will be the last one given. A double asterisk ``**`` denotes :dfn:`dictionary unpacking`. Its operand must be a :term:`mapping`. Each mapping item is added to the new dictionary. Later values replace values already set by -earlier key/datum pairs and earlier dictionary unpackings. +earlier dict items and earlier dictionary unpackings. .. versionadded:: 3.5 Unpacking into dictionary displays, originally proposed by :pep:`448`. @@ -344,7 +344,7 @@ in the new dictionary in the order they are produced. Restrictions on the types of the key values are listed earlier in section :ref:`types`. (To summarize, the key type should be :term:`hashable`, which excludes all mutable objects.) Clashes between duplicate keys are not detected; the last -datum (textually rightmost in the display) stored for a given key value +value (textually rightmost in the display) stored for a given key value prevails. .. versionchanged:: 3.8 diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 662a4b643c437..a9e65be1eda34 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -210,7 +210,7 @@ Assignment of an object to a single target is recursively defined as follows. If the primary is a mapping object (such as a dictionary), the subscript must have a type compatible with the mapping's key type, and the mapping is then - asked to create a key/datum pair which maps the subscript to the assigned + asked to create a key/value pair which maps the subscript to the assigned object. This can either replace an existing key/value pair with the same key value, or insert a new key/value pair (if no key with the same value existed). From webhook-mailer at python.org Sun Jul 2 23:23:31 2023 From: webhook-mailer at python.org (JelleZijlstra) Date: Mon, 03 Jul 2023 03:23:31 -0000 Subject: [Python-checkins] [3.12] Replace the esoteric term 'datum' when describing dict comprehensions (GH-106119) (#106348) Message-ID: https://github.com/python/cpython/commit/5e856049b19e2a0c78411b90a8fbc3198ca2a695 commit: 5e856049b19e2a0c78411b90a8fbc3198ca2a695 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: JelleZijlstra date: 2023-07-03T03:23:27Z summary: [3.12] Replace the esoteric term 'datum' when describing dict comprehensions (GH-106119) (#106348) Replace the esoteric term 'datum' when describing dict comprehensions (GH-106119) (cherry picked from commit 987b712b4aeeece336eed24fcc87a950a756c3e2) Co-authored-by: Ned Batchelder files: M Doc/reference/expressions.rst M Doc/reference/simple_stmts.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index b97a08f25d92a..08dcc8095bee8 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -298,27 +298,27 @@ Dictionary displays .. index:: pair: dictionary; display pair: dictionary; comprehensions - key, datum, key/datum pair + key, value, key/value pair pair: object; dictionary single: {} (curly brackets); dictionary expression single: : (colon); in dictionary expressions single: , (comma); in dictionary displays -A dictionary display is a possibly empty series of key/datum pairs enclosed in -curly braces: +A dictionary display is a possibly empty series of dict items (key/value pairs) +enclosed in curly braces: .. productionlist:: python-grammar - dict_display: "{" [`key_datum_list` | `dict_comprehension`] "}" - key_datum_list: `key_datum` ("," `key_datum`)* [","] - key_datum: `expression` ":" `expression` | "**" `or_expr` + dict_display: "{" [`dict_item_list` | `dict_comprehension`] "}" + dict_item_list: `dict_item` ("," `dict_item`)* [","] + dict_item: `expression` ":" `expression` | "**" `or_expr` dict_comprehension: `expression` ":" `expression` `comp_for` A dictionary display yields a new dictionary object. -If a comma-separated sequence of key/datum pairs is given, they are evaluated +If a comma-separated sequence of dict items is given, they are evaluated from left to right to define the entries of the dictionary: each key object is -used as a key into the dictionary to store the corresponding datum. This means -that you can specify the same key multiple times in the key/datum list, and the +used as a key into the dictionary to store the corresponding value. This means +that you can specify the same key multiple times in the dict item list, and the final dictionary's value for that key will be the last one given. .. index:: @@ -328,7 +328,7 @@ final dictionary's value for that key will be the last one given. A double asterisk ``**`` denotes :dfn:`dictionary unpacking`. Its operand must be a :term:`mapping`. Each mapping item is added to the new dictionary. Later values replace values already set by -earlier key/datum pairs and earlier dictionary unpackings. +earlier dict items and earlier dictionary unpackings. .. versionadded:: 3.5 Unpacking into dictionary displays, originally proposed by :pep:`448`. @@ -344,7 +344,7 @@ in the new dictionary in the order they are produced. Restrictions on the types of the key values are listed earlier in section :ref:`types`. (To summarize, the key type should be :term:`hashable`, which excludes all mutable objects.) Clashes between duplicate keys are not detected; the last -datum (textually rightmost in the display) stored for a given key value +value (textually rightmost in the display) stored for a given key value prevails. .. versionchanged:: 3.8 diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 662a4b643c437..a9e65be1eda34 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -210,7 +210,7 @@ Assignment of an object to a single target is recursively defined as follows. If the primary is a mapping object (such as a dictionary), the subscript must have a type compatible with the mapping's key type, and the mapping is then - asked to create a key/datum pair which maps the subscript to the assigned + asked to create a key/value pair which maps the subscript to the assigned object. This can either replace an existing key/value pair with the same key value, or insert a new key/value pair (if no key with the same value existed). From webhook-mailer at python.org Sun Jul 2 23:25:16 2023 From: webhook-mailer at python.org (JelleZijlstra) Date: Mon, 03 Jul 2023 03:25:16 -0000 Subject: [Python-checkins] [3.11] Replace the esoteric term 'datum' when describing dict comprehensions (GH-106119) (#106349) Message-ID: https://github.com/python/cpython/commit/e0d951d68aa360b30db0e85775814266e85272d2 commit: e0d951d68aa360b30db0e85775814266e85272d2 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: JelleZijlstra date: 2023-07-03T03:25:12Z summary: [3.11] Replace the esoteric term 'datum' when describing dict comprehensions (GH-106119) (#106349) Replace the esoteric term 'datum' when describing dict comprehensions (GH-106119) (cherry picked from commit 987b712b4aeeece336eed24fcc87a950a756c3e2) Co-authored-by: Ned Batchelder files: M Doc/reference/expressions.rst M Doc/reference/simple_stmts.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index ce9ff7f2208a8..2a16025ad6212 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -298,27 +298,27 @@ Dictionary displays .. index:: pair: dictionary; display pair: dictionary; comprehensions - key, datum, key/datum pair + key, value, key/value pair pair: object; dictionary single: {} (curly brackets); dictionary expression single: : (colon); in dictionary expressions single: , (comma); in dictionary displays -A dictionary display is a possibly empty series of key/datum pairs enclosed in -curly braces: +A dictionary display is a possibly empty series of dict items (key/value pairs) +enclosed in curly braces: .. productionlist:: python-grammar - dict_display: "{" [`key_datum_list` | `dict_comprehension`] "}" - key_datum_list: `key_datum` ("," `key_datum`)* [","] - key_datum: `expression` ":" `expression` | "**" `or_expr` + dict_display: "{" [`dict_item_list` | `dict_comprehension`] "}" + dict_item_list: `dict_item` ("," `dict_item`)* [","] + dict_item: `expression` ":" `expression` | "**" `or_expr` dict_comprehension: `expression` ":" `expression` `comp_for` A dictionary display yields a new dictionary object. -If a comma-separated sequence of key/datum pairs is given, they are evaluated +If a comma-separated sequence of dict items is given, they are evaluated from left to right to define the entries of the dictionary: each key object is -used as a key into the dictionary to store the corresponding datum. This means -that you can specify the same key multiple times in the key/datum list, and the +used as a key into the dictionary to store the corresponding value. This means +that you can specify the same key multiple times in the dict item list, and the final dictionary's value for that key will be the last one given. .. index:: @@ -328,7 +328,7 @@ final dictionary's value for that key will be the last one given. A double asterisk ``**`` denotes :dfn:`dictionary unpacking`. Its operand must be a :term:`mapping`. Each mapping item is added to the new dictionary. Later values replace values already set by -earlier key/datum pairs and earlier dictionary unpackings. +earlier dict items and earlier dictionary unpackings. .. versionadded:: 3.5 Unpacking into dictionary displays, originally proposed by :pep:`448`. @@ -344,7 +344,7 @@ in the new dictionary in the order they are produced. Restrictions on the types of the key values are listed earlier in section :ref:`types`. (To summarize, the key type should be :term:`hashable`, which excludes all mutable objects.) Clashes between duplicate keys are not detected; the last -datum (textually rightmost in the display) stored for a given key value +value (textually rightmost in the display) stored for a given key value prevails. .. versionchanged:: 3.8 diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index f7a8b44d19541..bf9a4e7d2a021 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -209,7 +209,7 @@ Assignment of an object to a single target is recursively defined as follows. If the primary is a mapping object (such as a dictionary), the subscript must have a type compatible with the mapping's key type, and the mapping is then - asked to create a key/datum pair which maps the subscript to the assigned + asked to create a key/value pair which maps the subscript to the assigned object. This can either replace an existing key/value pair with the same key value, or insert a new key/value pair (if no key with the same value existed). From webhook-mailer at python.org Mon Jul 3 01:44:41 2023 From: webhook-mailer at python.org (hauntsaninja) Date: Mon, 03 Jul 2023 05:44:41 -0000 Subject: [Python-checkins] Document PYTHONSAFEPATH along side -P (#106122) Message-ID: https://github.com/python/cpython/commit/0355625d94a50f4b816770bad946420d005900b8 commit: 0355625d94a50f4b816770bad946420d005900b8 branch: main author: Jeremy Paige committer: hauntsaninja <12621235+hauntsaninja at users.noreply.github.com> date: 2023-07-02T22:44:37-07:00 summary: Document PYTHONSAFEPATH along side -P (#106122) files: M Python/initconfig.c diff --git a/Python/initconfig.c b/Python/initconfig.c index 147cb370412cd..787e583262406 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -49,7 +49,7 @@ Options (and corresponding environment variables):\n\ .pyc extension; also PYTHONOPTIMIZE=x\n\ -OO : do -O changes and also discard docstrings; add .opt-2 before\n\ .pyc extension\n\ --P : don't prepend a potentially unsafe path to sys.path\n\ +-P : don't prepend a potentially unsafe path to sys.path; also PYTHONSAFEPATH\n\ -q : don't print version and copyright messages on interactive startup\n\ -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE\n\ -S : don't imply 'import site' on initialization\n\ @@ -144,7 +144,6 @@ static const char usage_envvars[] = "PYTHONSTARTUP: file executed on interactive startup (no default)\n" "PYTHONPATH : '%lc'-separated list of directories prefixed to the\n" " default module search path. The result is sys.path.\n" -"PYTHONSAFEPATH: don't prepend a potentially unsafe path to sys.path.\n" "PYTHONHOME : alternate directory (or %lc).\n" " The default module search path uses %s.\n" "PYTHONPLATLIBDIR : override sys.platlibdir.\n" @@ -184,6 +183,7 @@ static const char usage_envvars[] = " (-X int_max_str_digits=number)\n" "PYTHONNOUSERSITE : disable user site directory (-s)\n" "PYTHONOPTIMIZE : enable level 1 optimizations (-O)\n" +"PYTHONSAFEPATH : don't prepend a potentially unsafe path to sys.path (-P)\n" "PYTHONUNBUFFERED : disable stdout/stderr buffering (-u)\n" "PYTHONVERBOSE : trace import statements (-v)\n" "PYTHONWARNINGS=arg : warning control (-W arg)\n"; From webhook-mailer at python.org Mon Jul 3 02:56:57 2023 From: webhook-mailer at python.org (cjw296) Date: Mon, 03 Jul 2023 06:56:57 -0000 Subject: [Python-checkins] gh-61215: New mock to wait for multi-threaded events to happen (#16094) Message-ID: https://github.com/python/cpython/commit/d65b783b6966d233467a48ef633afb4aff9d5df8 commit: d65b783b6966d233467a48ef633afb4aff9d5df8 branch: main author: Mario Corchero committer: cjw296 date: 2023-07-03T07:56:54+01:00 summary: gh-61215: New mock to wait for multi-threaded events to happen (#16094) mock: Add `ThreadingMock` class Add a new class that allows to wait for a call to happen by using `Event` objects. This mock class can be used to test and validate expectations of multithreading code. It uses two attributes for events to distinguish calls with any argument and calls with specific arguments. The calls with specific arguments need a lock to prevent two calls in parallel from creating the same event twice. The timeout is configured at class and constructor level to allow users to set a timeout, we considered passing it as an argument to the function but it could collide with a function parameter. Alternatively we also considered passing it as positional only but from an API caller perspective it was unclear what the first number meant on the function call, think `mock.wait_until_called(1, "arg1", "arg2")`, where 1 is the timeout. Lastly we also considered adding the new attributes to magic mock directly rather than having a custom mock class for multi threading scenarios, but we preferred to have specialised class that can be composed if necessary. Additionally, having added it to `MagicMock` directly would have resulted in `AsyncMock` having this logic, which would not work as expected, since when if user "waits" on a coroutine does not have the same meaning as waiting on a standard call. Co-authored-by: Karthikeyan Singaravelan files: A Lib/test/test_unittest/testmock/testthreadingmock.py A Misc/NEWS.d/next/Library/2019-09-13-13-28-10.bpo-17013.NWcgE3.rst M Doc/library/unittest.mock.rst M Lib/unittest/mock.py diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 6c4d801f69f5a..756422bd6eab9 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -205,8 +205,10 @@ The Mock Class import asyncio import inspect import unittest + import threading from unittest.mock import sentinel, DEFAULT, ANY from unittest.mock import patch, call, Mock, MagicMock, PropertyMock, AsyncMock + from unittest.mock import ThreadingMock from unittest.mock import mock_open :class:`Mock` is a flexible mock object intended to replace the use of stubs and @@ -1099,6 +1101,51 @@ object:: [call('foo'), call('bar')] +.. class:: ThreadingMock(spec=None, side_effect=None, return_value=DEFAULT, wraps=None, name=None, spec_set=None, unsafe=False, *, timeout=UNSET, **kwargs) + + A version of :class:`MagicMock` for multithreading tests. The + :class:`ThreadingMock` object provides extra methods to wait for a call to + be invoked, rather than assert on it immediately. + + The default timeout is specified by the ``timeout`` argument, or if unset by the + :attr:`ThreadingMock.DEFAULT_TIMEOUT` attribute, which defaults to blocking (``None``). + + You can configure the global default timeout by setting :attr:`ThreadingMock.DEFAULT_TIMEOUT`. + + .. method:: wait_until_called(*, timeout=UNSET) + + Waits until the mock is called. + + If a timeout was passed at the creation of the mock or if a timeout + argument is passed to this function, the function raises an + :exc:`AssertionError` if the call is not performed in time. + + >>> mock = ThreadingMock() + >>> thread = threading.Thread(target=mock) + >>> thread.start() + >>> mock.wait_until_called(timeout=1) + >>> thread.join() + + .. method:: wait_until_any_call(*args, **kwargs) + + Waits until the the mock is called with the specified arguments. + + If a timeout was passed at the creation of the mock + the function raises an :exc:`AssertionError` if the call is not performed in time. + + >>> mock = ThreadingMock() + >>> thread = threading.Thread(target=mock, args=("arg1", "arg2",), kwargs={"arg": "thing"}) + >>> thread.start() + >>> mock.wait_until_any_call("arg1", "arg2", arg="thing") + >>> thread.join() + + .. attribute:: DEFAULT_TIMEOUT + + Global default timeout in seconds to create instances of :class:`ThreadingMock`. + + .. versionadded:: 3.13 + + Calling ~~~~~~~ diff --git a/Lib/test/test_unittest/testmock/testthreadingmock.py b/Lib/test/test_unittest/testmock/testthreadingmock.py new file mode 100644 index 0000000000000..6219cc58abdc4 --- /dev/null +++ b/Lib/test/test_unittest/testmock/testthreadingmock.py @@ -0,0 +1,278 @@ +import time +import unittest +import concurrent.futures + +from unittest.mock import patch, ThreadingMock, call + + +class Something: + def method_1(self): + pass + + def method_2(self): + pass + + +class TestThreadingMock(unittest.TestCase): + def _call_after_delay(self, func, /, *args, **kwargs): + time.sleep(kwargs.pop("delay")) + func(*args, **kwargs) + + def setUp(self): + self._executor = concurrent.futures.ThreadPoolExecutor(max_workers=5) + + def tearDown(self): + self._executor.shutdown() + + def run_async(self, func, /, *args, delay=0, **kwargs): + self._executor.submit( + self._call_after_delay, func, *args, **kwargs, delay=delay + ) + + def _make_mock(self, *args, **kwargs): + return ThreadingMock(*args, **kwargs) + + def test_spec(self): + waitable_mock = self._make_mock(spec=Something) + + with patch(f"{__name__}.Something", waitable_mock) as m: + something = m() + + self.assertIsInstance(something.method_1, ThreadingMock) + self.assertIsInstance(something.method_1().method_2(), ThreadingMock) + + with self.assertRaises(AttributeError): + m.test + + def test_side_effect(self): + waitable_mock = self._make_mock() + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + something.method_1.side_effect = [1] + + self.assertEqual(something.method_1(), 1) + + def test_instance_check(self): + waitable_mock = self._make_mock() + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + + self.assertIsInstance(something.method_1, ThreadingMock) + self.assertIsInstance(something.method_1().method_2(), ThreadingMock) + + def test_dynamic_child_mocks_are_threading_mocks(self): + waitable_mock = self._make_mock() + self.assertIsInstance(waitable_mock.child, ThreadingMock) + + def test_dynamic_child_mocks_inherit_timeout(self): + mock1 = self._make_mock() + self.assertIs(mock1._mock_wait_timeout, None) + mock2 = self._make_mock(timeout=2) + self.assertEqual(mock2._mock_wait_timeout, 2) + mock3 = self._make_mock(timeout=3) + self.assertEqual(mock3._mock_wait_timeout, 3) + + self.assertIs(mock1.child._mock_wait_timeout, None) + self.assertEqual(mock2.child._mock_wait_timeout, 2) + self.assertEqual(mock3.child._mock_wait_timeout, 3) + + self.assertEqual(mock2.really().__mul__().complex._mock_wait_timeout, 2) + + def test_no_name_clash(self): + waitable_mock = self._make_mock() + waitable_mock._event = "myevent" + waitable_mock.event = "myevent" + waitable_mock.timeout = "mytimeout" + waitable_mock("works") + waitable_mock.wait_until_called() + waitable_mock.wait_until_any_call("works") + + def test_wait_success(self): + waitable_mock = self._make_mock(spec=Something) + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + self.run_async(something.method_1, delay=0.01) + something.method_1.wait_until_called() + something.method_1.wait_until_any_call() + something.method_1.assert_called() + + def test_wait_success_with_instance_timeout(self): + waitable_mock = self._make_mock(timeout=1) + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + self.run_async(something.method_1, delay=0.01) + something.method_1.wait_until_called() + something.method_1.wait_until_any_call() + something.method_1.assert_called() + + def test_wait_failed_with_instance_timeout(self): + waitable_mock = self._make_mock(timeout=0.01) + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + self.run_async(something.method_1, delay=0.5) + self.assertRaises(AssertionError, waitable_mock.method_1.wait_until_called) + self.assertRaises( + AssertionError, waitable_mock.method_1.wait_until_any_call + ) + + def test_wait_success_with_timeout_override(self): + waitable_mock = self._make_mock(timeout=0.01) + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + self.run_async(something.method_1, delay=0.05) + something.method_1.wait_until_called(timeout=1) + + def test_wait_failed_with_timeout_override(self): + waitable_mock = self._make_mock(timeout=1) + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + self.run_async(something.method_1, delay=0.1) + with self.assertRaises(AssertionError): + something.method_1.wait_until_called(timeout=0.05) + with self.assertRaises(AssertionError): + something.method_1.wait_until_any_call(timeout=0.05) + + def test_wait_success_called_before(self): + waitable_mock = self._make_mock() + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + something.method_1() + something.method_1.wait_until_called() + something.method_1.wait_until_any_call() + something.method_1.assert_called() + + def test_wait_magic_method(self): + waitable_mock = self._make_mock() + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + self.run_async(something.method_1.__str__, delay=0.01) + something.method_1.__str__.wait_until_called() + something.method_1.__str__.assert_called() + + def test_wait_until_any_call_positional(self): + waitable_mock = self._make_mock(spec=Something) + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + self.run_async(something.method_1, 1, delay=0.1) + self.run_async(something.method_1, 2, delay=0.2) + self.run_async(something.method_1, 3, delay=0.3) + self.assertNotIn(call(1), something.method_1.mock_calls) + + something.method_1.wait_until_any_call(1) + something.method_1.assert_called_with(1) + self.assertNotIn(call(2), something.method_1.mock_calls) + self.assertNotIn(call(3), something.method_1.mock_calls) + + something.method_1.wait_until_any_call(3) + self.assertIn(call(2), something.method_1.mock_calls) + something.method_1.wait_until_any_call(2) + + def test_wait_until_any_call_keywords(self): + waitable_mock = self._make_mock(spec=Something) + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + self.run_async(something.method_1, a=1, delay=0.1) + self.run_async(something.method_1, b=2, delay=0.2) + self.run_async(something.method_1, c=3, delay=0.3) + self.assertNotIn(call(a=1), something.method_1.mock_calls) + + something.method_1.wait_until_any_call(a=1) + something.method_1.assert_called_with(a=1) + self.assertNotIn(call(b=2), something.method_1.mock_calls) + self.assertNotIn(call(c=3), something.method_1.mock_calls) + + something.method_1.wait_until_any_call(c=3) + self.assertIn(call(b=2), something.method_1.mock_calls) + something.method_1.wait_until_any_call(b=2) + + def test_wait_until_any_call_no_argument_fails_when_called_with_arg(self): + waitable_mock = self._make_mock(timeout=0.01) + + with patch(f"{__name__}.Something", waitable_mock): + something = Something() + something.method_1(1) + + something.method_1.assert_called_with(1) + with self.assertRaises(AssertionError): + something.method_1.wait_until_any_call() + + something.method_1() + something.method_1.wait_until_any_call() + + def test_wait_until_any_call_global_default(self): + with patch.object(ThreadingMock, "DEFAULT_TIMEOUT"): + ThreadingMock.DEFAULT_TIMEOUT = 0.01 + m = self._make_mock() + with self.assertRaises(AssertionError): + m.wait_until_any_call() + with self.assertRaises(AssertionError): + m.wait_until_called() + + m() + m.wait_until_any_call() + assert ThreadingMock.DEFAULT_TIMEOUT != 0.01 + + def test_wait_until_any_call_change_global_and_override(self): + with patch.object(ThreadingMock, "DEFAULT_TIMEOUT"): + ThreadingMock.DEFAULT_TIMEOUT = 0.01 + + m1 = self._make_mock() + self.run_async(m1, delay=0.1) + with self.assertRaises(AssertionError): + m1.wait_until_called() + + m2 = self._make_mock(timeout=1) + self.run_async(m2, delay=0.1) + m2.wait_until_called() + + m3 = self._make_mock() + self.run_async(m3, delay=0.1) + m3.wait_until_called(timeout=1) + + m4 = self._make_mock() + self.run_async(m4, delay=0.1) + m4.wait_until_called(timeout=None) + + m5 = self._make_mock(timeout=None) + self.run_async(m5, delay=0.1) + m5.wait_until_called() + + assert ThreadingMock.DEFAULT_TIMEOUT != 0.01 + + def test_reset_mock_resets_wait(self): + m = self._make_mock(timeout=0.01) + + with self.assertRaises(AssertionError): + m.wait_until_called() + with self.assertRaises(AssertionError): + m.wait_until_any_call() + m() + m.wait_until_called() + m.wait_until_any_call() + m.assert_called_once() + + m.reset_mock() + + with self.assertRaises(AssertionError): + m.wait_until_called() + with self.assertRaises(AssertionError): + m.wait_until_any_call() + m() + m.wait_until_called() + m.wait_until_any_call() + m.assert_called_once() + + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py index f10f9b04a5ab1..b542a9a2c51a9 100644 --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -14,6 +14,7 @@ 'call', 'create_autospec', 'AsyncMock', + 'ThreadingMock', 'FILTER_DIR', 'NonCallableMock', 'NonCallableMagicMock', @@ -32,6 +33,7 @@ import builtins import pkgutil from asyncio import iscoroutinefunction +import threading from types import CodeType, ModuleType, MethodType from unittest.util import safe_repr from functools import wraps, partial @@ -3003,6 +3005,98 @@ def __set__(self, obj, val): self(val) +_timeout_unset = sentinel.TIMEOUT_UNSET + +class ThreadingMixin(Base): + + DEFAULT_TIMEOUT = None + + def _get_child_mock(self, /, **kw): + if "timeout" in kw: + kw["timeout"] = kw.pop("timeout") + elif isinstance(kw.get("parent"), ThreadingMixin): + kw["timeout"] = kw["parent"]._mock_wait_timeout + elif isinstance(kw.get("_new_parent"), ThreadingMixin): + kw["timeout"] = kw["_new_parent"]._mock_wait_timeout + return super()._get_child_mock(**kw) + + def __init__(self, *args, timeout=_timeout_unset, **kwargs): + super().__init__(*args, **kwargs) + if timeout is _timeout_unset: + timeout = self.DEFAULT_TIMEOUT + self.__dict__["_mock_event"] = threading.Event() # Event for any call + self.__dict__["_mock_calls_events"] = [] # Events for each of the calls + self.__dict__["_mock_calls_events_lock"] = threading.Lock() + self.__dict__["_mock_wait_timeout"] = timeout + + def reset_mock(self, /, *args, **kwargs): + """ + See :func:`.Mock.reset_mock()` + """ + super().reset_mock(*args, **kwargs) + self.__dict__["_mock_event"] = threading.Event() + self.__dict__["_mock_calls_events"] = [] + + def __get_event(self, expected_args, expected_kwargs): + with self._mock_calls_events_lock: + for args, kwargs, event in self._mock_calls_events: + if (args, kwargs) == (expected_args, expected_kwargs): + return event + new_event = threading.Event() + self._mock_calls_events.append((expected_args, expected_kwargs, new_event)) + return new_event + + def _mock_call(self, *args, **kwargs): + ret_value = super()._mock_call(*args, **kwargs) + + call_event = self.__get_event(args, kwargs) + call_event.set() + + self._mock_event.set() + + return ret_value + + def wait_until_called(self, *, timeout=_timeout_unset): + """Wait until the mock object is called. + + `timeout` - time to wait for in seconds, waits forever otherwise. + Defaults to the constructor provided timeout. + Use None to block undefinetively. + """ + if timeout is _timeout_unset: + timeout = self._mock_wait_timeout + if not self._mock_event.wait(timeout=timeout): + msg = (f"{self._mock_name or 'mock'} was not called before" + f" timeout({timeout}).") + raise AssertionError(msg) + + def wait_until_any_call(self, *args, **kwargs): + """Wait until the mock object is called with given args. + + Waits for the timeout in seconds provided in the constructor. + """ + event = self.__get_event(args, kwargs) + if not event.wait(timeout=self._mock_wait_timeout): + expected_string = self._format_mock_call_signature(args, kwargs) + raise AssertionError(f'{expected_string} call not found') + + +class ThreadingMock(ThreadingMixin, MagicMixin, Mock): + """ + A mock that can be used to wait until on calls happening + in a different thread. + + The constructor can take a `timeout` argument which + controls the timeout in seconds for all `wait` calls of the mock. + + You can change the default timeout of all instances via the + `ThreadingMock.DEFAULT_TIMEOUT` attribute. + + If no timeout is set, it will block undefinetively. + """ + pass + + def seal(mock): """Disable the automatic generation of child mocks. diff --git a/Misc/NEWS.d/next/Library/2019-09-13-13-28-10.bpo-17013.NWcgE3.rst b/Misc/NEWS.d/next/Library/2019-09-13-13-28-10.bpo-17013.NWcgE3.rst new file mode 100644 index 0000000000000..ac746c45fa9ea --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-09-13-13-28-10.bpo-17013.NWcgE3.rst @@ -0,0 +1,3 @@ +Add ``ThreadingMock`` to :mod:`unittest.mock` that can be used to create +Mock objects that can wait until they are called. Patch by Karthikeyan +Singaravelan and Mario Corchero. From webhook-mailer at python.org Mon Jul 3 04:23:47 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 03 Jul 2023 08:23:47 -0000 Subject: [Python-checkins] gh-106320: Move _PyUnicodeWriter to the internal C API (#106342) Message-ID: https://github.com/python/cpython/commit/5ccbbe5bb9a659fa8f2fe551428c84cc14015f44 commit: 5ccbbe5bb9a659fa8f2fe551428c84cc14015f44 branch: main author: Victor Stinner committer: vstinner date: 2023-07-03T10:23:43+02:00 summary: gh-106320: Move _PyUnicodeWriter to the internal C API (#106342) Move also _PyUnicode_FormatAdvancedWriter(). CJK codecs and multibytecodec.c now define the Py_BUILD_CORE_MODULE macro. files: M Include/cpython/unicodeobject.h M Include/internal/pycore_complexobject.h M Include/internal/pycore_floatobject.h M Include/internal/pycore_unicodeobject.h M Modules/cjkcodecs/cjkcodecs.h M Modules/cjkcodecs/multibytecodec.c M Modules/cjkcodecs/multibytecodec.h M Tools/c-analyzer/c_parser/preprocessor/gcc.py diff --git a/Include/cpython/unicodeobject.h b/Include/cpython/unicodeobject.h index dee8b27d3d97d..c5892a80d2c54 100644 --- a/Include/cpython/unicodeobject.h +++ b/Include/cpython/unicodeobject.h @@ -480,131 +480,6 @@ PyAPI_FUNC(Py_UCS4) _PyUnicode_FindMaxChar ( Py_ssize_t start, Py_ssize_t end); -/* --- _PyUnicodeWriter API ----------------------------------------------- */ - -typedef struct { - PyObject *buffer; - void *data; - int kind; - Py_UCS4 maxchar; - Py_ssize_t size; - Py_ssize_t pos; - - /* minimum number of allocated characters (default: 0) */ - Py_ssize_t min_length; - - /* minimum character (default: 127, ASCII) */ - Py_UCS4 min_char; - - /* If non-zero, overallocate the buffer (default: 0). */ - unsigned char overallocate; - - /* If readonly is 1, buffer is a shared string (cannot be modified) - and size is set to 0. */ - unsigned char readonly; -} _PyUnicodeWriter ; - -/* Initialize a Unicode writer. - * - * By default, the minimum buffer size is 0 character and overallocation is - * disabled. Set min_length, min_char and overallocate attributes to control - * the allocation of the buffer. */ -PyAPI_FUNC(void) -_PyUnicodeWriter_Init(_PyUnicodeWriter *writer); - -/* Prepare the buffer to write 'length' characters - with the specified maximum character. - - Return 0 on success, raise an exception and return -1 on error. */ -#define _PyUnicodeWriter_Prepare(WRITER, LENGTH, MAXCHAR) \ - (((MAXCHAR) <= (WRITER)->maxchar \ - && (LENGTH) <= (WRITER)->size - (WRITER)->pos) \ - ? 0 \ - : (((LENGTH) == 0) \ - ? 0 \ - : _PyUnicodeWriter_PrepareInternal((WRITER), (LENGTH), (MAXCHAR)))) - -/* Don't call this function directly, use the _PyUnicodeWriter_Prepare() macro - instead. */ -PyAPI_FUNC(int) -_PyUnicodeWriter_PrepareInternal(_PyUnicodeWriter *writer, - Py_ssize_t length, Py_UCS4 maxchar); - -/* Prepare the buffer to have at least the kind KIND. - For example, kind=PyUnicode_2BYTE_KIND ensures that the writer will - support characters in range U+000-U+FFFF. - - Return 0 on success, raise an exception and return -1 on error. */ -#define _PyUnicodeWriter_PrepareKind(WRITER, KIND) \ - ((KIND) <= (WRITER)->kind \ - ? 0 \ - : _PyUnicodeWriter_PrepareKindInternal((WRITER), (KIND))) - -/* Don't call this function directly, use the _PyUnicodeWriter_PrepareKind() - macro instead. */ -PyAPI_FUNC(int) -_PyUnicodeWriter_PrepareKindInternal(_PyUnicodeWriter *writer, - int kind); - -/* Append a Unicode character. - Return 0 on success, raise an exception and return -1 on error. */ -PyAPI_FUNC(int) -_PyUnicodeWriter_WriteChar(_PyUnicodeWriter *writer, - Py_UCS4 ch - ); - -/* Append a Unicode string. - Return 0 on success, raise an exception and return -1 on error. */ -PyAPI_FUNC(int) -_PyUnicodeWriter_WriteStr(_PyUnicodeWriter *writer, - PyObject *str /* Unicode string */ - ); - -/* Append a substring of a Unicode string. - Return 0 on success, raise an exception and return -1 on error. */ -PyAPI_FUNC(int) -_PyUnicodeWriter_WriteSubstring(_PyUnicodeWriter *writer, - PyObject *str, /* Unicode string */ - Py_ssize_t start, - Py_ssize_t end - ); - -/* Append an ASCII-encoded byte string. - Return 0 on success, raise an exception and return -1 on error. */ -PyAPI_FUNC(int) -_PyUnicodeWriter_WriteASCIIString(_PyUnicodeWriter *writer, - const char *str, /* ASCII-encoded byte string */ - Py_ssize_t len /* number of bytes, or -1 if unknown */ - ); - -/* Append a latin1-encoded byte string. - Return 0 on success, raise an exception and return -1 on error. */ -PyAPI_FUNC(int) -_PyUnicodeWriter_WriteLatin1String(_PyUnicodeWriter *writer, - const char *str, /* latin1-encoded byte string */ - Py_ssize_t len /* length in bytes */ - ); - -/* Get the value of the writer as a Unicode string. Clear the - buffer of the writer. Raise an exception and return NULL - on error. */ -PyAPI_FUNC(PyObject *) -_PyUnicodeWriter_Finish(_PyUnicodeWriter *writer); - -/* Deallocate memory of a writer (clear its internal buffer). */ -PyAPI_FUNC(void) -_PyUnicodeWriter_Dealloc(_PyUnicodeWriter *writer); - - -/* Format the object based on the format_spec, as defined in PEP 3101 - (Advanced String Formatting). */ -PyAPI_FUNC(int) _PyUnicode_FormatAdvancedWriter( - _PyUnicodeWriter *writer, - PyObject *obj, - PyObject *format_spec, - Py_ssize_t start, - Py_ssize_t end); - /* --- Manage the default encoding ---------------------------------------- */ /* Returns a pointer to the default encoding (UTF-8) of the @@ -774,20 +649,6 @@ PyAPI_FUNC(PyObject *) _PyUnicode_XStrip( PyObject *sepobj ); -/* Using explicit passed-in values, insert the thousands grouping - into the string pointed to by buffer. For the argument descriptions, - see Objects/stringlib/localeutil.h */ -PyAPI_FUNC(Py_ssize_t) _PyUnicode_InsertThousandsGrouping( - _PyUnicodeWriter *writer, - Py_ssize_t n_buffer, - PyObject *digits, - Py_ssize_t d_pos, - Py_ssize_t n_digits, - Py_ssize_t min_width, - const char *grouping, - PyObject *thousands_sep, - Py_UCS4 *maxchar); - /* === Characters Type APIs =============================================== */ /* These should not be used directly. Use the Py_UNICODE_IS* and diff --git a/Include/internal/pycore_complexobject.h b/Include/internal/pycore_complexobject.h index fb344b7bb79bd..7843c0a008ebb 100644 --- a/Include/internal/pycore_complexobject.h +++ b/Include/internal/pycore_complexobject.h @@ -8,6 +8,8 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +#include "pycore_unicodeobject.h" // _PyUnicodeWriter + /* Operations on complex numbers from complexmodule.c */ PyAPI_FUNC(Py_complex) _Py_c_sum(Py_complex, Py_complex); diff --git a/Include/internal/pycore_floatobject.h b/Include/internal/pycore_floatobject.h index 27c63bc87f3ee..6abba04033d28 100644 --- a/Include/internal/pycore_floatobject.h +++ b/Include/internal/pycore_floatobject.h @@ -9,6 +9,8 @@ extern "C" { #endif +#include "pycore_unicodeobject.h" // _PyUnicodeWriter + /* runtime lifecycle */ extern void _PyFloat_InitState(PyInterpreterState *); diff --git a/Include/internal/pycore_unicodeobject.h b/Include/internal/pycore_unicodeobject.h index 1bb0f366e7816..a8c7f1957f360 100644 --- a/Include/internal/pycore_unicodeobject.h +++ b/Include/internal/pycore_unicodeobject.h @@ -14,7 +14,148 @@ extern "C" { void _PyUnicode_ExactDealloc(PyObject *op); Py_ssize_t _PyUnicode_InternedSize(void); -/* runtime lifecycle */ +/* --- _PyUnicodeWriter API ----------------------------------------------- */ + +typedef struct { + PyObject *buffer; + void *data; + int kind; + Py_UCS4 maxchar; + Py_ssize_t size; + Py_ssize_t pos; + + /* minimum number of allocated characters (default: 0) */ + Py_ssize_t min_length; + + /* minimum character (default: 127, ASCII) */ + Py_UCS4 min_char; + + /* If non-zero, overallocate the buffer (default: 0). */ + unsigned char overallocate; + + /* If readonly is 1, buffer is a shared string (cannot be modified) + and size is set to 0. */ + unsigned char readonly; +} _PyUnicodeWriter ; + +/* Initialize a Unicode writer. + * + * By default, the minimum buffer size is 0 character and overallocation is + * disabled. Set min_length, min_char and overallocate attributes to control + * the allocation of the buffer. */ +PyAPI_FUNC(void) +_PyUnicodeWriter_Init(_PyUnicodeWriter *writer); + +/* Prepare the buffer to write 'length' characters + with the specified maximum character. + + Return 0 on success, raise an exception and return -1 on error. */ +#define _PyUnicodeWriter_Prepare(WRITER, LENGTH, MAXCHAR) \ + (((MAXCHAR) <= (WRITER)->maxchar \ + && (LENGTH) <= (WRITER)->size - (WRITER)->pos) \ + ? 0 \ + : (((LENGTH) == 0) \ + ? 0 \ + : _PyUnicodeWriter_PrepareInternal((WRITER), (LENGTH), (MAXCHAR)))) + +/* Don't call this function directly, use the _PyUnicodeWriter_Prepare() macro + instead. */ +PyAPI_FUNC(int) +_PyUnicodeWriter_PrepareInternal(_PyUnicodeWriter *writer, + Py_ssize_t length, Py_UCS4 maxchar); + +/* Prepare the buffer to have at least the kind KIND. + For example, kind=PyUnicode_2BYTE_KIND ensures that the writer will + support characters in range U+000-U+FFFF. + + Return 0 on success, raise an exception and return -1 on error. */ +#define _PyUnicodeWriter_PrepareKind(WRITER, KIND) \ + ((KIND) <= (WRITER)->kind \ + ? 0 \ + : _PyUnicodeWriter_PrepareKindInternal((WRITER), (KIND))) + +/* Don't call this function directly, use the _PyUnicodeWriter_PrepareKind() + macro instead. */ +PyAPI_FUNC(int) +_PyUnicodeWriter_PrepareKindInternal(_PyUnicodeWriter *writer, + int kind); + +/* Append a Unicode character. + Return 0 on success, raise an exception and return -1 on error. */ +PyAPI_FUNC(int) +_PyUnicodeWriter_WriteChar(_PyUnicodeWriter *writer, + Py_UCS4 ch + ); + +/* Append a Unicode string. + Return 0 on success, raise an exception and return -1 on error. */ +PyAPI_FUNC(int) +_PyUnicodeWriter_WriteStr(_PyUnicodeWriter *writer, + PyObject *str /* Unicode string */ + ); + +/* Append a substring of a Unicode string. + Return 0 on success, raise an exception and return -1 on error. */ +PyAPI_FUNC(int) +_PyUnicodeWriter_WriteSubstring(_PyUnicodeWriter *writer, + PyObject *str, /* Unicode string */ + Py_ssize_t start, + Py_ssize_t end + ); + +/* Append an ASCII-encoded byte string. + Return 0 on success, raise an exception and return -1 on error. */ +PyAPI_FUNC(int) +_PyUnicodeWriter_WriteASCIIString(_PyUnicodeWriter *writer, + const char *str, /* ASCII-encoded byte string */ + Py_ssize_t len /* number of bytes, or -1 if unknown */ + ); + +/* Append a latin1-encoded byte string. + Return 0 on success, raise an exception and return -1 on error. */ +PyAPI_FUNC(int) +_PyUnicodeWriter_WriteLatin1String(_PyUnicodeWriter *writer, + const char *str, /* latin1-encoded byte string */ + Py_ssize_t len /* length in bytes */ + ); + +/* Get the value of the writer as a Unicode string. Clear the + buffer of the writer. Raise an exception and return NULL + on error. */ +PyAPI_FUNC(PyObject *) +_PyUnicodeWriter_Finish(_PyUnicodeWriter *writer); + +/* Deallocate memory of a writer (clear its internal buffer). */ +PyAPI_FUNC(void) +_PyUnicodeWriter_Dealloc(_PyUnicodeWriter *writer); + + +/* Format the object based on the format_spec, as defined in PEP 3101 + (Advanced String Formatting). */ +PyAPI_FUNC(int) _PyUnicode_FormatAdvancedWriter( + _PyUnicodeWriter *writer, + PyObject *obj, + PyObject *format_spec, + Py_ssize_t start, + Py_ssize_t end); + +/* --- Methods & Slots ---------------------------------------------------- */ + +/* Using explicit passed-in values, insert the thousands grouping + into the string pointed to by buffer. For the argument descriptions, + see Objects/stringlib/localeutil.h */ +PyAPI_FUNC(Py_ssize_t) _PyUnicode_InsertThousandsGrouping( + _PyUnicodeWriter *writer, + Py_ssize_t n_buffer, + PyObject *digits, + Py_ssize_t d_pos, + Py_ssize_t n_digits, + Py_ssize_t min_width, + const char *grouping, + PyObject *thousands_sep, + Py_UCS4 *maxchar); + +/* --- Runtime lifecycle -------------------------------------------------- */ extern void _PyUnicode_InitState(PyInterpreterState *); extern PyStatus _PyUnicode_InitGlobalObjects(PyInterpreterState *); @@ -24,7 +165,7 @@ extern void _PyUnicode_FiniTypes(PyInterpreterState *); extern PyTypeObject _PyUnicodeASCIIIter_Type; -/* other API */ +/* --- Other API ---------------------------------------------------------- */ struct _Py_unicode_runtime_ids { PyThread_type_lock lock; diff --git a/Modules/cjkcodecs/cjkcodecs.h b/Modules/cjkcodecs/cjkcodecs.h index 48cdcfb3ad308..97290aac3ba43 100644 --- a/Modules/cjkcodecs/cjkcodecs.h +++ b/Modules/cjkcodecs/cjkcodecs.h @@ -7,6 +7,10 @@ #ifndef _CJKCODECS_H_ #define _CJKCODECS_H_ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + #include "Python.h" #include "multibytecodec.h" diff --git a/Modules/cjkcodecs/multibytecodec.c b/Modules/cjkcodecs/multibytecodec.c index cf437d09f1fe8..3febd1a832f9c 100644 --- a/Modules/cjkcodecs/multibytecodec.c +++ b/Modules/cjkcodecs/multibytecodec.c @@ -4,6 +4,10 @@ * Written by Hye-Shik Chang */ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + #include "Python.h" #include "structmember.h" // PyMemberDef #include "multibytecodec.h" diff --git a/Modules/cjkcodecs/multibytecodec.h b/Modules/cjkcodecs/multibytecodec.h index f59362205d26f..5b85e2e3de316 100644 --- a/Modules/cjkcodecs/multibytecodec.h +++ b/Modules/cjkcodecs/multibytecodec.h @@ -10,6 +10,8 @@ extern "C" { #endif +#include "pycore_unicodeobject.h" // _PyUnicodeWriter + #ifdef uint16_t typedef uint16_t ucs2_t, DBCHAR; #else diff --git a/Tools/c-analyzer/c_parser/preprocessor/gcc.py b/Tools/c-analyzer/c_parser/preprocessor/gcc.py index 0929f7111eac9..415a2ba63dfe6 100644 --- a/Tools/c-analyzer/c_parser/preprocessor/gcc.py +++ b/Tools/c-analyzer/c_parser/preprocessor/gcc.py @@ -3,6 +3,14 @@ from . import common as _common +# Modules/socketmodule.h uses pycore_time.h which needs the Py_BUILD_CORE +# macro. Usually it's defined by the C file which includes it. +# Other header files have a similar issue. +NEED_BUILD_CORE = { + 'cjkcodecs.h', + 'multibytecodec.h', + 'socketmodule.h', +} TOOL = 'gcc' @@ -62,9 +70,7 @@ def preprocess(filename, filename = _normpath(filename, cwd) postargs = POST_ARGS - if os.path.basename(filename) == 'socketmodule.h': - # Modules/socketmodule.h uses pycore_time.h which needs Py_BUILD_CORE. - # Usually it's defined by the C file which includes it. + if os.path.basename(filename) in NEED_BUILD_CORE: postargs += ('-DPy_BUILD_CORE=1',) text = _common.preprocess( From webhook-mailer at python.org Mon Jul 3 05:39:14 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 03 Jul 2023 09:39:14 -0000 Subject: [Python-checkins] gh-106320: Create pycore_modsupport.h header file (#106355) Message-ID: https://github.com/python/cpython/commit/35963da40fb7bc93c55d94caf58ff9e268df951d commit: 35963da40fb7bc93c55d94caf58ff9e268df951d branch: main author: Victor Stinner committer: vstinner date: 2023-07-03T09:39:11Z summary: gh-106320: Create pycore_modsupport.h header file (#106355) Remove the following functions from the C API, move them to the internal C API: add a new pycore_modsupport.h internal header file: * PyModule_CreateInitialized() * _PyArg_NoKwnames() * _Py_VaBuildStack() No longer export these functions. files: A Include/internal/pycore_modsupport.h M Include/cpython/modsupport.h M Makefile.pre.in M Modules/_operator.c M Objects/boolobject.c M Objects/call.c M Objects/enumobject.c M Objects/floatobject.c M Objects/listobject.c M Objects/moduleobject.c M Objects/rangeobject.c M Objects/setobject.c M Objects/tupleobject.c M Objects/typeobject.c M Objects/weakrefobject.c M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters M Python/bltinmodule.c M Python/instrumentation.c M Python/sysmodule.c diff --git a/Include/cpython/modsupport.h b/Include/cpython/modsupport.h index a5d95d15440df..376336b13dcf8 100644 --- a/Include/cpython/modsupport.h +++ b/Include/cpython/modsupport.h @@ -11,12 +11,9 @@ PyAPI_FUNC(int) _PyArg_UnpackStack( ...); PyAPI_FUNC(int) _PyArg_NoKeywords(const char *funcname, PyObject *kwargs); -PyAPI_FUNC(int) _PyArg_NoKwnames(const char *funcname, PyObject *kwnames); PyAPI_FUNC(int) _PyArg_NoPositional(const char *funcname, PyObject *args); #define _PyArg_NoKeywords(funcname, kwargs) \ ((kwargs) == NULL || _PyArg_NoKeywords((funcname), (kwargs))) -#define _PyArg_NoKwnames(funcname, kwnames) \ - ((kwnames) == NULL || _PyArg_NoKwnames((funcname), (kwnames))) #define _PyArg_NoPositional(funcname, args) \ ((args) == NULL || _PyArg_NoPositional((funcname), (args))) @@ -29,13 +26,6 @@ PyAPI_FUNC(int) _PyArg_CheckPositional(const char *, Py_ssize_t, ((!_Py_ANY_VARARGS(max) && (min) <= (nargs) && (nargs) <= (max)) \ || _PyArg_CheckPositional((funcname), (nargs), (min), (max))) -PyAPI_FUNC(PyObject **) _Py_VaBuildStack( - PyObject **small_stack, - Py_ssize_t small_stack_len, - const char *format, - va_list va, - Py_ssize_t *p_nargs); - typedef struct _PyArg_Parser { int initialized; const char *format; @@ -83,5 +73,3 @@ PyAPI_FUNC(PyObject * const *) _PyArg_UnpackKeywordsWithVararg( (minpos) <= (nargs) && (nargs) <= (maxpos) && (args) != NULL) ? (args) : \ _PyArg_UnpackKeywords((args), (nargs), (kwargs), (kwnames), (parser), \ (minpos), (maxpos), (minkw), (buf))) - -PyAPI_FUNC(PyObject *) _PyModule_CreateInitialized(PyModuleDef*, int apiver); diff --git a/Include/internal/pycore_modsupport.h b/Include/internal/pycore_modsupport.h new file mode 100644 index 0000000000000..e577c6ba856b7 --- /dev/null +++ b/Include/internal/pycore_modsupport.h @@ -0,0 +1,29 @@ +#ifndef Py_INTERNAL_MODSUPPORT_H +#define Py_INTERNAL_MODSUPPORT_H +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef Py_BUILD_CORE +# error "this header requires Py_BUILD_CORE define" +#endif + + +extern int _PyArg_NoKwnames(const char *funcname, PyObject *kwnames); +#define _PyArg_NoKwnames(funcname, kwnames) \ + ((kwnames) == NULL || _PyArg_NoKwnames((funcname), (kwnames))) + +extern PyObject ** _Py_VaBuildStack( + PyObject **small_stack, + Py_ssize_t small_stack_len, + const char *format, + va_list va, + Py_ssize_t *p_nargs); + +extern PyObject* _PyModule_CreateInitialized(PyModuleDef*, int apiver); + +#ifdef __cplusplus +} +#endif +#endif // !Py_INTERNAL_MODSUPPORT_H + diff --git a/Makefile.pre.in b/Makefile.pre.in index 7560d17e3796f..dcb3b5eb06a1e 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1761,6 +1761,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/internal/pycore_intrinsics.h \ $(srcdir)/Include/internal/pycore_list.h \ $(srcdir)/Include/internal/pycore_long.h \ + $(srcdir)/Include/internal/pycore_modsupport.h \ $(srcdir)/Include/internal/pycore_moduleobject.h \ $(srcdir)/Include/internal/pycore_namespace.h \ $(srcdir)/Include/internal/pycore_object.h \ diff --git a/Modules/_operator.c b/Modules/_operator.c index 153e9e9e2f92c..108f45fb6dad9 100644 --- a/Modules/_operator.c +++ b/Modules/_operator.c @@ -1,7 +1,9 @@ #include "Python.h" +#include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_moduleobject.h" // _PyModule_GetState() -#include "structmember.h" // PyMemberDef #include "pycore_runtime.h" // _Py_ID() + +#include "structmember.h" // PyMemberDef #include "clinic/_operator.c.h" typedef struct { diff --git a/Objects/boolobject.c b/Objects/boolobject.c index f43e26f3f24e7..bbb187cb7121e 100644 --- a/Objects/boolobject.c +++ b/Objects/boolobject.c @@ -1,8 +1,9 @@ /* Boolean type, a subtype of int */ #include "Python.h" -#include "pycore_object.h" // _Py_FatalRefcountError() -#include "pycore_long.h" // FALSE_TAG TRUE_TAG +#include "pycore_long.h" // FALSE_TAG TRUE_TAG +#include "pycore_modsupport.h" // _PyArg_NoKwnames() +#include "pycore_object.h" // _Py_FatalRefcountError() #include "pycore_runtime.h" // _Py_ID() #include diff --git a/Objects/call.c b/Objects/call.c index 16c41ffe1d09b..5045c0dc92843 100644 --- a/Objects/call.c +++ b/Objects/call.c @@ -2,6 +2,7 @@ #include "pycore_call.h" // _PyObject_CallNoArgsTstate() #include "pycore_ceval.h" // _Py_EnterRecursiveCallTstate() #include "pycore_dict.h" // _PyDict_FromItems() +#include "pycore_modsupport.h" // _Py_VaBuildStack() #include "pycore_object.h" // _PyCFunctionWithKeywords_TrampolineCall() #include "pycore_pyerrors.h" // _PyErr_Occurred() #include "pycore_pystate.h" // _PyThreadState_GET() diff --git a/Objects/enumobject.c b/Objects/enumobject.c index c9d90584c26b7..556666779d852 100644 --- a/Objects/enumobject.c +++ b/Objects/enumobject.c @@ -3,6 +3,7 @@ #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_long.h" // _PyLong_GetOne() +#include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _PyObject_GC_TRACK() #include "clinic/enumobject.c.h" diff --git a/Objects/floatobject.c b/Objects/floatobject.c index 83a263c0d9c67..fa55481f09dec 100644 --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -9,6 +9,7 @@ #include "pycore_initconfig.h" // _PyStatus_OK() #include "pycore_interp.h" // _PyInterpreterState.float_state #include "pycore_long.h" // _PyLong_GetOne() +#include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _PyObject_Init() #include "pycore_pymath.h" // _PY_SHORT_FLOAT_REPR #include "pycore_pystate.h" // _PyInterpreterState_GET() diff --git a/Objects/listobject.c b/Objects/listobject.c index f1f324f7439b4..98fa08962b6aa 100644 --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -5,6 +5,7 @@ #include "pycore_interp.h" // PyInterpreterState.list #include "pycore_list.h" // struct _Py_list_state, _PyListIterObject #include "pycore_long.h" // _PyLong_DigitCount +#include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_tuple.h" // _PyTuple_FromArray() #include diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index bda25c881845c..d4fccae65244c 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -5,8 +5,9 @@ #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_interp.h" // PyInterpreterState.importlib #include "pycore_object.h" // _PyType_AllocNoTrack -#include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_moduleobject.h" // _PyModule_GetDef() +#include "pycore_modsupport.h" // _PyModule_CreateInitialized() +#include "pycore_pystate.h" // _PyInterpreterState_GET() #include "structmember.h" // PyMemberDef diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c index beb86b9623bdb..6dc41d71287ca 100644 --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -2,8 +2,9 @@ #include "Python.h" #include "pycore_abstract.h" // _PyIndex_Check() -#include "pycore_range.h" #include "pycore_long.h" // _PyLong_GetZero() +#include "pycore_modsupport.h" // _PyArg_NoKwnames() +#include "pycore_range.h" #include "pycore_tuple.h" // _PyTuple_ITEMS() #include "structmember.h" // PyMemberDef diff --git a/Objects/setobject.c b/Objects/setobject.c index 58f0ae73c0c40..4ac541b950975 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -32,6 +32,7 @@ */ #include "Python.h" +#include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include // offsetof() diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c index 991edcc86677d..e85af2b75e473 100644 --- a/Objects/tupleobject.c +++ b/Objects/tupleobject.c @@ -5,6 +5,7 @@ #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_gc.h" // _PyObject_GC_IS_TRACKED() #include "pycore_initconfig.h" // _PyStatus_OK() +#include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _PyObject_GC_TRACK(), _Py_FatalRefcountError() /*[clinic input] diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 3d3a63a75bd2f..87519efef081c 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -8,6 +8,7 @@ #include "pycore_frame.h" // _PyInterpreterFrame #include "pycore_long.h" // _PyLong_IsNegative() #include "pycore_memoryobject.h" // _PyMemoryView_FromBufferProc() +#include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_moduleobject.h" // _PyModule_GetDef() #include "pycore_object.h" // _PyType_HasFeature() #include "pycore_pyerrors.h" // _PyErr_Occurred() diff --git a/Objects/weakrefobject.c b/Objects/weakrefobject.c index f3f6c86637e9d..bac3e79bb7c25 100644 --- a/Objects/weakrefobject.c +++ b/Objects/weakrefobject.c @@ -1,4 +1,5 @@ #include "Python.h" +#include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _PyObject_GET_WEAKREFS_LISTPTR() #include "pycore_weakref.h" // _PyWeakref_GET_REF() #include "structmember.h" // PyMemberDef diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 79ce2d3d14017..760962e4c4b6a 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -241,6 +241,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index d47a22909e1e3..aaebe1908e30d 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -627,6 +627,9 @@ Include\internal + + Include\internal + Include\internal diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c index 68fe315338a54..9fe0067daa678 100644 --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -4,13 +4,14 @@ #include #include "pycore_ast.h" // _PyAST_Validate() #include "pycore_call.h" // _PyObject_CallNoArgs() +#include "pycore_ceval.h" // _PyEval_Vector() #include "pycore_compile.h" // _PyAST_Compile() #include "pycore_long.h" // _PyLong_CompactValue +#include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _Py_AddToAllObjects() #include "pycore_pyerrors.h" // _PyErr_NoMemory() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_tuple.h" // _PyTuple_FromArray() -#include "pycore_ceval.h" // _PyEval_Vector() #include "clinic/bltinmodule.c.h" diff --git a/Python/instrumentation.c b/Python/instrumentation.c index 03d7d2f215af7..e29748f0ad987 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -3,6 +3,7 @@ #include "pycore_frame.h" #include "pycore_interp.h" #include "pycore_long.h" +#include "pycore_modsupport.h" // _PyModule_CreateInitialized() #include "pycore_namespace.h" #include "pycore_object.h" #include "pycore_opcode.h" diff --git a/Python/sysmodule.c b/Python/sysmodule.c index 56d771f70ef53..0ac6edc5a16f8 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -20,6 +20,7 @@ Data members: #include "pycore_frame.h" // _PyInterpreterFrame #include "pycore_initconfig.h" // _PyStatus_EXCEPTION() #include "pycore_long.h" // _PY_LONG_MAX_STR_DIGITS_THRESHOLD +#include "pycore_modsupport.h" // _PyModule_CreateInitialized() #include "pycore_namespace.h" // _PyNamespace_New() #include "pycore_object.h" // _PyObject_IS_GC() #include "pycore_pathconfig.h" // _PyPathConfig_ComputeSysPath0() From webhook-mailer at python.org Mon Jul 3 06:06:57 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 03 Jul 2023 10:06:57 -0000 Subject: [Python-checkins] gh-104050: Annotate Argument Clinic DSLParser attributes (#106357) Message-ID: https://github.com/python/cpython/commit/18fedd04a7b4d54b5eb7a34a1e1c9ca42219f882 commit: 18fedd04a7b4d54b5eb7a34a1e1c9ca42219f882 branch: main author: Erlend E. Aasland committer: AlexWaygood date: 2023-07-03T10:06:54Z summary: gh-104050: Annotate Argument Clinic DSLParser attributes (#106357) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 27d3d57286888..125604bff97f7 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4293,6 +4293,19 @@ def dedent(self, line): StateKeeper = Callable[[str | None], None] class DSLParser: + function: Function | None + state: StateKeeper + keyword_only: bool + positional_only: bool + group: int + parameter_state: int + seen_positional_with_default: bool + indent: IndentStack + kind: str + coexist: bool + parameter_continuation: str + preserve_output: bool + def __init__(self, clinic: Clinic) -> None: self.clinic = clinic @@ -4312,7 +4325,7 @@ def __init__(self, clinic: Clinic) -> None: def reset(self) -> None: self.function = None - self.state: StateKeeper = self.state_dsl_start + self.state = self.state_dsl_start self.parameter_indent = None self.keyword_only = False self.positional_only = False From webhook-mailer at python.org Mon Jul 3 06:48:53 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 03 Jul 2023 10:48:53 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyErr C API functions (#106356) Message-ID: https://github.com/python/cpython/commit/c5afc97fc2f149758f597c21f65b4a1c770bb827 commit: c5afc97fc2f149758f597c21f65b4a1c770bb827 branch: main author: Victor Stinner committer: vstinner date: 2023-07-03T10:48:50Z summary: gh-106320: Remove private _PyErr C API functions (#106356) Remove private _PyErr C API functions: move them to the internal C API (pycore_pyerrors.h). files: M Include/cpython/pyerrors.h M Include/internal/pycore_pyerrors.h M Modules/_io/bufferedio.c M Modules/_sqlite/cursor.c M Objects/moduleobject.c M Objects/obmalloc.c M Objects/unicodeobject.c M Parser/pegen_errors.c M Python/importdl.c diff --git a/Include/cpython/pyerrors.h b/Include/cpython/pyerrors.h index 156665cbdb1ba..5c128211bd525 100644 --- a/Include/cpython/pyerrors.h +++ b/Include/cpython/pyerrors.h @@ -91,31 +91,14 @@ typedef PyOSErrorObject PyWindowsErrorObject; /* Error handling definitions */ PyAPI_FUNC(void) _PyErr_SetKeyError(PyObject *); -PyAPI_FUNC(_PyErr_StackItem*) _PyErr_GetTopmostException(PyThreadState *tstate); -PyAPI_FUNC(PyObject*) _PyErr_GetHandledException(PyThreadState *); -PyAPI_FUNC(void) _PyErr_SetHandledException(PyThreadState *, PyObject *); -PyAPI_FUNC(void) _PyErr_GetExcInfo(PyThreadState *, PyObject **, PyObject **, PyObject **); /* Context manipulation (PEP 3134) */ Py_DEPRECATED(3.12) PyAPI_FUNC(void) _PyErr_ChainExceptions(PyObject *, PyObject *, PyObject *); PyAPI_FUNC(void) _PyErr_ChainExceptions1(PyObject *); -/* Like PyErr_Format(), but saves current exception as __context__ and - __cause__. - */ -PyAPI_FUNC(PyObject *) _PyErr_FormatFromCause( - PyObject *exception, - const char *format, /* ASCII-encoded string */ - ... - ); - /* In exceptions.c */ -PyAPI_FUNC(int) _PyException_AddNote( - PyObject *exc, - PyObject *note); - PyAPI_FUNC(PyObject*) PyUnstable_Exc_PrepReraiseStar( PyObject *orig, PyObject *excs); @@ -123,7 +106,6 @@ PyAPI_FUNC(PyObject*) PyUnstable_Exc_PrepReraiseStar( /* In signalmodule.c */ int PySignal_SetWakeupFd(int fd); -PyAPI_FUNC(int) _PyErr_CheckSignals(void); /* Support for adding program text to SyntaxErrors */ @@ -143,18 +125,6 @@ PyAPI_FUNC(PyObject *) PyErr_ProgramTextObject( PyObject *filename, int lineno); -PyAPI_FUNC(PyObject *) _PyErr_ProgramDecodedTextObject( - PyObject *filename, - int lineno, - const char* encoding); - -PyAPI_FUNC(PyObject *) _PyUnicodeTranslateError_Create( - PyObject *object, - Py_ssize_t start, - Py_ssize_t end, - const char *reason /* UTF-8 encoded string */ - ); - PyAPI_FUNC(void) _PyErr_WriteUnraisableMsg( const char *err_msg, PyObject *obj); @@ -163,16 +133,4 @@ PyAPI_FUNC(void) _Py_NO_RETURN _Py_FatalErrorFunc( const char *func, const char *message); -PyAPI_FUNC(void) _Py_NO_RETURN _Py_FatalErrorFormat( - const char *func, - const char *format, - ...); - -extern PyObject *_PyErr_SetImportErrorWithNameFrom( - PyObject *, - PyObject *, - PyObject *, - PyObject *); - - #define Py_FatalError(message) _Py_FatalErrorFunc(__func__, (message)) diff --git a/Include/internal/pycore_pyerrors.h b/Include/internal/pycore_pyerrors.h index d75bef0c3b5d8..e3ba4b75e3cfc 100644 --- a/Include/internal/pycore_pyerrors.h +++ b/Include/internal/pycore_pyerrors.h @@ -9,6 +9,54 @@ extern "C" { #endif +/* Error handling definitions */ + +PyAPI_FUNC(_PyErr_StackItem*) _PyErr_GetTopmostException(PyThreadState *tstate); +PyAPI_FUNC(PyObject*) _PyErr_GetHandledException(PyThreadState *); +PyAPI_FUNC(void) _PyErr_SetHandledException(PyThreadState *, PyObject *); +PyAPI_FUNC(void) _PyErr_GetExcInfo(PyThreadState *, PyObject **, PyObject **, PyObject **); + +/* Like PyErr_Format(), but saves current exception as __context__ and + __cause__. + */ +PyAPI_FUNC(PyObject *) _PyErr_FormatFromCause( + PyObject *exception, + const char *format, /* ASCII-encoded string */ + ... + ); + +PyAPI_FUNC(int) _PyException_AddNote( + PyObject *exc, + PyObject *note); + +PyAPI_FUNC(int) _PyErr_CheckSignals(void); + +/* Support for adding program text to SyntaxErrors */ + +PyAPI_FUNC(PyObject *) _PyErr_ProgramDecodedTextObject( + PyObject *filename, + int lineno, + const char* encoding); + +PyAPI_FUNC(PyObject *) _PyUnicodeTranslateError_Create( + PyObject *object, + Py_ssize_t start, + Py_ssize_t end, + const char *reason /* UTF-8 encoded string */ + ); + +PyAPI_FUNC(void) _Py_NO_RETURN _Py_FatalErrorFormat( + const char *func, + const char *format, + ...); + +extern PyObject *_PyErr_SetImportErrorWithNameFrom( + PyObject *, + PyObject *, + PyObject *, + PyObject *); + + /* runtime lifecycle */ extern PyStatus _PyErr_InitTypes(PyInterpreterState *); diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c index 9c6a9cddba225..4d120d4e8af3a 100644 --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -10,6 +10,7 @@ #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_object.h" +#include "pycore_pyerrors.h" // _Py_FatalErrorFormat() #include "structmember.h" // PyMemberDef #include "_iomodule.h" diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c index caeedbddb8d88..dba8ab61e41e7 100644 --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -21,11 +21,17 @@ * 3. This notice may not be removed or altered from any source distribution. */ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + #include "cursor.h" #include "microprotocols.h" #include "module.h" #include "util.h" +#include "pycore_pyerrors.h" // _PyErr_FormatFromCause() + typedef enum { TYPE_LONG, TYPE_FLOAT, diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index d4fccae65244c..3e500b555e70f 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -4,9 +4,10 @@ #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_interp.h" // PyInterpreterState.importlib -#include "pycore_object.h" // _PyType_AllocNoTrack -#include "pycore_moduleobject.h" // _PyModule_GetDef() #include "pycore_modsupport.h" // _PyModule_CreateInitialized() +#include "pycore_moduleobject.h" // _PyModule_GetDef() +#include "pycore_object.h" // _PyType_AllocNoTrack +#include "pycore_pyerrors.h" // _PyErr_FormatFromCause() #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "structmember.h" // PyMemberDef diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index 9620a8fbb44ca..eb68d7c030d29 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -2,10 +2,10 @@ #include "Python.h" #include "pycore_code.h" // stats -#include "pycore_pystate.h" // _PyInterpreterState_GET - #include "pycore_obmalloc.h" +#include "pycore_pyerrors.h" // _Py_FatalErrorFormat() #include "pycore_pymem.h" +#include "pycore_pystate.h" // _PyInterpreterState_GET #include // malloc() #include diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 79402714f23b9..12e379d0c7e6c 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -50,6 +50,7 @@ OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. #include "pycore_long.h" // _PyLong_FormatWriter() #include "pycore_object.h" // _PyObject_GC_TRACK(), _Py_FatalRefcountError() #include "pycore_pathconfig.h" // _Py_DumpPathConfig() +#include "pycore_pyerrors.h" // _PyUnicodeTranslateError_Create() #include "pycore_pylifecycle.h" // _Py_SetFileSystemEncoding() #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_ucnhash.h" // _PyUnicode_Name_CAPI diff --git a/Parser/pegen_errors.c b/Parser/pegen_errors.c index af529057f50e7..e543d40ccd8ab 100644 --- a/Parser/pegen_errors.c +++ b/Parser/pegen_errors.c @@ -1,6 +1,7 @@ #include #include +#include "pycore_pyerrors.h" // _PyErr_ProgramDecodedTextObject() #include "tokenizer.h" #include "pegen.h" diff --git a/Python/importdl.c b/Python/importdl.c index eb6b808ecba1d..9ab0a5ad33aaa 100644 --- a/Python/importdl.c +++ b/Python/importdl.c @@ -4,6 +4,7 @@ #include "Python.h" #include "pycore_call.h" #include "pycore_import.h" +#include "pycore_pyerrors.h" // _PyErr_FormatFromCause() #include "pycore_pystate.h" #include "pycore_runtime.h" From webhook-mailer at python.org Mon Jul 3 07:00:38 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 11:00:38 -0000 Subject: [Python-checkins] gh-104146: Remove unused attr 'parameter_indent' from clinic.DLParser (#106358) Message-ID: https://github.com/python/cpython/commit/60e41a0cbbe1a43e504b50a454efe653aba78a1a commit: 60e41a0cbbe1a43e504b50a454efe653aba78a1a branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T11:00:35Z summary: gh-104146: Remove unused attr 'parameter_indent' from clinic.DLParser (#106358) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 125604bff97f7..2e46131d4e336 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4326,7 +4326,6 @@ def __init__(self, clinic: Clinic) -> None: def reset(self) -> None: self.function = None self.state = self.state_dsl_start - self.parameter_indent = None self.keyword_only = False self.positional_only = False self.group = 0 From webhook-mailer at python.org Mon Jul 3 09:15:03 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 13:15:03 -0000 Subject: [Python-checkins] gh-106359: Fix corner case bugs in Argument Clinic converter parser (#106361) Message-ID: https://github.com/python/cpython/commit/0da4c883cf4185efe27b711c3e0a1e6e94397610 commit: 0da4c883cf4185efe27b711c3e0a1e6e94397610 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T13:14:59Z summary: gh-106359: Fix corner case bugs in Argument Clinic converter parser (#106361) DSLParser.parse_converter() could return unusable kwdicts in some rare cases Co-authored-by: Alex Waygood files: A Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index a6ce9c2570cb6..fab2d2bb0bdbe 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -813,6 +813,22 @@ def test_other_bizarre_things_in_annotations_fail(self): ) self.assertEqual(s, expected_failure_message) + def test_kwarg_splats_disallowed_in_function_call_annotations(self): + expected_error_msg = ( + "Error on line 0:\n" + "Cannot use a kwarg splat in a function-call annotation\n" + ) + dataset = ( + 'module fo\nfo.barbaz\n o: bool(**{None: "bang!"})', + 'module fo\nfo.barbaz -> bool(**{None: "bang!"})', + 'module fo\nfo.barbaz -> bool(**{"bang": 42})', + 'module fo\nfo.barbaz\n o: bool(**{"bang": None})', + ) + for fn in dataset: + with self.subTest(fn=fn): + out = self.parse_function_should_fail(fn) + self.assertEqual(out, expected_error_msg) + def test_unused_param(self): block = self.parse(""" module foo diff --git a/Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst b/Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst new file mode 100644 index 0000000000000..600c265391ec5 --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst @@ -0,0 +1,2 @@ +Argument Clinic now explicitly forbids "kwarg splats" in function calls used as +annotations. diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 2e46131d4e336..47038f98aabd7 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4291,6 +4291,7 @@ def dedent(self, line): StateKeeper = Callable[[str | None], None] +ConverterArgs = dict[str, Any] class DSLParser: function: Function | None @@ -5033,10 +5034,10 @@ def bad_node(self, node): key = f"{parameter_name}_as_{c_name}" if c_name else parameter_name self.function.parameters[key] = p - KwargDict = dict[str | None, Any] - @staticmethod - def parse_converter(annotation: ast.expr | None) -> tuple[str, bool, KwargDict]: + def parse_converter( + annotation: ast.expr | None + ) -> tuple[str, bool, ConverterArgs]: match annotation: case ast.Constant(value=str() as value): return value, True, {} @@ -5044,10 +5045,11 @@ def parse_converter(annotation: ast.expr | None) -> tuple[str, bool, KwargDict]: return name, False, {} case ast.Call(func=ast.Name(name)): symbols = globals() - kwargs = { - node.arg: eval_ast_expr(node.value, symbols) - for node in annotation.keywords - } + kwargs: ConverterArgs = {} + for node in annotation.keywords: + if not isinstance(node.arg, str): + fail("Cannot use a kwarg splat in a function-call annotation") + kwargs[node.arg] = eval_ast_expr(node.value, symbols) return name, False, kwargs case _: fail( From webhook-mailer at python.org Mon Jul 3 09:42:24 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 03 Jul 2023 13:42:24 -0000 Subject: [Python-checkins] [3.12] gh-106359: Fix corner case bugs in Argument Clinic converter parser (GH-106361) (#106364) Message-ID: https://github.com/python/cpython/commit/5f20152f01febf4ffd9572a8ab3e980a4660ef69 commit: 5f20152f01febf4ffd9572a8ab3e980a4660ef69 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-03T13:42:20Z summary: [3.12] gh-106359: Fix corner case bugs in Argument Clinic converter parser (GH-106361) (#106364) gh-106359: Fix corner case bugs in Argument Clinic converter parser (GH-106361) DSLParser.parse_converter() could return unusable kwdicts in some rare cases (cherry picked from commit 0da4c883cf4185efe27b711c3e0a1e6e94397610) Co-authored-by: Erlend E. Aasland Co-authored-by: Alex Waygood files: A Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index bea7303805567..76a06df5fe2e5 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -813,6 +813,22 @@ def test_other_bizarre_things_in_annotations_fail(self): ) self.assertEqual(s, expected_failure_message) + def test_kwarg_splats_disallowed_in_function_call_annotations(self): + expected_error_msg = ( + "Error on line 0:\n" + "Cannot use a kwarg splat in a function-call annotation\n" + ) + dataset = ( + 'module fo\nfo.barbaz\n o: bool(**{None: "bang!"})', + 'module fo\nfo.barbaz -> bool(**{None: "bang!"})', + 'module fo\nfo.barbaz -> bool(**{"bang": 42})', + 'module fo\nfo.barbaz\n o: bool(**{"bang": None})', + ) + for fn in dataset: + with self.subTest(fn=fn): + out = self.parse_function_should_fail(fn) + self.assertEqual(out, expected_error_msg) + def test_unused_param(self): block = self.parse(""" module foo diff --git a/Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst b/Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst new file mode 100644 index 0000000000000..600c265391ec5 --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst @@ -0,0 +1,2 @@ +Argument Clinic now explicitly forbids "kwarg splats" in function calls used as +annotations. diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index d182e5e7764e4..d67479c7f8ab8 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4289,6 +4289,7 @@ def dedent(self, line): StateKeeper = Callable[[str | None], None] +ConverterArgs = dict[str, Any] class DSLParser: def __init__(self, clinic: Clinic) -> None: @@ -5016,10 +5017,10 @@ def bad_node(self, node): key = f"{parameter_name}_as_{c_name}" if c_name else parameter_name self.function.parameters[key] = p - KwargDict = dict[str | None, Any] - @staticmethod - def parse_converter(annotation: ast.expr | None) -> tuple[str, bool, KwargDict]: + def parse_converter( + annotation: ast.expr | None + ) -> tuple[str, bool, ConverterArgs]: match annotation: case ast.Constant(value=str() as value): return value, True, {} @@ -5027,10 +5028,11 @@ def parse_converter(annotation: ast.expr | None) -> tuple[str, bool, KwargDict]: return name, False, {} case ast.Call(func=ast.Name(name)): symbols = globals() - kwargs = { - node.arg: eval_ast_expr(node.value, symbols) - for node in annotation.keywords - } + kwargs: ConverterArgs = {} + for node in annotation.keywords: + if not isinstance(node.arg, str): + fail("Cannot use a kwarg splat in a function-call annotation") + kwargs[node.arg] = eval_ast_expr(node.value, symbols) return name, False, kwargs case _: fail( From webhook-mailer at python.org Mon Jul 3 10:03:35 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 14:03:35 -0000 Subject: [Python-checkins] gh-104050: Add more type hints to Argument Clinic DSLParser() (#106354) Message-ID: https://github.com/python/cpython/commit/2e2daac2f11cbd762601382e759048fdbabba5bc commit: 2e2daac2f11cbd762601382e759048fdbabba5bc branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T14:03:31Z summary: gh-104050: Add more type hints to Argument Clinic DSLParser() (#106354) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 47038f98aabd7..41d4274fc744e 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4,6 +4,7 @@ # Copyright 2012-2013 by Larry Hastings. # Licensed to the PSF under a contributor agreement. # +from __future__ import annotations import abc import ast @@ -2448,7 +2449,7 @@ def __init__( cls: Class | None = None, c_basename: str | None = None, full_name: str | None = None, - return_converter: ReturnConverterType, + return_converter: CReturnConverter, return_annotation = inspect.Signature.empty, docstring: str | None = None, kind: str = CALLABLE, @@ -2467,7 +2468,7 @@ def __init__( self.docstring = docstring or '' self.kind = kind self.coexist = coexist - self.self_converter = None + self.self_converter: self_converter | None = None # docstring_only means "don't generate a machine-readable # signature, just a normal docstring". it's True for # functions with optional groups because we can't represent @@ -2531,7 +2532,7 @@ class Parameter: def __init__( self, name: str, - kind: str, + kind: inspect._ParameterKind, *, default = inspect.Parameter.empty, function: Function, @@ -4539,7 +4540,7 @@ def state_dsl_start(self, line: str | None) -> None: self.next(self.state_modulename_name, line) - def state_modulename_name(self, line): + def state_modulename_name(self, line: str | None) -> None: # looking for declaration, which establishes the leftmost column # line should be # modulename.fnname [as c_basename] [-> return annotation] @@ -4556,13 +4557,14 @@ def state_modulename_name(self, line): # this line is permitted to start with whitespace. # we'll call this number of spaces F (for "function"). - if not line.strip(): + if not self.valid_line(line): return self.indent.infer(line) # are we cloning? before, equals, existing = line.rpartition('=') + c_basename: str | None if equals: full_name, _, c_basename = before.partition(' as ') full_name = full_name.strip() @@ -4665,8 +4667,9 @@ def state_modulename_name(self, line): if cls and type == "PyObject *": kwargs['type'] = cls.typedef sc = self.function.self_converter = self_converter(name, name, self.function, **kwargs) - p_self = Parameter(sc.name, inspect.Parameter.POSITIONAL_ONLY, function=self.function, converter=sc) - self.function.parameters[sc.name] = p_self + p_self = Parameter(name, inspect.Parameter.POSITIONAL_ONLY, + function=self.function, converter=sc) + self.function.parameters[name] = p_self (cls or module).functions.append(self.function) self.next(self.state_parameters_start) @@ -4740,7 +4743,7 @@ def state_modulename_name(self, line): ps_start, ps_left_square_before, ps_group_before, ps_required, \ ps_optional, ps_group_after, ps_right_square_after = range(7) - def state_parameters_start(self, line: str) -> None: + def state_parameters_start(self, line: str | None) -> None: if not self.valid_line(line): return From webhook-mailer at python.org Mon Jul 3 10:25:26 2023 From: webhook-mailer at python.org (carljm) Date: Mon, 03 Jul 2023 14:25:26 -0000 Subject: [Python-checkins] gh-91053: make func watcher tests resilient to other func watchers (#106286) Message-ID: https://github.com/python/cpython/commit/58906213cc5d8f2be311664766b4923ef29dae1f commit: 58906213cc5d8f2be311664766b4923ef29dae1f branch: main author: Carl Meyer committer: carljm date: 2023-07-03T08:25:22-06:00 summary: gh-91053: make func watcher tests resilient to other func watchers (#106286) files: M Modules/_testcapi/watchers.c diff --git a/Modules/_testcapi/watchers.c b/Modules/_testcapi/watchers.c index d2c71fb401d36..7167943fffab3 100644 --- a/Modules/_testcapi/watchers.c +++ b/Modules/_testcapi/watchers.c @@ -432,9 +432,9 @@ allocate_too_many_code_watchers(PyObject *self, PyObject *args) // Test function watchers -#define NUM_FUNC_WATCHERS 2 -static PyObject *pyfunc_watchers[NUM_FUNC_WATCHERS]; -static int func_watcher_ids[NUM_FUNC_WATCHERS] = {-1, -1}; +#define NUM_TEST_FUNC_WATCHERS 2 +static PyObject *pyfunc_watchers[NUM_TEST_FUNC_WATCHERS]; +static int func_watcher_ids[NUM_TEST_FUNC_WATCHERS] = {-1, -1}; static PyObject * get_id(PyObject *obj) @@ -508,7 +508,7 @@ second_func_watcher_callback(PyFunction_WatchEvent event, return call_pyfunc_watcher(pyfunc_watchers[1], event, func, new_value); } -static PyFunction_WatchCallback func_watcher_callbacks[NUM_FUNC_WATCHERS] = { +static PyFunction_WatchCallback func_watcher_callbacks[NUM_TEST_FUNC_WATCHERS] = { first_func_watcher_callback, second_func_watcher_callback }; @@ -533,26 +533,25 @@ add_func_watcher(PyObject *self, PyObject *func) return NULL; } int idx = -1; - for (int i = 0; i < NUM_FUNC_WATCHERS; i++) { + for (int i = 0; i < NUM_TEST_FUNC_WATCHERS; i++) { if (func_watcher_ids[i] == -1) { idx = i; break; } } if (idx == -1) { - PyErr_SetString(PyExc_RuntimeError, "no free watchers"); - return NULL; - } - PyObject *result = PyLong_FromLong(idx); - if (result == NULL) { + PyErr_SetString(PyExc_RuntimeError, "no free test watchers"); return NULL; } func_watcher_ids[idx] = PyFunction_AddWatcher(func_watcher_callbacks[idx]); if (func_watcher_ids[idx] < 0) { - Py_DECREF(result); return NULL; } pyfunc_watchers[idx] = Py_NewRef(func); + PyObject *result = PyLong_FromLong(func_watcher_ids[idx]); + if (result == NULL) { + return NULL; + } return result; } @@ -569,7 +568,7 @@ clear_func_watcher(PyObject *self, PyObject *watcher_id_obj) return NULL; } int idx = -1; - for (int i = 0; i < NUM_FUNC_WATCHERS; i++) { + for (int i = 0; i < NUM_TEST_FUNC_WATCHERS; i++) { if (func_watcher_ids[i] == wid) { idx = i; break; From webhook-mailer at python.org Mon Jul 3 11:13:24 2023 From: webhook-mailer at python.org (carljm) Date: Mon, 03 Jul 2023 15:13:24 -0000 Subject: [Python-checkins] [3.12] gh-91053: make func watcher tests resilient to other func watchers (GH-106286) (#106365) Message-ID: https://github.com/python/cpython/commit/887a7e603628a7718233cf8068d7be8ccd9d7254 commit: 887a7e603628a7718233cf8068d7be8ccd9d7254 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: carljm date: 2023-07-03T09:13:21-06:00 summary: [3.12] gh-91053: make func watcher tests resilient to other func watchers (GH-106286) (#106365) gh-91053: make func watcher tests resilient to other func watchers (GH-106286) (cherry picked from commit 58906213cc5d8f2be311664766b4923ef29dae1f) Co-authored-by: Carl Meyer files: M Modules/_testcapi/watchers.c diff --git a/Modules/_testcapi/watchers.c b/Modules/_testcapi/watchers.c index d2c71fb401d36..7167943fffab3 100644 --- a/Modules/_testcapi/watchers.c +++ b/Modules/_testcapi/watchers.c @@ -432,9 +432,9 @@ allocate_too_many_code_watchers(PyObject *self, PyObject *args) // Test function watchers -#define NUM_FUNC_WATCHERS 2 -static PyObject *pyfunc_watchers[NUM_FUNC_WATCHERS]; -static int func_watcher_ids[NUM_FUNC_WATCHERS] = {-1, -1}; +#define NUM_TEST_FUNC_WATCHERS 2 +static PyObject *pyfunc_watchers[NUM_TEST_FUNC_WATCHERS]; +static int func_watcher_ids[NUM_TEST_FUNC_WATCHERS] = {-1, -1}; static PyObject * get_id(PyObject *obj) @@ -508,7 +508,7 @@ second_func_watcher_callback(PyFunction_WatchEvent event, return call_pyfunc_watcher(pyfunc_watchers[1], event, func, new_value); } -static PyFunction_WatchCallback func_watcher_callbacks[NUM_FUNC_WATCHERS] = { +static PyFunction_WatchCallback func_watcher_callbacks[NUM_TEST_FUNC_WATCHERS] = { first_func_watcher_callback, second_func_watcher_callback }; @@ -533,26 +533,25 @@ add_func_watcher(PyObject *self, PyObject *func) return NULL; } int idx = -1; - for (int i = 0; i < NUM_FUNC_WATCHERS; i++) { + for (int i = 0; i < NUM_TEST_FUNC_WATCHERS; i++) { if (func_watcher_ids[i] == -1) { idx = i; break; } } if (idx == -1) { - PyErr_SetString(PyExc_RuntimeError, "no free watchers"); - return NULL; - } - PyObject *result = PyLong_FromLong(idx); - if (result == NULL) { + PyErr_SetString(PyExc_RuntimeError, "no free test watchers"); return NULL; } func_watcher_ids[idx] = PyFunction_AddWatcher(func_watcher_callbacks[idx]); if (func_watcher_ids[idx] < 0) { - Py_DECREF(result); return NULL; } pyfunc_watchers[idx] = Py_NewRef(func); + PyObject *result = PyLong_FromLong(func_watcher_ids[idx]); + if (result == NULL) { + return NULL; + } return result; } @@ -569,7 +568,7 @@ clear_func_watcher(PyObject *self, PyObject *watcher_id_obj) return NULL; } int idx = -1; - for (int i = 0; i < NUM_FUNC_WATCHERS; i++) { + for (int i = 0; i < NUM_TEST_FUNC_WATCHERS; i++) { if (func_watcher_ids[i] == wid) { idx = i; break; From webhook-mailer at python.org Mon Jul 3 16:05:15 2023 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 03 Jul 2023 20:05:15 -0000 Subject: [Python-checkins] gh-106290: Fix edge cases around uops (#106319) Message-ID: https://github.com/python/cpython/commit/2028a4f6d996d2a46cbc33d0b65fdae284ee71fc commit: 2028a4f6d996d2a46cbc33d0b65fdae284ee71fc branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-03T20:05:11Z summary: gh-106290: Fix edge cases around uops (#106319) - Tweak uops debugging output - Fix the bug from gh-106290 - Rename `SET_IP` to `SAVE_IP` (per https://github.com/faster-cpython/ideas/issues/558) - Add a `SAVE_IP` uop at the start of the trace (ditto) - Allow `unbound_local_error`; this gives us uops for `LOAD_FAST_CHECK`, `LOAD_CLOSURE`, and `DELETE_FAST` - Longer traces - Support `STORE_FAST_LOAD_FAST`, `STORE_FAST_STORE_FAST` - Add deps on pycore_uops.h to Makefile(.pre.in) files: M Include/internal/pycore_uops.h M Makefile.pre.in M Python/ceval.c M Python/executor_cases.c.h M Python/opcode_metadata.h M Python/optimizer.c M Tools/cases_generator/generate_cases.py diff --git a/Include/internal/pycore_uops.h b/Include/internal/pycore_uops.h index 0e88d7e7f4a3b..5ed275fb85767 100644 --- a/Include/internal/pycore_uops.h +++ b/Include/internal/pycore_uops.h @@ -8,7 +8,7 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -#define _Py_UOP_MAX_TRACE_LENGTH 16 +#define _Py_UOP_MAX_TRACE_LENGTH 32 typedef struct { int opcode; diff --git a/Makefile.pre.in b/Makefile.pre.in index dcb3b5eb06a1e..41623bd2f1da7 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1800,6 +1800,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/internal/pycore_unionobject.h \ $(srcdir)/Include/internal/pycore_unicodeobject.h \ $(srcdir)/Include/internal/pycore_unicodeobject_generated.h \ + $(srcdir)/Include/internal/pycore_uops.h \ $(srcdir)/Include/internal/pycore_warnings.h \ $(srcdir)/Include/internal/pycore_weakref.h \ $(DTRACE_HEADERS) \ diff --git a/Python/ceval.c b/Python/ceval.c index 80ae85c24f054..6ce1a6a636ed7 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2773,24 +2773,26 @@ void Py_LeaveRecursiveCall(void) _PyInterpreterFrame * _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject **stack_pointer) { -#ifdef LLTRACE +#ifdef Py_DEBUG char *uop_debug = Py_GETENV("PYTHONUOPSDEBUG"); int lltrace = 0; if (uop_debug != NULL && *uop_debug >= '0') { lltrace = *uop_debug - '0'; // TODO: Parse an int and all that } - if (lltrace >= 2) { - PyCodeObject *code = _PyFrame_GetCode(frame); - _Py_CODEUNIT *instr = frame->prev_instr + 1; - fprintf(stderr, - "Entering _PyUopExecute for %s (%s:%d) at offset %ld\n", - PyUnicode_AsUTF8(code->co_qualname), - PyUnicode_AsUTF8(code->co_filename), - code->co_firstlineno, - (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); - } +#define DPRINTF(level, ...) \ + if (lltrace >= (level)) { fprintf(stderr, __VA_ARGS__); } +#else +#define DPRINTF(level, ...) #endif + DPRINTF(3, + "Entering _PyUopExecute for %s (%s:%d) at offset %ld\n", + PyUnicode_AsUTF8(_PyFrame_GetCode(frame)->co_qualname), + PyUnicode_AsUTF8(_PyFrame_GetCode(frame)->co_filename), + _PyFrame_GetCode(frame)->co_firstlineno, + (long)(frame->prev_instr + 1 - + (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive)); + PyThreadState *tstate = _PyThreadState_GET(); _PyUOpExecutorObject *self = (_PyUOpExecutorObject *)executor; @@ -2803,7 +2805,7 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject } OBJECT_STAT_INC(optimization_traces_executed); - _Py_CODEUNIT *ip_offset = (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive - 1; + _Py_CODEUNIT *ip_offset = (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive; int pc = 0; int opcode; uint64_t operand; @@ -2812,14 +2814,11 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject opcode = self->trace[pc].opcode; operand = self->trace[pc].operand; oparg = (int)operand; -#ifdef LLTRACE - if (lltrace >= 3) { - const char *opname = opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode]; - int stack_level = (int)(stack_pointer - _PyFrame_Stackbase(frame)); - fprintf(stderr, " uop %s, operand %" PRIu64 ", stack_level %d\n", - opname, operand, stack_level); - } -#endif + DPRINTF(3, + " uop %s, operand %" PRIu64 ", stack_level %d\n", + opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode], + operand, + (int)(stack_pointer - _PyFrame_Stackbase(frame))); pc++; OBJECT_STAT_INC(optimization_uops_executed); switch (opcode) { @@ -2828,7 +2827,7 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject #define ENABLE_SPECIALIZATION 0 #include "executor_cases.c.h" - case SET_IP: + case SAVE_IP: { frame->prev_instr = ip_offset + oparg; break; @@ -2836,6 +2835,7 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject case EXIT_TRACE: { + frame->prev_instr--; // Back up to just before destination _PyFrame_SetStackPointer(frame, stack_pointer); Py_DECREF(self); return frame; @@ -2850,6 +2850,13 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject } } +unbound_local_error: + format_exc_check_arg(tstate, PyExc_UnboundLocalError, + UNBOUNDLOCAL_ERROR_MSG, + PyTuple_GetItem(_PyFrame_GetCode(frame)->co_localsplusnames, oparg) + ); + goto error; + pop_4_error: STACK_SHRINK(1); pop_3_error: @@ -2861,11 +2868,7 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject error: // On ERROR_IF we return NULL as the frame. // The caller recovers the frame from cframe.current_frame. -#ifdef LLTRACE - if (lltrace >= 2) { - fprintf(stderr, "Error: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand); - } -#endif + DPRINTF(2, "Error: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand); _PyFrame_SetStackPointer(frame, stack_pointer); Py_DECREF(self); return NULL; @@ -2873,11 +2876,8 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject deoptimize: // On DEOPT_IF we just repeat the last instruction. // This presumes nothing was popped from the stack (nor pushed). -#ifdef LLTRACE - if (lltrace >= 2) { - fprintf(stderr, "DEOPT: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand); - } -#endif + DPRINTF(2, "DEOPT: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand); + frame->prev_instr--; // Back up to just before destination _PyFrame_SetStackPointer(frame, stack_pointer); Py_DECREF(self); return frame; diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 546b3d9f50ac7..d1e0443db613d 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -7,13 +7,25 @@ break; } + case LOAD_FAST_CHECK: { + PyObject *value; + #line 182 "Python/bytecodes.c" + value = GETLOCAL(oparg); + if (value == NULL) goto unbound_local_error; + Py_INCREF(value); + #line 17 "Python/executor_cases.c.h" + STACK_GROW(1); + stack_pointer[-1] = value; + break; + } + case LOAD_FAST: { PyObject *value; #line 188 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value); - #line 17 "Python/executor_cases.c.h" + #line 29 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -25,7 +37,7 @@ value = GETLOCAL(oparg); // do not use SETLOCAL here, it decrefs the old value GETLOCAL(oparg) = NULL; - #line 29 "Python/executor_cases.c.h" + #line 41 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -36,7 +48,7 @@ #line 209 "Python/bytecodes.c" value = GETITEM(FRAME_CO_CONSTS, oparg); Py_INCREF(value); - #line 40 "Python/executor_cases.c.h" + #line 52 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -46,7 +58,7 @@ PyObject *value = stack_pointer[-1]; #line 214 "Python/bytecodes.c" SETLOCAL(oparg, value); - #line 50 "Python/executor_cases.c.h" + #line 62 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -54,7 +66,7 @@ case POP_TOP: { PyObject *value = stack_pointer[-1]; #line 237 "Python/bytecodes.c" - #line 58 "Python/executor_cases.c.h" + #line 70 "Python/executor_cases.c.h" Py_DECREF(value); STACK_SHRINK(1); break; @@ -64,7 +76,7 @@ PyObject *res; #line 241 "Python/bytecodes.c" res = NULL; - #line 68 "Python/executor_cases.c.h" + #line 80 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -75,7 +87,7 @@ PyObject *receiver = stack_pointer[-2]; #line 260 "Python/bytecodes.c" Py_DECREF(receiver); - #line 79 "Python/executor_cases.c.h" + #line 91 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = value; break; @@ -86,11 +98,11 @@ PyObject *res; #line 275 "Python/bytecodes.c" res = PyNumber_Negative(value); - #line 90 "Python/executor_cases.c.h" + #line 102 "Python/executor_cases.c.h" Py_DECREF(value); #line 277 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 94 "Python/executor_cases.c.h" + #line 106 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -101,7 +113,7 @@ #line 281 "Python/bytecodes.c" assert(PyBool_Check(value)); res = Py_IsFalse(value) ? Py_True : Py_False; - #line 105 "Python/executor_cases.c.h" + #line 117 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -111,7 +123,7 @@ #line 313 "Python/bytecodes.c" DEOPT_IF(!PyBool_Check(value), TO_BOOL); STAT_INC(TO_BOOL, hit); - #line 115 "Python/executor_cases.c.h" + #line 127 "Python/executor_cases.c.h" break; } @@ -126,12 +138,12 @@ res = Py_False; } else { - #line 130 "Python/executor_cases.c.h" + #line 142 "Python/executor_cases.c.h" Py_DECREF(value); #line 326 "Python/bytecodes.c" res = Py_True; } - #line 135 "Python/executor_cases.c.h" + #line 147 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -143,7 +155,7 @@ DEOPT_IF(!PyList_CheckExact(value), TO_BOOL); STAT_INC(TO_BOOL, hit); res = Py_SIZE(value) ? Py_True : Py_False; - #line 147 "Python/executor_cases.c.h" + #line 159 "Python/executor_cases.c.h" Py_DECREF(value); stack_pointer[-1] = res; break; @@ -157,7 +169,7 @@ DEOPT_IF(!Py_IsNone(value), TO_BOOL); STAT_INC(TO_BOOL, hit); res = Py_False; - #line 161 "Python/executor_cases.c.h" + #line 173 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -174,12 +186,12 @@ } else { assert(Py_SIZE(value)); - #line 178 "Python/executor_cases.c.h" + #line 190 "Python/executor_cases.c.h" Py_DECREF(value); #line 354 "Python/bytecodes.c" res = Py_True; } - #line 183 "Python/executor_cases.c.h" + #line 195 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -193,11 +205,11 @@ assert(version); DEOPT_IF(Py_TYPE(value)->tp_version_tag != version, TO_BOOL); STAT_INC(TO_BOOL, hit); - #line 197 "Python/executor_cases.c.h" + #line 209 "Python/executor_cases.c.h" Py_DECREF(value); #line 364 "Python/bytecodes.c" res = Py_True; - #line 201 "Python/executor_cases.c.h" + #line 213 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -207,11 +219,11 @@ PyObject *res; #line 368 "Python/bytecodes.c" res = PyNumber_Invert(value); - #line 211 "Python/executor_cases.c.h" + #line 223 "Python/executor_cases.c.h" Py_DECREF(value); #line 370 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 215 "Python/executor_cases.c.h" + #line 227 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -222,7 +234,7 @@ #line 386 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); - #line 226 "Python/executor_cases.c.h" + #line 238 "Python/executor_cases.c.h" break; } @@ -236,7 +248,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 240 "Python/executor_cases.c.h" + #line 252 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -252,7 +264,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 256 "Python/executor_cases.c.h" + #line 268 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -268,7 +280,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 272 "Python/executor_cases.c.h" + #line 284 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -280,7 +292,7 @@ #line 422 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); - #line 284 "Python/executor_cases.c.h" + #line 296 "Python/executor_cases.c.h" break; } @@ -294,7 +306,7 @@ ((PyFloatObject *)left)->ob_fval * ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 298 "Python/executor_cases.c.h" + #line 310 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -310,7 +322,7 @@ ((PyFloatObject *)left)->ob_fval + ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 314 "Python/executor_cases.c.h" + #line 326 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -326,7 +338,7 @@ ((PyFloatObject *)left)->ob_fval - ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 330 "Python/executor_cases.c.h" + #line 342 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -338,7 +350,7 @@ #line 458 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(!PyUnicode_CheckExact(right), BINARY_OP); - #line 342 "Python/executor_cases.c.h" + #line 354 "Python/executor_cases.c.h" break; } @@ -352,7 +364,7 @@ _Py_DECREF_SPECIALIZED(left, _PyUnicode_ExactDealloc); _Py_DECREF_SPECIALIZED(right, _PyUnicode_ExactDealloc); if (res == NULL) goto pop_2_error; - #line 356 "Python/executor_cases.c.h" + #line 368 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -376,7 +388,7 @@ } Py_DECREF(container); if (res == NULL) goto pop_3_error; - #line 380 "Python/executor_cases.c.h" + #line 392 "Python/executor_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = res; break; @@ -400,7 +412,7 @@ Py_DECREF(v); Py_DECREF(container); if (err) goto pop_4_error; - #line 404 "Python/executor_cases.c.h" + #line 416 "Python/executor_cases.c.h" STACK_SHRINK(4); break; } @@ -423,7 +435,7 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 427 "Python/executor_cases.c.h" + #line 439 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -447,7 +459,7 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(tuple); - #line 451 "Python/executor_cases.c.h" + #line 463 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -465,14 +477,14 @@ if (!_PyErr_Occurred(tstate)) { _PyErr_SetKeyError(sub); } - #line 469 "Python/executor_cases.c.h" + #line 481 "Python/executor_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); #line 603 "Python/bytecodes.c" if (true) goto pop_2_error; } Py_INCREF(res); // Do this before DECREF'ing dict, sub - #line 476 "Python/executor_cases.c.h" + #line 488 "Python/executor_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); STACK_SHRINK(1); @@ -485,7 +497,7 @@ PyObject *list = stack_pointer[-(2 + (oparg-1))]; #line 635 "Python/bytecodes.c" if (_PyList_AppendTakeRef((PyListObject *)list, v) < 0) goto pop_1_error; - #line 489 "Python/executor_cases.c.h" + #line 501 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -495,11 +507,11 @@ PyObject *set = stack_pointer[-(2 + (oparg-1))]; #line 639 "Python/bytecodes.c" int err = PySet_Add(set, v); - #line 499 "Python/executor_cases.c.h" + #line 511 "Python/executor_cases.c.h" Py_DECREF(v); #line 641 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 503 "Python/executor_cases.c.h" + #line 515 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -525,7 +537,7 @@ Py_DECREF(old_value); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 529 "Python/executor_cases.c.h" + #line 541 "Python/executor_cases.c.h" STACK_SHRINK(3); break; } @@ -540,7 +552,7 @@ int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub, value); Py_DECREF(dict); if (err) goto pop_3_error; - #line 544 "Python/executor_cases.c.h" + #line 556 "Python/executor_cases.c.h" STACK_SHRINK(3); break; } @@ -551,12 +563,12 @@ #line 697 "Python/bytecodes.c" /* del container[sub] */ int err = PyObject_DelItem(container, sub); - #line 555 "Python/executor_cases.c.h" + #line 567 "Python/executor_cases.c.h" Py_DECREF(container); Py_DECREF(sub); #line 700 "Python/bytecodes.c" if (err) goto pop_2_error; - #line 560 "Python/executor_cases.c.h" + #line 572 "Python/executor_cases.c.h" STACK_SHRINK(2); break; } @@ -567,11 +579,11 @@ #line 704 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_1); res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value); - #line 571 "Python/executor_cases.c.h" + #line 583 "Python/executor_cases.c.h" Py_DECREF(value); #line 707 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 575 "Python/executor_cases.c.h" + #line 587 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -583,12 +595,12 @@ #line 711 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_2); res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1); - #line 587 "Python/executor_cases.c.h" + #line 599 "Python/executor_cases.c.h" Py_DECREF(value2); Py_DECREF(value1); #line 714 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 592 "Python/executor_cases.c.h" + #line 604 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -610,14 +622,14 @@ "'async for' requires an object with " "__aiter__ method, got %.100s", type->tp_name); - #line 614 "Python/executor_cases.c.h" + #line 626 "Python/executor_cases.c.h" Py_DECREF(obj); #line 832 "Python/bytecodes.c" if (true) goto pop_1_error; } iter = (*getter)(obj); - #line 621 "Python/executor_cases.c.h" + #line 633 "Python/executor_cases.c.h" Py_DECREF(obj); #line 837 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; @@ -632,7 +644,7 @@ Py_DECREF(iter); if (true) goto pop_1_error; } - #line 636 "Python/executor_cases.c.h" + #line 648 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -683,7 +695,7 @@ Py_DECREF(next_iter); } } - #line 687 "Python/executor_cases.c.h" + #line 699 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = awaitable; break; @@ -699,7 +711,7 @@ format_awaitable_error(tstate, Py_TYPE(iterable), oparg); } - #line 703 "Python/executor_cases.c.h" + #line 715 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 904 "Python/bytecodes.c" @@ -718,7 +730,7 @@ } if (iter == NULL) goto pop_1_error; - #line 722 "Python/executor_cases.c.h" + #line 734 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -728,7 +740,7 @@ #line 1034 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; Py_XSETREF(exc_info->exc_value, exc_value); - #line 732 "Python/executor_cases.c.h" + #line 744 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -737,7 +749,7 @@ PyObject *value; #line 1085 "Python/bytecodes.c" value = Py_NewRef(PyExc_AssertionError); - #line 741 "Python/executor_cases.c.h" + #line 753 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -767,7 +779,7 @@ if (true) goto error; } } - #line 771 "Python/executor_cases.c.h" + #line 783 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = bc; break; @@ -782,7 +794,7 @@ if (ns == NULL) { _PyErr_Format(tstate, PyExc_SystemError, "no locals found when storing %R", name); - #line 786 "Python/executor_cases.c.h" + #line 798 "Python/executor_cases.c.h" Py_DECREF(v); #line 1121 "Python/bytecodes.c" if (true) goto pop_1_error; @@ -791,11 +803,11 @@ err = PyDict_SetItem(ns, name, v); else err = PyObject_SetItem(ns, name, v); - #line 795 "Python/executor_cases.c.h" + #line 807 "Python/executor_cases.c.h" Py_DECREF(v); #line 1128 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 799 "Python/executor_cases.c.h" + #line 811 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -818,7 +830,7 @@ name); goto error; } - #line 822 "Python/executor_cases.c.h" + #line 834 "Python/executor_cases.c.h" break; } @@ -832,7 +844,7 @@ STAT_INC(UNPACK_SEQUENCE, hit); values[0] = Py_NewRef(PyTuple_GET_ITEM(seq, 1)); values[1] = Py_NewRef(PyTuple_GET_ITEM(seq, 0)); - #line 836 "Python/executor_cases.c.h" + #line 848 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -850,7 +862,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 854 "Python/executor_cases.c.h" + #line 866 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -868,7 +880,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 872 "Python/executor_cases.c.h" + #line 884 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -881,11 +893,11 @@ int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); - #line 885 "Python/executor_cases.c.h" + #line 897 "Python/executor_cases.c.h" Py_DECREF(seq); #line 1211 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 889 "Python/executor_cases.c.h" + #line 901 "Python/executor_cases.c.h" STACK_GROW((oparg & 0xFF) + (oparg >> 8)); break; } @@ -895,11 +907,11 @@ #line 1242 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_SetAttr(owner, name, (PyObject *)NULL); - #line 899 "Python/executor_cases.c.h" + #line 911 "Python/executor_cases.c.h" Py_DECREF(owner); #line 1245 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 903 "Python/executor_cases.c.h" + #line 915 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -909,11 +921,11 @@ #line 1249 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyDict_SetItem(GLOBALS(), name, v); - #line 913 "Python/executor_cases.c.h" + #line 925 "Python/executor_cases.c.h" Py_DECREF(v); #line 1252 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 917 "Python/executor_cases.c.h" + #line 929 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -931,7 +943,7 @@ } goto error; } - #line 935 "Python/executor_cases.c.h" + #line 947 "Python/executor_cases.c.h" break; } @@ -945,7 +957,7 @@ if (true) goto error; } Py_INCREF(locals); - #line 949 "Python/executor_cases.c.h" + #line 961 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = locals; break; @@ -1011,11 +1023,20 @@ } } } - #line 1015 "Python/executor_cases.c.h" + #line 1027 "Python/executor_cases.c.h" stack_pointer[-1] = v; break; } + case DELETE_FAST: { + #line 1435 "Python/bytecodes.c" + PyObject *v = GETLOCAL(oparg); + if (v == NULL) goto unbound_local_error; + SETLOCAL(oparg, NULL); + #line 1037 "Python/executor_cases.c.h" + break; + } + case DELETE_DEREF: { #line 1452 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); @@ -1028,7 +1049,7 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 1032 "Python/executor_cases.c.h" + #line 1053 "Python/executor_cases.c.h" break; } @@ -1070,7 +1091,7 @@ } Py_INCREF(value); } - #line 1074 "Python/executor_cases.c.h" + #line 1095 "Python/executor_cases.c.h" stack_pointer[-1] = value; break; } @@ -1085,7 +1106,7 @@ if (true) goto error; } Py_INCREF(value); - #line 1089 "Python/executor_cases.c.h" + #line 1110 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -1098,7 +1119,7 @@ PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 1102 "Python/executor_cases.c.h" + #line 1123 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -1115,7 +1136,7 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 1119 "Python/executor_cases.c.h" + #line 1140 "Python/executor_cases.c.h" break; } @@ -1124,13 +1145,13 @@ PyObject *str; #line 1532 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 1128 "Python/executor_cases.c.h" + #line 1149 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } #line 1534 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1134 "Python/executor_cases.c.h" + #line 1155 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -1143,7 +1164,7 @@ #line 1538 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1147 "Python/executor_cases.c.h" + #line 1168 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -1156,7 +1177,7 @@ #line 1543 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1160 "Python/executor_cases.c.h" + #line 1181 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -1177,13 +1198,13 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 1181 "Python/executor_cases.c.h" + #line 1202 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 1559 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val)); - #line 1187 "Python/executor_cases.c.h" + #line 1208 "Python/executor_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); break; @@ -1194,11 +1215,11 @@ PyObject *set = stack_pointer[-(2 + (oparg-1))]; #line 1566 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 1198 "Python/executor_cases.c.h" + #line 1219 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 1568 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 1202 "Python/executor_cases.c.h" + #line 1223 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -1221,7 +1242,7 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 1225 "Python/executor_cases.c.h" + #line 1246 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -1239,13 +1260,13 @@ if (map == NULL) goto error; - #line 1243 "Python/executor_cases.c.h" + #line 1264 "Python/executor_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } #line 1597 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 1249 "Python/executor_cases.c.h" + #line 1270 "Python/executor_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -1293,7 +1314,7 @@ Py_DECREF(ann_dict); } } - #line 1297 "Python/executor_cases.c.h" + #line 1318 "Python/executor_cases.c.h" break; } @@ -1311,14 +1332,14 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 1315 "Python/executor_cases.c.h" + #line 1336 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); #line 1653 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 1322 "Python/executor_cases.c.h" + #line 1343 "Python/executor_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; break; @@ -1334,12 +1355,12 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 1338 "Python/executor_cases.c.h" + #line 1359 "Python/executor_cases.c.h" Py_DECREF(update); #line 1665 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1343 "Python/executor_cases.c.h" + #line 1364 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1352,12 +1373,12 @@ if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 1356 "Python/executor_cases.c.h" + #line 1377 "Python/executor_cases.c.h" Py_DECREF(update); #line 1676 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1361 "Python/executor_cases.c.h" + #line 1382 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1372,7 +1393,7 @@ /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 1376 "Python/executor_cases.c.h" + #line 1397 "Python/executor_cases.c.h" STACK_SHRINK(2); break; } @@ -1390,13 +1411,13 @@ STAT_INC(LOAD_SUPER_ATTR, hit); PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL); - #line 1394 "Python/executor_cases.c.h" + #line 1415 "Python/executor_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); #line 1772 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; - #line 1400 "Python/executor_cases.c.h" + #line 1421 "Python/executor_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -1433,7 +1454,7 @@ res = res2; res2 = NULL; } - #line 1437 "Python/executor_cases.c.h" + #line 1458 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; stack_pointer[-2] = res2; @@ -1456,7 +1477,7 @@ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1460 "Python/executor_cases.c.h" + #line 1481 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1482,7 +1503,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1486 "Python/executor_cases.c.h" + #line 1507 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1505,7 +1526,7 @@ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1509 "Python/executor_cases.c.h" + #line 1530 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1517,12 +1538,12 @@ PyObject *b; #line 2163 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 1521 "Python/executor_cases.c.h" + #line 1542 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); #line 2165 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1526 "Python/executor_cases.c.h" + #line 1547 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1534,13 +1555,13 @@ PyObject *b; #line 2169 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 1538 "Python/executor_cases.c.h" + #line 1559 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); #line 2171 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; - #line 1544 "Python/executor_cases.c.h" + #line 1565 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1553,7 +1574,7 @@ PyObject *match; #line 2176 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 1557 "Python/executor_cases.c.h" + #line 1578 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); #line 2178 "Python/bytecodes.c" @@ -1564,7 +1585,7 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 1568 "Python/executor_cases.c.h" + #line 1589 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); #line 2186 "Python/bytecodes.c" @@ -1576,7 +1597,7 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 1580 "Python/executor_cases.c.h" + #line 1601 "Python/executor_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; break; @@ -1589,18 +1610,18 @@ #line 2197 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 1593 "Python/executor_cases.c.h" + #line 1614 "Python/executor_cases.c.h" Py_DECREF(right); #line 2200 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 1600 "Python/executor_cases.c.h" + #line 1621 "Python/executor_cases.c.h" Py_DECREF(right); #line 2205 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1604 "Python/executor_cases.c.h" + #line 1625 "Python/executor_cases.c.h" stack_pointer[-1] = b; break; } @@ -1614,7 +1635,7 @@ if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 1618 "Python/executor_cases.c.h" + #line 1639 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; break; @@ -1630,7 +1651,7 @@ // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 1634 "Python/executor_cases.c.h" + #line 1655 "Python/executor_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); @@ -1642,7 +1663,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 1646 "Python/executor_cases.c.h" + #line 1667 "Python/executor_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; break; @@ -1654,7 +1675,7 @@ #line 2327 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 1658 "Python/executor_cases.c.h" + #line 1679 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1666,7 +1687,7 @@ #line 2332 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 1670 "Python/executor_cases.c.h" + #line 1691 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1680,7 +1701,7 @@ // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 1684 "Python/executor_cases.c.h" + #line 1705 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; break; @@ -1692,11 +1713,11 @@ #line 2343 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 1696 "Python/executor_cases.c.h" + #line 1717 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 2346 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 1700 "Python/executor_cases.c.h" + #line 1721 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -1727,11 +1748,11 @@ if (iter == NULL) { goto error; } - #line 1731 "Python/executor_cases.c.h" + #line 1752 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 2373 "Python/bytecodes.c" } - #line 1735 "Python/executor_cases.c.h" + #line 1756 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -1762,7 +1783,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 1766 "Python/executor_cases.c.h" + #line 1787 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1781,7 +1802,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 1785 "Python/executor_cases.c.h" + #line 1806 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -1798,7 +1819,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 1802 "Python/executor_cases.c.h" + #line 1823 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -1818,7 +1839,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 1822 "Python/executor_cases.c.h" + #line 1843 "Python/executor_cases.c.h" stack_pointer[-1] = func; break; } @@ -1851,7 +1872,7 @@ default: Py_UNREACHABLE(); } - #line 1855 "Python/executor_cases.c.h" + #line 1876 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; break; @@ -1864,13 +1885,13 @@ PyObject *slice; #line 3491 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 1868 "Python/executor_cases.c.h" + #line 1889 "Python/executor_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); #line 3493 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 1874 "Python/executor_cases.c.h" + #line 1895 "Python/executor_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -1887,7 +1908,7 @@ result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 1891 "Python/executor_cases.c.h" + #line 1912 "Python/executor_cases.c.h" stack_pointer[-1] = result; break; } @@ -1906,7 +1927,7 @@ else { res = value; } - #line 1910 "Python/executor_cases.c.h" + #line 1931 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -1920,7 +1941,7 @@ Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 1924 "Python/executor_cases.c.h" + #line 1945 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1932,7 +1953,7 @@ #line 3526 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 1936 "Python/executor_cases.c.h" + #line 1957 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; break; @@ -1943,7 +1964,7 @@ PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; #line 3551 "Python/bytecodes.c" assert(oparg >= 2); - #line 1947 "Python/executor_cases.c.h" + #line 1968 "Python/executor_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; break; diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index 6a42775e09120..ac3d800d8fe0f 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -20,7 +20,7 @@ 0) #define EXIT_TRACE 300 -#define SET_IP 301 +#define SAVE_IP 301 #define _GUARD_BOTH_INT 302 #define _BINARY_OP_MULTIPLY_INT 303 #define _BINARY_OP_ADD_INT 304 @@ -1164,6 +1164,7 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { }; const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [NOP] = { .nuops = 1, .uops = { { NOP, 0, 0 } } }, + [LOAD_FAST_CHECK] = { .nuops = 1, .uops = { { LOAD_FAST_CHECK, 0, 0 } } }, [LOAD_FAST] = { .nuops = 1, .uops = { { LOAD_FAST, 0, 0 } } }, [LOAD_FAST_AND_CLEAR] = { .nuops = 1, .uops = { { LOAD_FAST_AND_CLEAR, 0, 0 } } }, [LOAD_CONST] = { .nuops = 1, .uops = { { LOAD_CONST, 0, 0 } } }, @@ -1218,6 +1219,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [LOAD_LOCALS] = { .nuops = 1, .uops = { { _LOAD_LOCALS, 0, 0 } } }, [LOAD_NAME] = { .nuops = 2, .uops = { { _LOAD_LOCALS, 0, 0 }, { _LOAD_FROM_DICT_OR_GLOBALS, 0, 0 } } }, [LOAD_FROM_DICT_OR_GLOBALS] = { .nuops = 1, .uops = { { _LOAD_FROM_DICT_OR_GLOBALS, 0, 0 } } }, + [DELETE_FAST] = { .nuops = 1, .uops = { { DELETE_FAST, 0, 0 } } }, [DELETE_DEREF] = { .nuops = 1, .uops = { { DELETE_DEREF, 0, 0 } } }, [LOAD_FROM_DICT_OR_DEREF] = { .nuops = 1, .uops = { { LOAD_FROM_DICT_OR_DEREF, 0, 0 } } }, [LOAD_DEREF] = { .nuops = 1, .uops = { { LOAD_DEREF, 0, 0 } } }, @@ -1266,7 +1268,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { #ifdef Py_DEBUG const char * const _PyOpcode_uop_name[512] = { [300] = "EXIT_TRACE", - [301] = "SET_IP", + [301] = "SAVE_IP", [302] = "_GUARD_BOTH_INT", [303] = "_BINARY_OP_MULTIPLY_INT", [304] = "_BINARY_OP_ADD_INT", diff --git a/Python/optimizer.c b/Python/optimizer.c index b00825ade16e0..32f0b1477d203 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -282,11 +282,6 @@ PyUnstable_Optimizer_NewCounter(void) ///////////////////// Experimental UOp Optimizer ///////////////////// -#ifdef Py_DEBUG - /* For debugging the interpreter: */ -# define LLTRACE 1 /* Low-level trace feature */ -#endif - static void uop_dealloc(_PyUOpExecutorObject *self) { PyObject_Free(self); @@ -308,60 +303,81 @@ translate_bytecode_to_trace( _PyUOpInstruction *trace, int max_length) { -#ifdef LLTRACE + int trace_length = 0; + +#ifdef Py_DEBUG char *uop_debug = Py_GETENV("PYTHONUOPSDEBUG"); int lltrace = 0; if (uop_debug != NULL && *uop_debug >= '0') { lltrace = *uop_debug - '0'; // TODO: Parse an int and all that } - if (lltrace >= 4) { - fprintf(stderr, - "Optimizing %s (%s:%d) at offset %ld\n", - PyUnicode_AsUTF8(code->co_qualname), - PyUnicode_AsUTF8(code->co_filename), - code->co_firstlineno, - (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); - } -#define ADD_TO_TRACE(OPCODE, OPERAND) \ - if (lltrace >= 2) { \ - const char *opname = (OPCODE) < 256 ? _PyOpcode_OpName[(OPCODE)] : _PyOpcode_uop_name[(OPCODE)]; \ - fprintf(stderr, " ADD_TO_TRACE(%s, %" PRIu64 ")\n", opname, (uint64_t)(OPERAND)); \ - } \ - trace[trace_length].opcode = (OPCODE); \ - trace[trace_length].operand = (OPERAND); \ - trace_length++; +#define DPRINTF(level, ...) \ + if (lltrace >= (level)) { fprintf(stderr, __VA_ARGS__); } #else -#define ADD_TO_TRACE(OPCODE, OPERAND) \ - trace[trace_length].opcode = (OPCODE); \ - trace[trace_length].operand = (OPERAND); \ - trace_length++; +#define DPRINTF(level, ...) #endif - int trace_length = 0; - // Always reserve space for one uop, plus SET_UP, plus EXIT_TRACE - while (trace_length + 3 <= max_length) { +#define ADD_TO_TRACE(OPCODE, OPERAND) \ + DPRINTF(2, \ + " ADD_TO_TRACE(%s, %" PRIu64 ")\n", \ + (OPCODE) < 256 ? _PyOpcode_OpName[(OPCODE)] : _PyOpcode_uop_name[(OPCODE)], \ + (uint64_t)(OPERAND)); \ + assert(trace_length < max_length); \ + trace[trace_length].opcode = (OPCODE); \ + trace[trace_length].operand = (OPERAND); \ + trace_length++; + + DPRINTF(4, + "Optimizing %s (%s:%d) at offset %ld\n", + PyUnicode_AsUTF8(code->co_qualname), + PyUnicode_AsUTF8(code->co_filename), + code->co_firstlineno, + (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); + + for (;;) { + ADD_TO_TRACE(SAVE_IP, (int)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); int opcode = instr->op.code; uint64_t operand = instr->op.arg; switch (opcode) { case LOAD_FAST_LOAD_FAST: + case STORE_FAST_LOAD_FAST: + case STORE_FAST_STORE_FAST: { - // Reserve space for two uops (+ SETUP + EXIT_TRACE) + // Reserve space for two uops (+ SAVE_IP + EXIT_TRACE) if (trace_length + 4 > max_length) { + DPRINTF(1, "Ran out of space for LOAD_FAST_LOAD_FAST\n"); goto done; } uint64_t oparg1 = operand >> 4; uint64_t oparg2 = operand & 15; - ADD_TO_TRACE(LOAD_FAST, oparg1); - ADD_TO_TRACE(LOAD_FAST, oparg2); + switch (opcode) { + case LOAD_FAST_LOAD_FAST: + ADD_TO_TRACE(LOAD_FAST, oparg1); + ADD_TO_TRACE(LOAD_FAST, oparg2); + break; + case STORE_FAST_LOAD_FAST: + ADD_TO_TRACE(STORE_FAST, oparg1); + ADD_TO_TRACE(LOAD_FAST, oparg2); + break; + case STORE_FAST_STORE_FAST: + ADD_TO_TRACE(STORE_FAST, oparg1); + ADD_TO_TRACE(STORE_FAST, oparg2); + break; + default: + Py_FatalError("Missing case"); + } break; } default: { const struct opcode_macro_expansion *expansion = &_PyOpcode_macro_expansion[opcode]; if (expansion->nuops > 0) { - // Reserve space for nuops (+ SETUP + EXIT_TRACE) + // Reserve space for nuops (+ SAVE_IP + EXIT_TRACE) int nuops = expansion->nuops; if (trace_length + nuops + 2 > max_length) { + DPRINTF(1, + "Ran out of space for %s\n", + opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode]); goto done; } for (int i = 0; i < nuops; i++) { @@ -387,49 +403,45 @@ translate_bytecode_to_trace( Py_FatalError("garbled expansion"); } ADD_TO_TRACE(expansion->uops[i].uop, operand); - assert(expansion->uops[0].size == 0); // TODO } break; } - // fprintf(stderr, "Unsupported opcode %d\n", opcode); - goto done; // Break out of while loop + DPRINTF(2, + "Unsupported opcode %s\n", + opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode]); + goto done; // Break out of loop } } instr++; // Add cache size for opcode instr += _PyOpcode_Caches[_PyOpcode_Deopt[opcode]]; - ADD_TO_TRACE(SET_IP, (int)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); } + done: - if (trace_length > 0) { + // Skip short traces like SAVE_IP, LOAD_FAST, SAVE_IP, EXIT_TRACE + if (trace_length > 3) { ADD_TO_TRACE(EXIT_TRACE, 0); -#ifdef LLTRACE - if (lltrace >= 1) { - fprintf(stderr, - "Created a trace for %s (%s:%d) at offset %ld -- length %d\n", - PyUnicode_AsUTF8(code->co_qualname), - PyUnicode_AsUTF8(code->co_filename), - code->co_firstlineno, - (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive), - trace_length); - } -#endif + DPRINTF(1, + "Created a trace for %s (%s:%d) at offset %ld -- length %d\n", + PyUnicode_AsUTF8(code->co_qualname), + PyUnicode_AsUTF8(code->co_filename), + code->co_firstlineno, + (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive), + trace_length); + return trace_length; } else { -#ifdef LLTRACE - if (lltrace >= 4) { - fprintf(stderr, - "No trace for %s (%s:%d) at offset %ld\n", - PyUnicode_AsUTF8(code->co_qualname), - PyUnicode_AsUTF8(code->co_filename), - code->co_firstlineno, - (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); - } -#endif + DPRINTF(4, + "No trace for %s (%s:%d) at offset %ld\n", + PyUnicode_AsUTF8(code->co_qualname), + PyUnicode_AsUTF8(code->co_filename), + code->co_firstlineno, + (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); } - return trace_length; + return 0; #undef ADD_TO_TRACE +#undef DPRINTF } static int diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 38be98034e0d8..657dfa93fd537 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -308,8 +308,7 @@ class ActiveCacheEffect: FORBIDDEN_NAMES_IN_UOPS = ( - "resume_with_error", # Proxy for "goto", which isn't an IDENTIFIER - "unbound_local_error", + "resume_with_error", "kwnames", "next_instr", "oparg1", # Proxy for super-instructions like LOAD_FAST_LOAD_FAST @@ -401,20 +400,25 @@ def __init__(self, inst: parser.InstDef): def is_viable_uop(self) -> bool: """Whether this instruction is viable as a uop.""" if self.always_exits: + # print(f"Skipping {self.name} because it always exits") return False if self.instr_flags.HAS_ARG_FLAG: # If the instruction uses oparg, it cannot use any caches if self.active_caches: + # print(f"Skipping {self.name} because it uses oparg and caches") return False else: # If it doesn't use oparg, it can have one cache entry if len(self.active_caches) > 1: + # print(f"Skipping {self.name} because it has >1 cache entries") return False + res = True for forbidden in FORBIDDEN_NAMES_IN_UOPS: # TODO: Don't check in '#ifdef ENABLE_SPECIALIZATION' regions if variable_used(self.inst, forbidden): - return False - return True + # print(f"Skipping {self.name} because it uses {forbidden}") + res = False + return res def write(self, out: Formatter, tier: Tiers = TIER_ONE) -> None: """Write one instruction, sans prologue and epilogue.""" @@ -1323,7 +1327,7 @@ def add(name: str) -> None: self.out.emit(make_text(name, counter)) counter += 1 add("EXIT_TRACE") - add("SET_IP") + add("SAVE_IP") for instr in self.instrs.values(): if instr.kind == "op" and instr.is_viable_uop(): add(instr.name) From webhook-mailer at python.org Mon Jul 3 16:10:05 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Mon, 03 Jul 2023 20:10:05 -0000 Subject: [Python-checkins] gh-106078: Move `context template` to decimal module global state (#106346) Message-ID: https://github.com/python/cpython/commit/bf06c6825cadeda54a9c0848fa51463a0e0b2cf8 commit: bf06c6825cadeda54a9c0848fa51463a0e0b2cf8 branch: main author: Charlie Zhao committer: kumaraditya303 <59607654+kumaraditya303 at users.noreply.github.com> date: 2023-07-04T01:40:01+05:30 summary: gh-106078: Move `context template` to decimal module global state (#106346) files: M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/globals-to-fix.tsv diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index da62372500342..6da5095b91018 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -49,6 +49,14 @@ typedef struct { /* Top level Exception; inherits from ArithmeticError */ PyObject *DecimalException; + + /* Template for creating new thread contexts, calling Context() without + * arguments and initializing the module_context on first access. */ + PyObject *default_context_template; + + /* Basic and extended context templates */ + PyObject *basic_context_template; + PyObject *extended_context_template; } decimal_state; static decimal_state global_state; @@ -147,14 +155,6 @@ static PyDecContextObject *cached_context = NULL; static PyObject *current_context_var = NULL; #endif -/* Template for creating new thread contexts, calling Context() without - * arguments and initializing the module_context on first access. */ -static PyObject *default_context_template = NULL; -/* Basic and extended context templates */ -static PyObject *basic_context_template = NULL; -static PyObject *extended_context_template = NULL; - - /* Error codes for functions that return signals or conditions */ #define DEC_INVALID_SIGNALS (MPD_Max_status+1U) #define DEC_ERR_OCCURRED (DEC_INVALID_SIGNALS<<1) @@ -1272,8 +1272,8 @@ context_new(PyTypeObject *type, PyObject *args UNUSED, PyObject *kwds UNUSED) ctx = CTX(self); - if (default_context_template) { - *ctx = *CTX(default_context_template); + if (state->default_context_template) { + *ctx = *CTX(state->default_context_template); } else { *ctx = dflt_ctx; @@ -1576,7 +1576,7 @@ current_context_from_dict(void) } /* Set up a new thread local context. */ - tl_context = context_copy(default_context_template, NULL); + tl_context = context_copy(state->default_context_template, NULL); if (tl_context == NULL) { return NULL; } @@ -1649,9 +1649,9 @@ PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) /* If the new context is one of the templates, make a copy. * This is the current behavior of decimal.py. */ - if (v == default_context_template || - v == basic_context_template || - v == extended_context_template) { + if (v == state->default_context_template || + v == state->basic_context_template || + v == state->extended_context_template) { v = context_copy(v, NULL); if (v == NULL) { return NULL; @@ -1675,7 +1675,8 @@ PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) static PyObject * init_current_context(void) { - PyObject *tl_context = context_copy(default_context_template, NULL); + decimal_state *state = GLOBAL_STATE(); + PyObject *tl_context = context_copy(state->default_context_template, NULL); if (tl_context == NULL) { return NULL; } @@ -1730,9 +1731,9 @@ PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) /* If the new context is one of the templates, make a copy. * This is the current behavior of decimal.py. */ - if (v == default_context_template || - v == basic_context_template || - v == extended_context_template) { + if (v == state->default_context_template || + v == state->basic_context_template || + v == state->extended_context_template) { v = context_copy(v, NULL); if (v == NULL) { return NULL; @@ -5980,10 +5981,10 @@ PyInit__decimal(void) /* Init default context template first */ - ASSIGN_PTR(default_context_template, + ASSIGN_PTR(state->default_context_template, PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL)); CHECK_INT(PyModule_AddObject(m, "DefaultContext", - Py_NewRef(default_context_template))); + Py_NewRef(state->default_context_template))); #ifndef WITH_DECIMAL_CONTEXTVAR ASSIGN_PTR(tls_context_key, PyUnicode_FromString("___DECIMAL_CTX__")); @@ -5995,18 +5996,18 @@ PyInit__decimal(void) CHECK_INT(PyModule_AddObject(m, "HAVE_THREADS", Py_NewRef(Py_True))); /* Init basic context template */ - ASSIGN_PTR(basic_context_template, + ASSIGN_PTR(state->basic_context_template, PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL)); - init_basic_context(basic_context_template); + init_basic_context(state->basic_context_template); CHECK_INT(PyModule_AddObject(m, "BasicContext", - Py_NewRef(basic_context_template))); + Py_NewRef(state->basic_context_template))); /* Init extended context template */ - ASSIGN_PTR(extended_context_template, + ASSIGN_PTR(state->extended_context_template, PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL)); - init_extended_context(extended_context_template); + init_extended_context(state->extended_context_template); CHECK_INT(PyModule_AddObject(m, "ExtendedContext", - Py_NewRef(extended_context_template))); + Py_NewRef(state->extended_context_template))); /* Init mpd_ssize_t constants */ @@ -6046,14 +6047,14 @@ PyInit__decimal(void) Py_CLEAR(MutableMapping); /* GCOV_NOT_REACHED */ Py_CLEAR(SignalTuple); /* GCOV_NOT_REACHED */ Py_CLEAR(state->DecimalTuple); /* GCOV_NOT_REACHED */ - Py_CLEAR(default_context_template); /* GCOV_NOT_REACHED */ + Py_CLEAR(state->default_context_template); /* GCOV_NOT_REACHED */ #ifndef WITH_DECIMAL_CONTEXTVAR Py_CLEAR(tls_context_key); /* GCOV_NOT_REACHED */ #else Py_CLEAR(current_context_var); /* GCOV_NOT_REACHED */ #endif - Py_CLEAR(basic_context_template); /* GCOV_NOT_REACHED */ - Py_CLEAR(extended_context_template); /* GCOV_NOT_REACHED */ + Py_CLEAR(state->basic_context_template); /* GCOV_NOT_REACHED */ + Py_CLEAR(state->extended_context_template); /* GCOV_NOT_REACHED */ Py_CLEAR(m); /* GCOV_NOT_REACHED */ return NULL; /* GCOV_NOT_REACHED */ diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 8fdc54df2b072..ad3d9b6513d69 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -422,10 +422,7 @@ Modules/_datetimemodule.c - us_per_day - Modules/_datetimemodule.c - us_per_week - Modules/_datetimemodule.c - seconds_per_day - Modules/_decimal/_decimal.c - global_state - -Modules/_decimal/_decimal.c - basic_context_template - Modules/_decimal/_decimal.c - current_context_var - -Modules/_decimal/_decimal.c - default_context_template - -Modules/_decimal/_decimal.c - extended_context_template - Modules/_decimal/_decimal.c - round_map - Modules/_decimal/_decimal.c - Rational - Modules/_decimal/_decimal.c - SignalTuple - From webhook-mailer at python.org Mon Jul 3 16:16:53 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 20:16:53 -0000 Subject: [Python-checkins] gh-106368: Increase Argument Clinic test coverage (#106369) Message-ID: https://github.com/python/cpython/commit/7f4c8121db62a9f72f00f2d9f73381e82f289581 commit: 7f4c8121db62a9f72f00f2d9f73381e82f289581 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T20:16:50Z summary: gh-106368: Increase Argument Clinic test coverage (#106369) Add tests for 'self' and 'defining_class' converter requirements. files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index fab2d2bb0bdbe..51d2ac972752f 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -829,6 +829,63 @@ def test_kwarg_splats_disallowed_in_function_call_annotations(self): out = self.parse_function_should_fail(fn) self.assertEqual(out, expected_error_msg) + def test_self_param_placement(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'self' parameter, if specified, must be the very first thing " + "in the parameter block.\n" + ) + block = """ + module foo + foo.func + a: int + self: self(type="PyObject *") + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + + def test_self_param_cannot_be_optional(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'self' parameter cannot be marked optional.\n" + ) + block = """ + module foo + foo.func + self: self(type="PyObject *") = None + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + + def test_defining_class_param_placement(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'defining_class' parameter, if specified, must either be the " + "first thing in the parameter block, or come just after 'self'.\n" + ) + block = """ + module foo + foo.func + self: self(type="PyObject *") + a: int + cls: defining_class + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + + def test_defining_class_param_cannot_be_optional(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'defining_class' parameter cannot be marked optional.\n" + ) + block = """ + module foo + foo.func + cls: defining_class(type="PyObject *") = None + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + def test_unused_param(self): block = self.parse(""" module foo From webhook-mailer at python.org Mon Jul 3 16:28:31 2023 From: webhook-mailer at python.org (markshannon) Date: Mon, 03 Jul 2023 20:28:31 -0000 Subject: [Python-checkins] GH-104584: Fix ENTER_EXECUTOR (GH-106141) Message-ID: https://github.com/python/cpython/commit/e5862113dde7a66b08f1ece542a3cfaf0a3d9080 commit: e5862113dde7a66b08f1ece542a3cfaf0a3d9080 branch: main author: Mark Shannon committer: markshannon date: 2023-07-03T21:28:27+01:00 summary: GH-104584: Fix ENTER_EXECUTOR (GH-106141) * Check eval-breaker in ENTER_EXECUTOR. * Make sure that frame->prev_instr is set before entering executor. files: M Include/internal/pycore_ceval.h M Python/bytecodes.c M Python/ceval.c M Python/ceval_gil.c M Python/ceval_macros.h M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/opcode_metadata.h diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h index 9e9b523e7c222..46bc18cff86d5 100644 --- a/Include/internal/pycore_ceval.h +++ b/Include/internal/pycore_ceval.h @@ -152,6 +152,8 @@ extern struct _PyInterpreterFrame* _PyEval_GetFrame(void); extern PyObject* _Py_MakeCoro(PyFunctionObject *func); +/* Handle signals, pending calls, GIL drop request + and asynchronous exception */ extern int _Py_HandlePending(PyThreadState *tstate); diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 8aab7ebc5cf0c..f69ac2beef4c2 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -141,8 +141,8 @@ dummy_func( ERROR_IF(err, error); next_instr--; } - else if (_Py_atomic_load_relaxed_int32(&tstate->interp->ceval.eval_breaker) && oparg < 2) { - goto handle_eval_breaker; + else if (oparg < 2) { + CHECK_EVAL_BREAKER(); } } @@ -158,6 +158,9 @@ dummy_func( next_instr--; } else { + if (oparg < 2) { + CHECK_EVAL_BREAKER(); + } _PyFrame_SetStackPointer(frame, stack_pointer); int err = _Py_call_instrumentation( tstate, oparg > 0, frame, next_instr-1); @@ -168,9 +171,6 @@ dummy_func( next_instr = frame->prev_instr; DISPATCH(); } - if (_Py_atomic_load_relaxed_int32(&tstate->interp->ceval.eval_breaker) && oparg < 2) { - goto handle_eval_breaker; - } } } @@ -2223,6 +2223,7 @@ dummy_func( } inst(JUMP_BACKWARD, (--)) { + CHECK_EVAL_BREAKER(); _Py_CODEUNIT *here = next_instr - 1; assert(oparg <= INSTR_OFFSET()); JUMPBY(1-oparg); @@ -2240,7 +2241,6 @@ dummy_func( goto resume_frame; } #endif /* ENABLE_SPECIALIZATION */ - CHECK_EVAL_BREAKER(); } pseudo(JUMP) = { @@ -2254,8 +2254,13 @@ dummy_func( }; inst(ENTER_EXECUTOR, (--)) { + CHECK_EVAL_BREAKER(); + PyCodeObject *code = _PyFrame_GetCode(frame); _PyExecutorObject *executor = (_PyExecutorObject *)code->co_executors->executors[oparg&255]; + int original_oparg = executor->vm_data.oparg | (oparg & 0xfffff00); + JUMPBY(1-original_oparg); + frame->prev_instr = next_instr - 1; Py_INCREF(executor); frame = executor->execute(executor, frame, stack_pointer); if (frame == NULL) { @@ -3570,8 +3575,8 @@ dummy_func( } inst(INSTRUMENTED_JUMP_BACKWARD, ( -- )) { - INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); CHECK_EVAL_BREAKER(); + INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); } inst(INSTRUMENTED_POP_JUMP_IF_TRUE, ( -- )) { diff --git a/Python/ceval.c b/Python/ceval.c index 6ce1a6a636ed7..9bcb83f9c993c 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -763,68 +763,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int DISPATCH(); -handle_eval_breaker: - - /* Do periodic things, like check for signals and async I/0. - * We need to do reasonably frequently, but not too frequently. - * All loops should include a check of the eval breaker. - * We also check on return from any builtin function. - * - * ## More Details ### - * - * The eval loop (this function) normally executes the instructions - * of a code object sequentially. However, the runtime supports a - * number of out-of-band execution scenarios that may pause that - * sequential execution long enough to do that out-of-band work - * in the current thread using the current PyThreadState. - * - * The scenarios include: - * - * - cyclic garbage collection - * - GIL drop requests - * - "async" exceptions - * - "pending calls" (some only in the main thread) - * - signal handling (only in the main thread) - * - * When the need for one of the above is detected, the eval loop - * pauses long enough to handle the detected case. Then, if doing - * so didn't trigger an exception, the eval loop resumes executing - * the sequential instructions. - * - * To make this work, the eval loop periodically checks if any - * of the above needs to happen. The individual checks can be - * expensive if computed each time, so a while back we switched - * to using pre-computed, per-interpreter variables for the checks, - * and later consolidated that to a single "eval breaker" variable - * (now a PyInterpreterState field). - * - * For the longest time, the eval breaker check would happen - * frequently, every 5 or so times through the loop, regardless - * of what instruction ran last or what would run next. Then, in - * early 2021 (gh-18334, commit 4958f5d), we switched to checking - * the eval breaker less frequently, by hard-coding the check to - * specific places in the eval loop (e.g. certain instructions). - * The intent then was to check after returning from calls - * and on the back edges of loops. - * - * In addition to being more efficient, that approach keeps - * the eval loop from running arbitrary code between instructions - * that don't handle that well. (See gh-74174.) - * - * Currently, the eval breaker check happens here at the - * "handle_eval_breaker" label. Some instructions come here - * explicitly (goto) and some indirectly. Notably, the check - * happens on back edges in the control flow graph, which - * pretty much applies to all loops and most calls. - * (See bytecodes.c for exact information.) - * - * One consequence of this approach is that it might not be obvious - * how to force any specific thread to pick up the eval breaker, - * or for any specific thread to not pick it up. Mostly this - * involves judicious uses of locks and careful ordering of code, - * while avoiding code that might trigger the eval breaker - * until so desired. - */ if (_Py_HandlePending(tstate) != 0) { goto error; } @@ -2796,13 +2734,7 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject PyThreadState *tstate = _PyThreadState_GET(); _PyUOpExecutorObject *self = (_PyUOpExecutorObject *)executor; - // Equivalent to CHECK_EVAL_BREAKER() - _Py_CHECK_EMSCRIPTEN_SIGNALS_PERIODICALLY(); - if (_Py_atomic_load_relaxed_int32(&tstate->interp->ceval.eval_breaker)) { - if (_Py_HandlePending(tstate) != 0) { - goto error; - } - } + CHECK_EVAL_BREAKER(); OBJECT_STAT_INC(optimization_traces_executed); _Py_CODEUNIT *ip_offset = (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive; diff --git a/Python/ceval_gil.c b/Python/ceval_gil.c index c6b1f9ef689ee..7c9ad07cc7207 100644 --- a/Python/ceval_gil.c +++ b/Python/ceval_gil.c @@ -1052,8 +1052,65 @@ _PyEval_FiniState(struct _ceval_state *ceval) } } -/* Handle signals, pending calls, GIL drop request - and asynchronous exception */ + +/* Do periodic things, like check for signals and async I/0. +* We need to do reasonably frequently, but not too frequently. +* All loops should include a check of the eval breaker. +* We also check on return from any builtin function. +* +* ## More Details ### +* +* The eval loop (this function) normally executes the instructions +* of a code object sequentially. However, the runtime supports a +* number of out-of-band execution scenarios that may pause that +* sequential execution long enough to do that out-of-band work +* in the current thread using the current PyThreadState. +* +* The scenarios include: +* +* - cyclic garbage collection +* - GIL drop requests +* - "async" exceptions +* - "pending calls" (some only in the main thread) +* - signal handling (only in the main thread) +* +* When the need for one of the above is detected, the eval loop +* pauses long enough to handle the detected case. Then, if doing +* so didn't trigger an exception, the eval loop resumes executing +* the sequential instructions. +* +* To make this work, the eval loop periodically checks if any +* of the above needs to happen. The individual checks can be +* expensive if computed each time, so a while back we switched +* to using pre-computed, per-interpreter variables for the checks, +* and later consolidated that to a single "eval breaker" variable +* (now a PyInterpreterState field). +* +* For the longest time, the eval breaker check would happen +* frequently, every 5 or so times through the loop, regardless +* of what instruction ran last or what would run next. Then, in +* early 2021 (gh-18334, commit 4958f5d), we switched to checking +* the eval breaker less frequently, by hard-coding the check to +* specific places in the eval loop (e.g. certain instructions). +* The intent then was to check after returning from calls +* and on the back edges of loops. +* +* In addition to being more efficient, that approach keeps +* the eval loop from running arbitrary code between instructions +* that don't handle that well. (See gh-74174.) +* +* Currently, the eval breaker check happens on back edges in +* the control flow graph, which pretty much applies to all loops, +* and most calls. +* (See bytecodes.c for exact information.) +* +* One consequence of this approach is that it might not be obvious +* how to force any specific thread to pick up the eval breaker, +* or for any specific thread to not pick it up. Mostly this +* involves judicious uses of locks and careful ordering of code, +* while avoiding code that might trigger the eval breaker +* until so desired. +*/ int _Py_HandlePending(PyThreadState *tstate) { diff --git a/Python/ceval_macros.h b/Python/ceval_macros.h index 0d41ef5a14cef..72800aaaaa2ac 100644 --- a/Python/ceval_macros.h +++ b/Python/ceval_macros.h @@ -117,7 +117,9 @@ #define CHECK_EVAL_BREAKER() \ _Py_CHECK_EMSCRIPTEN_SIGNALS_PERIODICALLY(); \ if (_Py_atomic_load_relaxed_int32(&tstate->interp->ceval.eval_breaker)) { \ - goto handle_eval_breaker; \ + if (_Py_HandlePending(tstate) != 0) { \ + goto error; \ + } \ } diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index d1e0443db613d..39a4490e51c24 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1629,7 +1629,7 @@ case GET_LEN: { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2304 "Python/bytecodes.c" + #line 2309 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; @@ -1646,7 +1646,7 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2312 "Python/bytecodes.c" + #line 2317 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); @@ -1655,7 +1655,7 @@ Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2317 "Python/bytecodes.c" + #line 2322 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -1672,7 +1672,7 @@ case MATCH_MAPPING: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2327 "Python/bytecodes.c" + #line 2332 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; #line 1679 "Python/executor_cases.c.h" @@ -1684,7 +1684,7 @@ case MATCH_SEQUENCE: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2332 "Python/bytecodes.c" + #line 2337 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; #line 1691 "Python/executor_cases.c.h" @@ -1697,7 +1697,7 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2337 "Python/bytecodes.c" + #line 2342 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; @@ -1710,12 +1710,12 @@ case GET_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2343 "Python/bytecodes.c" + #line 2348 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); #line 1717 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2346 "Python/bytecodes.c" + #line 2351 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; #line 1721 "Python/executor_cases.c.h" stack_pointer[-1] = iter; @@ -1725,7 +1725,7 @@ case GET_YIELD_FROM_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2350 "Python/bytecodes.c" + #line 2355 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -1750,7 +1750,7 @@ } #line 1752 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2373 "Python/bytecodes.c" + #line 2378 "Python/bytecodes.c" } #line 1756 "Python/executor_cases.c.h" stack_pointer[-1] = iter; @@ -1762,7 +1762,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2605 "Python/bytecodes.c" + #line 2610 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -1792,7 +1792,7 @@ case PUSH_EXC_INFO: { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2644 "Python/bytecodes.c" + #line 2649 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -1811,7 +1811,7 @@ case EXIT_INIT_CHECK: { PyObject *should_be_none = stack_pointer[-1]; - #line 3013 "Python/bytecodes.c" + #line 3018 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -1827,7 +1827,7 @@ case MAKE_FUNCTION: { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3427 "Python/bytecodes.c" + #line 3432 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -1847,7 +1847,7 @@ case SET_FUNCTION_ATTRIBUTE: { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3441 "Python/bytecodes.c" + #line 3446 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -1883,13 +1883,13 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3491 "Python/bytecodes.c" + #line 3496 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); #line 1889 "Python/executor_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3493 "Python/bytecodes.c" + #line 3498 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } #line 1895 "Python/executor_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); @@ -1901,7 +1901,7 @@ case CONVERT_VALUE: { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3497 "Python/bytecodes.c" + #line 3502 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; @@ -1916,7 +1916,7 @@ case FORMAT_SIMPLE: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3506 "Python/bytecodes.c" + #line 3511 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -1936,7 +1936,7 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3519 "Python/bytecodes.c" + #line 3524 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); @@ -1950,7 +1950,7 @@ case COPY: { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3526 "Python/bytecodes.c" + #line 3531 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); #line 1957 "Python/executor_cases.c.h" @@ -1962,7 +1962,7 @@ case SWAP: { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3551 "Python/bytecodes.c" + #line 3556 "Python/bytecodes.c" assert(oparg >= 2); #line 1968 "Python/executor_cases.c.h" stack_pointer[-1] = bottom; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 21cd2d0126ab3..eb2422943984b 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -17,8 +17,8 @@ if (err) goto error; next_instr--; } - else if (_Py_atomic_load_relaxed_int32(&tstate->interp->ceval.eval_breaker) && oparg < 2) { - goto handle_eval_breaker; + else if (oparg < 2) { + CHECK_EVAL_BREAKER(); } #line 24 "Python/generated_cases.c.h" DISPATCH(); @@ -37,6 +37,9 @@ next_instr--; } else { + if (oparg < 2) { + CHECK_EVAL_BREAKER(); + } _PyFrame_SetStackPointer(frame, stack_pointer); int err = _Py_call_instrumentation( tstate, oparg > 0, frame, next_instr-1); @@ -47,9 +50,6 @@ next_instr = frame->prev_instr; DISPATCH(); } - if (_Py_atomic_load_relaxed_int32(&tstate->interp->ceval.eval_breaker) && oparg < 2) { - goto handle_eval_breaker; - } } #line 55 "Python/generated_cases.c.h" DISPATCH(); @@ -3182,6 +3182,7 @@ TARGET(JUMP_BACKWARD) { #line 2226 "Python/bytecodes.c" + CHECK_EVAL_BREAKER(); _Py_CODEUNIT *here = next_instr - 1; assert(oparg <= INSTR_OFFSET()); JUMPBY(1-oparg); @@ -3199,15 +3200,19 @@ goto resume_frame; } #endif /* ENABLE_SPECIALIZATION */ - #line 3203 "Python/generated_cases.c.h" - CHECK_EVAL_BREAKER(); + #line 3204 "Python/generated_cases.c.h" DISPATCH(); } TARGET(ENTER_EXECUTOR) { #line 2257 "Python/bytecodes.c" + CHECK_EVAL_BREAKER(); + PyCodeObject *code = _PyFrame_GetCode(frame); _PyExecutorObject *executor = (_PyExecutorObject *)code->co_executors->executors[oparg&255]; + int original_oparg = executor->vm_data.oparg | (oparg & 0xfffff00); + JUMPBY(1-original_oparg); + frame->prev_instr = next_instr - 1; Py_INCREF(executor); frame = executor->execute(executor, frame, stack_pointer); if (frame == NULL) { @@ -3215,81 +3220,81 @@ goto resume_with_error; } goto resume_frame; - #line 3219 "Python/generated_cases.c.h" + #line 3224 "Python/generated_cases.c.h" } TARGET(POP_JUMP_IF_FALSE) { PyObject *cond = stack_pointer[-1]; - #line 2269 "Python/bytecodes.c" + #line 2274 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsFalse(cond)); - #line 3227 "Python/generated_cases.c.h" + #line 3232 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_TRUE) { PyObject *cond = stack_pointer[-1]; - #line 2274 "Python/bytecodes.c" + #line 2279 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsTrue(cond)); - #line 3237 "Python/generated_cases.c.h" + #line 3242 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_NOT_NONE) { PyObject *value = stack_pointer[-1]; - #line 2279 "Python/bytecodes.c" + #line 2284 "Python/bytecodes.c" if (!Py_IsNone(value)) { - #line 3246 "Python/generated_cases.c.h" + #line 3251 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2281 "Python/bytecodes.c" + #line 2286 "Python/bytecodes.c" JUMPBY(oparg); } - #line 3251 "Python/generated_cases.c.h" + #line 3256 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_NONE) { PyObject *value = stack_pointer[-1]; - #line 2286 "Python/bytecodes.c" + #line 2291 "Python/bytecodes.c" if (Py_IsNone(value)) { JUMPBY(oparg); } else { - #line 3263 "Python/generated_cases.c.h" + #line 3268 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2291 "Python/bytecodes.c" + #line 2296 "Python/bytecodes.c" } - #line 3267 "Python/generated_cases.c.h" + #line 3272 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(JUMP_BACKWARD_NO_INTERRUPT) { - #line 2295 "Python/bytecodes.c" + #line 2300 "Python/bytecodes.c" /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost * generator or coroutine, so we deliberately do not check it here. * (see bpo-30039). */ JUMPBY(-oparg); - #line 3280 "Python/generated_cases.c.h" + #line 3285 "Python/generated_cases.c.h" DISPATCH(); } TARGET(GET_LEN) { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2304 "Python/bytecodes.c" + #line 2309 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 3293 "Python/generated_cases.c.h" + #line 3298 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; DISPATCH(); @@ -3300,16 +3305,16 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2312 "Python/bytecodes.c" + #line 2317 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 3309 "Python/generated_cases.c.h" + #line 3314 "Python/generated_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2317 "Python/bytecodes.c" + #line 2322 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -3317,7 +3322,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 3321 "Python/generated_cases.c.h" + #line 3326 "Python/generated_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; DISPATCH(); @@ -3326,10 +3331,10 @@ TARGET(MATCH_MAPPING) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2327 "Python/bytecodes.c" + #line 2332 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 3333 "Python/generated_cases.c.h" + #line 3338 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3338,10 +3343,10 @@ TARGET(MATCH_SEQUENCE) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2332 "Python/bytecodes.c" + #line 2337 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 3345 "Python/generated_cases.c.h" + #line 3350 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3351,11 +3356,11 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2337 "Python/bytecodes.c" + #line 2342 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 3359 "Python/generated_cases.c.h" + #line 3364 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; DISPATCH(); @@ -3364,14 +3369,14 @@ TARGET(GET_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2343 "Python/bytecodes.c" + #line 2348 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 3371 "Python/generated_cases.c.h" + #line 3376 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2346 "Python/bytecodes.c" + #line 2351 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 3375 "Python/generated_cases.c.h" + #line 3380 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3379,7 +3384,7 @@ TARGET(GET_YIELD_FROM_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2350 "Python/bytecodes.c" + #line 2355 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -3402,11 +3407,11 @@ if (iter == NULL) { goto error; } - #line 3406 "Python/generated_cases.c.h" + #line 3411 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2373 "Python/bytecodes.c" + #line 2378 "Python/bytecodes.c" } - #line 3410 "Python/generated_cases.c.h" + #line 3415 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3416,7 +3421,7 @@ static_assert(INLINE_CACHE_ENTRIES_FOR_ITER == 1, "incorrect cache size"); PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2391 "Python/bytecodes.c" + #line 2396 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -3448,7 +3453,7 @@ DISPATCH(); } // Common case: no jump, leave it to the code generator - #line 3452 "Python/generated_cases.c.h" + #line 3457 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3456,7 +3461,7 @@ } TARGET(INSTRUMENTED_FOR_ITER) { - #line 2425 "Python/bytecodes.c" + #line 2430 "Python/bytecodes.c" _Py_CODEUNIT *here = next_instr-1; _Py_CODEUNIT *target; PyObject *iter = TOP(); @@ -3482,14 +3487,14 @@ target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER + oparg + 1; } INSTRUMENTED_JUMP(here, target, PY_MONITORING_EVENT_BRANCH); - #line 3486 "Python/generated_cases.c.h" + #line 3491 "Python/generated_cases.c.h" DISPATCH(); } TARGET(FOR_ITER_LIST) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2453 "Python/bytecodes.c" + #line 2458 "Python/bytecodes.c" DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit); @@ -3510,7 +3515,7 @@ DISPATCH(); end_for_iter_list: // Common case: no jump, leave it to the code generator - #line 3514 "Python/generated_cases.c.h" + #line 3519 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3520,7 +3525,7 @@ TARGET(FOR_ITER_TUPLE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2476 "Python/bytecodes.c" + #line 2481 "Python/bytecodes.c" _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3541,7 +3546,7 @@ DISPATCH(); end_for_iter_tuple: // Common case: no jump, leave it to the code generator - #line 3545 "Python/generated_cases.c.h" + #line 3550 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3551,7 +3556,7 @@ TARGET(FOR_ITER_RANGE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2499 "Python/bytecodes.c" + #line 2504 "Python/bytecodes.c" _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3570,7 +3575,7 @@ if (next == NULL) { goto error; } - #line 3574 "Python/generated_cases.c.h" + #line 3579 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3579,7 +3584,7 @@ TARGET(FOR_ITER_GEN) { PyObject *iter = stack_pointer[-1]; - #line 2520 "Python/bytecodes.c" + #line 2525 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER); @@ -3595,14 +3600,14 @@ assert(next_instr[oparg].op.code == END_FOR || next_instr[oparg].op.code == INSTRUMENTED_END_FOR); DISPATCH_INLINED(gen_frame); - #line 3599 "Python/generated_cases.c.h" + #line 3604 "Python/generated_cases.c.h" } TARGET(BEFORE_ASYNC_WITH) { PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2538 "Python/bytecodes.c" + #line 2543 "Python/bytecodes.c" PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { @@ -3625,16 +3630,16 @@ Py_DECREF(enter); goto error; } - #line 3629 "Python/generated_cases.c.h" + #line 3634 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2561 "Python/bytecodes.c" + #line 2566 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3638 "Python/generated_cases.c.h" + #line 3643 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3645,7 +3650,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2570 "Python/bytecodes.c" + #line 2575 "Python/bytecodes.c" /* pop the context manager, push its __exit__ and the * value returned from calling its __enter__ */ @@ -3671,16 +3676,16 @@ Py_DECREF(enter); goto error; } - #line 3675 "Python/generated_cases.c.h" + #line 3680 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2596 "Python/bytecodes.c" + #line 2601 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3684 "Python/generated_cases.c.h" + #line 3689 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3692,7 +3697,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2605 "Python/bytecodes.c" + #line 2610 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -3713,7 +3718,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 3717 "Python/generated_cases.c.h" + #line 3722 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3722,7 +3727,7 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2644 "Python/bytecodes.c" + #line 2649 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -3732,7 +3737,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 3736 "Python/generated_cases.c.h" + #line 3741 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -3746,7 +3751,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2656 "Python/bytecodes.c" + #line 2661 "Python/bytecodes.c" /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3763,7 +3768,7 @@ assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; assert(oparg & 1); - #line 3767 "Python/generated_cases.c.h" + #line 3772 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3777,7 +3782,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2675 "Python/bytecodes.c" + #line 2680 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); assert(self_cls->tp_dictoffset == 0); @@ -3787,7 +3792,7 @@ res2 = Py_NewRef(descr); res = self; assert(oparg & 1); - #line 3791 "Python/generated_cases.c.h" + #line 3796 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3801,7 +3806,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2687 "Python/bytecodes.c" + #line 2692 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); Py_ssize_t dictoffset = self_cls->tp_dictoffset; @@ -3815,7 +3820,7 @@ res2 = Py_NewRef(descr); res = self; assert(oparg & 1); - #line 3819 "Python/generated_cases.c.h" + #line 3824 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3824,16 +3829,16 @@ } TARGET(KW_NAMES) { - #line 2703 "Python/bytecodes.c" + #line 2708 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(FRAME_CO_CONSTS)); kwnames = GETITEM(FRAME_CO_CONSTS, oparg); - #line 3832 "Python/generated_cases.c.h" + #line 3837 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_CALL) { - #line 2709 "Python/bytecodes.c" + #line 2714 "Python/bytecodes.c" int is_meth = PEEK(oparg+2) != NULL; int total_args = oparg + is_meth; PyObject *function = PEEK(total_args + 1); @@ -3846,7 +3851,7 @@ _PyCallCache *cache = (_PyCallCache *)next_instr; INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(CALL); - #line 3850 "Python/generated_cases.c.h" + #line 3855 "Python/generated_cases.c.h" } TARGET(CALL) { @@ -3856,7 +3861,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2755 "Python/bytecodes.c" + #line 2760 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3938,7 +3943,7 @@ Py_DECREF(args[i]); } if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3942 "Python/generated_cases.c.h" + #line 3947 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3950,7 +3955,7 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2843 "Python/bytecodes.c" + #line 2848 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit); @@ -3960,7 +3965,7 @@ PEEK(oparg + 2) = Py_NewRef(meth); // method Py_DECREF(callable); GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS); - #line 3964 "Python/generated_cases.c.h" + #line 3969 "Python/generated_cases.c.h" } TARGET(CALL_PY_EXACT_ARGS) { @@ -3969,7 +3974,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2855 "Python/bytecodes.c" + #line 2860 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -3995,7 +4000,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 3999 "Python/generated_cases.c.h" + #line 4004 "Python/generated_cases.c.h" } TARGET(CALL_PY_WITH_DEFAULTS) { @@ -4003,7 +4008,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2883 "Python/bytecodes.c" + #line 2888 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4039,7 +4044,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4043 "Python/generated_cases.c.h" + #line 4048 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_TYPE_1) { @@ -4047,7 +4052,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2921 "Python/bytecodes.c" + #line 2926 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4057,7 +4062,7 @@ res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(obj); Py_DECREF(&PyType_Type); // I.e., callable - #line 4061 "Python/generated_cases.c.h" + #line 4066 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4070,7 +4075,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2933 "Python/bytecodes.c" + #line 2938 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4081,7 +4086,7 @@ Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); // I.e., callable if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4085 "Python/generated_cases.c.h" + #line 4090 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4095,7 +4100,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2947 "Python/bytecodes.c" + #line 2952 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4106,7 +4111,7 @@ Py_DECREF(arg); Py_DECREF(&PyTuple_Type); // I.e., tuple if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4110 "Python/generated_cases.c.h" + #line 4115 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4119,7 +4124,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; - #line 2961 "Python/bytecodes.c" + #line 2966 "Python/bytecodes.c" /* This instruction does the following: * 1. Creates the object (by calling ``object.__new__``) * 2. Pushes a shim frame to the frame stack (to cleanup after ``__init__``) @@ -4169,12 +4174,12 @@ frame = cframe.current_frame = init_frame; CALL_STAT_INC(inlined_py_calls); goto start_frame; - #line 4173 "Python/generated_cases.c.h" + #line 4178 "Python/generated_cases.c.h" } TARGET(EXIT_INIT_CHECK) { PyObject *should_be_none = stack_pointer[-1]; - #line 3013 "Python/bytecodes.c" + #line 3018 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -4182,7 +4187,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 4186 "Python/generated_cases.c.h" + #line 4191 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -4192,7 +4197,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3023 "Python/bytecodes.c" + #line 3028 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4214,7 +4219,7 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4218 "Python/generated_cases.c.h" + #line 4223 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4228,7 +4233,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3048 "Python/bytecodes.c" + #line 3053 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4256,7 +4261,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4260 "Python/generated_cases.c.h" + #line 4265 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4270,7 +4275,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3079 "Python/bytecodes.c" + #line 3084 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4302,7 +4307,7 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 4306 "Python/generated_cases.c.h" + #line 4311 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4316,7 +4321,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3114 "Python/bytecodes.c" + #line 3119 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -4348,7 +4353,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4352 "Python/generated_cases.c.h" + #line 4357 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4362,7 +4367,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3149 "Python/bytecodes.c" + #line 3154 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -4387,7 +4392,7 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4391 "Python/generated_cases.c.h" + #line 4396 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4400,7 +4405,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3176 "Python/bytecodes.c" + #line 3181 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -4427,7 +4432,7 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4431 "Python/generated_cases.c.h" + #line 4436 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4439,7 +4444,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 3206 "Python/bytecodes.c" + #line 3211 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -4457,14 +4462,14 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 4461 "Python/generated_cases.c.h" + #line 4466 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3226 "Python/bytecodes.c" + #line 3231 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4495,7 +4500,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4499 "Python/generated_cases.c.h" + #line 4504 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4508,7 +4513,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3260 "Python/bytecodes.c" + #line 3265 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4537,7 +4542,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4541 "Python/generated_cases.c.h" + #line 4546 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4550,7 +4555,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3292 "Python/bytecodes.c" + #line 3297 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -4579,7 +4584,7 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4583 "Python/generated_cases.c.h" + #line 4588 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4592,7 +4597,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3324 "Python/bytecodes.c" + #line 3329 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4620,7 +4625,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4624 "Python/generated_cases.c.h" + #line 4629 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4630,9 +4635,9 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { - #line 3355 "Python/bytecodes.c" + #line 3360 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); - #line 4636 "Python/generated_cases.c.h" + #line 4641 "Python/generated_cases.c.h" } TARGET(CALL_FUNCTION_EX) { @@ -4641,7 +4646,7 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 3359 "Python/bytecodes.c" + #line 3364 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs)); @@ -4703,14 +4708,14 @@ } result = PyObject_Call(func, callargs, kwargs); } - #line 4707 "Python/generated_cases.c.h" + #line 4712 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 3421 "Python/bytecodes.c" + #line 3426 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4714 "Python/generated_cases.c.h" + #line 4719 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4721,7 +4726,7 @@ TARGET(MAKE_FUNCTION) { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3427 "Python/bytecodes.c" + #line 3432 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4733,7 +4738,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4737 "Python/generated_cases.c.h" + #line 4742 "Python/generated_cases.c.h" stack_pointer[-1] = func; DISPATCH(); } @@ -4741,7 +4746,7 @@ TARGET(SET_FUNCTION_ATTRIBUTE) { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3441 "Python/bytecodes.c" + #line 3446 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -4766,14 +4771,14 @@ default: Py_UNREACHABLE(); } - #line 4770 "Python/generated_cases.c.h" + #line 4775 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 3468 "Python/bytecodes.c" + #line 3473 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4794,7 +4799,7 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4798 "Python/generated_cases.c.h" + #line 4803 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4802,15 +4807,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3491 "Python/bytecodes.c" + #line 3496 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4808 "Python/generated_cases.c.h" + #line 4813 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3493 "Python/bytecodes.c" + #line 3498 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4814 "Python/generated_cases.c.h" + #line 4819 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4820,14 +4825,14 @@ TARGET(CONVERT_VALUE) { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3497 "Python/bytecodes.c" + #line 3502 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 4831 "Python/generated_cases.c.h" + #line 4836 "Python/generated_cases.c.h" stack_pointer[-1] = result; DISPATCH(); } @@ -4835,7 +4840,7 @@ TARGET(FORMAT_SIMPLE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3506 "Python/bytecodes.c" + #line 3511 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -4846,7 +4851,7 @@ else { res = value; } - #line 4850 "Python/generated_cases.c.h" + #line 4855 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -4855,12 +4860,12 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3519 "Python/bytecodes.c" + #line 3524 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 4864 "Python/generated_cases.c.h" + #line 4869 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -4869,10 +4874,10 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3526 "Python/bytecodes.c" + #line 3531 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4876 "Python/generated_cases.c.h" + #line 4881 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4884,7 +4889,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3531 "Python/bytecodes.c" + #line 3536 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -4899,12 +4904,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4903 "Python/generated_cases.c.h" + #line 4908 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3546 "Python/bytecodes.c" + #line 3551 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 4908 "Python/generated_cases.c.h" + #line 4913 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -4914,16 +4919,16 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3551 "Python/bytecodes.c" + #line 3556 "Python/bytecodes.c" assert(oparg >= 2); - #line 4920 "Python/generated_cases.c.h" + #line 4925 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } TARGET(INSTRUMENTED_INSTRUCTION) { - #line 3555 "Python/bytecodes.c" + #line 3560 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error; @@ -4935,48 +4940,48 @@ assert(next_opcode > 0 && next_opcode < 256); opcode = next_opcode; DISPATCH_GOTO(); - #line 4939 "Python/generated_cases.c.h" + #line 4944 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_JUMP_FORWARD) { - #line 3569 "Python/bytecodes.c" + #line 3574 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); - #line 4945 "Python/generated_cases.c.h" + #line 4950 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) { - #line 3573 "Python/bytecodes.c" - INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); - #line 4952 "Python/generated_cases.c.h" + #line 3578 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); + INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); + #line 4958 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { - #line 3578 "Python/bytecodes.c" + #line 3583 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsTrue(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4964 "Python/generated_cases.c.h" + #line 4969 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { - #line 3586 "Python/bytecodes.c" + #line 3591 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsFalse(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4975 "Python/generated_cases.c.h" + #line 4980 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { - #line 3594 "Python/bytecodes.c" + #line 3599 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -4988,12 +4993,12 @@ offset = 0; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4992 "Python/generated_cases.c.h" + #line 4997 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { - #line 3608 "Python/bytecodes.c" + #line 3613 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -5005,30 +5010,30 @@ offset = oparg; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5009 "Python/generated_cases.c.h" + #line 5014 "Python/generated_cases.c.h" DISPATCH(); } TARGET(EXTENDED_ARG) { - #line 3622 "Python/bytecodes.c" + #line 3627 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 5020 "Python/generated_cases.c.h" + #line 5025 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3630 "Python/bytecodes.c" + #line 3635 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); - #line 5027 "Python/generated_cases.c.h" + #line 5032 "Python/generated_cases.c.h" } TARGET(RESERVED) { - #line 3635 "Python/bytecodes.c" + #line 3640 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 5034 "Python/generated_cases.c.h" + #line 5039 "Python/generated_cases.c.h" } diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index ac3d800d8fe0f..82c9823589228 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -1087,7 +1087,7 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { [JUMP_BACKWARD] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [JUMP] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [JUMP_NO_INTERRUPT] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, - [ENTER_EXECUTOR] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, + [ENTER_EXECUTOR] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [POP_JUMP_IF_FALSE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [POP_JUMP_IF_TRUE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [POP_JUMP_IF_NOT_NONE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, From webhook-mailer at python.org Mon Jul 3 16:29:48 2023 From: webhook-mailer at python.org (barneygale) Date: Mon, 03 Jul 2023 20:29:48 -0000 Subject: [Python-checkins] GH-106330: Fix matching of empty path in `pathlib.PurePath.match()` (GH-106331) Message-ID: https://github.com/python/cpython/commit/b4efdf8cda8fbbd0ca53b457d5f6e46a59348caf commit: b4efdf8cda8fbbd0ca53b457d5f6e46a59348caf branch: main author: Barney Gale committer: barneygale date: 2023-07-03T21:29:44+01:00 summary: GH-106330: Fix matching of empty path in `pathlib.PurePath.match()` (GH-106331) We match paths using the `_lines` attribute, which is derived from the path's string representation. The bug arises because an empty path's string representation is `'.'` (not `''`), which is matched by the `'*'` wildcard. files: A Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Lib/pathlib.py b/Lib/pathlib.py index e15718dc98d67..f3813e0410990 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -463,8 +463,12 @@ def _lines(self): try: return self._lines_cached except AttributeError: - trans = _SWAP_SEP_AND_NEWLINE[self._flavour.sep] - self._lines_cached = str(self).translate(trans) + path_str = str(self) + if path_str == '.': + self._lines_cached = '' + else: + trans = _SWAP_SEP_AND_NEWLINE[self._flavour.sep] + self._lines_cached = path_str.translate(trans) return self._lines_cached def __eq__(self, other): diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index 464a835212d47..eb2b0cfb26e85 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -384,6 +384,10 @@ def test_match_common(self): self.assertTrue(P('A.py').match('a.PY', case_sensitive=False)) self.assertFalse(P('c:/a/B.Py').match('C:/A/*.pY', case_sensitive=True)) self.assertTrue(P('/a/b/c.py').match('/A/*/*.Py', case_sensitive=False)) + # Matching against empty path + self.assertFalse(P().match('*')) + self.assertTrue(P().match('**')) + self.assertFalse(P().match('**/*')) def test_ordering_common(self): # Ordering is tuple-alike. diff --git a/Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst b/Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst new file mode 100644 index 0000000000000..c1f55ab658b51 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst @@ -0,0 +1,2 @@ +Fix incorrect matching of empty paths in :meth:`pathlib.PurePath.match`. +This bug was introduced in Python 3.12.0 beta 1. From webhook-mailer at python.org Mon Jul 3 16:38:41 2023 From: webhook-mailer at python.org (rhettinger) Date: Mon, 03 Jul 2023 20:38:41 -0000 Subject: [Python-checkins] Small speed-up for the convolve() recipe. (GH-106371) Message-ID: https://github.com/python/cpython/commit/77090370952307730ea71d68b848cce0dc8cbd83 commit: 77090370952307730ea71d68b848cce0dc8cbd83 branch: main author: Raymond Hettinger committer: rhettinger date: 2023-07-03T15:38:38-05:00 summary: Small speed-up for the convolve() recipe. (GH-106371) files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 56d6599798af2..a2d1798a2c6da 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -1085,8 +1085,8 @@ The following recipes have a more mathematical flavor: kernel = tuple(kernel)[::-1] n = len(kernel) padded_signal = chain(repeat(0, n-1), signal, repeat(0, n-1)) - for window in sliding_window(padded_signal, n): - yield math.sumprod(kernel, window) + windowed_signal = sliding_window(padded_signal, n) + return map(math.sumprod, repeat(kernel), windowed_signal) def polynomial_from_roots(roots): """Compute a polynomial's coefficients from its roots. From webhook-mailer at python.org Mon Jul 3 16:45:53 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 20:45:53 -0000 Subject: [Python-checkins] [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106369) (#106370) Message-ID: https://github.com/python/cpython/commit/ddff4737bd1215b151d94aa27559721022ab2a47 commit: ddff4737bd1215b151d94aa27559721022ab2a47 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-03T20:45:49Z summary: [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106369) (#106370) Add tests for 'self' and 'defining_class' converter requirements. (cherry picked from commit 7f4c8121db62a9f72f00f2d9f73381e82f289581) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 76a06df5fe2e5..30e8dcc3e8ef1 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -829,6 +829,63 @@ def test_kwarg_splats_disallowed_in_function_call_annotations(self): out = self.parse_function_should_fail(fn) self.assertEqual(out, expected_error_msg) + def test_self_param_placement(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'self' parameter, if specified, must be the very first thing " + "in the parameter block.\n" + ) + block = """ + module foo + foo.func + a: int + self: self(type="PyObject *") + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + + def test_self_param_cannot_be_optional(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'self' parameter cannot be marked optional.\n" + ) + block = """ + module foo + foo.func + self: self(type="PyObject *") = None + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + + def test_defining_class_param_placement(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'defining_class' parameter, if specified, must either be the " + "first thing in the parameter block, or come just after 'self'.\n" + ) + block = """ + module foo + foo.func + self: self(type="PyObject *") + a: int + cls: defining_class + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + + def test_defining_class_param_cannot_be_optional(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'defining_class' parameter cannot be marked optional.\n" + ) + block = """ + module foo + foo.func + cls: defining_class(type="PyObject *") = None + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + def test_unused_param(self): block = self.parse(""" module foo From webhook-mailer at python.org Mon Jul 3 16:49:13 2023 From: webhook-mailer at python.org (rhettinger) Date: Mon, 03 Jul 2023 20:49:13 -0000 Subject: [Python-checkins] [3.12] Small speed-up for the convolve() recipe. (GH-106371) (GH-106375) Message-ID: https://github.com/python/cpython/commit/67127ca8e2e9c94619503390267467fb3ee216e1 commit: 67127ca8e2e9c94619503390267467fb3ee216e1 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: rhettinger date: 2023-07-03T15:49:09-05:00 summary: [3.12] Small speed-up for the convolve() recipe. (GH-106371) (GH-106375) files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 56d6599798af2..a2d1798a2c6da 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -1085,8 +1085,8 @@ The following recipes have a more mathematical flavor: kernel = tuple(kernel)[::-1] n = len(kernel) padded_signal = chain(repeat(0, n-1), signal, repeat(0, n-1)) - for window in sliding_window(padded_signal, n): - yield math.sumprod(kernel, window) + windowed_signal = sliding_window(padded_signal, n) + return map(math.sumprod, repeat(kernel), windowed_signal) def polynomial_from_roots(roots): """Compute a polynomial's coefficients from its roots. From webhook-mailer at python.org Mon Jul 3 17:16:25 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 21:16:25 -0000 Subject: [Python-checkins] gh-104683: Modernise Argument Clinic parameter state machine (#106362) Message-ID: https://github.com/python/cpython/commit/71b40443fed6acb58330ee262f8d674b394f41d3 commit: 71b40443fed6acb58330ee262f8d674b394f41d3 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T21:16:21Z summary: gh-104683: Modernise Argument Clinic parameter state machine (#106362) Use enums and pattern matching to make the code more readable. Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 41d4274fc744e..6380b9ce38f5f 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4294,6 +4294,37 @@ def dedent(self, line): StateKeeper = Callable[[str | None], None] ConverterArgs = dict[str, Any] +class ParamState(enum.IntEnum): + """Parameter parsing state. + + [ [ a, b, ] c, ] d, e, f=3, [ g, h, [ i ] ] <- line + 01 2 3 4 5 6 <- state transitions + """ + # Before we've seen anything. + # Legal transitions: to LEFT_SQUARE_BEFORE or REQUIRED + START = 0 + + # Left square backets before required params. + LEFT_SQUARE_BEFORE = 1 + + # In a group, before required params. + GROUP_BEFORE = 2 + + # Required params, positional-or-keyword or positional-only (we + # don't know yet). Renumber left groups! + REQUIRED = 3 + + # Positional-or-keyword or positional-only params that now must have + # default values. + OPTIONAL = 4 + + # In a group, after required params. + GROUP_AFTER = 5 + + # Right square brackets after required params. + RIGHT_SQUARE_AFTER = 6 + + class DSLParser: function: Function | None state: StateKeeper @@ -4331,7 +4362,7 @@ def reset(self) -> None: self.keyword_only = False self.positional_only = False self.group = 0 - self.parameter_state = self.ps_start + self.parameter_state: ParamState = ParamState.START self.seen_positional_with_default = False self.indent = IndentStack() self.kind = CALLABLE @@ -4726,22 +4757,8 @@ def state_modulename_name(self, line: str | None) -> None: # # These rules are enforced with a single state variable: # "parameter_state". (Previously the code was a miasma of ifs and - # separate boolean state variables.) The states are: - # - # [ [ a, b, ] c, ] d, e, f=3, [ g, h, [ i ] ] <- line - # 01 2 3 4 5 6 <- state transitions - # - # 0: ps_start. before we've seen anything. legal transitions are to 1 or 3. - # 1: ps_left_square_before. left square brackets before required parameters. - # 2: ps_group_before. in a group, before required parameters. - # 3: ps_required. required parameters, positional-or-keyword or positional-only - # (we don't know yet). (renumber left groups!) - # 4: ps_optional. positional-or-keyword or positional-only parameters that - # now must have default values. - # 5: ps_group_after. in a group, after required parameters. - # 6: ps_right_square_after. right square brackets after required parameters. - ps_start, ps_left_square_before, ps_group_before, ps_required, \ - ps_optional, ps_group_after, ps_right_square_after = range(7) + # separate boolean state variables.) The states are defined in the + # ParamState class. def state_parameters_start(self, line: str | None) -> None: if not self.valid_line(line): @@ -4759,8 +4776,8 @@ def to_required(self): """ Transition to the "required" parameter state. """ - if self.parameter_state != self.ps_required: - self.parameter_state = self.ps_required + if self.parameter_state is not ParamState.REQUIRED: + self.parameter_state = ParamState.REQUIRED for p in self.function.parameters.values(): p.group = -p.group @@ -4793,17 +4810,18 @@ def state_parameter(self, line): self.parse_special_symbol(line) return - if self.parameter_state in (self.ps_start, self.ps_required): - self.to_required() - elif self.parameter_state == self.ps_left_square_before: - self.parameter_state = self.ps_group_before - elif self.parameter_state == self.ps_group_before: - if not self.group: + match self.parameter_state: + case ParamState.START | ParamState.REQUIRED: self.to_required() - elif self.parameter_state in (self.ps_group_after, self.ps_optional): - pass - else: - fail("Function " + self.function.name + " has an unsupported group configuration. (Unexpected state " + str(self.parameter_state) + ".a)") + case ParamState.LEFT_SQUARE_BEFORE: + self.parameter_state = ParamState.GROUP_BEFORE + case ParamState.GROUP_BEFORE: + if not self.group: + self.to_required() + case ParamState.GROUP_AFTER | ParamState.OPTIONAL: + pass + case st: + fail(f"Function {self.function.name} has an unsupported group configuration. (Unexpected state {st}.a)") # handle "as" for parameters too c_name = None @@ -4863,8 +4881,9 @@ def state_parameter(self, line): name, legacy, kwargs = self.parse_converter(parameter.annotation) if not default: - if self.parameter_state == self.ps_optional: - fail("Can't have a parameter without a default (" + repr(parameter_name) + ")\nafter a parameter with a default!") + if self.parameter_state is ParamState.OPTIONAL: + fail(f"Can't have a parameter without a default ({parameter_name!r})\n" + "after a parameter with a default!") if is_vararg: value = NULL kwargs.setdefault('c_default', "NULL") @@ -4876,8 +4895,8 @@ def state_parameter(self, line): if is_vararg: fail("Vararg can't take a default value!") - if self.parameter_state == self.ps_required: - self.parameter_state = self.ps_optional + if self.parameter_state is ParamState.REQUIRED: + self.parameter_state = ParamState.OPTIONAL default = default.strip() bad = False ast_input = f"x = {default}" @@ -5001,14 +5020,14 @@ def bad_node(self, node): if isinstance(converter, self_converter): if len(self.function.parameters) == 1: - if (self.parameter_state != self.ps_required): + if self.parameter_state is not ParamState.REQUIRED: fail("A 'self' parameter cannot be marked optional.") if value is not unspecified: fail("A 'self' parameter cannot have a default value.") if self.group: fail("A 'self' parameter cannot be in an optional group.") kind = inspect.Parameter.POSITIONAL_ONLY - self.parameter_state = self.ps_start + self.parameter_state = ParamState.START self.function.parameters.clear() else: fail("A 'self' parameter, if specified, must be the very first thing in the parameter block.") @@ -5016,7 +5035,7 @@ def bad_node(self, node): if isinstance(converter, defining_class_converter): _lp = len(self.function.parameters) if _lp == 1: - if (self.parameter_state != self.ps_required): + if self.parameter_state is not ParamState.REQUIRED: fail("A 'defining_class' parameter cannot be marked optional.") if value is not unspecified: fail("A 'defining_class' parameter cannot have a default value.") @@ -5065,12 +5084,13 @@ def parse_special_symbol(self, symbol): fail("Function " + self.function.name + " uses '*' more than once.") self.keyword_only = True elif symbol == '[': - if self.parameter_state in (self.ps_start, self.ps_left_square_before): - self.parameter_state = self.ps_left_square_before - elif self.parameter_state in (self.ps_required, self.ps_group_after): - self.parameter_state = self.ps_group_after - else: - fail("Function " + self.function.name + " has an unsupported group configuration. (Unexpected state " + str(self.parameter_state) + ".b)") + match self.parameter_state: + case ParamState.START | ParamState.LEFT_SQUARE_BEFORE: + self.parameter_state = ParamState.LEFT_SQUARE_BEFORE + case ParamState.REQUIRED | ParamState.GROUP_AFTER: + self.parameter_state = ParamState.GROUP_AFTER + case st: + fail(f"Function {self.function.name} has an unsupported group configuration. (Unexpected state {st}.b)") self.group += 1 self.function.docstring_only = True elif symbol == ']': @@ -5079,20 +5099,27 @@ def parse_special_symbol(self, symbol): if not any(p.group == self.group for p in self.function.parameters.values()): fail("Function " + self.function.name + " has an empty group.\nAll groups must contain at least one parameter.") self.group -= 1 - if self.parameter_state in (self.ps_left_square_before, self.ps_group_before): - self.parameter_state = self.ps_group_before - elif self.parameter_state in (self.ps_group_after, self.ps_right_square_after): - self.parameter_state = self.ps_right_square_after - else: - fail("Function " + self.function.name + " has an unsupported group configuration. (Unexpected state " + str(self.parameter_state) + ".c)") + match self.parameter_state: + case ParamState.LEFT_SQUARE_BEFORE | ParamState.GROUP_BEFORE: + self.parameter_state = ParamState.GROUP_BEFORE + case ParamState.GROUP_AFTER | ParamState.RIGHT_SQUARE_AFTER: + self.parameter_state = ParamState.RIGHT_SQUARE_AFTER + case st: + fail(f"Function {self.function.name} has an unsupported group configuration. (Unexpected state {st}.c)") elif symbol == '/': if self.positional_only: fail("Function " + self.function.name + " uses '/' more than once.") self.positional_only = True - # ps_required and ps_optional are allowed here, that allows positional-only without option groups + # REQUIRED and OPTIONAL are allowed here, that allows positional-only without option groups # to work (and have default values!) - if (self.parameter_state not in (self.ps_required, self.ps_optional, self.ps_right_square_after, self.ps_group_before)) or self.group: - fail("Function " + self.function.name + " has an unsupported group configuration. (Unexpected state " + str(self.parameter_state) + ".d)") + allowed = { + ParamState.REQUIRED, + ParamState.OPTIONAL, + ParamState.RIGHT_SQUARE_AFTER, + ParamState.GROUP_BEFORE, + } + if (self.parameter_state not in allowed) or self.group: + fail(f"Function {self.function.name} has an unsupported group configuration. (Unexpected state {self.parameter_state}.d)") if self.keyword_only: fail("Function " + self.function.name + " mixes keyword-only and positional-only parameters, which is unsupported.") # fixup preceding parameters From webhook-mailer at python.org Mon Jul 3 17:57:23 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 21:57:23 -0000 Subject: [Python-checkins] gh-106368: Clean up Argument Clinic tests (#106373) Message-ID: https://github.com/python/cpython/commit/3ee8dac7a1b3882aa3aac7703bdae2de7b6402ad commit: 3ee8dac7a1b3882aa3aac7703bdae2de7b6402ad branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T21:57:20Z summary: gh-106368: Clean up Argument Clinic tests (#106373) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 51d2ac972752f..b3602887ab635 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -4,6 +4,7 @@ from test import support, test_tools from test.support import os_helper +from textwrap import dedent from unittest import TestCase import collections import inspect @@ -171,43 +172,43 @@ def test_solo_newline(self): def test_no_substitution(self): self._test(""" abc - """, """ + """, """ abc - """) + """) def test_empty_substitution(self): self._test(""" abc {name} def - """, """ + """, """ abc def - """, name='') + """, name='') def test_single_line_substitution(self): self._test(""" abc {name} def - """, """ + """, """ abc GARGLE def - """, name='GARGLE') + """, name='GARGLE') def test_multiline_substitution(self): self._test(""" abc {name} def - """, """ + """, """ abc bingle bungle def - """, name='bingle\nbungle\n') + """, name='bingle\nbungle\n') class InertParser: def __init__(self, clinic): @@ -240,9 +241,9 @@ def round_trip(self, input): def test_round_trip_1(self): self.round_trip(""" - verbatim text here - lah dee dah -""") + verbatim text here + lah dee dah + """) def test_round_trip_2(self): self.round_trip(""" verbatim text here @@ -286,22 +287,38 @@ def test_clinic_1(self): class ClinicParserTest(TestCase): + def checkDocstring(self, fn, expected): + self.assertTrue(hasattr(fn, "docstring")) + self.assertEqual(fn.docstring.strip(), + dedent(expected).strip()) + def test_trivial(self): parser = DSLParser(FakeClinic()) - block = clinic.Block("module os\nos.access") + block = clinic.Block(""" + module os + os.access + """) parser.parse(block) module, function = block.signatures self.assertEqual("access", function.name) self.assertEqual("os", module.name) def test_ignore_line(self): - block = self.parse("#\nmodule os\nos.access") + block = self.parse(dedent(""" + # + module os + os.access + """)) module, function = block.signatures self.assertEqual("access", function.name) self.assertEqual("os", module.name) def test_param(self): - function = self.parse_function("module os\nos.access\n path: int") + function = self.parse_function(""" + module os + os.access + path: int + """) self.assertEqual("access", function.name) self.assertEqual(2, len(function.parameters)) p = function.parameters['path'] @@ -309,236 +326,296 @@ def test_param(self): self.assertIsInstance(p.converter, clinic.int_converter) def test_param_default(self): - function = self.parse_function("module os\nos.access\n follow_symlinks: bool = True") + function = self.parse_function(""" + module os + os.access + follow_symlinks: bool = True + """) p = function.parameters['follow_symlinks'] self.assertEqual(True, p.default) def test_param_with_continuations(self): - function = self.parse_function("module os\nos.access\n follow_symlinks: \\\n bool \\\n =\\\n True") + function = self.parse_function(r""" + module os + os.access + follow_symlinks: \ + bool \ + = \ + True + """) p = function.parameters['follow_symlinks'] self.assertEqual(True, p.default) def test_param_default_expression(self): - function = self.parse_function("module os\nos.access\n follow_symlinks: int(c_default='MAXSIZE') = sys.maxsize") + function = self.parse_function(""" + module os + os.access + follow_symlinks: int(c_default='MAXSIZE') = sys.maxsize + """) p = function.parameters['follow_symlinks'] self.assertEqual(sys.maxsize, p.default) self.assertEqual("MAXSIZE", p.converter.c_default) - s = self.parse_function_should_fail("module os\nos.access\n follow_symlinks: int = sys.maxsize") - self.assertEqual(s, "Error on line 0:\nWhen you specify a named constant ('sys.maxsize') as your default value,\nyou MUST specify a valid c_default.\n") + expected_msg = ( + "Error on line 0:\n" + "When you specify a named constant ('sys.maxsize') as your default value,\n" + "you MUST specify a valid c_default.\n" + ) + out = self.parse_function_should_fail(""" + module os + os.access + follow_symlinks: int = sys.maxsize + """) + self.assertEqual(out, expected_msg) def test_param_no_docstring(self): function = self.parse_function(""" -module os -os.access - follow_symlinks: bool = True - something_else: str = ''""") + module os + os.access + follow_symlinks: bool = True + something_else: str = '' + """) p = function.parameters['follow_symlinks'] self.assertEqual(3, len(function.parameters)) - self.assertIsInstance(function.parameters['something_else'].converter, clinic.str_converter) + conv = function.parameters['something_else'].converter + self.assertIsInstance(conv, clinic.str_converter) def test_param_default_parameters_out_of_order(self): - s = self.parse_function_should_fail(""" -module os -os.access - follow_symlinks: bool = True - something_else: str""") - self.assertEqual(s, """Error on line 0: -Can't have a parameter without a default ('something_else') -after a parameter with a default! -""") + expected_msg = ( + "Error on line 0:\n" + "Can't have a parameter without a default ('something_else')\n" + "after a parameter with a default!\n" + ) + out = self.parse_function_should_fail(""" + module os + os.access + follow_symlinks: bool = True + something_else: str""") + self.assertEqual(out, expected_msg) def disabled_test_converter_arguments(self): - function = self.parse_function("module os\nos.access\n path: path_t(allow_fd=1)") + function = self.parse_function(""" + module os + os.access + path: path_t(allow_fd=1) + """) p = function.parameters['path'] self.assertEqual(1, p.converter.args['allow_fd']) def test_function_docstring(self): function = self.parse_function(""" -module os -os.stat as os_stat_fn + module os + os.stat as os_stat_fn - path: str - Path to be examined + path: str + Path to be examined -Perform a stat system call on the given path.""") - self.assertEqual(""" -stat($module, /, path) --- + Perform a stat system call on the given path. + """) + self.checkDocstring(function, """ + stat($module, /, path) + -- -Perform a stat system call on the given path. + Perform a stat system call on the given path. - path - Path to be examined -""".strip(), function.docstring) + path + Path to be examined + """) def test_explicit_parameters_in_docstring(self): - function = self.parse_function(""" -module foo -foo.bar - x: int - Documentation for x. - y: int + function = self.parse_function(dedent(""" + module foo + foo.bar + x: int + Documentation for x. + y: int -This is the documentation for foo. + This is the documentation for foo. -Okay, we're done here. -""") - self.assertEqual(""" -bar($module, /, x, y) --- + Okay, we're done here. + """)) + self.checkDocstring(function, """ + bar($module, /, x, y) + -- -This is the documentation for foo. + This is the documentation for foo. - x - Documentation for x. + x + Documentation for x. -Okay, we're done here. -""".strip(), function.docstring) + Okay, we're done here. + """) def test_parser_regression_special_character_in_parameter_column_of_docstring_first_line(self): - function = self.parse_function(""" -module os -os.stat - path: str -This/used to break Clinic! -""") - self.assertEqual("stat($module, /, path)\n--\n\nThis/used to break Clinic!", function.docstring) + function = self.parse_function(dedent(""" + module os + os.stat + path: str + This/used to break Clinic! + """)) + self.checkDocstring(function, """ + stat($module, /, path) + -- + + This/used to break Clinic! + """) def test_c_name(self): - function = self.parse_function("module os\nos.stat as os_stat_fn") + function = self.parse_function(""" + module os + os.stat as os_stat_fn + """) self.assertEqual("os_stat_fn", function.c_basename) def test_return_converter(self): - function = self.parse_function("module os\nos.stat -> int") + function = self.parse_function(""" + module os + os.stat -> int + """) self.assertIsInstance(function.return_converter, clinic.int_return_converter) def test_star(self): - function = self.parse_function("module os\nos.access\n *\n follow_symlinks: bool = True") + function = self.parse_function(""" + module os + os.access + * + follow_symlinks: bool = True + """) p = function.parameters['follow_symlinks'] self.assertEqual(inspect.Parameter.KEYWORD_ONLY, p.kind) self.assertEqual(0, p.group) def test_group(self): - function = self.parse_function("module window\nwindow.border\n [\n ls : int\n ]\n /\n") + function = self.parse_function(""" + module window + window.border + [ + ls: int + ] + / + """) p = function.parameters['ls'] self.assertEqual(1, p.group) def test_left_group(self): function = self.parse_function(""" -module curses -curses.addch - [ - y: int - Y-coordinate. - x: int - X-coordinate. - ] - ch: char - Character to add. - [ - attr: long - Attributes for the character. - ] - / -""") - for name, group in ( + module curses + curses.addch + [ + y: int + Y-coordinate. + x: int + X-coordinate. + ] + ch: char + Character to add. + [ + attr: long + Attributes for the character. + ] + / + """) + dataset = ( ('y', -1), ('x', -1), ('ch', 0), ('attr', 1), - ): - p = function.parameters[name] - self.assertEqual(p.group, group) - self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) - self.assertEqual(function.docstring.strip(), """ -addch([y, x,] ch, [attr]) - - - y - Y-coordinate. - x - X-coordinate. - ch - Character to add. - attr - Attributes for the character. - """.strip()) + ) + for name, group in dataset: + with self.subTest(name=name, group=group): + p = function.parameters[name] + self.assertEqual(p.group, group) + self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) + self.checkDocstring(function, """ + addch([y, x,] ch, [attr]) + + + y + Y-coordinate. + x + X-coordinate. + ch + Character to add. + attr + Attributes for the character. + """) def test_nested_groups(self): function = self.parse_function(""" -module curses -curses.imaginary - [ - [ - y1: int - Y-coordinate. - y2: int - Y-coordinate. - ] - x1: int - X-coordinate. - x2: int - X-coordinate. - ] - ch: char - Character to add. - [ - attr1: long - Attributes for the character. - attr2: long - Attributes for the character. - attr3: long - Attributes for the character. - [ - attr4: long - Attributes for the character. - attr5: long - Attributes for the character. - attr6: long - Attributes for the character. - ] - ] - / -""") - for name, group in ( + module curses + curses.imaginary + [ + [ + y1: int + Y-coordinate. + y2: int + Y-coordinate. + ] + x1: int + X-coordinate. + x2: int + X-coordinate. + ] + ch: char + Character to add. + [ + attr1: long + Attributes for the character. + attr2: long + Attributes for the character. + attr3: long + Attributes for the character. + [ + attr4: long + Attributes for the character. + attr5: long + Attributes for the character. + attr6: long + Attributes for the character. + ] + ] + / + """) + dataset = ( ('y1', -2), ('y2', -2), ('x1', -1), ('x2', -1), ('ch', 0), ('attr1', 1), ('attr2', 1), ('attr3', 1), ('attr4', 2), ('attr5', 2), ('attr6', 2), - ): - p = function.parameters[name] - self.assertEqual(p.group, group) - self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) - - self.assertEqual(function.docstring.strip(), """ -imaginary([[y1, y2,] x1, x2,] ch, [attr1, attr2, attr3, [attr4, attr5, - attr6]]) - - - y1 - Y-coordinate. - y2 - Y-coordinate. - x1 - X-coordinate. - x2 - X-coordinate. - ch - Character to add. - attr1 - Attributes for the character. - attr2 - Attributes for the character. - attr3 - Attributes for the character. - attr4 - Attributes for the character. - attr5 - Attributes for the character. - attr6 - Attributes for the character. - """.strip()) + ) + for name, group in dataset: + with self.subTest(name=name, group=group): + p = function.parameters[name] + self.assertEqual(p.group, group) + self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) + + self.checkDocstring(function, """ + imaginary([[y1, y2,] x1, x2,] ch, [attr1, attr2, attr3, [attr4, attr5, + attr6]]) + + + y1 + Y-coordinate. + y2 + Y-coordinate. + x1 + X-coordinate. + x2 + X-coordinate. + ch + Character to add. + attr1 + Attributes for the character. + attr2 + Attributes for the character. + attr3 + Attributes for the character. + attr4 + Attributes for the character. + attr5 + Attributes for the character. + attr6 + Attributes for the character. + """) def parse_function_should_fail(self, s): with support.captured_stdout() as stdout: @@ -547,104 +624,108 @@ def parse_function_should_fail(self, s): return stdout.getvalue() def test_disallowed_grouping__two_top_groups_on_left(self): - s = self.parse_function_should_fail(""" -module foo -foo.two_top_groups_on_left - [ - group1 : int - ] - [ - group2 : int - ] - param: int - """) - self.assertEqual(s, - ('Error on line 0:\n' - 'Function two_top_groups_on_left has an unsupported group configuration. (Unexpected state 2.b)\n')) + expected_msg = ( + 'Error on line 0:\n' + 'Function two_top_groups_on_left has an unsupported group ' + 'configuration. (Unexpected state 2.b)\n' + ) + out = self.parse_function_should_fail(""" + module foo + foo.two_top_groups_on_left + [ + group1 : int + ] + [ + group2 : int + ] + param: int + """) + self.assertEqual(out, expected_msg) def test_disallowed_grouping__two_top_groups_on_right(self): self.parse_function_should_fail(""" -module foo -foo.two_top_groups_on_right - param: int - [ - group1 : int - ] - [ - group2 : int - ] - """) + module foo + foo.two_top_groups_on_right + param: int + [ + group1 : int + ] + [ + group2 : int + ] + """) def test_disallowed_grouping__parameter_after_group_on_right(self): self.parse_function_should_fail(""" -module foo -foo.parameter_after_group_on_right - param: int - [ - [ - group1 : int - ] - group2 : int - ] - """) + module foo + foo.parameter_after_group_on_right + param: int + [ + [ + group1 : int + ] + group2 : int + ] + """) def test_disallowed_grouping__group_after_parameter_on_left(self): self.parse_function_should_fail(""" -module foo -foo.group_after_parameter_on_left - [ - group2 : int - [ - group1 : int - ] - ] - param: int - """) + module foo + foo.group_after_parameter_on_left + [ + group2 : int + [ + group1 : int + ] + ] + param: int + """) def test_disallowed_grouping__empty_group_on_left(self): self.parse_function_should_fail(""" -module foo -foo.empty_group - [ - [ - ] - group2 : int - ] - param: int - """) + module foo + foo.empty_group + [ + [ + ] + group2 : int + ] + param: int + """) def test_disallowed_grouping__empty_group_on_right(self): self.parse_function_should_fail(""" -module foo -foo.empty_group - param: int - [ - [ - ] - group2 : int - ] - """) + module foo + foo.empty_group + param: int + [ + [ + ] + group2 : int + ] + """) def test_no_parameters(self): function = self.parse_function(""" -module foo -foo.bar + module foo + foo.bar -Docstring + Docstring -""") + """) self.assertEqual("bar($module, /)\n--\n\nDocstring", function.docstring) self.assertEqual(1, len(function.parameters)) # self! def test_init_with_no_parameters(self): function = self.parse_function(""" -module foo -class foo.Bar "unused" "notneeded" -foo.Bar.__init__ + module foo + class foo.Bar "unused" "notneeded" + foo.Bar.__init__ -Docstring + Docstring + + """, signatures_in_block=3, function_index=2) -""", signatures_in_block=3, function_index=2) # self is not in the signature self.assertEqual("Bar()\n--\n\nDocstring", function.docstring) # but it *is* a parameter @@ -652,113 +733,117 @@ class foo.Bar "unused" "notneeded" def test_illegal_module_line(self): self.parse_function_should_fail(""" -module foo -foo.bar => int - / -""") + module foo + foo.bar => int + / + """) def test_illegal_c_basename(self): self.parse_function_should_fail(""" -module foo -foo.bar as 935 - / -""") + module foo + foo.bar as 935 + / + """) def test_single_star(self): self.parse_function_should_fail(""" -module foo -foo.bar - * - * -""") + module foo + foo.bar + * + * + """) def test_parameters_required_after_star_without_initial_parameters_or_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - * -""") + module foo + foo.bar + * + """) def test_parameters_required_after_star_without_initial_parameters_with_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - * -Docstring here. -""") + module foo + foo.bar + * + Docstring here. + """) def test_parameters_required_after_star_with_initial_parameters_without_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - this: int - * -""") + module foo + foo.bar + this: int + * + """) def test_parameters_required_after_star_with_initial_parameters_and_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - this: int - * -Docstring. -""") + module foo + foo.bar + this: int + * + Docstring. + """) def test_single_slash(self): self.parse_function_should_fail(""" -module foo -foo.bar - / - / -""") + module foo + foo.bar + / + / + """) def test_mix_star_and_slash(self): self.parse_function_should_fail(""" -module foo -foo.bar - x: int - y: int - * - z: int - / -""") + module foo + foo.bar + x: int + y: int + * + z: int + / + """) def test_parameters_not_permitted_after_slash_for_now(self): self.parse_function_should_fail(""" -module foo -foo.bar - / - x: int -""") + module foo + foo.bar + / + x: int + """) def test_parameters_no_more_than_one_vararg(self): - s = self.parse_function_should_fail(""" -module foo -foo.bar - *vararg1: object - *vararg2: object -""") - self.assertEqual(s, "Error on line 0:\nToo many var args\n") + expected_msg = ( + "Error on line 0:\n" + "Too many var args\n" + ) + out = self.parse_function_should_fail(""" + module foo + foo.bar + *vararg1: object + *vararg2: object + """) + self.assertEqual(out, expected_msg) def test_function_not_at_column_0(self): function = self.parse_function(""" - module foo - foo.bar - x: int - Nested docstring here, goeth. - * - y: str - Not at column 0! -""") - self.assertEqual(""" -bar($module, /, x, *, y) --- + module foo + foo.bar + x: int + Nested docstring here, goeth. + * + y: str + Not at column 0! + """) + self.checkDocstring(function, """ + bar($module, /, x, *, y) + -- -Not at column 0! + Not at column 0! - x - Nested docstring here, goeth. -""".strip(), function.docstring) + x + Nested docstring here, goeth. + """) def test_directive(self): c = FakeClinic() @@ -772,46 +857,39 @@ def test_directive(self): def test_legacy_converters(self): block = self.parse('module os\nos.access\n path: "s"') module, function = block.signatures - self.assertIsInstance((function.parameters['path']).converter, clinic.str_converter) + conv = (function.parameters['path']).converter + self.assertIsInstance(conv, clinic.str_converter) def test_legacy_converters_non_string_constant_annotation(self): - expected_failure_message = """\ -Error on line 0: -Annotations must be either a name, a function call, or a string. -""" - - s = self.parse_function_should_fail('module os\nos.access\n path: 42') - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail('module os\nos.access\n path: 42.42') - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail('module os\nos.access\n path: 42j') - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail('module os\nos.access\n path: b"42"') - self.assertEqual(s, expected_failure_message) - - def test_other_bizarre_things_in_annotations_fail(self): - expected_failure_message = """\ -Error on line 0: -Annotations must be either a name, a function call, or a string. -""" - - s = self.parse_function_should_fail( - 'module os\nos.access\n path: {"some": "dictionary"}' + expected_failure_message = ( + "Error on line 0:\n" + "Annotations must be either a name, a function call, or a string.\n" ) - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail( - 'module os\nos.access\n path: ["list", "of", "strings"]' + dataset = ( + 'module os\nos.access\n path: 42', + 'module os\nos.access\n path: 42.42', + 'module os\nos.access\n path: 42j', + 'module os\nos.access\n path: b"42"', ) - self.assertEqual(s, expected_failure_message) + for block in dataset: + with self.subTest(block=block): + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_failure_message) - s = self.parse_function_should_fail( - 'module os\nos.access\n path: (x for x in range(42))' + def test_other_bizarre_things_in_annotations_fail(self): + expected_failure_message = ( + "Error on line 0:\n" + "Annotations must be either a name, a function call, or a string.\n" + ) + dataset = ( + 'module os\nos.access\n path: {"some": "dictionary"}', + 'module os\nos.access\n path: ["list", "of", "strings"]', + 'module os\nos.access\n path: (x for x in range(42))', ) - self.assertEqual(s, expected_failure_message) + for block in dataset: + with self.subTest(block=block): + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_failure_message) def test_kwarg_splats_disallowed_in_function_call_annotations(self): expected_error_msg = ( @@ -945,10 +1023,16 @@ def test_scaffolding(self): self.assertEqual(repr(clinic.NULL), '') # test that fail fails + expected = ( + 'Error in file "clown.txt" on line 69:\n' + 'The igloos are melting!\n' + ) with support.captured_stdout() as stdout: with self.assertRaises(SystemExit): - clinic.fail('The igloos are melting!', filename='clown.txt', line_number=69) - self.assertEqual(stdout.getvalue(), 'Error in file "clown.txt" on line 69:\nThe igloos are melting!\n') + clinic.fail('The igloos are melting!', + filename='clown.txt', line_number=69) + actual = stdout.getvalue() + self.assertEqual(actual, expected) class ClinicExternalTest(TestCase): From webhook-mailer at python.org Mon Jul 3 18:07:09 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 22:07:09 -0000 Subject: [Python-checkins] [3.11] gh-106368: Increase Argument Clinic test coverage (#106369) (#106374) Message-ID: https://github.com/python/cpython/commit/52d5049cfd9f688d40f81194e3ccf7abbe724209 commit: 52d5049cfd9f688d40f81194e3ccf7abbe724209 branch: 3.11 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T22:07:05Z summary: [3.11] gh-106368: Increase Argument Clinic test coverage (#106369) (#106374) Add tests for 'self' and 'defining_class' converter requirements. (cherry picked from commit 7f4c8121db62a9f72f00f2d9f73381e82f289581) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 8d4f92fa310c3..c2d2085ecd6de 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -812,6 +812,63 @@ def test_other_bizarre_things_in_annotations_fail(self): ) self.assertEqual(s, expected_failure_message) + def test_self_param_placement(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'self' parameter, if specified, must be the very first thing " + "in the parameter block.\n" + ) + block = """ + module foo + foo.func + a: int + self: self(type="PyObject *") + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + + def test_self_param_cannot_be_optional(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'self' parameter cannot be marked optional.\n" + ) + block = """ + module foo + foo.func + self: self(type="PyObject *") = None + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + + def test_defining_class_param_placement(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'defining_class' parameter, if specified, must either be the " + "first thing in the parameter block, or come just after 'self'.\n" + ) + block = """ + module foo + foo.func + self: self(type="PyObject *") + a: int + cls: defining_class + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + + def test_defining_class_param_cannot_be_optional(self): + expected_error_msg = ( + "Error on line 0:\n" + "A 'defining_class' parameter cannot be marked optional.\n" + ) + block = """ + module foo + foo.func + cls: defining_class(type="PyObject *") = None + """ + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_error_msg) + def parse(self, text): c = FakeClinic() parser = DSLParser(c) From webhook-mailer at python.org Mon Jul 3 18:10:50 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 22:10:50 -0000 Subject: [Python-checkins] gh-104050: Annotate more Argument Clinic DSLParser state methods (#106376) Message-ID: https://github.com/python/cpython/commit/b24479dcba6e8952039066564d448d5ac4b37bef commit: b24479dcba6e8952039066564d448d5ac4b37bef branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T22:10:46Z summary: gh-104050: Annotate more Argument Clinic DSLParser state methods (#106376) Annotate the following methods: - state_parameter() - state_parameter_docstring_start() Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 6380b9ce38f5f..a07fcbd8cabf7 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4781,14 +4781,16 @@ def to_required(self): for p in self.function.parameters.values(): p.group = -p.group - def state_parameter(self, line): - if self.parameter_continuation: - line = self.parameter_continuation + ' ' + line.lstrip() - self.parameter_continuation = '' + def state_parameter(self, line: str | None) -> None: + assert isinstance(self.function, Function) if not self.valid_line(line): return + if self.parameter_continuation: + line = self.parameter_continuation + ' ' + line.lstrip() + self.parameter_continuation = '' + assert self.indent.depth == 2 indent = self.indent.infer(line) if indent == -1: @@ -4839,6 +4841,7 @@ def state_parameter(self, line): fields[0] = name line = ' '.join(fields) + default: str | None base, equals, default = line.rpartition('=') if not equals: base = default @@ -4861,7 +4864,9 @@ def state_parameter(self, line): if not module: fail("Function " + self.function.name + " has an invalid parameter declaration:\n\t" + line) - function_args = module.body[0].args + function = module.body[0] + assert isinstance(function, ast.FunctionDef) + function_args = function.args if len(function_args.args) > 1: fail("Function " + self.function.name + " has an invalid parameter declaration (comma?):\n\t" + line) @@ -4884,6 +4889,7 @@ def state_parameter(self, line): if self.parameter_state is ParamState.OPTIONAL: fail(f"Can't have a parameter without a default ({parameter_name!r})\n" "after a parameter with a default!") + value: Sentinels | Null if is_vararg: value = NULL kwargs.setdefault('c_default', "NULL") @@ -4946,8 +4952,11 @@ def bad_node(self, node): if bad: fail("Unsupported expression as default value: " + repr(default)) - expr = module.body[0].value + assignment = module.body[0] + assert isinstance(assignment, ast.Assign) + expr = assignment.value # mild hack: explicitly support NULL as a default value + c_default: str | None if isinstance(expr, ast.Name) and expr.id == 'NULL': value = NULL py_default = '' @@ -4964,7 +4973,7 @@ def bad_node(self, node): value = unknown elif isinstance(expr, ast.Attribute): a = [] - n = expr + n: ast.expr | ast.Attribute = expr while isinstance(n, ast.Attribute): a.append(n.attr) n = n.value @@ -4984,7 +4993,7 @@ def bad_node(self, node): else: value = ast.literal_eval(expr) py_default = repr(value) - if isinstance(value, (bool, None.__class__)): + if isinstance(value, (bool, NoneType)): c_default = "Py_" + py_default elif isinstance(value, str): c_default = c_repr(value) @@ -5011,6 +5020,7 @@ def bad_node(self, node): # but the parameter object gets the python name converter = dict[name](c_name or parameter_name, parameter_name, self.function, value, **kwargs) + kind: inspect._ParameterKind if is_vararg: kind = inspect.Parameter.VAR_POSITIONAL elif self.keyword_only: @@ -5130,7 +5140,7 @@ def parse_special_symbol(self, symbol): fail("Function " + self.function.name + " mixes keyword-only and positional-only parameters, which is unsupported.") p.kind = inspect.Parameter.POSITIONAL_ONLY - def state_parameter_docstring_start(self, line: str) -> None: + def state_parameter_docstring_start(self, line: str | None) -> None: self.parameter_docstring_indent = len(self.indent.margin) assert self.indent.depth == 3 return self.next(self.state_parameter_docstring, line) From webhook-mailer at python.org Mon Jul 3 18:27:37 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 22:27:37 -0000 Subject: [Python-checkins] [3.12] gh-106368: Clean up Argument Clinic tests (#106373) (#106379) Message-ID: https://github.com/python/cpython/commit/38fe0e7c2d0db1bba75bebc7dd5e890a93aa11b0 commit: 38fe0e7c2d0db1bba75bebc7dd5e890a93aa11b0 branch: 3.12 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T22:27:34Z summary: [3.12] gh-106368: Clean up Argument Clinic tests (#106373) (#106379) (cherry picked from commit 3ee8dac7a1b3882aa3aac7703bdae2de7b6402ad) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 30e8dcc3e8ef1..b3602887ab635 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -3,7 +3,8 @@ # Licensed to the PSF under a contributor agreement. from test import support, test_tools -from test.support import import_helper, os_helper +from test.support import os_helper +from textwrap import dedent from unittest import TestCase import collections import inspect @@ -171,43 +172,43 @@ def test_solo_newline(self): def test_no_substitution(self): self._test(""" abc - """, """ + """, """ abc - """) + """) def test_empty_substitution(self): self._test(""" abc {name} def - """, """ + """, """ abc def - """, name='') + """, name='') def test_single_line_substitution(self): self._test(""" abc {name} def - """, """ + """, """ abc GARGLE def - """, name='GARGLE') + """, name='GARGLE') def test_multiline_substitution(self): self._test(""" abc {name} def - """, """ + """, """ abc bingle bungle def - """, name='bingle\nbungle\n') + """, name='bingle\nbungle\n') class InertParser: def __init__(self, clinic): @@ -240,9 +241,9 @@ def round_trip(self, input): def test_round_trip_1(self): self.round_trip(""" - verbatim text here - lah dee dah -""") + verbatim text here + lah dee dah + """) def test_round_trip_2(self): self.round_trip(""" verbatim text here @@ -286,22 +287,38 @@ def test_clinic_1(self): class ClinicParserTest(TestCase): + def checkDocstring(self, fn, expected): + self.assertTrue(hasattr(fn, "docstring")) + self.assertEqual(fn.docstring.strip(), + dedent(expected).strip()) + def test_trivial(self): parser = DSLParser(FakeClinic()) - block = clinic.Block("module os\nos.access") + block = clinic.Block(""" + module os + os.access + """) parser.parse(block) module, function = block.signatures self.assertEqual("access", function.name) self.assertEqual("os", module.name) def test_ignore_line(self): - block = self.parse("#\nmodule os\nos.access") + block = self.parse(dedent(""" + # + module os + os.access + """)) module, function = block.signatures self.assertEqual("access", function.name) self.assertEqual("os", module.name) def test_param(self): - function = self.parse_function("module os\nos.access\n path: int") + function = self.parse_function(""" + module os + os.access + path: int + """) self.assertEqual("access", function.name) self.assertEqual(2, len(function.parameters)) p = function.parameters['path'] @@ -309,236 +326,296 @@ def test_param(self): self.assertIsInstance(p.converter, clinic.int_converter) def test_param_default(self): - function = self.parse_function("module os\nos.access\n follow_symlinks: bool = True") + function = self.parse_function(""" + module os + os.access + follow_symlinks: bool = True + """) p = function.parameters['follow_symlinks'] self.assertEqual(True, p.default) def test_param_with_continuations(self): - function = self.parse_function("module os\nos.access\n follow_symlinks: \\\n bool \\\n =\\\n True") + function = self.parse_function(r""" + module os + os.access + follow_symlinks: \ + bool \ + = \ + True + """) p = function.parameters['follow_symlinks'] self.assertEqual(True, p.default) def test_param_default_expression(self): - function = self.parse_function("module os\nos.access\n follow_symlinks: int(c_default='MAXSIZE') = sys.maxsize") + function = self.parse_function(""" + module os + os.access + follow_symlinks: int(c_default='MAXSIZE') = sys.maxsize + """) p = function.parameters['follow_symlinks'] self.assertEqual(sys.maxsize, p.default) self.assertEqual("MAXSIZE", p.converter.c_default) - s = self.parse_function_should_fail("module os\nos.access\n follow_symlinks: int = sys.maxsize") - self.assertEqual(s, "Error on line 0:\nWhen you specify a named constant ('sys.maxsize') as your default value,\nyou MUST specify a valid c_default.\n") + expected_msg = ( + "Error on line 0:\n" + "When you specify a named constant ('sys.maxsize') as your default value,\n" + "you MUST specify a valid c_default.\n" + ) + out = self.parse_function_should_fail(""" + module os + os.access + follow_symlinks: int = sys.maxsize + """) + self.assertEqual(out, expected_msg) def test_param_no_docstring(self): function = self.parse_function(""" -module os -os.access - follow_symlinks: bool = True - something_else: str = ''""") + module os + os.access + follow_symlinks: bool = True + something_else: str = '' + """) p = function.parameters['follow_symlinks'] self.assertEqual(3, len(function.parameters)) - self.assertIsInstance(function.parameters['something_else'].converter, clinic.str_converter) + conv = function.parameters['something_else'].converter + self.assertIsInstance(conv, clinic.str_converter) def test_param_default_parameters_out_of_order(self): - s = self.parse_function_should_fail(""" -module os -os.access - follow_symlinks: bool = True - something_else: str""") - self.assertEqual(s, """Error on line 0: -Can't have a parameter without a default ('something_else') -after a parameter with a default! -""") + expected_msg = ( + "Error on line 0:\n" + "Can't have a parameter without a default ('something_else')\n" + "after a parameter with a default!\n" + ) + out = self.parse_function_should_fail(""" + module os + os.access + follow_symlinks: bool = True + something_else: str""") + self.assertEqual(out, expected_msg) def disabled_test_converter_arguments(self): - function = self.parse_function("module os\nos.access\n path: path_t(allow_fd=1)") + function = self.parse_function(""" + module os + os.access + path: path_t(allow_fd=1) + """) p = function.parameters['path'] self.assertEqual(1, p.converter.args['allow_fd']) def test_function_docstring(self): function = self.parse_function(""" -module os -os.stat as os_stat_fn + module os + os.stat as os_stat_fn - path: str - Path to be examined + path: str + Path to be examined -Perform a stat system call on the given path.""") - self.assertEqual(""" -stat($module, /, path) --- + Perform a stat system call on the given path. + """) + self.checkDocstring(function, """ + stat($module, /, path) + -- -Perform a stat system call on the given path. + Perform a stat system call on the given path. - path - Path to be examined -""".strip(), function.docstring) + path + Path to be examined + """) def test_explicit_parameters_in_docstring(self): - function = self.parse_function(""" -module foo -foo.bar - x: int - Documentation for x. - y: int + function = self.parse_function(dedent(""" + module foo + foo.bar + x: int + Documentation for x. + y: int -This is the documentation for foo. + This is the documentation for foo. -Okay, we're done here. -""") - self.assertEqual(""" -bar($module, /, x, y) --- + Okay, we're done here. + """)) + self.checkDocstring(function, """ + bar($module, /, x, y) + -- -This is the documentation for foo. + This is the documentation for foo. - x - Documentation for x. + x + Documentation for x. -Okay, we're done here. -""".strip(), function.docstring) + Okay, we're done here. + """) def test_parser_regression_special_character_in_parameter_column_of_docstring_first_line(self): - function = self.parse_function(""" -module os -os.stat - path: str -This/used to break Clinic! -""") - self.assertEqual("stat($module, /, path)\n--\n\nThis/used to break Clinic!", function.docstring) + function = self.parse_function(dedent(""" + module os + os.stat + path: str + This/used to break Clinic! + """)) + self.checkDocstring(function, """ + stat($module, /, path) + -- + + This/used to break Clinic! + """) def test_c_name(self): - function = self.parse_function("module os\nos.stat as os_stat_fn") + function = self.parse_function(""" + module os + os.stat as os_stat_fn + """) self.assertEqual("os_stat_fn", function.c_basename) def test_return_converter(self): - function = self.parse_function("module os\nos.stat -> int") + function = self.parse_function(""" + module os + os.stat -> int + """) self.assertIsInstance(function.return_converter, clinic.int_return_converter) def test_star(self): - function = self.parse_function("module os\nos.access\n *\n follow_symlinks: bool = True") + function = self.parse_function(""" + module os + os.access + * + follow_symlinks: bool = True + """) p = function.parameters['follow_symlinks'] self.assertEqual(inspect.Parameter.KEYWORD_ONLY, p.kind) self.assertEqual(0, p.group) def test_group(self): - function = self.parse_function("module window\nwindow.border\n [\n ls : int\n ]\n /\n") + function = self.parse_function(""" + module window + window.border + [ + ls: int + ] + / + """) p = function.parameters['ls'] self.assertEqual(1, p.group) def test_left_group(self): function = self.parse_function(""" -module curses -curses.addch - [ - y: int - Y-coordinate. - x: int - X-coordinate. - ] - ch: char - Character to add. - [ - attr: long - Attributes for the character. - ] - / -""") - for name, group in ( + module curses + curses.addch + [ + y: int + Y-coordinate. + x: int + X-coordinate. + ] + ch: char + Character to add. + [ + attr: long + Attributes for the character. + ] + / + """) + dataset = ( ('y', -1), ('x', -1), ('ch', 0), ('attr', 1), - ): - p = function.parameters[name] - self.assertEqual(p.group, group) - self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) - self.assertEqual(function.docstring.strip(), """ -addch([y, x,] ch, [attr]) - - - y - Y-coordinate. - x - X-coordinate. - ch - Character to add. - attr - Attributes for the character. - """.strip()) + ) + for name, group in dataset: + with self.subTest(name=name, group=group): + p = function.parameters[name] + self.assertEqual(p.group, group) + self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) + self.checkDocstring(function, """ + addch([y, x,] ch, [attr]) + + + y + Y-coordinate. + x + X-coordinate. + ch + Character to add. + attr + Attributes for the character. + """) def test_nested_groups(self): function = self.parse_function(""" -module curses -curses.imaginary - [ - [ - y1: int - Y-coordinate. - y2: int - Y-coordinate. - ] - x1: int - X-coordinate. - x2: int - X-coordinate. - ] - ch: char - Character to add. - [ - attr1: long - Attributes for the character. - attr2: long - Attributes for the character. - attr3: long - Attributes for the character. - [ - attr4: long - Attributes for the character. - attr5: long - Attributes for the character. - attr6: long - Attributes for the character. - ] - ] - / -""") - for name, group in ( + module curses + curses.imaginary + [ + [ + y1: int + Y-coordinate. + y2: int + Y-coordinate. + ] + x1: int + X-coordinate. + x2: int + X-coordinate. + ] + ch: char + Character to add. + [ + attr1: long + Attributes for the character. + attr2: long + Attributes for the character. + attr3: long + Attributes for the character. + [ + attr4: long + Attributes for the character. + attr5: long + Attributes for the character. + attr6: long + Attributes for the character. + ] + ] + / + """) + dataset = ( ('y1', -2), ('y2', -2), ('x1', -1), ('x2', -1), ('ch', 0), ('attr1', 1), ('attr2', 1), ('attr3', 1), ('attr4', 2), ('attr5', 2), ('attr6', 2), - ): - p = function.parameters[name] - self.assertEqual(p.group, group) - self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) - - self.assertEqual(function.docstring.strip(), """ -imaginary([[y1, y2,] x1, x2,] ch, [attr1, attr2, attr3, [attr4, attr5, - attr6]]) - - - y1 - Y-coordinate. - y2 - Y-coordinate. - x1 - X-coordinate. - x2 - X-coordinate. - ch - Character to add. - attr1 - Attributes for the character. - attr2 - Attributes for the character. - attr3 - Attributes for the character. - attr4 - Attributes for the character. - attr5 - Attributes for the character. - attr6 - Attributes for the character. - """.strip()) + ) + for name, group in dataset: + with self.subTest(name=name, group=group): + p = function.parameters[name] + self.assertEqual(p.group, group) + self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) + + self.checkDocstring(function, """ + imaginary([[y1, y2,] x1, x2,] ch, [attr1, attr2, attr3, [attr4, attr5, + attr6]]) + + + y1 + Y-coordinate. + y2 + Y-coordinate. + x1 + X-coordinate. + x2 + X-coordinate. + ch + Character to add. + attr1 + Attributes for the character. + attr2 + Attributes for the character. + attr3 + Attributes for the character. + attr4 + Attributes for the character. + attr5 + Attributes for the character. + attr6 + Attributes for the character. + """) def parse_function_should_fail(self, s): with support.captured_stdout() as stdout: @@ -547,104 +624,108 @@ def parse_function_should_fail(self, s): return stdout.getvalue() def test_disallowed_grouping__two_top_groups_on_left(self): - s = self.parse_function_should_fail(""" -module foo -foo.two_top_groups_on_left - [ - group1 : int - ] - [ - group2 : int - ] - param: int - """) - self.assertEqual(s, - ('Error on line 0:\n' - 'Function two_top_groups_on_left has an unsupported group configuration. (Unexpected state 2.b)\n')) + expected_msg = ( + 'Error on line 0:\n' + 'Function two_top_groups_on_left has an unsupported group ' + 'configuration. (Unexpected state 2.b)\n' + ) + out = self.parse_function_should_fail(""" + module foo + foo.two_top_groups_on_left + [ + group1 : int + ] + [ + group2 : int + ] + param: int + """) + self.assertEqual(out, expected_msg) def test_disallowed_grouping__two_top_groups_on_right(self): self.parse_function_should_fail(""" -module foo -foo.two_top_groups_on_right - param: int - [ - group1 : int - ] - [ - group2 : int - ] - """) + module foo + foo.two_top_groups_on_right + param: int + [ + group1 : int + ] + [ + group2 : int + ] + """) def test_disallowed_grouping__parameter_after_group_on_right(self): self.parse_function_should_fail(""" -module foo -foo.parameter_after_group_on_right - param: int - [ - [ - group1 : int - ] - group2 : int - ] - """) + module foo + foo.parameter_after_group_on_right + param: int + [ + [ + group1 : int + ] + group2 : int + ] + """) def test_disallowed_grouping__group_after_parameter_on_left(self): self.parse_function_should_fail(""" -module foo -foo.group_after_parameter_on_left - [ - group2 : int - [ - group1 : int - ] - ] - param: int - """) + module foo + foo.group_after_parameter_on_left + [ + group2 : int + [ + group1 : int + ] + ] + param: int + """) def test_disallowed_grouping__empty_group_on_left(self): self.parse_function_should_fail(""" -module foo -foo.empty_group - [ - [ - ] - group2 : int - ] - param: int - """) + module foo + foo.empty_group + [ + [ + ] + group2 : int + ] + param: int + """) def test_disallowed_grouping__empty_group_on_right(self): self.parse_function_should_fail(""" -module foo -foo.empty_group - param: int - [ - [ - ] - group2 : int - ] - """) + module foo + foo.empty_group + param: int + [ + [ + ] + group2 : int + ] + """) def test_no_parameters(self): function = self.parse_function(""" -module foo -foo.bar + module foo + foo.bar -Docstring + Docstring -""") + """) self.assertEqual("bar($module, /)\n--\n\nDocstring", function.docstring) self.assertEqual(1, len(function.parameters)) # self! def test_init_with_no_parameters(self): function = self.parse_function(""" -module foo -class foo.Bar "unused" "notneeded" -foo.Bar.__init__ + module foo + class foo.Bar "unused" "notneeded" + foo.Bar.__init__ -Docstring + Docstring + + """, signatures_in_block=3, function_index=2) -""", signatures_in_block=3, function_index=2) # self is not in the signature self.assertEqual("Bar()\n--\n\nDocstring", function.docstring) # but it *is* a parameter @@ -652,113 +733,117 @@ class foo.Bar "unused" "notneeded" def test_illegal_module_line(self): self.parse_function_should_fail(""" -module foo -foo.bar => int - / -""") + module foo + foo.bar => int + / + """) def test_illegal_c_basename(self): self.parse_function_should_fail(""" -module foo -foo.bar as 935 - / -""") + module foo + foo.bar as 935 + / + """) def test_single_star(self): self.parse_function_should_fail(""" -module foo -foo.bar - * - * -""") + module foo + foo.bar + * + * + """) def test_parameters_required_after_star_without_initial_parameters_or_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - * -""") + module foo + foo.bar + * + """) def test_parameters_required_after_star_without_initial_parameters_with_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - * -Docstring here. -""") + module foo + foo.bar + * + Docstring here. + """) def test_parameters_required_after_star_with_initial_parameters_without_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - this: int - * -""") + module foo + foo.bar + this: int + * + """) def test_parameters_required_after_star_with_initial_parameters_and_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - this: int - * -Docstring. -""") + module foo + foo.bar + this: int + * + Docstring. + """) def test_single_slash(self): self.parse_function_should_fail(""" -module foo -foo.bar - / - / -""") + module foo + foo.bar + / + / + """) def test_mix_star_and_slash(self): self.parse_function_should_fail(""" -module foo -foo.bar - x: int - y: int - * - z: int - / -""") + module foo + foo.bar + x: int + y: int + * + z: int + / + """) def test_parameters_not_permitted_after_slash_for_now(self): self.parse_function_should_fail(""" -module foo -foo.bar - / - x: int -""") + module foo + foo.bar + / + x: int + """) def test_parameters_no_more_than_one_vararg(self): - s = self.parse_function_should_fail(""" -module foo -foo.bar - *vararg1: object - *vararg2: object -""") - self.assertEqual(s, "Error on line 0:\nToo many var args\n") + expected_msg = ( + "Error on line 0:\n" + "Too many var args\n" + ) + out = self.parse_function_should_fail(""" + module foo + foo.bar + *vararg1: object + *vararg2: object + """) + self.assertEqual(out, expected_msg) def test_function_not_at_column_0(self): function = self.parse_function(""" - module foo - foo.bar - x: int - Nested docstring here, goeth. - * - y: str - Not at column 0! -""") - self.assertEqual(""" -bar($module, /, x, *, y) --- + module foo + foo.bar + x: int + Nested docstring here, goeth. + * + y: str + Not at column 0! + """) + self.checkDocstring(function, """ + bar($module, /, x, *, y) + -- -Not at column 0! + Not at column 0! - x - Nested docstring here, goeth. -""".strip(), function.docstring) + x + Nested docstring here, goeth. + """) def test_directive(self): c = FakeClinic() @@ -772,46 +857,39 @@ def test_directive(self): def test_legacy_converters(self): block = self.parse('module os\nos.access\n path: "s"') module, function = block.signatures - self.assertIsInstance((function.parameters['path']).converter, clinic.str_converter) + conv = (function.parameters['path']).converter + self.assertIsInstance(conv, clinic.str_converter) def test_legacy_converters_non_string_constant_annotation(self): - expected_failure_message = """\ -Error on line 0: -Annotations must be either a name, a function call, or a string. -""" - - s = self.parse_function_should_fail('module os\nos.access\n path: 42') - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail('module os\nos.access\n path: 42.42') - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail('module os\nos.access\n path: 42j') - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail('module os\nos.access\n path: b"42"') - self.assertEqual(s, expected_failure_message) - - def test_other_bizarre_things_in_annotations_fail(self): - expected_failure_message = """\ -Error on line 0: -Annotations must be either a name, a function call, or a string. -""" - - s = self.parse_function_should_fail( - 'module os\nos.access\n path: {"some": "dictionary"}' + expected_failure_message = ( + "Error on line 0:\n" + "Annotations must be either a name, a function call, or a string.\n" ) - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail( - 'module os\nos.access\n path: ["list", "of", "strings"]' + dataset = ( + 'module os\nos.access\n path: 42', + 'module os\nos.access\n path: 42.42', + 'module os\nos.access\n path: 42j', + 'module os\nos.access\n path: b"42"', ) - self.assertEqual(s, expected_failure_message) + for block in dataset: + with self.subTest(block=block): + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_failure_message) - s = self.parse_function_should_fail( - 'module os\nos.access\n path: (x for x in range(42))' + def test_other_bizarre_things_in_annotations_fail(self): + expected_failure_message = ( + "Error on line 0:\n" + "Annotations must be either a name, a function call, or a string.\n" + ) + dataset = ( + 'module os\nos.access\n path: {"some": "dictionary"}', + 'module os\nos.access\n path: ["list", "of", "strings"]', + 'module os\nos.access\n path: (x for x in range(42))', ) - self.assertEqual(s, expected_failure_message) + for block in dataset: + with self.subTest(block=block): + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_failure_message) def test_kwarg_splats_disallowed_in_function_call_annotations(self): expected_error_msg = ( @@ -945,10 +1023,16 @@ def test_scaffolding(self): self.assertEqual(repr(clinic.NULL), '') # test that fail fails + expected = ( + 'Error in file "clown.txt" on line 69:\n' + 'The igloos are melting!\n' + ) with support.captured_stdout() as stdout: with self.assertRaises(SystemExit): - clinic.fail('The igloos are melting!', filename='clown.txt', line_number=69) - self.assertEqual(stdout.getvalue(), 'Error in file "clown.txt" on line 69:\nThe igloos are melting!\n') + clinic.fail('The igloos are melting!', + filename='clown.txt', line_number=69) + actual = stdout.getvalue() + self.assertEqual(actual, expected) class ClinicExternalTest(TestCase): From webhook-mailer at python.org Mon Jul 3 18:33:49 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 22:33:49 -0000 Subject: [Python-checkins] [3.11] gh-106368: Clean up Argument Clinic tests (#106373) (#106381) Message-ID: https://github.com/python/cpython/commit/3fdda976db62e1f4f3a8f7511c9c43527148c850 commit: 3fdda976db62e1f4f3a8f7511c9c43527148c850 branch: 3.11 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T22:33:45Z summary: [3.11] gh-106368: Clean up Argument Clinic tests (#106373) (#106381) (cherry picked from commit 3ee8dac7a1b3882aa3aac7703bdae2de7b6402ad) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index c2d2085ecd6de..26849ef9f211d 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -3,7 +3,8 @@ # Licensed to the PSF under a contributor agreement. from test import support, test_tools -from test.support import import_helper, os_helper +from test.support import os_helper +from textwrap import dedent from unittest import TestCase import collections import inspect @@ -170,43 +171,43 @@ def test_solo_newline(self): def test_no_substitution(self): self._test(""" abc - """, """ + """, """ abc - """) + """) def test_empty_substitution(self): self._test(""" abc {name} def - """, """ + """, """ abc def - """, name='') + """, name='') def test_single_line_substitution(self): self._test(""" abc {name} def - """, """ + """, """ abc GARGLE def - """, name='GARGLE') + """, name='GARGLE') def test_multiline_substitution(self): self._test(""" abc {name} def - """, """ + """, """ abc bingle bungle def - """, name='bingle\nbungle\n') + """, name='bingle\nbungle\n') class InertParser: def __init__(self, clinic): @@ -239,9 +240,9 @@ def round_trip(self, input): def test_round_trip_1(self): self.round_trip(""" - verbatim text here - lah dee dah -""") + verbatim text here + lah dee dah + """) def test_round_trip_2(self): self.round_trip(""" verbatim text here @@ -285,22 +286,38 @@ def test_clinic_1(self): class ClinicParserTest(TestCase): + def checkDocstring(self, fn, expected): + self.assertTrue(hasattr(fn, "docstring")) + self.assertEqual(fn.docstring.strip(), + dedent(expected).strip()) + def test_trivial(self): parser = DSLParser(FakeClinic()) - block = clinic.Block("module os\nos.access") + block = clinic.Block(""" + module os + os.access + """) parser.parse(block) module, function = block.signatures self.assertEqual("access", function.name) self.assertEqual("os", module.name) def test_ignore_line(self): - block = self.parse("#\nmodule os\nos.access") + block = self.parse(dedent(""" + # + module os + os.access + """)) module, function = block.signatures self.assertEqual("access", function.name) self.assertEqual("os", module.name) def test_param(self): - function = self.parse_function("module os\nos.access\n path: int") + function = self.parse_function(""" + module os + os.access + path: int + """) self.assertEqual("access", function.name) self.assertEqual(2, len(function.parameters)) p = function.parameters['path'] @@ -308,236 +325,296 @@ def test_param(self): self.assertIsInstance(p.converter, clinic.int_converter) def test_param_default(self): - function = self.parse_function("module os\nos.access\n follow_symlinks: bool = True") + function = self.parse_function(""" + module os + os.access + follow_symlinks: bool = True + """) p = function.parameters['follow_symlinks'] self.assertEqual(True, p.default) def test_param_with_continuations(self): - function = self.parse_function("module os\nos.access\n follow_symlinks: \\\n bool \\\n =\\\n True") + function = self.parse_function(r""" + module os + os.access + follow_symlinks: \ + bool \ + = \ + True + """) p = function.parameters['follow_symlinks'] self.assertEqual(True, p.default) def test_param_default_expression(self): - function = self.parse_function("module os\nos.access\n follow_symlinks: int(c_default='MAXSIZE') = sys.maxsize") + function = self.parse_function(""" + module os + os.access + follow_symlinks: int(c_default='MAXSIZE') = sys.maxsize + """) p = function.parameters['follow_symlinks'] self.assertEqual(sys.maxsize, p.default) self.assertEqual("MAXSIZE", p.converter.c_default) - s = self.parse_function_should_fail("module os\nos.access\n follow_symlinks: int = sys.maxsize") - self.assertEqual(s, "Error on line 0:\nWhen you specify a named constant ('sys.maxsize') as your default value,\nyou MUST specify a valid c_default.\n") + expected_msg = ( + "Error on line 0:\n" + "When you specify a named constant ('sys.maxsize') as your default value,\n" + "you MUST specify a valid c_default.\n" + ) + out = self.parse_function_should_fail(""" + module os + os.access + follow_symlinks: int = sys.maxsize + """) + self.assertEqual(out, expected_msg) def test_param_no_docstring(self): function = self.parse_function(""" -module os -os.access - follow_symlinks: bool = True - something_else: str = ''""") + module os + os.access + follow_symlinks: bool = True + something_else: str = '' + """) p = function.parameters['follow_symlinks'] self.assertEqual(3, len(function.parameters)) - self.assertIsInstance(function.parameters['something_else'].converter, clinic.str_converter) + conv = function.parameters['something_else'].converter + self.assertIsInstance(conv, clinic.str_converter) def test_param_default_parameters_out_of_order(self): - s = self.parse_function_should_fail(""" -module os -os.access - follow_symlinks: bool = True - something_else: str""") - self.assertEqual(s, """Error on line 0: -Can't have a parameter without a default ('something_else') -after a parameter with a default! -""") + expected_msg = ( + "Error on line 0:\n" + "Can't have a parameter without a default ('something_else')\n" + "after a parameter with a default!\n" + ) + out = self.parse_function_should_fail(""" + module os + os.access + follow_symlinks: bool = True + something_else: str""") + self.assertEqual(out, expected_msg) def disabled_test_converter_arguments(self): - function = self.parse_function("module os\nos.access\n path: path_t(allow_fd=1)") + function = self.parse_function(""" + module os + os.access + path: path_t(allow_fd=1) + """) p = function.parameters['path'] self.assertEqual(1, p.converter.args['allow_fd']) def test_function_docstring(self): function = self.parse_function(""" -module os -os.stat as os_stat_fn + module os + os.stat as os_stat_fn - path: str - Path to be examined + path: str + Path to be examined -Perform a stat system call on the given path.""") - self.assertEqual(""" -stat($module, /, path) --- + Perform a stat system call on the given path. + """) + self.checkDocstring(function, """ + stat($module, /, path) + -- -Perform a stat system call on the given path. + Perform a stat system call on the given path. - path - Path to be examined -""".strip(), function.docstring) + path + Path to be examined + """) def test_explicit_parameters_in_docstring(self): - function = self.parse_function(""" -module foo -foo.bar - x: int - Documentation for x. - y: int + function = self.parse_function(dedent(""" + module foo + foo.bar + x: int + Documentation for x. + y: int -This is the documentation for foo. + This is the documentation for foo. -Okay, we're done here. -""") - self.assertEqual(""" -bar($module, /, x, y) --- + Okay, we're done here. + """)) + self.checkDocstring(function, """ + bar($module, /, x, y) + -- -This is the documentation for foo. + This is the documentation for foo. - x - Documentation for x. + x + Documentation for x. -Okay, we're done here. -""".strip(), function.docstring) + Okay, we're done here. + """) def test_parser_regression_special_character_in_parameter_column_of_docstring_first_line(self): - function = self.parse_function(""" -module os -os.stat - path: str -This/used to break Clinic! -""") - self.assertEqual("stat($module, /, path)\n--\n\nThis/used to break Clinic!", function.docstring) + function = self.parse_function(dedent(""" + module os + os.stat + path: str + This/used to break Clinic! + """)) + self.checkDocstring(function, """ + stat($module, /, path) + -- + + This/used to break Clinic! + """) def test_c_name(self): - function = self.parse_function("module os\nos.stat as os_stat_fn") + function = self.parse_function(""" + module os + os.stat as os_stat_fn + """) self.assertEqual("os_stat_fn", function.c_basename) def test_return_converter(self): - function = self.parse_function("module os\nos.stat -> int") + function = self.parse_function(""" + module os + os.stat -> int + """) self.assertIsInstance(function.return_converter, clinic.int_return_converter) def test_star(self): - function = self.parse_function("module os\nos.access\n *\n follow_symlinks: bool = True") + function = self.parse_function(""" + module os + os.access + * + follow_symlinks: bool = True + """) p = function.parameters['follow_symlinks'] self.assertEqual(inspect.Parameter.KEYWORD_ONLY, p.kind) self.assertEqual(0, p.group) def test_group(self): - function = self.parse_function("module window\nwindow.border\n [\n ls : int\n ]\n /\n") + function = self.parse_function(""" + module window + window.border + [ + ls: int + ] + / + """) p = function.parameters['ls'] self.assertEqual(1, p.group) def test_left_group(self): function = self.parse_function(""" -module curses -curses.addch - [ - y: int - Y-coordinate. - x: int - X-coordinate. - ] - ch: char - Character to add. - [ - attr: long - Attributes for the character. - ] - / -""") - for name, group in ( + module curses + curses.addch + [ + y: int + Y-coordinate. + x: int + X-coordinate. + ] + ch: char + Character to add. + [ + attr: long + Attributes for the character. + ] + / + """) + dataset = ( ('y', -1), ('x', -1), ('ch', 0), ('attr', 1), - ): - p = function.parameters[name] - self.assertEqual(p.group, group) - self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) - self.assertEqual(function.docstring.strip(), """ -addch([y, x,] ch, [attr]) - - - y - Y-coordinate. - x - X-coordinate. - ch - Character to add. - attr - Attributes for the character. - """.strip()) + ) + for name, group in dataset: + with self.subTest(name=name, group=group): + p = function.parameters[name] + self.assertEqual(p.group, group) + self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) + self.checkDocstring(function, """ + addch([y, x,] ch, [attr]) + + + y + Y-coordinate. + x + X-coordinate. + ch + Character to add. + attr + Attributes for the character. + """) def test_nested_groups(self): function = self.parse_function(""" -module curses -curses.imaginary - [ - [ - y1: int - Y-coordinate. - y2: int - Y-coordinate. - ] - x1: int - X-coordinate. - x2: int - X-coordinate. - ] - ch: char - Character to add. - [ - attr1: long - Attributes for the character. - attr2: long - Attributes for the character. - attr3: long - Attributes for the character. - [ - attr4: long - Attributes for the character. - attr5: long - Attributes for the character. - attr6: long - Attributes for the character. - ] - ] - / -""") - for name, group in ( + module curses + curses.imaginary + [ + [ + y1: int + Y-coordinate. + y2: int + Y-coordinate. + ] + x1: int + X-coordinate. + x2: int + X-coordinate. + ] + ch: char + Character to add. + [ + attr1: long + Attributes for the character. + attr2: long + Attributes for the character. + attr3: long + Attributes for the character. + [ + attr4: long + Attributes for the character. + attr5: long + Attributes for the character. + attr6: long + Attributes for the character. + ] + ] + / + """) + dataset = ( ('y1', -2), ('y2', -2), ('x1', -1), ('x2', -1), ('ch', 0), ('attr1', 1), ('attr2', 1), ('attr3', 1), ('attr4', 2), ('attr5', 2), ('attr6', 2), - ): - p = function.parameters[name] - self.assertEqual(p.group, group) - self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) - - self.assertEqual(function.docstring.strip(), """ -imaginary([[y1, y2,] x1, x2,] ch, [attr1, attr2, attr3, [attr4, attr5, - attr6]]) - - - y1 - Y-coordinate. - y2 - Y-coordinate. - x1 - X-coordinate. - x2 - X-coordinate. - ch - Character to add. - attr1 - Attributes for the character. - attr2 - Attributes for the character. - attr3 - Attributes for the character. - attr4 - Attributes for the character. - attr5 - Attributes for the character. - attr6 - Attributes for the character. - """.strip()) + ) + for name, group in dataset: + with self.subTest(name=name, group=group): + p = function.parameters[name] + self.assertEqual(p.group, group) + self.assertEqual(p.kind, inspect.Parameter.POSITIONAL_ONLY) + + self.checkDocstring(function, """ + imaginary([[y1, y2,] x1, x2,] ch, [attr1, attr2, attr3, [attr4, attr5, + attr6]]) + + + y1 + Y-coordinate. + y2 + Y-coordinate. + x1 + X-coordinate. + x2 + X-coordinate. + ch + Character to add. + attr1 + Attributes for the character. + attr2 + Attributes for the character. + attr3 + Attributes for the character. + attr4 + Attributes for the character. + attr5 + Attributes for the character. + attr6 + Attributes for the character. + """) def parse_function_should_fail(self, s): with support.captured_stdout() as stdout: @@ -546,104 +623,108 @@ def parse_function_should_fail(self, s): return stdout.getvalue() def test_disallowed_grouping__two_top_groups_on_left(self): - s = self.parse_function_should_fail(""" -module foo -foo.two_top_groups_on_left - [ - group1 : int - ] - [ - group2 : int - ] - param: int - """) - self.assertEqual(s, - ('Error on line 0:\n' - 'Function two_top_groups_on_left has an unsupported group configuration. (Unexpected state 2.b)\n')) + expected_msg = ( + 'Error on line 0:\n' + 'Function two_top_groups_on_left has an unsupported group ' + 'configuration. (Unexpected state 2.b)\n' + ) + out = self.parse_function_should_fail(""" + module foo + foo.two_top_groups_on_left + [ + group1 : int + ] + [ + group2 : int + ] + param: int + """) + self.assertEqual(out, expected_msg) def test_disallowed_grouping__two_top_groups_on_right(self): self.parse_function_should_fail(""" -module foo -foo.two_top_groups_on_right - param: int - [ - group1 : int - ] - [ - group2 : int - ] - """) + module foo + foo.two_top_groups_on_right + param: int + [ + group1 : int + ] + [ + group2 : int + ] + """) def test_disallowed_grouping__parameter_after_group_on_right(self): self.parse_function_should_fail(""" -module foo -foo.parameter_after_group_on_right - param: int - [ - [ - group1 : int - ] - group2 : int - ] - """) + module foo + foo.parameter_after_group_on_right + param: int + [ + [ + group1 : int + ] + group2 : int + ] + """) def test_disallowed_grouping__group_after_parameter_on_left(self): self.parse_function_should_fail(""" -module foo -foo.group_after_parameter_on_left - [ - group2 : int - [ - group1 : int - ] - ] - param: int - """) + module foo + foo.group_after_parameter_on_left + [ + group2 : int + [ + group1 : int + ] + ] + param: int + """) def test_disallowed_grouping__empty_group_on_left(self): self.parse_function_should_fail(""" -module foo -foo.empty_group - [ - [ - ] - group2 : int - ] - param: int - """) + module foo + foo.empty_group + [ + [ + ] + group2 : int + ] + param: int + """) def test_disallowed_grouping__empty_group_on_right(self): self.parse_function_should_fail(""" -module foo -foo.empty_group - param: int - [ - [ - ] - group2 : int - ] - """) + module foo + foo.empty_group + param: int + [ + [ + ] + group2 : int + ] + """) def test_no_parameters(self): function = self.parse_function(""" -module foo -foo.bar + module foo + foo.bar -Docstring + Docstring -""") + """) self.assertEqual("bar($module, /)\n--\n\nDocstring", function.docstring) self.assertEqual(1, len(function.parameters)) # self! def test_init_with_no_parameters(self): function = self.parse_function(""" -module foo -class foo.Bar "unused" "notneeded" -foo.Bar.__init__ + module foo + class foo.Bar "unused" "notneeded" + foo.Bar.__init__ + + Docstring -Docstring + """, signatures_in_block=3, function_index=2) -""", signatures_in_block=3, function_index=2) # self is not in the signature self.assertEqual("Bar()\n--\n\nDocstring", function.docstring) # but it *is* a parameter @@ -651,113 +732,117 @@ class foo.Bar "unused" "notneeded" def test_illegal_module_line(self): self.parse_function_should_fail(""" -module foo -foo.bar => int - / -""") + module foo + foo.bar => int + / + """) def test_illegal_c_basename(self): self.parse_function_should_fail(""" -module foo -foo.bar as 935 - / -""") + module foo + foo.bar as 935 + / + """) def test_single_star(self): self.parse_function_should_fail(""" -module foo -foo.bar - * - * -""") + module foo + foo.bar + * + * + """) def test_parameters_required_after_star_without_initial_parameters_or_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - * -""") + module foo + foo.bar + * + """) def test_parameters_required_after_star_without_initial_parameters_with_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - * -Docstring here. -""") + module foo + foo.bar + * + Docstring here. + """) def test_parameters_required_after_star_with_initial_parameters_without_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - this: int - * -""") + module foo + foo.bar + this: int + * + """) def test_parameters_required_after_star_with_initial_parameters_and_docstring(self): self.parse_function_should_fail(""" -module foo -foo.bar - this: int - * -Docstring. -""") + module foo + foo.bar + this: int + * + Docstring. + """) def test_single_slash(self): self.parse_function_should_fail(""" -module foo -foo.bar - / - / -""") + module foo + foo.bar + / + / + """) def test_mix_star_and_slash(self): self.parse_function_should_fail(""" -module foo -foo.bar - x: int - y: int - * - z: int - / -""") + module foo + foo.bar + x: int + y: int + * + z: int + / + """) def test_parameters_not_permitted_after_slash_for_now(self): self.parse_function_should_fail(""" -module foo -foo.bar - / - x: int -""") + module foo + foo.bar + / + x: int + """) def test_parameters_no_more_than_one_vararg(self): - s = self.parse_function_should_fail(""" -module foo -foo.bar - *vararg1: object - *vararg2: object -""") - self.assertEqual(s, "Error on line 0:\nToo many var args\n") + expected_msg = ( + "Error on line 0:\n" + "Too many var args\n" + ) + out = self.parse_function_should_fail(""" + module foo + foo.bar + *vararg1: object + *vararg2: object + """) + self.assertEqual(out, expected_msg) def test_function_not_at_column_0(self): function = self.parse_function(""" - module foo - foo.bar - x: int - Nested docstring here, goeth. - * - y: str - Not at column 0! -""") - self.assertEqual(""" -bar($module, /, x, *, y) --- - -Not at column 0! - - x - Nested docstring here, goeth. -""".strip(), function.docstring) + module foo + foo.bar + x: int + Nested docstring here, goeth. + * + y: str + Not at column 0! + """) + self.checkDocstring(function, """ + bar($module, /, x, *, y) + -- + + Not at column 0! + + x + Nested docstring here, goeth. + """) def test_directive(self): c = FakeClinic() @@ -771,46 +856,39 @@ def test_directive(self): def test_legacy_converters(self): block = self.parse('module os\nos.access\n path: "s"') module, function = block.signatures - self.assertIsInstance((function.parameters['path']).converter, clinic.str_converter) + conv = (function.parameters['path']).converter + self.assertIsInstance(conv, clinic.str_converter) def test_legacy_converters_non_string_constant_annotation(self): - expected_failure_message = """\ -Error on line 0: -Annotations must be either a name, a function call, or a string. -""" - - s = self.parse_function_should_fail('module os\nos.access\n path: 42') - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail('module os\nos.access\n path: 42.42') - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail('module os\nos.access\n path: 42j') - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail('module os\nos.access\n path: b"42"') - self.assertEqual(s, expected_failure_message) - - def test_other_bizarre_things_in_annotations_fail(self): - expected_failure_message = """\ -Error on line 0: -Annotations must be either a name, a function call, or a string. -""" - - s = self.parse_function_should_fail( - 'module os\nos.access\n path: {"some": "dictionary"}' + expected_failure_message = ( + "Error on line 0:\n" + "Annotations must be either a name, a function call, or a string.\n" ) - self.assertEqual(s, expected_failure_message) - - s = self.parse_function_should_fail( - 'module os\nos.access\n path: ["list", "of", "strings"]' + dataset = ( + 'module os\nos.access\n path: 42', + 'module os\nos.access\n path: 42.42', + 'module os\nos.access\n path: 42j', + 'module os\nos.access\n path: b"42"', ) - self.assertEqual(s, expected_failure_message) + for block in dataset: + with self.subTest(block=block): + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_failure_message) - s = self.parse_function_should_fail( - 'module os\nos.access\n path: (x for x in range(42))' + def test_other_bizarre_things_in_annotations_fail(self): + expected_failure_message = ( + "Error on line 0:\n" + "Annotations must be either a name, a function call, or a string.\n" ) - self.assertEqual(s, expected_failure_message) + dataset = ( + 'module os\nos.access\n path: {"some": "dictionary"}', + 'module os\nos.access\n path: ["list", "of", "strings"]', + 'module os\nos.access\n path: (x for x in range(42))', + ) + for block in dataset: + with self.subTest(block=block): + out = self.parse_function_should_fail(block) + self.assertEqual(out, expected_failure_message) def test_self_param_placement(self): expected_error_msg = ( @@ -890,10 +968,16 @@ def test_scaffolding(self): self.assertEqual(repr(clinic.NULL), '') # test that fail fails + expected = ( + 'Error in file "clown.txt" on line 69:\n' + 'The igloos are melting!\n' + ) with support.captured_stdout() as stdout: with self.assertRaises(SystemExit): - clinic.fail('The igloos are melting!', filename='clown.txt', line_number=69) - self.assertEqual(stdout.getvalue(), 'Error in file "clown.txt" on line 69:\nThe igloos are melting!\n') + clinic.fail('The igloos are melting!', + filename='clown.txt', line_number=69) + actual = stdout.getvalue() + self.assertEqual(actual, expected) class ClinicExternalTest(TestCase): From webhook-mailer at python.org Mon Jul 3 18:35:49 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 03 Jul 2023 22:35:49 -0000 Subject: [Python-checkins] gh-106320: Remove more private _PyUnicode C API functions (#106382) Message-ID: https://github.com/python/cpython/commit/506cfdf141f03186d5cdf9bb31caa40294eba4e5 commit: 506cfdf141f03186d5cdf9bb31caa40294eba4e5 branch: main author: Victor Stinner committer: vstinner date: 2023-07-03T22:35:46Z summary: gh-106320: Remove more private _PyUnicode C API functions (#106382) Remove more private _PyUnicode C API functions: move them to the internal C API (pycore_unicodeobject.h). No longer export most pycore_unicodeobject.h functions. files: M Include/cpython/unicodeobject.h M Include/internal/pycore_unicodeobject.h M Python/future.c diff --git a/Include/cpython/unicodeobject.h b/Include/cpython/unicodeobject.h index c5892a80d2c54..fcd9c28e030c1 100644 --- a/Include/cpython/unicodeobject.h +++ b/Include/cpython/unicodeobject.h @@ -394,11 +394,6 @@ static inline int PyUnicode_READY(PyObject* Py_UNUSED(op)) } #define PyUnicode_READY(op) PyUnicode_READY(_PyObject_CAST(op)) -/* Get a copy of a Unicode string. */ -PyAPI_FUNC(PyObject*) _PyUnicode_Copy( - PyObject *unicode - ); - /* Copy character from one unicode object into another, this function performs character conversion when necessary and falls back to memcpy() if possible. @@ -425,17 +420,6 @@ PyAPI_FUNC(Py_ssize_t) PyUnicode_CopyCharacters( Py_ssize_t how_many ); -/* Unsafe version of PyUnicode_CopyCharacters(): don't check arguments and so - may crash if parameters are invalid (e.g. if the output string - is too short). */ -PyAPI_FUNC(void) _PyUnicode_FastCopyCharacters( - PyObject *to, - Py_ssize_t to_start, - PyObject *from, - Py_ssize_t from_start, - Py_ssize_t how_many - ); - /* Fill a string with a character: write fill_char into unicode[start:start+length]. @@ -451,15 +435,6 @@ PyAPI_FUNC(Py_ssize_t) PyUnicode_Fill( Py_UCS4 fill_char ); -/* Unsafe version of PyUnicode_Fill(): don't check arguments and so may crash - if parameters are invalid (e.g. if length is longer than the string). */ -PyAPI_FUNC(void) _PyUnicode_FastFill( - PyObject *unicode, - Py_ssize_t start, - Py_ssize_t length, - Py_UCS4 fill_char - ); - /* Create a new string from a buffer of Py_UCS1, Py_UCS2 or Py_UCS4 characters. Scan the string to find the maximum character. */ PyAPI_FUNC(PyObject*) PyUnicode_FromKindAndData( @@ -467,19 +442,6 @@ PyAPI_FUNC(PyObject*) PyUnicode_FromKindAndData( const void *buffer, Py_ssize_t size); -/* Create a new string from a buffer of ASCII characters. - WARNING: Don't check if the string contains any non-ASCII character. */ -PyAPI_FUNC(PyObject*) _PyUnicode_FromASCII( - const char *buffer, - Py_ssize_t size); - -/* Compute the maximum character of the substring unicode[start:end]. - Return 127 for an empty string. */ -PyAPI_FUNC(Py_UCS4) _PyUnicode_FindMaxChar ( - PyObject *unicode, - Py_ssize_t start, - Py_ssize_t end); - /* --- Manage the default encoding ---------------------------------------- */ /* Returns a pointer to the default encoding (UTF-8) of the @@ -618,37 +580,6 @@ PyAPI_FUNC(PyObject*) _PyUnicode_TransformDecimalAndSpaceToASCII( PyObject *unicode /* Unicode object */ ); -/* --- Methods & Slots ---------------------------------------------------- */ - -PyAPI_FUNC(PyObject *) _PyUnicode_JoinArray( - PyObject *separator, - PyObject *const *items, - Py_ssize_t seqlen - ); - -/* Test whether a unicode is equal to ASCII identifier. Return 1 if true, - 0 otherwise. The right argument must be ASCII identifier. - Any error occurs inside will be cleared before return. */ -PyAPI_FUNC(int) _PyUnicode_EqualToASCIIId( - PyObject *left, /* Left string */ - _Py_Identifier *right /* Right identifier */ - ); - -/* Test whether a unicode is equal to ASCII string. Return 1 if true, - 0 otherwise. The right argument must be ASCII-encoded string. - Any error occurs inside will be cleared before return. */ -PyAPI_FUNC(int) _PyUnicode_EqualToASCIIString( - PyObject *left, - const char *right /* ASCII-encoded string */ - ); - -/* Externally visible for str.strip(unicode) */ -PyAPI_FUNC(PyObject *) _PyUnicode_XStrip( - PyObject *self, - int striptype, - PyObject *sepobj - ); - /* === Characters Type APIs =============================================== */ /* These should not be used directly. Use the Py_UNICODE_IS* and diff --git a/Include/internal/pycore_unicodeobject.h b/Include/internal/pycore_unicodeobject.h index a8c7f1957f360..da01f57f96279 100644 --- a/Include/internal/pycore_unicodeobject.h +++ b/Include/internal/pycore_unicodeobject.h @@ -14,6 +14,44 @@ extern "C" { void _PyUnicode_ExactDealloc(PyObject *op); Py_ssize_t _PyUnicode_InternedSize(void); +/* Get a copy of a Unicode string. */ +PyAPI_FUNC(PyObject*) _PyUnicode_Copy( + PyObject *unicode + ); + +/* Unsafe version of PyUnicode_Fill(): don't check arguments and so may crash + if parameters are invalid (e.g. if length is longer than the string). */ +extern void _PyUnicode_FastFill( + PyObject *unicode, + Py_ssize_t start, + Py_ssize_t length, + Py_UCS4 fill_char + ); + +/* Unsafe version of PyUnicode_CopyCharacters(): don't check arguments and so + may crash if parameters are invalid (e.g. if the output string + is too short). */ +extern void _PyUnicode_FastCopyCharacters( + PyObject *to, + Py_ssize_t to_start, + PyObject *from, + Py_ssize_t from_start, + Py_ssize_t how_many + ); + +/* Create a new string from a buffer of ASCII characters. + WARNING: Don't check if the string contains any non-ASCII character. */ +extern PyObject* _PyUnicode_FromASCII( + const char *buffer, + Py_ssize_t size); + +/* Compute the maximum character of the substring unicode[start:end]. + Return 127 for an empty string. */ +extern Py_UCS4 _PyUnicode_FindMaxChar ( + PyObject *unicode, + Py_ssize_t start, + Py_ssize_t end); + /* --- _PyUnicodeWriter API ----------------------------------------------- */ typedef struct { @@ -141,10 +179,40 @@ PyAPI_FUNC(int) _PyUnicode_FormatAdvancedWriter( /* --- Methods & Slots ---------------------------------------------------- */ +extern PyObject* _PyUnicode_JoinArray( + PyObject *separator, + PyObject *const *items, + Py_ssize_t seqlen + ); + +/* Test whether a unicode is equal to ASCII identifier. Return 1 if true, + 0 otherwise. The right argument must be ASCII identifier. + Any error occurs inside will be cleared before return. */ +extern int _PyUnicode_EqualToASCIIId( + PyObject *left, /* Left string */ + _Py_Identifier *right /* Right identifier */ + ); + +/* Test whether a unicode is equal to ASCII string. Return 1 if true, + 0 otherwise. The right argument must be ASCII-encoded string. + Any error occurs inside will be cleared before return. */ +PyAPI_FUNC(int) _PyUnicode_EqualToASCIIString( + PyObject *left, + const char *right /* ASCII-encoded string */ + ); + +/* Externally visible for str.strip(unicode) */ +extern PyObject* _PyUnicode_XStrip( + PyObject *self, + int striptype, + PyObject *sepobj + ); + + /* Using explicit passed-in values, insert the thousands grouping into the string pointed to by buffer. For the argument descriptions, see Objects/stringlib/localeutil.h */ -PyAPI_FUNC(Py_ssize_t) _PyUnicode_InsertThousandsGrouping( +extern Py_ssize_t _PyUnicode_InsertThousandsGrouping( _PyUnicodeWriter *writer, Py_ssize_t n_buffer, PyObject *digits, diff --git a/Python/future.c b/Python/future.c index d56f733096468..0dbc7ede20f32 100644 --- a/Python/future.c +++ b/Python/future.c @@ -1,5 +1,6 @@ #include "Python.h" #include "pycore_ast.h" // _PyAST_GetDocString() +#include "pycore_unicodeobject.h" // _PyUnicode_EqualToASCIIString() #define UNDEFINED_FUTURE_FEATURE "future feature %.100s is not defined" From webhook-mailer at python.org Mon Jul 3 18:52:31 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 03 Jul 2023 22:52:31 -0000 Subject: [Python-checkins] gh-86085: Remove _PyCodec_Forget() declaration (#106377) Message-ID: https://github.com/python/cpython/commit/f6d2bb18aba844f6bb5836797c72eb791b7f3644 commit: f6d2bb18aba844f6bb5836797c72eb791b7f3644 branch: main author: Victor Stinner committer: vstinner date: 2023-07-04T00:52:27+02:00 summary: gh-86085: Remove _PyCodec_Forget() declaration (#106377) The code was already removed by: commit c9f696cb96d1c362d5cad871f61da520572d9b08. files: M Include/internal/pycore_codecs.h diff --git a/Include/internal/pycore_codecs.h b/Include/internal/pycore_codecs.h index 2f8d9d510019b..a2465192eacd5 100644 --- a/Include/internal/pycore_codecs.h +++ b/Include/internal/pycore_codecs.h @@ -6,8 +6,6 @@ extern "C" { extern PyObject* _PyCodec_Lookup(const char *encoding); -extern int _PyCodec_Forget(const char *encoding); - /* Text codec specific encoding and decoding API. Checks the encoding against a list of codecs which do not From webhook-mailer at python.org Mon Jul 3 19:02:10 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 03 Jul 2023 23:02:10 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyImport C API functions (#106383) Message-ID: https://github.com/python/cpython/commit/2e92edbf6de9578b30cca8e48c4bfb2ba71ae97a commit: 2e92edbf6de9578b30cca8e48c4bfb2ba71ae97a branch: main author: Victor Stinner committer: vstinner date: 2023-07-03T23:02:07Z summary: gh-106320: Remove private _PyImport C API functions (#106383) * Remove private _PyImport C API functions: move them to the internal C API (pycore_import.h). * No longer export most of these private functions. * _testcapi avoids private _PyImport_GetModuleAttrString(). files: M Include/cpython/import.h M Include/internal/pycore_import.h M Modules/_elementtree.c M Modules/_sqlite/connection.c M Modules/_sqlite/module.c M Modules/_testcapimodule.c M Modules/cjkcodecs/cjkcodecs.h M Modules/pyexpat.c diff --git a/Include/cpython/import.h b/Include/cpython/import.h index 2bca4ade4c4f2..cdfdd15bfa48d 100644 --- a/Include/cpython/import.h +++ b/Include/cpython/import.h @@ -4,23 +4,6 @@ PyMODINIT_FUNC PyInit__imp(void); -PyAPI_FUNC(int) _PyImport_IsInitialized(PyInterpreterState *); - -PyAPI_FUNC(PyObject *) _PyImport_GetModuleId(_Py_Identifier *name); -PyAPI_FUNC(int) _PyImport_SetModule(PyObject *name, PyObject *module); -PyAPI_FUNC(int) _PyImport_SetModuleString(const char *name, PyObject* module); - -PyAPI_FUNC(void) _PyImport_AcquireLock(PyInterpreterState *interp); -PyAPI_FUNC(int) _PyImport_ReleaseLock(PyInterpreterState *interp); - -PyAPI_FUNC(int) _PyImport_FixupBuiltin( - PyObject *mod, - const char *name, /* UTF-8 encoded string */ - PyObject *modules - ); -PyAPI_FUNC(int) _PyImport_FixupExtensionObject(PyObject*, PyObject *, - PyObject *, PyObject *); - struct _inittab { const char *name; /* ASCII encoded string */ PyObject* (*initfunc)(void); @@ -41,6 +24,3 @@ struct _frozen { collection of frozen modules: */ PyAPI_DATA(const struct _frozen *) PyImport_FrozenModules; - -PyAPI_DATA(PyObject *) _PyImport_GetModuleAttr(PyObject *, PyObject *); -PyAPI_DATA(PyObject *) _PyImport_GetModuleAttrString(const char *, const char *); diff --git a/Include/internal/pycore_import.h b/Include/internal/pycore_import.h index ee93f7d99d915..457a654aff464 100644 --- a/Include/internal/pycore_import.h +++ b/Include/internal/pycore_import.h @@ -7,6 +7,26 @@ extern "C" { #include "pycore_time.h" // _PyTime_t +extern int _PyImport_IsInitialized(PyInterpreterState *); + +PyAPI_FUNC(PyObject *) _PyImport_GetModuleId(_Py_Identifier *name); +PyAPI_FUNC(int) _PyImport_SetModule(PyObject *name, PyObject *module); +PyAPI_FUNC(int) _PyImport_SetModuleString(const char *name, PyObject* module); + +extern void _PyImport_AcquireLock(PyInterpreterState *interp); +extern int _PyImport_ReleaseLock(PyInterpreterState *interp); + +extern int _PyImport_FixupBuiltin( + PyObject *mod, + const char *name, /* UTF-8 encoded string */ + PyObject *modules + ); +extern int _PyImport_FixupExtensionObject(PyObject*, PyObject *, + PyObject *, PyObject *); + +PyAPI_DATA(PyObject *) _PyImport_GetModuleAttr(PyObject *, PyObject *); +PyAPI_DATA(PyObject *) _PyImport_GetModuleAttrString(const char *, const char *); + struct _import_runtime_state { /* The builtin modules (defined in config.c). */ diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 48280690a707a..3e742e067e7db 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -11,7 +11,12 @@ *-------------------------------------------------------------------- */ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + #include "Python.h" +#include "pycore_import.h" // _PyImport_GetModuleAttrString() #include "structmember.h" // PyMemberDef #include "expat.h" #include "pyexpat.h" diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index 967ba2812080e..d71cef14779e5 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -33,6 +33,7 @@ #include "blob.h" #include "prepare_protocol.h" #include "util.h" +#include "pycore_import.h" // _PyImport_GetModuleAttrString() #include "pycore_weakref.h" // _PyWeakref_IS_DEAD() #include diff --git a/Modules/_sqlite/module.c b/Modules/_sqlite/module.c index ea4d8c58b7ee0..368e581b4f335 100644 --- a/Modules/_sqlite/module.c +++ b/Modules/_sqlite/module.c @@ -21,6 +21,10 @@ * 3. This notice may not be removed or altered from any source distribution. */ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + #include "connection.h" #include "statement.h" #include "cursor.h" @@ -29,6 +33,8 @@ #include "row.h" #include "blob.h" +#include "pycore_import.h" // _PyImport_GetModuleAttrString() + #if SQLITE_VERSION_NUMBER < 3015002 #error "SQLite 3.15.2 or higher required" #endif diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index ce1131743eb2a..d1044b5445202 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -1267,9 +1267,15 @@ test_pep3118_obsolete_write_locks(PyObject* self, PyObject *Py_UNUSED(ignored)) if (ret != -1 || match == 0) goto error; + PyObject *mod_io = PyImport_ImportModule("_io"); + if (mod_io == NULL) { + return NULL; + } + /* bytesiobuf_getbuffer() */ - PyTypeObject *type = (PyTypeObject *)_PyImport_GetModuleAttrString( - "_io", "_BytesIOBuffer"); + PyTypeObject *type = (PyTypeObject *)PyObject_GetAttrString( + mod_io, "_BytesIOBuffer"); + Py_DECREF(mod_io); if (type == NULL) { return NULL; } diff --git a/Modules/cjkcodecs/cjkcodecs.h b/Modules/cjkcodecs/cjkcodecs.h index 97290aac3ba43..ee588785e7403 100644 --- a/Modules/cjkcodecs/cjkcodecs.h +++ b/Modules/cjkcodecs/cjkcodecs.h @@ -13,6 +13,7 @@ #include "Python.h" #include "multibytecodec.h" +#include "pycore_import.h" // _PyImport_GetModuleAttrString() /* a unicode "undefined" code point */ diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index e3333fff00b2b..28915359fb49e 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1,4 +1,9 @@ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + #include "Python.h" +#include "pycore_import.h" // _PyImport_SetModule() #include #include "structmember.h" // PyMemberDef From webhook-mailer at python.org Mon Jul 3 19:10:51 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 23:10:51 -0000 Subject: [Python-checkins] gh-106368: Harden Argument Clinic parser tests (#106384) Message-ID: https://github.com/python/cpython/commit/648688c137744a623a71dc2413d2879b80c99eae commit: 648688c137744a623a71dc2413d2879b80c99eae branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T23:10:47Z summary: gh-106368: Harden Argument Clinic parser tests (#106384) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index b3602887ab635..c5cfe53e0df99 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -643,7 +643,7 @@ def test_disallowed_grouping__two_top_groups_on_left(self): self.assertEqual(out, expected_msg) def test_disallowed_grouping__two_top_groups_on_right(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.two_top_groups_on_right param: int @@ -654,9 +654,14 @@ def test_disallowed_grouping__two_top_groups_on_right(self): group2 : int ] """) + msg = ( + "Function two_top_groups_on_right has an unsupported group " + "configuration. (Unexpected state 6.b)" + ) + self.assertIn(msg, out) def test_disallowed_grouping__parameter_after_group_on_right(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.parameter_after_group_on_right param: int @@ -667,9 +672,14 @@ def test_disallowed_grouping__parameter_after_group_on_right(self): group2 : int ] """) + msg = ( + "Function parameter_after_group_on_right has an unsupported group " + "configuration. (Unexpected state 6.a)" + ) + self.assertIn(msg, out) def test_disallowed_grouping__group_after_parameter_on_left(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.group_after_parameter_on_left [ @@ -680,9 +690,14 @@ def test_disallowed_grouping__group_after_parameter_on_left(self): ] param: int """) + msg = ( + "Function group_after_parameter_on_left has an unsupported group " + "configuration. (Unexpected state 2.b)" + ) + self.assertIn(msg, out) def test_disallowed_grouping__empty_group_on_left(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.empty_group [ @@ -692,9 +707,14 @@ def test_disallowed_grouping__empty_group_on_left(self): ] param: int """) + msg = ( + "Function empty_group has an empty group.\n" + "All groups must contain at least one parameter." + ) + self.assertIn(msg, out) def test_disallowed_grouping__empty_group_on_right(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.empty_group param: int @@ -704,6 +724,11 @@ def test_disallowed_grouping__empty_group_on_right(self): group2 : int ] """) + msg = ( + "Function empty_group has an empty group.\n" + "All groups must contain at least one parameter." + ) + self.assertIn(msg, out) def test_no_parameters(self): function = self.parse_function(""" @@ -732,69 +757,60 @@ class foo.Bar "unused" "notneeded" self.assertEqual(1, len(function.parameters)) def test_illegal_module_line(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar => int / """) + msg = "Illegal function name: foo.bar => int" + self.assertIn(msg, out) def test_illegal_c_basename(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar as 935 / """) + msg = "Illegal C basename: 935" + self.assertIn(msg, out) def test_single_star(self): - self.parse_function_should_fail(""" - module foo - foo.bar - * - * - """) - - def test_parameters_required_after_star_without_initial_parameters_or_docstring(self): - self.parse_function_should_fail(""" - module foo - foo.bar - * - """) - - def test_parameters_required_after_star_without_initial_parameters_with_docstring(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar * - Docstring here. - """) - - def test_parameters_required_after_star_with_initial_parameters_without_docstring(self): - self.parse_function_should_fail(""" - module foo - foo.bar - this: int * """) + self.assertIn("Function bar uses '*' more than once.", out) - def test_parameters_required_after_star_with_initial_parameters_and_docstring(self): - self.parse_function_should_fail(""" - module foo - foo.bar - this: int - * - Docstring. - """) + def test_parameters_required_after_star(self): + dataset = ( + "module foo\nfoo.bar\n *", + "module foo\nfoo.bar\n *\nDocstring here.", + "module foo\nfoo.bar\n this: int\n *", + "module foo\nfoo.bar\n this: int\n *\nDocstring.", + ) + msg = "Function bar specifies '*' without any parameters afterwards." + for block in dataset: + with self.subTest(block=block): + out = self.parse_function_should_fail(block) + self.assertIn(msg, out) def test_single_slash(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar / / """) + msg = ( + "Function bar has an unsupported group configuration. " + "(Unexpected state 0.d)" + ) + self.assertIn(msg, out) def test_mix_star_and_slash(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar x: int @@ -803,14 +819,24 @@ def test_mix_star_and_slash(self): z: int / """) + msg = ( + "Function bar mixes keyword-only and positional-only parameters, " + "which is unsupported." + ) + self.assertIn(msg, out) def test_parameters_not_permitted_after_slash_for_now(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar / x: int """) + msg = ( + "Function bar has an unsupported group configuration. " + "(Unexpected state 0.d)" + ) + self.assertIn(msg, out) def test_parameters_no_more_than_one_vararg(self): expected_msg = ( From webhook-mailer at python.org Mon Jul 3 19:35:45 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 23:35:45 -0000 Subject: [Python-checkins] [3.11] gh-106368: Harden Argument Clinic parser tests (GH-106384) (#106388) Message-ID: https://github.com/python/cpython/commit/76889641e63cb637d4fc6a85125d90c8edb428c1 commit: 76889641e63cb637d4fc6a85125d90c8edb428c1 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-03T23:35:42Z summary: [3.11] gh-106368: Harden Argument Clinic parser tests (GH-106384) (#106388) (cherry picked from commit 648688c137744a623a71dc2413d2879b80c99eae) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 26849ef9f211d..f107e14e38f20 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -642,7 +642,7 @@ def test_disallowed_grouping__two_top_groups_on_left(self): self.assertEqual(out, expected_msg) def test_disallowed_grouping__two_top_groups_on_right(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.two_top_groups_on_right param: int @@ -653,9 +653,14 @@ def test_disallowed_grouping__two_top_groups_on_right(self): group2 : int ] """) + msg = ( + "Function two_top_groups_on_right has an unsupported group " + "configuration. (Unexpected state 6.b)" + ) + self.assertIn(msg, out) def test_disallowed_grouping__parameter_after_group_on_right(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.parameter_after_group_on_right param: int @@ -666,9 +671,14 @@ def test_disallowed_grouping__parameter_after_group_on_right(self): group2 : int ] """) + msg = ( + "Function parameter_after_group_on_right has an unsupported group " + "configuration. (Unexpected state 6.a)" + ) + self.assertIn(msg, out) def test_disallowed_grouping__group_after_parameter_on_left(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.group_after_parameter_on_left [ @@ -679,9 +689,14 @@ def test_disallowed_grouping__group_after_parameter_on_left(self): ] param: int """) + msg = ( + "Function group_after_parameter_on_left has an unsupported group " + "configuration. (Unexpected state 2.b)" + ) + self.assertIn(msg, out) def test_disallowed_grouping__empty_group_on_left(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.empty_group [ @@ -691,9 +706,14 @@ def test_disallowed_grouping__empty_group_on_left(self): ] param: int """) + msg = ( + "Function empty_group has an empty group.\n" + "All groups must contain at least one parameter." + ) + self.assertIn(msg, out) def test_disallowed_grouping__empty_group_on_right(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.empty_group param: int @@ -703,6 +723,11 @@ def test_disallowed_grouping__empty_group_on_right(self): group2 : int ] """) + msg = ( + "Function empty_group has an empty group.\n" + "All groups must contain at least one parameter." + ) + self.assertIn(msg, out) def test_no_parameters(self): function = self.parse_function(""" @@ -731,69 +756,60 @@ class foo.Bar "unused" "notneeded" self.assertEqual(1, len(function.parameters)) def test_illegal_module_line(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar => int / """) + msg = "Illegal function name: foo.bar => int" + self.assertIn(msg, out) def test_illegal_c_basename(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar as 935 / """) + msg = "Illegal C basename: 935" + self.assertIn(msg, out) def test_single_star(self): - self.parse_function_should_fail(""" - module foo - foo.bar - * - * - """) - - def test_parameters_required_after_star_without_initial_parameters_or_docstring(self): - self.parse_function_should_fail(""" - module foo - foo.bar - * - """) - - def test_parameters_required_after_star_without_initial_parameters_with_docstring(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar * - Docstring here. - """) - - def test_parameters_required_after_star_with_initial_parameters_without_docstring(self): - self.parse_function_should_fail(""" - module foo - foo.bar - this: int * """) + self.assertIn("Function bar uses '*' more than once.", out) - def test_parameters_required_after_star_with_initial_parameters_and_docstring(self): - self.parse_function_should_fail(""" - module foo - foo.bar - this: int - * - Docstring. - """) + def test_parameters_required_after_star(self): + dataset = ( + "module foo\nfoo.bar\n *", + "module foo\nfoo.bar\n *\nDocstring here.", + "module foo\nfoo.bar\n this: int\n *", + "module foo\nfoo.bar\n this: int\n *\nDocstring.", + ) + msg = "Function bar specifies '*' without any parameters afterwards." + for block in dataset: + with self.subTest(block=block): + out = self.parse_function_should_fail(block) + self.assertIn(msg, out) def test_single_slash(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar / / """) + msg = ( + "Function bar has an unsupported group configuration. " + "(Unexpected state 0.d)" + ) + self.assertIn(msg, out) def test_mix_star_and_slash(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar x: int @@ -802,14 +818,24 @@ def test_mix_star_and_slash(self): z: int / """) + msg = ( + "Function bar mixes keyword-only and positional-only parameters, " + "which is unsupported." + ) + self.assertIn(msg, out) def test_parameters_not_permitted_after_slash_for_now(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar / x: int """) + msg = ( + "Function bar has an unsupported group configuration. " + "(Unexpected state 0.d)" + ) + self.assertIn(msg, out) def test_parameters_no_more_than_one_vararg(self): expected_msg = ( From webhook-mailer at python.org Mon Jul 3 19:37:51 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 23:37:51 -0000 Subject: [Python-checkins] [3.12] gh-106368: Harden Argument Clinic parser tests (GH-106384) (#106387) Message-ID: https://github.com/python/cpython/commit/6720003dae318c417687a8d8b1ecf54b7cad6554 commit: 6720003dae318c417687a8d8b1ecf54b7cad6554 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-03T23:37:47Z summary: [3.12] gh-106368: Harden Argument Clinic parser tests (GH-106384) (#106387) (cherry picked from commit 648688c137744a623a71dc2413d2879b80c99eae) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index b3602887ab635..c5cfe53e0df99 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -643,7 +643,7 @@ def test_disallowed_grouping__two_top_groups_on_left(self): self.assertEqual(out, expected_msg) def test_disallowed_grouping__two_top_groups_on_right(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.two_top_groups_on_right param: int @@ -654,9 +654,14 @@ def test_disallowed_grouping__two_top_groups_on_right(self): group2 : int ] """) + msg = ( + "Function two_top_groups_on_right has an unsupported group " + "configuration. (Unexpected state 6.b)" + ) + self.assertIn(msg, out) def test_disallowed_grouping__parameter_after_group_on_right(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.parameter_after_group_on_right param: int @@ -667,9 +672,14 @@ def test_disallowed_grouping__parameter_after_group_on_right(self): group2 : int ] """) + msg = ( + "Function parameter_after_group_on_right has an unsupported group " + "configuration. (Unexpected state 6.a)" + ) + self.assertIn(msg, out) def test_disallowed_grouping__group_after_parameter_on_left(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.group_after_parameter_on_left [ @@ -680,9 +690,14 @@ def test_disallowed_grouping__group_after_parameter_on_left(self): ] param: int """) + msg = ( + "Function group_after_parameter_on_left has an unsupported group " + "configuration. (Unexpected state 2.b)" + ) + self.assertIn(msg, out) def test_disallowed_grouping__empty_group_on_left(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.empty_group [ @@ -692,9 +707,14 @@ def test_disallowed_grouping__empty_group_on_left(self): ] param: int """) + msg = ( + "Function empty_group has an empty group.\n" + "All groups must contain at least one parameter." + ) + self.assertIn(msg, out) def test_disallowed_grouping__empty_group_on_right(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.empty_group param: int @@ -704,6 +724,11 @@ def test_disallowed_grouping__empty_group_on_right(self): group2 : int ] """) + msg = ( + "Function empty_group has an empty group.\n" + "All groups must contain at least one parameter." + ) + self.assertIn(msg, out) def test_no_parameters(self): function = self.parse_function(""" @@ -732,69 +757,60 @@ class foo.Bar "unused" "notneeded" self.assertEqual(1, len(function.parameters)) def test_illegal_module_line(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar => int / """) + msg = "Illegal function name: foo.bar => int" + self.assertIn(msg, out) def test_illegal_c_basename(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar as 935 / """) + msg = "Illegal C basename: 935" + self.assertIn(msg, out) def test_single_star(self): - self.parse_function_should_fail(""" - module foo - foo.bar - * - * - """) - - def test_parameters_required_after_star_without_initial_parameters_or_docstring(self): - self.parse_function_should_fail(""" - module foo - foo.bar - * - """) - - def test_parameters_required_after_star_without_initial_parameters_with_docstring(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar * - Docstring here. - """) - - def test_parameters_required_after_star_with_initial_parameters_without_docstring(self): - self.parse_function_should_fail(""" - module foo - foo.bar - this: int * """) + self.assertIn("Function bar uses '*' more than once.", out) - def test_parameters_required_after_star_with_initial_parameters_and_docstring(self): - self.parse_function_should_fail(""" - module foo - foo.bar - this: int - * - Docstring. - """) + def test_parameters_required_after_star(self): + dataset = ( + "module foo\nfoo.bar\n *", + "module foo\nfoo.bar\n *\nDocstring here.", + "module foo\nfoo.bar\n this: int\n *", + "module foo\nfoo.bar\n this: int\n *\nDocstring.", + ) + msg = "Function bar specifies '*' without any parameters afterwards." + for block in dataset: + with self.subTest(block=block): + out = self.parse_function_should_fail(block) + self.assertIn(msg, out) def test_single_slash(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar / / """) + msg = ( + "Function bar has an unsupported group configuration. " + "(Unexpected state 0.d)" + ) + self.assertIn(msg, out) def test_mix_star_and_slash(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar x: int @@ -803,14 +819,24 @@ def test_mix_star_and_slash(self): z: int / """) + msg = ( + "Function bar mixes keyword-only and positional-only parameters, " + "which is unsupported." + ) + self.assertIn(msg, out) def test_parameters_not_permitted_after_slash_for_now(self): - self.parse_function_should_fail(""" + out = self.parse_function_should_fail(""" module foo foo.bar / x: int """) + msg = ( + "Function bar has an unsupported group configuration. " + "(Unexpected state 0.d)" + ) + self.assertIn(msg, out) def test_parameters_no_more_than_one_vararg(self): expected_msg = ( From webhook-mailer at python.org Mon Jul 3 19:37:51 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 03 Jul 2023 23:37:51 -0000 Subject: [Python-checkins] gh-106320: Fix _PyImport_GetModuleAttr() declaration (#106386) Message-ID: https://github.com/python/cpython/commit/b4256135809d78d342e9d92e8bc3f527d3d3057f commit: b4256135809d78d342e9d92e8bc3f527d3d3057f branch: main author: Victor Stinner committer: vstinner date: 2023-07-03T23:37:48Z summary: gh-106320: Fix _PyImport_GetModuleAttr() declaration (#106386) Replace PyAPI_DATA() with PyAPI_FUNC(). files: M Include/internal/pycore_import.h diff --git a/Include/internal/pycore_import.h b/Include/internal/pycore_import.h index 457a654aff464..c048ae88d9000 100644 --- a/Include/internal/pycore_import.h +++ b/Include/internal/pycore_import.h @@ -24,8 +24,8 @@ extern int _PyImport_FixupBuiltin( extern int _PyImport_FixupExtensionObject(PyObject*, PyObject *, PyObject *, PyObject *); -PyAPI_DATA(PyObject *) _PyImport_GetModuleAttr(PyObject *, PyObject *); -PyAPI_DATA(PyObject *) _PyImport_GetModuleAttrString(const char *, const char *); +PyAPI_FUNC(PyObject *) _PyImport_GetModuleAttr(PyObject *, PyObject *); +PyAPI_FUNC(PyObject *) _PyImport_GetModuleAttrString(const char *, const char *); struct _import_runtime_state { From webhook-mailer at python.org Mon Jul 3 19:58:30 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 03 Jul 2023 23:58:30 -0000 Subject: [Python-checkins] gh-106368: Increase Argument Clinic test coverage (#106389) Message-ID: https://github.com/python/cpython/commit/3406f8cce542ea4edf4153c0fac5216df283a9b1 commit: 3406f8cce542ea4edf4153c0fac5216df283a9b1 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-03T23:58:27Z summary: gh-106368: Increase Argument Clinic test coverage (#106389) Add: - test_disallowed_gropuing__no_matching_bracket - test_double_slash files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index c5cfe53e0df99..03754d0bf123b 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -730,6 +730,18 @@ def test_disallowed_grouping__empty_group_on_right(self): ) self.assertIn(msg, out) + def test_disallowed_grouping__no_matching_bracket(self): + out = self.parse_function_should_fail(""" + module foo + foo.empty_group + param: int + ] + group2: int + ] + """) + msg = "Function empty_group has a ] without a matching [." + self.assertIn(msg, out) + def test_no_parameters(self): function = self.parse_function(""" module foo @@ -809,6 +821,18 @@ def test_single_slash(self): ) self.assertIn(msg, out) + def test_double_slash(self): + out = self.parse_function_should_fail(""" + module foo + foo.bar + a: int + / + b: int + / + """) + msg = "Function bar uses '/' more than once." + self.assertIn(msg, out) + def test_mix_star_and_slash(self): out = self.parse_function_should_fail(""" module foo From webhook-mailer at python.org Mon Jul 3 20:22:35 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 04 Jul 2023 00:22:35 -0000 Subject: [Python-checkins] [3.11] gh-106368: Increase Argument Clinic test coverage (GH-106389) (#106391) Message-ID: https://github.com/python/cpython/commit/3cc5c4a073d63534e45fc08d7906a77180e8aaa3 commit: 3cc5c4a073d63534e45fc08d7906a77180e8aaa3 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-04T00:22:32Z summary: [3.11] gh-106368: Increase Argument Clinic test coverage (GH-106389) (#106391) Add: - test_disallowed_gropuing__no_matching_bracket - test_double_slash (cherry picked from commit 3406f8cce542ea4edf4153c0fac5216df283a9b1) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index f107e14e38f20..dafdfc2240d0d 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -729,6 +729,18 @@ def test_disallowed_grouping__empty_group_on_right(self): ) self.assertIn(msg, out) + def test_disallowed_grouping__no_matching_bracket(self): + out = self.parse_function_should_fail(""" + module foo + foo.empty_group + param: int + ] + group2: int + ] + """) + msg = "Function empty_group has a ] without a matching [." + self.assertIn(msg, out) + def test_no_parameters(self): function = self.parse_function(""" module foo @@ -808,6 +820,18 @@ def test_single_slash(self): ) self.assertIn(msg, out) + def test_double_slash(self): + out = self.parse_function_should_fail(""" + module foo + foo.bar + a: int + / + b: int + / + """) + msg = "Function bar uses '/' more than once." + self.assertIn(msg, out) + def test_mix_star_and_slash(self): out = self.parse_function_should_fail(""" module foo From webhook-mailer at python.org Mon Jul 3 20:26:48 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 04 Jul 2023 00:26:48 -0000 Subject: [Python-checkins] [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106389) (#106390) Message-ID: https://github.com/python/cpython/commit/b84365fe3eaa52bac8ffa7369ad0496a807aa8a7 commit: b84365fe3eaa52bac8ffa7369ad0496a807aa8a7 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-04T00:26:44Z summary: [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106389) (#106390) Add: - test_disallowed_gropuing__no_matching_bracket - test_double_slash (cherry picked from commit 3406f8cce542ea4edf4153c0fac5216df283a9b1) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index c5cfe53e0df99..03754d0bf123b 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -730,6 +730,18 @@ def test_disallowed_grouping__empty_group_on_right(self): ) self.assertIn(msg, out) + def test_disallowed_grouping__no_matching_bracket(self): + out = self.parse_function_should_fail(""" + module foo + foo.empty_group + param: int + ] + group2: int + ] + """) + msg = "Function empty_group has a ] without a matching [." + self.assertIn(msg, out) + def test_no_parameters(self): function = self.parse_function(""" module foo @@ -809,6 +821,18 @@ def test_single_slash(self): ) self.assertIn(msg, out) + def test_double_slash(self): + out = self.parse_function_should_fail(""" + module foo + foo.bar + a: int + / + b: int + / + """) + msg = "Function bar uses '/' more than once." + self.assertIn(msg, out) + def test_mix_star_and_slash(self): out = self.parse_function_should_fail(""" module foo From webhook-mailer at python.org Tue Jul 4 03:29:56 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 04 Jul 2023 07:29:56 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyUnicode codecs C API functions (#106385) Message-ID: https://github.com/python/cpython/commit/d8c5d76da2d5c3e8f9c05fcfc59dc1aaaa1fe6e1 commit: d8c5d76da2d5c3e8f9c05fcfc59dc1aaaa1fe6e1 branch: main author: Victor Stinner committer: vstinner date: 2023-07-04T07:29:52Z summary: gh-106320: Remove private _PyUnicode codecs C API functions (#106385) Remove private _PyUnicode codecs C API functions: move them to the internal C API (pycore_unicodeobject.h). No longer export most of these functions. files: M Include/cpython/unicodeobject.h M Include/internal/pycore_unicodeobject.h M Parser/string_parser.c diff --git a/Include/cpython/unicodeobject.h b/Include/cpython/unicodeobject.h index fcd9c28e030c1..dc8f6437c0e2c 100644 --- a/Include/cpython/unicodeobject.h +++ b/Include/cpython/unicodeobject.h @@ -461,112 +461,6 @@ PyAPI_FUNC(const char *) PyUnicode_AsUTF8(PyObject *unicode); #define _PyUnicode_AsString PyUnicode_AsUTF8 -/* --- UTF-7 Codecs ------------------------------------------------------- */ - -PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF7( - PyObject *unicode, /* Unicode object */ - int base64SetO, /* Encode RFC2152 Set O characters in base64 */ - int base64WhiteSpace, /* Encode whitespace (sp, ht, nl, cr) in base64 */ - const char *errors /* error handling */ - ); - -/* --- UTF-8 Codecs ------------------------------------------------------- */ - -PyAPI_FUNC(PyObject*) _PyUnicode_AsUTF8String( - PyObject *unicode, - const char *errors); - -/* --- UTF-32 Codecs ------------------------------------------------------ */ - -PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF32( - PyObject *object, /* Unicode object */ - const char *errors, /* error handling */ - int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ - ); - -/* --- UTF-16 Codecs ------------------------------------------------------ */ - -/* Returns a Python string object holding the UTF-16 encoded value of - the Unicode data. - - If byteorder is not 0, output is written according to the following - byte order: - - byteorder == -1: little endian - byteorder == 0: native byte order (writes a BOM mark) - byteorder == 1: big endian - - If byteorder is 0, the output string will always start with the - Unicode BOM mark (U+FEFF). In the other two modes, no BOM mark is - prepended. -*/ -PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF16( - PyObject* unicode, /* Unicode object */ - const char *errors, /* error handling */ - int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ - ); - -/* --- Unicode-Escape Codecs ---------------------------------------------- */ - -/* Variant of PyUnicode_DecodeUnicodeEscape that supports partial decoding. */ -PyAPI_FUNC(PyObject*) _PyUnicode_DecodeUnicodeEscapeStateful( - const char *string, /* Unicode-Escape encoded string */ - Py_ssize_t length, /* size of string */ - const char *errors, /* error handling */ - Py_ssize_t *consumed /* bytes consumed */ -); -/* Helper for PyUnicode_DecodeUnicodeEscape that detects invalid escape - chars. */ -PyAPI_FUNC(PyObject*) _PyUnicode_DecodeUnicodeEscapeInternal( - const char *string, /* Unicode-Escape encoded string */ - Py_ssize_t length, /* size of string */ - const char *errors, /* error handling */ - Py_ssize_t *consumed, /* bytes consumed */ - const char **first_invalid_escape /* on return, points to first - invalid escaped char in - string. */ -); - -/* --- Raw-Unicode-Escape Codecs ---------------------------------------------- */ - -/* Variant of PyUnicode_DecodeRawUnicodeEscape that supports partial decoding. */ -PyAPI_FUNC(PyObject*) _PyUnicode_DecodeRawUnicodeEscapeStateful( - const char *string, /* Unicode-Escape encoded string */ - Py_ssize_t length, /* size of string */ - const char *errors, /* error handling */ - Py_ssize_t *consumed /* bytes consumed */ -); - -/* --- Latin-1 Codecs ----------------------------------------------------- */ - -PyAPI_FUNC(PyObject*) _PyUnicode_AsLatin1String( - PyObject* unicode, - const char* errors); - -/* --- ASCII Codecs ------------------------------------------------------- */ - -PyAPI_FUNC(PyObject*) _PyUnicode_AsASCIIString( - PyObject* unicode, - const char* errors); - -/* --- Character Map Codecs ----------------------------------------------- */ - -/* Translate an Unicode object by applying a character mapping table to - it and return the resulting Unicode object. - - The mapping table must map Unicode ordinal integers to Unicode strings, - Unicode ordinal integers or None (causing deletion of the character). - - Mapping tables may be dictionaries or sequences. Unmapped character - ordinals (ones which cause a LookupError) are left untouched and - are copied as-is. -*/ -PyAPI_FUNC(PyObject*) _PyUnicode_EncodeCharmap( - PyObject *unicode, /* Unicode object */ - PyObject *mapping, /* encoding mapping */ - const char *errors /* error handling */ - ); - /* --- Decimal Encoder ---------------------------------------------------- */ /* Coverts a Unicode object holding a decimal value to an ASCII string diff --git a/Include/internal/pycore_unicodeobject.h b/Include/internal/pycore_unicodeobject.h index da01f57f96279..dd20ac19d413b 100644 --- a/Include/internal/pycore_unicodeobject.h +++ b/Include/internal/pycore_unicodeobject.h @@ -177,6 +177,106 @@ PyAPI_FUNC(int) _PyUnicode_FormatAdvancedWriter( Py_ssize_t start, Py_ssize_t end); +/* --- UTF-7 Codecs ------------------------------------------------------- */ + +extern PyObject* _PyUnicode_EncodeUTF7( + PyObject *unicode, /* Unicode object */ + int base64SetO, /* Encode RFC2152 Set O characters in base64 */ + int base64WhiteSpace, /* Encode whitespace (sp, ht, nl, cr) in base64 */ + const char *errors); /* error handling */ + +/* --- UTF-8 Codecs ------------------------------------------------------- */ + +PyAPI_FUNC(PyObject*) _PyUnicode_AsUTF8String( + PyObject *unicode, + const char *errors); + +/* --- UTF-32 Codecs ------------------------------------------------------ */ + +PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF32( + PyObject *object, /* Unicode object */ + const char *errors, /* error handling */ + int byteorder); /* byteorder to use 0=BOM+native;-1=LE,1=BE */ + +/* --- UTF-16 Codecs ------------------------------------------------------ */ + +/* Returns a Python string object holding the UTF-16 encoded value of + the Unicode data. + + If byteorder is not 0, output is written according to the following + byte order: + + byteorder == -1: little endian + byteorder == 0: native byte order (writes a BOM mark) + byteorder == 1: big endian + + If byteorder is 0, the output string will always start with the + Unicode BOM mark (U+FEFF). In the other two modes, no BOM mark is + prepended. +*/ +PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF16( + PyObject* unicode, /* Unicode object */ + const char *errors, /* error handling */ + int byteorder); /* byteorder to use 0=BOM+native;-1=LE,1=BE */ + +/* --- Unicode-Escape Codecs ---------------------------------------------- */ + +/* Variant of PyUnicode_DecodeUnicodeEscape that supports partial decoding. */ +extern PyObject* _PyUnicode_DecodeUnicodeEscapeStateful( + const char *string, /* Unicode-Escape encoded string */ + Py_ssize_t length, /* size of string */ + const char *errors, /* error handling */ + Py_ssize_t *consumed); /* bytes consumed */ + +/* Helper for PyUnicode_DecodeUnicodeEscape that detects invalid escape + chars. */ +PyAPI_FUNC(PyObject*) _PyUnicode_DecodeUnicodeEscapeInternal( + const char *string, /* Unicode-Escape encoded string */ + Py_ssize_t length, /* size of string */ + const char *errors, /* error handling */ + Py_ssize_t *consumed, /* bytes consumed */ + const char **first_invalid_escape); /* on return, points to first + invalid escaped char in + string. */ + +/* --- Raw-Unicode-Escape Codecs ---------------------------------------------- */ + +/* Variant of PyUnicode_DecodeRawUnicodeEscape that supports partial decoding. */ +extern PyObject* _PyUnicode_DecodeRawUnicodeEscapeStateful( + const char *string, /* Unicode-Escape encoded string */ + Py_ssize_t length, /* size of string */ + const char *errors, /* error handling */ + Py_ssize_t *consumed); /* bytes consumed */ + +/* --- Latin-1 Codecs ----------------------------------------------------- */ + +extern PyObject* _PyUnicode_AsLatin1String( + PyObject* unicode, + const char* errors); + +/* --- ASCII Codecs ------------------------------------------------------- */ + +extern PyObject* _PyUnicode_AsASCIIString( + PyObject* unicode, + const char* errors); + +/* --- Character Map Codecs ----------------------------------------------- */ + +/* Translate an Unicode object by applying a character mapping table to + it and return the resulting Unicode object. + + The mapping table must map Unicode ordinal integers to Unicode strings, + Unicode ordinal integers or None (causing deletion of the character). + + Mapping tables may be dictionaries or sequences. Unmapped character + ordinals (ones which cause a LookupError) are left untouched and + are copied as-is. +*/ +extern PyObject* _PyUnicode_EncodeCharmap( + PyObject *unicode, /* Unicode object */ + PyObject *mapping, /* encoding mapping */ + const char *errors); /* error handling */ + /* --- Methods & Slots ---------------------------------------------------- */ extern PyObject* _PyUnicode_JoinArray( diff --git a/Parser/string_parser.c b/Parser/string_parser.c index 20459e8946349..bc1f99d607ae4 100644 --- a/Parser/string_parser.c +++ b/Parser/string_parser.c @@ -1,6 +1,7 @@ #include #include +#include "pycore_unicodeobject.h" // _PyUnicode_DecodeUnicodeEscapeInternal() #include "tokenizer.h" #include "pegen.h" From webhook-mailer at python.org Tue Jul 4 04:27:27 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 04 Jul 2023 08:27:27 -0000 Subject: [Python-checkins] gh-106320: Remove _PyBytesWriter C API (#106399) Message-ID: https://github.com/python/cpython/commit/ec931fc3943df0b94f2e250d7723892f2b3414bd commit: ec931fc3943df0b94f2e250d7723892f2b3414bd branch: main author: Victor Stinner committer: vstinner date: 2023-07-04T08:27:23Z summary: gh-106320: Remove _PyBytesWriter C API (#106399) Remove the _PyBytesWriter C API: move it to the internal C API (pycore_bytesobject.h). files: M Include/cpython/bytesobject.h M Include/internal/pycore_bytesobject.h M Include/internal/pycore_long.h M Modules/_pickle.c M Modules/_struct.c diff --git a/Include/cpython/bytesobject.h b/Include/cpython/bytesobject.h index e982031c107de..0af4c83b1e5bc 100644 --- a/Include/cpython/bytesobject.h +++ b/Include/cpython/bytesobject.h @@ -47,83 +47,3 @@ static inline Py_ssize_t PyBytes_GET_SIZE(PyObject *op) { /* _PyBytes_Join(sep, x) is like sep.join(x). sep must be PyBytesObject*, x must be an iterable object. */ PyAPI_FUNC(PyObject *) _PyBytes_Join(PyObject *sep, PyObject *x); - - -/* The _PyBytesWriter structure is big: it contains an embedded "stack buffer". - A _PyBytesWriter variable must be declared at the end of variables in a - function to optimize the memory allocation on the stack. */ -typedef struct { - /* bytes, bytearray or NULL (when the small buffer is used) */ - PyObject *buffer; - - /* Number of allocated size. */ - Py_ssize_t allocated; - - /* Minimum number of allocated bytes, - incremented by _PyBytesWriter_Prepare() */ - Py_ssize_t min_size; - - /* If non-zero, use a bytearray instead of a bytes object for buffer. */ - int use_bytearray; - - /* If non-zero, overallocate the buffer (default: 0). - This flag must be zero if use_bytearray is non-zero. */ - int overallocate; - - /* Stack buffer */ - int use_small_buffer; - char small_buffer[512]; -} _PyBytesWriter; - -/* Initialize a bytes writer - - By default, the overallocation is disabled. Set the overallocate attribute - to control the allocation of the buffer. */ -PyAPI_FUNC(void) _PyBytesWriter_Init(_PyBytesWriter *writer); - -/* Get the buffer content and reset the writer. - Return a bytes object, or a bytearray object if use_bytearray is non-zero. - Raise an exception and return NULL on error. */ -PyAPI_FUNC(PyObject *) _PyBytesWriter_Finish(_PyBytesWriter *writer, - void *str); - -/* Deallocate memory of a writer (clear its internal buffer). */ -PyAPI_FUNC(void) _PyBytesWriter_Dealloc(_PyBytesWriter *writer); - -/* Allocate the buffer to write size bytes. - Return the pointer to the beginning of buffer data. - Raise an exception and return NULL on error. */ -PyAPI_FUNC(void*) _PyBytesWriter_Alloc(_PyBytesWriter *writer, - Py_ssize_t size); - -/* Ensure that the buffer is large enough to write *size* bytes. - Add size to the writer minimum size (min_size attribute). - - str is the current pointer inside the buffer. - Return the updated current pointer inside the buffer. - Raise an exception and return NULL on error. */ -PyAPI_FUNC(void*) _PyBytesWriter_Prepare(_PyBytesWriter *writer, - void *str, - Py_ssize_t size); - -/* Resize the buffer to make it larger. - The new buffer may be larger than size bytes because of overallocation. - Return the updated current pointer inside the buffer. - Raise an exception and return NULL on error. - - Note: size must be greater than the number of allocated bytes in the writer. - - This function doesn't use the writer minimum size (min_size attribute). - - See also _PyBytesWriter_Prepare(). - */ -PyAPI_FUNC(void*) _PyBytesWriter_Resize(_PyBytesWriter *writer, - void *str, - Py_ssize_t size); - -/* Write bytes. - Raise an exception and return NULL on error. */ -PyAPI_FUNC(void*) _PyBytesWriter_WriteBytes(_PyBytesWriter *writer, - void *str, - const void *bytes, - Py_ssize_t size); diff --git a/Include/internal/pycore_bytesobject.h b/Include/internal/pycore_bytesobject.h index d36fa9569d64a..115c0c52c8f9a 100644 --- a/Include/internal/pycore_bytesobject.h +++ b/Include/internal/pycore_bytesobject.h @@ -41,6 +41,87 @@ PyAPI_FUNC(void) _PyBytes_Repeat(char* dest, Py_ssize_t len_dest, const char* src, Py_ssize_t len_src); +/* --- _PyBytesWriter ----------------------------------------------------- */ + +/* The _PyBytesWriter structure is big: it contains an embedded "stack buffer". + A _PyBytesWriter variable must be declared at the end of variables in a + function to optimize the memory allocation on the stack. */ +typedef struct { + /* bytes, bytearray or NULL (when the small buffer is used) */ + PyObject *buffer; + + /* Number of allocated size. */ + Py_ssize_t allocated; + + /* Minimum number of allocated bytes, + incremented by _PyBytesWriter_Prepare() */ + Py_ssize_t min_size; + + /* If non-zero, use a bytearray instead of a bytes object for buffer. */ + int use_bytearray; + + /* If non-zero, overallocate the buffer (default: 0). + This flag must be zero if use_bytearray is non-zero. */ + int overallocate; + + /* Stack buffer */ + int use_small_buffer; + char small_buffer[512]; +} _PyBytesWriter; + +/* Initialize a bytes writer + + By default, the overallocation is disabled. Set the overallocate attribute + to control the allocation of the buffer. */ +PyAPI_FUNC(void) _PyBytesWriter_Init(_PyBytesWriter *writer); + +/* Get the buffer content and reset the writer. + Return a bytes object, or a bytearray object if use_bytearray is non-zero. + Raise an exception and return NULL on error. */ +PyAPI_FUNC(PyObject *) _PyBytesWriter_Finish(_PyBytesWriter *writer, + void *str); + +/* Deallocate memory of a writer (clear its internal buffer). */ +PyAPI_FUNC(void) _PyBytesWriter_Dealloc(_PyBytesWriter *writer); + +/* Allocate the buffer to write size bytes. + Return the pointer to the beginning of buffer data. + Raise an exception and return NULL on error. */ +PyAPI_FUNC(void*) _PyBytesWriter_Alloc(_PyBytesWriter *writer, + Py_ssize_t size); + +/* Ensure that the buffer is large enough to write *size* bytes. + Add size to the writer minimum size (min_size attribute). + + str is the current pointer inside the buffer. + Return the updated current pointer inside the buffer. + Raise an exception and return NULL on error. */ +PyAPI_FUNC(void*) _PyBytesWriter_Prepare(_PyBytesWriter *writer, + void *str, + Py_ssize_t size); + +/* Resize the buffer to make it larger. + The new buffer may be larger than size bytes because of overallocation. + Return the updated current pointer inside the buffer. + Raise an exception and return NULL on error. + + Note: size must be greater than the number of allocated bytes in the writer. + + This function doesn't use the writer minimum size (min_size attribute). + + See also _PyBytesWriter_Prepare(). + */ +PyAPI_FUNC(void*) _PyBytesWriter_Resize(_PyBytesWriter *writer, + void *str, + Py_ssize_t size); + +/* Write bytes. + Raise an exception and return NULL on error. */ +PyAPI_FUNC(void*) _PyBytesWriter_WriteBytes(_PyBytesWriter *writer, + void *str, + const void *bytes, + Py_ssize_t size); + #ifdef __cplusplus } #endif diff --git a/Include/internal/pycore_long.h b/Include/internal/pycore_long.h index 64c00cb147548..3f01694e5f5ac 100644 --- a/Include/internal/pycore_long.h +++ b/Include/internal/pycore_long.h @@ -8,7 +8,8 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -#include "pycore_global_objects.h" // _PY_NSMALLNEGINTS +#include "pycore_bytesobject.h" // _PyBytesWriter +#include "pycore_global_objects.h"// _PY_NSMALLNEGINTS #include "pycore_runtime.h" // _PyRuntime /* diff --git a/Modules/_pickle.c b/Modules/_pickle.c index 4913a8dfee589..a68a0aaa64c2b 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -9,10 +9,11 @@ #endif #include "Python.h" +#include "pycore_bytesobject.h" // _PyBytesWriter #include "pycore_ceval.h" // _Py_EnterRecursiveCall() #include "pycore_moduleobject.h" // _PyModule_GetState() -#include "pycore_runtime.h" // _Py_ID() #include "pycore_pystate.h" // _PyThreadState_GET() +#include "pycore_runtime.h" // _Py_ID() #include "structmember.h" // PyMemberDef #include // strtol() diff --git a/Modules/_struct.c b/Modules/_struct.c index 0a6f076aac0c5..31c94927e91d6 100644 --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -8,6 +8,7 @@ #endif #include "Python.h" +#include "pycore_bytesobject.h" // _PyBytesWriter #include "pycore_moduleobject.h" // _PyModule_GetState() #include "structmember.h" // PyMemberDef #include From webhook-mailer at python.org Tue Jul 4 04:59:13 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 04 Jul 2023 08:59:13 -0000 Subject: [Python-checkins] gh-106320: Remove _PyUnicode_TransformDecimalAndSpaceToASCII() (#106398) Message-ID: https://github.com/python/cpython/commit/8a73b57b9b5f6e36dd5a4c279f4d606d9e71a31f commit: 8a73b57b9b5f6e36dd5a4c279f4d606d9e71a31f branch: main author: Victor Stinner committer: vstinner date: 2023-07-04T08:59:09Z summary: gh-106320: Remove _PyUnicode_TransformDecimalAndSpaceToASCII() (#106398) Remove private _PyUnicode_TransformDecimalAndSpaceToASCII() and other private _PyUnicode C API functions: move them to the internal C API (pycore_unicodeobject.h). No longer most of these functions. Replace _testcapi.unicode_transformdecimalandspacetoascii() with _testinternal._PyUnicode_TransformDecimalAndSpaceToASCII(). files: M Include/cpython/unicodeobject.h M Include/internal/pycore_unicodeobject.h M Lib/test/test_capi/test_unicode.py M Modules/_testcapi/unicode.c M Modules/_testinternalcapi.c M Python/pystrhex.c diff --git a/Include/cpython/unicodeobject.h b/Include/cpython/unicodeobject.h index dc8f6437c0e2c..e75b5e154943d 100644 --- a/Include/cpython/unicodeobject.h +++ b/Include/cpython/unicodeobject.h @@ -167,10 +167,6 @@ typedef struct { } data; /* Canonical, smallest-form Unicode buffer */ } PyUnicodeObject; -PyAPI_FUNC(int) _PyUnicode_CheckConsistency( - PyObject *op, - int check_content); - #define _PyASCIIObject_CAST(op) \ (assert(PyUnicode_Check(op)), \ @@ -461,19 +457,6 @@ PyAPI_FUNC(const char *) PyUnicode_AsUTF8(PyObject *unicode); #define _PyUnicode_AsString PyUnicode_AsUTF8 -/* --- Decimal Encoder ---------------------------------------------------- */ - -/* Coverts a Unicode object holding a decimal value to an ASCII string - for using in int, float and complex parsers. - Transforms code points that have decimal digit property to the - corresponding ASCII digit code points. Transforms spaces to ASCII. - Transforms code points starting from the first non-ASCII code point that - is neither a decimal digit nor a space to the end into '?'. */ - -PyAPI_FUNC(PyObject*) _PyUnicode_TransformDecimalAndSpaceToASCII( - PyObject *unicode /* Unicode object */ - ); - /* === Characters Type APIs =============================================== */ /* These should not be used directly. Use the Py_UNICODE_IS* and @@ -623,23 +606,3 @@ static inline int Py_UNICODE_ISALNUM(Py_UCS4 ch) { || Py_UNICODE_ISDIGIT(ch) || Py_UNICODE_ISNUMERIC(ch)); } - - -/* === Misc functions ===================================================== */ - -PyAPI_FUNC(PyObject*) _PyUnicode_FormatLong(PyObject *, int, int, int); - -/* Return an interned Unicode object for an Identifier; may fail if there is no memory.*/ -PyAPI_FUNC(PyObject*) _PyUnicode_FromId(_Py_Identifier*); - -/* Fast equality check when the inputs are known to be exact unicode types - and where the hash values are equal (i.e. a very probable match) */ -PyAPI_FUNC(int) _PyUnicode_EQ(PyObject *, PyObject *); - -/* Equality check. */ -PyAPI_FUNC(int) _PyUnicode_Equal(PyObject *, PyObject *); - -PyAPI_FUNC(int) _PyUnicode_WideCharString_Converter(PyObject *, void *); -PyAPI_FUNC(int) _PyUnicode_WideCharString_Opt_Converter(PyObject *, void *); - -PyAPI_FUNC(Py_ssize_t) _PyUnicode_ScanIdentifier(PyObject *); diff --git a/Include/internal/pycore_unicodeobject.h b/Include/internal/pycore_unicodeobject.h index dd20ac19d413b..ad59c3e385f2d 100644 --- a/Include/internal/pycore_unicodeobject.h +++ b/Include/internal/pycore_unicodeobject.h @@ -11,8 +11,12 @@ extern "C" { #include "pycore_fileutils.h" // _Py_error_handler #include "pycore_ucnhash.h" // _PyUnicode_Name_CAPI -void _PyUnicode_ExactDealloc(PyObject *op); -Py_ssize_t _PyUnicode_InternedSize(void); +PyAPI_FUNC(int) _PyUnicode_CheckConsistency( + PyObject *op, + int check_content); + +extern void _PyUnicode_ExactDealloc(PyObject *op); +extern Py_ssize_t _PyUnicode_InternedSize(void); /* Get a copy of a Unicode string. */ PyAPI_FUNC(PyObject*) _PyUnicode_Copy( @@ -277,6 +281,18 @@ extern PyObject* _PyUnicode_EncodeCharmap( PyObject *mapping, /* encoding mapping */ const char *errors); /* error handling */ +/* --- Decimal Encoder ---------------------------------------------------- */ + +/* Coverts a Unicode object holding a decimal value to an ASCII string + for using in int, float and complex parsers. + Transforms code points that have decimal digit property to the + corresponding ASCII digit code points. Transforms spaces to ASCII. + Transforms code points starting from the first non-ASCII code point that + is neither a decimal digit nor a space to the end into '?'. */ + +PyAPI_FUNC(PyObject*) _PyUnicode_TransformDecimalAndSpaceToASCII( + PyObject *unicode); /* Unicode object */ + /* --- Methods & Slots ---------------------------------------------------- */ extern PyObject* _PyUnicode_JoinArray( @@ -323,6 +339,25 @@ extern Py_ssize_t _PyUnicode_InsertThousandsGrouping( PyObject *thousands_sep, Py_UCS4 *maxchar); +/* --- Misc functions ----------------------------------------------------- */ + +extern PyObject* _PyUnicode_FormatLong(PyObject *, int, int, int); + +/* Return an interned Unicode object for an Identifier; may fail if there is no memory.*/ +PyAPI_FUNC(PyObject*) _PyUnicode_FromId(_Py_Identifier*); + +/* Fast equality check when the inputs are known to be exact unicode types + and where the hash values are equal (i.e. a very probable match) */ +extern int _PyUnicode_EQ(PyObject *, PyObject *); + +/* Equality check. */ +PyAPI_FUNC(int) _PyUnicode_Equal(PyObject *, PyObject *); + +extern int _PyUnicode_WideCharString_Converter(PyObject *, void *); +extern int _PyUnicode_WideCharString_Opt_Converter(PyObject *, void *); + +PyAPI_FUNC(Py_ssize_t) _PyUnicode_ScanIdentifier(PyObject *); + /* --- Runtime lifecycle -------------------------------------------------- */ extern void _PyUnicode_InitState(PyInterpreterState *); diff --git a/Lib/test/test_capi/test_unicode.py b/Lib/test/test_capi/test_unicode.py index ca914459a62be..622ee8993907f 100644 --- a/Lib/test/test_capi/test_unicode.py +++ b/Lib/test/test_capi/test_unicode.py @@ -7,6 +7,10 @@ import _testcapi except ImportError: _testcapi = None +try: + import _testinternalcapi +except ImportError: + _testinternalcapi = None NULL = None @@ -913,10 +917,10 @@ def test_getdefaultencoding(self): self.assertEqual(getdefaultencoding(), b'utf-8') @support.cpython_only - @unittest.skipIf(_testcapi is None, 'need _testcapi module') + @unittest.skipIf(_testinternalcapi is None, 'need _testinternalcapi module') def test_transform_decimal_and_space(self): """Test _PyUnicode_TransformDecimalAndSpaceToASCII()""" - from _testcapi import unicode_transformdecimalandspacetoascii as transform_decimal + from _testinternalcapi import _PyUnicode_TransformDecimalAndSpaceToASCII as transform_decimal self.assertEqual(transform_decimal('123'), '123') diff --git a/Modules/_testcapi/unicode.c b/Modules/_testcapi/unicode.c index 9c2760c3f763a..51d741a6b5ff1 100644 --- a/Modules/_testcapi/unicode.c +++ b/Modules/_testcapi/unicode.c @@ -660,14 +660,6 @@ unicode_getdefaultencoding(PyObject *self, PyObject *Py_UNUSED(ignored)) return PyBytes_FromString(s); } -/* Test _PyUnicode_TransformDecimalAndSpaceToASCII() */ -static PyObject * -unicode_transformdecimalandspacetoascii(PyObject *self, PyObject *arg) -{ - NULLABLE(arg); - return _PyUnicode_TransformDecimalAndSpaceToASCII(arg); -} - /* Test PyUnicode_DecodeUTF8() */ static PyObject * unicode_decodeutf8(PyObject *self, PyObject *args) @@ -1544,7 +1536,6 @@ static PyMethodDef TestMethods[] = { {"unicode_decodeutf8", unicode_decodeutf8, METH_VARARGS}, {"unicode_decodeutf8stateful",unicode_decodeutf8stateful, METH_VARARGS}, {"unicode_getdefaultencoding",unicode_getdefaultencoding, METH_NOARGS}, - {"unicode_transformdecimalandspacetoascii", unicode_transformdecimalandspacetoascii, METH_O}, {"unicode_concat", unicode_concat, METH_VARARGS}, {"unicode_splitlines", unicode_splitlines, METH_VARARGS}, {"unicode_split", unicode_split, METH_VARARGS}, diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 4875ee7bed168..14f91e8da1716 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -1253,6 +1253,17 @@ test_tstate_capi(PyObject *self, PyObject *Py_UNUSED(args)) } +/* Test _PyUnicode_TransformDecimalAndSpaceToASCII() */ +static PyObject * +unicode_transformdecimalandspacetoascii(PyObject *self, PyObject *arg) +{ + if (arg == Py_None) { + arg = NULL; + } + return _PyUnicode_TransformDecimalAndSpaceToASCII(arg); +} + + static PyMethodDef module_functions[] = { {"get_configs", get_configs, METH_NOARGS}, {"get_recursion_depth", get_recursion_depth, METH_NOARGS}, @@ -1304,6 +1315,7 @@ static PyMethodDef module_functions[] = { {"_PyTime_ObjectToTimeval", test_pytime_object_to_timeval, METH_VARARGS}, {"_PyTraceMalloc_GetTraceback", tracemalloc_get_traceback, METH_VARARGS}, {"test_tstate_capi", test_tstate_capi, METH_NOARGS, NULL}, + {"_PyUnicode_TransformDecimalAndSpaceToASCII", unicode_transformdecimalandspacetoascii, METH_O}, {NULL, NULL} /* sentinel */ }; diff --git a/Python/pystrhex.c b/Python/pystrhex.c index f798256e18ebd..ce456b79f1655 100644 --- a/Python/pystrhex.c +++ b/Python/pystrhex.c @@ -2,6 +2,7 @@ #include "Python.h" #include "pycore_strhex.h" // _Py_strhex_with_sep() +#include "pycore_unicodeobject.h" // _PyUnicode_CheckConsistency() #include // abs() static PyObject *_Py_strhex_impl(const char* argbuf, const Py_ssize_t arglen, From webhook-mailer at python.org Tue Jul 4 05:41:47 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 04 Jul 2023 09:41:47 -0000 Subject: [Python-checkins] gh-106320: Remove private pylifecycle.h functions (#106400) Message-ID: https://github.com/python/cpython/commit/c9ce983ae1a361f431a0303aeb6f4b8e1d674275 commit: c9ce983ae1a361f431a0303aeb6f4b8e1d674275 branch: main author: Victor Stinner committer: vstinner date: 2023-07-04T09:41:43Z summary: gh-106320: Remove private pylifecycle.h functions (#106400) Remove private pylifecycle.h functions: move them to the internal C API ( pycore_atexit.h, pycore_pylifecycle.h and pycore_signal.h). No longer export most of these functions. Move _testcapi.test_atexit() to _testinternalcapi. files: M Include/cpython/pylifecycle.h M Include/internal/pycore_atexit.h M Include/internal/pycore_pylifecycle.h M Include/internal/pycore_signal.h M Modules/_asynciomodule.c M Modules/_io/bufferedio.c M Modules/_posixsubprocess.c M Modules/_randommodule.c M Modules/_sqlite/connection.c M Modules/_testcapimodule.c M Modules/_testinternalcapi.c M Modules/_xxinterpchannelsmodule.c M Modules/getbuildinfo.c M Modules/posixmodule.c M Modules/readline.c M Modules/signalmodule.c M Python/_warnings.c M Python/bootstrap_hash.c M Python/preconfig.c diff --git a/Include/cpython/pylifecycle.h b/Include/cpython/pylifecycle.h index 1ca9ee91a72b1..8af34b0564251 100644 --- a/Include/cpython/pylifecycle.h +++ b/Include/cpython/pylifecycle.h @@ -19,13 +19,13 @@ PyAPI_FUNC(PyStatus) Py_PreInitializeFromArgs( Py_ssize_t argc, wchar_t **argv); -PyAPI_FUNC(int) _Py_IsCoreInitialized(void); - /* Initialization and finalization */ PyAPI_FUNC(PyStatus) Py_InitializeFromConfig( const PyConfig *config); + +// Python 3.8 provisional API (PEP 587) PyAPI_FUNC(PyStatus) _Py_InitializeMain(void); PyAPI_FUNC(int) Py_RunMain(void); @@ -33,31 +33,8 @@ PyAPI_FUNC(int) Py_RunMain(void); PyAPI_FUNC(void) _Py_NO_RETURN Py_ExitStatusException(PyStatus err); -/* Restore signals that the interpreter has called SIG_IGN on to SIG_DFL. */ -PyAPI_FUNC(void) _Py_RestoreSignals(void); - PyAPI_FUNC(int) Py_FdIsInteractive(FILE *, const char *); -PyAPI_FUNC(int) _Py_FdIsInteractive(FILE *fp, PyObject *filename); - -PyAPI_FUNC(const char *) _Py_gitidentifier(void); -PyAPI_FUNC(const char *) _Py_gitversion(void); - -PyAPI_FUNC(int) _Py_IsFinalizing(void); -PyAPI_FUNC(int) _Py_IsInterpreterFinalizing(PyInterpreterState *interp); - -/* Random */ -PyAPI_FUNC(int) _PyOS_URandom(void *buffer, Py_ssize_t size); -PyAPI_FUNC(int) _PyOS_URandomNonblock(void *buffer, Py_ssize_t size); - -/* Legacy locale support */ -PyAPI_FUNC(int) _Py_CoerceLegacyLocale(int warn); -PyAPI_FUNC(int) _Py_LegacyLocaleDetected(int warn); -PyAPI_FUNC(char *) _Py_SetLocaleFromEnv(int category); PyAPI_FUNC(PyStatus) Py_NewInterpreterFromConfig( PyThreadState **tstate_p, const PyInterpreterConfig *config); - -typedef void (*atexit_datacallbackfunc)(void *); -PyAPI_FUNC(int) _Py_AtExit( - PyInterpreterState *, atexit_datacallbackfunc, void *); diff --git a/Include/internal/pycore_atexit.h b/Include/internal/pycore_atexit.h index 63a2cd5d507d2..fc5cb6d882643 100644 --- a/Include/internal/pycore_atexit.h +++ b/Include/internal/pycore_atexit.h @@ -25,7 +25,8 @@ struct _atexit_runtime_state { //################### // interpreter atexit -struct atexit_callback; +typedef void (*atexit_datacallbackfunc)(void *); + typedef struct atexit_callback { atexit_datacallbackfunc func; void *data; @@ -50,6 +51,10 @@ struct atexit_state { int callback_len; }; +PyAPI_FUNC(int) _Py_AtExit( + PyInterpreterState *interp, + atexit_datacallbackfunc func, + void *data); #ifdef __cplusplus } diff --git a/Include/internal/pycore_pylifecycle.h b/Include/internal/pycore_pylifecycle.h index b07c2dba8de84..fb28652515909 100644 --- a/Include/internal/pycore_pylifecycle.h +++ b/Include/internal/pycore_pylifecycle.h @@ -23,7 +23,7 @@ extern PyStatus _PyUnicode_InitEncodings(PyThreadState *tstate); extern int _PyUnicode_EnableLegacyWindowsFSEncoding(void); #endif -PyAPI_FUNC(int) _Py_IsLocaleCoercionTarget(const char *ctype_loc); +extern int _Py_IsLocaleCoercionTarget(const char *ctype_loc); /* Various one-time initializers */ @@ -67,30 +67,49 @@ extern PyStatus _PyGILState_Init(PyInterpreterState *interp); extern PyStatus _PyGILState_SetTstate(PyThreadState *tstate); extern void _PyGILState_Fini(PyInterpreterState *interp); -PyAPI_FUNC(void) _PyGC_DumpShutdownStats(PyInterpreterState *interp); +extern void _PyGC_DumpShutdownStats(PyInterpreterState *interp); -PyAPI_FUNC(PyStatus) _Py_PreInitializeFromPyArgv( +extern PyStatus _Py_PreInitializeFromPyArgv( const PyPreConfig *src_config, const struct _PyArgv *args); -PyAPI_FUNC(PyStatus) _Py_PreInitializeFromConfig( +extern PyStatus _Py_PreInitializeFromConfig( const PyConfig *config, const struct _PyArgv *args); -PyAPI_FUNC(wchar_t *) _Py_GetStdlibDir(void); +extern wchar_t * _Py_GetStdlibDir(void); -PyAPI_FUNC(int) _Py_HandleSystemExit(int *exitcode_p); +extern int _Py_HandleSystemExit(int *exitcode_p); -PyAPI_FUNC(PyObject*) _PyErr_WriteUnraisableDefaultHook(PyObject *unraisable); +extern PyObject* _PyErr_WriteUnraisableDefaultHook(PyObject *unraisable); -PyAPI_FUNC(void) _PyErr_Print(PyThreadState *tstate); -PyAPI_FUNC(void) _PyErr_Display(PyObject *file, PyObject *exception, +extern void _PyErr_Print(PyThreadState *tstate); +extern void _PyErr_Display(PyObject *file, PyObject *exception, PyObject *value, PyObject *tb); -PyAPI_FUNC(void) _PyErr_DisplayException(PyObject *file, PyObject *exc); +extern void _PyErr_DisplayException(PyObject *file, PyObject *exc); -PyAPI_FUNC(void) _PyThreadState_DeleteCurrent(PyThreadState *tstate); +extern void _PyThreadState_DeleteCurrent(PyThreadState *tstate); extern void _PyAtExit_Call(PyInterpreterState *interp); +extern int _Py_IsCoreInitialized(void); + +extern int _Py_FdIsInteractive(FILE *fp, PyObject *filename); + +extern const char* _Py_gitidentifier(void); +extern const char* _Py_gitversion(void); + +extern int _Py_IsFinalizing(void); +PyAPI_FUNC(int) _Py_IsInterpreterFinalizing(PyInterpreterState *interp); + +/* Random */ +extern int _PyOS_URandom(void *buffer, Py_ssize_t size); +PyAPI_FUNC(int) _PyOS_URandomNonblock(void *buffer, Py_ssize_t size); + +/* Legacy locale support */ +extern int _Py_CoerceLegacyLocale(int warn); +extern int _Py_LegacyLocaleDetected(int warn); +PyAPI_FUNC(char*) _Py_SetLocaleFromEnv(int category); + #ifdef __cplusplus } #endif diff --git a/Include/internal/pycore_signal.h b/Include/internal/pycore_signal.h index ca3f69d09fc0c..1a454ba6f4e8f 100644 --- a/Include/internal/pycore_signal.h +++ b/Include/internal/pycore_signal.h @@ -11,10 +11,12 @@ extern "C" { #endif #include "pycore_atomic.h" // _Py_atomic_address - #include // NSIG +/* Restore signals that the interpreter has called SIG_IGN on to SIG_DFL. */ +PyAPI_FUNC(void) _Py_RestoreSignals(void); + #ifdef _SIG_MAXSIG // gh-91145: On FreeBSD, defines NSIG as 32: it doesn't include // realtime signals: [SIGRTMIN,SIGRTMAX]. Use _SIG_MAXSIG instead. For diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 05f94ef9ed281..3843f9c45d723 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -3,10 +3,11 @@ #endif #include "Python.h" +#include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_pyerrors.h" // _PyErr_ClearExcState() +#include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_runtime_init.h" // _Py_ID() -#include "pycore_moduleobject.h" // _PyModule_GetState() #include "structmember.h" // PyMemberDef #include // offsetof() diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c index 4d120d4e8af3a..e58e87926f673 100644 --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -11,6 +11,7 @@ #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_object.h" #include "pycore_pyerrors.h" // _Py_FatalErrorFormat() +#include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing() #include "structmember.h" // PyMemberDef #include "_iomodule.h" diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index 6caa4b8852911..ac2b0d4f55468 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -6,6 +6,7 @@ #include "Python.h" #include "pycore_fileutils.h" #include "pycore_pystate.h" +#include "pycore_signal.h" // _Py_RestoreSignals() #if defined(HAVE_PIPE2) && !defined(_GNU_SOURCE) # define _GNU_SOURCE #endif @@ -739,8 +740,9 @@ child_exec(char *const exec_array[], if (child_umask >= 0) umask(child_umask); /* umask() always succeeds. */ - if (restore_signals) + if (restore_signals) { _Py_RestoreSignals(); + } #ifdef VFORK_USABLE if (child_sigmask) { diff --git a/Modules/_randommodule.c b/Modules/_randommodule.c index fda5ef267fb47..7daa1f9327966 100644 --- a/Modules/_randommodule.c +++ b/Modules/_randommodule.c @@ -72,6 +72,7 @@ #include "Python.h" #include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_pylifecycle.h" // _PyOS_URandomNonblock() #include "pycore_runtime.h" #ifdef HAVE_PROCESS_H # include // getpid() diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index d71cef14779e5..bab743674b666 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -34,6 +34,7 @@ #include "prepare_protocol.h" #include "util.h" #include "pycore_import.h" // _PyImport_GetModuleAttrString() +#include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing() #include "pycore_weakref.h" // _PyWeakref_IS_DEAD() #include diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index d1044b5445202..2baf453f71026 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -3293,37 +3293,6 @@ function_set_kw_defaults(PyObject *self, PyObject *args) Py_RETURN_NONE; } -struct atexit_data { - int called; -}; - -static void -callback(void *data) -{ - ((struct atexit_data *)data)->called += 1; -} - -static PyObject * -test_atexit(PyObject *self, PyObject *Py_UNUSED(args)) -{ - PyThreadState *oldts = PyThreadState_Swap(NULL); - PyThreadState *tstate = Py_NewInterpreter(); - - struct atexit_data data = {0}; - int res = _Py_AtExit(tstate->interp, callback, (void *)&data); - Py_EndInterpreter(tstate); - PyThreadState_Swap(oldts); - if (res < 0) { - return NULL; - } - if (data.called == 0) { - PyErr_SetString(PyExc_RuntimeError, "atexit callback not called"); - return NULL; - } - Py_RETURN_NONE; -} - - static PyObject * check_pyimport_addmodule(PyObject *self, PyObject *args) { @@ -3613,7 +3582,6 @@ static PyMethodDef TestMethods[] = { {"function_set_defaults", function_set_defaults, METH_VARARGS, NULL}, {"function_get_kw_defaults", function_get_kw_defaults, METH_O, NULL}, {"function_set_kw_defaults", function_set_kw_defaults, METH_VARARGS, NULL}, - {"test_atexit", test_atexit, METH_NOARGS}, {"check_pyimport_addmodule", check_pyimport_addmodule, METH_VARARGS}, {"test_weakref_capi", test_weakref_capi, METH_NOARGS}, {NULL, NULL} /* sentinel */ diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 14f91e8da1716..84511d27e37f4 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -1264,6 +1264,38 @@ unicode_transformdecimalandspacetoascii(PyObject *self, PyObject *arg) } +struct atexit_data { + int called; +}; + +static void +callback(void *data) +{ + ((struct atexit_data *)data)->called += 1; +} + +static PyObject * +test_atexit(PyObject *self, PyObject *Py_UNUSED(args)) +{ + PyThreadState *oldts = PyThreadState_Swap(NULL); + PyThreadState *tstate = Py_NewInterpreter(); + + struct atexit_data data = {0}; + int res = _Py_AtExit(tstate->interp, callback, (void *)&data); + Py_EndInterpreter(tstate); + PyThreadState_Swap(oldts); + if (res < 0) { + return NULL; + } + + if (data.called == 0) { + PyErr_SetString(PyExc_RuntimeError, "atexit callback not called"); + return NULL; + } + Py_RETURN_NONE; +} + + static PyMethodDef module_functions[] = { {"get_configs", get_configs, METH_NOARGS}, {"get_recursion_depth", get_recursion_depth, METH_NOARGS}, @@ -1316,6 +1348,7 @@ static PyMethodDef module_functions[] = { {"_PyTraceMalloc_GetTraceback", tracemalloc_get_traceback, METH_VARARGS}, {"test_tstate_capi", test_tstate_capi, METH_NOARGS, NULL}, {"_PyUnicode_TransformDecimalAndSpaceToASCII", unicode_transformdecimalandspacetoascii, METH_O}, + {"test_atexit", test_atexit, METH_NOARGS}, {NULL, NULL} /* sentinel */ }; diff --git a/Modules/_xxinterpchannelsmodule.c b/Modules/_xxinterpchannelsmodule.c index 1d7e7f1d71af3..82472555ec7d6 100644 --- a/Modules/_xxinterpchannelsmodule.c +++ b/Modules/_xxinterpchannelsmodule.c @@ -1,9 +1,13 @@ - /* interpreters module */ /* low-level access to interpreter primitives */ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + #include "Python.h" #include "interpreteridobject.h" +#include "pycore_atexit.h" // _Py_AtExit() /* diff --git a/Modules/getbuildinfo.c b/Modules/getbuildinfo.c index a24750b76c09b..8d553d106c6ab 100644 --- a/Modules/getbuildinfo.c +++ b/Modules/getbuildinfo.c @@ -1,4 +1,9 @@ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + #include "Python.h" +#include "pycore_pylifecycle.h" // _Py_gitidentifier() #ifndef DONT_HAVE_STDIO_H #include diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index b9f42476e6b82..aef802c232c6c 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -19,6 +19,7 @@ #include "pycore_initconfig.h" // _PyStatus_EXCEPTION() #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_object.h" // _PyObject_LookupSpecial() +#include "pycore_pylifecycle.h" // _PyOS_URandom() #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_signal.h" // Py_NSIG diff --git a/Modules/readline.c b/Modules/readline.c index ff7075c6822e2..a592919692cb8 100644 --- a/Modules/readline.c +++ b/Modules/readline.c @@ -4,8 +4,13 @@ * recently, it was largely rewritten by Guido van Rossum. */ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + /* Standard definitions */ #include "Python.h" +#include "pycore_pylifecycle.h" // _Py_SetLocaleFromEnv() #include #include diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c index 00ea4343735da..3adb2e8dfe58d 100644 --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -13,7 +13,7 @@ #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_pyerrors.h" // _PyErr_SetString() #include "pycore_pystate.h" // _PyThreadState_GET() -#include "pycore_signal.h" // Py_NSIG +#include "pycore_signal.h" // _Py_RestoreSignals() #ifndef MS_WINDOWS # include "posixmodule.h" diff --git a/Python/_warnings.c b/Python/_warnings.c index e4941f7b068d3..e0580f01d9361 100644 --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -1,10 +1,11 @@ #include "Python.h" +#include "pycore_frame.h" #include "pycore_initconfig.h" #include "pycore_interp.h" // PyInterpreterState.warnings #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_pyerrors.h" +#include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing() #include "pycore_pystate.h" // _PyThreadState_GET() -#include "pycore_frame.h" #include "clinic/_warnings.c.h" #define MODULE_NAME "_warnings" diff --git a/Python/bootstrap_hash.c b/Python/bootstrap_hash.c index 587063ef1ab29..ef693e5df1fcc 100644 --- a/Python/bootstrap_hash.c +++ b/Python/bootstrap_hash.c @@ -1,6 +1,7 @@ #include "Python.h" -#include "pycore_initconfig.h" #include "pycore_fileutils.h" // _Py_fstat_noraise() +#include "pycore_initconfig.h" +#include "pycore_pylifecycle.h" // _PyOS_URandomNonblock() #include "pycore_runtime.h" // _PyRuntime #ifdef MS_WINDOWS diff --git a/Python/preconfig.c b/Python/preconfig.c index 77a86d651eb0f..5b26c75de8b3a 100644 --- a/Python/preconfig.c +++ b/Python/preconfig.c @@ -2,6 +2,7 @@ #include "pycore_fileutils.h" // DECODE_LOCALE_ERR #include "pycore_getopt.h" // _PyOS_GetOpt() #include "pycore_initconfig.h" // _PyArgv +#include "pycore_pylifecycle.h" // _Py_LegacyLocaleDetected() #include "pycore_pymem.h" // _PyMem_GetAllocatorName() #include "pycore_runtime.h" // _PyRuntime_Initialize() From webhook-mailer at python.org Tue Jul 4 06:04:01 2023 From: webhook-mailer at python.org (pablogsal) Date: Tue, 04 Jul 2023 10:04:01 -0000 Subject: [Python-checkins] GH-106008: Fix refleak when peepholing `None` comparisons (#106367) Message-ID: https://github.com/python/cpython/commit/e4ba71fe4b32ae0d7fb3319d697616470fba1e58 commit: e4ba71fe4b32ae0d7fb3319d697616470fba1e58 branch: main author: Brandt Bucher committer: pablogsal date: 2023-07-04T11:03:57+01:00 summary: GH-106008: Fix refleak when peepholing `None` comparisons (#106367) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-03-11-38-43.gh-issue-106008.HDf1zd.rst M Python/flowgraph.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-03-11-38-43.gh-issue-106008.HDf1zd.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-03-11-38-43.gh-issue-106008.HDf1zd.rst new file mode 100644 index 0000000000000..a57b892fd5324 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-03-11-38-43.gh-issue-106008.HDf1zd.rst @@ -0,0 +1,2 @@ +Fix possible reference leaks when failing to optimize comparisons with +:const:`None` in the bytecode compiler. diff --git a/Python/flowgraph.c b/Python/flowgraph.c index 213c993bb863a..e159a4356dfe4 100644 --- a/Python/flowgraph.c +++ b/Python/flowgraph.c @@ -1377,9 +1377,9 @@ optimize_basic_block(PyObject *const_cache, basicblock *bb, PyObject *consts) goto error; } if (!Py_IsNone(cnt)) { + Py_DECREF(cnt); break; } - Py_DECREF(cnt); if (bb->b_iused <= i + 2) { break; } From webhook-mailer at python.org Tue Jul 4 07:36:43 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 04 Jul 2023 11:36:43 -0000 Subject: [Python-checkins] gh-106368: Add tests for permutation helpers in Argument Clinic (#106407) Message-ID: https://github.com/python/cpython/commit/8f6df5e9cbc3a1689601714192aa6ecbb23e1927 commit: 8f6df5e9cbc3a1689601714192aa6ecbb23e1927 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-04T11:36:40Z summary: gh-106368: Add tests for permutation helpers in Argument Clinic (#106407) Added new test class PermutationTests() files: M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 03754d0bf123b..544e7323e4f60 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1586,5 +1586,111 @@ def test_cloned_func_with_converter_exception_message(self): self.assertEqual(func(), name) +class PermutationTests(unittest.TestCase): + """Test permutation support functions.""" + + def test_permute_left_option_groups(self): + expected = ( + (), + (3,), + (2, 3), + (1, 2, 3), + ) + data = list(zip([1, 2, 3])) # Generate a list of 1-tuples. + actual = tuple(clinic.permute_left_option_groups(data)) + self.assertEqual(actual, expected) + + def test_permute_right_option_groups(self): + expected = ( + (), + (1,), + (1, 2), + (1, 2, 3), + ) + data = list(zip([1, 2, 3])) # Generate a list of 1-tuples. + actual = tuple(clinic.permute_right_option_groups(data)) + self.assertEqual(actual, expected) + + def test_permute_optional_groups(self): + empty = { + "left": (), "required": (), "right": (), + "expected": ((),), + } + noleft1 = { + "left": (), "required": ("b",), "right": ("c",), + "expected": ( + ("b",), + ("b", "c"), + ), + } + noleft2 = { + "left": (), "required": ("b", "c",), "right": ("d",), + "expected": ( + ("b", "c"), + ("b", "c", "d"), + ), + } + noleft3 = { + "left": (), "required": ("b", "c",), "right": ("d", "e"), + "expected": ( + ("b", "c"), + ("b", "c", "d"), + ("b", "c", "d", "e"), + ), + } + noright1 = { + "left": ("a",), "required": ("b",), "right": (), + "expected": ( + ("b",), + ("a", "b"), + ), + } + noright2 = { + "left": ("a",), "required": ("b", "c"), "right": (), + "expected": ( + ("b", "c"), + ("a", "b", "c"), + ), + } + noright3 = { + "left": ("a", "b"), "required": ("c",), "right": (), + "expected": ( + ("c",), + ("b", "c"), + ("a", "b", "c"), + ), + } + leftandright1 = { + "left": ("a",), "required": ("b",), "right": ("c",), + "expected": ( + ("b",), + ("a", "b"), # Prefer left. + ("a", "b", "c"), + ), + } + leftandright2 = { + "left": ("a", "b"), "required": ("c", "d"), "right": ("e", "f"), + "expected": ( + ("c", "d"), + ("b", "c", "d"), # Prefer left. + ("a", "b", "c", "d"), # Prefer left. + ("a", "b", "c", "d", "e"), + ("a", "b", "c", "d", "e", "f"), + ), + } + dataset = ( + empty, + noleft1, noleft2, noleft3, + noright1, noright2, noright3, + leftandright1, leftandright2, + ) + for params in dataset: + with self.subTest(**params): + left, required, right, expected = params.values() + permutations = clinic.permute_optional_groups(left, required, right) + actual = tuple(permutations) + self.assertEqual(actual, expected) + + if __name__ == "__main__": unittest.main() diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index a07fcbd8cabf7..c02c82876591f 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -518,7 +518,7 @@ class PythonLanguage(Language): def permute_left_option_groups(l): """ - Given [1, 2, 3], should yield: + Given [(1,), (2,), (3,)], should yield: () (3,) (2, 3) @@ -533,7 +533,7 @@ def permute_left_option_groups(l): def permute_right_option_groups(l): """ - Given [1, 2, 3], should yield: + Given [(1,), (2,), (3,)], should yield: () (1,) (1, 2) From webhook-mailer at python.org Tue Jul 4 08:17:09 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 04 Jul 2023 12:17:09 -0000 Subject: [Python-checkins] [3.12] gh-106368: Add tests for permutation helpers in Argument Clinic (GH-106407) (#106409) Message-ID: https://github.com/python/cpython/commit/fda297031bba7c4e46b5959ebaa9c48ca11bb5f4 commit: fda297031bba7c4e46b5959ebaa9c48ca11bb5f4 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-04T14:17:05+02:00 summary: [3.12] gh-106368: Add tests for permutation helpers in Argument Clinic (GH-106407) (#106409) Added new test class PermutationTests() (cherry picked from commit 8f6df5e9cbc3a1689601714192aa6ecbb23e1927) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 03754d0bf123b..544e7323e4f60 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1586,5 +1586,111 @@ def test_cloned_func_with_converter_exception_message(self): self.assertEqual(func(), name) +class PermutationTests(unittest.TestCase): + """Test permutation support functions.""" + + def test_permute_left_option_groups(self): + expected = ( + (), + (3,), + (2, 3), + (1, 2, 3), + ) + data = list(zip([1, 2, 3])) # Generate a list of 1-tuples. + actual = tuple(clinic.permute_left_option_groups(data)) + self.assertEqual(actual, expected) + + def test_permute_right_option_groups(self): + expected = ( + (), + (1,), + (1, 2), + (1, 2, 3), + ) + data = list(zip([1, 2, 3])) # Generate a list of 1-tuples. + actual = tuple(clinic.permute_right_option_groups(data)) + self.assertEqual(actual, expected) + + def test_permute_optional_groups(self): + empty = { + "left": (), "required": (), "right": (), + "expected": ((),), + } + noleft1 = { + "left": (), "required": ("b",), "right": ("c",), + "expected": ( + ("b",), + ("b", "c"), + ), + } + noleft2 = { + "left": (), "required": ("b", "c",), "right": ("d",), + "expected": ( + ("b", "c"), + ("b", "c", "d"), + ), + } + noleft3 = { + "left": (), "required": ("b", "c",), "right": ("d", "e"), + "expected": ( + ("b", "c"), + ("b", "c", "d"), + ("b", "c", "d", "e"), + ), + } + noright1 = { + "left": ("a",), "required": ("b",), "right": (), + "expected": ( + ("b",), + ("a", "b"), + ), + } + noright2 = { + "left": ("a",), "required": ("b", "c"), "right": (), + "expected": ( + ("b", "c"), + ("a", "b", "c"), + ), + } + noright3 = { + "left": ("a", "b"), "required": ("c",), "right": (), + "expected": ( + ("c",), + ("b", "c"), + ("a", "b", "c"), + ), + } + leftandright1 = { + "left": ("a",), "required": ("b",), "right": ("c",), + "expected": ( + ("b",), + ("a", "b"), # Prefer left. + ("a", "b", "c"), + ), + } + leftandright2 = { + "left": ("a", "b"), "required": ("c", "d"), "right": ("e", "f"), + "expected": ( + ("c", "d"), + ("b", "c", "d"), # Prefer left. + ("a", "b", "c", "d"), # Prefer left. + ("a", "b", "c", "d", "e"), + ("a", "b", "c", "d", "e", "f"), + ), + } + dataset = ( + empty, + noleft1, noleft2, noleft3, + noright1, noright2, noright3, + leftandright1, leftandright2, + ) + for params in dataset: + with self.subTest(**params): + left, required, right, expected = params.values() + permutations = clinic.permute_optional_groups(left, required, right) + actual = tuple(permutations) + self.assertEqual(actual, expected) + + if __name__ == "__main__": unittest.main() diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index d67479c7f8ab8..fa6c1392e1f09 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -509,7 +509,7 @@ class PythonLanguage(Language): def permute_left_option_groups(l): """ - Given [1, 2, 3], should yield: + Given [(1,), (2,), (3,)], should yield: () (3,) (2, 3) @@ -524,7 +524,7 @@ def permute_left_option_groups(l): def permute_right_option_groups(l): """ - Given [1, 2, 3], should yield: + Given [(1,), (2,), (3,)], should yield: () (1,) (1, 2) From webhook-mailer at python.org Tue Jul 4 08:17:29 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 04 Jul 2023 12:17:29 -0000 Subject: [Python-checkins] [3.11] gh-106368: Add tests for permutation helpers in Argument Clinic (GH-106407) (#106410) Message-ID: https://github.com/python/cpython/commit/9f8aa7a6d79eb71c339fface8f8570607b8d1cf2 commit: 9f8aa7a6d79eb71c339fface8f8570607b8d1cf2 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-04T14:17:26+02:00 summary: [3.11] gh-106368: Add tests for permutation helpers in Argument Clinic (GH-106407) (#106410) Added new test class PermutationTests() (cherry picked from commit 8f6df5e9cbc3a1689601714192aa6ecbb23e1927) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index dafdfc2240d0d..8c9fe997f1adb 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1518,5 +1518,111 @@ def test_gh_99240_double_free(self): ac_tester.gh_99240_double_free('a', '\0b') +class PermutationTests(unittest.TestCase): + """Test permutation support functions.""" + + def test_permute_left_option_groups(self): + expected = ( + (), + (3,), + (2, 3), + (1, 2, 3), + ) + data = list(zip([1, 2, 3])) # Generate a list of 1-tuples. + actual = tuple(clinic.permute_left_option_groups(data)) + self.assertEqual(actual, expected) + + def test_permute_right_option_groups(self): + expected = ( + (), + (1,), + (1, 2), + (1, 2, 3), + ) + data = list(zip([1, 2, 3])) # Generate a list of 1-tuples. + actual = tuple(clinic.permute_right_option_groups(data)) + self.assertEqual(actual, expected) + + def test_permute_optional_groups(self): + empty = { + "left": (), "required": (), "right": (), + "expected": ((),), + } + noleft1 = { + "left": (), "required": ("b",), "right": ("c",), + "expected": ( + ("b",), + ("b", "c"), + ), + } + noleft2 = { + "left": (), "required": ("b", "c",), "right": ("d",), + "expected": ( + ("b", "c"), + ("b", "c", "d"), + ), + } + noleft3 = { + "left": (), "required": ("b", "c",), "right": ("d", "e"), + "expected": ( + ("b", "c"), + ("b", "c", "d"), + ("b", "c", "d", "e"), + ), + } + noright1 = { + "left": ("a",), "required": ("b",), "right": (), + "expected": ( + ("b",), + ("a", "b"), + ), + } + noright2 = { + "left": ("a",), "required": ("b", "c"), "right": (), + "expected": ( + ("b", "c"), + ("a", "b", "c"), + ), + } + noright3 = { + "left": ("a", "b"), "required": ("c",), "right": (), + "expected": ( + ("c",), + ("b", "c"), + ("a", "b", "c"), + ), + } + leftandright1 = { + "left": ("a",), "required": ("b",), "right": ("c",), + "expected": ( + ("b",), + ("a", "b"), # Prefer left. + ("a", "b", "c"), + ), + } + leftandright2 = { + "left": ("a", "b"), "required": ("c", "d"), "right": ("e", "f"), + "expected": ( + ("c", "d"), + ("b", "c", "d"), # Prefer left. + ("a", "b", "c", "d"), # Prefer left. + ("a", "b", "c", "d", "e"), + ("a", "b", "c", "d", "e", "f"), + ), + } + dataset = ( + empty, + noleft1, noleft2, noleft3, + noright1, noright2, noright3, + leftandright1, leftandright2, + ) + for params in dataset: + with self.subTest(**params): + left, required, right, expected = params.values() + permutations = clinic.permute_optional_groups(left, required, right) + actual = tuple(permutations) + self.assertEqual(actual, expected) + + if __name__ == "__main__": unittest.main() diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 82e4919242c71..b18c0205a8559 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -460,7 +460,7 @@ class PythonLanguage(Language): def permute_left_option_groups(l): """ - Given [1, 2, 3], should yield: + Given [(1,), (2,), (3,)], should yield: () (3,) (2, 3) @@ -475,7 +475,7 @@ def permute_left_option_groups(l): def permute_right_option_groups(l): """ - Given [1, 2, 3], should yield: + Given [(1,), (2,), (3,)], should yield: () (1,) (1, 2) From webhook-mailer at python.org Tue Jul 4 08:19:11 2023 From: webhook-mailer at python.org (lysnikolaou) Date: Tue, 04 Jul 2023 12:19:11 -0000 Subject: [Python-checkins] gh-106396: Special-case empty format spec to gen empty JoinedStr node (#106401) Message-ID: https://github.com/python/cpython/commit/dfe4de203881e8d068e6fc5b8e31075841a86d25 commit: dfe4de203881e8d068e6fc5b8e31075841a86d25 branch: main author: Lysandros Nikolaou committer: lysnikolaou date: 2023-07-04T14:19:08+02:00 summary: gh-106396: Special-case empty format spec to gen empty JoinedStr node (#106401) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst M Lib/test/test_fstring.py M Parser/action_helpers.c diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index ba223ae124a8f..cb14bba2602de 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -496,6 +496,24 @@ def test_ast_line_numbers_with_parentheses(self): self.assertEqual(wat2.end_col_offset, 17) self.assertEqual(fstring.end_col_offset, 18) + def test_ast_fstring_empty_format_spec(self): + expr = "f'{expr:}'" + + mod = ast.parse(expr) + self.assertEqual(type(mod), ast.Module) + self.assertEqual(len(mod.body), 1) + + fstring = mod.body[0].value + self.assertEqual(type(fstring), ast.JoinedStr) + self.assertEqual(len(fstring.values), 1) + + fv = fstring.values[0] + self.assertEqual(type(fv), ast.FormattedValue) + + format_spec = fv.format_spec + self.assertEqual(type(format_spec), ast.JoinedStr) + self.assertEqual(len(format_spec.values), 0) + def test_docstring(self): def f(): f'''Not a docstring''' diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst new file mode 100644 index 0000000000000..c5767e97271d9 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst @@ -0,0 +1,3 @@ +When the format specification of an f-string expression is empty, the parser now +generates an empty :class:`ast.JoinedStr` node for it instead of an one-element +:class:`ast.JoinedStr` with an empty string :class:`ast.Constant`. diff --git a/Parser/action_helpers.c b/Parser/action_helpers.c index 70c267bb212fc..36e0750220a30 100644 --- a/Parser/action_helpers.c +++ b/Parser/action_helpers.c @@ -997,6 +997,18 @@ _PyPegen_setup_full_format_spec(Parser *p, Token *colon, asdl_expr_seq *spec, in if (!spec) { return NULL; } + + // This is needed to keep compatibility with 3.11, where an empty format spec is parsed + // as an *empty* JoinedStr node, instead of having an empty constant in it. + if (asdl_seq_LEN(spec) == 1) { + expr_ty e = asdl_seq_GET(spec, 0); + if (e->kind == Constant_kind + && PyUnicode_Check(e->v.Constant.value) + && PyUnicode_GetLength(e->v.Constant.value) == 0) { + spec = _Py_asdl_expr_seq_new(0, arena); + } + } + expr_ty res = _PyAST_JoinedStr(spec, lineno, col_offset, end_lineno, end_col_offset, p->arena); if (!res) { return NULL; From webhook-mailer at python.org Tue Jul 4 09:00:52 2023 From: webhook-mailer at python.org (lysnikolaou) Date: Tue, 04 Jul 2023 13:00:52 -0000 Subject: [Python-checkins] [3.12] gh-106396: Special-case empty format spec to gen empty JoinedStr node (GH-106401) (#106416) Message-ID: https://github.com/python/cpython/commit/930df7b07e774636ad200a62a7b4b56564f502b0 commit: 930df7b07e774636ad200a62a7b4b56564f502b0 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: lysnikolaou date: 2023-07-04T13:00:47Z summary: [3.12] gh-106396: Special-case empty format spec to gen empty JoinedStr node (GH-106401) (#106416) (cherry picked from commit dfe4de203881e8d068e6fc5b8e31075841a86d25) Co-authored-by: Lysandros Nikolaou files: A Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst M Lib/test/test_fstring.py M Parser/action_helpers.c diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index ba223ae124a8f..cb14bba2602de 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -496,6 +496,24 @@ def test_ast_line_numbers_with_parentheses(self): self.assertEqual(wat2.end_col_offset, 17) self.assertEqual(fstring.end_col_offset, 18) + def test_ast_fstring_empty_format_spec(self): + expr = "f'{expr:}'" + + mod = ast.parse(expr) + self.assertEqual(type(mod), ast.Module) + self.assertEqual(len(mod.body), 1) + + fstring = mod.body[0].value + self.assertEqual(type(fstring), ast.JoinedStr) + self.assertEqual(len(fstring.values), 1) + + fv = fstring.values[0] + self.assertEqual(type(fv), ast.FormattedValue) + + format_spec = fv.format_spec + self.assertEqual(type(format_spec), ast.JoinedStr) + self.assertEqual(len(format_spec.values), 0) + def test_docstring(self): def f(): f'''Not a docstring''' diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst new file mode 100644 index 0000000000000..c5767e97271d9 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst @@ -0,0 +1,3 @@ +When the format specification of an f-string expression is empty, the parser now +generates an empty :class:`ast.JoinedStr` node for it instead of an one-element +:class:`ast.JoinedStr` with an empty string :class:`ast.Constant`. diff --git a/Parser/action_helpers.c b/Parser/action_helpers.c index dbad56b5164b6..e68a9cac25908 100644 --- a/Parser/action_helpers.c +++ b/Parser/action_helpers.c @@ -1003,6 +1003,18 @@ _PyPegen_setup_full_format_spec(Parser *p, Token *colon, asdl_expr_seq *spec, in if (!spec) { return NULL; } + + // This is needed to keep compatibility with 3.11, where an empty format spec is parsed + // as an *empty* JoinedStr node, instead of having an empty constant in it. + if (asdl_seq_LEN(spec) == 1) { + expr_ty e = asdl_seq_GET(spec, 0); + if (e->kind == Constant_kind + && PyUnicode_Check(e->v.Constant.value) + && PyUnicode_GetLength(e->v.Constant.value) == 0) { + spec = _Py_asdl_expr_seq_new(0, arena); + } + } + expr_ty res = _PyAST_JoinedStr(spec, lineno, col_offset, end_lineno, end_col_offset, p->arena); if (!res) { return NULL; From webhook-mailer at python.org Tue Jul 4 10:19:23 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 04 Jul 2023 14:19:23 -0000 Subject: [Python-checkins] gh-106406: Fix _Py_IsInterpreterFinalizing() in _winapi.c (#106408) Message-ID: https://github.com/python/cpython/commit/80f1c6c49b4cd2bf698eb2bc3d2f3da904880dd2 commit: 80f1c6c49b4cd2bf698eb2bc3d2f3da904880dd2 branch: main author: Nikita Sobolev committer: vstinner date: 2023-07-04T16:19:20+02:00 summary: gh-106406: Fix _Py_IsInterpreterFinalizing() in _winapi.c (#106408) files: M Modules/_winapi.c diff --git a/Modules/_winapi.c b/Modules/_winapi.c index d4291f557b6a6..313c12a34c672 100644 --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -36,6 +36,7 @@ #include "Python.h" #include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing() #include "pycore_pystate.h" // _PyInterpreterState_GET #include "structmember.h" // PyMemberDef From webhook-mailer at python.org Tue Jul 4 12:23:04 2023 From: webhook-mailer at python.org (markshannon) Date: Tue, 04 Jul 2023 16:23:04 -0000 Subject: [Python-checkins] GH-106360: Support very basic superblock introspection (#106422) Message-ID: https://github.com/python/cpython/commit/318ea2c72e9aed7ac92457c28747eda9424c8327 commit: 318ea2c72e9aed7ac92457c28747eda9424c8327 branch: main author: Mark Shannon committer: markshannon date: 2023-07-04T17:23:00+01:00 summary: GH-106360: Support very basic superblock introspection (#106422) * Add len() and indexing support to uop superblocks. files: M Include/cpython/optimizer.h M Include/internal/pycore_opcode.h M Lib/test/test_capi/test_misc.py M Modules/_testinternalcapi.c M Python/opcode_metadata.h M Python/optimizer.c M Tools/build/generate_opcode_h.py M Tools/cases_generator/generate_cases.py diff --git a/Include/cpython/optimizer.h b/Include/cpython/optimizer.h index 2664f5bc4b174..2260501bfd608 100644 --- a/Include/cpython/optimizer.h +++ b/Include/cpython/optimizer.h @@ -38,6 +38,8 @@ PyAPI_FUNC(void) PyUnstable_SetOptimizer(_PyOptimizerObject* optimizer); PyAPI_FUNC(_PyOptimizerObject *) PyUnstable_GetOptimizer(void); +PyAPI_FUNC(_PyExecutorObject *)PyUnstable_GetExecutor(PyCodeObject *code, int offset); + struct _PyInterpreterFrame * _PyOptimizer_BackEdge(struct _PyInterpreterFrame *frame, _Py_CODEUNIT *src, _Py_CODEUNIT *dest, PyObject **stack_pointer); diff --git a/Include/internal/pycore_opcode.h b/Include/internal/pycore_opcode.h index 428df4ccadbc1..c7c2fdcc32788 100644 --- a/Include/internal/pycore_opcode.h +++ b/Include/internal/pycore_opcode.h @@ -242,8 +242,11 @@ const uint8_t _PyOpcode_Deopt[256] = { }; #endif // NEED_OPCODE_TABLES -#ifdef Py_DEBUG -static const char *const _PyOpcode_OpName[268] = { + +extern const char *const _PyOpcode_OpName[268]; + +#ifdef NEED_OPCODE_TABLES +const char *const _PyOpcode_OpName[268] = { [CACHE] = "CACHE", [POP_TOP] = "POP_TOP", [PUSH_NULL] = "PUSH_NULL", @@ -513,7 +516,7 @@ static const char *const _PyOpcode_OpName[268] = { [STORE_FAST_MAYBE_NULL] = "STORE_FAST_MAYBE_NULL", [LOAD_CLOSURE] = "LOAD_CLOSURE", }; -#endif +#endif // NEED_OPCODE_TABLES #define EXTRA_CASES \ case 184: \ diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 9e825a343eb31..de9f00a9e5fb4 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2415,5 +2415,28 @@ def long_loop(): self.assertEqual(opt.get_count(), 10) +class TestUops(unittest.TestCase): + + def test_basic_loop(self): + + def testfunc(x): + i = 0 + while i < x: + i += 1 + + testfunc(1000) + + ex = None + for offset in range(0, 100, 2): + try: + ex = _testinternalcapi.get_executor(testfunc.__code__, offset) + break + except ValueError: + pass + if ex is None: + return + self.assertIn("SAVE_IP", str(ex)) + + if __name__ == "__main__": unittest.main() diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 84511d27e37f4..52e524a40672e 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -858,6 +858,26 @@ get_optimizer(PyObject *self, PyObject *Py_UNUSED(ignored)) return opt; } +static PyObject * +get_executor(PyObject *self, PyObject *const *args, Py_ssize_t nargs) +{ + + if (!_PyArg_CheckPositional("get_executor", nargs, 2, 2)) { + return NULL; + } + PyObject *code = args[0]; + PyObject *offset = args[1]; + long ioffset = PyLong_AsLong(offset); + if (ioffset == -1 && PyErr_Occurred()) { + return NULL; + } + if (!PyCode_Check(code)) { + PyErr_SetString(PyExc_TypeError, "first argument must be a code object"); + return NULL; + } + return (PyObject *)PyUnstable_GetExecutor((PyCodeObject *)code, ioffset); +} + static int _pending_callback(void *arg) { /* we assume the argument is callable object to which we own a reference */ @@ -1326,6 +1346,7 @@ static PyMethodDef module_functions[] = { {"iframe_getlasti", iframe_getlasti, METH_O, NULL}, {"get_optimizer", get_optimizer, METH_NOARGS, NULL}, {"set_optimizer", set_optimizer, METH_O, NULL}, + {"get_executor", _PyCFunction_CAST(get_executor), METH_FASTCALL, NULL}, {"get_counter_optimizer", get_counter_optimizer, METH_NOARGS, NULL}, {"get_uop_optimizer", get_uop_optimizer, METH_NOARGS, NULL}, {"pending_threadfunc", _PyCFunction_CAST(pending_threadfunc), diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index 82c9823589228..ac86a4abd9c1b 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -942,9 +942,7 @@ struct opcode_macro_expansion { #ifndef NEED_OPCODE_METADATA extern const struct opcode_metadata _PyOpcode_opcode_metadata[512]; extern const struct opcode_macro_expansion _PyOpcode_macro_expansion[256]; -#ifdef Py_DEBUG extern const char * const _PyOpcode_uop_name[512]; -#endif #else const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { [NOP] = { true, INSTR_FMT_IX, 0 }, @@ -1265,7 +1263,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [COPY] = { .nuops = 1, .uops = { { COPY, 0, 0 } } }, [SWAP] = { .nuops = 1, .uops = { { SWAP, 0, 0 } } }, }; -#ifdef Py_DEBUG +#ifdef NEED_OPCODE_METADATA const char * const _PyOpcode_uop_name[512] = { [300] = "EXIT_TRACE", [301] = "SAVE_IP", @@ -1282,5 +1280,5 @@ const char * const _PyOpcode_uop_name[512] = { [312] = "_LOAD_LOCALS", [313] = "_LOAD_FROM_DICT_OR_GLOBALS", }; -#endif +#endif // NEED_OPCODE_METADATA #endif diff --git a/Python/optimizer.c b/Python/optimizer.c index 32f0b1477d203..c3ab649b51b0e 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -188,6 +188,23 @@ _PyOptimizer_BackEdge(_PyInterpreterFrame *frame, _Py_CODEUNIT *src, _Py_CODEUNI return frame; } +_PyExecutorObject * +PyUnstable_GetExecutor(PyCodeObject *code, int offset) +{ + int code_len = (int)Py_SIZE(code); + for (int i = 0 ; i < code_len;) { + if (_PyCode_CODE(code)[i].op.code == ENTER_EXECUTOR && i*2 == offset) { + int oparg = _PyCode_CODE(code)[i].op.arg; + _PyExecutorObject *res = code->co_executors->executors[oparg]; + Py_INCREF(res); + return res; + } + i += _PyInstruction_GetLength(code, i); + } + PyErr_SetString(PyExc_ValueError, "no executor at given offset"); + return NULL; +} + /** Test support **/ @@ -287,6 +304,58 @@ uop_dealloc(_PyUOpExecutorObject *self) { PyObject_Free(self); } +static const char * +uop_name(int index) { + if (index < EXIT_TRACE) { + return _PyOpcode_OpName[index]; + } + return _PyOpcode_uop_name[index]; +} + +static Py_ssize_t +uop_len(_PyUOpExecutorObject *self) +{ + int count = 1; + for (; count < _Py_UOP_MAX_TRACE_LENGTH; count++) { + if (self->trace[count-1].opcode == EXIT_TRACE) { + break; + } + } + return count; +} + +static PyObject * +uop_item(_PyUOpExecutorObject *self, Py_ssize_t index) +{ + for (int i = 0; i < _Py_UOP_MAX_TRACE_LENGTH; i++) { + if (self->trace[i].opcode == EXIT_TRACE) { + break; + } + if (i != index) { + continue; + } + const char *name = uop_name(self->trace[i].opcode); + PyObject *oname = _PyUnicode_FromASCII(name, strlen(name)); + if (oname == NULL) { + return NULL; + } + PyObject *operand = PyLong_FromUnsignedLongLong(self->trace[i].operand); + if (operand == NULL) { + Py_DECREF(oname); + return NULL; + } + PyObject *args[2] = { oname, operand }; + return _PyTuple_FromArraySteal(args, 2); + } + PyErr_SetNone(PyExc_IndexError); + return NULL; +} + +PySequenceMethods uop_as_sequence = { + .sq_length = (lenfunc)uop_len, + .sq_item = (ssizeargfunc)uop_item, +}; + static PyTypeObject UOpExecutor_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) .tp_name = "uop_executor", @@ -294,6 +363,7 @@ static PyTypeObject UOpExecutor_Type = { .tp_itemsize = 0, .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_DISALLOW_INSTANTIATION, .tp_dealloc = (destructor)uop_dealloc, + .tp_as_sequence = &uop_as_sequence, }; static int diff --git a/Tools/build/generate_opcode_h.py b/Tools/build/generate_opcode_h.py index 4711fbbd1eb8c..2e841e6097aa2 100644 --- a/Tools/build/generate_opcode_h.py +++ b/Tools/build/generate_opcode_h.py @@ -184,14 +184,15 @@ def main(opcode_py, fobj.write(f"#define ENABLE_SPECIALIZATION {int(ENABLE_SPECIALIZATION)}") iobj.write("\n") - iobj.write("#ifdef Py_DEBUG\n") - iobj.write(f"static const char *const _PyOpcode_OpName[{NUM_OPCODES}] = {{\n") + iobj.write(f"\nextern const char *const _PyOpcode_OpName[{NUM_OPCODES}];\n") + iobj.write("\n#ifdef NEED_OPCODE_TABLES\n") + iobj.write(f"const char *const _PyOpcode_OpName[{NUM_OPCODES}] = {{\n") for op, name in enumerate(opname_including_specialized): if name[0] != "<": op = name iobj.write(f''' [{op}] = "{name}",\n''') iobj.write("};\n") - iobj.write("#endif\n") + iobj.write("#endif // NEED_OPCODE_TABLES\n") iobj.write("\n") iobj.write("#define EXTRA_CASES \\\n") diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 657dfa93fd537..a90abfe20c173 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -1224,9 +1224,7 @@ def write_metadata(self) -> None: self.out.emit("#ifndef NEED_OPCODE_METADATA") self.out.emit("extern const struct opcode_metadata _PyOpcode_opcode_metadata[512];") self.out.emit("extern const struct opcode_macro_expansion _PyOpcode_macro_expansion[256];") - self.out.emit("#ifdef Py_DEBUG") self.out.emit("extern const char * const _PyOpcode_uop_name[512];") - self.out.emit("#endif") self.out.emit("#else") self.out.emit("const struct opcode_metadata _PyOpcode_opcode_metadata[512] = {") @@ -1273,10 +1271,10 @@ def write_metadata(self) -> None: case _: typing.assert_never(thing) - self.out.emit("#ifdef Py_DEBUG") + self.out.emit("#ifdef NEED_OPCODE_METADATA") with self.out.block("const char * const _PyOpcode_uop_name[512] =", ";"): self.write_uop_items(lambda name, counter: f"[{counter}] = \"{name}\",") - self.out.emit("#endif") + self.out.emit("#endif // NEED_OPCODE_METADATA") self.out.emit("#endif") From webhook-mailer at python.org Tue Jul 4 12:55:49 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 04 Jul 2023 16:55:49 -0000 Subject: [Python-checkins] gh-106320: Remove _PyInterpreterState_HasFeature() (#106425) Message-ID: https://github.com/python/cpython/commit/aa85c93792876a602fc9ce2083b64958507d29bf commit: aa85c93792876a602fc9ce2083b64958507d29bf branch: main author: Victor Stinner committer: vstinner date: 2023-07-04T16:55:45Z summary: gh-106320: Remove _PyInterpreterState_HasFeature() (#106425) Remove the _PyInterpreterState_HasFeature() function from the C API: move it to the internal C API (pycore_interp.h). No longer export the function. files: M Include/cpython/pystate.h M Include/internal/pycore_interp.h diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index e08bcdaf19762..4254110889fc6 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -3,38 +3,6 @@ #endif -/* -Runtime Feature Flags - -Each flag indicate whether or not a specific runtime feature -is available in a given context. For example, forking the process -might not be allowed in the current interpreter (i.e. os.fork() would fail). -*/ - -/* Set if the interpreter share obmalloc runtime state - with the main interpreter. */ -#define Py_RTFLAGS_USE_MAIN_OBMALLOC (1UL << 5) - -/* Set if import should check a module for subinterpreter support. */ -#define Py_RTFLAGS_MULTI_INTERP_EXTENSIONS (1UL << 8) - -/* Set if threads are allowed. */ -#define Py_RTFLAGS_THREADS (1UL << 10) - -/* Set if daemon threads are allowed. */ -#define Py_RTFLAGS_DAEMON_THREADS (1UL << 11) - -/* Set if os.fork() is allowed. */ -#define Py_RTFLAGS_FORK (1UL << 15) - -/* Set if os.exec*() is allowed. */ -#define Py_RTFLAGS_EXEC (1UL << 16) - - -PyAPI_FUNC(int) _PyInterpreterState_HasFeature(PyInterpreterState *interp, - unsigned long feature); - - /* private interpreter helpers */ PyAPI_FUNC(int) _PyInterpreterState_RequiresIDRef(PyInterpreterState *); diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index a1f00df12d6ac..bb37cafe6286a 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -275,6 +275,38 @@ PyAPI_FUNC(int) _PyInterpreterState_GetConfigCopy( PyAPI_FUNC(int) _PyInterpreterState_SetConfig( const struct PyConfig *config); + +/* +Runtime Feature Flags + +Each flag indicate whether or not a specific runtime feature +is available in a given context. For example, forking the process +might not be allowed in the current interpreter (i.e. os.fork() would fail). +*/ + +/* Set if the interpreter share obmalloc runtime state + with the main interpreter. */ +#define Py_RTFLAGS_USE_MAIN_OBMALLOC (1UL << 5) + +/* Set if import should check a module for subinterpreter support. */ +#define Py_RTFLAGS_MULTI_INTERP_EXTENSIONS (1UL << 8) + +/* Set if threads are allowed. */ +#define Py_RTFLAGS_THREADS (1UL << 10) + +/* Set if daemon threads are allowed. */ +#define Py_RTFLAGS_DAEMON_THREADS (1UL << 11) + +/* Set if os.fork() is allowed. */ +#define Py_RTFLAGS_FORK (1UL << 15) + +/* Set if os.exec*() is allowed. */ +#define Py_RTFLAGS_EXEC (1UL << 16) + +extern int _PyInterpreterState_HasFeature(PyInterpreterState *interp, + unsigned long feature); + + #ifdef __cplusplus } #endif From webhook-mailer at python.org Tue Jul 4 13:02:52 2023 From: webhook-mailer at python.org (methane) Date: Tue, 04 Jul 2023 17:02:52 -0000 Subject: [Python-checkins] gh-106162: array: suppress warning in test_array (#106404) Message-ID: https://github.com/python/cpython/commit/8a4bba8b9764ba28667b137fe62c11aea672f500 commit: 8a4bba8b9764ba28667b137fe62c11aea672f500 branch: main author: Inada Naoki committer: methane date: 2023-07-05T02:02:49+09:00 summary: gh-106162: array: suppress warning in test_array (#106404) array: suppress warning in test_array files: M Lib/test/test_array.py diff --git a/Lib/test/test_array.py b/Lib/test/test_array.py index bec3766b87b20..a219fa365e7f2 100755 --- a/Lib/test/test_array.py +++ b/Lib/test/test_array.py @@ -215,6 +215,14 @@ class BaseTest: # outside: An entry that is not in example # minitemsize: the minimum guaranteed itemsize + def setUp(self): + self.enterContext(warnings.catch_warnings()) + warnings.filterwarnings( + "ignore", + message="The 'u' type code is deprecated and " + "will be removed in Python 3.16", + category=DeprecationWarning) + def assertEntryEqual(self, entry1, entry2): self.assertEqual(entry1, entry2) From webhook-mailer at python.org Tue Jul 4 13:34:47 2023 From: webhook-mailer at python.org (cjw296) Date: Tue, 04 Jul 2023 17:34:47 -0000 Subject: [Python-checkins] gh-61215: Rename `wait_until_any_call` to `wait_until_any_call_with` (#106414) Message-ID: https://github.com/python/cpython/commit/2dfc7fae787e65726f24bfe9efe05418b05ee8e2 commit: 2dfc7fae787e65726f24bfe9efe05418b05ee8e2 branch: main author: Mario Corchero committer: cjw296 date: 2023-07-04T18:34:43+01:00 summary: gh-61215: Rename `wait_until_any_call` to `wait_until_any_call_with` (#106414) mock: Rename `wait_until_any_call` to `wait_until_any_call_with` Rename the method to be more explicit that it expects the args and kwargs to wait for. files: M Doc/library/unittest.mock.rst M Lib/test/test_unittest/testmock/testthreadingmock.py M Lib/unittest/mock.py diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 756422bd6eab9..6d5f17d1c2c5c 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -1126,7 +1126,7 @@ object:: >>> mock.wait_until_called(timeout=1) >>> thread.join() - .. method:: wait_until_any_call(*args, **kwargs) + .. method:: wait_until_any_call_with(*args, **kwargs) Waits until the the mock is called with the specified arguments. @@ -1136,7 +1136,7 @@ object:: >>> mock = ThreadingMock() >>> thread = threading.Thread(target=mock, args=("arg1", "arg2",), kwargs={"arg": "thing"}) >>> thread.start() - >>> mock.wait_until_any_call("arg1", "arg2", arg="thing") + >>> mock.wait_until_any_call_with("arg1", "arg2", arg="thing") >>> thread.join() .. attribute:: DEFAULT_TIMEOUT diff --git a/Lib/test/test_unittest/testmock/testthreadingmock.py b/Lib/test/test_unittest/testmock/testthreadingmock.py index 6219cc58abdc4..2b4dca3a4be17 100644 --- a/Lib/test/test_unittest/testmock/testthreadingmock.py +++ b/Lib/test/test_unittest/testmock/testthreadingmock.py @@ -87,7 +87,7 @@ def test_no_name_clash(self): waitable_mock.timeout = "mytimeout" waitable_mock("works") waitable_mock.wait_until_called() - waitable_mock.wait_until_any_call("works") + waitable_mock.wait_until_any_call_with("works") def test_wait_success(self): waitable_mock = self._make_mock(spec=Something) @@ -96,7 +96,7 @@ def test_wait_success(self): something = Something() self.run_async(something.method_1, delay=0.01) something.method_1.wait_until_called() - something.method_1.wait_until_any_call() + something.method_1.wait_until_any_call_with() something.method_1.assert_called() def test_wait_success_with_instance_timeout(self): @@ -106,7 +106,7 @@ def test_wait_success_with_instance_timeout(self): something = Something() self.run_async(something.method_1, delay=0.01) something.method_1.wait_until_called() - something.method_1.wait_until_any_call() + something.method_1.wait_until_any_call_with() something.method_1.assert_called() def test_wait_failed_with_instance_timeout(self): @@ -117,7 +117,7 @@ def test_wait_failed_with_instance_timeout(self): self.run_async(something.method_1, delay=0.5) self.assertRaises(AssertionError, waitable_mock.method_1.wait_until_called) self.assertRaises( - AssertionError, waitable_mock.method_1.wait_until_any_call + AssertionError, waitable_mock.method_1.wait_until_any_call_with ) def test_wait_success_with_timeout_override(self): @@ -137,7 +137,7 @@ def test_wait_failed_with_timeout_override(self): with self.assertRaises(AssertionError): something.method_1.wait_until_called(timeout=0.05) with self.assertRaises(AssertionError): - something.method_1.wait_until_any_call(timeout=0.05) + something.method_1.wait_until_any_call_with(timeout=0.05) def test_wait_success_called_before(self): waitable_mock = self._make_mock() @@ -146,7 +146,7 @@ def test_wait_success_called_before(self): something = Something() something.method_1() something.method_1.wait_until_called() - something.method_1.wait_until_any_call() + something.method_1.wait_until_any_call_with() something.method_1.assert_called() def test_wait_magic_method(self): @@ -158,7 +158,7 @@ def test_wait_magic_method(self): something.method_1.__str__.wait_until_called() something.method_1.__str__.assert_called() - def test_wait_until_any_call_positional(self): + def test_wait_until_any_call_with_positional(self): waitable_mock = self._make_mock(spec=Something) with patch(f"{__name__}.Something", waitable_mock): @@ -168,16 +168,16 @@ def test_wait_until_any_call_positional(self): self.run_async(something.method_1, 3, delay=0.3) self.assertNotIn(call(1), something.method_1.mock_calls) - something.method_1.wait_until_any_call(1) + something.method_1.wait_until_any_call_with(1) something.method_1.assert_called_with(1) self.assertNotIn(call(2), something.method_1.mock_calls) self.assertNotIn(call(3), something.method_1.mock_calls) - something.method_1.wait_until_any_call(3) + something.method_1.wait_until_any_call_with(3) self.assertIn(call(2), something.method_1.mock_calls) - something.method_1.wait_until_any_call(2) + something.method_1.wait_until_any_call_with(2) - def test_wait_until_any_call_keywords(self): + def test_wait_until_any_call_with_keywords(self): waitable_mock = self._make_mock(spec=Something) with patch(f"{__name__}.Something", waitable_mock): @@ -187,16 +187,16 @@ def test_wait_until_any_call_keywords(self): self.run_async(something.method_1, c=3, delay=0.3) self.assertNotIn(call(a=1), something.method_1.mock_calls) - something.method_1.wait_until_any_call(a=1) + something.method_1.wait_until_any_call_with(a=1) something.method_1.assert_called_with(a=1) self.assertNotIn(call(b=2), something.method_1.mock_calls) self.assertNotIn(call(c=3), something.method_1.mock_calls) - something.method_1.wait_until_any_call(c=3) + something.method_1.wait_until_any_call_with(c=3) self.assertIn(call(b=2), something.method_1.mock_calls) - something.method_1.wait_until_any_call(b=2) + something.method_1.wait_until_any_call_with(b=2) - def test_wait_until_any_call_no_argument_fails_when_called_with_arg(self): + def test_wait_until_any_call_with_no_argument_fails_when_called_with_arg(self): waitable_mock = self._make_mock(timeout=0.01) with patch(f"{__name__}.Something", waitable_mock): @@ -205,25 +205,25 @@ def test_wait_until_any_call_no_argument_fails_when_called_with_arg(self): something.method_1.assert_called_with(1) with self.assertRaises(AssertionError): - something.method_1.wait_until_any_call() + something.method_1.wait_until_any_call_with() something.method_1() - something.method_1.wait_until_any_call() + something.method_1.wait_until_any_call_with() - def test_wait_until_any_call_global_default(self): + def test_wait_until_any_call_with_global_default(self): with patch.object(ThreadingMock, "DEFAULT_TIMEOUT"): ThreadingMock.DEFAULT_TIMEOUT = 0.01 m = self._make_mock() with self.assertRaises(AssertionError): - m.wait_until_any_call() + m.wait_until_any_call_with() with self.assertRaises(AssertionError): m.wait_until_called() m() - m.wait_until_any_call() + m.wait_until_any_call_with() assert ThreadingMock.DEFAULT_TIMEOUT != 0.01 - def test_wait_until_any_call_change_global_and_override(self): + def test_wait_until_any_call_with_change_global_and_override(self): with patch.object(ThreadingMock, "DEFAULT_TIMEOUT"): ThreadingMock.DEFAULT_TIMEOUT = 0.01 @@ -256,10 +256,10 @@ def test_reset_mock_resets_wait(self): with self.assertRaises(AssertionError): m.wait_until_called() with self.assertRaises(AssertionError): - m.wait_until_any_call() + m.wait_until_any_call_with() m() m.wait_until_called() - m.wait_until_any_call() + m.wait_until_any_call_with() m.assert_called_once() m.reset_mock() @@ -267,10 +267,10 @@ def test_reset_mock_resets_wait(self): with self.assertRaises(AssertionError): m.wait_until_called() with self.assertRaises(AssertionError): - m.wait_until_any_call() + m.wait_until_any_call_with() m() m.wait_until_called() - m.wait_until_any_call() + m.wait_until_any_call_with() m.assert_called_once() diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py index b542a9a2c51a9..7ef7e7180b31c 100644 --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -3070,7 +3070,7 @@ def wait_until_called(self, *, timeout=_timeout_unset): f" timeout({timeout}).") raise AssertionError(msg) - def wait_until_any_call(self, *args, **kwargs): + def wait_until_any_call_with(self, *args, **kwargs): """Wait until the mock object is called with given args. Waits for the timeout in seconds provided in the constructor. From webhook-mailer at python.org Tue Jul 4 14:20:04 2023 From: webhook-mailer at python.org (hugovk) Date: Tue, 04 Jul 2023 18:20:04 -0000 Subject: [Python-checkins] gh-106217: Truncate the issue body size of `new-bugs-announce-notifier` (#106423) Message-ID: https://github.com/python/cpython/commit/c5dacc8fa0c3013be8b457afac996bdae1dc12d2 commit: c5dacc8fa0c3013be8b457afac996bdae1dc12d2 branch: main author: Nikita Sobolev committer: hugovk date: 2023-07-04T21:20:00+03:00 summary: gh-106217: Truncate the issue body size of `new-bugs-announce-notifier` (#106423) files: M .github/workflows/new-bugs-announce-notifier.yml diff --git a/.github/workflows/new-bugs-announce-notifier.yml b/.github/workflows/new-bugs-announce-notifier.yml index e3572db670693..80514b4d2ca57 100644 --- a/.github/workflows/new-bugs-announce-notifier.yml +++ b/.github/workflows/new-bugs-announce-notifier.yml @@ -44,7 +44,7 @@ jobs: // We need to truncate the body size, because the max size for // the whole payload is 16kb. We want to be safe and assume that // body can take up to ~8kb of space. - body : issue.data.body.substring(8000) + body : issue.data.body.substring(0, 8000) }; const data = { From webhook-mailer at python.org Tue Jul 4 15:45:36 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Tue, 04 Jul 2023 19:45:36 -0000 Subject: [Python-checkins] Add some codeowners for `Tools/clinic/` (#106430) Message-ID: https://github.com/python/cpython/commit/eeb5c63179eba0390c8937d0803a41092225be7e commit: eeb5c63179eba0390c8937d0803a41092225be7e branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-04T19:45:32Z summary: Add some codeowners for `Tools/clinic/` (#106430) Co-authored-by: Erlend E. Aasland files: M .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index c044e9b8f88c5..5b471c79f75ee 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -173,3 +173,7 @@ Doc/c-api/stable.rst @encukou # zipfile.Path **/*zipfile/*_path.py @jaraco + +# Argument Clinic +/Tools/clinic/** @erlend-aasland @AlexWaygood +/Lib/test/test_clinic.py @erlend-aasland @AlexWaygood From webhook-mailer at python.org Tue Jul 4 16:51:39 2023 From: webhook-mailer at python.org (barneygale) Date: Tue, 04 Jul 2023 20:51:39 -0000 Subject: [Python-checkins] [3.12] GH-106330: Fix matching of empty path in `pathlib.PurePath.match()` (GH-106331) (GH-106372) Message-ID: https://github.com/python/cpython/commit/d5ed72b696f2d26d85f3599abf0693545a1ac4e2 commit: d5ed72b696f2d26d85f3599abf0693545a1ac4e2 branch: 3.12 author: Barney Gale committer: barneygale date: 2023-07-04T21:51:36+01:00 summary: [3.12] GH-106330: Fix matching of empty path in `pathlib.PurePath.match()` (GH-106331) (GH-106372) We match paths using the `_lines` attribute, which is derived from the path's string representation. The bug arises because an empty path's string representation is `'.'` (not `''`), which is matched by the `'*'` wildcard. (cherry picked from commit b4efdf8cda8fbbd0ca53b457d5f6e46a59348caf) files: A Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Lib/pathlib.py b/Lib/pathlib.py index d279fd2958b17..b99bf6e7dd240 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -127,12 +127,19 @@ def _compile_pattern_lines(pattern_lines, case_sensitive): # Match the start of the path, or just after a path separator parts = ['^'] for part in pattern_lines.splitlines(keepends=True): - # We slice off the common prefix and suffix added by translate() to - # ensure that re.DOTALL is not set, and the end of the string not - # matched, respectively. With DOTALL not set, '*' wildcards will not - # match path separators, because the '.' characters in the pattern - # will not match newlines. - parts.append(fnmatch.translate(part)[_FNMATCH_SLICE]) + if part == '*\n': + part = r'.+\n' + elif part == '*': + part = r'.+' + else: + # Any other component: pass to fnmatch.translate(). We slice off + # the common prefix and suffix added by translate() to ensure that + # re.DOTALL is not set, and the end of the string not matched, + # respectively. With DOTALL not set, '*' wildcards will not match + # path separators, because the '.' characters in the pattern will + # not match newlines. + part = fnmatch.translate(part)[_FNMATCH_SLICE] + parts.append(part) # Match the end of the path, always. parts.append(r'\Z') flags = re.MULTILINE @@ -501,8 +508,12 @@ def _lines(self): try: return self._lines_cached except AttributeError: - trans = _SWAP_SEP_AND_NEWLINE[self._flavour.sep] - self._lines_cached = str(self).translate(trans) + path_str = str(self) + if path_str == '.': + self._lines_cached = '' + else: + trans = _SWAP_SEP_AND_NEWLINE[self._flavour.sep] + self._lines_cached = path_str.translate(trans) return self._lines_cached def __eq__(self, other): diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index f716f1075bcb3..1d28a782f44ab 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -317,6 +317,10 @@ def test_match_common(self): self.assertTrue(P('A.py').match('a.PY', case_sensitive=False)) self.assertFalse(P('c:/a/B.Py').match('C:/A/*.pY', case_sensitive=True)) self.assertTrue(P('/a/b/c.py').match('/A/*/*.Py', case_sensitive=False)) + # Matching against empty path + self.assertFalse(P().match('*')) + self.assertTrue(P().match('**')) + self.assertFalse(P().match('**/*')) def test_ordering_common(self): # Ordering is tuple-alike. diff --git a/Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst b/Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst new file mode 100644 index 0000000000000..c1f55ab658b51 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst @@ -0,0 +1,2 @@ +Fix incorrect matching of empty paths in :meth:`pathlib.PurePath.match`. +This bug was introduced in Python 3.12.0 beta 1. From webhook-mailer at python.org Tue Jul 4 17:42:15 2023 From: webhook-mailer at python.org (gvanrossum) Date: Tue, 04 Jul 2023 21:42:15 -0000 Subject: [Python-checkins] gh-106320: Fix specialize.c compilation by including pycore_pylifecycle.h (#106434) Message-ID: https://github.com/python/cpython/commit/17af98227f883a93bfbd866b4bcc719ba9d682a1 commit: 17af98227f883a93bfbd866b4bcc719ba9d682a1 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-04T21:42:12Z summary: gh-106320: Fix specialize.c compilation by including pycore_pylifecycle.h (#106434) Compilation of Python/specialize.c was broken on macOS for me by gh-106400. files: M Python/specialize.c diff --git a/Python/specialize.c b/Python/specialize.c index 22c58e2c46fc3..a3fce2e677591 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -9,6 +9,7 @@ #include "pycore_opcode.h" // _PyOpcode_Caches #include "structmember.h" // struct PyMemberDef, T_OFFSET_EX #include "pycore_descrobject.h" +#include "pycore_pylifecycle.h" // _PyOS_URandomNonblock() #include // rand() From webhook-mailer at python.org Tue Jul 4 18:13:33 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 04 Jul 2023 22:13:33 -0000 Subject: [Python-checkins] gh-104050: Annotate toplevel functions in clinic.py (#106435) Message-ID: https://github.com/python/cpython/commit/110b97c94ce2544dc731298e35dfb003d58af626 commit: 110b97c94ce2544dc731298e35dfb003d58af626 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-04T22:13:30Z summary: gh-104050: Annotate toplevel functions in clinic.py (#106435) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index c02c82876591f..3d18d9560bc28 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -575,7 +575,7 @@ def permute_optional_groups(left, required, right): return tuple(accumulator) -def strip_leading_and_trailing_blank_lines(s): +def strip_leading_and_trailing_blank_lines(s: str) -> str: lines = s.rstrip().split('\n') while lines: line = lines[0] @@ -585,7 +585,11 @@ def strip_leading_and_trailing_blank_lines(s): return '\n'.join(lines) @functools.lru_cache() -def normalize_snippet(s, *, indent=0): +def normalize_snippet( + s: str, + *, + indent: int = 0 +) -> str: """ Reformats s: * removes leading and trailing blank lines @@ -599,7 +603,11 @@ def normalize_snippet(s, *, indent=0): return s -def declare_parser(f, *, hasformat=False): +def declare_parser( + f: Function, + *, + hasformat: bool = False +) -> str: """ Generates the code template for a static local PyArg_Parser variable, with an initializer. For core code (incl. builtin modules) the @@ -658,7 +666,10 @@ def declare_parser(f, *, hasformat=False): return normalize_snippet(declarations) -def wrap_declarations(text, length=78): +def wrap_declarations( + text: str, + length: int = 78 +) -> str: """ A simple-minded text wrapper for C function declarations. @@ -680,14 +691,14 @@ def wrap_declarations(text, length=78): if not after_l_paren: lines.append(line) continue - parameters, _, after_r_paren = after_l_paren.partition(')') + in_paren, _, after_r_paren = after_l_paren.partition(')') if not _: lines.append(line) continue - if ',' not in parameters: + if ',' not in in_paren: lines.append(line) continue - parameters = [x.strip() + ", " for x in parameters.split(',')] + parameters = [x.strip() + ", " for x in in_paren.split(',')] prefix += "(" if len(prefix) < length: spaces = " " * len(prefix) @@ -1589,7 +1600,12 @@ def OverrideStdioWith(stdout): sys.stdout = saved_stdout -def create_regex(before, after, word=True, whole_line=True): +def create_regex( + before: str, + after: str, + word: bool = True, + whole_line: bool = True +) -> re.Pattern[str]: """Create an re object for matching marker lines.""" group_re = r"\w+" if word else ".+" pattern = r'{}({}){}' @@ -1985,7 +2001,7 @@ def file_changed(filename: str, new_contents: str) -> bool: return True -def write_file(filename: str, new_contents: str): +def write_file(filename: str, new_contents: str) -> None: # Atomic write using a temporary file and os.replace() filename_new = f"{filename}.new" with open(filename_new, "w", encoding="utf-8") as fp: @@ -2602,7 +2618,10 @@ def __getattribute__(self, name: str): fail("Stepped on a land mine, trying to access attribute " + repr(name) + ":\n" + self.__message__) -def add_c_converter(f, name=None): +def add_c_converter( + f: type[CConverter], + name: str | None = None +) -> type[CConverter]: if not name: name = f.__name__ if not name.endswith('_converter'): @@ -2620,7 +2639,10 @@ def add_default_legacy_c_converter(cls): legacy_converters[cls.format_unit] = cls return cls -def add_legacy_c_converter(format_unit, **kwargs): +def add_legacy_c_converter( + format_unit: str, + **kwargs +) -> Callable[[ConverterType], ConverterType]: """ Adds a legacy converter. """ @@ -3887,7 +3909,9 @@ def parse_arg(self, argname: str, displayname: str) -> str: return super().parse_arg(argname, displayname) -def correct_name_for_self(f) -> tuple[str, str]: +def correct_name_for_self( + f: Function +) -> tuple[str, str]: if f.kind in (CALLABLE, METHOD_INIT): if f.cls: return "PyObject *", "self" @@ -3898,7 +3922,9 @@ def correct_name_for_self(f) -> tuple[str, str]: return "PyTypeObject *", "type" raise RuntimeError("Unhandled type of function f: " + repr(f.kind)) -def required_type_for_self_for_parser(f): +def required_type_for_self_for_parser( + f: Function +) -> str | None: type, _ = correct_name_for_self(f) if f.kind in (METHOD_INIT, METHOD_NEW, STATIC_METHOD, CLASS_METHOD): return type @@ -4193,7 +4219,12 @@ class float_return_converter(double_return_converter): cast = '(double)' -def eval_ast_expr(node, globals, *, filename='-'): +def eval_ast_expr( + node: ast.expr, + globals: dict[str, Any], + *, + filename: str = '-' +) -> FunctionType: """ Takes an ast.Expr node. Compiles and evaluates it. Returns the result of the expression. @@ -4205,8 +4236,8 @@ def eval_ast_expr(node, globals, *, filename='-'): if isinstance(node, ast.Expr): node = node.value - node = ast.Expression(node) - co = compile(node, filename, 'eval') + expr = ast.Expression(node) + co = compile(expr, filename, 'eval') fn = FunctionType(co, globals) return fn() From webhook-mailer at python.org Tue Jul 4 18:15:13 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 04 Jul 2023 22:15:13 -0000 Subject: [Python-checkins] gh-104050: Annotate Argument Clinic parameter permutation helpers (#106431) Message-ID: https://github.com/python/cpython/commit/22087516bca0b339a04742765febc9c20a9d6b21 commit: 22087516bca0b339a04742765febc9c20a9d6b21 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-04T22:15:10Z summary: gh-104050: Annotate Argument Clinic parameter permutation helpers (#106431) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 3d18d9560bc28..898361474f721 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -28,7 +28,12 @@ import textwrap import traceback -from collections.abc import Callable +from collections.abc import ( + Callable, + Iterable, + Iterator, + Sequence, +) from types import FunctionType, NoneType from typing import ( Any, @@ -516,7 +521,13 @@ class PythonLanguage(Language): checksum_line = "#/*[{dsl_name} end generated code: {arguments}]*/" -def permute_left_option_groups(l): +ParamGroup = Iterable["Parameter"] +ParamTuple = tuple["Parameter", ...] + + +def permute_left_option_groups( + l: Sequence[ParamGroup] +) -> Iterator[ParamTuple]: """ Given [(1,), (2,), (3,)], should yield: () @@ -525,13 +536,15 @@ def permute_left_option_groups(l): (1, 2, 3) """ yield tuple() - accumulator = [] + accumulator: list[Parameter] = [] for group in reversed(l): accumulator = list(group) + accumulator yield tuple(accumulator) -def permute_right_option_groups(l): +def permute_right_option_groups( + l: Sequence[ParamGroup] +) -> Iterator[ParamTuple]: """ Given [(1,), (2,), (3,)], should yield: () @@ -540,13 +553,17 @@ def permute_right_option_groups(l): (1, 2, 3) """ yield tuple() - accumulator = [] + accumulator: list[Parameter] = [] for group in l: accumulator.extend(group) yield tuple(accumulator) -def permute_optional_groups(left, required, right): +def permute_optional_groups( + left: Sequence[ParamGroup], + required: ParamGroup, + right: Sequence[ParamGroup] +) -> tuple[ParamTuple, ...]: """ Generator function that computes the set of acceptable argument lists for the provided iterables of @@ -561,7 +578,7 @@ def permute_optional_groups(left, required, right): if left: raise ValueError("required is empty but left is not") - accumulator = [] + accumulator: list[ParamTuple] = [] counts = set() for r in permute_right_option_groups(right): for l in permute_left_option_groups(left): From webhook-mailer at python.org Tue Jul 4 18:36:00 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 04 Jul 2023 22:36:00 -0000 Subject: [Python-checkins] gh-106368: Add tests for formatting helpers in Argument Clinic (#106415) Message-ID: https://github.com/python/cpython/commit/2fb9480c8313ab524d333b18e4af09f05f5b8afa commit: 2fb9480c8313ab524d333b18e4af09f05f5b8afa branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-04T22:35:57Z summary: gh-106368: Add tests for formatting helpers in Argument Clinic (#106415) Co-authored-by: Alex Waygood files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 544e7323e4f60..c095d14234375 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1692,5 +1692,169 @@ def test_permute_optional_groups(self): self.assertEqual(actual, expected) +class FormatHelperTests(unittest.TestCase): + + def test_strip_leading_and_trailing_blank_lines(self): + dataset = ( + # Input lines, expected output. + ("a\nb", "a\nb"), + ("a\nb\n", "a\nb"), + ("a\nb ", "a\nb"), + ("\na\nb\n\n", "a\nb"), + ("\n\na\nb\n\n", "a\nb"), + ("\n\na\n\nb\n\n", "a\n\nb"), + # Note, leading whitespace is preserved: + (" a\nb", " a\nb"), + (" a\nb ", " a\nb"), + (" \n \n a\nb \n \n ", " a\nb"), + ) + for lines, expected in dataset: + with self.subTest(lines=lines, expected=expected): + out = clinic.strip_leading_and_trailing_blank_lines(lines) + self.assertEqual(out, expected) + + def test_normalize_snippet(self): + snippet = """ + one + two + three + """ + + # Expected outputs: + zero_indent = ( + "one\n" + "two\n" + "three" + ) + four_indent = ( + " one\n" + " two\n" + " three" + ) + eight_indent = ( + " one\n" + " two\n" + " three" + ) + expected_outputs = {0: zero_indent, 4: four_indent, 8: eight_indent} + for indent, expected in expected_outputs.items(): + with self.subTest(indent=indent): + actual = clinic.normalize_snippet(snippet, indent=indent) + self.assertEqual(actual, expected) + + def test_accumulator(self): + acc = clinic.text_accumulator() + self.assertEqual(acc.output(), "") + acc.append("a") + self.assertEqual(acc.output(), "a") + self.assertEqual(acc.output(), "") + acc.append("b") + self.assertEqual(acc.output(), "b") + self.assertEqual(acc.output(), "") + acc.append("c") + acc.append("d") + self.assertEqual(acc.output(), "cd") + self.assertEqual(acc.output(), "") + + def test_quoted_for_c_string(self): + dataset = ( + # input, expected + (r"abc", r"abc"), + (r"\abc", r"\\abc"), + (r"\a\bc", r"\\a\\bc"), + (r"\a\\bc", r"\\a\\\\bc"), + (r'"abc"', r'\"abc\"'), + (r"'a'", r"\'a\'"), + ) + for line, expected in dataset: + with self.subTest(line=line, expected=expected): + out = clinic.quoted_for_c_string(line) + self.assertEqual(out, expected) + + def test_rstrip_lines(self): + lines = ( + "a \n" + "b\n" + " c\n" + " d \n" + ) + expected = ( + "a\n" + "b\n" + " c\n" + " d\n" + ) + out = clinic.rstrip_lines(lines) + self.assertEqual(out, expected) + + def test_format_escape(self): + line = "{}, {a}" + expected = "{{}}, {{a}}" + out = clinic.format_escape(line) + self.assertEqual(out, expected) + + def test_indent_all_lines(self): + # Blank lines are expected to be unchanged. + self.assertEqual(clinic.indent_all_lines("", prefix="bar"), "") + + lines = ( + "one\n" + "two" # The missing newline is deliberate. + ) + expected = ( + "barone\n" + "bartwo" + ) + out = clinic.indent_all_lines(lines, prefix="bar") + self.assertEqual(out, expected) + + # If last line is empty, expect it to be unchanged. + lines = ( + "\n" + "one\n" + "two\n" + "" + ) + expected = ( + "bar\n" + "barone\n" + "bartwo\n" + "" + ) + out = clinic.indent_all_lines(lines, prefix="bar") + self.assertEqual(out, expected) + + def test_suffix_all_lines(self): + # Blank lines are expected to be unchanged. + self.assertEqual(clinic.suffix_all_lines("", suffix="foo"), "") + + lines = ( + "one\n" + "two" # The missing newline is deliberate. + ) + expected = ( + "onefoo\n" + "twofoo" + ) + out = clinic.suffix_all_lines(lines, suffix="foo") + self.assertEqual(out, expected) + + # If last line is empty, expect it to be unchanged. + lines = ( + "\n" + "one\n" + "two\n" + "" + ) + expected = ( + "foo\n" + "onefoo\n" + "twofoo\n" + "" + ) + out = clinic.suffix_all_lines(lines, suffix="foo") + self.assertEqual(out, expected) + + if __name__ == "__main__": unittest.main() From webhook-mailer at python.org Tue Jul 4 19:00:35 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Tue, 04 Jul 2023 23:00:35 -0000 Subject: [Python-checkins] [3.12] gh-106368: Add tests for formatting helpers in Argument Clinic (GH-106415) (#106438) Message-ID: https://github.com/python/cpython/commit/6f684044a98f3ae26f15c2de5bd61a30927eddf0 commit: 6f684044a98f3ae26f15c2de5bd61a30927eddf0 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-04T23:00:32Z summary: [3.12] gh-106368: Add tests for formatting helpers in Argument Clinic (GH-106415) (#106438) gh-106368: Add tests for formatting helpers in Argument Clinic (GH-106415) (cherry picked from commit 2fb9480c8313ab524d333b18e4af09f05f5b8afa) Co-authored-by: Erlend E. Aasland Co-authored-by: Alex Waygood files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 544e7323e4f60..c095d14234375 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1692,5 +1692,169 @@ def test_permute_optional_groups(self): self.assertEqual(actual, expected) +class FormatHelperTests(unittest.TestCase): + + def test_strip_leading_and_trailing_blank_lines(self): + dataset = ( + # Input lines, expected output. + ("a\nb", "a\nb"), + ("a\nb\n", "a\nb"), + ("a\nb ", "a\nb"), + ("\na\nb\n\n", "a\nb"), + ("\n\na\nb\n\n", "a\nb"), + ("\n\na\n\nb\n\n", "a\n\nb"), + # Note, leading whitespace is preserved: + (" a\nb", " a\nb"), + (" a\nb ", " a\nb"), + (" \n \n a\nb \n \n ", " a\nb"), + ) + for lines, expected in dataset: + with self.subTest(lines=lines, expected=expected): + out = clinic.strip_leading_and_trailing_blank_lines(lines) + self.assertEqual(out, expected) + + def test_normalize_snippet(self): + snippet = """ + one + two + three + """ + + # Expected outputs: + zero_indent = ( + "one\n" + "two\n" + "three" + ) + four_indent = ( + " one\n" + " two\n" + " three" + ) + eight_indent = ( + " one\n" + " two\n" + " three" + ) + expected_outputs = {0: zero_indent, 4: four_indent, 8: eight_indent} + for indent, expected in expected_outputs.items(): + with self.subTest(indent=indent): + actual = clinic.normalize_snippet(snippet, indent=indent) + self.assertEqual(actual, expected) + + def test_accumulator(self): + acc = clinic.text_accumulator() + self.assertEqual(acc.output(), "") + acc.append("a") + self.assertEqual(acc.output(), "a") + self.assertEqual(acc.output(), "") + acc.append("b") + self.assertEqual(acc.output(), "b") + self.assertEqual(acc.output(), "") + acc.append("c") + acc.append("d") + self.assertEqual(acc.output(), "cd") + self.assertEqual(acc.output(), "") + + def test_quoted_for_c_string(self): + dataset = ( + # input, expected + (r"abc", r"abc"), + (r"\abc", r"\\abc"), + (r"\a\bc", r"\\a\\bc"), + (r"\a\\bc", r"\\a\\\\bc"), + (r'"abc"', r'\"abc\"'), + (r"'a'", r"\'a\'"), + ) + for line, expected in dataset: + with self.subTest(line=line, expected=expected): + out = clinic.quoted_for_c_string(line) + self.assertEqual(out, expected) + + def test_rstrip_lines(self): + lines = ( + "a \n" + "b\n" + " c\n" + " d \n" + ) + expected = ( + "a\n" + "b\n" + " c\n" + " d\n" + ) + out = clinic.rstrip_lines(lines) + self.assertEqual(out, expected) + + def test_format_escape(self): + line = "{}, {a}" + expected = "{{}}, {{a}}" + out = clinic.format_escape(line) + self.assertEqual(out, expected) + + def test_indent_all_lines(self): + # Blank lines are expected to be unchanged. + self.assertEqual(clinic.indent_all_lines("", prefix="bar"), "") + + lines = ( + "one\n" + "two" # The missing newline is deliberate. + ) + expected = ( + "barone\n" + "bartwo" + ) + out = clinic.indent_all_lines(lines, prefix="bar") + self.assertEqual(out, expected) + + # If last line is empty, expect it to be unchanged. + lines = ( + "\n" + "one\n" + "two\n" + "" + ) + expected = ( + "bar\n" + "barone\n" + "bartwo\n" + "" + ) + out = clinic.indent_all_lines(lines, prefix="bar") + self.assertEqual(out, expected) + + def test_suffix_all_lines(self): + # Blank lines are expected to be unchanged. + self.assertEqual(clinic.suffix_all_lines("", suffix="foo"), "") + + lines = ( + "one\n" + "two" # The missing newline is deliberate. + ) + expected = ( + "onefoo\n" + "twofoo" + ) + out = clinic.suffix_all_lines(lines, suffix="foo") + self.assertEqual(out, expected) + + # If last line is empty, expect it to be unchanged. + lines = ( + "\n" + "one\n" + "two\n" + "" + ) + expected = ( + "foo\n" + "onefoo\n" + "twofoo\n" + "" + ) + out = clinic.suffix_all_lines(lines, suffix="foo") + self.assertEqual(out, expected) + + if __name__ == "__main__": unittest.main() From webhook-mailer at python.org Tue Jul 4 19:08:01 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 04 Jul 2023 23:08:01 -0000 Subject: [Python-checkins] gh-104050: Partially annotate Argument Clinic CLanguage class (#106437) Message-ID: https://github.com/python/cpython/commit/7bb9fa5ae486912d5d0a9372f213ba6a72c4cde1 commit: 7bb9fa5ae486912d5d0a9372f213ba6a72c4cde1 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-04T23:07:57Z summary: gh-104050: Partially annotate Argument Clinic CLanguage class (#106437) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 898361474f721..5f5d024b5aa6f 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -752,10 +752,14 @@ def __init__(self, filename): self.cpp = cpp.Monitor(filename) self.cpp.fail = fail - def parse_line(self, line): + def parse_line(self, line: str) -> None: self.cpp.writeline(line) - def render(self, clinic, signatures): + def render( + self, + clinic: Clinic | None, + signatures: Iterable[Function] + ) -> str: function = None for o in signatures: if isinstance(o, Function): @@ -764,7 +768,10 @@ def render(self, clinic, signatures): function = o return self.render_function(clinic, function) - def docstring_for_c_string(self, f): + def docstring_for_c_string( + self, + f: Function + ) -> str: if re.search(r'[^\x00-\x7F]', f.docstring): warn("Non-ascii character appear in docstring.") @@ -1345,7 +1352,7 @@ def parser_body(prototype, *fields, declarations=''): return d2 @staticmethod - def group_to_variable_name(group): + def group_to_variable_name(group: int) -> str: adjective = "left_" if group < 0 else "right_" return "group_" + adjective + str(abs(group)) @@ -1441,8 +1448,12 @@ def render_option_group_parsing(self, f, template_dict): add("}") template_dict['option_group_parsing'] = format_escape(output()) - def render_function(self, clinic, f): - if not f: + def render_function( + self, + clinic: Clinic | None, + f: Function | None + ) -> str: + if f is None or clinic is None: return "" add, output = text_accumulator() @@ -1504,10 +1515,12 @@ def render_function(self, clinic, f): template_dict = {} + assert isinstance(f.full_name, str) full_name = f.full_name template_dict['full_name'] = full_name if new_or_init: + assert isinstance(f.cls, Class) name = f.cls.name else: name = f.name From webhook-mailer at python.org Wed Jul 5 01:08:29 2023 From: webhook-mailer at python.org (hauntsaninja) Date: Wed, 05 Jul 2023 05:08:29 -0000 Subject: [Python-checkins] tp_flags docs: fix indentation (#106420) Message-ID: https://github.com/python/cpython/commit/ad075682ba49c3d90cb9b09341f8bf2ea56761d8 commit: ad075682ba49c3d90cb9b09341f8bf2ea56761d8 branch: main author: Jelle Zijlstra committer: hauntsaninja <12621235+hauntsaninja at users.noreply.github.com> date: 2023-07-05T05:08:25Z summary: tp_flags docs: fix indentation (#106420) files: M Doc/c-api/typeobj.rst diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index c6e783acdf065..239c191457f51 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1143,14 +1143,14 @@ and :c:type:`PyType_Type` effectively act as defaults.) :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is inherited whenever :c:member:`~PyTypeObject.tp_descr_get` is inherited. - .. data:: Py_TPFLAGS_MANAGED_DICT + .. data:: Py_TPFLAGS_MANAGED_DICT - This bit indicates that instances of the class have a ``__dict__`` - attribute, and that the space for the dictionary is managed by the VM. + This bit indicates that instances of the class have a ``__dict__`` + attribute, and that the space for the dictionary is managed by the VM. - If this flag is set, :const:`Py_TPFLAGS_HAVE_GC` should also be set. + If this flag is set, :const:`Py_TPFLAGS_HAVE_GC` should also be set. - .. versionadded:: 3.12 + .. versionadded:: 3.12 **Inheritance:** @@ -1158,12 +1158,12 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:member:`~PyTypeObject.tp_dictoffset` field is set in a superclass. - .. data:: Py_TPFLAGS_MANAGED_WEAKREF + .. data:: Py_TPFLAGS_MANAGED_WEAKREF - This bit indicates that instances of the class should be weakly - referenceable. + This bit indicates that instances of the class should be weakly + referenceable. - .. versionadded:: 3.12 + .. versionadded:: 3.12 **Inheritance:** From webhook-mailer at python.org Wed Jul 5 02:12:50 2023 From: webhook-mailer at python.org (hauntsaninja) Date: Wed, 05 Jul 2023 06:12:50 -0000 Subject: [Python-checkins] [3.12] tp_flags docs: fix indentation (GH-106420) (#106442) Message-ID: https://github.com/python/cpython/commit/5784acd08c89d91f616faba3a0b7b88e0ba4ce67 commit: 5784acd08c89d91f616faba3a0b7b88e0ba4ce67 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hauntsaninja <12621235+hauntsaninja at users.noreply.github.com> date: 2023-07-04T23:12:47-07:00 summary: [3.12] tp_flags docs: fix indentation (GH-106420) (#106442) Co-authored-by: Jelle Zijlstra files: M Doc/c-api/typeobj.rst diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index c6e783acdf065..239c191457f51 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1143,14 +1143,14 @@ and :c:type:`PyType_Type` effectively act as defaults.) :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is inherited whenever :c:member:`~PyTypeObject.tp_descr_get` is inherited. - .. data:: Py_TPFLAGS_MANAGED_DICT + .. data:: Py_TPFLAGS_MANAGED_DICT - This bit indicates that instances of the class have a ``__dict__`` - attribute, and that the space for the dictionary is managed by the VM. + This bit indicates that instances of the class have a ``__dict__`` + attribute, and that the space for the dictionary is managed by the VM. - If this flag is set, :const:`Py_TPFLAGS_HAVE_GC` should also be set. + If this flag is set, :const:`Py_TPFLAGS_HAVE_GC` should also be set. - .. versionadded:: 3.12 + .. versionadded:: 3.12 **Inheritance:** @@ -1158,12 +1158,12 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:member:`~PyTypeObject.tp_dictoffset` field is set in a superclass. - .. data:: Py_TPFLAGS_MANAGED_WEAKREF + .. data:: Py_TPFLAGS_MANAGED_WEAKREF - This bit indicates that instances of the class should be weakly - referenceable. + This bit indicates that instances of the class should be weakly + referenceable. - .. versionadded:: 3.12 + .. versionadded:: 3.12 **Inheritance:** From webhook-mailer at python.org Wed Jul 5 04:33:09 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 05 Jul 2023 08:33:09 -0000 Subject: [Python-checkins] gh-104683: Rename Lib/test/clinic.test as Lib/test/clinic.test.c (#106443) Message-ID: https://github.com/python/cpython/commit/a941bd6c53ac4646926292557a7bb2a86f8025c3 commit: a941bd6c53ac4646926292557a7bb2a86f8025c3 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-05T08:33:05Z summary: gh-104683: Rename Lib/test/clinic.test as Lib/test/clinic.test.c (#106443) files: A Lib/test/clinic.test.c D Lib/test/clinic.test M Lib/test/test_clinic.py diff --git a/Lib/test/clinic.test b/Lib/test/clinic.test.c similarity index 100% rename from Lib/test/clinic.test rename to Lib/test/clinic.test.c diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index c095d14234375..7c46e8a81803a 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1089,15 +1089,16 @@ class ClinicExternalTest(TestCase): maxDiff = None def test_external(self): + CLINIC_TEST = 'clinic.test.c' # bpo-42398: Test that the destination file is left unchanged if the # content does not change. Moreover, check also that the file # modification time does not change in this case. - source = support.findfile('clinic.test') + source = support.findfile(CLINIC_TEST) with open(source, 'r', encoding='utf-8') as f: orig_contents = f.read() with os_helper.temp_dir() as tmp_dir: - testfile = os.path.join(tmp_dir, 'clinic.test.c') + testfile = os.path.join(tmp_dir, CLINIC_TEST) with open(testfile, 'w', encoding='utf-8') as f: f.write(orig_contents) old_mtime_ns = os.stat(testfile).st_mtime_ns From webhook-mailer at python.org Wed Jul 5 04:58:03 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 05 Jul 2023 08:58:03 -0000 Subject: [Python-checkins] [3.12] gh-104683: Rename Lib/test/clinic.test as Lib/test/clinic.test.c (GH-106443) (#106444) Message-ID: https://github.com/python/cpython/commit/00c522a81c726f3a15b16ae67bba4840431afdaf commit: 00c522a81c726f3a15b16ae67bba4840431afdaf branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-05T08:57:59Z summary: [3.12] gh-104683: Rename Lib/test/clinic.test as Lib/test/clinic.test.c (GH-106443) (#106444) gh-104683: Rename Lib/test/clinic.test as Lib/test/clinic.test.c (GH-106443) (cherry picked from commit a941bd6c53ac4646926292557a7bb2a86f8025c3) Co-authored-by: Erlend E. Aasland files: A Lib/test/clinic.test.c D Lib/test/clinic.test M Lib/test/test_clinic.py diff --git a/Lib/test/clinic.test b/Lib/test/clinic.test.c similarity index 100% rename from Lib/test/clinic.test rename to Lib/test/clinic.test.c diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index c095d14234375..7c46e8a81803a 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1089,15 +1089,16 @@ class ClinicExternalTest(TestCase): maxDiff = None def test_external(self): + CLINIC_TEST = 'clinic.test.c' # bpo-42398: Test that the destination file is left unchanged if the # content does not change. Moreover, check also that the file # modification time does not change in this case. - source = support.findfile('clinic.test') + source = support.findfile(CLINIC_TEST) with open(source, 'r', encoding='utf-8') as f: orig_contents = f.read() with os_helper.temp_dir() as tmp_dir: - testfile = os.path.join(tmp_dir, 'clinic.test.c') + testfile = os.path.join(tmp_dir, CLINIC_TEST) with open(testfile, 'w', encoding='utf-8') as f: f.write(orig_contents) old_mtime_ns = os.stat(testfile).st_mtime_ns From webhook-mailer at python.org Wed Jul 5 05:03:29 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 05 Jul 2023 09:03:29 -0000 Subject: [Python-checkins] [3.11] gh-104683: Rename Lib/test/clinic.test as Lib/test/clinic.test.c (GH-106443) (#106445) Message-ID: https://github.com/python/cpython/commit/edb29f0fbed652f1b03c107db8d055dfb8f179e1 commit: edb29f0fbed652f1b03c107db8d055dfb8f179e1 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-05T09:03:25Z summary: [3.11] gh-104683: Rename Lib/test/clinic.test as Lib/test/clinic.test.c (GH-106443) (#106445) gh-104683: Rename Lib/test/clinic.test as Lib/test/clinic.test.c (GH-106443) (cherry picked from commit a941bd6c53ac4646926292557a7bb2a86f8025c3) Co-authored-by: Erlend E. Aasland files: A Lib/test/clinic.test.c D Lib/test/clinic.test M Lib/test/test_clinic.py diff --git a/Lib/test/clinic.test b/Lib/test/clinic.test.c similarity index 100% rename from Lib/test/clinic.test rename to Lib/test/clinic.test.c diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 8c9fe997f1adb..f2e0c5e03f8b9 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1034,15 +1034,16 @@ class ClinicExternalTest(TestCase): maxDiff = None def test_external(self): + CLINIC_TEST = 'clinic.test.c' # bpo-42398: Test that the destination file is left unchanged if the # content does not change. Moreover, check also that the file # modification time does not change in this case. - source = support.findfile('clinic.test') + source = support.findfile(CLINIC_TEST) with open(source, 'r', encoding='utf-8') as f: orig_contents = f.read() with os_helper.temp_dir() as tmp_dir: - testfile = os.path.join(tmp_dir, 'clinic.test.c') + testfile = os.path.join(tmp_dir, CLINIC_TEST) with open(testfile, 'w', encoding='utf-8') as f: f.write(orig_contents) old_mtime_ns = os.stat(testfile).st_mtime_ns From webhook-mailer at python.org Wed Jul 5 05:42:51 2023 From: webhook-mailer at python.org (lysnikolaou) Date: Wed, 05 Jul 2023 09:42:51 -0000 Subject: [Python-checkins] [3.12] gh-100238: Use setuptools in peg-generator and reenable tests (GH-104798) (#105135) Message-ID: https://github.com/python/cpython/commit/637102980d12e5ff99b69ddffd5f05ecfe0adeb6 commit: 637102980d12e5ff99b69ddffd5f05ecfe0adeb6 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: lysnikolaou date: 2023-07-05T11:42:47+02:00 summary: [3.12] gh-100238: Use setuptools in peg-generator and reenable tests (GH-104798) (#105135) (cherry picked from commit afa759fb800be416f69e3e9c9b3efe68006316f5) Co-authored-by: Lysandros Nikolaou files: M Lib/test/support/__init__.py M Lib/test/test_peg_generator/__init__.py M Lib/test/test_peg_generator/test_c_parser.py M Lib/test/test_peg_generator/test_pegen.py M Tools/peg_generator/pegen/build.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index c59508b40d320..3f1cc3a083a1c 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1868,15 +1868,16 @@ def missing_compiler_executable(cmd_names=[]): missing. """ - # TODO (PEP 632): alternate check without using distutils - from distutils import ccompiler, sysconfig, spawn, errors + from setuptools._distutils import ccompiler, sysconfig, spawn + from setuptools import errors + compiler = ccompiler.new_compiler() sysconfig.customize_compiler(compiler) if compiler.compiler_type == "msvc": # MSVC has no executables, so check whether initialization succeeds try: compiler.initialize() - except errors.DistutilsPlatformError: + except errors.PlatformError: return "msvc" for name in compiler.executables: if cmd_names and name not in cmd_names: diff --git a/Lib/test/test_peg_generator/__init__.py b/Lib/test/test_peg_generator/__init__.py index 7c402c3d7c5ac..77f72fcc7c6e3 100644 --- a/Lib/test/test_peg_generator/__init__.py +++ b/Lib/test/test_peg_generator/__init__.py @@ -3,9 +3,6 @@ from test import support from test.support import load_package_tests -# TODO: gh-92584: peg_generator uses distutils which was removed in Python 3.12 -raise unittest.SkipTest("distutils has been removed in Python 3.12") - if support.check_sanitizer(address=True, memory=True): # bpo-46633: Skip the test because it is too slow when Python is built diff --git a/Lib/test/test_peg_generator/test_c_parser.py b/Lib/test/test_peg_generator/test_c_parser.py index d34ffef0dbc5e..af39faeba9435 100644 --- a/Lib/test/test_peg_generator/test_c_parser.py +++ b/Lib/test/test_peg_generator/test_c_parser.py @@ -1,3 +1,5 @@ +import contextlib +import subprocess import sysconfig import textwrap import unittest @@ -8,7 +10,7 @@ from test import test_tools from test import support -from test.support import os_helper +from test.support import os_helper, import_helper from test.support.script_helper import assert_python_ok _py_cflags_nodist = sysconfig.get_config_var("PY_CFLAGS_NODIST") @@ -88,6 +90,16 @@ def setUpClass(cls): cls.library_dir = tempfile.mkdtemp(dir=cls.tmp_base) cls.addClassCleanup(shutil.rmtree, cls.library_dir) + with contextlib.ExitStack() as stack: + python_exe = stack.enter_context(support.setup_venv_with_pip_setuptools_wheel("venv")) + sitepackages = subprocess.check_output( + [python_exe, "-c", "import sysconfig; print(sysconfig.get_path('platlib'))"], + text=True, + ).strip() + stack.enter_context(import_helper.DirsOnSysPath(sitepackages)) + cls.addClassCleanup(stack.pop_all().close) + + @support.requires_venv_with_pip() def setUp(self): self._backup_config_vars = dict(sysconfig._CONFIG_VARS) cmd = support.missing_compiler_executable() diff --git a/Lib/test/test_peg_generator/test_pegen.py b/Lib/test/test_peg_generator/test_pegen.py index 30e992ed213c6..d92da7b29bff9 100644 --- a/Lib/test/test_peg_generator/test_pegen.py +++ b/Lib/test/test_peg_generator/test_pegen.py @@ -794,7 +794,7 @@ def test_soft_keyword(self) -> None: start: | "number" n=NUMBER { eval(n.string) } | "string" n=STRING { n.string } - | SOFT_KEYWORD l=NAME n=(NUMBER | NAME | STRING) { f"{l.string} = {n.string}"} + | SOFT_KEYWORD l=NAME n=(NUMBER | NAME | STRING) { l.string + " = " + n.string } """ parser_class = make_parser(grammar) self.assertEqual(parse_string("number 1", parser_class), 1) diff --git a/Tools/peg_generator/pegen/build.py b/Tools/peg_generator/pegen/build.py index 5805ff6371744..aace684045b9f 100644 --- a/Tools/peg_generator/pegen/build.py +++ b/Tools/peg_generator/pegen/build.py @@ -1,4 +1,5 @@ import itertools +import os import pathlib import sys import sysconfig @@ -27,6 +28,46 @@ def get_extra_flags(compiler_flags: str, compiler_py_flags_nodist: str) -> List[ return f"{flags} {py_flags_nodist}".split() +def fixup_build_ext(cmd): + """Function needed to make build_ext tests pass. + + When Python was built with --enable-shared on Unix, -L. is not enough to + find libpython.so, because regrtest runs in a tempdir, not in the + source directory where the .so lives. + + When Python was built with in debug mode on Windows, build_ext commands + need their debug attribute set, and it is not done automatically for + some reason. + + This function handles both of these things. Example use: + + cmd = build_ext(dist) + support.fixup_build_ext(cmd) + cmd.ensure_finalized() + + Unlike most other Unix platforms, Mac OS X embeds absolute paths + to shared libraries into executables, so the fixup is not needed there. + + Taken from distutils (was part of the CPython stdlib until Python 3.11) + """ + if os.name == 'nt': + cmd.debug = sys.executable.endswith('_d.exe') + elif sysconfig.get_config_var('Py_ENABLE_SHARED'): + # To further add to the shared builds fun on Unix, we can't just add + # library_dirs to the Extension() instance because that doesn't get + # plumbed through to the final compiler command. + runshared = sysconfig.get_config_var('RUNSHARED') + if runshared is None: + cmd.library_dirs = ['.'] + else: + if sys.platform == 'darwin': + cmd.library_dirs = [] + else: + name, equals, value = runshared.partition('=') + cmd.library_dirs = [d for d in value.split(os.pathsep) if d] + + + def compile_c_extension( generated_source_path: str, build_dir: Optional[str] = None, @@ -49,16 +90,15 @@ def compile_c_extension( static library of the common parser sources (this is useful in case you are creating multiple extensions). """ - import distutils.log - from distutils.core import Distribution, Extension - from distutils.tests.support import fixup_build_ext # type: ignore + import setuptools.logging - from distutils.ccompiler import new_compiler - from distutils.dep_util import newer_group - from distutils.sysconfig import customize_compiler + from setuptools import Extension, Distribution + from setuptools._distutils.dep_util import newer_group + from setuptools._distutils.ccompiler import new_compiler + from setuptools._distutils.sysconfig import customize_compiler if verbose: - distutils.log.set_threshold(distutils.log.DEBUG) + setuptools.logging.set_threshold(setuptools.logging.logging.DEBUG) source_file_path = pathlib.Path(generated_source_path) extension_name = source_file_path.stem From webhook-mailer at python.org Wed Jul 5 06:51:20 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 10:51:20 -0000 Subject: [Python-checkins] [3.12] gh-104692: Include commoninstall as a prerequisite for bininstall (GH-104693) (#105428) Message-ID: https://github.com/python/cpython/commit/2edec6ad9f6cbce217427e2352e577a7a83cd774 commit: 2edec6ad9f6cbce217427e2352e577a7a83cd774 branch: 3.12 author: Jeffery To committer: ambv date: 2023-07-05T12:51:16+02:00 summary: [3.12] gh-104692: Include commoninstall as a prerequisite for bininstall (GH-104693) (#105428) This ensures that `commoninstall` is completed before `bininstall` is started when parallel builds are used (`make -j install`), and so the `python3` symlink is only installed after all standard library modules are installed. (cherry picked from commit 990cb3676c2edb7e5787372d6cbe360a73367f4c) files: A Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst M Makefile.pre.in diff --git a/Makefile.pre.in b/Makefile.pre.in index e2adc3cb49f2a..12788d11d1d14 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -2014,7 +2014,11 @@ altbininstall: $(BUILDPYTHON) @FRAMEWORKPYTHONW@ fi .PHONY: bininstall -bininstall: altbininstall +# We depend on commoninstall here to make sure the installation is already usable +# before we possibly overwrite the global 'python3' symlink to avoid causing +# problems for anything else trying to run 'python3' while we install, particularly +# if we're installing in parallel with -j. +bininstall: commoninstall altbininstall if test ! -d $(DESTDIR)$(LIBPC); then \ echo "Creating directory $(LIBPC)"; \ $(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$(LIBPC); \ diff --git a/Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst b/Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst new file mode 100644 index 0000000000000..2936990999e1a --- /dev/null +++ b/Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst @@ -0,0 +1,6 @@ +Include ``commoninstall`` as a prerequisite for ``bininstall`` + +This ensures that ``commoninstall`` is completed before ``bininstall`` +is started when parallel builds are used (``make -j install``), and so +the ``python3`` symlink is only installed after all standard library +modules are installed. From webhook-mailer at python.org Wed Jul 5 06:51:30 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 10:51:30 -0000 Subject: [Python-checkins] [3.11] gh-104692: Include commoninstall as a prerequisite for bininstall (GH-104693) (#105429) Message-ID: https://github.com/python/cpython/commit/6c6a2a9143c301bd6101fb18fd7f3b96adafc43c commit: 6c6a2a9143c301bd6101fb18fd7f3b96adafc43c branch: 3.11 author: Jeffery To committer: ambv date: 2023-07-05T12:51:26+02:00 summary: [3.11] gh-104692: Include commoninstall as a prerequisite for bininstall (GH-104693) (#105429) This ensures that `commoninstall` is completed before `bininstall` is started when parallel builds are used (`make -j install`), and so the `python3` symlink is only installed after all standard library modules are installed. . (cherry picked from commit 990cb3676c2edb7e5787372d6cbe360a73367f4c) Co-authored-by: Erlend E. Aasland files: A Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst M Makefile.pre.in diff --git a/Makefile.pre.in b/Makefile.pre.in index 3ea8653de0069..3948184ad8ea3 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1839,7 +1839,11 @@ altbininstall: $(BUILDPYTHON) @FRAMEWORKPYTHONW@ $(DESTDIR)$(BINDIR)/python$(VERSION)$(EXE); \ fi -bininstall: altbininstall +# We depend on commoninstall here to make sure the installation is already usable +# before we possibly overwrite the global 'python3' symlink to avoid causing +# problems for anything else trying to run 'python3' while we install, particularly +# if we're installing in parallel with -j. +bininstall: commoninstall altbininstall if test ! -d $(DESTDIR)$(LIBPC); then \ echo "Creating directory $(LIBPC)"; \ $(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$(LIBPC); \ diff --git a/Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst b/Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst new file mode 100644 index 0000000000000..2936990999e1a --- /dev/null +++ b/Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst @@ -0,0 +1,6 @@ +Include ``commoninstall`` as a prerequisite for ``bininstall`` + +This ensures that ``commoninstall`` is completed before ``bininstall`` +is started when parallel builds are used (``make -j install``), and so +the ``python3`` symlink is only installed after all standard library +modules are installed. From webhook-mailer at python.org Wed Jul 5 06:52:20 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 10:52:20 -0000 Subject: [Python-checkins] [3.12] gh-89392: Make test_decimal discoverable (GH-106209) (#106230) Message-ID: https://github.com/python/cpython/commit/334b95b2434880e6138d430054c3054266e6020f commit: 334b95b2434880e6138d430054c3054266e6020f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T12:52:16+02:00 summary: [3.12] gh-89392: Make test_decimal discoverable (GH-106209) (#106230) gh-89392: Make test_decimal discoverable (GH-106209) (cherry picked from commit 0e24499129f3917b199a6d46fa33eeedd2c447fc) Co-authored-by: Serhiy Storchaka files: M Lib/test/test_decimal.py diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py index 67ccaab40c5ed..749496e3e9455 100644 --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -20,7 +20,7 @@ This test module can be called from command line with one parameter (Arithmetic or Behaviour) to test each part, or without parameter to test both parts. If -you're working through IDLE, you can import this test module and call test_main() +you're working through IDLE, you can import this test module and call test() with the corresponding argument. """ @@ -32,7 +32,7 @@ import unittest import numbers import locale -from test.support import (run_unittest, run_doctest, is_resource_enabled, +from test.support import (is_resource_enabled, requires_IEEE_754, requires_docstrings, requires_legacy_unicode_capi, check_sanitizer) from test.support import (TestFailed, @@ -62,6 +62,7 @@ fractions = {C:cfractions, P:pfractions} sys.modules['decimal'] = orig_sys_decimal +requires_cdecimal = unittest.skipUnless(C, "test requires C version") # Useful Test Constant Signals = { @@ -99,7 +100,7 @@ def assert_signals(cls, context, attr, expected): ] # Tests are built around these assumed context defaults. -# test_main() restores the original context. +# test() restores the original context. ORIGINAL_CONTEXT = { C: C.getcontext().copy() if C else None, P: P.getcontext().copy() @@ -133,7 +134,7 @@ def init(m): EXTRA_FUNCTIONALITY, "test requires regular build") -class IBMTestCases(unittest.TestCase): +class IBMTestCases: """Class which tests the Decimal class against the IBM test cases.""" def setUp(self): @@ -488,14 +489,10 @@ def change_max_exponent(self, exp): def change_clamp(self, clamp): self.context.clamp = clamp -class CIBMTestCases(IBMTestCases): - decimal = C -class PyIBMTestCases(IBMTestCases): - decimal = P # The following classes test the behaviour of Decimal according to PEP 327 -class ExplicitConstructionTest(unittest.TestCase): +class ExplicitConstructionTest: '''Unit tests for Explicit Construction cases of Decimal.''' def test_explicit_empty(self): @@ -838,12 +835,13 @@ def test_unicode_digits(self): for input, expected in test_values.items(): self.assertEqual(str(Decimal(input)), expected) -class CExplicitConstructionTest(ExplicitConstructionTest): + at requires_cdecimal +class CExplicitConstructionTest(ExplicitConstructionTest, unittest.TestCase): decimal = C -class PyExplicitConstructionTest(ExplicitConstructionTest): +class PyExplicitConstructionTest(ExplicitConstructionTest, unittest.TestCase): decimal = P -class ImplicitConstructionTest(unittest.TestCase): +class ImplicitConstructionTest: '''Unit tests for Implicit Construction cases of Decimal.''' def test_implicit_from_None(self): @@ -920,12 +918,13 @@ def __ne__(self, other): self.assertEqual(eval('Decimal(10)' + sym + 'E()'), '10' + rop + 'str') -class CImplicitConstructionTest(ImplicitConstructionTest): + at requires_cdecimal +class CImplicitConstructionTest(ImplicitConstructionTest, unittest.TestCase): decimal = C -class PyImplicitConstructionTest(ImplicitConstructionTest): +class PyImplicitConstructionTest(ImplicitConstructionTest, unittest.TestCase): decimal = P -class FormatTest(unittest.TestCase): +class FormatTest: '''Unit tests for the format function.''' def test_formatting(self): Decimal = self.decimal.Decimal @@ -1262,12 +1261,13 @@ def __init__(self, a): a = A.from_float(42) self.assertEqual(self.decimal.Decimal, a.a_type) -class CFormatTest(FormatTest): + at requires_cdecimal +class CFormatTest(FormatTest, unittest.TestCase): decimal = C -class PyFormatTest(FormatTest): +class PyFormatTest(FormatTest, unittest.TestCase): decimal = P -class ArithmeticOperatorsTest(unittest.TestCase): +class ArithmeticOperatorsTest: '''Unit tests for all arithmetic operators, binary and unary.''' def test_addition(self): @@ -1523,14 +1523,17 @@ def test_nan_comparisons(self): equality_ops = operator.eq, operator.ne # results when InvalidOperation is not trapped - for x, y in qnan_pairs + snan_pairs: - for op in order_ops + equality_ops: - got = op(x, y) - expected = True if op is operator.ne else False - self.assertIs(expected, got, - "expected {0!r} for operator.{1}({2!r}, {3!r}); " - "got {4!r}".format( - expected, op.__name__, x, y, got)) + with localcontext() as ctx: + ctx.traps[InvalidOperation] = 0 + + for x, y in qnan_pairs + snan_pairs: + for op in order_ops + equality_ops: + got = op(x, y) + expected = True if op is operator.ne else False + self.assertIs(expected, got, + "expected {0!r} for operator.{1}({2!r}, {3!r}); " + "got {4!r}".format( + expected, op.__name__, x, y, got)) # repeat the above, but this time trap the InvalidOperation with localcontext() as ctx: @@ -1562,9 +1565,10 @@ def test_copy_sign(self): self.assertEqual(Decimal(1).copy_sign(-2), d) self.assertRaises(TypeError, Decimal(1).copy_sign, '-2') -class CArithmeticOperatorsTest(ArithmeticOperatorsTest): + at requires_cdecimal +class CArithmeticOperatorsTest(ArithmeticOperatorsTest, unittest.TestCase): decimal = C -class PyArithmeticOperatorsTest(ArithmeticOperatorsTest): +class PyArithmeticOperatorsTest(ArithmeticOperatorsTest, unittest.TestCase): decimal = P # The following are two functions used to test threading in the next class @@ -1654,7 +1658,7 @@ def thfunc2(cls): @threading_helper.requires_working_threading() -class ThreadingTest(unittest.TestCase): +class ThreadingTest: '''Unit tests for thread local contexts in Decimal.''' # Take care executing this test from IDLE, there's an issue in threading @@ -1699,13 +1703,14 @@ def test_threading(self): DefaultContext.Emin = save_emin -class CThreadingTest(ThreadingTest): + at requires_cdecimal +class CThreadingTest(ThreadingTest, unittest.TestCase): decimal = C -class PyThreadingTest(ThreadingTest): +class PyThreadingTest(ThreadingTest, unittest.TestCase): decimal = P -class UsabilityTest(unittest.TestCase): +class UsabilityTest: '''Unit tests for Usability cases of Decimal.''' def test_comparison_operators(self): @@ -2521,9 +2526,10 @@ def test_conversions_from_int(self): self.assertEqual(Decimal(-12).fma(45, Decimal(67)), Decimal(-12).fma(Decimal(45), Decimal(67))) -class CUsabilityTest(UsabilityTest): + at requires_cdecimal +class CUsabilityTest(UsabilityTest, unittest.TestCase): decimal = C -class PyUsabilityTest(UsabilityTest): +class PyUsabilityTest(UsabilityTest, unittest.TestCase): decimal = P def setUp(self): @@ -2535,7 +2541,7 @@ def tearDown(self): sys.set_int_max_str_digits(self._previous_int_limit) super().tearDown() -class PythonAPItests(unittest.TestCase): +class PythonAPItests: def test_abc(self): Decimal = self.decimal.Decimal @@ -2884,12 +2890,13 @@ def test_exception_hierarchy(self): self.assertTrue(issubclass(decimal.DivisionUndefined, ZeroDivisionError)) self.assertTrue(issubclass(decimal.InvalidContext, InvalidOperation)) -class CPythonAPItests(PythonAPItests): + at requires_cdecimal +class CPythonAPItests(PythonAPItests, unittest.TestCase): decimal = C -class PyPythonAPItests(PythonAPItests): +class PyPythonAPItests(PythonAPItests, unittest.TestCase): decimal = P -class ContextAPItests(unittest.TestCase): +class ContextAPItests: def test_none_args(self): Context = self.decimal.Context @@ -3635,12 +3642,13 @@ def test_to_integral_value(self): self.assertRaises(TypeError, c.to_integral_value, '10') self.assertRaises(TypeError, c.to_integral_value, 10, 'x') -class CContextAPItests(ContextAPItests): + at requires_cdecimal +class CContextAPItests(ContextAPItests, unittest.TestCase): decimal = C -class PyContextAPItests(ContextAPItests): +class PyContextAPItests(ContextAPItests, unittest.TestCase): decimal = P -class ContextWithStatement(unittest.TestCase): +class ContextWithStatement: # Can't do these as docstrings until Python 2.6 # as doctest can't handle __future__ statements @@ -3704,9 +3712,13 @@ def test_localcontext_kwargs(self): def test_local_context_kwargs_does_not_overwrite_existing_argument(self): ctx = self.decimal.getcontext() - ctx.prec = 28 + orig_prec = ctx.prec with self.decimal.localcontext(prec=10) as ctx2: - self.assertEqual(ctx.prec, 28) + self.assertEqual(ctx2.prec, 10) + self.assertEqual(ctx.prec, orig_prec) + with self.decimal.localcontext(prec=20) as ctx2: + self.assertEqual(ctx2.prec, 20) + self.assertEqual(ctx.prec, orig_prec) def test_nested_with_statements(self): # Use a copy of the supplied context in the block @@ -3800,12 +3812,13 @@ def test_with_statements_gc3(self): self.assertEqual(c4.prec, 4) del c4 -class CContextWithStatement(ContextWithStatement): + at requires_cdecimal +class CContextWithStatement(ContextWithStatement, unittest.TestCase): decimal = C -class PyContextWithStatement(ContextWithStatement): +class PyContextWithStatement(ContextWithStatement, unittest.TestCase): decimal = P -class ContextFlags(unittest.TestCase): +class ContextFlags: def test_flags_irrelevant(self): # check that the result (numeric result + flags raised) of an @@ -4072,12 +4085,13 @@ def test_float_operation_default(self): self.assertTrue(context.traps[FloatOperation]) self.assertTrue(context.traps[Inexact]) -class CContextFlags(ContextFlags): + at requires_cdecimal +class CContextFlags(ContextFlags, unittest.TestCase): decimal = C -class PyContextFlags(ContextFlags): +class PyContextFlags(ContextFlags, unittest.TestCase): decimal = P -class SpecialContexts(unittest.TestCase): +class SpecialContexts: """Test the context templates.""" def test_context_templates(self): @@ -4157,12 +4171,13 @@ def test_default_context(self): if ex: raise ex -class CSpecialContexts(SpecialContexts): + at requires_cdecimal +class CSpecialContexts(SpecialContexts, unittest.TestCase): decimal = C -class PySpecialContexts(SpecialContexts): +class PySpecialContexts(SpecialContexts, unittest.TestCase): decimal = P -class ContextInputValidation(unittest.TestCase): +class ContextInputValidation: def test_invalid_context(self): Context = self.decimal.Context @@ -4224,12 +4239,13 @@ def test_invalid_context(self): self.assertRaises(TypeError, Context, flags=(0,1)) self.assertRaises(TypeError, Context, traps=(1,0)) -class CContextInputValidation(ContextInputValidation): + at requires_cdecimal +class CContextInputValidation(ContextInputValidation, unittest.TestCase): decimal = C -class PyContextInputValidation(ContextInputValidation): +class PyContextInputValidation(ContextInputValidation, unittest.TestCase): decimal = P -class ContextSubclassing(unittest.TestCase): +class ContextSubclassing: def test_context_subclassing(self): decimal = self.decimal @@ -4338,12 +4354,14 @@ def __init__(self, prec=None, rounding=None, Emin=None, Emax=None, for signal in OrderedSignals[decimal]: self.assertFalse(c.traps[signal]) -class CContextSubclassing(ContextSubclassing): + at requires_cdecimal +class CContextSubclassing(ContextSubclassing, unittest.TestCase): decimal = C -class PyContextSubclassing(ContextSubclassing): +class PyContextSubclassing(ContextSubclassing, unittest.TestCase): decimal = P @skip_if_extra_functionality + at requires_cdecimal class CheckAttributes(unittest.TestCase): def test_module_attributes(self): @@ -4373,7 +4391,7 @@ def test_decimal_attributes(self): y = [s for s in dir(C.Decimal(9)) if '__' in s or not s.startswith('_')] self.assertEqual(set(x) - set(y), set()) -class Coverage(unittest.TestCase): +class Coverage: def test_adjusted(self): Decimal = self.decimal.Decimal @@ -4630,9 +4648,10 @@ def test_copy(self): y = c.copy_sign(x, 1) self.assertEqual(y, -x) -class CCoverage(Coverage): + at requires_cdecimal +class CCoverage(Coverage, unittest.TestCase): decimal = C -class PyCoverage(Coverage): +class PyCoverage(Coverage, unittest.TestCase): decimal = P def setUp(self): @@ -4885,6 +4904,7 @@ def test_constants(self): self.assertEqual(C.DecTraps, C.DecErrors|C.DecOverflow|C.DecUnderflow) + at requires_cdecimal class CWhitebox(unittest.TestCase): """Whitebox testing for _decimal""" @@ -5663,7 +5683,7 @@ def test_maxcontext_exact_arith(self): @requires_docstrings - at unittest.skipUnless(C, "test requires C version") + at requires_cdecimal class SignatureTest(unittest.TestCase): """Function signatures""" @@ -5799,52 +5819,10 @@ def doit(ty): doit('Context') -all_tests = [ - CExplicitConstructionTest, PyExplicitConstructionTest, - CImplicitConstructionTest, PyImplicitConstructionTest, - CFormatTest, PyFormatTest, - CArithmeticOperatorsTest, PyArithmeticOperatorsTest, - CThreadingTest, PyThreadingTest, - CUsabilityTest, PyUsabilityTest, - CPythonAPItests, PyPythonAPItests, - CContextAPItests, PyContextAPItests, - CContextWithStatement, PyContextWithStatement, - CContextFlags, PyContextFlags, - CSpecialContexts, PySpecialContexts, - CContextInputValidation, PyContextInputValidation, - CContextSubclassing, PyContextSubclassing, - CCoverage, PyCoverage, - CFunctionality, PyFunctionality, - CWhitebox, PyWhitebox, - CIBMTestCases, PyIBMTestCases, -] - -# Delete C tests if _decimal.so is not present. -if not C: - all_tests = all_tests[1::2] -else: - all_tests.insert(0, CheckAttributes) - all_tests.insert(1, SignatureTest) - - -def test_main(arith=None, verbose=None, todo_tests=None, debug=None): - """ Execute the tests. - - Runs all arithmetic tests if arith is True or if the "decimal" resource - is enabled in regrtest.py - """ - - init(C) - init(P) - global TEST_ALL, DEBUG - TEST_ALL = arith if arith is not None else is_resource_enabled('decimal') - DEBUG = debug - - if todo_tests is None: - test_classes = all_tests - else: - test_classes = [CIBMTestCases, PyIBMTestCases] - +def load_tests(loader, tests, pattern): + if TODO_TESTS is not None: + # Run only Arithmetic tests + tests = loader.suiteClass() # Dynamically build custom test definition for each file in the test # directory and add the definitions to the DecimalTest class. This # procedure insures that new files do not get skipped. @@ -5852,34 +5830,69 @@ def test_main(arith=None, verbose=None, todo_tests=None, debug=None): if '.decTest' not in filename or filename.startswith("."): continue head, tail = filename.split('.') - if todo_tests is not None and head not in todo_tests: + if TODO_TESTS is not None and head not in TODO_TESTS: continue tester = lambda self, f=filename: self.eval_file(directory + f) - setattr(CIBMTestCases, 'test_' + head, tester) - setattr(PyIBMTestCases, 'test_' + head, tester) + setattr(IBMTestCases, 'test_' + head, tester) del filename, head, tail, tester + for prefix, mod in ('C', C), ('Py', P): + if not mod: + continue + test_class = type(prefix + 'IBMTestCases', + (IBMTestCases, unittest.TestCase), + {'decimal': mod}) + tests.addTest(loader.loadTestsFromTestCase(test_class)) + + if TODO_TESTS is None: + from doctest import DocTestSuite, IGNORE_EXCEPTION_DETAIL + for mod in C, P: + if not mod: + continue + def setUp(slf, mod=mod): + sys.modules['decimal'] = mod + def tearDown(slf): + sys.modules['decimal'] = orig_sys_decimal + optionflags = IGNORE_EXCEPTION_DETAIL if mod is C else 0 + sys.modules['decimal'] = mod + tests.addTest(DocTestSuite(mod, setUp=setUp, tearDown=tearDown, + optionflags=optionflags)) + sys.modules['decimal'] = orig_sys_decimal + return tests + +def setUpModule(): + init(C) + init(P) + global TEST_ALL + TEST_ALL = ARITH if ARITH is not None else is_resource_enabled('decimal') + +def tearDownModule(): + if C: C.setcontext(ORIGINAL_CONTEXT[C]) + P.setcontext(ORIGINAL_CONTEXT[P]) + if not C: + warnings.warn('C tests skipped: no module named _decimal.', + UserWarning) + if not orig_sys_decimal is sys.modules['decimal']: + raise TestFailed("Internal error: unbalanced number of changes to " + "sys.modules['decimal'].") + + +ARITH = None +TEST_ALL = True +TODO_TESTS = None +DEBUG = False + +def test(arith=None, verbose=None, todo_tests=None, debug=None): + """ Execute the tests. + Runs all arithmetic tests if arith is True or if the "decimal" resource + is enabled in regrtest.py + """ - try: - run_unittest(*test_classes) - if todo_tests is None: - from doctest import IGNORE_EXCEPTION_DETAIL - savedecimal = sys.modules['decimal'] - if C: - sys.modules['decimal'] = C - run_doctest(C, verbose, optionflags=IGNORE_EXCEPTION_DETAIL) - sys.modules['decimal'] = P - run_doctest(P, verbose) - sys.modules['decimal'] = savedecimal - finally: - if C: C.setcontext(ORIGINAL_CONTEXT[C]) - P.setcontext(ORIGINAL_CONTEXT[P]) - if not C: - warnings.warn('C tests skipped: no module named _decimal.', - UserWarning) - if not orig_sys_decimal is sys.modules['decimal']: - raise TestFailed("Internal error: unbalanced number of changes to " - "sys.modules['decimal'].") + global ARITH, TODO_TESTS, DEBUG + ARITH = arith + TODO_TESTS = todo_tests + DEBUG = debug + unittest.main(__name__, verbosity=2 if verbose else 1, exit=False, argv=[__name__]) if __name__ == '__main__': @@ -5890,8 +5903,8 @@ def test_main(arith=None, verbose=None, todo_tests=None, debug=None): (opt, args) = p.parse_args() if opt.skip: - test_main(arith=False, verbose=True) + test(arith=False, verbose=True) elif args: - test_main(arith=True, verbose=True, todo_tests=args, debug=opt.debug) + test(arith=True, verbose=True, todo_tests=args, debug=opt.debug) else: - test_main(arith=True, verbose=True) + test(arith=True, verbose=True) From webhook-mailer at python.org Wed Jul 5 06:54:30 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 10:54:30 -0000 Subject: [Python-checkins] [3.12] Display the sanitizer config in the regrtest header. (GH-105301) (#105342) Message-ID: https://github.com/python/cpython/commit/fc2393e4177d32dab0c73f69472dc1b726dbeeaf commit: fc2393e4177d32dab0c73f69472dc1b726dbeeaf branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T12:54:26+02:00 summary: [3.12] Display the sanitizer config in the regrtest header. (GH-105301) (#105342) Display the sanitizer config in the regrtest header. (GH-105301) Display the sanitizers present in libregrtest. Having this in the CI output for tests with the relevant environment variable displayed will help make it easier to do what we need to create an equivalent local test run. (cherry picked from commit 852348ab65783601e0844b6647ea033668b45c11) Co-authored-by: Gregory P. Smith files: M Lib/test/libregrtest/main.py M Lib/test/support/__init__.py diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index 3c3509d030337..9001ca33b6166 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -526,6 +526,26 @@ def display_header(self): print("== CPU count:", cpu_count) print("== encodings: locale=%s, FS=%s" % (locale.getencoding(), sys.getfilesystemencoding())) + asan = support.check_sanitizer(address=True) + msan = support.check_sanitizer(memory=True) + ubsan = support.check_sanitizer(ub=True) + # This makes it easier to remember what to set in your local + # environment when trying to reproduce a sanitizer failure. + if asan or msan or ubsan: + names = [n for n in (asan and "address", + msan and "memory", + ubsan and "undefined behavior") + if n] + print(f"== sanitizers: {', '.join(names)}") + a_opts = os.environ.get("ASAN_OPTIONS") + if asan and a_opts is not None: + print(f"== ASAN_OPTIONS={a_opts}") + m_opts = os.environ.get("ASAN_OPTIONS") + if msan and m_opts is not None: + print(f"== MSAN_OPTIONS={m_opts}") + ub_opts = os.environ.get("UBSAN_OPTIONS") + if ubsan and ub_opts is not None: + print(f"== UBSAN_OPTIONS={ub_opts}") def no_tests_run(self): return not any((self.good, self.bad, self.skipped, self.interrupted, diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 3f1cc3a083a1c..3b332f49951f0 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -414,7 +414,7 @@ def check_sanitizer(*, address=False, memory=False, ub=False): ) address_sanitizer = ( '-fsanitize=address' in _cflags or - '--with-memory-sanitizer' in _config_args + '--with-address-sanitizer' in _config_args ) ub_sanitizer = ( '-fsanitize=undefined' in _cflags or From webhook-mailer at python.org Wed Jul 5 07:00:59 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:00:59 -0000 Subject: [Python-checkins] [3.11] gh-89392: Make test_decimal discoverable (GH-106209) (#106229) Message-ID: https://github.com/python/cpython/commit/33fb89552b2a0e3dcc1e6651bec41657ecd080c3 commit: 33fb89552b2a0e3dcc1e6651bec41657ecd080c3 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:00:56+02:00 summary: [3.11] gh-89392: Make test_decimal discoverable (GH-106209) (#106229) (cherry picked from commit 0e24499129f3917b199a6d46fa33eeedd2c447fc) Co-authored-by: Serhiy Storchaka files: M Lib/test/test_decimal.py diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py index 67ccaab40c5ed..749496e3e9455 100644 --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -20,7 +20,7 @@ This test module can be called from command line with one parameter (Arithmetic or Behaviour) to test each part, or without parameter to test both parts. If -you're working through IDLE, you can import this test module and call test_main() +you're working through IDLE, you can import this test module and call test() with the corresponding argument. """ @@ -32,7 +32,7 @@ import unittest import numbers import locale -from test.support import (run_unittest, run_doctest, is_resource_enabled, +from test.support import (is_resource_enabled, requires_IEEE_754, requires_docstrings, requires_legacy_unicode_capi, check_sanitizer) from test.support import (TestFailed, @@ -62,6 +62,7 @@ fractions = {C:cfractions, P:pfractions} sys.modules['decimal'] = orig_sys_decimal +requires_cdecimal = unittest.skipUnless(C, "test requires C version") # Useful Test Constant Signals = { @@ -99,7 +100,7 @@ def assert_signals(cls, context, attr, expected): ] # Tests are built around these assumed context defaults. -# test_main() restores the original context. +# test() restores the original context. ORIGINAL_CONTEXT = { C: C.getcontext().copy() if C else None, P: P.getcontext().copy() @@ -133,7 +134,7 @@ def init(m): EXTRA_FUNCTIONALITY, "test requires regular build") -class IBMTestCases(unittest.TestCase): +class IBMTestCases: """Class which tests the Decimal class against the IBM test cases.""" def setUp(self): @@ -488,14 +489,10 @@ def change_max_exponent(self, exp): def change_clamp(self, clamp): self.context.clamp = clamp -class CIBMTestCases(IBMTestCases): - decimal = C -class PyIBMTestCases(IBMTestCases): - decimal = P # The following classes test the behaviour of Decimal according to PEP 327 -class ExplicitConstructionTest(unittest.TestCase): +class ExplicitConstructionTest: '''Unit tests for Explicit Construction cases of Decimal.''' def test_explicit_empty(self): @@ -838,12 +835,13 @@ def test_unicode_digits(self): for input, expected in test_values.items(): self.assertEqual(str(Decimal(input)), expected) -class CExplicitConstructionTest(ExplicitConstructionTest): + at requires_cdecimal +class CExplicitConstructionTest(ExplicitConstructionTest, unittest.TestCase): decimal = C -class PyExplicitConstructionTest(ExplicitConstructionTest): +class PyExplicitConstructionTest(ExplicitConstructionTest, unittest.TestCase): decimal = P -class ImplicitConstructionTest(unittest.TestCase): +class ImplicitConstructionTest: '''Unit tests for Implicit Construction cases of Decimal.''' def test_implicit_from_None(self): @@ -920,12 +918,13 @@ def __ne__(self, other): self.assertEqual(eval('Decimal(10)' + sym + 'E()'), '10' + rop + 'str') -class CImplicitConstructionTest(ImplicitConstructionTest): + at requires_cdecimal +class CImplicitConstructionTest(ImplicitConstructionTest, unittest.TestCase): decimal = C -class PyImplicitConstructionTest(ImplicitConstructionTest): +class PyImplicitConstructionTest(ImplicitConstructionTest, unittest.TestCase): decimal = P -class FormatTest(unittest.TestCase): +class FormatTest: '''Unit tests for the format function.''' def test_formatting(self): Decimal = self.decimal.Decimal @@ -1262,12 +1261,13 @@ def __init__(self, a): a = A.from_float(42) self.assertEqual(self.decimal.Decimal, a.a_type) -class CFormatTest(FormatTest): + at requires_cdecimal +class CFormatTest(FormatTest, unittest.TestCase): decimal = C -class PyFormatTest(FormatTest): +class PyFormatTest(FormatTest, unittest.TestCase): decimal = P -class ArithmeticOperatorsTest(unittest.TestCase): +class ArithmeticOperatorsTest: '''Unit tests for all arithmetic operators, binary and unary.''' def test_addition(self): @@ -1523,14 +1523,17 @@ def test_nan_comparisons(self): equality_ops = operator.eq, operator.ne # results when InvalidOperation is not trapped - for x, y in qnan_pairs + snan_pairs: - for op in order_ops + equality_ops: - got = op(x, y) - expected = True if op is operator.ne else False - self.assertIs(expected, got, - "expected {0!r} for operator.{1}({2!r}, {3!r}); " - "got {4!r}".format( - expected, op.__name__, x, y, got)) + with localcontext() as ctx: + ctx.traps[InvalidOperation] = 0 + + for x, y in qnan_pairs + snan_pairs: + for op in order_ops + equality_ops: + got = op(x, y) + expected = True if op is operator.ne else False + self.assertIs(expected, got, + "expected {0!r} for operator.{1}({2!r}, {3!r}); " + "got {4!r}".format( + expected, op.__name__, x, y, got)) # repeat the above, but this time trap the InvalidOperation with localcontext() as ctx: @@ -1562,9 +1565,10 @@ def test_copy_sign(self): self.assertEqual(Decimal(1).copy_sign(-2), d) self.assertRaises(TypeError, Decimal(1).copy_sign, '-2') -class CArithmeticOperatorsTest(ArithmeticOperatorsTest): + at requires_cdecimal +class CArithmeticOperatorsTest(ArithmeticOperatorsTest, unittest.TestCase): decimal = C -class PyArithmeticOperatorsTest(ArithmeticOperatorsTest): +class PyArithmeticOperatorsTest(ArithmeticOperatorsTest, unittest.TestCase): decimal = P # The following are two functions used to test threading in the next class @@ -1654,7 +1658,7 @@ def thfunc2(cls): @threading_helper.requires_working_threading() -class ThreadingTest(unittest.TestCase): +class ThreadingTest: '''Unit tests for thread local contexts in Decimal.''' # Take care executing this test from IDLE, there's an issue in threading @@ -1699,13 +1703,14 @@ def test_threading(self): DefaultContext.Emin = save_emin -class CThreadingTest(ThreadingTest): + at requires_cdecimal +class CThreadingTest(ThreadingTest, unittest.TestCase): decimal = C -class PyThreadingTest(ThreadingTest): +class PyThreadingTest(ThreadingTest, unittest.TestCase): decimal = P -class UsabilityTest(unittest.TestCase): +class UsabilityTest: '''Unit tests for Usability cases of Decimal.''' def test_comparison_operators(self): @@ -2521,9 +2526,10 @@ def test_conversions_from_int(self): self.assertEqual(Decimal(-12).fma(45, Decimal(67)), Decimal(-12).fma(Decimal(45), Decimal(67))) -class CUsabilityTest(UsabilityTest): + at requires_cdecimal +class CUsabilityTest(UsabilityTest, unittest.TestCase): decimal = C -class PyUsabilityTest(UsabilityTest): +class PyUsabilityTest(UsabilityTest, unittest.TestCase): decimal = P def setUp(self): @@ -2535,7 +2541,7 @@ def tearDown(self): sys.set_int_max_str_digits(self._previous_int_limit) super().tearDown() -class PythonAPItests(unittest.TestCase): +class PythonAPItests: def test_abc(self): Decimal = self.decimal.Decimal @@ -2884,12 +2890,13 @@ def test_exception_hierarchy(self): self.assertTrue(issubclass(decimal.DivisionUndefined, ZeroDivisionError)) self.assertTrue(issubclass(decimal.InvalidContext, InvalidOperation)) -class CPythonAPItests(PythonAPItests): + at requires_cdecimal +class CPythonAPItests(PythonAPItests, unittest.TestCase): decimal = C -class PyPythonAPItests(PythonAPItests): +class PyPythonAPItests(PythonAPItests, unittest.TestCase): decimal = P -class ContextAPItests(unittest.TestCase): +class ContextAPItests: def test_none_args(self): Context = self.decimal.Context @@ -3635,12 +3642,13 @@ def test_to_integral_value(self): self.assertRaises(TypeError, c.to_integral_value, '10') self.assertRaises(TypeError, c.to_integral_value, 10, 'x') -class CContextAPItests(ContextAPItests): + at requires_cdecimal +class CContextAPItests(ContextAPItests, unittest.TestCase): decimal = C -class PyContextAPItests(ContextAPItests): +class PyContextAPItests(ContextAPItests, unittest.TestCase): decimal = P -class ContextWithStatement(unittest.TestCase): +class ContextWithStatement: # Can't do these as docstrings until Python 2.6 # as doctest can't handle __future__ statements @@ -3704,9 +3712,13 @@ def test_localcontext_kwargs(self): def test_local_context_kwargs_does_not_overwrite_existing_argument(self): ctx = self.decimal.getcontext() - ctx.prec = 28 + orig_prec = ctx.prec with self.decimal.localcontext(prec=10) as ctx2: - self.assertEqual(ctx.prec, 28) + self.assertEqual(ctx2.prec, 10) + self.assertEqual(ctx.prec, orig_prec) + with self.decimal.localcontext(prec=20) as ctx2: + self.assertEqual(ctx2.prec, 20) + self.assertEqual(ctx.prec, orig_prec) def test_nested_with_statements(self): # Use a copy of the supplied context in the block @@ -3800,12 +3812,13 @@ def test_with_statements_gc3(self): self.assertEqual(c4.prec, 4) del c4 -class CContextWithStatement(ContextWithStatement): + at requires_cdecimal +class CContextWithStatement(ContextWithStatement, unittest.TestCase): decimal = C -class PyContextWithStatement(ContextWithStatement): +class PyContextWithStatement(ContextWithStatement, unittest.TestCase): decimal = P -class ContextFlags(unittest.TestCase): +class ContextFlags: def test_flags_irrelevant(self): # check that the result (numeric result + flags raised) of an @@ -4072,12 +4085,13 @@ def test_float_operation_default(self): self.assertTrue(context.traps[FloatOperation]) self.assertTrue(context.traps[Inexact]) -class CContextFlags(ContextFlags): + at requires_cdecimal +class CContextFlags(ContextFlags, unittest.TestCase): decimal = C -class PyContextFlags(ContextFlags): +class PyContextFlags(ContextFlags, unittest.TestCase): decimal = P -class SpecialContexts(unittest.TestCase): +class SpecialContexts: """Test the context templates.""" def test_context_templates(self): @@ -4157,12 +4171,13 @@ def test_default_context(self): if ex: raise ex -class CSpecialContexts(SpecialContexts): + at requires_cdecimal +class CSpecialContexts(SpecialContexts, unittest.TestCase): decimal = C -class PySpecialContexts(SpecialContexts): +class PySpecialContexts(SpecialContexts, unittest.TestCase): decimal = P -class ContextInputValidation(unittest.TestCase): +class ContextInputValidation: def test_invalid_context(self): Context = self.decimal.Context @@ -4224,12 +4239,13 @@ def test_invalid_context(self): self.assertRaises(TypeError, Context, flags=(0,1)) self.assertRaises(TypeError, Context, traps=(1,0)) -class CContextInputValidation(ContextInputValidation): + at requires_cdecimal +class CContextInputValidation(ContextInputValidation, unittest.TestCase): decimal = C -class PyContextInputValidation(ContextInputValidation): +class PyContextInputValidation(ContextInputValidation, unittest.TestCase): decimal = P -class ContextSubclassing(unittest.TestCase): +class ContextSubclassing: def test_context_subclassing(self): decimal = self.decimal @@ -4338,12 +4354,14 @@ def __init__(self, prec=None, rounding=None, Emin=None, Emax=None, for signal in OrderedSignals[decimal]: self.assertFalse(c.traps[signal]) -class CContextSubclassing(ContextSubclassing): + at requires_cdecimal +class CContextSubclassing(ContextSubclassing, unittest.TestCase): decimal = C -class PyContextSubclassing(ContextSubclassing): +class PyContextSubclassing(ContextSubclassing, unittest.TestCase): decimal = P @skip_if_extra_functionality + at requires_cdecimal class CheckAttributes(unittest.TestCase): def test_module_attributes(self): @@ -4373,7 +4391,7 @@ def test_decimal_attributes(self): y = [s for s in dir(C.Decimal(9)) if '__' in s or not s.startswith('_')] self.assertEqual(set(x) - set(y), set()) -class Coverage(unittest.TestCase): +class Coverage: def test_adjusted(self): Decimal = self.decimal.Decimal @@ -4630,9 +4648,10 @@ def test_copy(self): y = c.copy_sign(x, 1) self.assertEqual(y, -x) -class CCoverage(Coverage): + at requires_cdecimal +class CCoverage(Coverage, unittest.TestCase): decimal = C -class PyCoverage(Coverage): +class PyCoverage(Coverage, unittest.TestCase): decimal = P def setUp(self): @@ -4885,6 +4904,7 @@ def test_constants(self): self.assertEqual(C.DecTraps, C.DecErrors|C.DecOverflow|C.DecUnderflow) + at requires_cdecimal class CWhitebox(unittest.TestCase): """Whitebox testing for _decimal""" @@ -5663,7 +5683,7 @@ def test_maxcontext_exact_arith(self): @requires_docstrings - at unittest.skipUnless(C, "test requires C version") + at requires_cdecimal class SignatureTest(unittest.TestCase): """Function signatures""" @@ -5799,52 +5819,10 @@ def doit(ty): doit('Context') -all_tests = [ - CExplicitConstructionTest, PyExplicitConstructionTest, - CImplicitConstructionTest, PyImplicitConstructionTest, - CFormatTest, PyFormatTest, - CArithmeticOperatorsTest, PyArithmeticOperatorsTest, - CThreadingTest, PyThreadingTest, - CUsabilityTest, PyUsabilityTest, - CPythonAPItests, PyPythonAPItests, - CContextAPItests, PyContextAPItests, - CContextWithStatement, PyContextWithStatement, - CContextFlags, PyContextFlags, - CSpecialContexts, PySpecialContexts, - CContextInputValidation, PyContextInputValidation, - CContextSubclassing, PyContextSubclassing, - CCoverage, PyCoverage, - CFunctionality, PyFunctionality, - CWhitebox, PyWhitebox, - CIBMTestCases, PyIBMTestCases, -] - -# Delete C tests if _decimal.so is not present. -if not C: - all_tests = all_tests[1::2] -else: - all_tests.insert(0, CheckAttributes) - all_tests.insert(1, SignatureTest) - - -def test_main(arith=None, verbose=None, todo_tests=None, debug=None): - """ Execute the tests. - - Runs all arithmetic tests if arith is True or if the "decimal" resource - is enabled in regrtest.py - """ - - init(C) - init(P) - global TEST_ALL, DEBUG - TEST_ALL = arith if arith is not None else is_resource_enabled('decimal') - DEBUG = debug - - if todo_tests is None: - test_classes = all_tests - else: - test_classes = [CIBMTestCases, PyIBMTestCases] - +def load_tests(loader, tests, pattern): + if TODO_TESTS is not None: + # Run only Arithmetic tests + tests = loader.suiteClass() # Dynamically build custom test definition for each file in the test # directory and add the definitions to the DecimalTest class. This # procedure insures that new files do not get skipped. @@ -5852,34 +5830,69 @@ def test_main(arith=None, verbose=None, todo_tests=None, debug=None): if '.decTest' not in filename or filename.startswith("."): continue head, tail = filename.split('.') - if todo_tests is not None and head not in todo_tests: + if TODO_TESTS is not None and head not in TODO_TESTS: continue tester = lambda self, f=filename: self.eval_file(directory + f) - setattr(CIBMTestCases, 'test_' + head, tester) - setattr(PyIBMTestCases, 'test_' + head, tester) + setattr(IBMTestCases, 'test_' + head, tester) del filename, head, tail, tester + for prefix, mod in ('C', C), ('Py', P): + if not mod: + continue + test_class = type(prefix + 'IBMTestCases', + (IBMTestCases, unittest.TestCase), + {'decimal': mod}) + tests.addTest(loader.loadTestsFromTestCase(test_class)) + + if TODO_TESTS is None: + from doctest import DocTestSuite, IGNORE_EXCEPTION_DETAIL + for mod in C, P: + if not mod: + continue + def setUp(slf, mod=mod): + sys.modules['decimal'] = mod + def tearDown(slf): + sys.modules['decimal'] = orig_sys_decimal + optionflags = IGNORE_EXCEPTION_DETAIL if mod is C else 0 + sys.modules['decimal'] = mod + tests.addTest(DocTestSuite(mod, setUp=setUp, tearDown=tearDown, + optionflags=optionflags)) + sys.modules['decimal'] = orig_sys_decimal + return tests + +def setUpModule(): + init(C) + init(P) + global TEST_ALL + TEST_ALL = ARITH if ARITH is not None else is_resource_enabled('decimal') + +def tearDownModule(): + if C: C.setcontext(ORIGINAL_CONTEXT[C]) + P.setcontext(ORIGINAL_CONTEXT[P]) + if not C: + warnings.warn('C tests skipped: no module named _decimal.', + UserWarning) + if not orig_sys_decimal is sys.modules['decimal']: + raise TestFailed("Internal error: unbalanced number of changes to " + "sys.modules['decimal'].") + + +ARITH = None +TEST_ALL = True +TODO_TESTS = None +DEBUG = False + +def test(arith=None, verbose=None, todo_tests=None, debug=None): + """ Execute the tests. + Runs all arithmetic tests if arith is True or if the "decimal" resource + is enabled in regrtest.py + """ - try: - run_unittest(*test_classes) - if todo_tests is None: - from doctest import IGNORE_EXCEPTION_DETAIL - savedecimal = sys.modules['decimal'] - if C: - sys.modules['decimal'] = C - run_doctest(C, verbose, optionflags=IGNORE_EXCEPTION_DETAIL) - sys.modules['decimal'] = P - run_doctest(P, verbose) - sys.modules['decimal'] = savedecimal - finally: - if C: C.setcontext(ORIGINAL_CONTEXT[C]) - P.setcontext(ORIGINAL_CONTEXT[P]) - if not C: - warnings.warn('C tests skipped: no module named _decimal.', - UserWarning) - if not orig_sys_decimal is sys.modules['decimal']: - raise TestFailed("Internal error: unbalanced number of changes to " - "sys.modules['decimal'].") + global ARITH, TODO_TESTS, DEBUG + ARITH = arith + TODO_TESTS = todo_tests + DEBUG = debug + unittest.main(__name__, verbosity=2 if verbose else 1, exit=False, argv=[__name__]) if __name__ == '__main__': @@ -5890,8 +5903,8 @@ def test_main(arith=None, verbose=None, todo_tests=None, debug=None): (opt, args) = p.parse_args() if opt.skip: - test_main(arith=False, verbose=True) + test(arith=False, verbose=True) elif args: - test_main(arith=True, verbose=True, todo_tests=args, debug=opt.debug) + test(arith=True, verbose=True, todo_tests=args, debug=opt.debug) else: - test_main(arith=True, verbose=True) + test(arith=True, verbose=True) From webhook-mailer at python.org Wed Jul 5 07:02:26 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:02:26 -0000 Subject: [Python-checkins] [3.12] GH-104554: Add RTSPS support to `urllib/parse.py` (GH-104605) (#105759) Message-ID: https://github.com/python/cpython/commit/53605f285a5e50cd804f3b38aa9e3a71546142b3 commit: 53605f285a5e50cd804f3b38aa9e3a71546142b3 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:02:22+02:00 summary: [3.12] GH-104554: Add RTSPS support to `urllib/parse.py` (GH-104605) (#105759) RTSPS is the permanent scheme defined in https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml alongside RTSP and RTSPU schemes. (cherry picked from commit f3266c05b6186ab6d1db0799c06b8f76aefe7cf1) Co-authored-by: zentarim <33746047+zentarim at users.noreply.github.com> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst M Doc/library/urllib.parse.rst M Lib/urllib/parse.py diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index 5a9a53f83dace..e1aa4ebb0964d 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -23,9 +23,9 @@ to an absolute URL given a "base URL." The module has been designed to match the internet RFC on Relative Uniform Resource Locators. It supports the following URL schemes: ``file``, ``ftp``, ``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``mailto``, ``mms``, -``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtspu``, ``sftp``, -``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``, ``telnet``, -``wais``, ``ws``, ``wss``. +``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtsps``, ``rtspu``, +``sftp``, ``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``, +``telnet``, ``wais``, ``ws``, ``wss``. The :mod:`urllib.parse` module defines functions that fall into two broad categories: URL parsing and URL quoting. These are covered in detail in diff --git a/Lib/urllib/parse.py b/Lib/urllib/parse.py index b73b34428764f..c129b0d7971d7 100644 --- a/Lib/urllib/parse.py +++ b/Lib/urllib/parse.py @@ -52,18 +52,18 @@ uses_relative = ['', 'ftp', 'http', 'gopher', 'nntp', 'imap', 'wais', 'file', 'https', 'shttp', 'mms', - 'prospero', 'rtsp', 'rtspu', 'sftp', + 'prospero', 'rtsp', 'rtsps', 'rtspu', 'sftp', 'svn', 'svn+ssh', 'ws', 'wss'] uses_netloc = ['', 'ftp', 'http', 'gopher', 'nntp', 'telnet', 'imap', 'wais', 'file', 'mms', 'https', 'shttp', - 'snews', 'prospero', 'rtsp', 'rtspu', 'rsync', + 'snews', 'prospero', 'rtsp', 'rtsps', 'rtspu', 'rsync', 'svn', 'svn+ssh', 'sftp', 'nfs', 'git', 'git+ssh', 'ws', 'wss', 'itms-services'] uses_params = ['', 'ftp', 'hdl', 'prospero', 'http', 'imap', - 'https', 'shttp', 'rtsp', 'rtspu', 'sip', 'sips', - 'mms', 'sftp', 'tel'] + 'https', 'shttp', 'rtsp', 'rtsps', 'rtspu', 'sip', + 'sips', 'mms', 'sftp', 'tel'] # These are not actually used anymore, but should stay for backwards # compatibility. (They are undocumented, but have a public-looking name.) @@ -72,7 +72,7 @@ 'telnet', 'wais', 'imap', 'snews', 'sip', 'sips'] uses_query = ['', 'http', 'wais', 'imap', 'https', 'shttp', 'mms', - 'gopher', 'rtsp', 'rtspu', 'sip', 'sips'] + 'gopher', 'rtsp', 'rtsps', 'rtspu', 'sip', 'sips'] uses_fragment = ['', 'ftp', 'hdl', 'http', 'gopher', 'news', 'nntp', 'wais', 'https', 'shttp', 'snews', diff --git a/Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst b/Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst new file mode 100644 index 0000000000000..9ef8c67459c40 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst @@ -0,0 +1 @@ +Add RTSPS scheme support in urllib.parse From webhook-mailer at python.org Wed Jul 5 07:02:40 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:02:40 -0000 Subject: [Python-checkins] [3.11] GH-104554: Add RTSPS support to `urllib/parse.py` (GH-104605) (#105760) Message-ID: https://github.com/python/cpython/commit/23b731a10d5885e4afee6a0a4e0395862f5eac69 commit: 23b731a10d5885e4afee6a0a4e0395862f5eac69 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:02:36+02:00 summary: [3.11] GH-104554: Add RTSPS support to `urllib/parse.py` (GH-104605) (#105760) RTSPS is the permanent scheme defined in https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml alongside RTSP and RTSPU schemes. --------- (cherry picked from commit f3266c05b6186ab6d1db0799c06b8f76aefe7cf1) Co-authored-by: zentarim <33746047+zentarim at users.noreply.github.com> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst M Doc/library/urllib.parse.rst M Lib/urllib/parse.py diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index a326e82e308d7..184d24f2c8e49 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -23,9 +23,9 @@ to an absolute URL given a "base URL." The module has been designed to match the internet RFC on Relative Uniform Resource Locators. It supports the following URL schemes: ``file``, ``ftp``, ``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``mailto``, ``mms``, -``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtspu``, ``sftp``, -``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``, ``telnet``, -``wais``, ``ws``, ``wss``. +``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtsps``, ``rtspu``, +``sftp``, ``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``, +``telnet``, ``wais``, ``ws``, ``wss``. The :mod:`urllib.parse` module defines functions that fall into two broad categories: URL parsing and URL quoting. These are covered in detail in diff --git a/Lib/urllib/parse.py b/Lib/urllib/parse.py index e5f0b784bf693..c5bbe713ccc58 100644 --- a/Lib/urllib/parse.py +++ b/Lib/urllib/parse.py @@ -52,18 +52,18 @@ uses_relative = ['', 'ftp', 'http', 'gopher', 'nntp', 'imap', 'wais', 'file', 'https', 'shttp', 'mms', - 'prospero', 'rtsp', 'rtspu', 'sftp', + 'prospero', 'rtsp', 'rtsps', 'rtspu', 'sftp', 'svn', 'svn+ssh', 'ws', 'wss'] uses_netloc = ['', 'ftp', 'http', 'gopher', 'nntp', 'telnet', 'imap', 'wais', 'file', 'mms', 'https', 'shttp', - 'snews', 'prospero', 'rtsp', 'rtspu', 'rsync', + 'snews', 'prospero', 'rtsp', 'rtsps', 'rtspu', 'rsync', 'svn', 'svn+ssh', 'sftp', 'nfs', 'git', 'git+ssh', 'ws', 'wss'] uses_params = ['', 'ftp', 'hdl', 'prospero', 'http', 'imap', - 'https', 'shttp', 'rtsp', 'rtspu', 'sip', 'sips', - 'mms', 'sftp', 'tel'] + 'https', 'shttp', 'rtsp', 'rtsps', 'rtspu', 'sip', + 'sips', 'mms', 'sftp', 'tel'] # These are not actually used anymore, but should stay for backwards # compatibility. (They are undocumented, but have a public-looking name.) @@ -72,7 +72,7 @@ 'telnet', 'wais', 'imap', 'snews', 'sip', 'sips'] uses_query = ['', 'http', 'wais', 'imap', 'https', 'shttp', 'mms', - 'gopher', 'rtsp', 'rtspu', 'sip', 'sips'] + 'gopher', 'rtsp', 'rtsps', 'rtspu', 'sip', 'sips'] uses_fragment = ['', 'ftp', 'hdl', 'http', 'gopher', 'news', 'nntp', 'wais', 'https', 'shttp', 'snews', diff --git a/Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst b/Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst new file mode 100644 index 0000000000000..9ef8c67459c40 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst @@ -0,0 +1 @@ +Add RTSPS scheme support in urllib.parse From webhook-mailer at python.org Wed Jul 5 07:05:53 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:05:53 -0000 Subject: [Python-checkins] [3.12] gh-105063: Disable test_peg_generator.TestCParser bco. ref leaks (GH-106024) (#106450) Message-ID: https://github.com/python/cpython/commit/b314194c3c05d388892be2a6bf359fec20d9504c commit: b314194c3c05d388892be2a6bf359fec20d9504c branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:05:50+02:00 summary: [3.12] gh-105063: Disable test_peg_generator.TestCParser bco. ref leaks (GH-106024) (#106450) Since gh-104798 (Use setuptools in peg-generator and reenable tests), the TestCParser test case has been producing ref leaks. (cherry picked from commit 41ad4dfc04c201728ce9fa12b1a96922dd15a368) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_peg_generator/test_c_parser.py diff --git a/Lib/test/test_peg_generator/test_c_parser.py b/Lib/test/test_peg_generator/test_c_parser.py index af39faeba9435..f9105a9f23bd6 100644 --- a/Lib/test/test_peg_generator/test_c_parser.py +++ b/Lib/test/test_peg_generator/test_c_parser.py @@ -74,8 +74,18 @@ def test_parse(self): @support.requires_subprocess() class TestCParser(unittest.TestCase): + _has_run = False + @classmethod def setUpClass(cls): + if cls._has_run: + # Since gh-104798 (Use setuptools in peg-generator and reenable + # tests), this test case has been producing ref leaks. Initial + # debugging points to bug(s) in setuptools and/or importlib. + # See gh-105063 for more info. + raise unittest.SkipTest("gh-105063: can not rerun because of ref. leaks") + cls._has_run = True + # When running under regtest, a separate tempdir is used # as the current directory and watched for left-overs. # Reusing that as the base for temporary directories From webhook-mailer at python.org Wed Jul 5 07:18:43 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:18:43 -0000 Subject: [Python-checkins] [3.10] [3.11] Add single value `agen.athrow(value)` signature to the 3.11 docs gh-105269 (GH-105468) (#105480) Message-ID: https://github.com/python/cpython/commit/18514431570e9928094df0b0a74a1808fbd054dd commit: 18514431570e9928094df0b0a74a1808fbd054dd branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:18:39+02:00 summary: [3.10] [3.11] Add single value `agen.athrow(value)` signature to the 3.11 docs gh-105269 (GH-105468) (#105480) (cherry picked from commit acf3916e84158308660ed07c474a564e045d6884) Co-authored-by: Federico Caselli files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 447fbc2a88f89..baa6695969be9 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -735,7 +735,8 @@ which are used to control the execution of a generator function. because there is no yield expression that could receive the value. -.. coroutinemethod:: agen.athrow(type[, value[, traceback]]) +.. coroutinemethod:: agen.athrow(value) + agen.athrow(type[, value[, traceback]]) Returns an awaitable that raises an exception of type ``type`` at the point where the asynchronous generator was paused, and returns the next value From webhook-mailer at python.org Wed Jul 5 07:18:53 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:18:53 -0000 Subject: [Python-checkins] [3.9] [3.11] Add single value `agen.athrow(value)` signature to the 3.11 docs gh-105269 (GH-105468) (#105477) Message-ID: https://github.com/python/cpython/commit/ce93371488b0fcd942567c2ee43f7d2e13e597fb commit: ce93371488b0fcd942567c2ee43f7d2e13e597fb branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:18:49+02:00 summary: [3.9] [3.11] Add single value `agen.athrow(value)` signature to the 3.11 docs gh-105269 (GH-105468) (#105477) (cherry picked from commit acf3916e84158308660ed07c474a564e045d6884) Co-authored-by: Federico Caselli files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 9f6f6f7d81527..d877b61b4b194 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -725,7 +725,8 @@ which are used to control the execution of a generator function. because there is no yield expression that could receive the value. -.. coroutinemethod:: agen.athrow(type[, value[, traceback]]) +.. coroutinemethod:: agen.athrow(value) + agen.athrow(type[, value[, traceback]]) Returns an awaitable that raises an exception of type ``type`` at the point where the asynchronous generator was paused, and returns the next value From webhook-mailer at python.org Wed Jul 5 07:19:05 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:19:05 -0000 Subject: [Python-checkins] [3.8] [3.11] Add single value `agen.athrow(value)` signature to the 3.11 docs gh-105269 (GH-105468) (#105478) Message-ID: https://github.com/python/cpython/commit/db42de40002cf769903ae055f24f57c1cb2a9540 commit: db42de40002cf769903ae055f24f57c1cb2a9540 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:19:01+02:00 summary: [3.8] [3.11] Add single value `agen.athrow(value)` signature to the 3.11 docs gh-105269 (GH-105468) (#105478) (cherry picked from commit acf3916e84158308660ed07c474a564e045d6884) Co-authored-by: Federico Caselli files: M Doc/reference/expressions.rst diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 0ac45995aef9c..d5b34379e2b59 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -710,7 +710,8 @@ which are used to control the execution of a generator function. because there is no yield expression that could receive the value. -.. coroutinemethod:: agen.athrow(type[, value[, traceback]]) +.. coroutinemethod:: agen.athrow(value) + agen.athrow(type[, value[, traceback]]) Returns an awaitable that raises an exception of type ``type`` at the point where the asynchronous generator was paused, and returns the next value From webhook-mailer at python.org Wed Jul 5 07:20:32 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:20:32 -0000 Subject: [Python-checkins] [3.10] CI: Bump macOS build to use OpenSSL v3.0 (GH-105538) (#105869) Message-ID: https://github.com/python/cpython/commit/073c66070778c8bc2a35085ae303be47d583c7af commit: 073c66070778c8bc2a35085ae303be47d583c7af branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:20:29+02:00 summary: [3.10] CI: Bump macOS build to use OpenSSL v3.0 (GH-105538) (#105869) (cherry picked from commit 34e93d3998bab8acd651c50724eb1977f4860a08) Co-authored-by: Erlend E. Aasland files: M .github/workflows/build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index f51a93e311209..8e91a17cd1744 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -181,7 +181,7 @@ jobs: steps: - uses: actions/checkout at v3 - name: Install Homebrew dependencies - run: brew install pkg-config openssl at 1.1 xz gdbm tcl-tk + run: brew install pkg-config openssl at 3.0 xz gdbm tcl-tk - name: Configure CPython run: | CPPFLAGS="-I$(brew --prefix gdbm)/include -I$(brew --prefix xz)/include" \ @@ -189,7 +189,7 @@ jobs: ./configure \ --with-pydebug \ --prefix=/opt/python-dev \ - --with-openssl="$(brew --prefix openssl at 1.1)" + --with-openssl="$(brew --prefix openssl at 3.0)" - name: Build CPython run: make -j4 - name: Display build info From webhook-mailer at python.org Wed Jul 5 07:20:48 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:20:48 -0000 Subject: [Python-checkins] [3.9] CI: Bump macOS build to use OpenSSL v3.0 (GH-105538) (#105871) Message-ID: https://github.com/python/cpython/commit/38b489bae817eef648fa14e775296c6245da2cb2 commit: 38b489bae817eef648fa14e775296c6245da2cb2 branch: 3.9 author: Erlend E. Aasland committer: ambv date: 2023-07-05T13:20:44+02:00 summary: [3.9] CI: Bump macOS build to use OpenSSL v3.0 (GH-105538) (#105871) (cherry picked from commit 34e93d3998bab8acd651c50724eb1977f4860a08) Co-authored-by: Erlend E. Aasland files: M .github/workflows/build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index ab2d0e4fd1e4b..37fd2ee863f72 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -153,15 +153,16 @@ jobs: PYTHONSTRICTEXTENSIONBUILD: 1 steps: - uses: actions/checkout at v3 + - name: Install Homebrew dependencies + run: brew install pkg-config openssl at 3.0 xz gdbm tcl-tk - name: Configure CPython run: | - brew install pkg-config openssl at 1.1 xz gdbm tcl-tk CC=clang \ CPPFLAGS="-I$(brew --prefix gdbm)/include -I$(brew --prefix xz)/include" \ LDFLAGS="-L$(brew --prefix gdbm)/lib -L$(brew --prefix xz)/lib" \ ./configure --prefix=/opt/python-dev \ --with-pydebug \ - --with-openssl="$(brew --prefix openssl at 1.1)" \ + --with-openssl="$(brew --prefix openssl at 3.0)" \ --with-tcltk-libs="$(pkg-config --libs tk)" \ --with-tcltk-includes="$(pkg-config --cflags tk)" - name: Build CPython From webhook-mailer at python.org Wed Jul 5 07:21:02 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:21:02 -0000 Subject: [Python-checkins] [3.8] CI: Bump macOS build to use OpenSSL v3.0 (GH-105538) (#105872) Message-ID: https://github.com/python/cpython/commit/1663f8ba8405fefc82f74f8d1ed86d6f26250658 commit: 1663f8ba8405fefc82f74f8d1ed86d6f26250658 branch: 3.8 author: Erlend E. Aasland committer: ambv date: 2023-07-05T13:20:58+02:00 summary: [3.8] CI: Bump macOS build to use OpenSSL v3.0 (GH-105538) (#105872) (cherry picked from commit 34e93d3998bab8acd651c50724eb1977f4860a08) Co-authored-by: Erlend E. Aasland files: M .github/workflows/build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 8102541bc3354..b7bc77181515b 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -132,16 +132,17 @@ jobs: HOMEBREW_NO_INSTALL_CLEANUP: 1 steps: - uses: actions/checkout at v2 + - name: Install Homebrew dependencies + run: brew install pkg-config openssl at 3.0 xz gdbm tcl-tk - name: Configure CPython run: | - brew install pkg-config openssl at 1.1 xz gdbm tcl-tk SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk \ CC=clang \ CPPFLAGS="-I$(brew --prefix gdbm)/include -I$(brew --prefix xz)/include" \ LDFLAGS="-L$(brew --prefix gdbm)/lib -L$(brew --prefix xz)/lib" \ ./configure --prefix=/opt/python-dev \ --with-pydebug \ - --with-openssl="$(brew --prefix openssl at 1.1)" \ + --with-openssl="$(brew --prefix openssl at 3.0)" \ --with-tcltk-libs="$(pkg-config --libs tk)" \ --with-tcltk-includes="$(pkg-config --cflags tk)" - name: Build CPython From webhook-mailer at python.org Wed Jul 5 07:21:22 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:21:22 -0000 Subject: [Python-checkins] [3.10] gh-105993: Add possible `None` return type to `asyncio.EventLoop.start_tls` docs (GH-105995) (#106190) Message-ID: https://github.com/python/cpython/commit/f91dfdf5ff9f68a4b012e1b70ab9997c6dc1542d commit: f91dfdf5ff9f68a4b012e1b70ab9997c6dc1542d branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:21:19+02:00 summary: [3.10] gh-105993: Add possible `None` return type to `asyncio.EventLoop.start_tls` docs (GH-105995) (#106190) (cherry picked from commit 6b52a581c151914e59c8c367a03bc7309713a73b) Co-authored-by: Sam Bull files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index f68fa013d1b8e..9e59b5b0dc202 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -830,6 +830,9 @@ TLS Upgrade object only because the coder caches *protocol*-side data and sporadically exchanges extra TLS session packets with *transport*. + In some situations (e.g. when the passed transport is already closing) this + may return ``None``. + Parameters: * *transport* and *protocol* instances that methods like From webhook-mailer at python.org Wed Jul 5 07:21:41 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:21:41 -0000 Subject: [Python-checkins] [3.12] Document PYTHONSAFEPATH along side -P (GH-106122) (#106352) Message-ID: https://github.com/python/cpython/commit/da672b2d245fb439f4ff895636bf284e352fa631 commit: da672b2d245fb439f4ff895636bf284e352fa631 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:21:38+02:00 summary: [3.12] Document PYTHONSAFEPATH along side -P (GH-106122) (#106352) Document PYTHONSAFEPATH along side -P (GH-106122) (cherry picked from commit 0355625d94a50f4b816770bad946420d005900b8) Co-authored-by: Jeremy Paige files: M Python/initconfig.c diff --git a/Python/initconfig.c b/Python/initconfig.c index 0d42b7ea082d6..4e5d4bb9876e3 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -49,7 +49,7 @@ Options (and corresponding environment variables):\n\ .pyc extension; also PYTHONOPTIMIZE=x\n\ -OO : do -O changes and also discard docstrings; add .opt-2 before\n\ .pyc extension\n\ --P : don't prepend a potentially unsafe path to sys.path\n\ +-P : don't prepend a potentially unsafe path to sys.path; also PYTHONSAFEPATH\n\ -q : don't print version and copyright messages on interactive startup\n\ -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE\n\ -S : don't imply 'import site' on initialization\n\ @@ -144,7 +144,6 @@ static const char usage_envvars[] = "PYTHONSTARTUP: file executed on interactive startup (no default)\n" "PYTHONPATH : '%lc'-separated list of directories prefixed to the\n" " default module search path. The result is sys.path.\n" -"PYTHONSAFEPATH: don't prepend a potentially unsafe path to sys.path.\n" "PYTHONHOME : alternate directory (or %lc).\n" " The default module search path uses %s.\n" "PYTHONPLATLIBDIR : override sys.platlibdir.\n" @@ -184,6 +183,7 @@ static const char usage_envvars[] = " (-X int_max_str_digits=number)\n" "PYTHONNOUSERSITE : disable user site directory (-s)\n" "PYTHONOPTIMIZE : enable level 1 optimizations (-O)\n" +"PYTHONSAFEPATH : don't prepend a potentially unsafe path to sys.path (-P)\n" "PYTHONUNBUFFERED : disable stdout/stderr buffering (-u)\n" "PYTHONVERBOSE : trace import statements (-v)\n" "PYTHONWARNINGS=arg : warning control (-W arg)\n"; From webhook-mailer at python.org Wed Jul 5 07:21:49 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:21:49 -0000 Subject: [Python-checkins] [3.11] Document PYTHONSAFEPATH along side -P (GH-106122) (#106353) Message-ID: https://github.com/python/cpython/commit/ddfe8eb45f23885af0b89901df6907172a37f7f4 commit: ddfe8eb45f23885af0b89901df6907172a37f7f4 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:21:45+02:00 summary: [3.11] Document PYTHONSAFEPATH along side -P (GH-106122) (#106353) (cherry picked from commit 0355625d94a50f4b816770bad946420d005900b8) Co-authored-by: Jeremy Paige files: M Python/initconfig.c diff --git a/Python/initconfig.c b/Python/initconfig.c index d81cbaff7ec97..50d2498a330c3 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -49,7 +49,7 @@ Options (and corresponding environment variables):\n\ .pyc extension; also PYTHONOPTIMIZE=x\n\ -OO : do -O changes and also discard docstrings; add .opt-2 before\n\ .pyc extension\n\ --P : don't prepend a potentially unsafe path to sys.path\n\ +-P : don't prepend a potentially unsafe path to sys.path; also PYTHONSAFEPATH\n\ -q : don't print version and copyright messages on interactive startup\n\ -s : don't add user site directory to sys.path; also PYTHONNOUSERSITE\n\ -S : don't imply 'import site' on initialization\n\ @@ -132,7 +132,6 @@ static const char usage_envvars[] = "PYTHONSTARTUP: file executed on interactive startup (no default)\n" "PYTHONPATH : '%lc'-separated list of directories prefixed to the\n" " default module search path. The result is sys.path.\n" -"PYTHONSAFEPATH: don't prepend a potentially unsafe path to sys.path.\n" "PYTHONHOME : alternate directory (or %lc).\n" " The default module search path uses %s.\n" "PYTHONPLATLIBDIR : override sys.platlibdir.\n" @@ -172,6 +171,7 @@ static const char usage_envvars[] = " (-X int_max_str_digits=number)\n" "PYTHONNOUSERSITE : disable user site directory (-s)\n" "PYTHONOPTIMIZE : enable level 1 optimizations (-O)\n" +"PYTHONSAFEPATH : don't prepend a potentially unsafe path to sys.path (-P)\n" "PYTHONUNBUFFERED : disable stdout/stderr buffering (-u)\n" "PYTHONVERBOSE : trace import statements (-v)\n" "PYTHONWARNINGS=arg : warning control (-W arg)\n"; From webhook-mailer at python.org Wed Jul 5 07:23:25 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:23:25 -0000 Subject: [Python-checkins] gh-64595: Fix regression in file write logic in Argument Clinic (#106449) Message-ID: https://github.com/python/cpython/commit/9d1d4f9c73a71192b22ab52a2eb9278737f98ddb commit: 9d1d4f9c73a71192b22ab52a2eb9278737f98ddb branch: main author: Erlend E. Aasland committer: ambv date: 2023-07-05T13:23:22+02:00 summary: gh-64595: Fix regression in file write logic in Argument Clinic (#106449) Revert the two commits that introduced the regressions: - gh-104152 - gh-104507 files: M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 7c46e8a81803a..685ba58642a5a 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -100,9 +100,8 @@ def test_eol(self): # the last line of the block got corrupted. c = clinic.Clinic(clinic.CLanguage(None), filename="file") raw = "/*[clinic]\nfoo\n[clinic]*/" - cooked, _ = c.parse(raw) - lines = cooked.splitlines() - end_line = lines[2].rstrip() + cooked = c.parse(raw).splitlines() + end_line = cooked[2].rstrip() # this test is redundant, it's just here explicitly to catch # the regression test so we don't forget what it looked like self.assertNotEqual(end_line, "[clinic]*/[clinic]*/") @@ -261,7 +260,7 @@ def _test_clinic(self, input, output): c = clinic.Clinic(language, filename="file") c.parsers['inert'] = InertParser(c) c.parsers['copy'] = CopyParser(c) - computed, _ = c.parse(input) + computed = c.parse(input) self.assertEqual(output, computed) def test_clinic_1(self): diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 5f5d024b5aa6f..7ada7e9d917b3 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2021,22 +2021,20 @@ def dump(self): extensions['py'] = PythonLanguage -def file_changed(filename: str, new_contents: str) -> bool: - """Return true if file contents changed (meaning we must update it)""" +def write_file(filename: str, new_contents: str) -> None: try: - with open(filename, encoding="utf-8") as fp: + with open(filename, 'r', encoding="utf-8") as fp: old_contents = fp.read() - return old_contents != new_contents - except FileNotFoundError: - return True - -def write_file(filename: str, new_contents: str) -> None: + if old_contents == new_contents: + # no change: avoid modifying the file modification time + return + except FileNotFoundError: + pass # Atomic write using a temporary file and os.replace() filename_new = f"{filename}.new" with open(filename_new, "w", encoding="utf-8") as fp: fp.write(new_contents) - try: os.replace(filename_new, filename) except: @@ -2214,8 +2212,6 @@ def parse(self, input): traceback.format_exc().rstrip()) printer.print_block(block) - clinic_out = [] - # these are destinations not buffers for name, destination in self.destinations.items(): if destination.type == 'suppress': @@ -2223,7 +2219,6 @@ def parse(self, input): output = destination.dump() if output: - block = Block("", dsl_name="clinic", output=output) if destination.type == 'buffer': @@ -2255,11 +2250,10 @@ def parse(self, input): block.input = 'preserve\n' printer_2 = BlockPrinter(self.language) printer_2.print_block(block, core_includes=True) - pair = destination.filename, printer_2.f.getvalue() - clinic_out.append(pair) + write_file(destination.filename, printer_2.f.getvalue()) continue - return printer.f.getvalue(), clinic_out + return printer.f.getvalue() def _module_and_class(self, fields): @@ -2321,14 +2315,9 @@ def parse_file( assert isinstance(language, CLanguage) clinic = Clinic(language, verify=verify, filename=filename) - src_out, clinic_out = clinic.parse(raw) - - changes = [(fn, data) for fn, data in clinic_out if file_changed(fn, data)] - if changes: - # Always (re)write the source file. - write_file(output, src_out) - for fn, data in clinic_out: - write_file(fn, data) + cooked = clinic.parse(raw) + + write_file(output, cooked) def compute_checksum( From webhook-mailer at python.org Wed Jul 5 07:40:03 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:40:03 -0000 Subject: [Python-checkins] [3.12] gh-105497: [Enum] Fix Flag inversion when alias/mask members exist. (GH-105542) (#105572) Message-ID: https://github.com/python/cpython/commit/74d84cf84d18e8cf69ffef0d7955f80fbc47220a commit: 74d84cf84d18e8cf69ffef0d7955f80fbc47220a branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:39:59+02:00 summary: [3.12] gh-105497: [Enum] Fix Flag inversion when alias/mask members exist. (GH-105542) (#105572) gh-105497: [Enum] Fix Flag inversion when alias/mask members exist. (GH-105542) When inverting a Flag member (or boundary STRICT), only consider other canonical flags; when inverting an IntFlag member (or boundary KEEP), also consider aliases. (cherry picked from commit 59f009e5898a006cdc8f5249be589de6edfe5cd0) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 62df304057a10..92a9632dc892d 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1463,12 +1463,11 @@ def _missing_(cls, value): else: pseudo_member._name_ = None # use setdefault in case another thread already created a composite - # with this value, but only if all members are known - # note: zero is a special case -- add it - if not unknown: - pseudo_member = cls._value2member_map_.setdefault(value, pseudo_member) - if neg_value is not None: - cls._value2member_map_[neg_value] = pseudo_member + # with this value + # note: zero is a special case -- always add it + pseudo_member = cls._value2member_map_.setdefault(value, pseudo_member) + if neg_value is not None: + cls._value2member_map_[neg_value] = pseudo_member return pseudo_member def __contains__(self, other): @@ -1544,8 +1543,8 @@ def __invert__(self): # use all bits self._inverted_ = self.__class__(~self._value_) else: - # calculate flags not in this member - self._inverted_ = self.__class__(self._flag_mask_ ^ self._value_) + # use canonical bits (i.e. calculate flags not in this member) + self._inverted_ = self.__class__(self._singles_mask_ ^ self._value_) if isinstance(self._inverted_, self.__class__): self._inverted_._inverted_ = self return self._inverted_ diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index b4ac3abfc7006..291213a13f36b 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -3045,6 +3045,33 @@ class Color(Flag): WHITE = RED|GREEN|BLUE BLANCO = RED|GREEN|BLUE + class Complete(Flag): + A = 0x01 + B = 0x02 + + class Partial(Flag): + A = 0x01 + B = 0x02 + MASK = 0xff + + class CompleteInt(IntFlag): + A = 0x01 + B = 0x02 + + class PartialInt(IntFlag): + A = 0x01 + B = 0x02 + MASK = 0xff + + class CompleteIntStrict(IntFlag, boundary=STRICT): + A = 0x01 + B = 0x02 + + class PartialIntStrict(IntFlag, boundary=STRICT): + A = 0x01 + B = 0x02 + MASK = 0xff + def test_or(self): Perm = self.Perm for i in Perm: @@ -3103,6 +3130,18 @@ def test_invert(self): Open = self.Open self.assertIs(Open.WO & ~Open.WO, Open.RO) self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) + Complete = self.Complete + self.assertIs(~Complete.A, Complete.B) + Partial = self.Partial + self.assertIs(~Partial.A, Partial.B) + CompleteInt = self.CompleteInt + self.assertIs(~CompleteInt.A, CompleteInt.B) + PartialInt = self.PartialInt + self.assertIs(~PartialInt.A, PartialInt(254)) + CompleteIntStrict = self.CompleteIntStrict + self.assertIs(~CompleteIntStrict.A, CompleteIntStrict.B) + PartialIntStrict = self.PartialIntStrict + self.assertIs(~PartialIntStrict.A, PartialIntStrict.B) def test_bool(self): Perm = self.Perm diff --git a/Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst b/Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst new file mode 100644 index 0000000000000..2d4e2091b5071 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst @@ -0,0 +1 @@ +Fix flag inversion when alias/mask members exist. From webhook-mailer at python.org Wed Jul 5 07:59:21 2023 From: webhook-mailer at python.org (ambv) Date: Wed, 05 Jul 2023 11:59:21 -0000 Subject: [Python-checkins] [3.11] gh-105497: [Enum] Fix Flag inversion when alias/mask members exist. (GH-105542) (#105571) Message-ID: https://github.com/python/cpython/commit/3f244b2f6e355a15658a7b58792819b4143be0d9 commit: 3f244b2f6e355a15658a7b58792819b4143be0d9 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-05T13:59:18+02:00 summary: [3.11] gh-105497: [Enum] Fix Flag inversion when alias/mask members exist. (GH-105542) (#105571) When inverting a Flag member (or boundary STRICT), only consider other canonical flags; when inverting an IntFlag member (or boundary KEEP), also consider aliases. (cherry picked from commit 59f009e5898a006cdc8f5249be589de6edfe5cd0) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index f07fb37852e68..d59026162bdc6 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1446,12 +1446,11 @@ def _missing_(cls, value): else: pseudo_member._name_ = None # use setdefault in case another thread already created a composite - # with this value, but only if all members are known - # note: zero is a special case -- add it - if not unknown: - pseudo_member = cls._value2member_map_.setdefault(value, pseudo_member) - if neg_value is not None: - cls._value2member_map_[neg_value] = pseudo_member + # with this value + # note: zero is a special case -- always add it + pseudo_member = cls._value2member_map_.setdefault(value, pseudo_member) + if neg_value is not None: + cls._value2member_map_[neg_value] = pseudo_member return pseudo_member def __contains__(self, other): @@ -1527,8 +1526,8 @@ def __invert__(self): # use all bits self._inverted_ = self.__class__(~self._value_) else: - # calculate flags not in this member - self._inverted_ = self.__class__(self._flag_mask_ ^ self._value_) + # use canonical bits (i.e. calculate flags not in this member) + self._inverted_ = self.__class__(self._singles_mask_ ^ self._value_) if isinstance(self._inverted_, self.__class__): self._inverted_._inverted_ = self return self._inverted_ diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 50048253a8096..ce8182e789b9a 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -2913,6 +2913,33 @@ class Color(Flag): WHITE = RED|GREEN|BLUE BLANCO = RED|GREEN|BLUE + class Complete(Flag): + A = 0x01 + B = 0x02 + + class Partial(Flag): + A = 0x01 + B = 0x02 + MASK = 0xff + + class CompleteInt(IntFlag): + A = 0x01 + B = 0x02 + + class PartialInt(IntFlag): + A = 0x01 + B = 0x02 + MASK = 0xff + + class CompleteIntStrict(IntFlag, boundary=STRICT): + A = 0x01 + B = 0x02 + + class PartialIntStrict(IntFlag, boundary=STRICT): + A = 0x01 + B = 0x02 + MASK = 0xff + def test_or(self): Perm = self.Perm for i in Perm: @@ -2971,6 +2998,18 @@ def test_invert(self): Open = self.Open self.assertIs(Open.WO & ~Open.WO, Open.RO) self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) + Complete = self.Complete + self.assertIs(~Complete.A, Complete.B) + Partial = self.Partial + self.assertIs(~Partial.A, Partial.B) + CompleteInt = self.CompleteInt + self.assertIs(~CompleteInt.A, CompleteInt.B) + PartialInt = self.PartialInt + self.assertIs(~PartialInt.A, PartialInt(254)) + CompleteIntStrict = self.CompleteIntStrict + self.assertIs(~CompleteIntStrict.A, CompleteIntStrict.B) + PartialIntStrict = self.PartialIntStrict + self.assertIs(~PartialIntStrict.A, PartialIntStrict.B) def test_bool(self): Perm = self.Perm diff --git a/Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst b/Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst new file mode 100644 index 0000000000000..2d4e2091b5071 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst @@ -0,0 +1 @@ +Fix flag inversion when alias/mask members exist. From webhook-mailer at python.org Wed Jul 5 09:17:13 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 05 Jul 2023 13:17:13 -0000 Subject: [Python-checkins] [3.12] gh-64595: Fix regression in file write logic in Argument Clinic (#106449) (#106452) Message-ID: https://github.com/python/cpython/commit/b72601e7e5a8e3286b1345ac14fa21ac2d3056dc commit: b72601e7e5a8e3286b1345ac14fa21ac2d3056dc branch: 3.12 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-05T13:17:09Z summary: [3.12] gh-64595: Fix regression in file write logic in Argument Clinic (#106449) (#106452) Revert the two commits that introduced the regressions: - gh-104152 - gh-104507 (cherry picked from commit 9d1d4f9c73a71192b22ab52a2eb9278737f98ddb) files: M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 7c46e8a81803a..685ba58642a5a 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -100,9 +100,8 @@ def test_eol(self): # the last line of the block got corrupted. c = clinic.Clinic(clinic.CLanguage(None), filename="file") raw = "/*[clinic]\nfoo\n[clinic]*/" - cooked, _ = c.parse(raw) - lines = cooked.splitlines() - end_line = lines[2].rstrip() + cooked = c.parse(raw).splitlines() + end_line = cooked[2].rstrip() # this test is redundant, it's just here explicitly to catch # the regression test so we don't forget what it looked like self.assertNotEqual(end_line, "[clinic]*/[clinic]*/") @@ -261,7 +260,7 @@ def _test_clinic(self, input, output): c = clinic.Clinic(language, filename="file") c.parsers['inert'] = InertParser(c) c.parsers['copy'] = CopyParser(c) - computed, _ = c.parse(input) + computed = c.parse(input) self.assertEqual(output, computed) def test_clinic_1(self): diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index fa6c1392e1f09..04951e97bbd80 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1966,22 +1966,20 @@ def dump(self): extensions['py'] = PythonLanguage -def file_changed(filename: str, new_contents: str) -> bool: - """Return true if file contents changed (meaning we must update it)""" +def write_file(filename: str, new_contents: str) -> None: try: - with open(filename, encoding="utf-8") as fp: + with open(filename, 'r', encoding="utf-8") as fp: old_contents = fp.read() - return old_contents != new_contents - except FileNotFoundError: - return True - -def write_file(filename: str, new_contents: str): + if old_contents == new_contents: + # no change: avoid modifying the file modification time + return + except FileNotFoundError: + pass # Atomic write using a temporary file and os.replace() filename_new = f"{filename}.new" with open(filename_new, "w", encoding="utf-8") as fp: fp.write(new_contents) - try: os.replace(filename_new, filename) except: @@ -2159,8 +2157,6 @@ def parse(self, input): traceback.format_exc().rstrip()) printer.print_block(block) - clinic_out = [] - # these are destinations not buffers for name, destination in self.destinations.items(): if destination.type == 'suppress': @@ -2168,7 +2164,6 @@ def parse(self, input): output = destination.dump() if output: - block = Block("", dsl_name="clinic", output=output) if destination.type == 'buffer': @@ -2200,11 +2195,10 @@ def parse(self, input): block.input = 'preserve\n' printer_2 = BlockPrinter(self.language) printer_2.print_block(block, core_includes=True) - pair = destination.filename, printer_2.f.getvalue() - clinic_out.append(pair) + write_file(destination.filename, printer_2.f.getvalue()) continue - return printer.f.getvalue(), clinic_out + return printer.f.getvalue() def _module_and_class(self, fields): @@ -2266,14 +2260,9 @@ def parse_file( assert isinstance(language, CLanguage) clinic = Clinic(language, verify=verify, filename=filename) - src_out, clinic_out = clinic.parse(raw) - - changes = [(fn, data) for fn, data in clinic_out if file_changed(fn, data)] - if changes: - # Always (re)write the source file. - write_file(output, src_out) - for fn, data in clinic_out: - write_file(fn, data) + cooked = clinic.parse(raw) + + write_file(output, cooked) def compute_checksum( From webhook-mailer at python.org Wed Jul 5 11:07:06 2023 From: webhook-mailer at python.org (gvanrossum) Date: Wed, 05 Jul 2023 15:07:06 -0000 Subject: [Python-checkins] Clarify state of CancelledError in doc (#106453) Message-ID: https://github.com/python/cpython/commit/12a98138083589314d3da14bc97f2d8517947437 commit: 12a98138083589314d3da14bc97f2d8517947437 branch: main author: Kristj?n Valur J?nsson committer: gvanrossum date: 2023-07-05T08:07:02-07:00 summary: Clarify state of CancelledError in doc (#106453) This change makes it explicit that asyncio.CancelledError is not a subclass of Exception. files: M Doc/library/asyncio-exceptions.rst diff --git a/Doc/library/asyncio-exceptions.rst b/Doc/library/asyncio-exceptions.rst index 9250f01b8a089..7ad9103ca3fdf 100644 --- a/Doc/library/asyncio-exceptions.rst +++ b/Doc/library/asyncio-exceptions.rst @@ -31,7 +31,7 @@ Exceptions .. versionchanged:: 3.8 - :exc:`CancelledError` is now a subclass of :class:`BaseException`. + :exc:`CancelledError` is now a subclass of :class:`BaseException` rather than :class:`Exception`. .. exception:: InvalidStateError From webhook-mailer at python.org Wed Jul 5 12:17:41 2023 From: webhook-mailer at python.org (warsaw) Date: Wed, 05 Jul 2023 16:17:41 -0000 Subject: [Python-checkins] gh-102542 Remove unused bytes object and bytes slicing (#106433) Message-ID: https://github.com/python/cpython/commit/70e2a42647f2f4b53d0f07c0c7db48ea27e066fa commit: 70e2a42647f2f4b53d0f07c0c7db48ea27e066fa branch: main author: JosephSBoyle <48555120+JosephSBoyle at users.noreply.github.com> committer: warsaw date: 2023-07-05T09:17:37-07:00 summary: gh-102542 Remove unused bytes object and bytes slicing (#106433) Remove unused bytes object and bytes slicing Co-authored-by: Shantanu <12621235+hauntsaninja at users.noreply.github.com> files: M Lib/email/mime/audio.py diff --git a/Lib/email/mime/audio.py b/Lib/email/mime/audio.py index 065819b2a2101..aa0c4905cbb2b 100644 --- a/Lib/email/mime/audio.py +++ b/Lib/email/mime/audio.py @@ -6,7 +6,6 @@ __all__ = ['MIMEAudio'] -from io import BytesIO from email import encoders from email.mime.nonmultipart import MIMENonMultipart @@ -59,10 +58,8 @@ def _what(data): # sndhdr.what() had a pretty cruddy interface, unfortunately. This is why # we re-do it here. It would be easier to reverse engineer the Unix 'file' # command and use the standard 'magic' file, as shipped with a modern Unix. - hdr = data[:512] - fakefile = BytesIO(hdr) for testfn in _rules: - if res := testfn(hdr, fakefile): + if res := testfn(data): return res else: return None @@ -74,7 +71,7 @@ def rule(rulefunc): @rule -def _aiff(h, f): +def _aiff(h): if not h.startswith(b'FORM'): return None if h[8:12] in {b'AIFC', b'AIFF'}: @@ -84,7 +81,7 @@ def _aiff(h, f): @rule -def _au(h, f): +def _au(h): if h.startswith(b'.snd'): return 'basic' else: @@ -92,7 +89,7 @@ def _au(h, f): @rule -def _wav(h, f): +def _wav(h): # 'RIFF' 'WAVE' 'fmt ' if not h.startswith(b'RIFF') or h[8:12] != b'WAVE' or h[12:16] != b'fmt ': return None From webhook-mailer at python.org Wed Jul 5 14:17:00 2023 From: webhook-mailer at python.org (gvanrossum) Date: Wed, 05 Jul 2023 18:17:00 -0000 Subject: [Python-checkins] [3.11] Clarify state of CancelledError in doc (GH-106453) (#106456) Message-ID: https://github.com/python/cpython/commit/ada22db1478f27af0ee40a5d67d220d05d4897c8 commit: ada22db1478f27af0ee40a5d67d220d05d4897c8 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gvanrossum date: 2023-07-05T11:16:56-07:00 summary: [3.11] Clarify state of CancelledError in doc (GH-106453) (#106456) Clarify state of CancelledError in doc (GH-106453) This change makes it explicit that asyncio.CancelledError is not a subclass of Exception. (cherry picked from commit 12a98138083589314d3da14bc97f2d8517947437) Co-authored-by: Kristj?n Valur J?nsson files: M Doc/library/asyncio-exceptions.rst diff --git a/Doc/library/asyncio-exceptions.rst b/Doc/library/asyncio-exceptions.rst index 9250f01b8a089..7ad9103ca3fdf 100644 --- a/Doc/library/asyncio-exceptions.rst +++ b/Doc/library/asyncio-exceptions.rst @@ -31,7 +31,7 @@ Exceptions .. versionchanged:: 3.8 - :exc:`CancelledError` is now a subclass of :class:`BaseException`. + :exc:`CancelledError` is now a subclass of :class:`BaseException` rather than :class:`Exception`. .. exception:: InvalidStateError From webhook-mailer at python.org Wed Jul 5 16:38:56 2023 From: webhook-mailer at python.org (JelleZijlstra) Date: Wed, 05 Jul 2023 20:38:56 -0000 Subject: [Python-checkins] shlex docs: remove outdated note (#106463) Message-ID: https://github.com/python/cpython/commit/c16ea94abc73c0098b484f7e2ec23bfd9c36b67c commit: c16ea94abc73c0098b484f7e2ec23bfd9c36b67c branch: main author: Jelle Zijlstra committer: JelleZijlstra date: 2023-07-05T13:38:53-07:00 summary: shlex docs: remove outdated note (#106463) As the versionchanged notice says, this note is no longer true on 3.12+. files: M Doc/library/shlex.rst diff --git a/Doc/library/shlex.rst b/Doc/library/shlex.rst index 0bad51833aae1..f94833ad5331a 100644 --- a/Doc/library/shlex.rst +++ b/Doc/library/shlex.rst @@ -30,12 +30,6 @@ The :mod:`shlex` module defines the following functions: in POSIX mode by default, but uses non-POSIX mode if the *posix* argument is false. - .. note:: - - Since the :func:`split` function instantiates a :class:`~shlex.shlex` - instance, passing ``None`` for *s* will read the string to split from - standard input. - .. versionchanged:: 3.12 Passing ``None`` for *s* argument now raises an exception, rather than reading :data:`sys.stdin`. From webhook-mailer at python.org Wed Jul 5 16:57:52 2023 From: webhook-mailer at python.org (JelleZijlstra) Date: Wed, 05 Jul 2023 20:57:52 -0000 Subject: [Python-checkins] [3.12] shlex docs: remove outdated note (GH-106463) (#106466) Message-ID: https://github.com/python/cpython/commit/a49a29f22bcc39376e760823ec512df831d2e828 commit: a49a29f22bcc39376e760823ec512df831d2e828 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: JelleZijlstra date: 2023-07-05T13:57:49-07:00 summary: [3.12] shlex docs: remove outdated note (GH-106463) (#106466) shlex docs: remove outdated note (GH-106463) As the versionchanged notice says, this note is no longer true on 3.12+. (cherry picked from commit c16ea94abc73c0098b484f7e2ec23bfd9c36b67c) Co-authored-by: Jelle Zijlstra files: M Doc/library/shlex.rst diff --git a/Doc/library/shlex.rst b/Doc/library/shlex.rst index 0bad51833aae1..f94833ad5331a 100644 --- a/Doc/library/shlex.rst +++ b/Doc/library/shlex.rst @@ -30,12 +30,6 @@ The :mod:`shlex` module defines the following functions: in POSIX mode by default, but uses non-POSIX mode if the *posix* argument is false. - .. note:: - - Since the :func:`split` function instantiates a :class:`~shlex.shlex` - instance, passing ``None`` for *s* will read the string to split from - standard input. - .. versionchanged:: 3.12 Passing ``None`` for *s* argument now raises an exception, rather than reading :data:`sys.stdin`. From webhook-mailer at python.org Wed Jul 5 18:19:53 2023 From: webhook-mailer at python.org (corona10) Date: Wed, 05 Jul 2023 22:19:53 -0000 Subject: [Python-checkins] gh-96844: Improve error message of list.remove (gh-106455) Message-ID: https://github.com/python/cpython/commit/217f47d6e5e56bca78b8556e910cd00890f6f84a commit: 217f47d6e5e56bca78b8556e910cd00890f6f84a branch: main author: Dong-hee Na committer: corona10 date: 2023-07-06T07:19:49+09:00 summary: gh-96844: Improve error message of list.remove (gh-106455) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-06-00-35-44.gh-issue-96844.kwvoS-.rst M Doc/library/doctest.rst M Lib/test/test_xml_etree.py M Objects/listobject.c diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst index d6e4dca086067..92da6133f9bf0 100644 --- a/Doc/library/doctest.rst +++ b/Doc/library/doctest.rst @@ -409,10 +409,10 @@ Simple example:: >>> [1, 2, 3].remove(42) Traceback (most recent call last): File "", line 1, in - ValueError: list.remove(x): x not in list + ValueError: 42 is not in list -That doctest succeeds if :exc:`ValueError` is raised, with the ``list.remove(x): -x not in list`` detail as shown. +That doctest succeeds if :exc:`ValueError` is raised, with the ``42 is not in list`` +detail as shown. The expected output for an exception must start with a traceback header, which may be either of the following two lines, indented the same as the first line of diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index 11efee00582e0..b9352cb865d02 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -328,7 +328,7 @@ def test_simpleops(self): self.serialize_check(element, '') # 5 with self.assertRaises(ValueError) as cm: element.remove(subelement) - self.assertEqual(str(cm.exception), 'list.remove(x): x not in list') + self.assertIn('not in list', str(cm.exception)) self.serialize_check(element, '') # 6 element[0:0] = [subelement, subelement, subelement] self.serialize_check(element[1], '') diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-06-00-35-44.gh-issue-96844.kwvoS-.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-06-00-35-44.gh-issue-96844.kwvoS-.rst new file mode 100644 index 0000000000000..55334173bc002 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-06-00-35-44.gh-issue-96844.kwvoS-.rst @@ -0,0 +1 @@ +Improve error message of :meth:`list.remove`. Patch by Dong-hee Na. diff --git a/Objects/listobject.c b/Objects/listobject.c index 98fa08962b6aa..144ede6351e03 100644 --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -2694,7 +2694,7 @@ list_remove(PyListObject *self, PyObject *value) else if (cmp < 0) return NULL; } - PyErr_SetString(PyExc_ValueError, "list.remove(x): x not in list"); + PyErr_Format(PyExc_ValueError, "%R is not in list", value); return NULL; } From webhook-mailer at python.org Wed Jul 5 19:01:40 2023 From: webhook-mailer at python.org (carljm) Date: Wed, 05 Jul 2023 23:01:40 -0000 Subject: [Python-checkins] gh-106292: restore checking __dict__ in cached_property.__get__ (#106380) Message-ID: https://github.com/python/cpython/commit/838406b4fc044c0b2f397c23275c69f16a76205b commit: 838406b4fc044c0b2f397c23275c69f16a76205b branch: main author: Carl Meyer committer: carljm date: 2023-07-05T17:01:35-06:00 summary: gh-106292: restore checking __dict__ in cached_property.__get__ (#106380) * gh-106292: restore checking __dict__ in cached_property.__get__ Co-authored-by: Dong-hee Na files: A Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst M Lib/functools.py M Lib/test/test_functools.py diff --git a/Lib/functools.py b/Lib/functools.py index 4d5e270900784..8518450a8d499 100644 --- a/Lib/functools.py +++ b/Lib/functools.py @@ -957,9 +957,10 @@ def __isabstractmethod__(self): ################################################################################ -### cached_property() - computed once per instance, cached as attribute +### cached_property() - property result cached as instance attribute ################################################################################ +_NOT_FOUND = object() class cached_property: def __init__(self, func): @@ -990,15 +991,17 @@ def __get__(self, instance, owner=None): f"instance to cache {self.attrname!r} property." ) raise TypeError(msg) from None - val = self.func(instance) - try: - cache[self.attrname] = val - except TypeError: - msg = ( - f"The '__dict__' attribute on {type(instance).__name__!r} instance " - f"does not support item assignment for caching {self.attrname!r} property." - ) - raise TypeError(msg) from None + val = cache.get(self.attrname, _NOT_FOUND) + if val is _NOT_FOUND: + val = self.func(instance) + try: + cache[self.attrname] = val + except TypeError: + msg = ( + f"The '__dict__' attribute on {type(instance).__name__!r} instance " + f"does not support item assignment for caching {self.attrname!r} property." + ) + raise TypeError(msg) from None return val __class_getitem__ = classmethod(GenericAlias) diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py index d668fa4c3adf5..c4eca0f5b7951 100644 --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -3037,6 +3037,25 @@ def test_access_from_class(self): def test_doc(self): self.assertEqual(CachedCostItem.cost.__doc__, "The cost of the item.") + def test_subclass_with___set__(self): + """Caching still works for a subclass defining __set__.""" + class readonly_cached_property(py_functools.cached_property): + def __set__(self, obj, value): + raise AttributeError("read only property") + + class Test: + def __init__(self, prop): + self._prop = prop + + @readonly_cached_property + def prop(self): + return self._prop + + t = Test(1) + self.assertEqual(t.prop, 1) + t._prop = 999 + self.assertEqual(t.prop, 1) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst b/Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst new file mode 100644 index 0000000000000..233509344d509 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst @@ -0,0 +1,4 @@ +Check for an instance-dict cached value in the :meth:`__get__` method of +:func:`functools.cached_property`. This better matches the pre-3.12 behavior +and improves compatibility for users subclassing +:func:`functools.cached_property` and adding a :meth:`__set__` method. From webhook-mailer at python.org Wed Jul 5 19:05:06 2023 From: webhook-mailer at python.org (carljm) Date: Wed, 05 Jul 2023 23:05:06 -0000 Subject: [Python-checkins] gh-105340: include hidden fast-locals in locals() (#105715) Message-ID: https://github.com/python/cpython/commit/104d7b760fed18055e4f04e5da3ca619e28bfc81 commit: 104d7b760fed18055e4f04e5da3ca619e28bfc81 branch: main author: Carl Meyer committer: carljm date: 2023-07-05T17:05:02-06:00 summary: gh-105340: include hidden fast-locals in locals() (#105715) * gh-105340: include hidden fast-locals in locals() files: A Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst M Include/internal/pycore_ceval.h M Include/internal/pycore_frame.h M Lib/test/test_listcomps.py M Objects/frameobject.c M Objects/object.c M Python/bltinmodule.c M Python/ceval.c diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h index 46bc18cff86d5..e729904ff2c4c 100644 --- a/Include/internal/pycore_ceval.h +++ b/Include/internal/pycore_ceval.h @@ -156,6 +156,7 @@ extern PyObject* _Py_MakeCoro(PyFunctionObject *func); and asynchronous exception */ extern int _Py_HandlePending(PyThreadState *tstate); +extern PyObject * _PyEval_GetFrameLocals(void); #ifdef __cplusplus diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h index 952a96009dda6..efc19e33ec5dc 100644 --- a/Include/internal/pycore_frame.h +++ b/Include/internal/pycore_frame.h @@ -234,6 +234,9 @@ _PyFrame_ClearExceptCode(_PyInterpreterFrame * frame); int _PyFrame_Traverse(_PyInterpreterFrame *frame, visitproc visit, void *arg); +PyObject * +_PyFrame_GetLocals(_PyInterpreterFrame *frame, int include_hidden); + int _PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame); diff --git a/Lib/test/test_listcomps.py b/Lib/test/test_listcomps.py index c2cf058c321fa..9f28ced32bd26 100644 --- a/Lib/test/test_listcomps.py +++ b/Lib/test/test_listcomps.py @@ -539,6 +539,28 @@ def b(): self._check_in_scopes(code, {"x": True, "y": ["b"]}, scopes=["function"]) self._check_in_scopes(code, raises=NameError, scopes=["class"]) + def test_iter_var_available_in_locals(self): + code = """ + l = [1, 2] + y = 0 + items = [locals()["x"] for x in l] + items2 = [vars()["x"] for x in l] + items3 = [("x" in dir()) for x in l] + items4 = [eval("x") for x in l] + # x is available, and does not overwrite y + [exec("y = x") for x in l] + """ + self._check_in_scopes( + code, + { + "items": [1, 2], + "items2": [1, 2], + "items3": [True, True], + "items4": [1, 2], + "y": 0 + } + ) + __test__ = {'doctests' : doctests} diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst new file mode 100644 index 0000000000000..f6d4fa8fc4d74 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst @@ -0,0 +1,2 @@ +Include the comprehension iteration variable in ``locals()`` inside a +module- or class-scope comprehension. diff --git a/Objects/frameobject.c b/Objects/frameobject.c index 0158d72d6b4f9..18820551a0547 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -1200,15 +1200,28 @@ frame_get_var(_PyInterpreterFrame *frame, PyCodeObject *co, int i, return 1; } -int -_PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame) + +PyObject * +_PyFrame_GetLocals(_PyInterpreterFrame *frame, int include_hidden) { /* Merge fast locals into f->f_locals */ PyObject *locals = frame->f_locals; if (locals == NULL) { locals = frame->f_locals = PyDict_New(); if (locals == NULL) { - return -1; + return NULL; + } + } + PyObject *hidden = NULL; + + /* If include_hidden, "hidden" fast locals (from inlined comprehensions in + module/class scopes) will be included in the returned dict, but not in + frame->f_locals; the returned dict will be a modified copy. Non-hidden + locals will still be updated in frame->f_locals. */ + if (include_hidden) { + hidden = PyDict_New(); + if (hidden == NULL) { + return NULL; } } @@ -1224,6 +1237,11 @@ _PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame) PyObject *name = PyTuple_GET_ITEM(co->co_localsplusnames, i); _PyLocals_Kind kind = _PyLocals_GetKind(co->co_localspluskinds, i); if (kind & CO_FAST_HIDDEN) { + if (include_hidden && value != NULL) { + if (PyObject_SetItem(hidden, name, value) != 0) { + goto error; + } + } continue; } if (value == NULL) { @@ -1232,16 +1250,53 @@ _PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame) PyErr_Clear(); } else { - return -1; + goto error; } } } else { if (PyObject_SetItem(locals, name, value) != 0) { - return -1; + goto error; } } } + + if (include_hidden && PyDict_Size(hidden)) { + PyObject *innerlocals = PyDict_New(); + if (innerlocals == NULL) { + goto error; + } + if (PyDict_Merge(innerlocals, locals, 1) != 0) { + Py_DECREF(innerlocals); + goto error; + } + if (PyDict_Merge(innerlocals, hidden, 1) != 0) { + Py_DECREF(innerlocals); + goto error; + } + locals = innerlocals; + } + else { + Py_INCREF(locals); + } + Py_CLEAR(hidden); + + return locals; + + error: + Py_XDECREF(hidden); + return NULL; +} + + +int +_PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame) +{ + PyObject *locals = _PyFrame_GetLocals(frame, 0); + if (locals == NULL) { + return -1; + } + Py_DECREF(locals); return 0; } diff --git a/Objects/object.c b/Objects/object.c index b20e87ef3fb23..c27b13e9e0c31 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -1690,13 +1690,15 @@ _dir_locals(void) PyObject *names; PyObject *locals; - locals = PyEval_GetLocals(); + locals = _PyEval_GetFrameLocals(); if (locals == NULL) return NULL; names = PyMapping_Keys(locals); - if (!names) + Py_DECREF(locals); + if (!names) { return NULL; + } if (!PyList_Check(names)) { PyErr_Format(PyExc_TypeError, "dir(): expected keys() of locals to be a list, " @@ -1708,7 +1710,6 @@ _dir_locals(void) Py_DECREF(names); return NULL; } - /* the locals don't need to be DECREF'd */ return names; } diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c index 9fe0067daa678..49efafc07f424 100644 --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -908,7 +908,7 @@ builtin_eval_impl(PyObject *module, PyObject *source, PyObject *globals, PyObject *locals) /*[clinic end generated code: output=0a0824aa70093116 input=11ee718a8640e527]*/ { - PyObject *result, *source_copy; + PyObject *result = NULL, *source_copy; const char *str; if (locals != Py_None && !PyMapping_Check(locals)) { @@ -924,19 +924,25 @@ builtin_eval_impl(PyObject *module, PyObject *source, PyObject *globals, if (globals == Py_None) { globals = PyEval_GetGlobals(); if (locals == Py_None) { - locals = PyEval_GetLocals(); + locals = _PyEval_GetFrameLocals(); if (locals == NULL) return NULL; } + else { + Py_INCREF(locals); + } } else if (locals == Py_None) - locals = globals; + locals = Py_NewRef(globals); + else { + Py_INCREF(locals); + } if (globals == NULL || locals == NULL) { PyErr_SetString(PyExc_TypeError, "eval must be given globals and locals " "when called without a frame"); - return NULL; + goto error; } int r = PyDict_Contains(globals, &_Py_ID(__builtins__)); @@ -944,34 +950,38 @@ builtin_eval_impl(PyObject *module, PyObject *source, PyObject *globals, r = PyDict_SetItem(globals, &_Py_ID(__builtins__), PyEval_GetBuiltins()); } if (r < 0) { - return NULL; + goto error; } if (PyCode_Check(source)) { if (PySys_Audit("exec", "O", source) < 0) { - return NULL; + goto error; } if (PyCode_GetNumFree((PyCodeObject *)source) > 0) { PyErr_SetString(PyExc_TypeError, "code object passed to eval() may not contain free variables"); - return NULL; + goto error; } - return PyEval_EvalCode(source, globals, locals); + result = PyEval_EvalCode(source, globals, locals); } + else { + PyCompilerFlags cf = _PyCompilerFlags_INIT; + cf.cf_flags = PyCF_SOURCE_IS_UTF8; + str = _Py_SourceAsString(source, "eval", "string, bytes or code", &cf, &source_copy); + if (str == NULL) + goto error; - PyCompilerFlags cf = _PyCompilerFlags_INIT; - cf.cf_flags = PyCF_SOURCE_IS_UTF8; - str = _Py_SourceAsString(source, "eval", "string, bytes or code", &cf, &source_copy); - if (str == NULL) - return NULL; + while (*str == ' ' || *str == '\t') + str++; - while (*str == ' ' || *str == '\t') - str++; + (void)PyEval_MergeCompilerFlags(&cf); + result = PyRun_StringFlags(str, Py_eval_input, globals, locals, &cf); + Py_XDECREF(source_copy); + } - (void)PyEval_MergeCompilerFlags(&cf); - result = PyRun_StringFlags(str, Py_eval_input, globals, locals, &cf); - Py_XDECREF(source_copy); + error: + Py_XDECREF(locals); return result; } @@ -1006,36 +1016,43 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, if (globals == Py_None) { globals = PyEval_GetGlobals(); if (locals == Py_None) { - locals = PyEval_GetLocals(); + locals = _PyEval_GetFrameLocals(); if (locals == NULL) return NULL; } + else { + Py_INCREF(locals); + } if (!globals || !locals) { PyErr_SetString(PyExc_SystemError, "globals and locals cannot be NULL"); return NULL; } } - else if (locals == Py_None) - locals = globals; + else if (locals == Py_None) { + locals = Py_NewRef(globals); + } + else { + Py_INCREF(locals); + } if (!PyDict_Check(globals)) { PyErr_Format(PyExc_TypeError, "exec() globals must be a dict, not %.100s", Py_TYPE(globals)->tp_name); - return NULL; + goto error; } if (!PyMapping_Check(locals)) { PyErr_Format(PyExc_TypeError, "locals must be a mapping or None, not %.100s", Py_TYPE(locals)->tp_name); - return NULL; + goto error; } int r = PyDict_Contains(globals, &_Py_ID(__builtins__)); if (r == 0) { r = PyDict_SetItem(globals, &_Py_ID(__builtins__), PyEval_GetBuiltins()); } if (r < 0) { - return NULL; + goto error; } if (closure == Py_None) { @@ -1048,7 +1065,7 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, if (closure) { PyErr_SetString(PyExc_TypeError, "cannot use a closure with this code object"); - return NULL; + goto error; } } else { int closure_is_ok = @@ -1068,12 +1085,12 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, PyErr_Format(PyExc_TypeError, "code object requires a closure of exactly length %zd", num_free); - return NULL; + goto error; } } if (PySys_Audit("exec", "O", source) < 0) { - return NULL; + goto error; } if (!closure) { @@ -1100,7 +1117,7 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, "string, bytes or code", &cf, &source_copy); if (str == NULL) - return NULL; + goto error; if (PyEval_MergeCompilerFlags(&cf)) v = PyRun_StringFlags(str, Py_file_input, globals, locals, &cf); @@ -1109,9 +1126,14 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, Py_XDECREF(source_copy); } if (v == NULL) - return NULL; + goto error; + Py_DECREF(locals); Py_DECREF(v); Py_RETURN_NONE; + + error: + Py_XDECREF(locals); + return NULL; } @@ -1721,10 +1743,7 @@ static PyObject * builtin_locals_impl(PyObject *module) /*[clinic end generated code: output=b46c94015ce11448 input=7874018d478d5c4b]*/ { - PyObject *d; - - d = PyEval_GetLocals(); - return Py_XNewRef(d); + return _PyEval_GetFrameLocals(); } @@ -2442,7 +2461,7 @@ builtin_vars_impl(PyObject *module, PyObject *object) PyObject *d; if (object == NULL) { - d = Py_XNewRef(PyEval_GetLocals()); + d = _PyEval_GetFrameLocals(); } else { if (_PyObject_LookupAttr(object, &_Py_ID(__dict__), &d) == 0) { diff --git a/Python/ceval.c b/Python/ceval.c index 9bcb83f9c993c..6714229fd0784 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2261,6 +2261,19 @@ PyEval_GetLocals(void) return locals; } +PyObject * +_PyEval_GetFrameLocals(void) +{ + PyThreadState *tstate = _PyThreadState_GET(); + _PyInterpreterFrame *current_frame = _PyThreadState_GetFrame(tstate); + if (current_frame == NULL) { + _PyErr_SetString(tstate, PyExc_SystemError, "frame does not exist"); + return NULL; + } + + return _PyFrame_GetLocals(current_frame, 1); +} + PyObject * PyEval_GetGlobals(void) { From webhook-mailer at python.org Wed Jul 5 19:31:40 2023 From: webhook-mailer at python.org (carljm) Date: Wed, 05 Jul 2023 23:31:40 -0000 Subject: [Python-checkins] [3.12] gh-105340: include hidden fast-locals in locals() (GH-105715) (#106470) Message-ID: https://github.com/python/cpython/commit/bb17e6f5de2bca85746a06d504029f90e85e3892 commit: bb17e6f5de2bca85746a06d504029f90e85e3892 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: carljm date: 2023-07-05T23:31:37Z summary: [3.12] gh-105340: include hidden fast-locals in locals() (GH-105715) (#106470) gh-105340: include hidden fast-locals in locals() (GH-105715) * gh-105340: include hidden fast-locals in locals() (cherry picked from commit 104d7b760fed18055e4f04e5da3ca619e28bfc81) Co-authored-by: Carl Meyer files: A Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst M Include/internal/pycore_ceval.h M Include/internal/pycore_frame.h M Lib/test/test_listcomps.py M Objects/frameobject.c M Objects/object.c M Python/bltinmodule.c M Python/ceval.c diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h index 9e9b523e7c222..fc0f72efdae48 100644 --- a/Include/internal/pycore_ceval.h +++ b/Include/internal/pycore_ceval.h @@ -154,6 +154,7 @@ extern PyObject* _Py_MakeCoro(PyFunctionObject *func); extern int _Py_HandlePending(PyThreadState *tstate); +extern PyObject * _PyEval_GetFrameLocals(void); #ifdef __cplusplus diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h index a72e03f1438fc..158db2cf9df82 100644 --- a/Include/internal/pycore_frame.h +++ b/Include/internal/pycore_frame.h @@ -226,6 +226,9 @@ _PyFrame_ClearExceptCode(_PyInterpreterFrame * frame); int _PyFrame_Traverse(_PyInterpreterFrame *frame, visitproc visit, void *arg); +PyObject * +_PyFrame_GetLocals(_PyInterpreterFrame *frame, int include_hidden); + int _PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame); diff --git a/Lib/test/test_listcomps.py b/Lib/test/test_listcomps.py index c2cf058c321fa..9f28ced32bd26 100644 --- a/Lib/test/test_listcomps.py +++ b/Lib/test/test_listcomps.py @@ -539,6 +539,28 @@ def b(): self._check_in_scopes(code, {"x": True, "y": ["b"]}, scopes=["function"]) self._check_in_scopes(code, raises=NameError, scopes=["class"]) + def test_iter_var_available_in_locals(self): + code = """ + l = [1, 2] + y = 0 + items = [locals()["x"] for x in l] + items2 = [vars()["x"] for x in l] + items3 = [("x" in dir()) for x in l] + items4 = [eval("x") for x in l] + # x is available, and does not overwrite y + [exec("y = x") for x in l] + """ + self._check_in_scopes( + code, + { + "items": [1, 2], + "items2": [1, 2], + "items3": [True, True], + "items4": [1, 2], + "y": 0 + } + ) + __test__ = {'doctests' : doctests} diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst new file mode 100644 index 0000000000000..f6d4fa8fc4d74 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst @@ -0,0 +1,2 @@ +Include the comprehension iteration variable in ``locals()`` inside a +module- or class-scope comprehension. diff --git a/Objects/frameobject.c b/Objects/frameobject.c index 2c90a6b71311c..e62cfab80b2e3 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -1199,15 +1199,28 @@ frame_get_var(_PyInterpreterFrame *frame, PyCodeObject *co, int i, return 1; } -int -_PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame) + +PyObject * +_PyFrame_GetLocals(_PyInterpreterFrame *frame, int include_hidden) { /* Merge fast locals into f->f_locals */ PyObject *locals = frame->f_locals; if (locals == NULL) { locals = frame->f_locals = PyDict_New(); if (locals == NULL) { - return -1; + return NULL; + } + } + PyObject *hidden = NULL; + + /* If include_hidden, "hidden" fast locals (from inlined comprehensions in + module/class scopes) will be included in the returned dict, but not in + frame->f_locals; the returned dict will be a modified copy. Non-hidden + locals will still be updated in frame->f_locals. */ + if (include_hidden) { + hidden = PyDict_New(); + if (hidden == NULL) { + return NULL; } } @@ -1223,6 +1236,11 @@ _PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame) PyObject *name = PyTuple_GET_ITEM(co->co_localsplusnames, i); _PyLocals_Kind kind = _PyLocals_GetKind(co->co_localspluskinds, i); if (kind & CO_FAST_HIDDEN) { + if (include_hidden && value != NULL) { + if (PyObject_SetItem(hidden, name, value) != 0) { + goto error; + } + } continue; } if (value == NULL) { @@ -1231,16 +1249,53 @@ _PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame) PyErr_Clear(); } else { - return -1; + goto error; } } } else { if (PyObject_SetItem(locals, name, value) != 0) { - return -1; + goto error; } } } + + if (include_hidden && PyDict_Size(hidden)) { + PyObject *innerlocals = PyDict_New(); + if (innerlocals == NULL) { + goto error; + } + if (PyDict_Merge(innerlocals, locals, 1) != 0) { + Py_DECREF(innerlocals); + goto error; + } + if (PyDict_Merge(innerlocals, hidden, 1) != 0) { + Py_DECREF(innerlocals); + goto error; + } + locals = innerlocals; + } + else { + Py_INCREF(locals); + } + Py_CLEAR(hidden); + + return locals; + + error: + Py_XDECREF(hidden); + return NULL; +} + + +int +_PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame) +{ + PyObject *locals = _PyFrame_GetLocals(frame, 0); + if (locals == NULL) { + return -1; + } + Py_DECREF(locals); return 0; } diff --git a/Objects/object.c b/Objects/object.c index ece0c5e21e77f..b8bdf459201d9 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -1704,13 +1704,15 @@ _dir_locals(void) PyObject *names; PyObject *locals; - locals = PyEval_GetLocals(); + locals = _PyEval_GetFrameLocals(); if (locals == NULL) return NULL; names = PyMapping_Keys(locals); - if (!names) + Py_DECREF(locals); + if (!names) { return NULL; + } if (!PyList_Check(names)) { PyErr_Format(PyExc_TypeError, "dir(): expected keys() of locals to be a list, " @@ -1722,7 +1724,6 @@ _dir_locals(void) Py_DECREF(names); return NULL; } - /* the locals don't need to be DECREF'd */ return names; } diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c index 45ce3b71e84ec..7f366b43599ae 100644 --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -907,7 +907,7 @@ builtin_eval_impl(PyObject *module, PyObject *source, PyObject *globals, PyObject *locals) /*[clinic end generated code: output=0a0824aa70093116 input=11ee718a8640e527]*/ { - PyObject *result, *source_copy; + PyObject *result = NULL, *source_copy; const char *str; if (locals != Py_None && !PyMapping_Check(locals)) { @@ -923,19 +923,25 @@ builtin_eval_impl(PyObject *module, PyObject *source, PyObject *globals, if (globals == Py_None) { globals = PyEval_GetGlobals(); if (locals == Py_None) { - locals = PyEval_GetLocals(); + locals = _PyEval_GetFrameLocals(); if (locals == NULL) return NULL; } + else { + Py_INCREF(locals); + } } else if (locals == Py_None) - locals = globals; + locals = Py_NewRef(globals); + else { + Py_INCREF(locals); + } if (globals == NULL || locals == NULL) { PyErr_SetString(PyExc_TypeError, "eval must be given globals and locals " "when called without a frame"); - return NULL; + goto error; } int r = PyDict_Contains(globals, &_Py_ID(__builtins__)); @@ -943,34 +949,38 @@ builtin_eval_impl(PyObject *module, PyObject *source, PyObject *globals, r = PyDict_SetItem(globals, &_Py_ID(__builtins__), PyEval_GetBuiltins()); } if (r < 0) { - return NULL; + goto error; } if (PyCode_Check(source)) { if (PySys_Audit("exec", "O", source) < 0) { - return NULL; + goto error; } if (PyCode_GetNumFree((PyCodeObject *)source) > 0) { PyErr_SetString(PyExc_TypeError, "code object passed to eval() may not contain free variables"); - return NULL; + goto error; } - return PyEval_EvalCode(source, globals, locals); + result = PyEval_EvalCode(source, globals, locals); } + else { + PyCompilerFlags cf = _PyCompilerFlags_INIT; + cf.cf_flags = PyCF_SOURCE_IS_UTF8; + str = _Py_SourceAsString(source, "eval", "string, bytes or code", &cf, &source_copy); + if (str == NULL) + goto error; - PyCompilerFlags cf = _PyCompilerFlags_INIT; - cf.cf_flags = PyCF_SOURCE_IS_UTF8; - str = _Py_SourceAsString(source, "eval", "string, bytes or code", &cf, &source_copy); - if (str == NULL) - return NULL; + while (*str == ' ' || *str == '\t') + str++; - while (*str == ' ' || *str == '\t') - str++; + (void)PyEval_MergeCompilerFlags(&cf); + result = PyRun_StringFlags(str, Py_eval_input, globals, locals, &cf); + Py_XDECREF(source_copy); + } - (void)PyEval_MergeCompilerFlags(&cf); - result = PyRun_StringFlags(str, Py_eval_input, globals, locals, &cf); - Py_XDECREF(source_copy); + error: + Py_XDECREF(locals); return result; } @@ -1005,36 +1015,43 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, if (globals == Py_None) { globals = PyEval_GetGlobals(); if (locals == Py_None) { - locals = PyEval_GetLocals(); + locals = _PyEval_GetFrameLocals(); if (locals == NULL) return NULL; } + else { + Py_INCREF(locals); + } if (!globals || !locals) { PyErr_SetString(PyExc_SystemError, "globals and locals cannot be NULL"); return NULL; } } - else if (locals == Py_None) - locals = globals; + else if (locals == Py_None) { + locals = Py_NewRef(globals); + } + else { + Py_INCREF(locals); + } if (!PyDict_Check(globals)) { PyErr_Format(PyExc_TypeError, "exec() globals must be a dict, not %.100s", Py_TYPE(globals)->tp_name); - return NULL; + goto error; } if (!PyMapping_Check(locals)) { PyErr_Format(PyExc_TypeError, "locals must be a mapping or None, not %.100s", Py_TYPE(locals)->tp_name); - return NULL; + goto error; } int r = PyDict_Contains(globals, &_Py_ID(__builtins__)); if (r == 0) { r = PyDict_SetItem(globals, &_Py_ID(__builtins__), PyEval_GetBuiltins()); } if (r < 0) { - return NULL; + goto error; } if (closure == Py_None) { @@ -1047,7 +1064,7 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, if (closure) { PyErr_SetString(PyExc_TypeError, "cannot use a closure with this code object"); - return NULL; + goto error; } } else { int closure_is_ok = @@ -1067,12 +1084,12 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, PyErr_Format(PyExc_TypeError, "code object requires a closure of exactly length %zd", num_free); - return NULL; + goto error; } } if (PySys_Audit("exec", "O", source) < 0) { - return NULL; + goto error; } if (!closure) { @@ -1099,7 +1116,7 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, "string, bytes or code", &cf, &source_copy); if (str == NULL) - return NULL; + goto error; if (PyEval_MergeCompilerFlags(&cf)) v = PyRun_StringFlags(str, Py_file_input, globals, locals, &cf); @@ -1108,9 +1125,14 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, Py_XDECREF(source_copy); } if (v == NULL) - return NULL; + goto error; + Py_DECREF(locals); Py_DECREF(v); Py_RETURN_NONE; + + error: + Py_XDECREF(locals); + return NULL; } @@ -1720,10 +1742,7 @@ static PyObject * builtin_locals_impl(PyObject *module) /*[clinic end generated code: output=b46c94015ce11448 input=7874018d478d5c4b]*/ { - PyObject *d; - - d = PyEval_GetLocals(); - return Py_XNewRef(d); + return _PyEval_GetFrameLocals(); } @@ -2443,7 +2462,7 @@ builtin_vars_impl(PyObject *module, PyObject *object) PyObject *d; if (object == NULL) { - d = Py_XNewRef(PyEval_GetLocals()); + d = _PyEval_GetFrameLocals(); } else { if (_PyObject_LookupAttr(object, &_Py_ID(__dict__), &d) == 0) { diff --git a/Python/ceval.c b/Python/ceval.c index 4762dfac4812a..c883a903e964f 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2312,6 +2312,19 @@ PyEval_GetLocals(void) return locals; } +PyObject * +_PyEval_GetFrameLocals(void) +{ + PyThreadState *tstate = _PyThreadState_GET(); + _PyInterpreterFrame *current_frame = _PyThreadState_GetFrame(tstate); + if (current_frame == NULL) { + _PyErr_SetString(tstate, PyExc_SystemError, "frame does not exist"); + return NULL; + } + + return _PyFrame_GetLocals(current_frame, 1); +} + PyObject * PyEval_GetGlobals(void) { From webhook-mailer at python.org Wed Jul 5 19:31:57 2023 From: webhook-mailer at python.org (carljm) Date: Wed, 05 Jul 2023 23:31:57 -0000 Subject: [Python-checkins] [3.12] gh-106292: restore checking __dict__ in cached_property.__get__ (GH-106380) (#106469) Message-ID: https://github.com/python/cpython/commit/7b615a1573b8a119e8ad27915912da745756a547 commit: 7b615a1573b8a119e8ad27915912da745756a547 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: carljm date: 2023-07-05T23:31:54Z summary: [3.12] gh-106292: restore checking __dict__ in cached_property.__get__ (GH-106380) (#106469) gh-106292: restore checking __dict__ in cached_property.__get__ (GH-106380) * gh-106292: restore checking __dict__ in cached_property.__get__ (cherry picked from commit 838406b4fc044c0b2f397c23275c69f16a76205b) Co-authored-by: Carl Meyer Co-authored-by: Dong-hee Na files: A Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst M Lib/functools.py M Lib/test/test_functools.py diff --git a/Lib/functools.py b/Lib/functools.py index 72b2103e7a554..2ae4290f98398 100644 --- a/Lib/functools.py +++ b/Lib/functools.py @@ -956,9 +956,10 @@ def __isabstractmethod__(self): ################################################################################ -### cached_property() - computed once per instance, cached as attribute +### cached_property() - property result cached as instance attribute ################################################################################ +_NOT_FOUND = object() class cached_property: def __init__(self, func): @@ -989,15 +990,17 @@ def __get__(self, instance, owner=None): f"instance to cache {self.attrname!r} property." ) raise TypeError(msg) from None - val = self.func(instance) - try: - cache[self.attrname] = val - except TypeError: - msg = ( - f"The '__dict__' attribute on {type(instance).__name__!r} instance " - f"does not support item assignment for caching {self.attrname!r} property." - ) - raise TypeError(msg) from None + val = cache.get(self.attrname, _NOT_FOUND) + if val is _NOT_FOUND: + val = self.func(instance) + try: + cache[self.attrname] = val + except TypeError: + msg = ( + f"The '__dict__' attribute on {type(instance).__name__!r} instance " + f"does not support item assignment for caching {self.attrname!r} property." + ) + raise TypeError(msg) from None return val __class_getitem__ = classmethod(GenericAlias) diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py index d668fa4c3adf5..c4eca0f5b7951 100644 --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -3037,6 +3037,25 @@ def test_access_from_class(self): def test_doc(self): self.assertEqual(CachedCostItem.cost.__doc__, "The cost of the item.") + def test_subclass_with___set__(self): + """Caching still works for a subclass defining __set__.""" + class readonly_cached_property(py_functools.cached_property): + def __set__(self, obj, value): + raise AttributeError("read only property") + + class Test: + def __init__(self, prop): + self._prop = prop + + @readonly_cached_property + def prop(self): + return self._prop + + t = Test(1) + self.assertEqual(t.prop, 1) + t._prop = 999 + self.assertEqual(t.prop, 1) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst b/Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst new file mode 100644 index 0000000000000..233509344d509 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst @@ -0,0 +1,4 @@ +Check for an instance-dict cached value in the :meth:`__get__` method of +:func:`functools.cached_property`. This better matches the pre-3.12 behavior +and improves compatibility for users subclassing +:func:`functools.cached_property` and adding a :meth:`__set__` method. From webhook-mailer at python.org Wed Jul 5 20:12:25 2023 From: webhook-mailer at python.org (carljm) Date: Thu, 06 Jul 2023 00:12:25 -0000 Subject: [Python-checkins] gh-105256: What's New note for comprehension over locals() (#106378) Message-ID: https://github.com/python/cpython/commit/13aefd175e3c04529251f175c23cb3ed88451fd0 commit: 13aefd175e3c04529251f175c23cb3ed88451fd0 branch: main author: Carl Meyer committer: carljm date: 2023-07-05T18:12:21-06:00 summary: gh-105256: What's New note for comprehension over locals() (#106378) files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 1af80ea62b392..a892f92dd2628 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -253,6 +253,12 @@ Inlining does result in a few visible behavior changes: * Calling :func:`locals` inside a comprehension now includes variables from outside the comprehension, and no longer includes the synthetic ``.0`` variable for the comprehension "argument". +* A comprehension iterating directly over ``locals()`` (e.g. ``[k for k in + locals()]``) may see "RuntimeError: dictionary changed size during iteration" + when run under tracing (e.g. code coverage measurement). This is the same + behavior already seen in e.g. ``for k in locals():``. To avoid the error, first + create a list of keys to iterate over: ``keys = list(locals()); [k for k in + keys]``. Contributed by Carl Meyer and Vladimir Matveev in :pep:`709`. From webhook-mailer at python.org Wed Jul 5 20:53:33 2023 From: webhook-mailer at python.org (carljm) Date: Thu, 06 Jul 2023 00:53:33 -0000 Subject: [Python-checkins] [3.12] gh-105256: What's New note for comprehension over locals() (GH-106378) (#106471) Message-ID: https://github.com/python/cpython/commit/1e4c2359e4b88a237ca0ac7d1d0cac8411ba0278 commit: 1e4c2359e4b88a237ca0ac7d1d0cac8411ba0278 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: carljm date: 2023-07-05T18:53:30-06:00 summary: [3.12] gh-105256: What's New note for comprehension over locals() (GH-106378) (#106471) gh-105256: What's New note for comprehension over locals() (GH-106378) (cherry picked from commit 13aefd175e3c04529251f175c23cb3ed88451fd0) Co-authored-by: Carl Meyer files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index ec13ed57a5c45..eba3e226fa372 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -253,6 +253,12 @@ Inlining does result in a few visible behavior changes: * Calling :func:`locals` inside a comprehension now includes variables from outside the comprehension, and no longer includes the synthetic ``.0`` variable for the comprehension "argument". +* A comprehension iterating directly over ``locals()`` (e.g. ``[k for k in + locals()]``) may see "RuntimeError: dictionary changed size during iteration" + when run under tracing (e.g. code coverage measurement). This is the same + behavior already seen in e.g. ``for k in locals():``. To avoid the error, first + create a list of keys to iterate over: ``keys = list(locals()); [k for k in + keys]``. Contributed by Carl Meyer and Vladimir Matveev in :pep:`709`. From webhook-mailer at python.org Thu Jul 6 01:41:08 2023 From: webhook-mailer at python.org (hugovk) Date: Thu, 06 Jul 2023 05:41:08 -0000 Subject: [Python-checkins] Doc: Add missing ref labels to exception groups/notes sections (#106465) Message-ID: https://github.com/python/cpython/commit/38aa89a52ed5194f70bbf07d699a2dd3720e2efd commit: 38aa89a52ed5194f70bbf07d699a2dd3720e2efd branch: main author: C.A.M. Gerlach committer: hugovk date: 2023-07-05T23:41:04-06:00 summary: Doc: Add missing ref labels to exception groups/notes sections (#106465) files: M Doc/library/exceptions.rst M Doc/tutorial/errors.rst diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index 9e1335436ee87..4651eddf84370 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -871,6 +871,8 @@ The following exceptions are used as warning categories; see the .. versionadded:: 3.2 +.. _lib-exception-groups: + Exception groups ---------------- diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index ca5dc3314c63b..6419ff621f1b3 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -578,6 +578,8 @@ the following pattern:: ... +.. _tut-exception-notes: + Enriching Exceptions with Notes =============================== From webhook-mailer at python.org Thu Jul 6 01:51:33 2023 From: webhook-mailer at python.org (hugovk) Date: Thu, 06 Jul 2023 05:51:33 -0000 Subject: [Python-checkins] [3.12] Doc: Add missing ref labels to exception groups/notes sections (GH-106465) (#106473) Message-ID: https://github.com/python/cpython/commit/84cda268bd2c65c99d2ff69731d83e694146de70 commit: 84cda268bd2c65c99d2ff69731d83e694146de70 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-05T23:51:30-06:00 summary: [3.12] Doc: Add missing ref labels to exception groups/notes sections (GH-106465) (#106473) Co-authored-by: C.A.M. Gerlach files: M Doc/library/exceptions.rst M Doc/tutorial/errors.rst diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index 49ede20070d8a..d54c49fff6b62 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -871,6 +871,8 @@ The following exceptions are used as warning categories; see the .. versionadded:: 3.2 +.. _lib-exception-groups: + Exception groups ---------------- diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index ca5dc3314c63b..6419ff621f1b3 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -578,6 +578,8 @@ the following pattern:: ... +.. _tut-exception-notes: + Enriching Exceptions with Notes =============================== From webhook-mailer at python.org Thu Jul 6 02:05:08 2023 From: webhook-mailer at python.org (hugovk) Date: Thu, 06 Jul 2023 06:05:08 -0000 Subject: [Python-checkins] [3.11] Doc: Add missing ref labels to exception groups/notes sections (GH-106465) (#106474) Message-ID: https://github.com/python/cpython/commit/80117dd45059a9dd8f2a2c9149df8f86608f0ca0 commit: 80117dd45059a9dd8f2a2c9149df8f86608f0ca0 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-06T00:05:04-06:00 summary: [3.11] Doc: Add missing ref labels to exception groups/notes sections (GH-106465) (#106474) Co-authored-by: C.A.M. Gerlach files: M Doc/library/exceptions.rst M Doc/tutorial/errors.rst diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index 49ede20070d8a..d54c49fff6b62 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -871,6 +871,8 @@ The following exceptions are used as warning categories; see the .. versionadded:: 3.2 +.. _lib-exception-groups: + Exception groups ---------------- diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index ca5dc3314c63b..6419ff621f1b3 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -578,6 +578,8 @@ the following pattern:: ... +.. _tut-exception-notes: + Enriching Exceptions with Notes =============================== From webhook-mailer at python.org Thu Jul 6 03:02:25 2023 From: webhook-mailer at python.org (vsajip) Date: Thu, 06 Jul 2023 07:02:25 -0000 Subject: [Python-checkins] gh-106238: Handle KeyboardInterrupt during logging._acquireLock() (GH-106239) Message-ID: https://github.com/python/cpython/commit/99b00efd5edfd5b26bf9e2a35cbfc96277fdcbb1 commit: 99b00efd5edfd5b26bf9e2a35cbfc96277fdcbb1 branch: main author: Ariel Eizenberg committer: vsajip date: 2023-07-06T08:02:22+01:00 summary: gh-106238: Handle KeyboardInterrupt during logging._acquireLock() (GH-106239) Co-authored-by: Ariel Eizenberg files: A Misc/NEWS.d/next/Library/2023-06-29-12-40-52.gh-issue-106238.VulKb9.rst M Lib/logging/__init__.py diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py index ba2ed44b09507..fe2039af0334a 100644 --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -238,7 +238,11 @@ def _acquireLock(): This should be released with _releaseLock(). """ if _lock: - _lock.acquire() + try: + _lock.acquire() + except BaseException: + _lock.release() + raise def _releaseLock(): """ diff --git a/Misc/NEWS.d/next/Library/2023-06-29-12-40-52.gh-issue-106238.VulKb9.rst b/Misc/NEWS.d/next/Library/2023-06-29-12-40-52.gh-issue-106238.VulKb9.rst new file mode 100644 index 0000000000000..52e78382fd618 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-29-12-40-52.gh-issue-106238.VulKb9.rst @@ -0,0 +1 @@ +Fix rare concurrency bug in lock acquisition by the logging package. From webhook-mailer at python.org Thu Jul 6 07:04:46 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Thu, 06 Jul 2023 11:04:46 -0000 Subject: [Python-checkins] [3.12] Clarify state of CancelledError in doc (GH-106453) (#106454) Message-ID: https://github.com/python/cpython/commit/e229225fbaa2de58b5143fea25b9d47480393f56 commit: e229225fbaa2de58b5143fea25b9d47480393f56 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 <59607654+kumaraditya303 at users.noreply.github.com> date: 2023-07-06T16:34:43+05:30 summary: [3.12] Clarify state of CancelledError in doc (GH-106453) (#106454) Clarify state of CancelledError in doc (GH-106453) This change makes it explicit that asyncio.CancelledError is not a subclass of Exception. (cherry picked from commit 12a98138083589314d3da14bc97f2d8517947437) Co-authored-by: Kristj?n Valur J?nsson files: M Doc/library/asyncio-exceptions.rst diff --git a/Doc/library/asyncio-exceptions.rst b/Doc/library/asyncio-exceptions.rst index 9250f01b8a089..7ad9103ca3fdf 100644 --- a/Doc/library/asyncio-exceptions.rst +++ b/Doc/library/asyncio-exceptions.rst @@ -31,7 +31,7 @@ Exceptions .. versionchanged:: 3.8 - :exc:`CancelledError` is now a subclass of :class:`BaseException`. + :exc:`CancelledError` is now a subclass of :class:`BaseException` rather than :class:`Exception`. .. exception:: InvalidStateError From webhook-mailer at python.org Thu Jul 6 09:23:06 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 06 Jul 2023 13:23:06 -0000 Subject: [Python-checkins] gh-104683: clinic.py: Don't needlessly reimplement `contextlib.redirect_stdout` (#106478) Message-ID: https://github.com/python/cpython/commit/d0c6ba956fca28785ad4dea6423cd44fd1124cad commit: d0c6ba956fca28785ad4dea6423cd44fd1124cad branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-06T14:23:02+01:00 summary: gh-104683: clinic.py: Don't needlessly reimplement `contextlib.redirect_stdout` (#106478) clinic.py: Don't needlessly reimplement `contextlib.redirect_stdout` files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 7ada7e9d917b3..306dca8d44dd2 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1617,19 +1617,6 @@ def render_function( return clinic.get_destination('block').dump() - - - at contextlib.contextmanager -def OverrideStdioWith(stdout): - saved_stdout = sys.stdout - sys.stdout = stdout - try: - yield - finally: - assert sys.stdout is stdout - sys.stdout = saved_stdout - - def create_regex( before: str, after: str, @@ -2331,17 +2318,14 @@ def compute_checksum( return s - - class PythonParser: def __init__(self, clinic: Clinic) -> None: pass def parse(self, block: Block) -> None: - s = io.StringIO() - with OverrideStdioWith(s): + with contextlib.redirect_stdout(io.StringIO()) as s: exec(block.input) - block.output = s.getvalue() + block.output = s.getvalue() class Module: From webhook-mailer at python.org Thu Jul 6 11:06:22 2023 From: webhook-mailer at python.org (hugovk) Date: Thu, 06 Jul 2023 15:06:22 -0000 Subject: [Python-checkins] Introduce a gate/check GHA job (#97533) Message-ID: https://github.com/python/cpython/commit/e7cd55753b00d6f7e5c8998bae73ebaa9e86398d commit: e7cd55753b00d6f7e5c8998bae73ebaa9e86398d branch: main author: Sviatoslav Sydorenko committer: hugovk date: 2023-07-06T18:06:18+03:00 summary: Introduce a gate/check GHA job (#97533) Co-authored-by: Hugo van Kemenade files: M .github/workflows/build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 34fcce445d0cd..06551b13219c2 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -519,3 +519,60 @@ jobs: run: make pythoninfo - name: Tests run: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu" + + all-required-green: # This job does nothing and is only used for the branch protection + name: All required checks pass + if: always() + + needs: + - check_source # Transitive dependency, needed to access `run_tests` value + - check-docs + - check_generated_files + - build_win32 + - build_win_amd64 + - build_macos + - build_ubuntu + - build_ubuntu_ssltests + - test_hypothesis + - build_asan + + runs-on: ubuntu-latest + + steps: + - name: Check whether the needed jobs succeeded or failed + uses: re-actors/alls-green at 05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe + with: + allowed-failures: >- + build_macos, + build_ubuntu_ssltests, + build_win32, + test_hypothesis, + allowed-skips: >- + ${{ + !fromJSON(needs.check_source.outputs.run-docs) + && ' + check-docs, + ' + || '' + }} + ${{ + needs.check_source.outputs.run_tests != 'true' + && ' + check_generated_files, + build_win32, + build_win_amd64, + build_macos, + build_ubuntu, + build_ubuntu_ssltests, + build_asan, + ' + || '' + }} + ${{ + !fromJSON(needs.check_source.outputs.run_hypothesis) + && ' + test_hypothesis, + ' + || '' + }} + jobs: ${{ toJSON(needs) }} From webhook-mailer at python.org Thu Jul 6 13:54:49 2023 From: webhook-mailer at python.org (brettcannon) Date: Thu, 06 Jul 2023 17:54:49 -0000 Subject: [Python-checkins] gh-106458: Mark `testthreadingmock.py` with `@requires_working_threading` (GH-106366) Message-ID: https://github.com/python/cpython/commit/56353b10023ff12c7c8d6288ae4bf7bdcd5d4b6c commit: 56353b10023ff12c7c8d6288ae4bf7bdcd5d4b6c branch: main author: Mario Corchero committer: brettcannon date: 2023-07-06T10:54:45-07:00 summary: gh-106458: Mark `testthreadingmock.py` with `@requires_working_threading` (GH-106366) Mark `testthreadingmock.py` with `threading_helper.requires_working_threading`. Also add longer delays to reduce the change of a race conditions on the tests that validate short timeouts. files: M Lib/test/test_unittest/testmock/testthreadingmock.py diff --git a/Lib/test/test_unittest/testmock/testthreadingmock.py b/Lib/test/test_unittest/testmock/testthreadingmock.py index 2b4dca3a4be17..c6f179490d01f 100644 --- a/Lib/test/test_unittest/testmock/testthreadingmock.py +++ b/Lib/test/test_unittest/testmock/testthreadingmock.py @@ -2,9 +2,13 @@ import unittest import concurrent.futures +from test.support import threading_helper from unittest.mock import patch, ThreadingMock, call +threading_helper.requires_working_threading(module=True) + + class Something: def method_1(self): pass @@ -133,11 +137,9 @@ def test_wait_failed_with_timeout_override(self): with patch(f"{__name__}.Something", waitable_mock): something = Something() - self.run_async(something.method_1, delay=0.1) + self.run_async(something.method_1, delay=0.5) with self.assertRaises(AssertionError): something.method_1.wait_until_called(timeout=0.05) - with self.assertRaises(AssertionError): - something.method_1.wait_until_any_call_with(timeout=0.05) def test_wait_success_called_before(self): waitable_mock = self._make_mock() @@ -163,10 +165,10 @@ def test_wait_until_any_call_with_positional(self): with patch(f"{__name__}.Something", waitable_mock): something = Something() - self.run_async(something.method_1, 1, delay=0.1) - self.run_async(something.method_1, 2, delay=0.2) - self.run_async(something.method_1, 3, delay=0.3) + self.run_async(something.method_1, 1, delay=0.2) self.assertNotIn(call(1), something.method_1.mock_calls) + self.run_async(something.method_1, 2, delay=0.5) + self.run_async(something.method_1, 3, delay=0.6) something.method_1.wait_until_any_call_with(1) something.method_1.assert_called_with(1) @@ -182,10 +184,10 @@ def test_wait_until_any_call_with_keywords(self): with patch(f"{__name__}.Something", waitable_mock): something = Something() - self.run_async(something.method_1, a=1, delay=0.1) - self.run_async(something.method_1, b=2, delay=0.2) - self.run_async(something.method_1, c=3, delay=0.3) + self.run_async(something.method_1, a=1, delay=0.2) self.assertNotIn(call(a=1), something.method_1.mock_calls) + self.run_async(something.method_1, b=2, delay=0.5) + self.run_async(something.method_1, c=3, delay=0.6) something.method_1.wait_until_any_call_with(a=1) something.method_1.assert_called_with(a=1) From webhook-mailer at python.org Thu Jul 6 14:39:57 2023 From: webhook-mailer at python.org (gvanrossum) Date: Thu, 06 Jul 2023 18:39:57 -0000 Subject: [Python-checkins] gh-104584: Fix error handling from backedge optimization (#106484) Message-ID: https://github.com/python/cpython/commit/003ba71dcbe94f0d5cb1d0c56d7f1d5a6dae56f7 commit: 003ba71dcbe94f0d5cb1d0c56d7f1d5a6dae56f7 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-06T18:39:53Z summary: gh-104584: Fix error handling from backedge optimization (#106484) When `_PyOptimizer_BackEdge` returns `NULL`, we should restore `next_instr` (and `stack_pointer`). To accomplish this we should jump to `resume_with_error` instead of just `error`. The problem this causes is subtle -- the only repro I have is in PR gh-106393, at commit d7df54b139bcc47f5ea094bfaa9824f79bc45adc. But the fix is real (as shown later in that PR). While we're at it, also improve the debug output: the offsets at which traces are identified are now measured in bytes, and always show the start offset. This makes it easier to correlate executor calls with optimizer calls, and either with `dis` output. * Issue: gh-104584 files: M Python/bytecodes.c M Python/ceval.c M Python/generated_cases.c.h M Python/optimizer.c diff --git a/Python/bytecodes.c b/Python/bytecodes.c index f69ac2beef4c2..70b52391e6bda 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2234,7 +2234,7 @@ dummy_func( frame = _PyOptimizer_BackEdge(frame, here, next_instr, stack_pointer); if (frame == NULL) { frame = cframe.current_frame; - goto error; + goto resume_with_error; } assert(frame == cframe.current_frame); here[1].cache &= ((1 << OPTIMIZER_BITS_IN_COUNTER) -1); diff --git a/Python/ceval.c b/Python/ceval.c index 6714229fd0784..0ee95bc3a3a4b 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2737,11 +2737,11 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject #endif DPRINTF(3, - "Entering _PyUopExecute for %s (%s:%d) at offset %ld\n", + "Entering _PyUopExecute for %s (%s:%d) at byte offset %ld\n", PyUnicode_AsUTF8(_PyFrame_GetCode(frame)->co_qualname), PyUnicode_AsUTF8(_PyFrame_GetCode(frame)->co_filename), _PyFrame_GetCode(frame)->co_firstlineno, - (long)(frame->prev_instr + 1 - + 2 * (long)(frame->prev_instr + 1 - (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive)); PyThreadState *tstate = _PyThreadState_GET(); diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index eb2422943984b..6000ab2da1133 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3193,7 +3193,7 @@ frame = _PyOptimizer_BackEdge(frame, here, next_instr, stack_pointer); if (frame == NULL) { frame = cframe.current_frame; - goto error; + goto resume_with_error; } assert(frame == cframe.current_frame); here[1].cache &= ((1 << OPTIMIZER_BITS_IN_COUNTER) -1); diff --git a/Python/optimizer.c b/Python/optimizer.c index c3ab649b51b0e..2c1be61f8d614 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -181,6 +181,7 @@ _PyOptimizer_BackEdge(_PyInterpreterFrame *frame, _Py_CODEUNIT *src, _Py_CODEUNI } insert_executor(code, src, index, executor); assert(frame->prev_instr == src); + frame->prev_instr = dest - 1; return executor->execute(executor, frame, stack_pointer); jump_to_destination: frame->prev_instr = dest - 1; @@ -201,7 +202,7 @@ PyUnstable_GetExecutor(PyCodeObject *code, int offset) } i += _PyInstruction_GetLength(code, i); } - PyErr_SetString(PyExc_ValueError, "no executor at given offset"); + PyErr_SetString(PyExc_ValueError, "no executor at given byte offset"); return NULL; } @@ -373,6 +374,9 @@ translate_bytecode_to_trace( _PyUOpInstruction *trace, int max_length) { +#ifdef Py_DEBUG + _Py_CODEUNIT *initial_instr = instr; +#endif int trace_length = 0; #ifdef Py_DEBUG @@ -398,11 +402,11 @@ translate_bytecode_to_trace( trace_length++; DPRINTF(4, - "Optimizing %s (%s:%d) at offset %ld\n", + "Optimizing %s (%s:%d) at byte offset %ld\n", PyUnicode_AsUTF8(code->co_qualname), PyUnicode_AsUTF8(code->co_filename), code->co_firstlineno, - (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); + 2 * (long)(initial_instr - (_Py_CODEUNIT *)code->co_code_adaptive)); for (;;) { ADD_TO_TRACE(SAVE_IP, (int)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); @@ -492,21 +496,21 @@ translate_bytecode_to_trace( if (trace_length > 3) { ADD_TO_TRACE(EXIT_TRACE, 0); DPRINTF(1, - "Created a trace for %s (%s:%d) at offset %ld -- length %d\n", + "Created a trace for %s (%s:%d) at byte offset %ld -- length %d\n", PyUnicode_AsUTF8(code->co_qualname), PyUnicode_AsUTF8(code->co_filename), code->co_firstlineno, - (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive), + 2 * (long)(initial_instr - (_Py_CODEUNIT *)code->co_code_adaptive), trace_length); return trace_length; } else { DPRINTF(4, - "No trace for %s (%s:%d) at offset %ld\n", + "No trace for %s (%s:%d) at byte offset %ld\n", PyUnicode_AsUTF8(code->co_qualname), PyUnicode_AsUTF8(code->co_filename), code->co_firstlineno, - (long)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); + 2 * (long)(initial_instr - (_Py_CODEUNIT *)code->co_code_adaptive)); } return 0; From webhook-mailer at python.org Thu Jul 6 17:17:15 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 06 Jul 2023 21:17:15 -0000 Subject: [Python-checkins] gh-104683: clinic.py: refactor four simple classes as dataclasses (#106476) Message-ID: https://github.com/python/cpython/commit/5548097925b9924ebf761376d632c5198d01ebd5 commit: 5548097925b9924ebf761376d632c5198d01ebd5 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-06T22:17:12+01:00 summary: gh-104683: clinic.py: refactor four simple classes as dataclasses (#106476) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 306dca8d44dd2..24d6255f262da 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -13,6 +13,7 @@ import contextlib import copy import cpp +import dataclasses as dc import enum import functools import hashlib @@ -1858,11 +1859,10 @@ def is_stop_line(line): return Block(input_output(), dsl_name, output=output) + at dc.dataclass(slots=True) class BlockPrinter: - - def __init__(self, language, f=None): - self.language = language - self.f = f or io.StringIO() + language: Language + f: io.StringIO = dc.field(default_factory=io.StringIO) def print_block(self, block, *, core_includes=False): input = block.input @@ -2328,15 +2328,13 @@ def parse(self, block: Block) -> None: block.output = s.getvalue() + at dc.dataclass(repr=False) class Module: - def __init__( - self, - name: str, - module = None - ) -> None: - self.name = name - self.module = self.parent = module + name: str + module: Module | None = None + def __post_init__(self) -> None: + self.parent = self.module self.modules: ModuleDict = {} self.classes: ClassDict = {} self.functions: list[Function] = [] @@ -2345,22 +2343,16 @@ def __repr__(self) -> str: return "" + at dc.dataclass(repr=False) class Class: - def __init__( - self, - name: str, - module: Module | None = None, - cls = None, - typedef: str | None = None, - type_object: str | None = None - ) -> None: - self.name = name - self.module = module - self.cls = cls - self.typedef = typedef - self.type_object = type_object - self.parent = cls or module - + name: str + module: Module | None = None + cls: Class | None = None + typedef: str | None = None + type_object: str | None = None + + def __post_init__(self) -> None: + self.parent = self.cls or self.module self.classes: ClassDict = {} self.functions: list[Function] = [] @@ -2606,13 +2598,10 @@ def get_displayname(self, i: int) -> str: return f'"argument {i}"' + at dc.dataclass class LandMine: # try to access any - def __init__(self, message: str) -> None: - self.__message__ = message - - def __repr__(self) -> str: - return '" + __message__: str def __getattribute__(self, name: str): if name in ('__repr__', '__message__'): From webhook-mailer at python.org Thu Jul 6 18:34:04 2023 From: webhook-mailer at python.org (benjaminp) Date: Thu, 06 Jul 2023 22:34:04 -0000 Subject: [Python-checkins] closes gh-106479: fix typo in __cplusplus macro (gh-106480) Message-ID: https://github.com/python/cpython/commit/67a798888dcde13bbb1e17cfcc3c742c94e67a07 commit: 67a798888dcde13bbb1e17cfcc3c742c94e67a07 branch: main author: Dustin Rodrigues committer: benjaminp date: 2023-07-06T17:34:00-05:00 summary: closes gh-106479: fix typo in __cplusplus macro (gh-106480) files: M Lib/test/_testcppext.cpp diff --git a/Lib/test/_testcppext.cpp b/Lib/test/_testcppext.cpp index 0e381a78c5cee..82b471312dd2b 100644 --- a/Lib/test/_testcppext.cpp +++ b/Lib/test/_testcppext.cpp @@ -86,7 +86,7 @@ test_api_casts(PyObject *Py_UNUSED(module), PyObject *Py_UNUSED(args)) // gh-93442: Pass 0 as NULL for PyObject* Py_XINCREF(0); Py_XDECREF(0); -#if _cplusplus >= 201103 +#if __cplusplus >= 201103 // Test nullptr passed as PyObject* Py_XINCREF(nullptr); Py_XDECREF(nullptr); From webhook-mailer at python.org Thu Jul 6 18:46:00 2023 From: webhook-mailer at python.org (gvanrossum) Date: Thu, 06 Jul 2023 22:46:00 -0000 Subject: [Python-checkins] gh-104584: Clean up and fix uops tests and fix crash (#106492) Message-ID: https://github.com/python/cpython/commit/76fac7bce55302a8e9a524d72f5384fd89e6dfde commit: 76fac7bce55302a8e9a524d72f5384fd89e6dfde branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-06T15:45:56-07:00 summary: gh-104584: Clean up and fix uops tests and fix crash (#106492) The uops test wasn't testing anything by default, and was failing when run with -Xuops. Made the two executor-related context managers global, so TestUops can use them (notably `with temporary_optimizer(opt)`). Made clear_executor() a little more thorough. Fixed a crash upon finalizing a uop optimizer, by adding a `tp_dealloc` handler. files: M Lib/test/test_capi/test_misc.py M Python/optimizer.c diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index de9f00a9e5fb4..8d597e7990255 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2343,23 +2343,26 @@ def func(): names = ["func", "outer", "outer", "inner", "inner", "outer", "inner"] self.do_test(func, names) -class TestOptimizerAPI(unittest.TestCase): - @contextlib.contextmanager - def temporary_optimizer(self, opt): - _testinternalcapi.set_optimizer(opt) - try: - yield - finally: - _testinternalcapi.set_optimizer(None) + at contextlib.contextmanager +def temporary_optimizer(opt): + _testinternalcapi.set_optimizer(opt) + try: + yield + finally: + _testinternalcapi.set_optimizer(None) - @contextlib.contextmanager - def clear_executors(self, func): - try: - yield - finally: - #Clear executors - func.__code__ = func.__code__.replace() + at contextlib.contextmanager +def clear_executors(func): + # Clear executors in func before and after running a block + func.__code__ = func.__code__.replace() + try: + yield + finally: + func.__code__ = func.__code__.replace() + + +class TestOptimizerAPI(unittest.TestCase): def test_get_set_optimizer(self): self.assertEqual(_testinternalcapi.get_optimizer(), None) @@ -2381,9 +2384,9 @@ def loop(): for repeat in range(5): opt = _testinternalcapi.get_counter_optimizer() - with self.temporary_optimizer(opt): + with temporary_optimizer(opt): self.assertEqual(opt.get_count(), 0) - with self.clear_executors(loop): + with clear_executors(loop): loop() self.assertEqual(opt.get_count(), 1000) @@ -2409,7 +2412,7 @@ def long_loop(): long_loop = ns['long_loop'] opt = _testinternalcapi.get_counter_optimizer() - with self.temporary_optimizer(opt): + with temporary_optimizer(opt): self.assertEqual(opt.get_count(), 0) long_loop() self.assertEqual(opt.get_count(), 10) @@ -2418,24 +2421,27 @@ def long_loop(): class TestUops(unittest.TestCase): def test_basic_loop(self): - def testfunc(x): i = 0 while i < x: i += 1 - testfunc(1000) + opt = _testinternalcapi.get_uop_optimizer() + + with temporary_optimizer(opt): + testfunc(1000) ex = None - for offset in range(0, 100, 2): + for offset in range(0, len(testfunc.__code__.co_code), 2): try: ex = _testinternalcapi.get_executor(testfunc.__code__, offset) break except ValueError: pass - if ex is None: - return - self.assertIn("SAVE_IP", str(ex)) + self.assertIsNotNone(ex) + uops = {opname for opname, _ in ex} + self.assertIn("SAVE_IP", uops) + self.assertIn("LOAD_FAST", uops) if __name__ == "__main__": diff --git a/Python/optimizer.c b/Python/optimizer.c index 2c1be61f8d614..d2fdca59fe46c 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -532,7 +532,7 @@ uop_optimize( return trace_length; } OBJECT_STAT_INC(optimization_traces_created); - _PyUOpExecutorObject *executor = (_PyUOpExecutorObject *)_PyObject_New(&UOpExecutor_Type); + _PyUOpExecutorObject *executor = PyObject_New(_PyUOpExecutorObject, &UOpExecutor_Type); if (executor == NULL) { return -1; } @@ -542,18 +542,24 @@ uop_optimize( return 1; } +static void +uop_opt_dealloc(PyObject *self) { + PyObject_Free(self); +} + static PyTypeObject UOpOptimizer_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) .tp_name = "uop_optimizer", .tp_basicsize = sizeof(_PyOptimizerObject), .tp_itemsize = 0, .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_DISALLOW_INSTANTIATION, + .tp_dealloc = uop_opt_dealloc, }; PyObject * PyUnstable_Optimizer_NewUOpOptimizer(void) { - _PyOptimizerObject *opt = (_PyOptimizerObject *)_PyObject_New(&UOpOptimizer_Type); + _PyOptimizerObject *opt = PyObject_New(_PyOptimizerObject, &UOpOptimizer_Type); if (opt == NULL) { return NULL; } From webhook-mailer at python.org Thu Jul 6 18:46:53 2023 From: webhook-mailer at python.org (gpshead) Date: Thu, 06 Jul 2023 22:46:53 -0000 Subject: [Python-checkins] gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (#106464) Message-ID: https://github.com/python/cpython/commit/c60df361ce2d734148d503f4a711e67c110fe223 commit: c60df361ce2d734148d503f4a711e67c110fe223 branch: main author: Gregory P. Smith committer: gpshead date: 2023-07-06T22:46:50Z summary: gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (#106464) Prevent `multiprocessing.spawn` from failing to *import* in environments where `sys.executable` is `None`. This regressed in 3.11 with the addition of support for path-like objects in multiprocessing. Adds a test decorator to have tests only run when part of test_multiprocessing_spawn to `_test_multiprocessing.py` so we can start to avoid re-running the same not-global-state specific test in all 3 modes when there is no need. files: A Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst M Lib/multiprocessing/spawn.py M Lib/test/_test_multiprocessing.py diff --git a/Lib/multiprocessing/spawn.py b/Lib/multiprocessing/spawn.py index 09f8a229d7ccc..f1af770910471 100644 --- a/Lib/multiprocessing/spawn.py +++ b/Lib/multiprocessing/spawn.py @@ -31,11 +31,13 @@ WINSERVICE = False else: WINEXE = getattr(sys, 'frozen', False) - WINSERVICE = sys.executable.lower().endswith("pythonservice.exe") + WINSERVICE = sys.executable and sys.executable.lower().endswith("pythonservice.exe") def set_executable(exe): global _python_exe - if sys.platform == 'win32': + if exe is None: + _python_exe = exe + elif sys.platform == 'win32': _python_exe = os.fsdecode(exe) else: _python_exe = os.fsencode(exe) diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index c101fe980ceed..c1f9487ae8051 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -13,6 +13,7 @@ import os import gc import errno +import functools import signal import array import socket @@ -31,6 +32,7 @@ from test.support import hashlib_helper from test.support import import_helper from test.support import os_helper +from test.support import script_helper from test.support import socket_helper from test.support import threading_helper from test.support import warnings_helper @@ -171,6 +173,59 @@ def check_enough_semaphores(): "to run the test (required: %d)." % nsems_min) +def only_run_in_spawn_testsuite(reason): + """Returns a decorator: raises SkipTest when SM != spawn at test time. + + This can be useful to save overall Python test suite execution time. + "spawn" is the universal mode available on all platforms so this limits the + decorated test to only execute within test_multiprocessing_spawn. + + This would not be necessary if we refactored our test suite to split things + into other test files when they are not start method specific to be rerun + under all start methods. + """ + + def decorator(test_item): + + @functools.wraps(test_item) + def spawn_check_wrapper(*args, **kwargs): + if (start_method := multiprocessing.get_start_method()) != "spawn": + raise unittest.SkipTest(f"{start_method=}, not 'spawn'; {reason}") + return test_item(*args, **kwargs) + + return spawn_check_wrapper + + return decorator + + +class TestInternalDecorators(unittest.TestCase): + """Logic within a test suite that could errantly skip tests? Test it!""" + + @unittest.skipIf(sys.platform == "win32", "test requires that fork exists.") + def test_only_run_in_spawn_testsuite(self): + if multiprocessing.get_start_method() != "spawn": + raise unittest.SkipTest("only run in test_multiprocessing_spawn.") + + try: + @only_run_in_spawn_testsuite("testing this decorator") + def return_four_if_spawn(): + return 4 + except Exception as err: + self.fail(f"expected decorated `def` not to raise; caught {err}") + + orig_start_method = multiprocessing.get_start_method(allow_none=True) + try: + multiprocessing.set_start_method("spawn", force=True) + self.assertEqual(return_four_if_spawn(), 4) + multiprocessing.set_start_method("fork", force=True) + with self.assertRaises(unittest.SkipTest) as ctx: + return_four_if_spawn() + self.assertIn("testing this decorator", str(ctx.exception)) + self.assertIn("start_method=", str(ctx.exception)) + finally: + multiprocessing.set_start_method(orig_start_method, force=True) + + # # Creates a wrapper for a function which records the time it takes to finish # @@ -5815,6 +5870,7 @@ def test_namespace(self): class TestNamedResource(unittest.TestCase): + @only_run_in_spawn_testsuite("spawn specific test.") def test_global_named_resource_spawn(self): # # gh-90549: Check that global named resources in main module @@ -5825,22 +5881,18 @@ def test_global_named_resource_spawn(self): with open(testfn, 'w', encoding='utf-8') as f: f.write(textwrap.dedent('''\ import multiprocessing as mp - ctx = mp.get_context('spawn') - global_resource = ctx.Semaphore() - def submain(): pass - if __name__ == '__main__': p = ctx.Process(target=submain) p.start() p.join() ''')) - rc, out, err = test.support.script_helper.assert_python_ok(testfn) + rc, out, err = script_helper.assert_python_ok(testfn) # on error, err = 'UserWarning: resource_tracker: There appear to # be 1 leaked semaphore objects to clean up at shutdown' - self.assertEqual(err, b'') + self.assertFalse(err, msg=err.decode('utf-8')) class MiscTestCase(unittest.TestCase): @@ -5849,6 +5901,24 @@ def test__all__(self): support.check__all__(self, multiprocessing, extra=multiprocessing.__all__, not_exported=['SUBDEBUG', 'SUBWARNING']) + @only_run_in_spawn_testsuite("avoids redundant testing.") + def test_spawn_sys_executable_none_allows_import(self): + # Regression test for a bug introduced in + # https://github.com/python/cpython/issues/90876 that caused an + # ImportError in multiprocessing when sys.executable was None. + # This can be true in embedded environments. + rc, out, err = script_helper.assert_python_ok( + "-c", + """if 1: + import sys + sys.executable = None + assert "multiprocessing" not in sys.modules, "already imported!" + import multiprocessing + import multiprocessing.spawn # This should not fail\n""", + ) + self.assertEqual(rc, 0) + self.assertFalse(err, msg=err.decode('utf-8')) + # # Mixins diff --git a/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst b/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst new file mode 100644 index 0000000000000..3e062b5add6d8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst @@ -0,0 +1,3 @@ +Prevent :mod:`multiprocessing.spawn` from failing to *import* in environments +where ``sys.executable`` is ``None``. This regressed in 3.11 with the addition +of support for path-like objects in multiprocessing. From webhook-mailer at python.org Thu Jul 6 19:25:09 2023 From: webhook-mailer at python.org (gpshead) Date: Thu, 06 Jul 2023 23:25:09 -0000 Subject: [Python-checkins] [3.12] gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (GH-106464) (#106494) Message-ID: https://github.com/python/cpython/commit/4787eaed06cf83ed53270c61ac915dc0b15b20e8 commit: 4787eaed06cf83ed53270c61ac915dc0b15b20e8 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gpshead date: 2023-07-06T16:25:06-07:00 summary: [3.12] gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (GH-106464) (#106494) gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (GH-106464) Prevent `multiprocessing.spawn` from failing to *import* in environments where `sys.executable` is `None`. This regressed in 3.11 with the addition of support for path-like objects in multiprocessing. Adds a test decorator to have tests only run when part of test_multiprocessing_spawn to `_test_multiprocessing.py` so we can start to avoid re-running the same not-global-state specific test in all 3 modes when there is no need. (cherry picked from commit c60df361ce2d734148d503f4a711e67c110fe223) Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst M Lib/multiprocessing/spawn.py M Lib/test/_test_multiprocessing.py diff --git a/Lib/multiprocessing/spawn.py b/Lib/multiprocessing/spawn.py index 09f8a229d7ccc..f1af770910471 100644 --- a/Lib/multiprocessing/spawn.py +++ b/Lib/multiprocessing/spawn.py @@ -31,11 +31,13 @@ WINSERVICE = False else: WINEXE = getattr(sys, 'frozen', False) - WINSERVICE = sys.executable.lower().endswith("pythonservice.exe") + WINSERVICE = sys.executable and sys.executable.lower().endswith("pythonservice.exe") def set_executable(exe): global _python_exe - if sys.platform == 'win32': + if exe is None: + _python_exe = exe + elif sys.platform == 'win32': _python_exe = os.fsdecode(exe) else: _python_exe = os.fsencode(exe) diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index 767f049b0d717..c13b049645ac9 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -13,6 +13,7 @@ import os import gc import errno +import functools import signal import array import socket @@ -31,6 +32,7 @@ from test.support import hashlib_helper from test.support import import_helper from test.support import os_helper +from test.support import script_helper from test.support import socket_helper from test.support import threading_helper from test.support import warnings_helper @@ -171,6 +173,59 @@ def check_enough_semaphores(): "to run the test (required: %d)." % nsems_min) +def only_run_in_spawn_testsuite(reason): + """Returns a decorator: raises SkipTest when SM != spawn at test time. + + This can be useful to save overall Python test suite execution time. + "spawn" is the universal mode available on all platforms so this limits the + decorated test to only execute within test_multiprocessing_spawn. + + This would not be necessary if we refactored our test suite to split things + into other test files when they are not start method specific to be rerun + under all start methods. + """ + + def decorator(test_item): + + @functools.wraps(test_item) + def spawn_check_wrapper(*args, **kwargs): + if (start_method := multiprocessing.get_start_method()) != "spawn": + raise unittest.SkipTest(f"{start_method=}, not 'spawn'; {reason}") + return test_item(*args, **kwargs) + + return spawn_check_wrapper + + return decorator + + +class TestInternalDecorators(unittest.TestCase): + """Logic within a test suite that could errantly skip tests? Test it!""" + + @unittest.skipIf(sys.platform == "win32", "test requires that fork exists.") + def test_only_run_in_spawn_testsuite(self): + if multiprocessing.get_start_method() != "spawn": + raise unittest.SkipTest("only run in test_multiprocessing_spawn.") + + try: + @only_run_in_spawn_testsuite("testing this decorator") + def return_four_if_spawn(): + return 4 + except Exception as err: + self.fail(f"expected decorated `def` not to raise; caught {err}") + + orig_start_method = multiprocessing.get_start_method(allow_none=True) + try: + multiprocessing.set_start_method("spawn", force=True) + self.assertEqual(return_four_if_spawn(), 4) + multiprocessing.set_start_method("fork", force=True) + with self.assertRaises(unittest.SkipTest) as ctx: + return_four_if_spawn() + self.assertIn("testing this decorator", str(ctx.exception)) + self.assertIn("start_method=", str(ctx.exception)) + finally: + multiprocessing.set_start_method(orig_start_method, force=True) + + # # Creates a wrapper for a function which records the time it takes to finish # @@ -5815,6 +5870,7 @@ def test_namespace(self): class TestNamedResource(unittest.TestCase): + @only_run_in_spawn_testsuite("spawn specific test.") def test_global_named_resource_spawn(self): # # gh-90549: Check that global named resources in main module @@ -5825,22 +5881,18 @@ def test_global_named_resource_spawn(self): with open(testfn, 'w', encoding='utf-8') as f: f.write(textwrap.dedent('''\ import multiprocessing as mp - ctx = mp.get_context('spawn') - global_resource = ctx.Semaphore() - def submain(): pass - if __name__ == '__main__': p = ctx.Process(target=submain) p.start() p.join() ''')) - rc, out, err = test.support.script_helper.assert_python_ok(testfn) + rc, out, err = script_helper.assert_python_ok(testfn) # on error, err = 'UserWarning: resource_tracker: There appear to # be 1 leaked semaphore objects to clean up at shutdown' - self.assertEqual(err, b'') + self.assertFalse(err, msg=err.decode('utf-8')) class MiscTestCase(unittest.TestCase): @@ -5849,6 +5901,24 @@ def test__all__(self): support.check__all__(self, multiprocessing, extra=multiprocessing.__all__, not_exported=['SUBDEBUG', 'SUBWARNING']) + @only_run_in_spawn_testsuite("avoids redundant testing.") + def test_spawn_sys_executable_none_allows_import(self): + # Regression test for a bug introduced in + # https://github.com/python/cpython/issues/90876 that caused an + # ImportError in multiprocessing when sys.executable was None. + # This can be true in embedded environments. + rc, out, err = script_helper.assert_python_ok( + "-c", + """if 1: + import sys + sys.executable = None + assert "multiprocessing" not in sys.modules, "already imported!" + import multiprocessing + import multiprocessing.spawn # This should not fail\n""", + ) + self.assertEqual(rc, 0) + self.assertFalse(err, msg=err.decode('utf-8')) + # # Mixins diff --git a/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst b/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst new file mode 100644 index 0000000000000..3e062b5add6d8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst @@ -0,0 +1,3 @@ +Prevent :mod:`multiprocessing.spawn` from failing to *import* in environments +where ``sys.executable`` is ``None``. This regressed in 3.11 with the addition +of support for path-like objects in multiprocessing. From webhook-mailer at python.org Thu Jul 6 19:25:24 2023 From: webhook-mailer at python.org (gpshead) Date: Thu, 06 Jul 2023 23:25:24 -0000 Subject: [Python-checkins] [3.11] gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (GH-106464) (#106495) Message-ID: https://github.com/python/cpython/commit/1931c2a438c50e6250725c84dff94fc760b9b951 commit: 1931c2a438c50e6250725c84dff94fc760b9b951 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gpshead date: 2023-07-06T16:25:20-07:00 summary: [3.11] gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (GH-106464) (#106495) gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (GH-106464) Prevent `multiprocessing.spawn` from failing to *import* in environments where `sys.executable` is `None`. This regressed in 3.11 with the addition of support for path-like objects in multiprocessing. Adds a test decorator to have tests only run when part of test_multiprocessing_spawn to `_test_multiprocessing.py` so we can start to avoid re-running the same not-global-state specific test in all 3 modes when there is no need. (cherry picked from commit c60df361ce2d734148d503f4a711e67c110fe223) Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst M Lib/multiprocessing/spawn.py M Lib/test/_test_multiprocessing.py diff --git a/Lib/multiprocessing/spawn.py b/Lib/multiprocessing/spawn.py index 09f8a229d7ccc..f1af770910471 100644 --- a/Lib/multiprocessing/spawn.py +++ b/Lib/multiprocessing/spawn.py @@ -31,11 +31,13 @@ WINSERVICE = False else: WINEXE = getattr(sys, 'frozen', False) - WINSERVICE = sys.executable.lower().endswith("pythonservice.exe") + WINSERVICE = sys.executable and sys.executable.lower().endswith("pythonservice.exe") def set_executable(exe): global _python_exe - if sys.platform == 'win32': + if exe is None: + _python_exe = exe + elif sys.platform == 'win32': _python_exe = os.fsdecode(exe) else: _python_exe = os.fsencode(exe) diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index b50a1543205ce..aa302c21000dd 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -13,6 +13,7 @@ import os import gc import errno +import functools import signal import array import socket @@ -31,6 +32,7 @@ from test.support import hashlib_helper from test.support import import_helper from test.support import os_helper +from test.support import script_helper from test.support import socket_helper from test.support import threading_helper from test.support import warnings_helper @@ -170,6 +172,59 @@ def check_enough_semaphores(): "to run the test (required: %d)." % nsems_min) +def only_run_in_spawn_testsuite(reason): + """Returns a decorator: raises SkipTest when SM != spawn at test time. + + This can be useful to save overall Python test suite execution time. + "spawn" is the universal mode available on all platforms so this limits the + decorated test to only execute within test_multiprocessing_spawn. + + This would not be necessary if we refactored our test suite to split things + into other test files when they are not start method specific to be rerun + under all start methods. + """ + + def decorator(test_item): + + @functools.wraps(test_item) + def spawn_check_wrapper(*args, **kwargs): + if (start_method := multiprocessing.get_start_method()) != "spawn": + raise unittest.SkipTest(f"{start_method=}, not 'spawn'; {reason}") + return test_item(*args, **kwargs) + + return spawn_check_wrapper + + return decorator + + +class TestInternalDecorators(unittest.TestCase): + """Logic within a test suite that could errantly skip tests? Test it!""" + + @unittest.skipIf(sys.platform == "win32", "test requires that fork exists.") + def test_only_run_in_spawn_testsuite(self): + if multiprocessing.get_start_method() != "spawn": + raise unittest.SkipTest("only run in test_multiprocessing_spawn.") + + try: + @only_run_in_spawn_testsuite("testing this decorator") + def return_four_if_spawn(): + return 4 + except Exception as err: + self.fail(f"expected decorated `def` not to raise; caught {err}") + + orig_start_method = multiprocessing.get_start_method(allow_none=True) + try: + multiprocessing.set_start_method("spawn", force=True) + self.assertEqual(return_four_if_spawn(), 4) + multiprocessing.set_start_method("fork", force=True) + with self.assertRaises(unittest.SkipTest) as ctx: + return_four_if_spawn() + self.assertIn("testing this decorator", str(ctx.exception)) + self.assertIn("start_method=", str(ctx.exception)) + finally: + multiprocessing.set_start_method(orig_start_method, force=True) + + # # Creates a wrapper for a function which records the time it takes to finish # @@ -5775,6 +5830,7 @@ def test_namespace(self): class TestNamedResource(unittest.TestCase): + @only_run_in_spawn_testsuite("spawn specific test.") def test_global_named_resource_spawn(self): # # gh-90549: Check that global named resources in main module @@ -5785,22 +5841,18 @@ def test_global_named_resource_spawn(self): with open(testfn, 'w', encoding='utf-8') as f: f.write(textwrap.dedent('''\ import multiprocessing as mp - ctx = mp.get_context('spawn') - global_resource = ctx.Semaphore() - def submain(): pass - if __name__ == '__main__': p = ctx.Process(target=submain) p.start() p.join() ''')) - rc, out, err = test.support.script_helper.assert_python_ok(testfn) + rc, out, err = script_helper.assert_python_ok(testfn) # on error, err = 'UserWarning: resource_tracker: There appear to # be 1 leaked semaphore objects to clean up at shutdown' - self.assertEqual(err, b'') + self.assertFalse(err, msg=err.decode('utf-8')) class MiscTestCase(unittest.TestCase): @@ -5809,6 +5861,24 @@ def test__all__(self): support.check__all__(self, multiprocessing, extra=multiprocessing.__all__, not_exported=['SUBDEBUG', 'SUBWARNING']) + @only_run_in_spawn_testsuite("avoids redundant testing.") + def test_spawn_sys_executable_none_allows_import(self): + # Regression test for a bug introduced in + # https://github.com/python/cpython/issues/90876 that caused an + # ImportError in multiprocessing when sys.executable was None. + # This can be true in embedded environments. + rc, out, err = script_helper.assert_python_ok( + "-c", + """if 1: + import sys + sys.executable = None + assert "multiprocessing" not in sys.modules, "already imported!" + import multiprocessing + import multiprocessing.spawn # This should not fail\n""", + ) + self.assertEqual(rc, 0) + self.assertFalse(err, msg=err.decode('utf-8')) + # # Mixins diff --git a/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst b/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst new file mode 100644 index 0000000000000..3e062b5add6d8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst @@ -0,0 +1,3 @@ +Prevent :mod:`multiprocessing.spawn` from failing to *import* in environments +where ``sys.executable`` is ``None``. This regressed in 3.11 with the addition +of support for path-like objects in multiprocessing. From webhook-mailer at python.org Thu Jul 6 19:30:36 2023 From: webhook-mailer at python.org (benjaminp) Date: Thu, 06 Jul 2023 23:30:36 -0000 Subject: [Python-checkins] [3.12] closes gh-106479: fix typo in __cplusplus macro (gh-106480) (#106493) Message-ID: https://github.com/python/cpython/commit/5293e0108986eadb7e1f5c51e126673f4196ca79 commit: 5293e0108986eadb7e1f5c51e126673f4196ca79 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: benjaminp date: 2023-07-06T18:30:32-05:00 summary: [3.12] closes gh-106479: fix typo in __cplusplus macro (gh-106480) (#106493) closes gh-106479: fix typo in __cplusplus macro (gh-106480) (cherry picked from commit 67a798888dcde13bbb1e17cfcc3c742c94e67a07) Co-authored-by: Dustin Rodrigues files: M Lib/test/_testcppext.cpp diff --git a/Lib/test/_testcppext.cpp b/Lib/test/_testcppext.cpp index 0e381a78c5cee..82b471312dd2b 100644 --- a/Lib/test/_testcppext.cpp +++ b/Lib/test/_testcppext.cpp @@ -86,7 +86,7 @@ test_api_casts(PyObject *Py_UNUSED(module), PyObject *Py_UNUSED(args)) // gh-93442: Pass 0 as NULL for PyObject* Py_XINCREF(0); Py_XDECREF(0); -#if _cplusplus >= 201103 +#if __cplusplus >= 201103 // Test nullptr passed as PyObject* Py_XINCREF(nullptr); Py_XDECREF(nullptr); From webhook-mailer at python.org Thu Jul 6 19:46:09 2023 From: webhook-mailer at python.org (gvanrossum) Date: Thu, 06 Jul 2023 23:46:09 -0000 Subject: [Python-checkins] gh-104584: Handle EXTENDED_ARG in superblock creation (#106489) Message-ID: https://github.com/python/cpython/commit/e1d45b8ed43e1590862319fec33539f8adbc0849 commit: e1d45b8ed43e1590862319fec33539f8adbc0849 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-06T16:46:06-07:00 summary: gh-104584: Handle EXTENDED_ARG in superblock creation (#106489) With test. files: M Lib/test/test_capi/test_misc.py M Python/optimizer.c diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 8d597e7990255..181e6b8077f9f 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -7,6 +7,7 @@ import importlib.machinery import importlib.util import json +import opcode import os import pickle import queue @@ -2352,6 +2353,7 @@ def temporary_optimizer(opt): finally: _testinternalcapi.set_optimizer(None) + @contextlib.contextmanager def clear_executors(func): # Clear executors in func before and after running a block @@ -2418,6 +2420,19 @@ def long_loop(): self.assertEqual(opt.get_count(), 10) + +def get_first_executor(code): + co_code = code.co_code + JUMP_BACKWARD = opcode.opmap["JUMP_BACKWARD"] + for i in range(0, len(co_code), 2): + if co_code[i] == JUMP_BACKWARD or 1: + try: + return _testinternalcapi.get_executor(code, i) + except ValueError: + pass + return None + + class TestUops(unittest.TestCase): def test_basic_loop(self): @@ -2443,6 +2458,47 @@ def testfunc(x): self.assertIn("SAVE_IP", uops) self.assertIn("LOAD_FAST", uops) + def test_extended_arg(self): + "Check EXTENDED_ARG handling in superblock creation" + def many_vars(): + # 260 vars, so z9 should have index 259 + a0 = a1 = a2 = a3 = a4 = a5 = a6 = a7 = a8 = a9 = 42 + b0 = b1 = b2 = b3 = b4 = b5 = b6 = b7 = b8 = b9 = 42 + c0 = c1 = c2 = c3 = c4 = c5 = c6 = c7 = c8 = c9 = 42 + d0 = d1 = d2 = d3 = d4 = d5 = d6 = d7 = d8 = d9 = 42 + e0 = e1 = e2 = e3 = e4 = e5 = e6 = e7 = e8 = e9 = 42 + f0 = f1 = f2 = f3 = f4 = f5 = f6 = f7 = f8 = f9 = 42 + g0 = g1 = g2 = g3 = g4 = g5 = g6 = g7 = g8 = g9 = 42 + h0 = h1 = h2 = h3 = h4 = h5 = h6 = h7 = h8 = h9 = 42 + i0 = i1 = i2 = i3 = i4 = i5 = i6 = i7 = i8 = i9 = 42 + j0 = j1 = j2 = j3 = j4 = j5 = j6 = j7 = j8 = j9 = 42 + k0 = k1 = k2 = k3 = k4 = k5 = k6 = k7 = k8 = k9 = 42 + l0 = l1 = l2 = l3 = l4 = l5 = l6 = l7 = l8 = l9 = 42 + m0 = m1 = m2 = m3 = m4 = m5 = m6 = m7 = m8 = m9 = 42 + n0 = n1 = n2 = n3 = n4 = n5 = n6 = n7 = n8 = n9 = 42 + o0 = o1 = o2 = o3 = o4 = o5 = o6 = o7 = o8 = o9 = 42 + p0 = p1 = p2 = p3 = p4 = p5 = p6 = p7 = p8 = p9 = 42 + q0 = q1 = q2 = q3 = q4 = q5 = q6 = q7 = q8 = q9 = 42 + r0 = r1 = r2 = r3 = r4 = r5 = r6 = r7 = r8 = r9 = 42 + s0 = s1 = s2 = s3 = s4 = s5 = s6 = s7 = s8 = s9 = 42 + t0 = t1 = t2 = t3 = t4 = t5 = t6 = t7 = t8 = t9 = 42 + u0 = u1 = u2 = u3 = u4 = u5 = u6 = u7 = u8 = u9 = 42 + v0 = v1 = v2 = v3 = v4 = v5 = v6 = v7 = v8 = v9 = 42 + w0 = w1 = w2 = w3 = w4 = w5 = w6 = w7 = w8 = w9 = 42 + x0 = x1 = x2 = x3 = x4 = x5 = x6 = x7 = x8 = x9 = 42 + y0 = y1 = y2 = y3 = y4 = y5 = y6 = y7 = y8 = y9 = 42 + z0 = z1 = z2 = z3 = z4 = z5 = z6 = z7 = z8 = z9 = 42 + while z9 > 0: + z9 = z9 - 1 + + opt = _testinternalcapi.get_uop_optimizer() + with temporary_optimizer(opt): + ex = get_first_executor(many_vars.__code__) + self.assertIsNone(ex) + many_vars() + ex = get_first_executor(many_vars.__code__) + self.assertIn(("LOAD_FAST", 259), list(ex)) + if __name__ == "__main__": unittest.main() diff --git a/Python/optimizer.c b/Python/optimizer.c index d2fdca59fe46c..db117bb180c1c 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -412,6 +412,13 @@ translate_bytecode_to_trace( ADD_TO_TRACE(SAVE_IP, (int)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); int opcode = instr->op.code; uint64_t operand = instr->op.arg; + int extras = 0; + while (opcode == EXTENDED_ARG) { + instr++; + extras += 1; + opcode = instr->op.code; + operand = (operand << 8) | instr->op.arg; + } switch (opcode) { case LOAD_FAST_LOAD_FAST: case STORE_FAST_LOAD_FAST: @@ -458,6 +465,15 @@ translate_bytecode_to_trace( int offset = expansion->uops[i].offset; switch (expansion->uops[i].size) { case 0: + if (extras && OPCODE_HAS_JUMP(opcode)) { + if (opcode == JUMP_BACKWARD_NO_INTERRUPT) { + operand -= extras; + } + else { + assert(opcode != JUMP_BACKWARD); + operand += extras; + } + } break; case 1: operand = read_u16(&instr[offset].cache); From webhook-mailer at python.org Fri Jul 7 06:09:30 2023 From: webhook-mailer at python.org (markshannon) Date: Fri, 07 Jul 2023 10:09:30 -0000 Subject: [Python-checkins] GH-106057: Handle recursion errors in inline class calls properly. (GH-106108) Message-ID: https://github.com/python/cpython/commit/24fb627ea7a4d57cf479b7516bafdb6c253a1645 commit: 24fb627ea7a4d57cf479b7516bafdb6c253a1645 branch: main author: Mark Shannon committer: markshannon date: 2023-07-07T11:09:26+01:00 summary: GH-106057: Handle recursion errors in inline class calls properly. (GH-106108) files: M Lib/test/test_class.py M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h diff --git a/Lib/test/test_class.py b/Lib/test/test_class.py index d7a48e55b1018..894e0ca67deab 100644 --- a/Lib/test/test_class.py +++ b/Lib/test/test_class.py @@ -740,6 +740,21 @@ class A(0, 1, 2, 3, 4, 5, 6, 7, **d): pass class A(0, *range(1, 8), **d, foo='bar'): pass self.assertEqual(A, (tuple(range(8)), {'foo': 'bar'})) + def testClassCallRecursionLimit(self): + class C: + def __init__(self): + self.c = C() + + with self.assertRaises(RecursionError): + C() + + def add_one_level(): + #Each call to C() consumes 2 levels, so offset by 1. + C() + + with self.assertRaises(RecursionError): + add_one_level() + if __name__ == '__main__': unittest.main() diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 70b52391e6bda..89b077ac42834 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2985,9 +2985,6 @@ dummy_func( goto error; } Py_DECREF(tp); - if (_Py_EnterRecursivePy(tstate)) { - goto exit_unwind; - } _PyInterpreterFrame *shim = _PyFrame_PushTrampolineUnchecked( tstate, (PyCodeObject *)&_Py_InitCleanup, 1, 0); assert(_PyCode_CODE((PyCodeObject *)shim->f_executable)[1].op.code == EXIT_INIT_CHECK); @@ -3011,6 +3008,10 @@ dummy_func( shim->previous = frame; frame = cframe.current_frame = init_frame; CALL_STAT_INC(inlined_py_calls); + /* Account for pushing the extra frame. + * We don't check recursion depth here, + * as it will be checked after start_frame */ + tstate->py_recursion_remaining--; goto start_frame; } diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 39a4490e51c24..29ebb0b3e8b2d 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1811,7 +1811,7 @@ case EXIT_INIT_CHECK: { PyObject *should_be_none = stack_pointer[-1]; - #line 3018 "Python/bytecodes.c" + #line 3019 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -1827,7 +1827,7 @@ case MAKE_FUNCTION: { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3432 "Python/bytecodes.c" + #line 3433 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -1847,7 +1847,7 @@ case SET_FUNCTION_ATTRIBUTE: { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3446 "Python/bytecodes.c" + #line 3447 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -1883,13 +1883,13 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3496 "Python/bytecodes.c" + #line 3497 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); #line 1889 "Python/executor_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3498 "Python/bytecodes.c" + #line 3499 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } #line 1895 "Python/executor_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); @@ -1901,7 +1901,7 @@ case CONVERT_VALUE: { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3502 "Python/bytecodes.c" + #line 3503 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; @@ -1916,7 +1916,7 @@ case FORMAT_SIMPLE: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3511 "Python/bytecodes.c" + #line 3512 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -1936,7 +1936,7 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3524 "Python/bytecodes.c" + #line 3525 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); @@ -1950,7 +1950,7 @@ case COPY: { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3531 "Python/bytecodes.c" + #line 3532 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); #line 1957 "Python/executor_cases.c.h" @@ -1962,7 +1962,7 @@ case SWAP: { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3556 "Python/bytecodes.c" + #line 3557 "Python/bytecodes.c" assert(oparg >= 2); #line 1968 "Python/executor_cases.c.h" stack_pointer[-1] = bottom; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 6000ab2da1133..eb8b50e5c905d 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -4147,9 +4147,6 @@ goto error; } Py_DECREF(tp); - if (_Py_EnterRecursivePy(tstate)) { - goto exit_unwind; - } _PyInterpreterFrame *shim = _PyFrame_PushTrampolineUnchecked( tstate, (PyCodeObject *)&_Py_InitCleanup, 1, 0); assert(_PyCode_CODE((PyCodeObject *)shim->f_executable)[1].op.code == EXIT_INIT_CHECK); @@ -4173,13 +4170,17 @@ shim->previous = frame; frame = cframe.current_frame = init_frame; CALL_STAT_INC(inlined_py_calls); + /* Account for pushing the extra frame. + * We don't check recursion depth here, + * as it will be checked after start_frame */ + tstate->py_recursion_remaining--; goto start_frame; - #line 4178 "Python/generated_cases.c.h" + #line 4179 "Python/generated_cases.c.h" } TARGET(EXIT_INIT_CHECK) { PyObject *should_be_none = stack_pointer[-1]; - #line 3018 "Python/bytecodes.c" + #line 3019 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -4187,7 +4188,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 4191 "Python/generated_cases.c.h" + #line 4192 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -4197,7 +4198,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3028 "Python/bytecodes.c" + #line 3029 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4219,7 +4220,7 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4223 "Python/generated_cases.c.h" + #line 4224 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4233,7 +4234,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3053 "Python/bytecodes.c" + #line 3054 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4261,7 +4262,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4265 "Python/generated_cases.c.h" + #line 4266 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4275,7 +4276,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3084 "Python/bytecodes.c" + #line 3085 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4307,7 +4308,7 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 4311 "Python/generated_cases.c.h" + #line 4312 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4321,7 +4322,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3119 "Python/bytecodes.c" + #line 3120 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -4353,7 +4354,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4357 "Python/generated_cases.c.h" + #line 4358 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4367,7 +4368,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3154 "Python/bytecodes.c" + #line 3155 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -4392,7 +4393,7 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4396 "Python/generated_cases.c.h" + #line 4397 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4405,7 +4406,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3181 "Python/bytecodes.c" + #line 3182 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -4432,7 +4433,7 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4436 "Python/generated_cases.c.h" + #line 4437 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4444,7 +4445,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 3211 "Python/bytecodes.c" + #line 3212 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -4462,14 +4463,14 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 4466 "Python/generated_cases.c.h" + #line 4467 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3231 "Python/bytecodes.c" + #line 3232 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4500,7 +4501,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4504 "Python/generated_cases.c.h" + #line 4505 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4513,7 +4514,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3265 "Python/bytecodes.c" + #line 3266 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4542,7 +4543,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4546 "Python/generated_cases.c.h" + #line 4547 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4555,7 +4556,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3297 "Python/bytecodes.c" + #line 3298 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -4584,7 +4585,7 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4588 "Python/generated_cases.c.h" + #line 4589 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4597,7 +4598,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3329 "Python/bytecodes.c" + #line 3330 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4625,7 +4626,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4629 "Python/generated_cases.c.h" + #line 4630 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4635,9 +4636,9 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { - #line 3360 "Python/bytecodes.c" + #line 3361 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); - #line 4641 "Python/generated_cases.c.h" + #line 4642 "Python/generated_cases.c.h" } TARGET(CALL_FUNCTION_EX) { @@ -4646,7 +4647,7 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 3364 "Python/bytecodes.c" + #line 3365 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs)); @@ -4708,14 +4709,14 @@ } result = PyObject_Call(func, callargs, kwargs); } - #line 4712 "Python/generated_cases.c.h" + #line 4713 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 3426 "Python/bytecodes.c" + #line 3427 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4719 "Python/generated_cases.c.h" + #line 4720 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4726,7 +4727,7 @@ TARGET(MAKE_FUNCTION) { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3432 "Python/bytecodes.c" + #line 3433 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4738,7 +4739,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4742 "Python/generated_cases.c.h" + #line 4743 "Python/generated_cases.c.h" stack_pointer[-1] = func; DISPATCH(); } @@ -4746,7 +4747,7 @@ TARGET(SET_FUNCTION_ATTRIBUTE) { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3446 "Python/bytecodes.c" + #line 3447 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -4771,14 +4772,14 @@ default: Py_UNREACHABLE(); } - #line 4775 "Python/generated_cases.c.h" + #line 4776 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 3473 "Python/bytecodes.c" + #line 3474 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4799,7 +4800,7 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4803 "Python/generated_cases.c.h" + #line 4804 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4807,15 +4808,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3496 "Python/bytecodes.c" + #line 3497 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4813 "Python/generated_cases.c.h" + #line 4814 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3498 "Python/bytecodes.c" + #line 3499 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4819 "Python/generated_cases.c.h" + #line 4820 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4825,14 +4826,14 @@ TARGET(CONVERT_VALUE) { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3502 "Python/bytecodes.c" + #line 3503 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 4836 "Python/generated_cases.c.h" + #line 4837 "Python/generated_cases.c.h" stack_pointer[-1] = result; DISPATCH(); } @@ -4840,7 +4841,7 @@ TARGET(FORMAT_SIMPLE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3511 "Python/bytecodes.c" + #line 3512 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -4851,7 +4852,7 @@ else { res = value; } - #line 4855 "Python/generated_cases.c.h" + #line 4856 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -4860,12 +4861,12 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3524 "Python/bytecodes.c" + #line 3525 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 4869 "Python/generated_cases.c.h" + #line 4870 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -4874,10 +4875,10 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3531 "Python/bytecodes.c" + #line 3532 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4881 "Python/generated_cases.c.h" + #line 4882 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4889,7 +4890,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3536 "Python/bytecodes.c" + #line 3537 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -4904,12 +4905,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4908 "Python/generated_cases.c.h" + #line 4909 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3551 "Python/bytecodes.c" + #line 3552 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 4913 "Python/generated_cases.c.h" + #line 4914 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -4919,16 +4920,16 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3556 "Python/bytecodes.c" + #line 3557 "Python/bytecodes.c" assert(oparg >= 2); - #line 4925 "Python/generated_cases.c.h" + #line 4926 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } TARGET(INSTRUMENTED_INSTRUCTION) { - #line 3560 "Python/bytecodes.c" + #line 3561 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error; @@ -4940,48 +4941,48 @@ assert(next_opcode > 0 && next_opcode < 256); opcode = next_opcode; DISPATCH_GOTO(); - #line 4944 "Python/generated_cases.c.h" + #line 4945 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_JUMP_FORWARD) { - #line 3574 "Python/bytecodes.c" + #line 3575 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); - #line 4950 "Python/generated_cases.c.h" + #line 4951 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) { - #line 3578 "Python/bytecodes.c" + #line 3579 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); - #line 4958 "Python/generated_cases.c.h" + #line 4959 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { - #line 3583 "Python/bytecodes.c" + #line 3584 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsTrue(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4969 "Python/generated_cases.c.h" + #line 4970 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { - #line 3591 "Python/bytecodes.c" + #line 3592 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsFalse(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4980 "Python/generated_cases.c.h" + #line 4981 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { - #line 3599 "Python/bytecodes.c" + #line 3600 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -4993,12 +4994,12 @@ offset = 0; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4997 "Python/generated_cases.c.h" + #line 4998 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { - #line 3613 "Python/bytecodes.c" + #line 3614 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -5010,30 +5011,30 @@ offset = oparg; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5014 "Python/generated_cases.c.h" + #line 5015 "Python/generated_cases.c.h" DISPATCH(); } TARGET(EXTENDED_ARG) { - #line 3627 "Python/bytecodes.c" + #line 3628 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 5025 "Python/generated_cases.c.h" + #line 5026 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3635 "Python/bytecodes.c" + #line 3636 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); - #line 5032 "Python/generated_cases.c.h" + #line 5033 "Python/generated_cases.c.h" } TARGET(RESERVED) { - #line 3640 "Python/bytecodes.c" + #line 3641 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 5039 "Python/generated_cases.c.h" + #line 5040 "Python/generated_cases.c.h" } From webhook-mailer at python.org Fri Jul 7 07:41:58 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Fri, 07 Jul 2023 11:41:58 -0000 Subject: [Python-checkins] gh-106503: asyncio._SelectorSocketTransport: fix cyclic reference on close(). (#106504) Message-ID: https://github.com/python/cpython/commit/3e5ce7968f5ab715f649e296e1f6b499621b8091 commit: 3e5ce7968f5ab715f649e296e1f6b499621b8091 branch: main author: Andrew Geng committer: kumaraditya303 <59607654+kumaraditya303 at users.noreply.github.com> date: 2023-07-07T17:11:54+05:30 summary: gh-106503: asyncio._SelectorSocketTransport: fix cyclic reference on close(). (#106504) files: A Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst M Lib/asyncio/selector_events.py M Misc/ACKS diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index fa2422b7fba4a..f895750e3cf95 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -1202,6 +1202,7 @@ def _reset_empty_waiter(self): def close(self): self._read_ready_cb = None + self._write_ready = None super().close() diff --git a/Misc/ACKS b/Misc/ACKS index 454b63155f013..ef0029a7e4119 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -615,6 +615,7 @@ Marius Gedminas Jan-Philip Gehrcke Thomas Gellekum Gabriel Genellina +Andrew Geng Philip Georgi Christos Georgiou Elazar (?????) Gershuni diff --git a/Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst b/Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst new file mode 100644 index 0000000000000..b8dd850386e86 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst @@ -0,0 +1,2 @@ +Fix ref cycle in :class:`!asyncio._SelectorSocketTransport` by removing +``_write_ready`` in ``close``. From webhook-mailer at python.org Fri Jul 7 08:08:25 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Fri, 07 Jul 2023 12:08:25 -0000 Subject: [Python-checkins] [3.12] gh-106503: asyncio._SelectorSocketTransport: fix cyclic reference on close(). (GH-106504) (#106514) Message-ID: https://github.com/python/cpython/commit/7e883d76c0a17a7c97782384d8e2ec025eade91b commit: 7e883d76c0a17a7c97782384d8e2ec025eade91b branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 <59607654+kumaraditya303 at users.noreply.github.com> date: 2023-07-07T12:08:22Z summary: [3.12] gh-106503: asyncio._SelectorSocketTransport: fix cyclic reference on close(). (GH-106504) (#106514) gh-106503: asyncio._SelectorSocketTransport: fix cyclic reference on close(). (GH-106504) (cherry picked from commit 3e5ce7968f5ab715f649e296e1f6b499621b8091) Co-authored-by: Andrew Geng files: A Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst M Lib/asyncio/selector_events.py M Misc/ACKS diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index fa2422b7fba4a..f895750e3cf95 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -1202,6 +1202,7 @@ def _reset_empty_waiter(self): def close(self): self._read_ready_cb = None + self._write_ready = None super().close() diff --git a/Misc/ACKS b/Misc/ACKS index 454b63155f013..ef0029a7e4119 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -615,6 +615,7 @@ Marius Gedminas Jan-Philip Gehrcke Thomas Gellekum Gabriel Genellina +Andrew Geng Philip Georgi Christos Georgiou Elazar (?????) Gershuni diff --git a/Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst b/Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst new file mode 100644 index 0000000000000..b8dd850386e86 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst @@ -0,0 +1,2 @@ +Fix ref cycle in :class:`!asyncio._SelectorSocketTransport` by removing +``_write_ready`` in ``close``. From webhook-mailer at python.org Fri Jul 7 09:10:10 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Fri, 07 Jul 2023 13:10:10 -0000 Subject: [Python-checkins] gh-104683: clinic.py: refactor `Parameter` and `Function` as dataclasses (#106477) Message-ID: https://github.com/python/cpython/commit/363f4f99c524a6d763d1548986a79c42cc7ca292 commit: 363f4f99c524a6d763d1548986a79c42cc7ca292 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-07T13:10:07Z summary: gh-104683: clinic.py: refactor `Parameter` and `Function` as dataclasses (#106477) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 24d6255f262da..19c5f573920cb 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2439,6 +2439,8 @@ def __repr__(self) -> str: ParamDict = dict[str, "Parameter"] ReturnConverterType = Callable[..., "CReturnConverter"] + + at dc.dataclass(repr=False) class Function: """ Mutable duck type for inspect.Function. @@ -2450,49 +2452,34 @@ class Function: It will always be true that (not docstring) or ((not docstring[0].isspace()) and (docstring.rstrip() == docstring)) """ + parameters: ParamDict = dc.field(default_factory=dict) + _: dc.KW_ONLY + name: str + module: Module + cls: Class | None = None + c_basename: str | None = None + full_name: str | None = None + return_converter: CReturnConverter + return_annotation: object = inspect.Signature.empty + docstring: str = '' + kind: str = CALLABLE + coexist: bool = False + # docstring_only means "don't generate a machine-readable + # signature, just a normal docstring". it's True for + # functions with optional groups because we can't represent + # those accurately with inspect.Signature in 3.4. + docstring_only: bool = False - def __init__( - self, - parameters: ParamDict | None = None, - *, - name: str, - module: Module, - cls: Class | None = None, - c_basename: str | None = None, - full_name: str | None = None, - return_converter: CReturnConverter, - return_annotation = inspect.Signature.empty, - docstring: str | None = None, - kind: str = CALLABLE, - coexist: bool = False, - docstring_only: bool = False - ) -> None: - self.parameters = parameters or {} - self.return_annotation = return_annotation - self.name = name - self.full_name = full_name - self.module = module - self.cls = cls - self.parent = cls or module - self.c_basename = c_basename - self.return_converter = return_converter - self.docstring = docstring or '' - self.kind = kind - self.coexist = coexist + def __post_init__(self) -> None: + self.parent: Class | Module = self.cls or self.module self.self_converter: self_converter | None = None - # docstring_only means "don't generate a machine-readable - # signature, just a normal docstring". it's True for - # functions with optional groups because we can't represent - # those accurately with inspect.Signature in 3.4. - self.docstring_only = docstring_only - - self.rendered_parameters = None + self.__render_parameters__: list[Parameter] | None = None - __render_parameters__ = None @property - def render_parameters(self): + def render_parameters(self) -> list[Parameter]: if not self.__render_parameters__: - self.__render_parameters__ = l = [] + l: list[Parameter] = [] + self.__render_parameters__ = l for p in self.parameters.values(): p = p.copy() p.converter.pre_render() @@ -2517,17 +2504,8 @@ def methoddef_flags(self) -> str | None: def __repr__(self) -> str: return '' - def copy(self, **overrides) -> "Function": - kwargs = { - 'name': self.name, 'module': self.module, 'parameters': self.parameters, - 'cls': self.cls, 'c_basename': self.c_basename, - 'full_name': self.full_name, - 'return_converter': self.return_converter, 'return_annotation': self.return_annotation, - 'docstring': self.docstring, 'kind': self.kind, 'coexist': self.coexist, - 'docstring_only': self.docstring_only, - } - kwargs.update(overrides) - f = Function(**kwargs) + def copy(self, **overrides: Any) -> Function: + f = dc.replace(self, **overrides) f.parameters = { name: value.copy(function=f) for name, value in f.parameters.items() @@ -2535,31 +2513,21 @@ def copy(self, **overrides) -> "Function": return f + at dc.dataclass(repr=False, slots=True) class Parameter: """ Mutable duck type of inspect.Parameter. """ - - def __init__( - self, - name: str, - kind: inspect._ParameterKind, - *, - default = inspect.Parameter.empty, - function: Function, - converter: "CConverter", - annotation = inspect.Parameter.empty, - docstring: str | None = None, - group: int = 0 - ) -> None: - self.name = name - self.kind = kind - self.default = default - self.function = function - self.converter = converter - self.annotation = annotation - self.docstring = docstring or '' - self.group = group + name: str + kind: inspect._ParameterKind + _: dc.KW_ONLY + default: object = inspect.Parameter.empty + function: Function + converter: CConverter + annotation: object = inspect.Parameter.empty + docstring: str = '' + group: int = 0 + right_bracket_count: int = dc.field(init=False, default=0) def __repr__(self) -> str: return '' @@ -2576,18 +2544,19 @@ def is_vararg(self) -> bool: def is_optional(self) -> bool: return not self.is_vararg() and (self.default is not unspecified) - def copy(self, **overrides) -> "Parameter": - kwargs = { - 'name': self.name, 'kind': self.kind, 'default':self.default, - 'function': self.function, 'converter': self.converter, 'annotation': self.annotation, - 'docstring': self.docstring, 'group': self.group, - } - kwargs.update(overrides) - if 'converter' not in overrides: + def copy( + self, + /, + *, + converter: CConverter | None = None, + function: Function | None = None, + **overrides: Any + ) -> Parameter: + function = function or self.function + if not converter: converter = copy.copy(self.converter) - converter.function = kwargs['function'] - kwargs['converter'] = converter - return Parameter(**kwargs) + converter.function = function + return dc.replace(self, **overrides, function=function, converter=converter) def get_displayname(self, i: int) -> str: if i == 0: @@ -2761,7 +2730,7 @@ def __init__(self, # Positional args: name: str, py_name: str, - function, + function: Function, default: object = unspecified, *, # Keyword only args: c_default: str | None = None, @@ -2800,7 +2769,9 @@ def __init__(self, # about the function in the init. # (that breaks if we get cloned.) # so after this change we will noisily fail. - self.function = LandMine("Don't access members of self.function inside converter_init!") + self.function: Function | LandMine = LandMine( + "Don't access members of self.function inside converter_init!" + ) self.converter_init(**kwargs) self.function = function @@ -2810,7 +2781,7 @@ def converter_init(self): def is_optional(self) -> bool: return (self.default is not unspecified) - def _render_self(self, parameter: str, data: CRenderData) -> None: + def _render_self(self, parameter: Parameter, data: CRenderData) -> None: self.parameter = parameter name = self.parser_name @@ -2870,7 +2841,7 @@ def _render_non_self(self, parameter, data): if cleanup: data.cleanup.append('/* Cleanup for ' + name + ' */\n' + cleanup.rstrip() + "\n") - def render(self, parameter: str, data: CRenderData) -> None: + def render(self, parameter: Parameter, data: CRenderData) -> None: """ parameter is a clinic.Parameter instance. data is a CRenderData instance. @@ -5246,7 +5217,7 @@ def format_docstring(self): assert isinstance(parameters[0].converter, self_converter) # self is always positional-only. assert parameters[0].is_positional_only() - parameters[0].right_bracket_count = 0 + assert parameters[0].right_bracket_count == 0 positional_only = True for p in parameters[1:]: if not p.is_positional_only(): From webhook-mailer at python.org Fri Jul 7 13:42:14 2023 From: webhook-mailer at python.org (gvanrossum) Date: Fri, 07 Jul 2023 17:42:14 -0000 Subject: [Python-checkins] gh-104584: Move super-instruction special-casing to generator (#106500) Message-ID: https://github.com/python/cpython/commit/11038c56ad80dde2cdf65190e78a59d579c94b3a commit: 11038c56ad80dde2cdf65190e78a59d579c94b3a branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-07T17:42:10Z summary: gh-104584: Move super-instruction special-casing to generator (#106500) Instead of special-casing specific instructions, we add a few more special values to the 'size' field of expansions, so in the future we can automatically handle additional super-instructions in the generator. files: M Python/opcode_metadata.h M Python/optimizer.c M Tools/cases_generator/generate_cases.py diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index ac86a4abd9c1b..d29f7216ea65e 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -934,6 +934,12 @@ struct opcode_macro_expansion { struct { int16_t uop; int8_t size; int8_t offset; } uops[8]; }; +#define OPARG_FULL 0 +#define OPARG_CACHE_1 1 +#define OPARG_CACHE_2 2 +#define OPARG_CACHE_4 4 +#define OPARG_TOP 5 +#define OPARG_BOTTOM 6 #define OPCODE_METADATA_FMT(OP) (_PyOpcode_opcode_metadata[(OP)].instr_format) #define SAME_OPCODE_METADATA(OP1, OP2) \ @@ -1165,8 +1171,11 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [LOAD_FAST_CHECK] = { .nuops = 1, .uops = { { LOAD_FAST_CHECK, 0, 0 } } }, [LOAD_FAST] = { .nuops = 1, .uops = { { LOAD_FAST, 0, 0 } } }, [LOAD_FAST_AND_CLEAR] = { .nuops = 1, .uops = { { LOAD_FAST_AND_CLEAR, 0, 0 } } }, + [LOAD_FAST_LOAD_FAST] = { .nuops = 2, .uops = { { LOAD_FAST, 5, 0 }, { LOAD_FAST, 6, 0 } } }, [LOAD_CONST] = { .nuops = 1, .uops = { { LOAD_CONST, 0, 0 } } }, [STORE_FAST] = { .nuops = 1, .uops = { { STORE_FAST, 0, 0 } } }, + [STORE_FAST_LOAD_FAST] = { .nuops = 2, .uops = { { STORE_FAST, 5, 0 }, { LOAD_FAST, 6, 0 } } }, + [STORE_FAST_STORE_FAST] = { .nuops = 2, .uops = { { STORE_FAST, 5, 0 }, { STORE_FAST, 6, 0 } } }, [POP_TOP] = { .nuops = 1, .uops = { { POP_TOP, 0, 0 } } }, [PUSH_NULL] = { .nuops = 1, .uops = { { PUSH_NULL, 0, 0 } } }, [END_FOR] = { .nuops = 2, .uops = { { POP_TOP, 0, 0 }, { POP_TOP, 0, 0 } } }, diff --git a/Python/optimizer.c b/Python/optimizer.c index db117bb180c1c..2870f2fd05052 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -411,44 +411,15 @@ translate_bytecode_to_trace( for (;;) { ADD_TO_TRACE(SAVE_IP, (int)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); int opcode = instr->op.code; - uint64_t operand = instr->op.arg; + int oparg = instr->op.arg; int extras = 0; while (opcode == EXTENDED_ARG) { instr++; extras += 1; opcode = instr->op.code; - operand = (operand << 8) | instr->op.arg; + oparg = (oparg << 8) | instr->op.arg; } switch (opcode) { - case LOAD_FAST_LOAD_FAST: - case STORE_FAST_LOAD_FAST: - case STORE_FAST_STORE_FAST: - { - // Reserve space for two uops (+ SAVE_IP + EXIT_TRACE) - if (trace_length + 4 > max_length) { - DPRINTF(1, "Ran out of space for LOAD_FAST_LOAD_FAST\n"); - goto done; - } - uint64_t oparg1 = operand >> 4; - uint64_t oparg2 = operand & 15; - switch (opcode) { - case LOAD_FAST_LOAD_FAST: - ADD_TO_TRACE(LOAD_FAST, oparg1); - ADD_TO_TRACE(LOAD_FAST, oparg2); - break; - case STORE_FAST_LOAD_FAST: - ADD_TO_TRACE(STORE_FAST, oparg1); - ADD_TO_TRACE(LOAD_FAST, oparg2); - break; - case STORE_FAST_STORE_FAST: - ADD_TO_TRACE(STORE_FAST, oparg1); - ADD_TO_TRACE(STORE_FAST, oparg2); - break; - default: - Py_FatalError("Missing case"); - } - break; - } default: { const struct opcode_macro_expansion *expansion = &_PyOpcode_macro_expansion[opcode]; @@ -462,9 +433,11 @@ translate_bytecode_to_trace( goto done; } for (int i = 0; i < nuops; i++) { + uint64_t operand; int offset = expansion->uops[i].offset; switch (expansion->uops[i].size) { - case 0: + case OPARG_FULL: + operand = oparg; if (extras && OPCODE_HAS_JUMP(opcode)) { if (opcode == JUMP_BACKWARD_NO_INTERRUPT) { operand -= extras; @@ -475,19 +448,25 @@ translate_bytecode_to_trace( } } break; - case 1: + case OPARG_CACHE_1: operand = read_u16(&instr[offset].cache); break; - case 2: + case OPARG_CACHE_2: operand = read_u32(&instr[offset].cache); break; - case 4: + case OPARG_CACHE_4: operand = read_u64(&instr[offset].cache); break; + case OPARG_TOP: // First half of super-instr + operand = oparg >> 4; + break; + case OPARG_BOTTOM: // Second half of super-instr + operand = oparg & 0xF; + break; default: fprintf(stderr, - "opcode=%d, operand=%" PRIu64 "; nuops=%d, i=%d; size=%d, offset=%d\n", - opcode, operand, nuops, i, + "opcode=%d, oparg=%d; nuops=%d, i=%d; size=%d, offset=%d\n", + opcode, oparg, nuops, i, expansion->uops[i].size, expansion->uops[i].offset); Py_FatalError("garbled expansion"); diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index a90abfe20c173..632834ce23166 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -40,6 +40,17 @@ UNUSED = "unused" BITS_PER_CODE_UNIT = 16 +# Constants used instead of size for macro expansions. +# Note: 1, 2, 4 must match actual cache entry sizes. +OPARG_SIZES = { + "OPARG_FULL": 0, + "OPARG_CACHE_1": 1, + "OPARG_CACHE_2": 2, + "OPARG_CACHE_4": 4, + "OPARG_TOP": 5, + "OPARG_BOTTOM": 6, +} + RESERVED_WORDS = { "co_consts" : "Use FRAME_CO_CONSTS.", "co_names": "Use FRAME_CO_NAMES.", @@ -1213,7 +1224,10 @@ def write_metadata(self) -> None: self.out.emit("struct { int16_t uop; int8_t size; int8_t offset; } uops[8];") self.out.emit("") + for key, value in OPARG_SIZES.items(): + self.out.emit(f"#define {key} {value}") self.out.emit("") + self.out.emit("#define OPCODE_METADATA_FMT(OP) " "(_PyOpcode_opcode_metadata[(OP)].instr_format)") self.out.emit("#define SAME_OPCODE_METADATA(OP1, OP2) \\") @@ -1263,6 +1277,9 @@ def write_metadata(self) -> None: # Construct a dummy Component -- input/output mappings are not used part = Component(instr, [], [], instr.active_caches) self.write_macro_expansions(instr.name, [part]) + elif instr.kind == "inst" and variable_used(instr.inst, "oparg1"): + assert variable_used(instr.inst, "oparg2"), "Half super-instr?" + self.write_super_expansions(instr.name) case parser.Macro(): mac = self.macro_instrs[thing.name] self.write_macro_expansions(mac.name, mac.parts) @@ -1342,7 +1359,7 @@ def write_macro_expansions(self, name: str, parts: MacroParts) -> None: print(f"NOTE: Part {part.instr.name} of {name} is not a viable uop") return if part.instr.instr_flags.HAS_ARG_FLAG or not part.active_caches: - size, offset = 0, 0 + size, offset = OPARG_SIZES["OPARG_FULL"], 0 else: # If this assert triggers, is_viable_uops() lied assert len(part.active_caches) == 1, (name, part.instr.name) @@ -1350,10 +1367,50 @@ def write_macro_expansions(self, name: str, parts: MacroParts) -> None: size, offset = cache.effect.size, cache.offset expansions.append((part.instr.name, size, offset)) assert len(expansions) > 0, f"Macro {name} has empty expansion?!" + self.write_expansions(name, expansions) + + def write_super_expansions(self, name: str) -> None: + """Write special macro expansions for super-instructions. + + If you get an assertion failure here, you probably have accidentally + violated one of the assumptions here. + + - A super-instruction's name is of the form FIRST_SECOND where + FIRST and SECOND are regular instructions whose name has the + form FOO_BAR. Thus, there must be exactly 3 underscores. + Example: LOAD_CONST_STORE_FAST. + + - A super-instruction's body uses `oparg1 and `oparg2`, and no + other instruction's body uses those variable names. + + - A super-instruction has no active (used) cache entries. + + In the expansion, the first instruction's operand is all but the + bottom 4 bits of the super-instruction's oparg, and the second + instruction's operand is the bottom 4 bits. We use the special + size codes OPARG_TOP and OPARG_BOTTOM for these. + """ + pieces = name.split("_") + assert len(pieces) == 4, f"{name} doesn't look like a super-instr" + name1 = "_".join(pieces[:2]) + name2 = "_".join(pieces[2:]) + assert name1 in self.instrs, f"{name1} doesn't match any instr" + assert name2 in self.instrs, f"{name2} doesn't match any instr" + instr1 = self.instrs[name1] + instr2 = self.instrs[name2] + assert not instr1.active_caches, f"{name1} has active caches" + assert not instr2.active_caches, f"{name2} has active caches" + expansions = [ + (name1, OPARG_SIZES["OPARG_TOP"], 0), + (name2, OPARG_SIZES["OPARG_BOTTOM"], 0), + ] + self.write_expansions(name, expansions) + + def write_expansions(self, name: str, expansions: list[tuple[str, int, int]]) -> None: pieces = [f"{{ {name}, {size}, {offset} }}" for name, size, offset in expansions] self.out.emit( f"[{name}] = " - f"{{ .nuops = {len(expansions)}, .uops = {{ {', '.join(pieces)} }} }}," + f"{{ .nuops = {len(pieces)}, .uops = {{ {', '.join(pieces)} }} }}," ) def emit_metadata_entry( From webhook-mailer at python.org Fri Jul 7 14:03:31 2023 From: webhook-mailer at python.org (gvanrossum) Date: Fri, 07 Jul 2023 18:03:31 -0000 Subject: [Python-checkins] gh-104584: Allow unspecialized instructions in superblocks (#106497) Message-ID: https://github.com/python/cpython/commit/b3648f036e502db7e7da951ec4eb1f205cb3d74e commit: b3648f036e502db7e7da951ec4eb1f205cb3d74e branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-07T18:03:27Z summary: gh-104584: Allow unspecialized instructions in superblocks (#106497) This adds several of unspecialized opcodes to superblocks: TO_BOOL, BINARY_SUBSCR, STORE_SUBSCR, UNPACK_SEQUENCE, LOAD_GLOBAL, LOAD_ATTR, COMPARE_OP, BINARY_OP. While we may not want that eventually, for now this helps finding bugs. There is a rudimentary test checking for UNPACK_SEQUENCE. Once we're ready to undo this, that would be simple: just replace the call to variable_used_unspecialized with a call to variable_used (as shown in a comment). Or add individual opcdes to FORBIDDEN_NAMES_IN_UOPS. files: M Lib/test/test_capi/test_misc.py M Python/executor_cases.c.h M Python/opcode_metadata.h M Tools/cases_generator/generate_cases.py diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 181e6b8077f9f..5f39f23401c3f 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2499,6 +2499,34 @@ def many_vars(): ex = get_first_executor(many_vars.__code__) self.assertIn(("LOAD_FAST", 259), list(ex)) + def test_unspecialized_unpack(self): + # An example of an unspecialized opcode + def testfunc(x): + i = 0 + while i < x: + i += 1 + a, b = {1: 2, 3: 3} + assert a == 1 and b == 3 + i = 0 + while i < x: + i += 1 + + opt = _testinternalcapi.get_uop_optimizer() + + with temporary_optimizer(opt): + testfunc(10) + + ex = None + for offset in range(0, len(testfunc.__code__.co_code), 2): + try: + ex = _testinternalcapi.get_executor(testfunc.__code__, offset) + break + except ValueError: + pass + self.assertIsNotNone(ex) + uops = {opname for opname, _ in ex} + self.assertIn("UNPACK_SEQUENCE", uops) + if __name__ == "__main__": unittest.main() diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 29ebb0b3e8b2d..32efeb099d9b3 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -118,12 +118,38 @@ break; } + case TO_BOOL: { + static_assert(INLINE_CACHE_ENTRIES_TO_BOOL == 3, "incorrect cache size"); + PyObject *value = stack_pointer[-1]; + PyObject *res; + #line 296 "Python/bytecodes.c" + #if ENABLE_SPECIALIZATION + _PyToBoolCache *cache = (_PyToBoolCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { + next_instr--; + _Py_Specialize_ToBool(value, next_instr); + DISPATCH_SAME_OPARG(); + } + STAT_INC(TO_BOOL, deferred); + DECREMENT_ADAPTIVE_COUNTER(cache->counter); + #endif /* ENABLE_SPECIALIZATION */ + int err = PyObject_IsTrue(value); + #line 138 "Python/executor_cases.c.h" + Py_DECREF(value); + #line 308 "Python/bytecodes.c" + if (err < 0) goto pop_1_error; + res = err ? Py_True : Py_False; + #line 143 "Python/executor_cases.c.h" + stack_pointer[-1] = res; + break; + } + case TO_BOOL_BOOL: { PyObject *value = stack_pointer[-1]; #line 313 "Python/bytecodes.c" DEOPT_IF(!PyBool_Check(value), TO_BOOL); STAT_INC(TO_BOOL, hit); - #line 127 "Python/executor_cases.c.h" + #line 153 "Python/executor_cases.c.h" break; } @@ -138,12 +164,12 @@ res = Py_False; } else { - #line 142 "Python/executor_cases.c.h" + #line 168 "Python/executor_cases.c.h" Py_DECREF(value); #line 326 "Python/bytecodes.c" res = Py_True; } - #line 147 "Python/executor_cases.c.h" + #line 173 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -155,7 +181,7 @@ DEOPT_IF(!PyList_CheckExact(value), TO_BOOL); STAT_INC(TO_BOOL, hit); res = Py_SIZE(value) ? Py_True : Py_False; - #line 159 "Python/executor_cases.c.h" + #line 185 "Python/executor_cases.c.h" Py_DECREF(value); stack_pointer[-1] = res; break; @@ -169,7 +195,7 @@ DEOPT_IF(!Py_IsNone(value), TO_BOOL); STAT_INC(TO_BOOL, hit); res = Py_False; - #line 173 "Python/executor_cases.c.h" + #line 199 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -186,12 +212,12 @@ } else { assert(Py_SIZE(value)); - #line 190 "Python/executor_cases.c.h" + #line 216 "Python/executor_cases.c.h" Py_DECREF(value); #line 354 "Python/bytecodes.c" res = Py_True; } - #line 195 "Python/executor_cases.c.h" + #line 221 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -205,11 +231,11 @@ assert(version); DEOPT_IF(Py_TYPE(value)->tp_version_tag != version, TO_BOOL); STAT_INC(TO_BOOL, hit); - #line 209 "Python/executor_cases.c.h" + #line 235 "Python/executor_cases.c.h" Py_DECREF(value); #line 364 "Python/bytecodes.c" res = Py_True; - #line 213 "Python/executor_cases.c.h" + #line 239 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -219,11 +245,11 @@ PyObject *res; #line 368 "Python/bytecodes.c" res = PyNumber_Invert(value); - #line 223 "Python/executor_cases.c.h" + #line 249 "Python/executor_cases.c.h" Py_DECREF(value); #line 370 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 227 "Python/executor_cases.c.h" + #line 253 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -234,7 +260,7 @@ #line 386 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); - #line 238 "Python/executor_cases.c.h" + #line 264 "Python/executor_cases.c.h" break; } @@ -248,7 +274,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 252 "Python/executor_cases.c.h" + #line 278 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -264,7 +290,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 268 "Python/executor_cases.c.h" + #line 294 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -280,7 +306,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 284 "Python/executor_cases.c.h" + #line 310 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -292,7 +318,7 @@ #line 422 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); - #line 296 "Python/executor_cases.c.h" + #line 322 "Python/executor_cases.c.h" break; } @@ -306,7 +332,7 @@ ((PyFloatObject *)left)->ob_fval * ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 310 "Python/executor_cases.c.h" + #line 336 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -322,7 +348,7 @@ ((PyFloatObject *)left)->ob_fval + ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 326 "Python/executor_cases.c.h" + #line 352 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -338,7 +364,7 @@ ((PyFloatObject *)left)->ob_fval - ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 342 "Python/executor_cases.c.h" + #line 368 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -350,7 +376,7 @@ #line 458 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(!PyUnicode_CheckExact(right), BINARY_OP); - #line 354 "Python/executor_cases.c.h" + #line 380 "Python/executor_cases.c.h" break; } @@ -364,7 +390,35 @@ _Py_DECREF_SPECIALIZED(left, _PyUnicode_ExactDealloc); _Py_DECREF_SPECIALIZED(right, _PyUnicode_ExactDealloc); if (res == NULL) goto pop_2_error; - #line 368 "Python/executor_cases.c.h" + #line 394 "Python/executor_cases.c.h" + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + + case BINARY_SUBSCR: { + static_assert(INLINE_CACHE_ENTRIES_BINARY_SUBSCR == 1, "incorrect cache size"); + PyObject *sub = stack_pointer[-1]; + PyObject *container = stack_pointer[-2]; + PyObject *res; + #line 517 "Python/bytecodes.c" + #if ENABLE_SPECIALIZATION + _PyBinarySubscrCache *cache = (_PyBinarySubscrCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { + next_instr--; + _Py_Specialize_BinarySubscr(container, sub, next_instr); + DISPATCH_SAME_OPARG(); + } + STAT_INC(BINARY_SUBSCR, deferred); + DECREMENT_ADAPTIVE_COUNTER(cache->counter); + #endif /* ENABLE_SPECIALIZATION */ + res = PyObject_GetItem(container, sub); + #line 417 "Python/executor_cases.c.h" + Py_DECREF(container); + Py_DECREF(sub); + #line 529 "Python/bytecodes.c" + if (res == NULL) goto pop_2_error; + #line 422 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -388,7 +442,7 @@ } Py_DECREF(container); if (res == NULL) goto pop_3_error; - #line 392 "Python/executor_cases.c.h" + #line 446 "Python/executor_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = res; break; @@ -412,7 +466,7 @@ Py_DECREF(v); Py_DECREF(container); if (err) goto pop_4_error; - #line 416 "Python/executor_cases.c.h" + #line 470 "Python/executor_cases.c.h" STACK_SHRINK(4); break; } @@ -435,7 +489,7 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 439 "Python/executor_cases.c.h" + #line 493 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -459,7 +513,7 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(tuple); - #line 463 "Python/executor_cases.c.h" + #line 517 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -477,14 +531,14 @@ if (!_PyErr_Occurred(tstate)) { _PyErr_SetKeyError(sub); } - #line 481 "Python/executor_cases.c.h" + #line 535 "Python/executor_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); #line 603 "Python/bytecodes.c" if (true) goto pop_2_error; } Py_INCREF(res); // Do this before DECREF'ing dict, sub - #line 488 "Python/executor_cases.c.h" + #line 542 "Python/executor_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); STACK_SHRINK(1); @@ -497,7 +551,7 @@ PyObject *list = stack_pointer[-(2 + (oparg-1))]; #line 635 "Python/bytecodes.c" if (_PyList_AppendTakeRef((PyListObject *)list, v) < 0) goto pop_1_error; - #line 501 "Python/executor_cases.c.h" + #line 555 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -507,15 +561,47 @@ PyObject *set = stack_pointer[-(2 + (oparg-1))]; #line 639 "Python/bytecodes.c" int err = PySet_Add(set, v); - #line 511 "Python/executor_cases.c.h" + #line 565 "Python/executor_cases.c.h" Py_DECREF(v); #line 641 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 515 "Python/executor_cases.c.h" + #line 569 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } + case STORE_SUBSCR: { + static_assert(INLINE_CACHE_ENTRIES_STORE_SUBSCR == 1, "incorrect cache size"); + PyObject *sub = stack_pointer[-1]; + PyObject *container = stack_pointer[-2]; + PyObject *v = stack_pointer[-3]; + uint16_t counter = (uint16_t)operand; + #line 651 "Python/bytecodes.c" + #if ENABLE_SPECIALIZATION + if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { + next_instr--; + _Py_Specialize_StoreSubscr(container, sub, next_instr); + DISPATCH_SAME_OPARG(); + } + STAT_INC(STORE_SUBSCR, deferred); + _PyStoreSubscrCache *cache = (_PyStoreSubscrCache *)next_instr; + DECREMENT_ADAPTIVE_COUNTER(cache->counter); + #else + (void)counter; // Unused. + #endif /* ENABLE_SPECIALIZATION */ + /* container[sub] = v */ + int err = PyObject_SetItem(container, sub, v); + #line 595 "Python/executor_cases.c.h" + Py_DECREF(v); + Py_DECREF(container); + Py_DECREF(sub); + #line 666 "Python/bytecodes.c" + if (err) goto pop_3_error; + #line 601 "Python/executor_cases.c.h" + STACK_SHRINK(3); + break; + } + case STORE_SUBSCR_LIST_INT: { PyObject *sub = stack_pointer[-1]; PyObject *list = stack_pointer[-2]; @@ -537,7 +623,7 @@ Py_DECREF(old_value); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 541 "Python/executor_cases.c.h" + #line 627 "Python/executor_cases.c.h" STACK_SHRINK(3); break; } @@ -552,7 +638,7 @@ int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub, value); Py_DECREF(dict); if (err) goto pop_3_error; - #line 556 "Python/executor_cases.c.h" + #line 642 "Python/executor_cases.c.h" STACK_SHRINK(3); break; } @@ -563,12 +649,12 @@ #line 697 "Python/bytecodes.c" /* del container[sub] */ int err = PyObject_DelItem(container, sub); - #line 567 "Python/executor_cases.c.h" + #line 653 "Python/executor_cases.c.h" Py_DECREF(container); Py_DECREF(sub); #line 700 "Python/bytecodes.c" if (err) goto pop_2_error; - #line 572 "Python/executor_cases.c.h" + #line 658 "Python/executor_cases.c.h" STACK_SHRINK(2); break; } @@ -579,11 +665,11 @@ #line 704 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_1); res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value); - #line 583 "Python/executor_cases.c.h" + #line 669 "Python/executor_cases.c.h" Py_DECREF(value); #line 707 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 587 "Python/executor_cases.c.h" + #line 673 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -595,12 +681,12 @@ #line 711 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_2); res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1); - #line 599 "Python/executor_cases.c.h" + #line 685 "Python/executor_cases.c.h" Py_DECREF(value2); Py_DECREF(value1); #line 714 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 604 "Python/executor_cases.c.h" + #line 690 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -622,14 +708,14 @@ "'async for' requires an object with " "__aiter__ method, got %.100s", type->tp_name); - #line 626 "Python/executor_cases.c.h" + #line 712 "Python/executor_cases.c.h" Py_DECREF(obj); #line 832 "Python/bytecodes.c" if (true) goto pop_1_error; } iter = (*getter)(obj); - #line 633 "Python/executor_cases.c.h" + #line 719 "Python/executor_cases.c.h" Py_DECREF(obj); #line 837 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; @@ -644,7 +730,7 @@ Py_DECREF(iter); if (true) goto pop_1_error; } - #line 648 "Python/executor_cases.c.h" + #line 734 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -695,7 +781,7 @@ Py_DECREF(next_iter); } } - #line 699 "Python/executor_cases.c.h" + #line 785 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = awaitable; break; @@ -711,7 +797,7 @@ format_awaitable_error(tstate, Py_TYPE(iterable), oparg); } - #line 715 "Python/executor_cases.c.h" + #line 801 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 904 "Python/bytecodes.c" @@ -730,7 +816,7 @@ } if (iter == NULL) goto pop_1_error; - #line 734 "Python/executor_cases.c.h" + #line 820 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -740,7 +826,7 @@ #line 1034 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; Py_XSETREF(exc_info->exc_value, exc_value); - #line 744 "Python/executor_cases.c.h" + #line 830 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -749,7 +835,7 @@ PyObject *value; #line 1085 "Python/bytecodes.c" value = Py_NewRef(PyExc_AssertionError); - #line 753 "Python/executor_cases.c.h" + #line 839 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -779,7 +865,7 @@ if (true) goto error; } } - #line 783 "Python/executor_cases.c.h" + #line 869 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = bc; break; @@ -794,7 +880,7 @@ if (ns == NULL) { _PyErr_Format(tstate, PyExc_SystemError, "no locals found when storing %R", name); - #line 798 "Python/executor_cases.c.h" + #line 884 "Python/executor_cases.c.h" Py_DECREF(v); #line 1121 "Python/bytecodes.c" if (true) goto pop_1_error; @@ -803,11 +889,11 @@ err = PyDict_SetItem(ns, name, v); else err = PyObject_SetItem(ns, name, v); - #line 807 "Python/executor_cases.c.h" + #line 893 "Python/executor_cases.c.h" Py_DECREF(v); #line 1128 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 811 "Python/executor_cases.c.h" + #line 897 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -830,7 +916,33 @@ name); goto error; } - #line 834 "Python/executor_cases.c.h" + #line 920 "Python/executor_cases.c.h" + break; + } + + case UNPACK_SEQUENCE: { + static_assert(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE == 1, "incorrect cache size"); + PyObject *seq = stack_pointer[-1]; + #line 1158 "Python/bytecodes.c" + #if ENABLE_SPECIALIZATION + _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { + next_instr--; + _Py_Specialize_UnpackSequence(seq, next_instr, oparg); + DISPATCH_SAME_OPARG(); + } + STAT_INC(UNPACK_SEQUENCE, deferred); + DECREMENT_ADAPTIVE_COUNTER(cache->counter); + #endif /* ENABLE_SPECIALIZATION */ + PyObject **top = stack_pointer + oparg - 1; + int res = unpack_iterable(tstate, seq, oparg, -1, top); + #line 940 "Python/executor_cases.c.h" + Py_DECREF(seq); + #line 1171 "Python/bytecodes.c" + if (res == 0) goto pop_1_error; + #line 944 "Python/executor_cases.c.h" + STACK_SHRINK(1); + STACK_GROW(oparg); break; } @@ -844,7 +956,7 @@ STAT_INC(UNPACK_SEQUENCE, hit); values[0] = Py_NewRef(PyTuple_GET_ITEM(seq, 1)); values[1] = Py_NewRef(PyTuple_GET_ITEM(seq, 0)); - #line 848 "Python/executor_cases.c.h" + #line 960 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -862,7 +974,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 866 "Python/executor_cases.c.h" + #line 978 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -880,7 +992,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 884 "Python/executor_cases.c.h" + #line 996 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -893,11 +1005,11 @@ int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); - #line 897 "Python/executor_cases.c.h" + #line 1009 "Python/executor_cases.c.h" Py_DECREF(seq); #line 1211 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 901 "Python/executor_cases.c.h" + #line 1013 "Python/executor_cases.c.h" STACK_GROW((oparg & 0xFF) + (oparg >> 8)); break; } @@ -907,11 +1019,11 @@ #line 1242 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_SetAttr(owner, name, (PyObject *)NULL); - #line 911 "Python/executor_cases.c.h" + #line 1023 "Python/executor_cases.c.h" Py_DECREF(owner); #line 1245 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 915 "Python/executor_cases.c.h" + #line 1027 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -921,11 +1033,11 @@ #line 1249 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyDict_SetItem(GLOBALS(), name, v); - #line 925 "Python/executor_cases.c.h" + #line 1037 "Python/executor_cases.c.h" Py_DECREF(v); #line 1252 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 929 "Python/executor_cases.c.h" + #line 1041 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -943,7 +1055,7 @@ } goto error; } - #line 947 "Python/executor_cases.c.h" + #line 1059 "Python/executor_cases.c.h" break; } @@ -957,7 +1069,7 @@ if (true) goto error; } Py_INCREF(locals); - #line 961 "Python/executor_cases.c.h" + #line 1073 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = locals; break; @@ -1023,17 +1135,81 @@ } } } - #line 1027 "Python/executor_cases.c.h" + #line 1139 "Python/executor_cases.c.h" stack_pointer[-1] = v; break; } + case LOAD_GLOBAL: { + static_assert(INLINE_CACHE_ENTRIES_LOAD_GLOBAL == 4, "incorrect cache size"); + PyObject *null = NULL; + PyObject *v; + #line 1351 "Python/bytecodes.c" + #if ENABLE_SPECIALIZATION + _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { + PyObject *name = GETITEM(FRAME_CO_NAMES, oparg>>1); + next_instr--; + _Py_Specialize_LoadGlobal(GLOBALS(), BUILTINS(), next_instr, name); + DISPATCH_SAME_OPARG(); + } + STAT_INC(LOAD_GLOBAL, deferred); + DECREMENT_ADAPTIVE_COUNTER(cache->counter); + #endif /* ENABLE_SPECIALIZATION */ + PyObject *name = GETITEM(FRAME_CO_NAMES, oparg>>1); + if (PyDict_CheckExact(GLOBALS()) + && PyDict_CheckExact(BUILTINS())) + { + v = _PyDict_LoadGlobal((PyDictObject *)GLOBALS(), + (PyDictObject *)BUILTINS(), + name); + if (v == NULL) { + if (!_PyErr_Occurred(tstate)) { + /* _PyDict_LoadGlobal() returns NULL without raising + * an exception if the key doesn't exist */ + format_exc_check_arg(tstate, PyExc_NameError, + NAME_ERROR_MSG, name); + } + if (true) goto error; + } + Py_INCREF(v); + } + else { + /* Slow-path if globals or builtins is not a dict */ + + /* namespace 1: globals */ + v = PyObject_GetItem(GLOBALS(), name); + if (v == NULL) { + if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) goto error; + _PyErr_Clear(tstate); + + /* namespace 2: builtins */ + v = PyObject_GetItem(BUILTINS(), name); + if (v == NULL) { + if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { + format_exc_check_arg( + tstate, PyExc_NameError, + NAME_ERROR_MSG, name); + } + if (true) goto error; + } + } + } + null = NULL; + #line 1200 "Python/executor_cases.c.h" + STACK_GROW(1); + STACK_GROW(((oparg & 1) ? 1 : 0)); + stack_pointer[-1] = v; + if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = null; } + break; + } + case DELETE_FAST: { #line 1435 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL); - #line 1037 "Python/executor_cases.c.h" + #line 1213 "Python/executor_cases.c.h" break; } @@ -1049,7 +1225,7 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 1053 "Python/executor_cases.c.h" + #line 1229 "Python/executor_cases.c.h" break; } @@ -1091,7 +1267,7 @@ } Py_INCREF(value); } - #line 1095 "Python/executor_cases.c.h" + #line 1271 "Python/executor_cases.c.h" stack_pointer[-1] = value; break; } @@ -1106,7 +1282,7 @@ if (true) goto error; } Py_INCREF(value); - #line 1110 "Python/executor_cases.c.h" + #line 1286 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -1119,7 +1295,7 @@ PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 1123 "Python/executor_cases.c.h" + #line 1299 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -1136,7 +1312,7 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 1140 "Python/executor_cases.c.h" + #line 1316 "Python/executor_cases.c.h" break; } @@ -1145,13 +1321,13 @@ PyObject *str; #line 1532 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 1149 "Python/executor_cases.c.h" + #line 1325 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } #line 1534 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1155 "Python/executor_cases.c.h" + #line 1331 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -1164,7 +1340,7 @@ #line 1538 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1168 "Python/executor_cases.c.h" + #line 1344 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -1177,7 +1353,7 @@ #line 1543 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1181 "Python/executor_cases.c.h" + #line 1357 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -1198,13 +1374,13 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 1202 "Python/executor_cases.c.h" + #line 1378 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 1559 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val)); - #line 1208 "Python/executor_cases.c.h" + #line 1384 "Python/executor_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); break; @@ -1215,11 +1391,11 @@ PyObject *set = stack_pointer[-(2 + (oparg-1))]; #line 1566 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 1219 "Python/executor_cases.c.h" + #line 1395 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 1568 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 1223 "Python/executor_cases.c.h" + #line 1399 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -1242,7 +1418,7 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 1246 "Python/executor_cases.c.h" + #line 1422 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -1260,13 +1436,13 @@ if (map == NULL) goto error; - #line 1264 "Python/executor_cases.c.h" + #line 1440 "Python/executor_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } #line 1597 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 1270 "Python/executor_cases.c.h" + #line 1446 "Python/executor_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -1314,7 +1490,7 @@ Py_DECREF(ann_dict); } } - #line 1318 "Python/executor_cases.c.h" + #line 1494 "Python/executor_cases.c.h" break; } @@ -1332,14 +1508,14 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 1336 "Python/executor_cases.c.h" + #line 1512 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); #line 1653 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 1343 "Python/executor_cases.c.h" + #line 1519 "Python/executor_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; break; @@ -1355,12 +1531,12 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 1359 "Python/executor_cases.c.h" + #line 1535 "Python/executor_cases.c.h" Py_DECREF(update); #line 1665 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1364 "Python/executor_cases.c.h" + #line 1540 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1373,12 +1549,12 @@ if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 1377 "Python/executor_cases.c.h" + #line 1553 "Python/executor_cases.c.h" Py_DECREF(update); #line 1676 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1382 "Python/executor_cases.c.h" + #line 1558 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1393,7 +1569,7 @@ /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 1397 "Python/executor_cases.c.h" + #line 1573 "Python/executor_cases.c.h" STACK_SHRINK(2); break; } @@ -1411,13 +1587,13 @@ STAT_INC(LOAD_SUPER_ATTR, hit); PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL); - #line 1415 "Python/executor_cases.c.h" + #line 1591 "Python/executor_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); #line 1772 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; - #line 1421 "Python/executor_cases.c.h" + #line 1597 "Python/executor_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -1454,13 +1630,110 @@ res = res2; res2 = NULL; } - #line 1458 "Python/executor_cases.c.h" + #line 1634 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; stack_pointer[-2] = res2; break; } + case LOAD_ATTR: { + static_assert(INLINE_CACHE_ENTRIES_LOAD_ATTR == 9, "incorrect cache size"); + PyObject *owner = stack_pointer[-1]; + PyObject *res2 = NULL; + PyObject *res; + #line 1815 "Python/bytecodes.c" + #if ENABLE_SPECIALIZATION + _PyAttrCache *cache = (_PyAttrCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { + PyObject *name = GETITEM(FRAME_CO_NAMES, oparg>>1); + next_instr--; + _Py_Specialize_LoadAttr(owner, next_instr, name); + DISPATCH_SAME_OPARG(); + } + STAT_INC(LOAD_ATTR, deferred); + DECREMENT_ADAPTIVE_COUNTER(cache->counter); + #endif /* ENABLE_SPECIALIZATION */ + PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 1); + if (oparg & 1) { + /* Designed to work in tandem with CALL, pushes two values. */ + PyObject* meth = NULL; + if (_PyObject_GetMethod(owner, name, &meth)) { + /* We can bypass temporary bound method object. + meth is unbound method and obj is self. + + meth | self | arg1 | ... | argN + */ + assert(meth != NULL); // No errors on this branch + res2 = meth; + res = owner; // Transfer ownership + } + else { + /* meth is not an unbound method (but a regular attr, or + something was returned by a descriptor protocol). Set + the second element of the stack to NULL, to signal + CALL that it's not a method call. + + NULL | meth | arg1 | ... | argN + */ + #line 1680 "Python/executor_cases.c.h" + Py_DECREF(owner); + #line 1849 "Python/bytecodes.c" + if (meth == NULL) goto pop_1_error; + res2 = NULL; + res = meth; + } + } + else { + /* Classic, pushes one value. */ + res = PyObject_GetAttr(owner, name); + #line 1691 "Python/executor_cases.c.h" + Py_DECREF(owner); + #line 1858 "Python/bytecodes.c" + if (res == NULL) goto pop_1_error; + } + #line 1696 "Python/executor_cases.c.h" + STACK_GROW(((oparg & 1) ? 1 : 0)); + stack_pointer[-1] = res; + if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } + break; + } + + case COMPARE_OP: { + static_assert(INLINE_CACHE_ENTRIES_COMPARE_OP == 1, "incorrect cache size"); + PyObject *right = stack_pointer[-1]; + PyObject *left = stack_pointer[-2]; + PyObject *res; + #line 2091 "Python/bytecodes.c" + #if ENABLE_SPECIALIZATION + _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { + next_instr--; + _Py_Specialize_CompareOp(left, right, next_instr, oparg); + DISPATCH_SAME_OPARG(); + } + STAT_INC(COMPARE_OP, deferred); + DECREMENT_ADAPTIVE_COUNTER(cache->counter); + #endif /* ENABLE_SPECIALIZATION */ + assert((oparg >> 5) <= Py_GE); + res = PyObject_RichCompare(left, right, oparg >> 5); + #line 1721 "Python/executor_cases.c.h" + Py_DECREF(left); + Py_DECREF(right); + #line 2104 "Python/bytecodes.c" + if (res == NULL) goto pop_2_error; + if (oparg & 16) { + int res_bool = PyObject_IsTrue(res); + Py_DECREF(res); + if (res_bool < 0) goto pop_2_error; + res = res_bool ? Py_True : Py_False; + } + #line 1732 "Python/executor_cases.c.h" + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + case COMPARE_OP_FLOAT: { PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; @@ -1477,7 +1750,7 @@ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1481 "Python/executor_cases.c.h" + #line 1754 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1503,7 +1776,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1507 "Python/executor_cases.c.h" + #line 1780 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1526,7 +1799,7 @@ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1530 "Python/executor_cases.c.h" + #line 1803 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1538,12 +1811,12 @@ PyObject *b; #line 2163 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 1542 "Python/executor_cases.c.h" + #line 1815 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); #line 2165 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1547 "Python/executor_cases.c.h" + #line 1820 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1555,13 +1828,13 @@ PyObject *b; #line 2169 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 1559 "Python/executor_cases.c.h" + #line 1832 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); #line 2171 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; - #line 1565 "Python/executor_cases.c.h" + #line 1838 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1574,7 +1847,7 @@ PyObject *match; #line 2176 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 1578 "Python/executor_cases.c.h" + #line 1851 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); #line 2178 "Python/bytecodes.c" @@ -1585,7 +1858,7 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 1589 "Python/executor_cases.c.h" + #line 1862 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); #line 2186 "Python/bytecodes.c" @@ -1597,7 +1870,7 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 1601 "Python/executor_cases.c.h" + #line 1874 "Python/executor_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; break; @@ -1610,18 +1883,18 @@ #line 2197 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 1614 "Python/executor_cases.c.h" + #line 1887 "Python/executor_cases.c.h" Py_DECREF(right); #line 2200 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 1621 "Python/executor_cases.c.h" + #line 1894 "Python/executor_cases.c.h" Py_DECREF(right); #line 2205 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1625 "Python/executor_cases.c.h" + #line 1898 "Python/executor_cases.c.h" stack_pointer[-1] = b; break; } @@ -1635,7 +1908,7 @@ if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 1639 "Python/executor_cases.c.h" + #line 1912 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; break; @@ -1651,7 +1924,7 @@ // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 1655 "Python/executor_cases.c.h" + #line 1928 "Python/executor_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); @@ -1663,7 +1936,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 1667 "Python/executor_cases.c.h" + #line 1940 "Python/executor_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; break; @@ -1675,7 +1948,7 @@ #line 2332 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 1679 "Python/executor_cases.c.h" + #line 1952 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1687,7 +1960,7 @@ #line 2337 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 1691 "Python/executor_cases.c.h" + #line 1964 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1701,7 +1974,7 @@ // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 1705 "Python/executor_cases.c.h" + #line 1978 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; break; @@ -1713,11 +1986,11 @@ #line 2348 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 1717 "Python/executor_cases.c.h" + #line 1990 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 2351 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 1721 "Python/executor_cases.c.h" + #line 1994 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -1748,11 +2021,11 @@ if (iter == NULL) { goto error; } - #line 1752 "Python/executor_cases.c.h" + #line 2025 "Python/executor_cases.c.h" Py_DECREF(iterable); #line 2378 "Python/bytecodes.c" } - #line 1756 "Python/executor_cases.c.h" + #line 2029 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -1783,7 +2056,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 1787 "Python/executor_cases.c.h" + #line 2060 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1802,7 +2075,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 1806 "Python/executor_cases.c.h" + #line 2079 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -1819,7 +2092,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 1823 "Python/executor_cases.c.h" + #line 2096 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -1839,7 +2112,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 1843 "Python/executor_cases.c.h" + #line 2116 "Python/executor_cases.c.h" stack_pointer[-1] = func; break; } @@ -1872,7 +2145,7 @@ default: Py_UNREACHABLE(); } - #line 1876 "Python/executor_cases.c.h" + #line 2149 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; break; @@ -1885,13 +2158,13 @@ PyObject *slice; #line 3497 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 1889 "Python/executor_cases.c.h" + #line 2162 "Python/executor_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); #line 3499 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 1895 "Python/executor_cases.c.h" + #line 2168 "Python/executor_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -1908,7 +2181,7 @@ result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 1912 "Python/executor_cases.c.h" + #line 2185 "Python/executor_cases.c.h" stack_pointer[-1] = result; break; } @@ -1927,7 +2200,7 @@ else { res = value; } - #line 1931 "Python/executor_cases.c.h" + #line 2204 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -1941,7 +2214,7 @@ Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 1945 "Python/executor_cases.c.h" + #line 2218 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1953,18 +2226,49 @@ #line 3532 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 1957 "Python/executor_cases.c.h" + #line 2230 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; break; } + case BINARY_OP: { + static_assert(INLINE_CACHE_ENTRIES_BINARY_OP == 1, "incorrect cache size"); + PyObject *rhs = stack_pointer[-1]; + PyObject *lhs = stack_pointer[-2]; + PyObject *res; + #line 3537 "Python/bytecodes.c" + #if ENABLE_SPECIALIZATION + _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { + next_instr--; + _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg, &GETLOCAL(0)); + DISPATCH_SAME_OPARG(); + } + STAT_INC(BINARY_OP, deferred); + DECREMENT_ADAPTIVE_COUNTER(cache->counter); + #endif /* ENABLE_SPECIALIZATION */ + assert(0 <= oparg); + assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); + assert(binary_ops[oparg]); + res = binary_ops[oparg](lhs, rhs); + #line 2256 "Python/executor_cases.c.h" + Py_DECREF(lhs); + Py_DECREF(rhs); + #line 3552 "Python/bytecodes.c" + if (res == NULL) goto pop_2_error; + #line 2261 "Python/executor_cases.c.h" + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + case SWAP: { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; #line 3557 "Python/bytecodes.c" assert(oparg >= 2); - #line 1968 "Python/executor_cases.c.h" + #line 2272 "Python/executor_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; break; diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index d29f7216ea65e..70e1ca44ce766 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -1182,6 +1182,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [END_SEND] = { .nuops = 1, .uops = { { END_SEND, 0, 0 } } }, [UNARY_NEGATIVE] = { .nuops = 1, .uops = { { UNARY_NEGATIVE, 0, 0 } } }, [UNARY_NOT] = { .nuops = 1, .uops = { { UNARY_NOT, 0, 0 } } }, + [TO_BOOL] = { .nuops = 1, .uops = { { TO_BOOL, 0, 0 } } }, [TO_BOOL_BOOL] = { .nuops = 1, .uops = { { TO_BOOL_BOOL, 0, 0 } } }, [TO_BOOL_INT] = { .nuops = 1, .uops = { { TO_BOOL_INT, 0, 0 } } }, [TO_BOOL_LIST] = { .nuops = 1, .uops = { { TO_BOOL_LIST, 0, 0 } } }, @@ -1196,6 +1197,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [BINARY_OP_ADD_FLOAT] = { .nuops = 2, .uops = { { _GUARD_BOTH_FLOAT, 0, 0 }, { _BINARY_OP_ADD_FLOAT, 0, 0 } } }, [BINARY_OP_SUBTRACT_FLOAT] = { .nuops = 2, .uops = { { _GUARD_BOTH_FLOAT, 0, 0 }, { _BINARY_OP_SUBTRACT_FLOAT, 0, 0 } } }, [BINARY_OP_ADD_UNICODE] = { .nuops = 2, .uops = { { _GUARD_BOTH_UNICODE, 0, 0 }, { _BINARY_OP_ADD_UNICODE, 0, 0 } } }, + [BINARY_SUBSCR] = { .nuops = 1, .uops = { { BINARY_SUBSCR, 0, 0 } } }, [BINARY_SLICE] = { .nuops = 1, .uops = { { BINARY_SLICE, 0, 0 } } }, [STORE_SLICE] = { .nuops = 1, .uops = { { STORE_SLICE, 0, 0 } } }, [BINARY_SUBSCR_LIST_INT] = { .nuops = 1, .uops = { { BINARY_SUBSCR_LIST_INT, 0, 0 } } }, @@ -1203,6 +1205,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [BINARY_SUBSCR_DICT] = { .nuops = 1, .uops = { { BINARY_SUBSCR_DICT, 0, 0 } } }, [LIST_APPEND] = { .nuops = 1, .uops = { { LIST_APPEND, 0, 0 } } }, [SET_ADD] = { .nuops = 1, .uops = { { SET_ADD, 0, 0 } } }, + [STORE_SUBSCR] = { .nuops = 1, .uops = { { STORE_SUBSCR, 1, 0 } } }, [STORE_SUBSCR_LIST_INT] = { .nuops = 1, .uops = { { STORE_SUBSCR_LIST_INT, 0, 0 } } }, [STORE_SUBSCR_DICT] = { .nuops = 1, .uops = { { STORE_SUBSCR_DICT, 0, 0 } } }, [DELETE_SUBSCR] = { .nuops = 1, .uops = { { DELETE_SUBSCR, 0, 0 } } }, @@ -1216,6 +1219,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [LOAD_BUILD_CLASS] = { .nuops = 1, .uops = { { LOAD_BUILD_CLASS, 0, 0 } } }, [STORE_NAME] = { .nuops = 1, .uops = { { STORE_NAME, 0, 0 } } }, [DELETE_NAME] = { .nuops = 1, .uops = { { DELETE_NAME, 0, 0 } } }, + [UNPACK_SEQUENCE] = { .nuops = 1, .uops = { { UNPACK_SEQUENCE, 0, 0 } } }, [UNPACK_SEQUENCE_TWO_TUPLE] = { .nuops = 1, .uops = { { UNPACK_SEQUENCE_TWO_TUPLE, 0, 0 } } }, [UNPACK_SEQUENCE_TUPLE] = { .nuops = 1, .uops = { { UNPACK_SEQUENCE_TUPLE, 0, 0 } } }, [UNPACK_SEQUENCE_LIST] = { .nuops = 1, .uops = { { UNPACK_SEQUENCE_LIST, 0, 0 } } }, @@ -1226,6 +1230,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [LOAD_LOCALS] = { .nuops = 1, .uops = { { _LOAD_LOCALS, 0, 0 } } }, [LOAD_NAME] = { .nuops = 2, .uops = { { _LOAD_LOCALS, 0, 0 }, { _LOAD_FROM_DICT_OR_GLOBALS, 0, 0 } } }, [LOAD_FROM_DICT_OR_GLOBALS] = { .nuops = 1, .uops = { { _LOAD_FROM_DICT_OR_GLOBALS, 0, 0 } } }, + [LOAD_GLOBAL] = { .nuops = 1, .uops = { { LOAD_GLOBAL, 0, 0 } } }, [DELETE_FAST] = { .nuops = 1, .uops = { { DELETE_FAST, 0, 0 } } }, [DELETE_DEREF] = { .nuops = 1, .uops = { { DELETE_DEREF, 0, 0 } } }, [LOAD_FROM_DICT_OR_DEREF] = { .nuops = 1, .uops = { { LOAD_FROM_DICT_OR_DEREF, 0, 0 } } }, @@ -1246,6 +1251,8 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [MAP_ADD] = { .nuops = 1, .uops = { { MAP_ADD, 0, 0 } } }, [LOAD_SUPER_ATTR_ATTR] = { .nuops = 1, .uops = { { LOAD_SUPER_ATTR_ATTR, 0, 0 } } }, [LOAD_SUPER_ATTR_METHOD] = { .nuops = 1, .uops = { { LOAD_SUPER_ATTR_METHOD, 0, 0 } } }, + [LOAD_ATTR] = { .nuops = 1, .uops = { { LOAD_ATTR, 0, 0 } } }, + [COMPARE_OP] = { .nuops = 1, .uops = { { COMPARE_OP, 0, 0 } } }, [COMPARE_OP_FLOAT] = { .nuops = 1, .uops = { { COMPARE_OP_FLOAT, 0, 0 } } }, [COMPARE_OP_INT] = { .nuops = 1, .uops = { { COMPARE_OP_INT, 0, 0 } } }, [COMPARE_OP_STR] = { .nuops = 1, .uops = { { COMPARE_OP_STR, 0, 0 } } }, @@ -1270,6 +1277,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [FORMAT_SIMPLE] = { .nuops = 1, .uops = { { FORMAT_SIMPLE, 0, 0 } } }, [FORMAT_WITH_SPEC] = { .nuops = 1, .uops = { { FORMAT_WITH_SPEC, 0, 0 } } }, [COPY] = { .nuops = 1, .uops = { { COPY, 0, 0 } } }, + [BINARY_OP] = { .nuops = 1, .uops = { { BINARY_OP, 0, 0 } } }, [SWAP] = { .nuops = 1, .uops = { { SWAP, 0, 0 } } }, }; #ifdef NEED_OPCODE_METADATA diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 632834ce23166..14269ca8cbe75 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -425,8 +425,9 @@ def is_viable_uop(self) -> bool: return False res = True for forbidden in FORBIDDEN_NAMES_IN_UOPS: - # TODO: Don't check in '#ifdef ENABLE_SPECIALIZATION' regions - if variable_used(self.inst, forbidden): + # NOTE: To disallow unspecialized uops, use + # if variable_used(self.inst, forbidden): + if variable_used_unspecialized(self.inst, forbidden): # print(f"Skipping {self.name} because it uses {forbidden}") res = False return res @@ -1644,6 +1645,27 @@ def variable_used(node: parser.Node, name: str) -> bool: ) +def variable_used_unspecialized(node: parser.Node, name: str) -> bool: + """Like variable_used(), but skips #if ENABLE_SPECIALIZATION blocks.""" + tokens: list[lx.Token] = [] + skipping = False + for i, token in enumerate(node.tokens): + if token.kind == "MACRO": + text = "".join(token.text.split()) + # TODO: Handle nested #if + if text == "#if": + if ( + i + 1 < len(node.tokens) + and node.tokens[i + 1].text == "ENABLE_SPECIALIZATION" + ): + skipping = True + elif text in ("#else", "#endif"): + skipping = False + if not skipping: + tokens.append(token) + return any(token.kind == "IDENTIFIER" and token.text == name for token in tokens) + + def main(): """Parse command line, parse input, analyze, write output.""" args = arg_parser.parse_args() # Prints message and sys.exit(2) on error From webhook-mailer at python.org Fri Jul 7 14:16:23 2023 From: webhook-mailer at python.org (benjaminp) Date: Fri, 07 Jul 2023 18:16:23 -0000 Subject: [Python-checkins] Delete dead ceval code. (gh-106486) Message-ID: https://github.com/python/cpython/commit/a8554588bad5ca9e890830281b8d70ecbf6f8e23 commit: a8554588bad5ca9e890830281b8d70ecbf6f8e23 branch: main author: Benjamin Peterson committer: benjaminp date: 2023-07-07T13:16:19-05:00 summary: Delete dead ceval code. (gh-106486) files: M Python/ceval.c diff --git a/Python/ceval.c b/Python/ceval.c index 0ee95bc3a3a4b..1b8650a650412 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -763,11 +763,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int DISPATCH(); - if (_Py_HandlePending(tstate) != 0) { - goto error; - } - DISPATCH(); - { /* Start instructions */ #if !USE_COMPUTED_GOTOS From webhook-mailer at python.org Fri Jul 7 14:41:45 2023 From: webhook-mailer at python.org (gvanrossum) Date: Fri, 07 Jul 2023 18:41:45 -0000 Subject: [Python-checkins] gh-104584: Replace ENTER_EXECUTOR with the original in trace projection (#106526) Message-ID: https://github.com/python/cpython/commit/80b9b3a51757ebb1e3547afc349a229706eadfde commit: 80b9b3a51757ebb1e3547afc349a229706eadfde branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-07T11:41:42-07:00 summary: gh-104584: Replace ENTER_EXECUTOR with the original in trace projection (#106526) files: M Python/optimizer.c diff --git a/Python/optimizer.c b/Python/optimizer.c index 2870f2fd05052..1d731ed2309c3 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -419,6 +419,12 @@ translate_bytecode_to_trace( opcode = instr->op.code; oparg = (oparg << 8) | instr->op.arg; } + if (opcode == ENTER_EXECUTOR) { + _PyExecutorObject *executor = (_PyExecutorObject *)code->co_executors->executors[oparg&255]; + opcode = executor->vm_data.opcode; + DPRINTF(2, " * ENTER_EXECUTOR -> %s\n", _PyOpcode_OpName[opcode]); + oparg = (oparg & 0xffffff00) | executor->vm_data.oparg; + } switch (opcode) { default: { From webhook-mailer at python.org Fri Jul 7 16:42:44 2023 From: webhook-mailer at python.org (brettcannon) Date: Fri, 07 Jul 2023 20:42:44 -0000 Subject: [Python-checkins] gh-106300: Improve `assertRaises(Exception)` usages in tests (GH-106302) Message-ID: https://github.com/python/cpython/commit/6e6a4cd52332017b10c8d88fbbbfe015948093f4 commit: 6e6a4cd52332017b10c8d88fbbbfe015948093f4 branch: main author: Nikita Sobolev committer: brettcannon date: 2023-07-07T13:42:40-07:00 summary: gh-106300: Improve `assertRaises(Exception)` usages in tests (GH-106302) files: M Lib/test/test_abc.py M Lib/test/test_codecs.py M Lib/test/test_email/test_message.py M Lib/test/test_importlib/test_main.py M Lib/test/test_mailbox.py M Lib/test/test_shutil.py M Lib/test/test_unittest/testmock/testasync.py diff --git a/Lib/test/test_abc.py b/Lib/test/test_abc.py index 86f31a9acb4d5..5ce57cc209ea8 100644 --- a/Lib/test/test_abc.py +++ b/Lib/test/test_abc.py @@ -448,15 +448,16 @@ class S(metaclass=abc_ABCMeta): # Also check that issubclass() propagates exceptions raised by # __subclasses__. + class CustomError(Exception): ... exc_msg = "exception from __subclasses__" def raise_exc(): - raise Exception(exc_msg) + raise CustomError(exc_msg) class S(metaclass=abc_ABCMeta): __subclasses__ = raise_exc - with self.assertRaisesRegex(Exception, exc_msg): + with self.assertRaisesRegex(CustomError, exc_msg): issubclass(int, S) def test_subclasshook(self): diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index 376175f90f63e..91d7eaf997ae2 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2822,14 +2822,15 @@ def test_binary_to_text_denylists_text_transforms(self): def test_custom_zlib_error_is_noted(self): # Check zlib codec gives a good error for malformed input msg = "decoding with 'zlib_codec' codec failed" - with self.assertRaises(Exception) as failure: + with self.assertRaises(zlib.error) as failure: codecs.decode(b"hello", "zlib_codec") self.assertEqual(msg, failure.exception.__notes__[0]) def test_custom_hex_error_is_noted(self): # Check hex codec gives a good error for malformed input + import binascii msg = "decoding with 'hex_codec' codec failed" - with self.assertRaises(Exception) as failure: + with self.assertRaises(binascii.Error) as failure: codecs.decode(b"hello", "hex_codec") self.assertEqual(msg, failure.exception.__notes__[0]) diff --git a/Lib/test/test_email/test_message.py b/Lib/test/test_email/test_message.py index 4c754bf40fc30..d3f396f02e7a7 100644 --- a/Lib/test/test_email/test_message.py +++ b/Lib/test/test_email/test_message.py @@ -696,14 +696,16 @@ def subtype_as_add(self, method, subtype, outcome): self.assertIsNone(part['Content-Disposition']) class _TestSetRaisingContentManager: + class CustomError(Exception): + pass def set_content(self, msg, content, *args, **kw): - raise Exception('test') + raise self.CustomError('test') def test_default_content_manager_for_add_comes_from_policy(self): cm = self._TestSetRaisingContentManager() m = self.message(policy=self.policy.clone(content_manager=cm)) for method in ('add_related', 'add_alternative', 'add_attachment'): - with self.assertRaises(Exception) as ar: + with self.assertRaises(self._TestSetRaisingContentManager.CustomError) as ar: getattr(m, method)('') self.assertEqual(str(ar.exception), 'test') diff --git a/Lib/test/test_importlib/test_main.py b/Lib/test/test_importlib/test_main.py index 6469aad7130fc..3b49227255eb5 100644 --- a/Lib/test/test_importlib/test_main.py +++ b/Lib/test/test_importlib/test_main.py @@ -68,7 +68,7 @@ def test_abc_enforced(self): dict(name=''), ) def test_invalid_inputs_to_from_name(self, name): - with self.assertRaises(Exception): + with self.assertRaises(ValueError): Distribution.from_name(name) diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py index 4c592eaf34da2..4977a9369ddf8 100644 --- a/Lib/test/test_mailbox.py +++ b/Lib/test/test_mailbox.py @@ -116,10 +116,13 @@ def test_add_nonascii_string_header_raises(self): self.assertMailboxEmpty() def test_add_that_raises_leaves_mailbox_empty(self): + class CustomError(Exception): ... + exc_msg = "a fake error" + def raiser(*args, **kw): - raise Exception("a fake error") + raise CustomError(exc_msg) support.patch(self, email.generator.BytesGenerator, 'flatten', raiser) - with self.assertRaises(Exception): + with self.assertRaisesRegex(CustomError, exc_msg): self._box.add(email.message_from_string("From: Alph?so")) self.assertEqual(len(self._box), 0) self._box.close() diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 878a21a629f3c..93f20a6ff4133 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -2738,7 +2738,7 @@ def test_regular_copy(self): def test_same_file(self): self.addCleanup(self.reset) with self.get_files() as (src, dst): - with self.assertRaises(Exception): + with self.assertRaises((OSError, _GiveupOnFastCopy)): self.zerocopy_fun(src, src) # Make sure src file is not corrupted. self.assertEqual(read_file(TESTFN, binary=True), self.FILEDATA) diff --git a/Lib/test/test_unittest/testmock/testasync.py b/Lib/test/test_unittest/testmock/testasync.py index e9e1f63e84d4e..f57b83f457f27 100644 --- a/Lib/test/test_unittest/testmock/testasync.py +++ b/Lib/test/test_unittest/testmock/testasync.py @@ -459,9 +459,10 @@ async def addition(self, var): pass self.assertEqual(output, 10) async def test_add_side_effect_exception(self): + class CustomError(Exception): pass async def addition(var): pass - mock = AsyncMock(addition, side_effect=Exception('err')) - with self.assertRaises(Exception): + mock = AsyncMock(addition, side_effect=CustomError('side-effect')) + with self.assertRaisesRegex(CustomError, 'side-effect'): await mock(5) async def test_add_side_effect_coroutine(self): From webhook-mailer at python.org Fri Jul 7 17:25:27 2023 From: webhook-mailer at python.org (brettcannon) Date: Fri, 07 Jul 2023 21:25:27 -0000 Subject: [Python-checkins] [3.12] gh-106300: Improve `assertRaises(Exception)` usages in tests (GH-106302) (GH-106534) Message-ID: https://github.com/python/cpython/commit/2ade2fc148fb25a0306d5b14f705396d98c8b926 commit: 2ade2fc148fb25a0306d5b14f705396d98c8b926 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: brettcannon date: 2023-07-07T14:25:24-07:00 summary: [3.12] gh-106300: Improve `assertRaises(Exception)` usages in tests (GH-106302) (GH-106534) gh-106300: Improve `assertRaises(Exception)` usages in tests (GH-106302) (cherry picked from commit 6e6a4cd52332017b10c8d88fbbbfe015948093f4) Co-authored-by: Nikita Sobolev files: M Lib/test/test_abc.py M Lib/test/test_codecs.py M Lib/test/test_email/test_message.py M Lib/test/test_importlib/test_main.py M Lib/test/test_mailbox.py M Lib/test/test_shutil.py M Lib/test/test_unittest/testmock/testasync.py diff --git a/Lib/test/test_abc.py b/Lib/test/test_abc.py index 86f31a9acb4d5..5ce57cc209ea8 100644 --- a/Lib/test/test_abc.py +++ b/Lib/test/test_abc.py @@ -448,15 +448,16 @@ class S(metaclass=abc_ABCMeta): # Also check that issubclass() propagates exceptions raised by # __subclasses__. + class CustomError(Exception): ... exc_msg = "exception from __subclasses__" def raise_exc(): - raise Exception(exc_msg) + raise CustomError(exc_msg) class S(metaclass=abc_ABCMeta): __subclasses__ = raise_exc - with self.assertRaisesRegex(Exception, exc_msg): + with self.assertRaisesRegex(CustomError, exc_msg): issubclass(int, S) def test_subclasshook(self): diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index 376175f90f63e..91d7eaf997ae2 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2822,14 +2822,15 @@ def test_binary_to_text_denylists_text_transforms(self): def test_custom_zlib_error_is_noted(self): # Check zlib codec gives a good error for malformed input msg = "decoding with 'zlib_codec' codec failed" - with self.assertRaises(Exception) as failure: + with self.assertRaises(zlib.error) as failure: codecs.decode(b"hello", "zlib_codec") self.assertEqual(msg, failure.exception.__notes__[0]) def test_custom_hex_error_is_noted(self): # Check hex codec gives a good error for malformed input + import binascii msg = "decoding with 'hex_codec' codec failed" - with self.assertRaises(Exception) as failure: + with self.assertRaises(binascii.Error) as failure: codecs.decode(b"hello", "hex_codec") self.assertEqual(msg, failure.exception.__notes__[0]) diff --git a/Lib/test/test_email/test_message.py b/Lib/test/test_email/test_message.py index 4c754bf40fc30..d3f396f02e7a7 100644 --- a/Lib/test/test_email/test_message.py +++ b/Lib/test/test_email/test_message.py @@ -696,14 +696,16 @@ def subtype_as_add(self, method, subtype, outcome): self.assertIsNone(part['Content-Disposition']) class _TestSetRaisingContentManager: + class CustomError(Exception): + pass def set_content(self, msg, content, *args, **kw): - raise Exception('test') + raise self.CustomError('test') def test_default_content_manager_for_add_comes_from_policy(self): cm = self._TestSetRaisingContentManager() m = self.message(policy=self.policy.clone(content_manager=cm)) for method in ('add_related', 'add_alternative', 'add_attachment'): - with self.assertRaises(Exception) as ar: + with self.assertRaises(self._TestSetRaisingContentManager.CustomError) as ar: getattr(m, method)('') self.assertEqual(str(ar.exception), 'test') diff --git a/Lib/test/test_importlib/test_main.py b/Lib/test/test_importlib/test_main.py index 46cd2b696d4cc..81f683799cbc8 100644 --- a/Lib/test/test_importlib/test_main.py +++ b/Lib/test/test_importlib/test_main.py @@ -69,7 +69,7 @@ def test_abc_enforced(self): dict(name=''), ) def test_invalid_inputs_to_from_name(self, name): - with self.assertRaises(Exception): + with self.assertRaises(ValueError): Distribution.from_name(name) diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py index 4c592eaf34da2..4977a9369ddf8 100644 --- a/Lib/test/test_mailbox.py +++ b/Lib/test/test_mailbox.py @@ -116,10 +116,13 @@ def test_add_nonascii_string_header_raises(self): self.assertMailboxEmpty() def test_add_that_raises_leaves_mailbox_empty(self): + class CustomError(Exception): ... + exc_msg = "a fake error" + def raiser(*args, **kw): - raise Exception("a fake error") + raise CustomError(exc_msg) support.patch(self, email.generator.BytesGenerator, 'flatten', raiser) - with self.assertRaises(Exception): + with self.assertRaisesRegex(CustomError, exc_msg): self._box.add(email.message_from_string("From: Alph?so")) self.assertEqual(len(self._box), 0) self._box.close() diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 36f0b8a31a371..d74eee767c991 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -2739,7 +2739,7 @@ def test_regular_copy(self): def test_same_file(self): self.addCleanup(self.reset) with self.get_files() as (src, dst): - with self.assertRaises(Exception): + with self.assertRaises((OSError, _GiveupOnFastCopy)): self.zerocopy_fun(src, src) # Make sure src file is not corrupted. self.assertEqual(read_file(TESTFN, binary=True), self.FILEDATA) diff --git a/Lib/test/test_unittest/testmock/testasync.py b/Lib/test/test_unittest/testmock/testasync.py index 5f12f9f956674..edd9a5dfc1f89 100644 --- a/Lib/test/test_unittest/testmock/testasync.py +++ b/Lib/test/test_unittest/testmock/testasync.py @@ -436,9 +436,10 @@ async def addition(self, var): pass self.assertEqual(output, 10) async def test_add_side_effect_exception(self): + class CustomError(Exception): pass async def addition(var): pass - mock = AsyncMock(addition, side_effect=Exception('err')) - with self.assertRaises(Exception): + mock = AsyncMock(addition, side_effect=CustomError('side-effect')) + with self.assertRaisesRegex(CustomError, 'side-effect'): await mock(5) async def test_add_side_effect_coroutine(self): From webhook-mailer at python.org Fri Jul 7 18:02:17 2023 From: webhook-mailer at python.org (brettcannon) Date: Fri, 07 Jul 2023 22:02:17 -0000 Subject: [Python-checkins] gh-103200: Fix performance issues with `zipimport.invalidate_caches()` (GH-103208) Message-ID: https://github.com/python/cpython/commit/1fb9bd222bfe96cdf8a82701a3192e45d0811555 commit: 1fb9bd222bfe96cdf8a82701a3192e45d0811555 branch: main author: Desmond Cheong committer: brettcannon date: 2023-07-07T22:02:13Z summary: gh-103200: Fix performance issues with `zipimport.invalidate_caches()` (GH-103208) Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Brett Cannon files: A Misc/NEWS.d/next/Library/2023-04-03-08-09-40.gh-issue-103200.lq1Etz.rst M Lib/test/test_zipimport.py M Lib/zipimport.py diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py index 14c19719e260c..c12798d221e9b 100644 --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -520,10 +520,10 @@ def testInvalidateCaches(self): z.writestr(zinfo, data) zi = zipimport.zipimporter(TEMP_ZIP) - self.assertEqual(zi._files.keys(), files.keys()) + self.assertEqual(zi._get_files().keys(), files.keys()) # Check that the file information remains accurate after reloading zi.invalidate_caches() - self.assertEqual(zi._files.keys(), files.keys()) + self.assertEqual(zi._get_files().keys(), files.keys()) # Add a new file to the ZIP archive newfile = {"spam2" + pyc_ext: (NOW, test_pyc)} files.update(newfile) @@ -535,17 +535,54 @@ def testInvalidateCaches(self): z.writestr(zinfo, data) # Check that we can detect the new file after invalidating the cache zi.invalidate_caches() - self.assertEqual(zi._files.keys(), files.keys()) + self.assertEqual(zi._get_files().keys(), files.keys()) spec = zi.find_spec('spam2') self.assertIsNotNone(spec) self.assertIsInstance(spec.loader, zipimport.zipimporter) # Check that the cached data is removed if the file is deleted os.remove(TEMP_ZIP) zi.invalidate_caches() - self.assertFalse(zi._files) + self.assertFalse(zi._get_files()) self.assertIsNone(zipimport._zip_directory_cache.get(zi.archive)) self.assertIsNone(zi.find_spec("name_does_not_matter")) + def testInvalidateCachesWithMultipleZipimports(self): + packdir = TESTPACK + os.sep + packdir2 = packdir + TESTPACK2 + os.sep + files = {packdir + "__init__" + pyc_ext: (NOW, test_pyc), + packdir2 + "__init__" + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc), + "spam" + pyc_ext: (NOW, test_pyc)} + self.addCleanup(os_helper.unlink, TEMP_ZIP) + with ZipFile(TEMP_ZIP, "w") as z: + for name, (mtime, data) in files.items(): + zinfo = ZipInfo(name, time.localtime(mtime)) + zinfo.compress_type = self.compression + zinfo.comment = b"spam" + z.writestr(zinfo, data) + + zi = zipimport.zipimporter(TEMP_ZIP) + self.assertEqual(zi._get_files().keys(), files.keys()) + # Zipimporter for the same path. + zi2 = zipimport.zipimporter(TEMP_ZIP) + self.assertEqual(zi2._get_files().keys(), files.keys()) + # Add a new file to the ZIP archive to make the cache wrong. + newfile = {"spam2" + pyc_ext: (NOW, test_pyc)} + files.update(newfile) + with ZipFile(TEMP_ZIP, "a") as z: + for name, (mtime, data) in newfile.items(): + zinfo = ZipInfo(name, time.localtime(mtime)) + zinfo.compress_type = self.compression + zinfo.comment = b"spam" + z.writestr(zinfo, data) + # Invalidate the cache of the first zipimporter. + zi.invalidate_caches() + # Check that the second zipimporter detects the new file and isn't using a stale cache. + self.assertEqual(zi2._get_files().keys(), files.keys()) + spec = zi2.find_spec('spam2') + self.assertIsNotNone(spec) + self.assertIsInstance(spec.loader, zipimport.zipimporter) + def testZipImporterMethodsInSubDirectory(self): packdir = TESTPACK + os.sep packdir2 = packdir + TESTPACK2 + os.sep diff --git a/Lib/zipimport.py b/Lib/zipimport.py index a7333a4c4906d..5b9f614f02f7a 100644 --- a/Lib/zipimport.py +++ b/Lib/zipimport.py @@ -88,12 +88,8 @@ def __init__(self, path): raise ZipImportError('not a Zip file', path=path) break - try: - files = _zip_directory_cache[path] - except KeyError: - files = _read_directory(path) - _zip_directory_cache[path] = files - self._files = files + if path not in _zip_directory_cache: + _zip_directory_cache[path] = _read_directory(path) self.archive = path # a prefix directory following the ZIP file path. self.prefix = _bootstrap_external._path_join(*prefix[::-1]) @@ -152,7 +148,7 @@ def get_data(self, pathname): key = pathname[len(self.archive + path_sep):] try: - toc_entry = self._files[key] + toc_entry = self._get_files()[key] except KeyError: raise OSError(0, '', key) return _get_data(self.archive, toc_entry) @@ -189,7 +185,7 @@ def get_source(self, fullname): fullpath = f'{path}.py' try: - toc_entry = self._files[fullpath] + toc_entry = self._get_files()[fullpath] except KeyError: # we have the module, but no source return None @@ -268,14 +264,22 @@ def get_resource_reader(self, fullname): return ZipReader(self, fullname) - def invalidate_caches(self): - """Reload the file data of the archive path.""" + def _get_files(self): + """Return the files within the archive path.""" try: - self._files = _read_directory(self.archive) - _zip_directory_cache[self.archive] = self._files - except ZipImportError: - _zip_directory_cache.pop(self.archive, None) - self._files = {} + files = _zip_directory_cache[self.archive] + except KeyError: + try: + files = _zip_directory_cache[self.archive] = _read_directory(self.archive) + except ZipImportError: + files = {} + + return files + + + def invalidate_caches(self): + """Invalidates the cache of file data of the archive path.""" + _zip_directory_cache.pop(self.archive, None) def __repr__(self): @@ -305,15 +309,15 @@ def _is_dir(self, path): # of a namespace package. We test by seeing if the name, with an # appended path separator, exists. dirpath = path + path_sep - # If dirpath is present in self._files, we have a directory. - return dirpath in self._files + # If dirpath is present in self._get_files(), we have a directory. + return dirpath in self._get_files() # Return some information about a module. def _get_module_info(self, fullname): path = _get_module_path(self, fullname) for suffix, isbytecode, ispackage in _zip_searchorder: fullpath = path + suffix - if fullpath in self._files: + if fullpath in self._get_files(): return ispackage return None @@ -656,7 +660,7 @@ def _get_mtime_and_size_of_source(self, path): # strip 'c' or 'o' from *.py[co] assert path[-1:] in ('c', 'o') path = path[:-1] - toc_entry = self._files[path] + toc_entry = self._get_files()[path] # fetch the time stamp of the .py file for comparison # with an embedded pyc time stamp time = toc_entry[5] @@ -676,7 +680,7 @@ def _get_pyc_source(self, path): path = path[:-1] try: - toc_entry = self._files[path] + toc_entry = self._get_files()[path] except KeyError: return None else: @@ -692,7 +696,7 @@ def _get_module_code(self, fullname): fullpath = path + suffix _bootstrap._verbose_message('trying {}{}{}', self.archive, path_sep, fullpath, verbosity=2) try: - toc_entry = self._files[fullpath] + toc_entry = self._get_files()[fullpath] except KeyError: pass else: diff --git a/Misc/NEWS.d/next/Library/2023-04-03-08-09-40.gh-issue-103200.lq1Etz.rst b/Misc/NEWS.d/next/Library/2023-04-03-08-09-40.gh-issue-103200.lq1Etz.rst new file mode 100644 index 0000000000000..e264e0c81d3d9 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-04-03-08-09-40.gh-issue-103200.lq1Etz.rst @@ -0,0 +1 @@ +Fix cache repopulation semantics of zipimport.invalidate_caches(). The cache is now repopulated upon retrieving files with an invalid cache, not when the cache is invalidated. From webhook-mailer at python.org Fri Jul 7 18:03:54 2023 From: webhook-mailer at python.org (vstinner) Date: Fri, 07 Jul 2023 22:03:54 -0000 Subject: [Python-checkins] gh-106535: Document PEP 387 Soft Deprecation (#106536) Message-ID: https://github.com/python/cpython/commit/d524b6f61f0b9fe4c359373185bf08bab423a218 commit: d524b6f61f0b9fe4c359373185bf08bab423a218 branch: main author: Victor Stinner committer: vstinner date: 2023-07-08T00:03:51+02:00 summary: gh-106535: Document PEP 387 Soft Deprecation (#106536) Mark the optparse module as soft deprecated. files: M Doc/glossary.rst M Doc/library/optparse.rst diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 931360370d084..a4650a6c3efa2 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -1107,6 +1107,21 @@ Glossary when several are given, such as in ``variable_name[1:3:5]``. The bracket (subscript) notation uses :class:`slice` objects internally. + soft deprecated + A soft deprecation can be used when using an API which should no longer + be used to write new code, but it remains safe to continue using it in + existing code. The API remains documented and tested, but will not be + developed further (no enhancement). + + The main difference between a "soft" and a (regular) "hard" deprecation + is that the soft deprecation does not imply scheduling the removal of the + deprecated API. + + Another difference is that a soft deprecation does not issue a warning. + + See `PEP 387: Soft Deprecation + `_. + special method .. index:: pair: special; method diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index 0cff381745236..01177a04ab434 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -11,8 +11,9 @@ **Source code:** :source:`Lib/optparse.py` .. deprecated:: 3.2 - The :mod:`optparse` module is deprecated and will not be developed further; - development will continue with the :mod:`argparse` module. + The :mod:`optparse` module is :term:`soft deprecated` and will not be + developed further; development will continue with the :mod:`argparse` + module. -------------- From webhook-mailer at python.org Fri Jul 7 18:50:54 2023 From: webhook-mailer at python.org (vstinner) Date: Fri, 07 Jul 2023 22:50:54 -0000 Subject: [Python-checkins] gh-105373: Doc lists pending C API removals (#106537) Message-ID: https://github.com/python/cpython/commit/1b2938122d0bb97e802881321cf3ac87bb2bfaef commit: 1b2938122d0bb97e802881321cf3ac87bb2bfaef branch: main author: Victor Stinner committer: vstinner date: 2023-07-08T00:50:51+02:00 summary: gh-105373: Doc lists pending C API removals (#106537) files: M Doc/whatsnew/3.13.rst diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 8f737c5935f67..fbd34122ac572 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -470,6 +470,7 @@ Deprecated * :c:func:`Py_GetPythonHome`: get :c:member:`PyConfig.home` or :envvar:`PYTHONHOME` environment variable instead. + Functions scheduled for removal in Python 3.15. (Contributed by Victor Stinner in :gh:`105145`.) * Deprecate the :c:func:`PyImport_ImportModuleNoBlock` function which is just @@ -612,3 +613,82 @@ Removed `__ can be used to get :c:func:`PyInterpreterState_Get()` on Python 3.8 and older. (Contributed by Victor Stinner in :gh:`106320`.) + +Pending Removal in Python 3.15 +------------------------------ + +* :c:func:`PyImport_ImportModuleNoBlock`: use :c:func:`PyImport_ImportModule`. +* :c:func:`PyWeakref_GET_OBJECT`: use :c:func:`PyWeakref_GetRef` instead. +* :c:func:`PyWeakref_GetObject`: use :c:func:`PyWeakref_GetRef` instead. +* :c:type:`!Py_UNICODE_WIDE` type: use ``wchar_t`` instead. +* :c:type:`Py_UNICODE` type: use ``wchar_t`` instead. +* Python initialization functions: + + * :c:func:`PySys_ResetWarnOptions`: clear :data:`sys.warnoptions` and + :data:`!warnings.filters` instead. + * :c:func:`Py_GetExecPrefix`: get :data:`sys.exec_prefix` instead. + * :c:func:`Py_GetPath`: get :data:`sys.path` instead. + * :c:func:`Py_GetPrefix`: get :data:`sys.prefix` instead. + * :c:func:`Py_GetProgramFullPath`: get :data:`sys.executable` instead. + * :c:func:`Py_GetProgramName`: get :data:`sys.executable` instead. + * :c:func:`Py_GetPythonHome`: get :c:member:`PyConfig.home` or + :envvar:`PYTHONHOME` environment variable instead. + +Pending Removal in Future Versions +---------------------------------- + +The following APIs were deprecated in earlier Python versions and will be +removed, although there is currently no date scheduled for their removal. + +* :const:`Py_TPFLAGS_HAVE_FINALIZE`: no needed since Python 3.8. +* :c:func:`PyErr_Fetch`: use :c:func:`PyErr_GetRaisedException`. +* :c:func:`PyErr_NormalizeException`: use :c:func:`PyErr_GetRaisedException`. +* :c:func:`PyErr_Restore`: use :c:func:`PyErr_SetRaisedException`. +* :c:func:`PyModule_GetFilename`: use :c:func:`PyModule_GetFilenameObject`. +* :c:func:`PyOS_AfterFork`: use :c:func:`PyOS_AfterFork_Child()`. +* :c:func:`PySlice_GetIndicesEx`. +* :c:func:`!PyUnicode_AsDecodedObject`. +* :c:func:`!PyUnicode_AsDecodedUnicode`. +* :c:func:`!PyUnicode_AsEncodedObject`. +* :c:func:`!PyUnicode_AsEncodedUnicode`. +* :c:func:`PyUnicode_READY`: not needed since Python 3.12. +* :c:func:`!_PyErr_ChainExceptions`. +* :c:member:`!PyBytesObject.ob_shash` member: + call :c:func:`PyObject_Hash` instead. +* :c:member:`!PyDictObject.ma_version_tag` member. +* Global configuration variables: + + * :c:var:`Py_DebugFlag`: use :c:member:`PyConfig.parser_debug` + * :c:var:`Py_VerboseFlag`: use :c:member:`PyConfig.verbose` + * :c:var:`Py_QuietFlag`: use :c:member:`PyConfig.quiet` + * :c:var:`Py_InteractiveFlag`: use :c:member:`PyConfig.interactive` + * :c:var:`Py_InspectFlag`: use :c:member:`PyConfig.inspect` + * :c:var:`Py_OptimizeFlag`: use :c:member:`PyConfig.optimization_level` + * :c:var:`Py_NoSiteFlag`: use :c:member:`PyConfig.site_import` + * :c:var:`Py_BytesWarningFlag`: use :c:member:`PyConfig.bytes_warning` + * :c:var:`Py_FrozenFlag`: use :c:member:`PyConfig.pathconfig_warnings` + * :c:var:`Py_IgnoreEnvironmentFlag`: use :c:member:`PyConfig.use_environment` + * :c:var:`Py_DontWriteBytecodeFlag`: use :c:member:`PyConfig.write_bytecode` + * :c:var:`Py_NoUserSiteDirectory`: use :c:member:`PyConfig.user_site_directory` + * :c:var:`Py_UnbufferedStdioFlag`: use :c:member:`PyConfig.buffered_stdio` + * :c:var:`Py_HashRandomizationFlag`: use :c:member:`PyConfig.use_hash_seed` + and :c:member:`PyConfig.hash_seed` + * :c:var:`Py_IsolatedFlag`: use :c:member:`PyConfig.isolated` + * :c:var:`Py_LegacyWindowsFSEncodingFlag`: use :c:member:`PyPreConfig.legacy_windows_fs_encoding` + * :c:var:`Py_LegacyWindowsStdioFlag`: use :c:member:`PyConfig.legacy_windows_stdio` + * :c:var:`!Py_FileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding` + * :c:var:`!Py_HasFileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding` + * :c:var:`!Py_FileSystemDefaultEncodeErrors`: use :c:member:`PyConfig.filesystem_errors` + * :c:var:`!Py_UTF8Mode`: use :c:member:`PyPreConfig.utf8_mode` (see :c:func:`Py_PreInitialize`) + + The :c:func:`Py_InitializeFromConfig` API should be used with + :c:type:`PyConfig` instead. + +* TLS API: + + * :c:func:`PyThread_create_key`: use :c:func:`PyThread_tss_alloc`. + * :c:func:`PyThread_delete_key`: use :c:func:`PyThread_tss_free`. + * :c:func:`PyThread_set_key_value`: use :c:func:`PyThread_tss_set`. + * :c:func:`PyThread_get_key_value`: use :c:func:`PyThread_tss_get`. + * :c:func:`PyThread_delete_key_value`: use :c:func:`PyThread_tss_delete`. + * :c:func:`PyThread_ReInitTLS`: no longer needed. From webhook-mailer at python.org Fri Jul 7 19:49:23 2023 From: webhook-mailer at python.org (vstinner) Date: Fri, 07 Jul 2023 23:49:23 -0000 Subject: [Python-checkins] gh-105373: Remove C API global config vars in Python 3.14 (#106538) Message-ID: https://github.com/python/cpython/commit/ffe70c4d1c85f07d9b92bf0673e715fdaa2526ed commit: ffe70c4d1c85f07d9b92bf0673e715fdaa2526ed branch: main author: Victor Stinner committer: vstinner date: 2023-07-08T01:49:20+02:00 summary: gh-105373: Remove C API global config vars in Python 3.14 (#106538) Schedule the removal of C API global configuration variables in Python 3.14. Announce the removal to help C extension maintainers to upgrade their code. files: M Doc/c-api/init.rst M Doc/whatsnew/3.13.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 1dab0af2659b4..e7b2937d38dcf 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -87,7 +87,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-b` option. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_DebugFlag @@ -101,7 +101,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-d` option and the :envvar:`PYTHONDEBUG` environment variable. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_DontWriteBytecodeFlag @@ -115,7 +115,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-B` option and the :envvar:`PYTHONDONTWRITEBYTECODE` environment variable. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_FrozenFlag @@ -128,7 +128,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Private flag used by ``_freeze_module`` and ``frozenmain`` programs. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_HashRandomizationFlag @@ -143,7 +143,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. If the flag is non-zero, read the :envvar:`PYTHONHASHSEED` environment variable to initialize the secret hash seed. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_IgnoreEnvironmentFlag @@ -156,7 +156,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-E` and :option:`-I` options. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_InspectFlag @@ -171,7 +171,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-i` option and the :envvar:`PYTHONINSPECT` environment variable. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_InteractiveFlag @@ -196,7 +196,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. versionadded:: 3.4 - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_LegacyWindowsFSEncodingFlag @@ -215,7 +215,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. availability:: Windows. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_LegacyWindowsStdioFlag @@ -233,7 +233,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. availability:: Windows. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_NoSiteFlag @@ -248,7 +248,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-S` option. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_NoUserSiteDirectory @@ -262,7 +262,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-s` and :option:`-I` options, and the :envvar:`PYTHONNOUSERSITE` environment variable. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_OptimizeFlag @@ -273,7 +273,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-O` option and the :envvar:`PYTHONOPTIMIZE` environment variable. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_QuietFlag @@ -287,7 +287,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. versionadded:: 3.2 - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_UnbufferedStdioFlag @@ -300,7 +300,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-u` option and the :envvar:`PYTHONUNBUFFERED` environment variable. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 .. c:var:: int Py_VerboseFlag @@ -316,7 +316,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-v` option and the :envvar:`PYTHONVERBOSE` environment variable. - .. deprecated:: 3.12 + .. deprecated-removed:: 3.12 3.14 Initializing and finalizing the interpreter diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index fbd34122ac572..b376f846b725e 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -614,6 +614,37 @@ Removed :c:func:`PyInterpreterState_Get()` on Python 3.8 and older. (Contributed by Victor Stinner in :gh:`106320`.) +Pending Removal in Python 3.14 +------------------------------ + +* Global configuration variables: + + * :c:var:`Py_DebugFlag`: use :c:member:`PyConfig.parser_debug` + * :c:var:`Py_VerboseFlag`: use :c:member:`PyConfig.verbose` + * :c:var:`Py_QuietFlag`: use :c:member:`PyConfig.quiet` + * :c:var:`Py_InteractiveFlag`: use :c:member:`PyConfig.interactive` + * :c:var:`Py_InspectFlag`: use :c:member:`PyConfig.inspect` + * :c:var:`Py_OptimizeFlag`: use :c:member:`PyConfig.optimization_level` + * :c:var:`Py_NoSiteFlag`: use :c:member:`PyConfig.site_import` + * :c:var:`Py_BytesWarningFlag`: use :c:member:`PyConfig.bytes_warning` + * :c:var:`Py_FrozenFlag`: use :c:member:`PyConfig.pathconfig_warnings` + * :c:var:`Py_IgnoreEnvironmentFlag`: use :c:member:`PyConfig.use_environment` + * :c:var:`Py_DontWriteBytecodeFlag`: use :c:member:`PyConfig.write_bytecode` + * :c:var:`Py_NoUserSiteDirectory`: use :c:member:`PyConfig.user_site_directory` + * :c:var:`Py_UnbufferedStdioFlag`: use :c:member:`PyConfig.buffered_stdio` + * :c:var:`Py_HashRandomizationFlag`: use :c:member:`PyConfig.use_hash_seed` + and :c:member:`PyConfig.hash_seed` + * :c:var:`Py_IsolatedFlag`: use :c:member:`PyConfig.isolated` + * :c:var:`Py_LegacyWindowsFSEncodingFlag`: use :c:member:`PyPreConfig.legacy_windows_fs_encoding` + * :c:var:`Py_LegacyWindowsStdioFlag`: use :c:member:`PyConfig.legacy_windows_stdio` + * :c:var:`!Py_FileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding` + * :c:var:`!Py_HasFileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding` + * :c:var:`!Py_FileSystemDefaultEncodeErrors`: use :c:member:`PyConfig.filesystem_errors` + * :c:var:`!Py_UTF8Mode`: use :c:member:`PyPreConfig.utf8_mode` (see :c:func:`Py_PreInitialize`) + + The :c:func:`Py_InitializeFromConfig` API should be used with + :c:type:`PyConfig` instead. + Pending Removal in Python 3.15 ------------------------------ @@ -656,34 +687,6 @@ removed, although there is currently no date scheduled for their removal. * :c:member:`!PyBytesObject.ob_shash` member: call :c:func:`PyObject_Hash` instead. * :c:member:`!PyDictObject.ma_version_tag` member. -* Global configuration variables: - - * :c:var:`Py_DebugFlag`: use :c:member:`PyConfig.parser_debug` - * :c:var:`Py_VerboseFlag`: use :c:member:`PyConfig.verbose` - * :c:var:`Py_QuietFlag`: use :c:member:`PyConfig.quiet` - * :c:var:`Py_InteractiveFlag`: use :c:member:`PyConfig.interactive` - * :c:var:`Py_InspectFlag`: use :c:member:`PyConfig.inspect` - * :c:var:`Py_OptimizeFlag`: use :c:member:`PyConfig.optimization_level` - * :c:var:`Py_NoSiteFlag`: use :c:member:`PyConfig.site_import` - * :c:var:`Py_BytesWarningFlag`: use :c:member:`PyConfig.bytes_warning` - * :c:var:`Py_FrozenFlag`: use :c:member:`PyConfig.pathconfig_warnings` - * :c:var:`Py_IgnoreEnvironmentFlag`: use :c:member:`PyConfig.use_environment` - * :c:var:`Py_DontWriteBytecodeFlag`: use :c:member:`PyConfig.write_bytecode` - * :c:var:`Py_NoUserSiteDirectory`: use :c:member:`PyConfig.user_site_directory` - * :c:var:`Py_UnbufferedStdioFlag`: use :c:member:`PyConfig.buffered_stdio` - * :c:var:`Py_HashRandomizationFlag`: use :c:member:`PyConfig.use_hash_seed` - and :c:member:`PyConfig.hash_seed` - * :c:var:`Py_IsolatedFlag`: use :c:member:`PyConfig.isolated` - * :c:var:`Py_LegacyWindowsFSEncodingFlag`: use :c:member:`PyPreConfig.legacy_windows_fs_encoding` - * :c:var:`Py_LegacyWindowsStdioFlag`: use :c:member:`PyConfig.legacy_windows_stdio` - * :c:var:`!Py_FileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding` - * :c:var:`!Py_HasFileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding` - * :c:var:`!Py_FileSystemDefaultEncodeErrors`: use :c:member:`PyConfig.filesystem_errors` - * :c:var:`!Py_UTF8Mode`: use :c:member:`PyPreConfig.utf8_mode` (see :c:func:`Py_PreInitialize`) - - The :c:func:`Py_InitializeFromConfig` API should be used with - :c:type:`PyConfig` instead. - * TLS API: * :c:func:`PyThread_create_key`: use :c:func:`PyThread_tss_alloc`. From webhook-mailer at python.org Sat Jul 8 00:50:13 2023 From: webhook-mailer at python.org (gvanrossum) Date: Sat, 08 Jul 2023 04:50:13 -0000 Subject: [Python-checkins] Restore previous behavior of 'make regen-cases' (#106541) Message-ID: https://github.com/python/cpython/commit/48d5d32b80efe506e087e9b5a3302bf8df54aef4 commit: 48d5d32b80efe506e087e9b5a3302bf8df54aef4 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-07T21:50:09-07:00 summary: Restore previous behavior of 'make regen-cases' (#106541) When running 'make regen-cases' just to check whether anything changed, it's annoying that even if nothing changes, the output files are touched, causing an expensiv rebuild of _bootstrap_python and anything it creates. So use consistently for all output files. files: M Makefile.pre.in diff --git a/Makefile.pre.in b/Makefile.pre.in index 41623bd2f1da7..073b4bcc271ff 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1544,7 +1544,18 @@ regen-opcode-targets: regen-cases: # Regenerate various files from Python/bytecodes.c PYTHONPATH=$(srcdir)/Tools/cases_generator \ - $(PYTHON_FOR_REGEN) $(srcdir)/Tools/cases_generator/generate_cases.py -l + $(PYTHON_FOR_REGEN) \ + $(srcdir)/Tools/cases_generator/generate_cases.py \ + --emit-line-directives \ + -o $(srcdir)/Python/generated_cases.c.h.new \ + -m $(srcdir)/Python/opcode_metadata.h.new \ + -e $(srcdir)/Python/executor_cases.c.h.new \ + -p $(srcdir)/Lib/_opcode_metadata.py.new \ + $(srcdir)/Python/bytecodes.c + $(UPDATE_FILE) $(srcdir)/Python/generated_cases.c.h $(srcdir)/Python/generated_cases.c.h.new + $(UPDATE_FILE) $(srcdir)/Python/opcode_metadata.h $(srcdir)/Python/opcode_metadata.h.new + $(UPDATE_FILE) $(srcdir)/Python/executor_cases.c.h $(srcdir)/Python/executor_cases.c.h.new + $(UPDATE_FILE) $(srcdir)/Lib/_opcode_metadata.py $(srcdir)/Lib/_opcode_metadata.py.new Python/compile.o: $(srcdir)/Python/opcode_metadata.h From webhook-mailer at python.org Sat Jul 8 03:44:28 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sat, 08 Jul 2023 07:44:28 -0000 Subject: [Python-checkins] gh-106078: Move static objects related to `CONTEXTVAR` to the decimal module global state (#106395) Message-ID: https://github.com/python/cpython/commit/1c9e4934621627fbbfeada8d9dd87ecba4e446b0 commit: 1c9e4934621627fbbfeada8d9dd87ecba4e446b0 branch: main author: Charlie Zhao committer: kumaraditya303 <59607654+kumaraditya303 at users.noreply.github.com> date: 2023-07-08T13:14:24+05:30 summary: gh-106078: Move static objects related to `CONTEXTVAR` to the decimal module global state (#106395) Co-authored-by: Erlend E. Aasland files: M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/globals-to-fix.tsv diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 6da5095b91018..7e1809cfb98b2 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -39,6 +39,8 @@ #include "docstrings.h" +struct PyDecContextObject; + typedef struct { PyTypeObject *PyDecContextManager_Type; PyTypeObject *PyDecContext_Type; @@ -50,6 +52,15 @@ typedef struct { /* Top level Exception; inherits from ArithmeticError */ PyObject *DecimalException; +#ifndef WITH_DECIMAL_CONTEXTVAR + /* Key for thread state dictionary */ + PyObject *tls_context_key; + /* Invariant: NULL or the most recently accessed thread local context */ + struct PyDecContextObject *cached_context; +#else + PyObject *current_context_var; +#endif + /* Template for creating new thread contexts, calling Context() without * arguments and initializing the module_context on first access. */ PyObject *default_context_template; @@ -104,7 +115,7 @@ typedef struct { uint32_t *flags; } PyDecSignalDictObject; -typedef struct { +typedef struct PyDecContextObject { PyObject_HEAD mpd_context_t ctx; PyObject *traps; @@ -119,7 +130,6 @@ typedef struct { PyObject *global; } PyDecContextManagerObject; - #undef MPD #undef CTX #define PyDec_CheckExact(st, v) Py_IS_TYPE(v, (st)->PyDec_Type) @@ -145,16 +155,6 @@ incr_false(void) return Py_NewRef(Py_False); } - -#ifndef WITH_DECIMAL_CONTEXTVAR -/* Key for thread state dictionary */ -static PyObject *tls_context_key = NULL; -/* Invariant: NULL or the most recently accessed thread local context */ -static PyDecContextObject *cached_context = NULL; -#else -static PyObject *current_context_var = NULL; -#endif - /* Error codes for functions that return signals or conditions */ #define DEC_INVALID_SIGNALS (MPD_Max_status+1U) #define DEC_ERR_OCCURRED (DEC_INVALID_SIGNALS<<1) @@ -1565,7 +1565,8 @@ current_context_from_dict(void) return NULL; } - PyObject *tl_context = PyDict_GetItemWithError(dict, tls_context_key); + PyObject *tl_context; + tl_context = PyDict_GetItemWithError(dict, modstate->tls_context_key); if (tl_context != NULL) { /* We already have a thread local context. */ CONTEXT_CHECK(modstate, tl_context); @@ -1576,13 +1577,13 @@ current_context_from_dict(void) } /* Set up a new thread local context. */ - tl_context = context_copy(state->default_context_template, NULL); + tl_context = context_copy(modstate->default_context_template, NULL); if (tl_context == NULL) { return NULL; } CTX(tl_context)->status = 0; - if (PyDict_SetItem(dict, tls_context_key, tl_context) < 0) { + if (PyDict_SetItem(dict, modstate->tls_context_key, tl_context) < 0) { Py_DECREF(tl_context); return NULL; } @@ -1591,8 +1592,8 @@ current_context_from_dict(void) /* Cache the context of the current thread, assuming that it * will be accessed several times before a thread switch. */ - cached_context = (PyDecContextObject *)tl_context; - cached_context->tstate = tstate; + modstate->cached_context = (PyDecContextObject *)tl_context; + modstate->cached_context->tstate = tstate; /* Borrowed reference with refcount==1 */ return tl_context; @@ -1603,8 +1604,9 @@ static PyObject * current_context(void) { PyThreadState *tstate = _PyThreadState_GET(); - if (cached_context && cached_context->tstate == tstate) { - return (PyObject *)cached_context; + decimal_state *modstate = GLOBAL_STATE(); + if (modstate->cached_context && modstate->cached_context->tstate == tstate) { + return (PyObject *)(modstate->cached_context); } return current_context_from_dict(); @@ -1662,8 +1664,8 @@ PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) Py_INCREF(v); } - cached_context = NULL; - if (PyDict_SetItem(dict, tls_context_key, v) < 0) { + state->cached_context = NULL; + if (PyDict_SetItem(dict, state->tls_context_key, v) < 0) { Py_DECREF(v); return NULL; } @@ -1682,7 +1684,7 @@ init_current_context(void) } CTX(tl_context)->status = 0; - PyObject *tok = PyContextVar_Set(current_context_var, tl_context); + PyObject *tok = PyContextVar_Set(state->current_context_var, tl_context); if (tok == NULL) { Py_DECREF(tl_context); return NULL; @@ -1696,7 +1698,8 @@ static inline PyObject * current_context(void) { PyObject *tl_context; - if (PyContextVar_Get(current_context_var, NULL, &tl_context) < 0) { + decimal_state *state = GLOBAL_STATE(); + if (PyContextVar_Get(state->current_context_var, NULL, &tl_context) < 0) { return NULL; } @@ -1744,7 +1747,7 @@ PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) Py_INCREF(v); } - PyObject *tok = PyContextVar_Set(current_context_var, v); + PyObject *tok = PyContextVar_Set(state->current_context_var, v); Py_DECREF(v); if (tok == NULL) { return NULL; @@ -5987,10 +5990,11 @@ PyInit__decimal(void) Py_NewRef(state->default_context_template))); #ifndef WITH_DECIMAL_CONTEXTVAR - ASSIGN_PTR(tls_context_key, PyUnicode_FromString("___DECIMAL_CTX__")); + ASSIGN_PTR(state->tls_context_key, + PyUnicode_FromString("___DECIMAL_CTX__")); CHECK_INT(PyModule_AddObject(m, "HAVE_CONTEXTVAR", Py_NewRef(Py_False))); #else - ASSIGN_PTR(current_context_var, PyContextVar_New("decimal_context", NULL)); + ASSIGN_PTR(state->current_context_var, PyContextVar_New("decimal_context", NULL)); CHECK_INT(PyModule_AddObject(m, "HAVE_CONTEXTVAR", Py_NewRef(Py_True))); #endif CHECK_INT(PyModule_AddObject(m, "HAVE_THREADS", Py_NewRef(Py_True))); @@ -6049,9 +6053,9 @@ PyInit__decimal(void) Py_CLEAR(state->DecimalTuple); /* GCOV_NOT_REACHED */ Py_CLEAR(state->default_context_template); /* GCOV_NOT_REACHED */ #ifndef WITH_DECIMAL_CONTEXTVAR - Py_CLEAR(tls_context_key); /* GCOV_NOT_REACHED */ + Py_CLEAR(state->tls_context_key); /* GCOV_NOT_REACHED */ #else - Py_CLEAR(current_context_var); /* GCOV_NOT_REACHED */ + Py_CLEAR(state->current_context_var); /* GCOV_NOT_REACHED */ #endif Py_CLEAR(state->basic_context_template); /* GCOV_NOT_REACHED */ Py_CLEAR(state->extended_context_template); /* GCOV_NOT_REACHED */ diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index ad3d9b6513d69..12f91726b2d2f 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -422,7 +422,6 @@ Modules/_datetimemodule.c - us_per_day - Modules/_datetimemodule.c - us_per_week - Modules/_datetimemodule.c - seconds_per_day - Modules/_decimal/_decimal.c - global_state - -Modules/_decimal/_decimal.c - current_context_var - Modules/_decimal/_decimal.c - round_map - Modules/_decimal/_decimal.c - Rational - Modules/_decimal/_decimal.c - SignalTuple - From webhook-mailer at python.org Sat Jul 8 03:47:04 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 08 Jul 2023 07:47:04 -0000 Subject: [Python-checkins] gh-106524: Fix a crash in _sre.template() (GH-106525) Message-ID: https://github.com/python/cpython/commit/2ef1dc37f02b08536b677dd23ec51541a60effd7 commit: 2ef1dc37f02b08536b677dd23ec51541a60effd7 branch: main author: Radislav Chugunov <52372310+chgnrdv at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-08T10:47:01+03:00 summary: gh-106524: Fix a crash in _sre.template() (GH-106525) Some items remained uninitialized if _sre.template() was called with invalid indices. Then attempt to clear them in the destructor led to dereferencing of uninitialized pointer. files: A Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst M Lib/test/test_re.py M Modules/_sre/sre.c diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py index e4d14356402db..8f26d0aacf4ee 100644 --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -2418,6 +2418,16 @@ def test_regression_gh94675(self): p.terminate() p.join() + def test_sre_template_invalid_group_index(self): + # see gh-106524 + import _sre + with self.assertRaises(TypeError) as cm: + _sre.template("", ["", -1, ""]) + self.assertIn("invalid template", str(cm.exception)) + with self.assertRaises(TypeError) as cm: + _sre.template("", ["", (), ""]) + self.assertIn("an integer is required", str(cm.exception)) + def get_debug_out(pat): with captured_stdout() as out: diff --git a/Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst b/Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst new file mode 100644 index 0000000000000..f3fd070e391a6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst @@ -0,0 +1 @@ +Fix crash in :func:`!_sre.template` with templates containing invalid group indices. diff --git a/Modules/_sre/sre.c b/Modules/_sre/sre.c index 3f11916f9e172..98602b4c1eaaf 100644 --- a/Modules/_sre/sre.c +++ b/Modules/_sre/sre.c @@ -1544,10 +1544,12 @@ _sre_template_impl(PyObject *module, PyObject *pattern, PyObject *template) for (Py_ssize_t i = 0; i < n; i++) { Py_ssize_t index = PyLong_AsSsize_t(PyList_GET_ITEM(template, 2*i+1)); if (index == -1 && PyErr_Occurred()) { + Py_SET_SIZE(self, i); Py_DECREF(self); return NULL; } if (index < 0) { + Py_SET_SIZE(self, i); goto bad_template; } self->items[i].index = index; From webhook-mailer at python.org Sat Jul 8 04:12:36 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 08 Jul 2023 08:12:36 -0000 Subject: [Python-checkins] [3.12] gh-106524: Fix a crash in _sre.template() (GH-106525) (GH-106544) Message-ID: https://github.com/python/cpython/commit/8fdb0587935b66c3942c1740ab5e387b2ea415bb commit: 8fdb0587935b66c3942c1740ab5e387b2ea415bb branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-08T08:12:33Z summary: [3.12] gh-106524: Fix a crash in _sre.template() (GH-106525) (GH-106544) Some items remained uninitialized if _sre.template() was called with invalid indices. Then attempt to clear them in the destructor led to dereferencing of uninitialized pointer. (cherry picked from commit 2ef1dc37f02b08536b677dd23ec51541a60effd7) Co-authored-by: Radislav Chugunov <52372310+chgnrdv at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst M Lib/test/test_re.py M Modules/_sre/sre.c diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py index 11628a236ade9..a05a1c99def0b 100644 --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -2441,6 +2441,16 @@ def test_regression_gh94675(self): p.terminate() p.join() + def test_sre_template_invalid_group_index(self): + # see gh-106524 + import _sre + with self.assertRaises(TypeError) as cm: + _sre.template("", ["", -1, ""]) + self.assertIn("invalid template", str(cm.exception)) + with self.assertRaises(TypeError) as cm: + _sre.template("", ["", (), ""]) + self.assertIn("an integer is required", str(cm.exception)) + def get_debug_out(pat): with captured_stdout() as out: diff --git a/Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst b/Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst new file mode 100644 index 0000000000000..f3fd070e391a6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst @@ -0,0 +1 @@ +Fix crash in :func:`!_sre.template` with templates containing invalid group indices. diff --git a/Modules/_sre/sre.c b/Modules/_sre/sre.c index f8a1a05a31888..2f1c7324a0fa4 100644 --- a/Modules/_sre/sre.c +++ b/Modules/_sre/sre.c @@ -1549,10 +1549,12 @@ _sre_template_impl(PyObject *module, PyObject *pattern, PyObject *template) for (Py_ssize_t i = 0; i < n; i++) { Py_ssize_t index = PyLong_AsSsize_t(PyList_GET_ITEM(template, 2*i+1)); if (index == -1 && PyErr_Occurred()) { + Py_SET_SIZE(self, i); Py_DECREF(self); return NULL; } if (index < 0) { + Py_SET_SIZE(self, i); goto bad_template; } self->items[i].index = index; From webhook-mailer at python.org Sat Jul 8 04:22:36 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 08 Jul 2023 08:22:36 -0000 Subject: [Python-checkins] [3.11] gh-106300: Improve `assertRaises(Exception)` usages in tests (GH-106302). (GH-106545) Message-ID: https://github.com/python/cpython/commit/6cd08a566f11ff6862ccf8637d01b179da46515a commit: 6cd08a566f11ff6862ccf8637d01b179da46515a branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-08T08:22:33Z summary: [3.11] gh-106300: Improve `assertRaises(Exception)` usages in tests (GH-106302). (GH-106545) (cherry picked from commit 6e6a4cd52332017b10c8d88fbbbfe015948093f4) Co-authored-by: Nikita Sobolev files: M Lib/test/test_abc.py M Lib/test/test_codecs.py M Lib/test/test_email/test_message.py M Lib/test/test_importlib/test_main.py M Lib/test/test_mailbox.py M Lib/test/test_shutil.py M Lib/unittest/test/testmock/testasync.py diff --git a/Lib/test/test_abc.py b/Lib/test/test_abc.py index 1e7a0351db489..ed2e0d160e3a8 100644 --- a/Lib/test/test_abc.py +++ b/Lib/test/test_abc.py @@ -448,15 +448,16 @@ class S(metaclass=abc_ABCMeta): # Also check that issubclass() propagates exceptions raised by # __subclasses__. + class CustomError(Exception): ... exc_msg = "exception from __subclasses__" def raise_exc(): - raise Exception(exc_msg) + raise CustomError(exc_msg) class S(metaclass=abc_ABCMeta): __subclasses__ = raise_exc - with self.assertRaisesRegex(Exception, exc_msg): + with self.assertRaisesRegex(CustomError, exc_msg): issubclass(int, S) def test_subclasshook(self): diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index 934e4bb347ef3..e170a30263f8b 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2823,15 +2823,16 @@ def test_binary_to_text_denylists_text_transforms(self): def test_custom_zlib_error_is_wrapped(self): # Check zlib codec gives a good error for malformed input msg = "^decoding with 'zlib_codec' codec failed" - with self.assertRaisesRegex(Exception, msg) as failure: + with self.assertRaises(zlib.error) as failure: codecs.decode(b"hello", "zlib_codec") self.assertIsInstance(failure.exception.__cause__, type(failure.exception)) def test_custom_hex_error_is_wrapped(self): # Check hex codec gives a good error for malformed input + import binascii msg = "^decoding with 'hex_codec' codec failed" - with self.assertRaisesRegex(Exception, msg) as failure: + with self.assertRaises(binascii.Error) as failure: codecs.decode(b"hello", "hex_codec") self.assertIsInstance(failure.exception.__cause__, type(failure.exception)) diff --git a/Lib/test/test_email/test_message.py b/Lib/test/test_email/test_message.py index 4c754bf40fc30..d3f396f02e7a7 100644 --- a/Lib/test/test_email/test_message.py +++ b/Lib/test/test_email/test_message.py @@ -696,14 +696,16 @@ def subtype_as_add(self, method, subtype, outcome): self.assertIsNone(part['Content-Disposition']) class _TestSetRaisingContentManager: + class CustomError(Exception): + pass def set_content(self, msg, content, *args, **kw): - raise Exception('test') + raise self.CustomError('test') def test_default_content_manager_for_add_comes_from_policy(self): cm = self._TestSetRaisingContentManager() m = self.message(policy=self.policy.clone(content_manager=cm)) for method in ('add_related', 'add_alternative', 'add_attachment'): - with self.assertRaises(Exception) as ar: + with self.assertRaises(self._TestSetRaisingContentManager.CustomError) as ar: getattr(m, method)('') self.assertEqual(str(ar.exception), 'test') diff --git a/Lib/test/test_importlib/test_main.py b/Lib/test/test_importlib/test_main.py index d9d067c4b23d6..7ef442f3407c4 100644 --- a/Lib/test/test_importlib/test_main.py +++ b/Lib/test/test_importlib/test_main.py @@ -56,7 +56,7 @@ def test_new_style_classes(self): dict(name=''), ) def test_invalid_inputs_to_from_name(self, name): - with self.assertRaises(Exception): + with self.assertRaises(ValueError): Distribution.from_name(name) diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py index 07c2764dfd1b2..90642a9a8a810 100644 --- a/Lib/test/test_mailbox.py +++ b/Lib/test/test_mailbox.py @@ -116,10 +116,13 @@ def test_add_nonascii_string_header_raises(self): self.assertMailboxEmpty() def test_add_that_raises_leaves_mailbox_empty(self): + class CustomError(Exception): ... + exc_msg = "a fake error" + def raiser(*args, **kw): - raise Exception("a fake error") + raise CustomError(exc_msg) support.patch(self, email.generator.BytesGenerator, 'flatten', raiser) - with self.assertRaises(Exception): + with self.assertRaisesRegex(CustomError, exc_msg): self._box.add(email.message_from_string("From: Alph?so")) self.assertEqual(len(self._box), 0) self._box.close() diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 9bf41453667ac..bd82aa54368eb 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -2451,7 +2451,7 @@ def test_regular_copy(self): def test_same_file(self): self.addCleanup(self.reset) with self.get_files() as (src, dst): - with self.assertRaises(Exception): + with self.assertRaises((OSError, _GiveupOnFastCopy)): self.zerocopy_fun(src, src) # Make sure src file is not corrupted. self.assertEqual(read_file(TESTFN, binary=True), self.FILEDATA) diff --git a/Lib/unittest/test/testmock/testasync.py b/Lib/unittest/test/testmock/testasync.py index df260abde950b..b7c5d6dba579f 100644 --- a/Lib/unittest/test/testmock/testasync.py +++ b/Lib/unittest/test/testmock/testasync.py @@ -427,9 +427,10 @@ async def addition(self, var): pass self.assertEqual(output, 10) async def test_add_side_effect_exception(self): + class CustomError(Exception): pass async def addition(var): pass - mock = AsyncMock(addition, side_effect=Exception('err')) - with self.assertRaises(Exception): + mock = AsyncMock(addition, side_effect=CustomError('side-effect')) + with self.assertRaisesRegex(CustomError, 'side-effect'): await mock(5) async def test_add_side_effect_coroutine(self): From webhook-mailer at python.org Sat Jul 8 04:44:54 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sat, 08 Jul 2023 08:44:54 -0000 Subject: [Python-checkins] gh-105873: Make `_xxsubinterpreters` use exception type name in shared exception (#105874) Message-ID: https://github.com/python/cpython/commit/69a39bd9ad52241ca0e9a1926b4536c73017d067 commit: 69a39bd9ad52241ca0e9a1926b4536c73017d067 branch: main author: Radislav Chugunov <52372310+chgnrdv at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-08T08:44:50Z summary: gh-105873: Make `_xxsubinterpreters` use exception type name in shared exception (#105874) files: M Lib/test/test__xxsubinterpreters.py M Lib/test/test_importlib/test_util.py M Modules/_xxsubinterpretersmodule.c diff --git a/Lib/test/test__xxsubinterpreters.py b/Lib/test/test__xxsubinterpreters.py index 1ee18774d1720..ac2280eb7090e 100644 --- a/Lib/test/test__xxsubinterpreters.py +++ b/Lib/test/test__xxsubinterpreters.py @@ -731,10 +731,10 @@ def assert_run_failed(self, exctype, msg=None): yield if msg is None: self.assertEqual(str(caught.exception).split(':')[0], - str(exctype)) + exctype.__name__) else: self.assertEqual(str(caught.exception), - "{}: {}".format(exctype, msg)) + "{}: {}".format(exctype.__name__, msg)) def test_invalid_syntax(self): with self.assert_run_failed(SyntaxError): diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py index e967adc9451c8..5da72a21f586e 100644 --- a/Lib/test/test_importlib/test_util.py +++ b/Lib/test/test_importlib/test_util.py @@ -655,7 +655,7 @@ def test_magic_number(self): @unittest.skipIf(_interpreters is None, 'subinterpreters required') class IncompatibleExtensionModuleRestrictionsTests(unittest.TestCase): - ERROR = re.compile("^: module (.*) does not support loading in subinterpreters") + ERROR = re.compile("^ImportError: module (.*) does not support loading in subinterpreters") def run_with_own_gil(self, script): interpid = _interpreters.create(isolated=True) diff --git a/Modules/_xxsubinterpretersmodule.c b/Modules/_xxsubinterpretersmodule.c index 40dea170fd1f8..d2e0593872c5f 100644 --- a/Modules/_xxsubinterpretersmodule.c +++ b/Modules/_xxsubinterpretersmodule.c @@ -273,7 +273,7 @@ _sharedexception_bind(PyObject *exc, _sharedexception *sharedexc) assert(exc != NULL); const char *failure = NULL; - PyObject *nameobj = PyUnicode_FromFormat("%S", Py_TYPE(exc)); + PyObject *nameobj = PyUnicode_FromString(Py_TYPE(exc)->tp_name); if (nameobj == NULL) { failure = "unable to format exception type name"; goto error; From webhook-mailer at python.org Sat Jul 8 04:48:36 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sat, 08 Jul 2023 08:48:36 -0000 Subject: [Python-checkins] gh-101880: add link to object.__hash__() in hash() builtin documentation (#101883) Message-ID: https://github.com/python/cpython/commit/ec7180bd1b3c156d4484e8e6babc5ecb707420e3 commit: ec7180bd1b3c156d4484e8e6babc5ecb707420e3 branch: main author: Owain Davies <116417456+OTheDev at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-08T08:48:33Z summary: gh-101880: add link to object.__hash__() in hash() builtin documentation (#101883) files: M Doc/library/functions.rst diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 9b9731e918985..d8091f0b093aa 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -794,7 +794,7 @@ are always available. They are listed here in alphabetical order. For objects with custom :meth:`__hash__` methods, note that :func:`hash` truncates the return value based on the bit width of the host machine. - See :meth:`__hash__` for details. + See :meth:`__hash__ ` for details. .. function:: help() help(request) From webhook-mailer at python.org Sat Jul 8 04:52:53 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sat, 08 Jul 2023 08:52:53 -0000 Subject: [Python-checkins] [3.12] gh-101880: add link to object.__hash__() in hash() builtin documentation (GH-101883) (#106546) Message-ID: https://github.com/python/cpython/commit/48f58e02430f046d4b039cefce3856b934b02627 commit: 48f58e02430f046d4b039cefce3856b934b02627 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-08T14:22:50+05:30 summary: [3.12] gh-101880: add link to object.__hash__() in hash() builtin documentation (GH-101883) (#106546) gh-101880: add link to object.__hash__() in hash() builtin documentation (GH-101883) (cherry picked from commit ec7180bd1b3c156d4484e8e6babc5ecb707420e3) Co-authored-by: Owain Davies <116417456+OTheDev at users.noreply.github.com> files: M Doc/library/functions.rst diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 9b9731e918985..d8091f0b093aa 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -794,7 +794,7 @@ are always available. They are listed here in alphabetical order. For objects with custom :meth:`__hash__` methods, note that :func:`hash` truncates the return value based on the bit width of the host machine. - See :meth:`__hash__` for details. + See :meth:`__hash__ ` for details. .. function:: help() help(request) From webhook-mailer at python.org Sat Jul 8 04:53:04 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sat, 08 Jul 2023 08:53:04 -0000 Subject: [Python-checkins] [3.11] gh-101880: add link to object.__hash__() in hash() builtin documentation (GH-101883) (#106547) Message-ID: https://github.com/python/cpython/commit/44c335e47d63510912638bd8b74afd8abb9ddcde commit: 44c335e47d63510912638bd8b74afd8abb9ddcde branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-08T14:23:00+05:30 summary: [3.11] gh-101880: add link to object.__hash__() in hash() builtin documentation (GH-101883) (#106547) gh-101880: add link to object.__hash__() in hash() builtin documentation (GH-101883) (cherry picked from commit ec7180bd1b3c156d4484e8e6babc5ecb707420e3) Co-authored-by: Owain Davies <116417456+OTheDev at users.noreply.github.com> files: M Doc/library/functions.rst diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 2a0e456d869f0..3d7e6603a7f35 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -793,7 +793,7 @@ are always available. They are listed here in alphabetical order. For objects with custom :meth:`__hash__` methods, note that :func:`hash` truncates the return value based on the bit width of the host machine. - See :meth:`__hash__` for details. + See :meth:`__hash__ ` for details. .. function:: help() help(request) From webhook-mailer at python.org Sat Jul 8 07:31:28 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 08 Jul 2023 11:31:28 -0000 Subject: [Python-checkins] gh-106510: Fix DEBUG output for atomic group (GH-106511) Message-ID: https://github.com/python/cpython/commit/74ec02e9490d8aa086aa9ad9d1d34d2ad999b5af commit: 74ec02e9490d8aa086aa9ad9d1d34d2ad999b5af branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-08T14:31:25+03:00 summary: gh-106510: Fix DEBUG output for atomic group (GH-106511) files: A Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst M Lib/re/_parser.py M Lib/test/test_re.py diff --git a/Lib/re/_parser.py b/Lib/re/_parser.py index 6c8a4eccc0e0b..22d10ab6e31d3 100644 --- a/Lib/re/_parser.py +++ b/Lib/re/_parser.py @@ -113,7 +113,6 @@ def __init__(self, state, data=None): self.width = None def dump(self, level=0): - nl = True seqtypes = (tuple, list) for op, av in self.data: print(level*" " + str(op), end='') @@ -135,6 +134,9 @@ def dump(self, level=0): if item_no: print(level*" " + "ELSE") item_no.dump(level+1) + elif isinstance(av, SubPattern): + print() + av.dump(level+1) elif isinstance(av, seqtypes): nl = False for a in av: diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py index 8f26d0aacf4ee..b1699b0f83dbd 100644 --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -2489,7 +2489,10 @@ def test_debug_flag(self): def test_atomic_group(self): self.assertEqual(get_debug_out(r'(?>ab?)'), '''\ -ATOMIC_GROUP [(LITERAL, 97), (MAX_REPEAT, (0, 1, [(LITERAL, 98)]))] +ATOMIC_GROUP + LITERAL 97 + MAX_REPEAT 0 1 + LITERAL 98 0. INFO 4 0b0 1 2 (to 5) 5: ATOMIC_GROUP 11 (to 17) diff --git a/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst b/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst new file mode 100644 index 0000000000000..e0646fa9bc021 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst @@ -0,0 +1 @@ +Improve debug output for atomic groups in regular expressions. From webhook-mailer at python.org Sat Jul 8 08:15:25 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 08 Jul 2023 12:15:25 -0000 Subject: [Python-checkins] [3.11] gh-106510: Fix DEBUG output for atomic group (GH-106511) (GH-106549) Message-ID: https://github.com/python/cpython/commit/2d037fb406fd8662862c5da40a23033690235f1d commit: 2d037fb406fd8662862c5da40a23033690235f1d branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-08T15:15:22+03:00 summary: [3.11] gh-106510: Fix DEBUG output for atomic group (GH-106511) (GH-106549) (cherry picked from commit 74ec02e9490d8aa086aa9ad9d1d34d2ad999b5af) files: A Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst M Lib/re/_parser.py M Lib/test/test_re.py diff --git a/Lib/re/_parser.py b/Lib/re/_parser.py index f747a0396b1f3..6b715f54032f9 100644 --- a/Lib/re/_parser.py +++ b/Lib/re/_parser.py @@ -114,7 +114,6 @@ def __init__(self, state, data=None): self.width = None def dump(self, level=0): - nl = True seqtypes = (tuple, list) for op, av in self.data: print(level*" " + str(op), end='') @@ -136,6 +135,9 @@ def dump(self, level=0): if item_no: print(level*" " + "ELSE") item_no.dump(level+1) + elif isinstance(av, SubPattern): + print() + av.dump(level+1) elif isinstance(av, seqtypes): nl = False for a in av: diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py index 59d0b7b5c4896..167b4400f34c0 100644 --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -2533,7 +2533,10 @@ def test_debug_flag(self): def test_atomic_group(self): self.assertEqual(get_debug_out(r'(?>ab?)'), '''\ -ATOMIC_GROUP [(LITERAL, 97), (MAX_REPEAT, (0, 1, [(LITERAL, 98)]))] +ATOMIC_GROUP + LITERAL 97 + MAX_REPEAT 0 1 + LITERAL 98 0. INFO 4 0b0 1 2 (to 5) 5: ATOMIC_GROUP 11 (to 17) diff --git a/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst b/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst new file mode 100644 index 0000000000000..e0646fa9bc021 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst @@ -0,0 +1 @@ +Improve debug output for atomic groups in regular expressions. From webhook-mailer at python.org Sat Jul 8 08:15:58 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 08 Jul 2023 12:15:58 -0000 Subject: [Python-checkins] [3.12] gh-106510: Fix DEBUG output for atomic group (GH-106511) (GH-106548) Message-ID: https://github.com/python/cpython/commit/60ade0cb2929b180a50feb288a760038a072ef76 commit: 60ade0cb2929b180a50feb288a760038a072ef76 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-08T15:15:54+03:00 summary: [3.12] gh-106510: Fix DEBUG output for atomic group (GH-106511) (GH-106548) (cherry picked from commit 74ec02e9490d8aa086aa9ad9d1d34d2ad999b5af) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst M Lib/re/_parser.py M Lib/test/test_re.py diff --git a/Lib/re/_parser.py b/Lib/re/_parser.py index 5709acb626723..74bda2f103083 100644 --- a/Lib/re/_parser.py +++ b/Lib/re/_parser.py @@ -114,7 +114,6 @@ def __init__(self, state, data=None): self.width = None def dump(self, level=0): - nl = True seqtypes = (tuple, list) for op, av in self.data: print(level*" " + str(op), end='') @@ -136,6 +135,9 @@ def dump(self, level=0): if item_no: print(level*" " + "ELSE") item_no.dump(level+1) + elif isinstance(av, SubPattern): + print() + av.dump(level+1) elif isinstance(av, seqtypes): nl = False for a in av: diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py index a05a1c99def0b..406b665c192df 100644 --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -2512,7 +2512,10 @@ def test_debug_flag(self): def test_atomic_group(self): self.assertEqual(get_debug_out(r'(?>ab?)'), '''\ -ATOMIC_GROUP [(LITERAL, 97), (MAX_REPEAT, (0, 1, [(LITERAL, 98)]))] +ATOMIC_GROUP + LITERAL 97 + MAX_REPEAT 0 1 + LITERAL 98 0. INFO 4 0b0 1 2 (to 5) 5: ATOMIC_GROUP 11 (to 17) diff --git a/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst b/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst new file mode 100644 index 0000000000000..e0646fa9bc021 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst @@ -0,0 +1 @@ +Improve debug output for atomic groups in regular expressions. From webhook-mailer at python.org Sat Jul 8 11:00:42 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 08 Jul 2023 15:00:42 -0000 Subject: [Python-checkins] gh-106508: Improve debugging of the _sre module (GH-106509) Message-ID: https://github.com/python/cpython/commit/b305c69d1085c9e6c6875559109f73b827cb6fe0 commit: b305c69d1085c9e6c6875559109f73b827cb6fe0 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-08T18:00:39+03:00 summary: gh-106508: Improve debugging of the _sre module (GH-106509) Now the VERBOSE macro can control tracing on per-pattern basis: * 0 -- disabled * 1 -- only if the DEBUG flag set * 2 -- always files: M Modules/_sre/sre.c M Modules/_sre/sre.h M Modules/_sre/sre_lib.h diff --git a/Modules/_sre/sre.c b/Modules/_sre/sre.c index 98602b4c1eaaf..f34a353432dec 100644 --- a/Modules/_sre/sre.c +++ b/Modules/_sre/sre.c @@ -49,8 +49,14 @@ static const char copyright[] = #include -/* defining this one enables tracing */ -#undef VERBOSE +/* Defining this one controls tracing: + * 0 -- disabled + * 1 -- only if the DEBUG flag set + * 2 -- always + */ +#ifndef VERBOSE +# define VERBOSE 0 +#endif /* -------------------------------------------------------------------- */ @@ -70,10 +76,21 @@ static const char copyright[] = #define SRE_ERROR_MEMORY -9 /* out of memory */ #define SRE_ERROR_INTERRUPTED -10 /* signal handler raised exception */ -#if defined(VERBOSE) -#define TRACE(v) printf v +#if VERBOSE == 0 +# define INIT_TRACE(state) +# define TRACE(v) +#elif VERBOSE == 1 +# define INIT_TRACE(state) int _debug = (state)->debug +# define TRACE(v) do { \ + if (_debug) { \ + printf v; \ + } \ + } while (0) +#elif VERBOSE == 2 +# define INIT_TRACE(state) +# define TRACE(v) printf v #else -#define TRACE(v) +# error VERBOSE must be 0, 1 or 2 #endif /* -------------------------------------------------------------------- */ @@ -198,6 +215,7 @@ data_stack_dealloc(SRE_STATE* state) static int data_stack_grow(SRE_STATE* state, Py_ssize_t size) { + INIT_TRACE(state); Py_ssize_t minsize, cursize; minsize = state->data_stack_base+size; cursize = state->data_stack_size; @@ -449,6 +467,7 @@ state_init(SRE_STATE* state, PatternObject* pattern, PyObject* string, state->charsize = charsize; state->match_all = 0; state->must_advance = 0; + state->debug = ((pattern->flags & SRE_FLAG_DEBUG) != 0); state->beginning = ptr; @@ -641,6 +660,7 @@ _sre_SRE_Pattern_match_impl(PatternObject *self, PyTypeObject *cls, if (!state_init(&state, (PatternObject *)self, string, pos, endpos)) return NULL; + INIT_TRACE(&state); state.ptr = state.start; TRACE(("|%p|%p|MATCH\n", PatternObject_GetCode(self), state.ptr)); @@ -684,6 +704,7 @@ _sre_SRE_Pattern_fullmatch_impl(PatternObject *self, PyTypeObject *cls, if (!state_init(&state, self, string, pos, endpos)) return NULL; + INIT_TRACE(&state); state.ptr = state.start; TRACE(("|%p|%p|FULLMATCH\n", PatternObject_GetCode(self), state.ptr)); @@ -730,6 +751,7 @@ _sre_SRE_Pattern_search_impl(PatternObject *self, PyTypeObject *cls, if (!state_init(&state, self, string, pos, endpos)) return NULL; + INIT_TRACE(&state); TRACE(("|%p|%p|SEARCH\n", PatternObject_GetCode(self), state.ptr)); status = sre_search(&state, PatternObject_GetCode(self)); diff --git a/Modules/_sre/sre.h b/Modules/_sre/sre.h index d967d9ea04ba7..f60078d6bb999 100644 --- a/Modules/_sre/sre.h +++ b/Modules/_sre/sre.h @@ -84,6 +84,7 @@ typedef struct { int charsize; /* character size */ int match_all; int must_advance; + int debug; /* marks */ int lastmark; int lastindex; diff --git a/Modules/_sre/sre_lib.h b/Modules/_sre/sre_lib.h index fb4c18b63d643..c1a774f69090b 100644 --- a/Modules/_sre/sre_lib.h +++ b/Modules/_sre/sre_lib.h @@ -209,6 +209,7 @@ SRE(count)(SRE_STATE* state, const SRE_CODE* pattern, Py_ssize_t maxcount) const SRE_CHAR* ptr = (const SRE_CHAR *)state->ptr; const SRE_CHAR* end = (const SRE_CHAR *)state->end; Py_ssize_t i; + INIT_TRACE(state); /* adjust end */ if (maxcount < end - ptr && maxcount != SRE_MAXREPEAT) @@ -567,6 +568,7 @@ SRE(match)(SRE_STATE* state, const SRE_CODE* pattern, int toplevel) SRE(match_context)* ctx; SRE(match_context)* nextctx; + INIT_TRACE(state); TRACE(("|%p|%p|ENTER\n", pattern, state->ptr)); @@ -1639,6 +1641,7 @@ SRE(search)(SRE_STATE* state, SRE_CODE* pattern) SRE_CODE* charset = NULL; SRE_CODE* overlap = NULL; int flags = 0; + INIT_TRACE(state); if (ptr > end) return 0; From webhook-mailer at python.org Sat Jul 8 11:51:48 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 08 Jul 2023 15:51:48 -0000 Subject: [Python-checkins] gh-106535: Soft deprecate the getopt module (#105735) Message-ID: https://github.com/python/cpython/commit/da98ed0aa040791ef08b24befab697038c8c9fd5 commit: da98ed0aa040791ef08b24befab697038c8c9fd5 branch: main author: Victor Stinner committer: vstinner date: 2023-07-08T17:51:45+02:00 summary: gh-106535: Soft deprecate the getopt module (#105735) The getopt module exists since the initial revision of the Python source code (1990). The optparse module was added to Python 2.3. When Python 2.7 added the 3rd argparse module, the optparse module was soft deprecated. Soft deprecate the getopt module. files: M Doc/library/getopt.rst diff --git a/Doc/library/getopt.rst b/Doc/library/getopt.rst index 336deab28cb8a..ada68b240143e 100644 --- a/Doc/library/getopt.rst +++ b/Doc/library/getopt.rst @@ -7,18 +7,23 @@ **Source code:** :source:`Lib/getopt.py` +.. deprecated:: 3.13 + The :mod:`getopt` module is :term:`soft deprecated` and will not be + developed further; development will continue with the :mod:`argparse` + module. + .. note:: The :mod:`getopt` module is a parser for command line options whose API is - designed to be familiar to users of the C :c:func:`getopt` function. Users who - are unfamiliar with the C :c:func:`getopt` function or who would like to write + designed to be familiar to users of the C :c:func:`!getopt` function. Users who + are unfamiliar with the C :c:func:`!getopt` function or who would like to write less code and get better help and error messages should consider using the :mod:`argparse` module instead. -------------- This module helps scripts to parse the command line arguments in ``sys.argv``. -It supports the same conventions as the Unix :c:func:`getopt` function (including +It supports the same conventions as the Unix :c:func:`!getopt` function (including the special meanings of arguments of the form '``-``' and '``--``'). Long options similar to those supported by GNU software may be used as well via an optional third argument. @@ -33,11 +38,11 @@ exception: be parsed, without the leading reference to the running program. Typically, this means ``sys.argv[1:]``. *shortopts* is the string of option letters that the script wants to recognize, with options that require an argument followed by a - colon (``':'``; i.e., the same format that Unix :c:func:`getopt` uses). + colon (``':'``; i.e., the same format that Unix :c:func:`!getopt` uses). .. note:: - Unlike GNU :c:func:`getopt`, after a non-option argument, all further + Unlike GNU :c:func:`!getopt`, after a non-option argument, all further arguments are considered also non-options. This is similar to the way non-GNU Unix systems work. @@ -71,7 +76,7 @@ exception: non-option argument is encountered. If the first character of the option string is ``'+'``, or if the environment - variable :envvar:`POSIXLY_CORRECT` is set, then option processing stops as + variable :envvar:`!POSIXLY_CORRECT` is set, then option processing stops as soon as a non-option argument is encountered. @@ -81,9 +86,9 @@ exception: an option requiring an argument is given none. The argument to the exception is a string indicating the cause of the error. For long options, an argument given to an option which does not require one will also cause this exception to be - raised. The attributes :attr:`msg` and :attr:`opt` give the error message and + raised. The attributes :attr:`!msg` and :attr:`!opt` give the error message and related option; if there is no specific option to which the exception relates, - :attr:`opt` is an empty string. + :attr:`!opt` is an empty string. .. XXX deprecated? .. exception:: error From webhook-mailer at python.org Sun Jul 9 04:32:54 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 09 Jul 2023 08:32:54 -0000 Subject: [Python-checkins] gh-105376: Remove logging.warn() and LoggerAdapter.warn() (#106553) Message-ID: https://github.com/python/cpython/commit/dcc028d92428bd57358a5028ada2a53fc79fc365 commit: dcc028d92428bd57358a5028ada2a53fc79fc365 branch: main author: Victor Stinner committer: vstinner date: 2023-07-09T10:32:50+02:00 summary: gh-105376: Remove logging.warn() and LoggerAdapter.warn() (#106553) files: M Doc/library/logging.rst M Doc/whatsnew/3.13.rst M Lib/logging/__init__.py M Misc/NEWS.d/next/Library/2023-06-06-15-32-44.gh-issue-105376.W4oDQp.rst diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index d7a389fa13d6a..4e07eabd57f5e 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -1015,6 +1015,10 @@ interchangeably. Attribute :attr:`manager` and method :meth:`_log` were added, which delegate to the underlying logger and allow adapters to be nested. +.. versionchanged:: 3.13 + Remove the undocumented ``warn()`` method which was an alias to the + ``warning()`` method. + Thread Safety ------------- @@ -1162,6 +1166,10 @@ functions. identical to ``warning``. As ``warn`` is deprecated, please do not use it - use ``warning`` instead. + .. versionchanged:: 3.13 + Remove the undocumented ``warn()`` function which was an alias to the + :func:`warning` function. + .. function:: error(msg, *args, **kwargs) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index b376f846b725e..7a4ec5fd91303 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -348,10 +348,11 @@ Removed use ``locale.setlocale(locale.LC_ALL, "")`` instead. (Contributed by Victor Stinner in :gh:`104783`.) -* Remove the undocumented and untested ``logging.Logger.warn()`` method, - deprecated since Python 3.3, which was an alias to the - :meth:`logging.Logger.warning` method: use the :meth:`logging.Logger.warning` - method instead. +* :mod:`logging`: Remove undocumented and untested ``Logger.warn()`` and + ``LoggerAdapter.warn()`` methods and ``logging.warn()`` function. Deprecated + since Python 3.3, they were aliases to the :meth:`logging.Logger.warning` + method, :meth:`!logging.LoggerAdapter.warning` method and + :func:`logging.warning` function. (Contributed by Victor Stinner in :gh:`105376`.) * Remove *cafile*, *capath* and *cadefault* parameters of the diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py index fe2039af0334a..2a011b6d2ff15 100644 --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -37,7 +37,7 @@ 'captureWarnings', 'critical', 'debug', 'disable', 'error', 'exception', 'fatal', 'getLevelName', 'getLogger', 'getLoggerClass', 'info', 'log', 'makeLogRecord', 'setLoggerClass', 'shutdown', - 'warn', 'warning', 'getLogRecordFactory', 'setLogRecordFactory', + 'warning', 'getLogRecordFactory', 'setLogRecordFactory', 'lastResort', 'raiseExceptions', 'getLevelNamesMapping', 'getHandlerByName', 'getHandlerNames'] @@ -1928,11 +1928,6 @@ def warning(self, msg, *args, **kwargs): """ self.log(WARNING, msg, *args, **kwargs) - def warn(self, msg, *args, **kwargs): - warnings.warn("The 'warn' method is deprecated, " - "use 'warning' instead", DeprecationWarning, 2) - self.warning(msg, *args, **kwargs) - def error(self, msg, *args, **kwargs): """ Delegate an error call to the underlying logger. @@ -2206,11 +2201,6 @@ def warning(msg, *args, **kwargs): basicConfig() root.warning(msg, *args, **kwargs) -def warn(msg, *args, **kwargs): - warnings.warn("The 'warn' function is deprecated, " - "use 'warning' instead", DeprecationWarning, 2) - warning(msg, *args, **kwargs) - def info(msg, *args, **kwargs): """ Log a message with severity 'INFO' on the root logger. If the logger has diff --git a/Misc/NEWS.d/next/Library/2023-06-06-15-32-44.gh-issue-105376.W4oDQp.rst b/Misc/NEWS.d/next/Library/2023-06-06-15-32-44.gh-issue-105376.W4oDQp.rst index a7d3172ca4c64..2ed6b5e0a7ac0 100644 --- a/Misc/NEWS.d/next/Library/2023-06-06-15-32-44.gh-issue-105376.W4oDQp.rst +++ b/Misc/NEWS.d/next/Library/2023-06-06-15-32-44.gh-issue-105376.W4oDQp.rst @@ -1,4 +1,5 @@ -Remove the undocumented and untested ``logging.Logger.warn()`` method, -deprecated since Python 3.3, which was an alias to the -:meth:`logging.Logger.warning` method: use the :meth:`logging.Logger.warning` -method instead. Patch by Victor Stinner. +:mod:`logging`: Remove undocumented and untested ``Logger.warn()`` and +``LoggerAdapter.warn()`` methods and ``logging.warn()`` function. Deprecated +since Python 3.3, they were aliases to the :meth:`logging.Logger.warning` +method, :meth:`!logging.LoggerAdapter.warning` method and +:func:`logging.warning` function. Patch by Victor Stinner. From webhook-mailer at python.org Sun Jul 9 05:48:40 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 09 Jul 2023 09:48:40 -0000 Subject: [Python-checkins] Move implementation specific RE tests to separate class (GH-106563) Message-ID: https://github.com/python/cpython/commit/8cb6f9761e3c1cff3210697e3670b57591bf2e7a commit: 8cb6f9761e3c1cff3210697e3670b57591bf2e7a branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-09T12:48:36+03:00 summary: Move implementation specific RE tests to separate class (GH-106563) files: M Lib/test/test_re.py diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py index b1699b0f83dbd..a6f5af17d7d51 100644 --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -1046,33 +1046,6 @@ def test_ignore_case_range(self): def test_category(self): self.assertEqual(re.match(r"(\s)", " ").group(1), " ") - @cpython_only - def test_case_helpers(self): - import _sre - for i in range(128): - c = chr(i) - lo = ord(c.lower()) - self.assertEqual(_sre.ascii_tolower(i), lo) - self.assertEqual(_sre.unicode_tolower(i), lo) - iscased = c in string.ascii_letters - self.assertEqual(_sre.ascii_iscased(i), iscased) - self.assertEqual(_sre.unicode_iscased(i), iscased) - - for i in list(range(128, 0x1000)) + [0x10400, 0x10428]: - c = chr(i) - self.assertEqual(_sre.ascii_tolower(i), i) - if i != 0x0130: - self.assertEqual(_sre.unicode_tolower(i), ord(c.lower())) - iscased = c != c.lower() or c != c.upper() - self.assertFalse(_sre.ascii_iscased(i)) - self.assertEqual(_sre.unicode_iscased(i), - c != c.lower() or c != c.upper()) - - self.assertEqual(_sre.ascii_tolower(0x0130), 0x0130) - self.assertEqual(_sre.unicode_tolower(0x0130), ord('i')) - self.assertFalse(_sre.ascii_iscased(0x0130)) - self.assertTrue(_sre.unicode_iscased(0x0130)) - def test_not_literal(self): self.assertEqual(re.search(r"\s([^a])", " b").group(1), "b") self.assertEqual(re.search(r"\s([^a]*)", " bb").group(1), "bb") @@ -1770,20 +1743,6 @@ def test_bug_6509(self): pat = re.compile(b'..') self.assertEqual(pat.sub(lambda m: b'bytes', b'a5'), b'bytes') - def test_dealloc(self): - # issue 3299: check for segfault in debug build - import _sre - # the overflow limit is different on wide and narrow builds and it - # depends on the definition of SRE_CODE (see sre.h). - # 2**128 should be big enough to overflow on both. For smaller values - # a RuntimeError is raised instead of OverflowError. - long_overflow = 2**128 - self.assertRaises(TypeError, re.finditer, "a", {}) - with self.assertRaises(OverflowError): - _sre.compile("abc", 0, [long_overflow], 0, {}, ()) - with self.assertRaises(TypeError): - _sre.compile({}, 0, [], 0, [], []) - def test_search_dot_unicode(self): self.assertTrue(re.search("123.*-", '123abc-')) self.assertTrue(re.search("123.*-", '123\xe9-')) @@ -1841,21 +1800,6 @@ def test_repeat_minmax_overflow(self): self.assertRaises(OverflowError, re.compile, r".{%d,}?" % 2**128) self.assertRaises(OverflowError, re.compile, r".{%d,%d}" % (2**129, 2**128)) - @cpython_only - def test_repeat_minmax_overflow_maxrepeat(self): - try: - from _sre import MAXREPEAT - except ImportError: - self.skipTest('requires _sre.MAXREPEAT constant') - string = "x" * 100000 - self.assertIsNone(re.match(r".{%d}" % (MAXREPEAT - 1), string)) - self.assertEqual(re.match(r".{,%d}" % (MAXREPEAT - 1), string).span(), - (0, 100000)) - self.assertIsNone(re.match(r".{%d,}?" % (MAXREPEAT - 1), string)) - self.assertRaises(OverflowError, re.compile, r".{%d}" % MAXREPEAT) - self.assertRaises(OverflowError, re.compile, r".{,%d}" % MAXREPEAT) - self.assertRaises(OverflowError, re.compile, r".{%d,}?" % MAXREPEAT) - def test_backref_group_name_in_exception(self): # Issue 17341: Poor error message when compiling invalid regex self.checkPatternError('(?P=)', @@ -2418,16 +2362,6 @@ def test_regression_gh94675(self): p.terminate() p.join() - def test_sre_template_invalid_group_index(self): - # see gh-106524 - import _sre - with self.assertRaises(TypeError) as cm: - _sre.template("", ["", -1, ""]) - self.assertIn("invalid template", str(cm.exception)) - with self.assertRaises(TypeError) as cm: - _sre.template("", ["", (), ""]) - self.assertIn("an integer is required", str(cm.exception)) - def get_debug_out(pat): with captured_stdout() as out: @@ -2676,6 +2610,75 @@ def test_deprecated_modules(self): self.assertTrue(hasattr(mod, attr)) del sys.modules[name] + @cpython_only + def test_case_helpers(self): + import _sre + for i in range(128): + c = chr(i) + lo = ord(c.lower()) + self.assertEqual(_sre.ascii_tolower(i), lo) + self.assertEqual(_sre.unicode_tolower(i), lo) + iscased = c in string.ascii_letters + self.assertEqual(_sre.ascii_iscased(i), iscased) + self.assertEqual(_sre.unicode_iscased(i), iscased) + + for i in list(range(128, 0x1000)) + [0x10400, 0x10428]: + c = chr(i) + self.assertEqual(_sre.ascii_tolower(i), i) + if i != 0x0130: + self.assertEqual(_sre.unicode_tolower(i), ord(c.lower())) + iscased = c != c.lower() or c != c.upper() + self.assertFalse(_sre.ascii_iscased(i)) + self.assertEqual(_sre.unicode_iscased(i), + c != c.lower() or c != c.upper()) + + self.assertEqual(_sre.ascii_tolower(0x0130), 0x0130) + self.assertEqual(_sre.unicode_tolower(0x0130), ord('i')) + self.assertFalse(_sre.ascii_iscased(0x0130)) + self.assertTrue(_sre.unicode_iscased(0x0130)) + + @cpython_only + def test_dealloc(self): + # issue 3299: check for segfault in debug build + import _sre + # the overflow limit is different on wide and narrow builds and it + # depends on the definition of SRE_CODE (see sre.h). + # 2**128 should be big enough to overflow on both. For smaller values + # a RuntimeError is raised instead of OverflowError. + long_overflow = 2**128 + self.assertRaises(TypeError, re.finditer, "a", {}) + with self.assertRaises(OverflowError): + _sre.compile("abc", 0, [long_overflow], 0, {}, ()) + with self.assertRaises(TypeError): + _sre.compile({}, 0, [], 0, [], []) + + @cpython_only + def test_repeat_minmax_overflow_maxrepeat(self): + try: + from _sre import MAXREPEAT + except ImportError: + self.skipTest('requires _sre.MAXREPEAT constant') + string = "x" * 100000 + self.assertIsNone(re.match(r".{%d}" % (MAXREPEAT - 1), string)) + self.assertEqual(re.match(r".{,%d}" % (MAXREPEAT - 1), string).span(), + (0, 100000)) + self.assertIsNone(re.match(r".{%d,}?" % (MAXREPEAT - 1), string)) + self.assertRaises(OverflowError, re.compile, r".{%d}" % MAXREPEAT) + self.assertRaises(OverflowError, re.compile, r".{,%d}" % MAXREPEAT) + self.assertRaises(OverflowError, re.compile, r".{%d,}?" % MAXREPEAT) + + @cpython_only + def test_sre_template_invalid_group_index(self): + # see gh-106524 + import _sre + with self.assertRaises(TypeError) as cm: + _sre.template("", ["", -1, ""]) + self.assertIn("invalid template", str(cm.exception)) + with self.assertRaises(TypeError) as cm: + _sre.template("", ["", (), ""]) + self.assertIn("an integer is required", str(cm.exception)) + + class ExternalTests(unittest.TestCase): def test_re_benchmarks(self): From webhook-mailer at python.org Sun Jul 9 06:14:43 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 09 Jul 2023 10:14:43 -0000 Subject: [Python-checkins] [3.12] Move implementation specific RE tests to separate class (GH-106563) (#106564) Message-ID: https://github.com/python/cpython/commit/559267fcd359501679530c3ff1ccfa8e9c95ea66 commit: 559267fcd359501679530c3ff1ccfa8e9c95ea66 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-09T10:14:39Z summary: [3.12] Move implementation specific RE tests to separate class (GH-106563) (#106564) Move implementation specific RE tests to separate class (GH-106563) (cherry picked from commit 8cb6f9761e3c1cff3210697e3670b57591bf2e7a) Co-authored-by: Serhiy Storchaka files: M Lib/test/test_re.py diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py index 406b665c192df..50b9ad701f0ce 100644 --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -1046,33 +1046,6 @@ def test_ignore_case_range(self): def test_category(self): self.assertEqual(re.match(r"(\s)", " ").group(1), " ") - @cpython_only - def test_case_helpers(self): - import _sre - for i in range(128): - c = chr(i) - lo = ord(c.lower()) - self.assertEqual(_sre.ascii_tolower(i), lo) - self.assertEqual(_sre.unicode_tolower(i), lo) - iscased = c in string.ascii_letters - self.assertEqual(_sre.ascii_iscased(i), iscased) - self.assertEqual(_sre.unicode_iscased(i), iscased) - - for i in list(range(128, 0x1000)) + [0x10400, 0x10428]: - c = chr(i) - self.assertEqual(_sre.ascii_tolower(i), i) - if i != 0x0130: - self.assertEqual(_sre.unicode_tolower(i), ord(c.lower())) - iscased = c != c.lower() or c != c.upper() - self.assertFalse(_sre.ascii_iscased(i)) - self.assertEqual(_sre.unicode_iscased(i), - c != c.lower() or c != c.upper()) - - self.assertEqual(_sre.ascii_tolower(0x0130), 0x0130) - self.assertEqual(_sre.unicode_tolower(0x0130), ord('i')) - self.assertFalse(_sre.ascii_iscased(0x0130)) - self.assertTrue(_sre.unicode_iscased(0x0130)) - def test_not_literal(self): self.assertEqual(re.search(r"\s([^a])", " b").group(1), "b") self.assertEqual(re.search(r"\s([^a]*)", " bb").group(1), "bb") @@ -1769,20 +1742,6 @@ def test_bug_6509(self): pat = re.compile(b'..') self.assertEqual(pat.sub(lambda m: b'bytes', b'a5'), b'bytes') - def test_dealloc(self): - # issue 3299: check for segfault in debug build - import _sre - # the overflow limit is different on wide and narrow builds and it - # depends on the definition of SRE_CODE (see sre.h). - # 2**128 should be big enough to overflow on both. For smaller values - # a RuntimeError is raised instead of OverflowError. - long_overflow = 2**128 - self.assertRaises(TypeError, re.finditer, "a", {}) - with self.assertRaises(OverflowError): - _sre.compile("abc", 0, [long_overflow], 0, {}, ()) - with self.assertRaises(TypeError): - _sre.compile({}, 0, [], 0, [], []) - def test_search_dot_unicode(self): self.assertTrue(re.search("123.*-", '123abc-')) self.assertTrue(re.search("123.*-", '123\xe9-')) @@ -1840,21 +1799,6 @@ def test_repeat_minmax_overflow(self): self.assertRaises(OverflowError, re.compile, r".{%d,}?" % 2**128) self.assertRaises(OverflowError, re.compile, r".{%d,%d}" % (2**129, 2**128)) - @cpython_only - def test_repeat_minmax_overflow_maxrepeat(self): - try: - from _sre import MAXREPEAT - except ImportError: - self.skipTest('requires _sre.MAXREPEAT constant') - string = "x" * 100000 - self.assertIsNone(re.match(r".{%d}" % (MAXREPEAT - 1), string)) - self.assertEqual(re.match(r".{,%d}" % (MAXREPEAT - 1), string).span(), - (0, 100000)) - self.assertIsNone(re.match(r".{%d,}?" % (MAXREPEAT - 1), string)) - self.assertRaises(OverflowError, re.compile, r".{%d}" % MAXREPEAT) - self.assertRaises(OverflowError, re.compile, r".{,%d}" % MAXREPEAT) - self.assertRaises(OverflowError, re.compile, r".{%d,}?" % MAXREPEAT) - def test_backref_group_name_in_exception(self): # Issue 17341: Poor error message when compiling invalid regex self.checkPatternError('(?P=)', @@ -2441,16 +2385,6 @@ def test_regression_gh94675(self): p.terminate() p.join() - def test_sre_template_invalid_group_index(self): - # see gh-106524 - import _sre - with self.assertRaises(TypeError) as cm: - _sre.template("", ["", -1, ""]) - self.assertIn("invalid template", str(cm.exception)) - with self.assertRaises(TypeError) as cm: - _sre.template("", ["", (), ""]) - self.assertIn("an integer is required", str(cm.exception)) - def get_debug_out(pat): with captured_stdout() as out: @@ -2699,6 +2633,75 @@ def test_deprecated_modules(self): self.assertTrue(hasattr(mod, attr)) del sys.modules[name] + @cpython_only + def test_case_helpers(self): + import _sre + for i in range(128): + c = chr(i) + lo = ord(c.lower()) + self.assertEqual(_sre.ascii_tolower(i), lo) + self.assertEqual(_sre.unicode_tolower(i), lo) + iscased = c in string.ascii_letters + self.assertEqual(_sre.ascii_iscased(i), iscased) + self.assertEqual(_sre.unicode_iscased(i), iscased) + + for i in list(range(128, 0x1000)) + [0x10400, 0x10428]: + c = chr(i) + self.assertEqual(_sre.ascii_tolower(i), i) + if i != 0x0130: + self.assertEqual(_sre.unicode_tolower(i), ord(c.lower())) + iscased = c != c.lower() or c != c.upper() + self.assertFalse(_sre.ascii_iscased(i)) + self.assertEqual(_sre.unicode_iscased(i), + c != c.lower() or c != c.upper()) + + self.assertEqual(_sre.ascii_tolower(0x0130), 0x0130) + self.assertEqual(_sre.unicode_tolower(0x0130), ord('i')) + self.assertFalse(_sre.ascii_iscased(0x0130)) + self.assertTrue(_sre.unicode_iscased(0x0130)) + + @cpython_only + def test_dealloc(self): + # issue 3299: check for segfault in debug build + import _sre + # the overflow limit is different on wide and narrow builds and it + # depends on the definition of SRE_CODE (see sre.h). + # 2**128 should be big enough to overflow on both. For smaller values + # a RuntimeError is raised instead of OverflowError. + long_overflow = 2**128 + self.assertRaises(TypeError, re.finditer, "a", {}) + with self.assertRaises(OverflowError): + _sre.compile("abc", 0, [long_overflow], 0, {}, ()) + with self.assertRaises(TypeError): + _sre.compile({}, 0, [], 0, [], []) + + @cpython_only + def test_repeat_minmax_overflow_maxrepeat(self): + try: + from _sre import MAXREPEAT + except ImportError: + self.skipTest('requires _sre.MAXREPEAT constant') + string = "x" * 100000 + self.assertIsNone(re.match(r".{%d}" % (MAXREPEAT - 1), string)) + self.assertEqual(re.match(r".{,%d}" % (MAXREPEAT - 1), string).span(), + (0, 100000)) + self.assertIsNone(re.match(r".{%d,}?" % (MAXREPEAT - 1), string)) + self.assertRaises(OverflowError, re.compile, r".{%d}" % MAXREPEAT) + self.assertRaises(OverflowError, re.compile, r".{,%d}" % MAXREPEAT) + self.assertRaises(OverflowError, re.compile, r".{%d,}?" % MAXREPEAT) + + @cpython_only + def test_sre_template_invalid_group_index(self): + # see gh-106524 + import _sre + with self.assertRaises(TypeError) as cm: + _sre.template("", ["", -1, ""]) + self.assertIn("invalid template", str(cm.exception)) + with self.assertRaises(TypeError) as cm: + _sre.template("", ["", (), ""]) + self.assertIn("an integer is required", str(cm.exception)) + + class ExternalTests(unittest.TestCase): def test_re_benchmarks(self): From webhook-mailer at python.org Sun Jul 9 08:08:21 2023 From: webhook-mailer at python.org (corona10) Date: Sun, 09 Jul 2023 12:08:21 -0000 Subject: [Python-checkins] gh-104469: Convert_testcapi/vectorcall.c to use AC (gh-106557) Message-ID: https://github.com/python/cpython/commit/d137c2cae28b79555433079d917c3e0614bdcd61 commit: d137c2cae28b79555433079d917c3e0614bdcd61 branch: main author: littlebutt's workshop committer: corona10 date: 2023-07-09T21:08:18+09:00 summary: gh-104469: Convert_testcapi/vectorcall.c to use AC (gh-106557) files: M Modules/_testcapi/clinic/vectorcall.c.h M Modules/_testcapi/vectorcall.c diff --git a/Modules/_testcapi/clinic/vectorcall.c.h b/Modules/_testcapi/clinic/vectorcall.c.h index 765afeda9b306..728c0d382565a 100644 --- a/Modules/_testcapi/clinic/vectorcall.c.h +++ b/Modules/_testcapi/clinic/vectorcall.c.h @@ -8,6 +8,106 @@ preserve #endif +PyDoc_STRVAR(_testcapi_pyobject_fastcalldict__doc__, +"pyobject_fastcalldict($module, func, func_args, kwargs, /)\n" +"--\n" +"\n"); + +#define _TESTCAPI_PYOBJECT_FASTCALLDICT_METHODDEF \ + {"pyobject_fastcalldict", _PyCFunction_CAST(_testcapi_pyobject_fastcalldict), METH_FASTCALL, _testcapi_pyobject_fastcalldict__doc__}, + +static PyObject * +_testcapi_pyobject_fastcalldict_impl(PyObject *module, PyObject *func, + PyObject *func_args, PyObject *kwargs); + +static PyObject * +_testcapi_pyobject_fastcalldict(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *func; + PyObject *func_args; + PyObject *__clinic_kwargs; + + if (!_PyArg_CheckPositional("pyobject_fastcalldict", nargs, 3, 3)) { + goto exit; + } + func = args[0]; + func_args = args[1]; + __clinic_kwargs = args[2]; + return_value = _testcapi_pyobject_fastcalldict_impl(module, func, func_args, __clinic_kwargs); + +exit: + return return_value; +} + +PyDoc_STRVAR(_testcapi_pyobject_vectorcall__doc__, +"pyobject_vectorcall($module, func, func_args, kwnames, /)\n" +"--\n" +"\n"); + +#define _TESTCAPI_PYOBJECT_VECTORCALL_METHODDEF \ + {"pyobject_vectorcall", _PyCFunction_CAST(_testcapi_pyobject_vectorcall), METH_FASTCALL, _testcapi_pyobject_vectorcall__doc__}, + +static PyObject * +_testcapi_pyobject_vectorcall_impl(PyObject *module, PyObject *func, + PyObject *func_args, PyObject *kwnames); + +static PyObject * +_testcapi_pyobject_vectorcall(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *func; + PyObject *func_args; + PyObject *__clinic_kwnames; + + if (!_PyArg_CheckPositional("pyobject_vectorcall", nargs, 3, 3)) { + goto exit; + } + func = args[0]; + func_args = args[1]; + __clinic_kwnames = args[2]; + return_value = _testcapi_pyobject_vectorcall_impl(module, func, func_args, __clinic_kwnames); + +exit: + return return_value; +} + +PyDoc_STRVAR(_testcapi_pyvectorcall_call__doc__, +"pyvectorcall_call($module, func, argstuple, kwargs=, /)\n" +"--\n" +"\n"); + +#define _TESTCAPI_PYVECTORCALL_CALL_METHODDEF \ + {"pyvectorcall_call", _PyCFunction_CAST(_testcapi_pyvectorcall_call), METH_FASTCALL, _testcapi_pyvectorcall_call__doc__}, + +static PyObject * +_testcapi_pyvectorcall_call_impl(PyObject *module, PyObject *func, + PyObject *argstuple, PyObject *kwargs); + +static PyObject * +_testcapi_pyvectorcall_call(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *func; + PyObject *argstuple; + PyObject *__clinic_kwargs = NULL; + + if (!_PyArg_CheckPositional("pyvectorcall_call", nargs, 2, 3)) { + goto exit; + } + func = args[0]; + argstuple = args[1]; + if (nargs < 3) { + goto skip_optional; + } + __clinic_kwargs = args[2]; +skip_optional: + return_value = _testcapi_pyvectorcall_call_impl(module, func, argstuple, __clinic_kwargs); + +exit: + return return_value; +} + PyDoc_STRVAR(_testcapi_VectorCallClass_set_vectorcall__doc__, "set_vectorcall($self, type, /)\n" "--\n" @@ -110,4 +210,4 @@ _testcapi_has_vectorcall_flag(PyObject *module, PyObject *arg) exit: return return_value; } -/*[clinic end generated code: output=609569aa9942584f input=a9049054013a1b77]*/ +/*[clinic end generated code: output=beaf6beac3d13c25 input=a9049054013a1b77]*/ diff --git a/Modules/_testcapi/vectorcall.c b/Modules/_testcapi/vectorcall.c index 4935fd1b6a7ba..5ee468bd28c85 100644 --- a/Modules/_testcapi/vectorcall.c +++ b/Modules/_testcapi/vectorcall.c @@ -4,6 +4,10 @@ #include "structmember.h" // PyMemberDef #include // offsetof +/*[clinic input] +module _testcapi +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=6361033e795369fc]*/ /* Test PEP 590 - Vectorcall */ @@ -25,18 +29,22 @@ fastcall_args(PyObject *args, PyObject ***stack, Py_ssize_t *nargs) return 0; } +/*[clinic input] +_testcapi.pyobject_fastcalldict + func: object + func_args: object + kwargs: object + / +[clinic start generated code]*/ static PyObject * -test_pyobject_fastcalldict(PyObject *self, PyObject *args) +_testcapi_pyobject_fastcalldict_impl(PyObject *module, PyObject *func, + PyObject *func_args, PyObject *kwargs) +/*[clinic end generated code: output=35902ece94de4418 input=b9c0196ca7d5f9e4]*/ { - PyObject *func, *func_args, *kwargs; PyObject **stack; Py_ssize_t nargs; - if (!PyArg_ParseTuple(args, "OOO", &func, &func_args, &kwargs)) { - return NULL; - } - if (fastcall_args(func_args, &stack, &nargs) < 0) { return NULL; } @@ -52,17 +60,22 @@ test_pyobject_fastcalldict(PyObject *self, PyObject *args) return PyObject_VectorcallDict(func, stack, nargs, kwargs); } +/*[clinic input] +_testcapi.pyobject_vectorcall + func: object + func_args: object + kwnames: object + / +[clinic start generated code]*/ + static PyObject * -test_pyobject_vectorcall(PyObject *self, PyObject *args) +_testcapi_pyobject_vectorcall_impl(PyObject *module, PyObject *func, + PyObject *func_args, PyObject *kwnames) +/*[clinic end generated code: output=ff77245bc6afe0d8 input=a0668dfef625764c]*/ { - PyObject *func, *func_args, *kwnames = NULL; PyObject **stack; Py_ssize_t nargs, nkw; - if (!PyArg_ParseTuple(args, "OOO", &func, &func_args, &kwnames)) { - return NULL; - } - if (fastcall_args(func_args, &stack, &nargs) < 0) { return NULL; } @@ -103,17 +116,19 @@ function_setvectorcall(PyObject *self, PyObject *func) Py_RETURN_NONE; } +/*[clinic input] +_testcapi.pyvectorcall_call + func: object + argstuple: object + kwargs: object = NULL + / +[clinic start generated code]*/ + static PyObject * -test_pyvectorcall_call(PyObject *self, PyObject *args) +_testcapi_pyvectorcall_call_impl(PyObject *module, PyObject *func, + PyObject *argstuple, PyObject *kwargs) +/*[clinic end generated code: output=809046fe78511306 input=4376ee7cabd698ce]*/ { - PyObject *func; - PyObject *argstuple; - PyObject *kwargs = NULL; - - if (!PyArg_ParseTuple(args, "OO|O", &func, &argstuple, &kwargs)) { - return NULL; - } - if (!PyTuple_Check(argstuple)) { PyErr_SetString(PyExc_TypeError, "args must be a tuple"); return NULL; @@ -242,10 +257,10 @@ _testcapi_has_vectorcall_flag_impl(PyObject *module, PyTypeObject *type) } static PyMethodDef TestMethods[] = { - {"pyobject_fastcalldict", test_pyobject_fastcalldict, METH_VARARGS}, - {"pyobject_vectorcall", test_pyobject_vectorcall, METH_VARARGS}, + _TESTCAPI_PYOBJECT_FASTCALLDICT_METHODDEF + _TESTCAPI_PYOBJECT_VECTORCALL_METHODDEF {"function_setvectorcall", function_setvectorcall, METH_O}, - {"pyvectorcall_call", test_pyvectorcall_call, METH_VARARGS}, + _TESTCAPI_PYVECTORCALL_CALL_METHODDEF _TESTCAPI_MAKE_VECTORCALL_CLASS_METHODDEF _TESTCAPI_HAS_VECTORCALL_FLAG_METHODDEF {NULL}, From webhook-mailer at python.org Sun Jul 9 08:23:31 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 09 Jul 2023 12:23:31 -0000 Subject: [Python-checkins] [3.11] Move implementation specific RE tests to separate class (GH-106563) (GH-106565) Message-ID: https://github.com/python/cpython/commit/769b7d2d0b3fa912400587fbe0133fa0f7899395 commit: 769b7d2d0b3fa912400587fbe0133fa0f7899395 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-09T15:23:27+03:00 summary: [3.11] Move implementation specific RE tests to separate class (GH-106563) (GH-106565) (cherry picked from commit 8cb6f9761e3c1cff3210697e3670b57591bf2e7a) Co-authored-by: Serhiy Storchaka files: M Lib/test/test_re.py diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py index 167b4400f34c0..38d6db7c4091b 100644 --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -1077,33 +1077,6 @@ def test_ignore_case_range(self): def test_category(self): self.assertEqual(re.match(r"(\s)", " ").group(1), " ") - @cpython_only - def test_case_helpers(self): - import _sre - for i in range(128): - c = chr(i) - lo = ord(c.lower()) - self.assertEqual(_sre.ascii_tolower(i), lo) - self.assertEqual(_sre.unicode_tolower(i), lo) - iscased = c in string.ascii_letters - self.assertEqual(_sre.ascii_iscased(i), iscased) - self.assertEqual(_sre.unicode_iscased(i), iscased) - - for i in list(range(128, 0x1000)) + [0x10400, 0x10428]: - c = chr(i) - self.assertEqual(_sre.ascii_tolower(i), i) - if i != 0x0130: - self.assertEqual(_sre.unicode_tolower(i), ord(c.lower())) - iscased = c != c.lower() or c != c.upper() - self.assertFalse(_sre.ascii_iscased(i)) - self.assertEqual(_sre.unicode_iscased(i), - c != c.lower() or c != c.upper()) - - self.assertEqual(_sre.ascii_tolower(0x0130), 0x0130) - self.assertEqual(_sre.unicode_tolower(0x0130), ord('i')) - self.assertFalse(_sre.ascii_iscased(0x0130)) - self.assertTrue(_sre.unicode_iscased(0x0130)) - def test_not_literal(self): self.assertEqual(re.search(r"\s([^a])", " b").group(1), "b") self.assertEqual(re.search(r"\s([^a]*)", " bb").group(1), "bb") @@ -1800,20 +1773,6 @@ def test_bug_6509(self): pat = re.compile(b'..') self.assertEqual(pat.sub(lambda m: b'bytes', b'a5'), b'bytes') - def test_dealloc(self): - # issue 3299: check for segfault in debug build - import _sre - # the overflow limit is different on wide and narrow builds and it - # depends on the definition of SRE_CODE (see sre.h). - # 2**128 should be big enough to overflow on both. For smaller values - # a RuntimeError is raised instead of OverflowError. - long_overflow = 2**128 - self.assertRaises(TypeError, re.finditer, "a", {}) - with self.assertRaises(OverflowError): - _sre.compile("abc", 0, [long_overflow], 0, {}, ()) - with self.assertRaises(TypeError): - _sre.compile({}, 0, [], 0, [], []) - def test_search_dot_unicode(self): self.assertTrue(re.search("123.*-", '123abc-')) self.assertTrue(re.search("123.*-", '123\xe9-')) @@ -1871,21 +1830,6 @@ def test_repeat_minmax_overflow(self): self.assertRaises(OverflowError, re.compile, r".{%d,}?" % 2**128) self.assertRaises(OverflowError, re.compile, r".{%d,%d}" % (2**129, 2**128)) - @cpython_only - def test_repeat_minmax_overflow_maxrepeat(self): - try: - from _sre import MAXREPEAT - except ImportError: - self.skipTest('requires _sre.MAXREPEAT constant') - string = "x" * 100000 - self.assertIsNone(re.match(r".{%d}" % (MAXREPEAT - 1), string)) - self.assertEqual(re.match(r".{,%d}" % (MAXREPEAT - 1), string).span(), - (0, 100000)) - self.assertIsNone(re.match(r".{%d,}?" % (MAXREPEAT - 1), string)) - self.assertRaises(OverflowError, re.compile, r".{%d}" % MAXREPEAT) - self.assertRaises(OverflowError, re.compile, r".{,%d}" % MAXREPEAT) - self.assertRaises(OverflowError, re.compile, r".{%d,}?" % MAXREPEAT) - def test_backref_group_name_in_exception(self): # Issue 17341: Poor error message when compiling invalid regex self.checkPatternError('(?P=)', @@ -2720,6 +2664,64 @@ def test_deprecated_modules(self): self.assertTrue(hasattr(mod, attr)) del sys.modules[name] + @cpython_only + def test_case_helpers(self): + import _sre + for i in range(128): + c = chr(i) + lo = ord(c.lower()) + self.assertEqual(_sre.ascii_tolower(i), lo) + self.assertEqual(_sre.unicode_tolower(i), lo) + iscased = c in string.ascii_letters + self.assertEqual(_sre.ascii_iscased(i), iscased) + self.assertEqual(_sre.unicode_iscased(i), iscased) + + for i in list(range(128, 0x1000)) + [0x10400, 0x10428]: + c = chr(i) + self.assertEqual(_sre.ascii_tolower(i), i) + if i != 0x0130: + self.assertEqual(_sre.unicode_tolower(i), ord(c.lower())) + iscased = c != c.lower() or c != c.upper() + self.assertFalse(_sre.ascii_iscased(i)) + self.assertEqual(_sre.unicode_iscased(i), + c != c.lower() or c != c.upper()) + + self.assertEqual(_sre.ascii_tolower(0x0130), 0x0130) + self.assertEqual(_sre.unicode_tolower(0x0130), ord('i')) + self.assertFalse(_sre.ascii_iscased(0x0130)) + self.assertTrue(_sre.unicode_iscased(0x0130)) + + @cpython_only + def test_dealloc(self): + # issue 3299: check for segfault in debug build + import _sre + # the overflow limit is different on wide and narrow builds and it + # depends on the definition of SRE_CODE (see sre.h). + # 2**128 should be big enough to overflow on both. For smaller values + # a RuntimeError is raised instead of OverflowError. + long_overflow = 2**128 + self.assertRaises(TypeError, re.finditer, "a", {}) + with self.assertRaises(OverflowError): + _sre.compile("abc", 0, [long_overflow], 0, {}, ()) + with self.assertRaises(TypeError): + _sre.compile({}, 0, [], 0, [], []) + + @cpython_only + def test_repeat_minmax_overflow_maxrepeat(self): + try: + from _sre import MAXREPEAT + except ImportError: + self.skipTest('requires _sre.MAXREPEAT constant') + string = "x" * 100000 + self.assertIsNone(re.match(r".{%d}" % (MAXREPEAT - 1), string)) + self.assertEqual(re.match(r".{,%d}" % (MAXREPEAT - 1), string).span(), + (0, 100000)) + self.assertIsNone(re.match(r".{%d,}?" % (MAXREPEAT - 1), string)) + self.assertRaises(OverflowError, re.compile, r".{%d}" % MAXREPEAT) + self.assertRaises(OverflowError, re.compile, r".{,%d}" % MAXREPEAT) + self.assertRaises(OverflowError, re.compile, r".{%d,}?" % MAXREPEAT) + + class ExternalTests(unittest.TestCase): def test_re_benchmarks(self): From webhook-mailer at python.org Sun Jul 9 08:27:07 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 09 Jul 2023 12:27:07 -0000 Subject: [Python-checkins] gh-106303: Use _PyObject_LookupAttr() instead of PyObject_GetAttr() (GH-106304) Message-ID: https://github.com/python/cpython/commit/93d292c2b3f8e85ef562c37f59678c639b9b8fcb commit: 93d292c2b3f8e85ef562c37f59678c639b9b8fcb branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-09T15:27:03+03:00 summary: gh-106303: Use _PyObject_LookupAttr() instead of PyObject_GetAttr() (GH-106304) It simplifies and speed up the code. files: M Include/internal/pycore_global_objects_fini_generated.h M Include/internal/pycore_global_strings.h M Include/internal/pycore_runtime_init_generated.h M Include/internal/pycore_unicodeobject_generated.h M Objects/funcobject.c M Python/ceval.c diff --git a/Include/internal/pycore_global_objects_fini_generated.h b/Include/internal/pycore_global_objects_fini_generated.h index f85207b4bde29..6d50ffd0a02f1 100644 --- a/Include/internal/pycore_global_objects_fini_generated.h +++ b/Include/internal/pycore_global_objects_fini_generated.h @@ -668,6 +668,7 @@ _PyStaticObjects_CheckRefcnt(PyInterpreterState *interp) { _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__lshift__)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__lt__)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__main__)); + _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__match_args__)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__matmul__)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__missing__)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__mod__)); diff --git a/Include/internal/pycore_global_strings.h b/Include/internal/pycore_global_strings.h index 3c9c12202ba1b..bb1fb13f342fc 100644 --- a/Include/internal/pycore_global_strings.h +++ b/Include/internal/pycore_global_strings.h @@ -157,6 +157,7 @@ struct _Py_global_strings { STRUCT_FOR_ID(__lshift__) STRUCT_FOR_ID(__lt__) STRUCT_FOR_ID(__main__) + STRUCT_FOR_ID(__match_args__) STRUCT_FOR_ID(__matmul__) STRUCT_FOR_ID(__missing__) STRUCT_FOR_ID(__mod__) diff --git a/Include/internal/pycore_runtime_init_generated.h b/Include/internal/pycore_runtime_init_generated.h index 9a28368a124ce..2d66647438b19 100644 --- a/Include/internal/pycore_runtime_init_generated.h +++ b/Include/internal/pycore_runtime_init_generated.h @@ -663,6 +663,7 @@ extern "C" { INIT_ID(__lshift__), \ INIT_ID(__lt__), \ INIT_ID(__main__), \ + INIT_ID(__match_args__), \ INIT_ID(__matmul__), \ INIT_ID(__missing__), \ INIT_ID(__mod__), \ diff --git a/Include/internal/pycore_unicodeobject_generated.h b/Include/internal/pycore_unicodeobject_generated.h index 9e13a9491b7da..59f40075f9398 100644 --- a/Include/internal/pycore_unicodeobject_generated.h +++ b/Include/internal/pycore_unicodeobject_generated.h @@ -315,6 +315,9 @@ _PyUnicode_InitStaticStrings(PyInterpreterState *interp) { string = &_Py_ID(__main__); assert(_PyUnicode_CheckConsistency(string, 1)); _PyUnicode_InternInPlace(interp, &string); + string = &_Py_ID(__match_args__); + assert(_PyUnicode_CheckConsistency(string, 1)); + _PyUnicode_InternInPlace(interp, &string); string = &_Py_ID(__matmul__); assert(_PyUnicode_CheckConsistency(string, 1)); _PyUnicode_InternInPlace(interp, &string); diff --git a/Objects/funcobject.c b/Objects/funcobject.c index f43e3a2787b84..a8a9ee2b9bad4 100644 --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -942,17 +942,12 @@ PyTypeObject PyFunction_Type = { static int functools_copy_attr(PyObject *wrapper, PyObject *wrapped, PyObject *name) { - PyObject *value = PyObject_GetAttr(wrapped, name); - if (value == NULL) { - if (PyErr_ExceptionMatches(PyExc_AttributeError)) { - PyErr_Clear(); - return 0; - } - return -1; + PyObject *value; + int res = _PyObject_LookupAttr(wrapped, name, &value); + if (value != NULL) { + res = PyObject_SetAttr(wrapper, name, value); + Py_DECREF(value); } - - int res = PyObject_SetAttr(wrapper, name, value); - Py_DECREF(value); return res; } diff --git a/Python/ceval.c b/Python/ceval.c index 1b8650a650412..da1135549a255 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -418,10 +418,8 @@ match_class_attr(PyThreadState *tstate, PyObject *subject, PyObject *type, } return NULL; } - PyObject *attr = PyObject_GetAttr(subject, name); - if (attr == NULL && _PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { - _PyErr_Clear(tstate); - } + PyObject *attr; + (void)_PyObject_LookupAttr(subject, name, &attr); return attr; } @@ -456,7 +454,9 @@ match_class(PyThreadState *tstate, PyObject *subject, PyObject *type, // First, the positional subpatterns: if (nargs) { int match_self = 0; - match_args = PyObject_GetAttrString(type, "__match_args__"); + if (_PyObject_LookupAttr(type, &_Py_ID(__match_args__), &match_args) < 0) { + goto fail; + } if (match_args) { if (!PyTuple_CheckExact(match_args)) { const char *e = "%s.__match_args__ must be a tuple (got %s)"; @@ -466,8 +466,7 @@ match_class(PyThreadState *tstate, PyObject *subject, PyObject *type, goto fail; } } - else if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { - _PyErr_Clear(tstate); + else { // _Py_TPFLAGS_MATCH_SELF is only acknowledged if the type does not // define __match_args__. This is natural behavior for subclasses: // it's as if __match_args__ is some "magic" value that is lost as @@ -476,9 +475,6 @@ match_class(PyThreadState *tstate, PyObject *subject, PyObject *type, match_self = PyType_HasFeature((PyTypeObject*)type, _Py_TPFLAGS_MATCH_SELF); } - else { - goto fail; - } assert(PyTuple_CheckExact(match_args)); Py_ssize_t allowed = match_self ? 1 : PyTuple_GET_SIZE(match_args); if (allowed < nargs) { From webhook-mailer at python.org Sun Jul 9 11:26:30 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 09 Jul 2023 15:26:30 -0000 Subject: [Python-checkins] gh-105373: Doc lists pending removals (#106540) Message-ID: https://github.com/python/cpython/commit/1e12c8cfa373e57aaec65a574e5e4932bbbc0d4f commit: 1e12c8cfa373e57aaec65a574e5e4932bbbc0d4f branch: main author: Victor Stinner committer: vstinner date: 2023-07-09T17:26:26+02:00 summary: gh-105373: Doc lists pending removals (#106540) files: M Doc/whatsnew/3.13.rst diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 7a4ec5fd91303..a01fc3b34b6fe 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -90,7 +90,7 @@ Improved Modules array ----- -* Add ``'w'`` type code that can be used for Unicode strings. +* Add ``'w'`` type code (``Py_UCS4``) that can be used for Unicode strings. It can be used instead of ``'u'`` type code, which is deprecated. (Contributed by Inada Naoki in :gh:`80480`.) @@ -146,11 +146,11 @@ Deprecated methods of the :class:`wave.Wave_read` and :class:`wave.Wave_write` classes. They will be removed in Python 3.15. (Contributed by Victor Stinner in :gh:`105096`.) -* Creating a :class:`typing.NamedTuple` class using keyword arguments to denote +* :mod:`typing`: Creating a :class:`typing.NamedTuple` class using keyword arguments to denote the fields (``NT = NamedTuple("NT", x=int, y=int)``) is deprecated, and will be disallowed in Python 3.15. Use the class-based syntax or the functional syntax instead. (Contributed by Alex Waygood in :gh:`105566`.) -* When using the functional syntax to create a :class:`typing.NamedTuple` +* :mod:`typing`: When using the functional syntax to create a :class:`typing.NamedTuple` class or a :class:`typing.TypedDict` class, failing to pass a value to the 'fields' parameter (``NT = NamedTuple("NT")`` or ``TD = TypedDict("TD")``) is deprecated. Passing ``None`` to the 'fields' parameter @@ -172,6 +172,297 @@ Deprecated Replace ``ctypes.SetPointerType(item_type, size)`` with ``item_type * size``. (Contributed by Victor Stinner in :gh:`105733`.) +Pending Removal in Python 3.14 +------------------------------ + +* :mod:`argparse`: The *type*, *choices*, and *metavar* parameters + of :class:`!argparse.BooleanOptionalAction` are deprecated + and will be removed in 3.14. + (Contributed by Nikita Sobolev in :gh:`92248`.) + +* :mod:`ast`: The following features have been deprecated in documentation + since Python 3.8, now cause a :exc:`DeprecationWarning` to be emitted at + runtime when they are accessed or used, and will be removed in Python 3.14: + + * :class:`!ast.Num` + * :class:`!ast.Str` + * :class:`!ast.Bytes` + * :class:`!ast.NameConstant` + * :class:`!ast.Ellipsis` + + Use :class:`ast.Constant` instead. + (Contributed by Serhiy Storchaka in :gh:`90953`.) + +* :mod:`collections.abc`: Deprecated :class:`~collections.abc.ByteString`. + Prefer :class:`!Sequence` or :class:`~collections.abc.Buffer`. + For use in typing, prefer a union, like ``bytes | bytearray``, + or :class:`collections.abc.Buffer`. + (Contributed by Shantanu Jain in :gh:`91896`.) + +* :mod:`email`: Deprecated the *isdst* parameter in :func:`email.utils.localtime`. + (Contributed by Alan Williams in :gh:`72346`.) + +* :mod:`importlib`: ``__package__`` and ``__cached__`` will cease to be set or + taken into consideration by the import system (:gh:`97879`). + +* :mod:`importlib.abc` deprecated classes: + + * :class:`!importlib.abc.ResourceReader` + * :class:`!importlib.abc.Traversable` + * :class:`!importlib.abc.TraversableResources` + + Use :mod:`importlib.resources.abc` classes instead: + + * :class:`importlib.resources.abc.Traversable` + * :class:`importlib.resources.abc.TraversableResources` + + (Contributed by Jason R. Coombs and Hugo van Kemenade in :gh:`93963`.) + +* :mod:`itertools` had undocumented, inefficient, historically buggy, + and inconsistent support for copy, deepcopy, and pickle operations. + This will be removed in 3.14 for a significant reduction in code + volume and maintenance burden. + (Contributed by Raymond Hettinger in :gh:`101588`.) + +* :mod:`multiprocessing`: The default start method will change to a safer one on + Linux, BSDs, and other non-macOS POSIX platforms where ``'fork'`` is currently + the default (:gh:`84559`). Adding a runtime warning about this was deemed too + disruptive as the majority of code is not expected to care. Use the + :func:`~multiprocessing.get_context` or + :func:`~multiprocessing.set_start_method` APIs to explicitly specify when + your code *requires* ``'fork'``. See :ref:`multiprocessing-start-methods`. + +* :mod:`pathlib`: :meth:`~pathlib.PurePath.is_relative_to`, + :meth:`~pathlib.PurePath.relative_to`: passing additional arguments is + deprecated. + +* :func:`pkgutil.find_loader` and :func:`pkgutil.get_loader` + now raise :exc:`DeprecationWarning`; + use :func:`importlib.util.find_spec` instead. + (Contributed by Nikita Sobolev in :gh:`97850`.) + +* :mod:`pty`: + + * ``master_open()``: use :func:`pty.openpty`. + * ``slave_open()``: use :func:`pty.openpty`. + +* :func:`shutil.rmtree` *onerror* parameter is deprecated in 3.12, + and will be removed in 3.14: use the *onexc* parameter instead. + +* :mod:`sqlite3`: + + * :data:`~sqlite3.version` and :data:`~sqlite3.version_info`. + + * :meth:`~sqlite3.Cursor.execute` and :meth:`~sqlite3.Cursor.executemany` + if :ref:`named placeholders ` are used and + *parameters* is a sequence instead of a :class:`dict`. + + * date and datetime adapter, date and timestamp converter: + see the :mod:`sqlite3` documentation for suggested replacement recipes. + +* :class:`types.CodeType`: Accessing ``co_lnotab`` was deprecated in :pep:`626` + since 3.10 and was planned to be removed in 3.12, + but it only got a proper :exc:`DeprecationWarning` in 3.12. + May be removed in 3.14. + (Contributed by Nikita Sobolev in :gh:`101866`.) + +* :mod:`typing`: :class:`~typing.ByteString`, deprecated since Python 3.9, + now causes a :exc:`DeprecationWarning` to be emitted when it is used. + +* :class:`!urllib.parse.Quoter`. + +* :mod:`xml.etree.ElementTree`: Testing the truth value of an + :class:`~xml.etree.ElementTree.Element` is deprecated and will raise an + exception in Python 3.14. + +Pending Removal in Python 3.15 +------------------------------ + +* :class:`typing.NamedTuple`: + + * The undocumented keyword argument syntax for creating NamedTuple classes + (``NT = NamedTuple("NT", x=int)``) is deprecated, and will be disallowed in + 3.15. Use the class-based syntax or the functional syntax instead. + + * When using the functional syntax to create a NamedTuple class, failing to + pass a value to the 'fields' parameter (``NT = NamedTuple("NT")``) is + deprecated. Passing ``None`` to the 'fields' parameter + (``NT = NamedTuple("NT", None)``) is also deprecated. Both will be + disallowed in Python 3.15. To create a NamedTuple class with 0 fields, use + ``class NT(NamedTuple): pass`` or ``NT = NamedTuple("NT", [])``. + +* :class:`typing.TypedDict`: When using the functional syntax to create a + TypedDict class, failing to pass a value to the 'fields' parameter (``TD = + TypedDict("TD")``) is deprecated. Passing ``None`` to the 'fields' parameter + (``TD = TypedDict("TD", None)``) is also deprecated. Both will be disallowed + in Python 3.15. To create a TypedDict class with 0 fields, use ``class + TD(TypedDict): pass`` or ``TD = TypedDict("TD", {})``. + +* :mod:`wave`: Deprecate the ``getmark()``, ``setmark()`` and ``getmarkers()`` + methods of the :class:`wave.Wave_read` and :class:`wave.Wave_write` classes. + They will be removed in Python 3.15. + (Contributed by Victor Stinner in :gh:`105096`.) + +Pending Removal in Python 3.16 +------------------------------ + +* :class:`array.array` ``'u'`` type (``wchar_t``): + use the ``'w'`` type instead (``Py_UCS4``). + +Pending Removal in Future Versions +---------------------------------- + +The following APIs were deprecated in earlier Python versions and will be removed, +although there is currently no date scheduled for their removal. + +* :mod:`argparse`: Nesting argument groups and nesting mutually exclusive + groups are deprecated. + +* :mod:`builtins`: + + * ``~bool``, bitwise inversion on bool. + * ``bool(NotImplemented)``. + * Generators: ``throw(type, exc, tb)`` and ``athrow(type, exc, tb)`` + signature is deprecated: use ``throw(exc)`` and ``athrow(exc)`` instead, + the single argument signature. + * Currently Python accepts numeric literals immediately followed by keywords, + for example ``0in x``, ``1or x``, ``0if 1else 2``. It allows confusing and + ambiguous expressions like ``[0x1for x in y]`` (which can be interpreted as + ``[0x1 for x in y]`` or ``[0x1f or x in y]``). A syntax warning is raised + if the numeric literal is immediately followed by one of keywords + :keyword:`and`, :keyword:`else`, :keyword:`for`, :keyword:`if`, + :keyword:`in`, :keyword:`is` and :keyword:`or`. In a future release it + will be changed to a syntax error. (:gh:`87999`) + * Support for ``__index__()`` and ``__int__()`` method returning non-int type: + these methods will be required to return an instance of a strict subclass of + :class:`int`. + * Support for ``__float__()`` method returning a strict subclass of + :class:`float`: these methods will be required to return an instance of + :class:`float`. + * Support for ``__complex__()`` method returning a strict subclass of + :class:`complex`: these methods will be required to return an instance of + :class:`complex`. + * Delegation of ``int()`` to ``__trunc__()`` method. + +* :mod:`calendar`: ``calendar.January`` and ``calendar.February`` constants are + deprecated and replaced by :data:`calendar.Month.JANUARY` and + :data:`calendar.Month.FEBRUARY`. + (Contributed by Prince Roshan in :gh:`103636`.) + +* :mod:`datetime`: + + * :meth:`~datetime.datetime.utcnow`: + use ``datetime.datetime.now(tz=datetime.UTC)``. + * :meth:`~datetime.datetime.utcfromtimestamp`: + use ``datetime.datetime.fromtimestamp(timestamp, tz=datetime.UTC)``. + +* :mod:`gettext`: Plural value must be an integer. + +* :mod:`importlib`: + + * ``load_module()`` method: use ``exec_module()`` instead. + * :func:`~importlib.util.cache_from_source` *debug_override* parameter is + deprecated: use the *optimization* parameter instead. + +* :mod:`importlib.metadata`: + + * ``EntryPoints`` tuple interface. + * Implicit ``None`` on return values. + +* :mod:`importlib.resources`: First parameter to files is renamed to 'anchor'. +* :mod:`importlib.resources` deprecated methods: + + * ``contents()`` + * ``is_resource()`` + * ``open_binary()`` + * ``open_text()`` + * ``path()`` + * ``read_binary()`` + * ``read_text()`` + + Use ``files()`` instead. Refer to `importlib-resources: Migrating from Legacy + `_ + for migration advice. + +* :func:`locale.getdefaultlocale`: use :func:`locale.setlocale()`, + :func:`locale.getencoding()` and :func:`locale.getlocale()` instead + (:gh:`90817`) + +* :mod:`mailbox`: Use of StringIO input and text mode is deprecated, use + BytesIO and binary mode instead. + +* :mod:`os`: Calling :func:`os.register_at_fork` in multi-threaded process. + +* :class:`!pydoc.ErrorDuringImport`: A tuple value for *exc_info* parameter is + deprecated, use an exception instance. + +* :mod:`re`: bad character in group name. + +* :mod:`ssl` options and protocols: + + * :class:`ssl.SSLContext` without protocol argument is deprecated. + * :class:`ssl.SSLContext`: :meth:`~ssl.SSLContext.set_npn_protocols` and + :meth:`!~ssl.SSLContext.selected_npn_protocol` are deprecated: use ALPN + instead. + * ``ssl.OP_NO_SSL*`` options + * ``ssl.OP_NO_TLS*`` options + * ``ssl.PROTOCOL_SSLv3`` + * ``ssl.PROTOCOL_TLS`` + * ``ssl.PROTOCOL_TLSv1`` + * ``ssl.PROTOCOL_TLSv1_1`` + * ``ssl.PROTOCOL_TLSv1_2`` + * ``ssl.TLSVersion.SSLv3`` + * ``ssl.TLSVersion.TLSv1`` + * ``ssl.TLSVersion.TLSv1_1`` + +* :mod:`!sre_compile`, :mod:`!sre_constants` and :mod:`!sre_parse` modules. + +* ``types.CodeType.co_lnotab``: use the ``co_lines`` attribute instead. + +* :class:`typing.Text` (:gh:`92332`). + +* :func:`sysconfig.is_python_build` *check_home* parameter is deprecated and + ignored. + +* :mod:`threading` methods: + + * :meth:`!threading.Condition.notifyAll`: use :meth:`~threading.Condition.notify_all`. + * :meth:`!threading.Event.isSet`: use :meth:`~threading.Event.is_set`. + * :meth:`!threading.Thread.isDaemon`, :meth:`threading.Thread.setDaemon`: + use :attr:`threading.Thread.daemon` attribute. + * :meth:`!threading.Thread.getName`, :meth:`threading.Thread.setName`: + use :attr:`threading.Thread.name` attribute. + * :meth:`!threading.currentThread`: use :meth:`threading.current_thread`. + * :meth:`!threading.activeCount`: use :meth:`threading.active_count`. + +* :class:`unittest.IsolatedAsyncioTestCase`: it is deprecated to return a value + that is not None from a test case. + +* :mod:`urllib.request`: :class:`~urllib.request.URLopener` and + :class:`~urllib.request.FancyURLopener` style of invoking requests is + deprecated. Use newer :func:`~urllib.request.urlopen` functions and methods. + +* :func:`!urllib.parse.to_bytes`. + +* :mod:`urllib.parse` deprecated functions: :func:`~urllib.parse.urlparse` instead + + * ``splitattr()`` + * ``splithost()`` + * ``splitnport()`` + * ``splitpasswd()`` + * ``splitport()`` + * ``splitquery()`` + * ``splittag()`` + * ``splittype()`` + * ``splituser()`` + * ``splitvalue()`` + +* :mod:`wsgiref`: ``SimpleHandler.stdout.write()`` should not do partial + writes. + +* :meth:`zipimport.zipimporter.load_module` is deprecated: + use :meth:`~zipimport.zipimporter.exec_module` instead. + Removed ======= @@ -618,6 +909,8 @@ Removed Pending Removal in Python 3.14 ------------------------------ +* Creating immutable types (:data:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable + bases using the C API. * Global configuration variables: * :c:var:`Py_DebugFlag`: use :c:member:`PyConfig.parser_debug` From webhook-mailer at python.org Sun Jul 9 11:50:29 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 09 Jul 2023 15:50:29 -0000 Subject: [Python-checkins] gh-105927: PyWeakref_GetRef() returns 1 on success (#106561) Message-ID: https://github.com/python/cpython/commit/ee46cb6aa959d891b0a480fea29f1eb991e0fad8 commit: ee46cb6aa959d891b0a480fea29f1eb991e0fad8 branch: main author: Victor Stinner committer: vstinner date: 2023-07-09T15:50:26Z summary: gh-105927: PyWeakref_GetRef() returns 1 on success (#106561) PyWeakref_GetRef() now returns 1 on success, and return 0 if the reference is dead. Co-authored-by: Serhiy Storchaka files: M Doc/c-api/weakref.rst M Modules/_testcapimodule.c M Objects/weakrefobject.c diff --git a/Doc/c-api/weakref.rst b/Doc/c-api/weakref.rst index 04781f78d2346..038f54a9751fd 100644 --- a/Doc/c-api/weakref.rst +++ b/Doc/c-api/weakref.rst @@ -55,9 +55,11 @@ as much as it can. Get a :term:`strong reference` to the referenced object from a weak reference, *ref*, into *\*pobj*. - Return 0 on success. Raise an exception and return -1 on error. - If the referent is no longer live, set *\*pobj* to ``NULL`` and return 0. + * On success, set *\*pobj* to a new :term:`strong reference` to the + referenced object and return 1. + * If the reference is dead, set *\*pobj* to ``NULL`` and return 0. + * On error, raise an exception and return -1. .. versionadded:: 3.13 diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 2baf453f71026..50eaff9917fd1 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -3376,7 +3376,7 @@ test_weakref_capi(PyObject *Py_UNUSED(module), PyObject *Py_UNUSED(args)) // test PyWeakref_GetRef(), reference is alive PyObject *ref = Py_True; // marker to check that value was set - assert(PyWeakref_GetRef(weakref, &ref) == 0); + assert(PyWeakref_GetRef(weakref, &ref) == 1); assert(ref == obj); assert(Py_REFCNT(obj) == (refcnt + 1)); Py_DECREF(ref); diff --git a/Objects/weakrefobject.c b/Objects/weakrefobject.c index bac3e79bb7c25..e9563729bf82b 100644 --- a/Objects/weakrefobject.c +++ b/Objects/weakrefobject.c @@ -913,7 +913,7 @@ PyWeakref_GetRef(PyObject *ref, PyObject **pobj) return -1; } *pobj = _PyWeakref_GET_REF(ref); - return 0; + return (*pobj != NULL); } From webhook-mailer at python.org Sun Jul 9 16:46:19 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sun, 09 Jul 2023 20:46:19 -0000 Subject: [Python-checkins] gh-106461: typing: Consolidate docs on `Callable` (#106462) Message-ID: https://github.com/python/cpython/commit/ca8b55c7f54b38e264056148075a8061a7082013 commit: ca8b55c7f54b38e264056148075a8061a7082013 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-09T21:46:15+01:00 summary: gh-106461: typing: Consolidate docs on `Callable` (#106462) files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 7ac1062eb26d7..11af3ea3c9030 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -258,18 +258,21 @@ See :pep:`484` for more details. The performance of calling ``NewType`` has been restored to its level in Python 3.9. +.. _annotating-callables: -Callable -======== +Annotating callable objects +=========================== -Frameworks expecting callback functions of specific signatures might be -type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. +Functions -- or other :term:`callable` objects -- can be annotated using +:class:`collections.abc.Callable` or :data:`typing.Callable`. +``Callable[[int], str]`` signifies a function that takes a single parameter +of type :class:`int` and returns a :class:`str`. For example: .. testcode:: - from collections.abc import Callable + from collections.abc import Callable, Awaitable def feeder(get_next_item: Callable[[], str]) -> None: ... # Body @@ -283,9 +286,49 @@ For example: callback: Callable[[str], Awaitable[None]] = on_update -It is possible to declare the return type of a callable without specifying -the call signature by substituting a literal ellipsis -for the list of arguments in the type hint: ``Callable[..., ReturnType]``. +The subscription syntax must always be used with exactly two values: the +argument list and the return type. The argument list must be a list of types, +a :class:`ParamSpec`, :data:`Concatenate`, or an ellipsis. The return type must +be a single type. + +If a literal ellipsis ``...`` is given as the argument list, it indicates that +a callable with any arbitrary parameter list would be acceptable: + +.. testcode:: + + def concat(x: str, y: str) -> str: + return x + y + + x: Callable[..., str] + x = str # OK + x = concat # Also OK + +``Callable`` cannot express complex signatures such as functions that take a +variadic number of arguments, :func:`overloaded functions `, or +functions that have keyword-only parameters. However, these signatures can be +expressed by defining a :class:`Protocol` class with a +:meth:`~object.__call__` method: + +.. testcode:: + + from collections.abc import Iterable + from typing import Protocol + + class Combiner(Protocol): + def __call__(self, *vals: bytes, maxlen: int | None = None) -> list[bytes]: ... + + def batch_proc(data: Iterable[bytes], cb_results: Combiner) -> bytes: + for item in data: + ... + + def good_cb(*vals: bytes, maxlen: int | None = None) -> list[bytes]: + ... + def bad_cb(*vals: bytes, maxitems: int | None) -> list[bytes]: + ... + + batch_proc([], good_cb) # OK + batch_proc([], bad_cb) # Error! Argument 2 has incompatible type because of + # different name and kind in the callback Callables which take other callables as arguments may indicate that their parameter types are dependent on each other using :class:`ParamSpec`. @@ -1043,56 +1086,16 @@ These can be used as types in annotations. They all support subscription using Optional can now be written as ``X | None``. See :ref:`union type expressions`. -.. data:: Callable - - Deprecated alias to :class:`collections.abc.Callable`. - - ``Callable[[int], str]`` signifies a function that takes a single parameter - of type :class:`int` and returns a :class:`str`. - - The subscription syntax must always be used with exactly two - values: the argument list and the return type. The argument list - must be a list of types, a :class:`ParamSpec`, :data:`Concatenate`, - or an ellipsis. The return type must be a single type. - - There is no syntax to indicate optional or keyword arguments; - such function types are rarely used as callback types. - ``Callable[..., ReturnType]`` (literal ellipsis) can be used to - type hint a callable taking any number of arguments and returning - ``ReturnType``. A plain :data:`Callable` is equivalent to - ``Callable[..., Any]``, and in turn to - :class:`collections.abc.Callable`. - - Callables which take other callables as arguments may indicate that their - parameter types are dependent on each other using :class:`ParamSpec`. - Additionally, if that callable adds or removes arguments from other - callables, the :data:`Concatenate` operator may be used. They - take the form ``Callable[ParamSpecVariable, ReturnType]`` and - ``Callable[Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable], ReturnType]`` - respectively. - - .. deprecated:: 3.9 - :class:`collections.abc.Callable` now supports subscripting (``[]``). - See :pep:`585` and :ref:`types-genericalias`. - - .. versionchanged:: 3.10 - ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`. - See :pep:`612` for more details. - - .. seealso:: - The documentation for :class:`ParamSpec` and :class:`Concatenate` provide - examples of usage with ``Callable``. - .. data:: Concatenate Special form for annotating higher-order functions. - ``Concatenate`` can be used in conjunction with :data:`Callable` and + ``Concatenate`` can be used in conjunction with :ref:`Callable ` and :class:`ParamSpec` to annotate a higher-order callable which adds, removes, or transforms parameters of another callable. Usage is in the form ``Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable]``. ``Concatenate`` - is currently only valid when used as the first argument to a :data:`Callable`. + is currently only valid when used as the first argument to a :ref:`Callable `. The last parameter to ``Concatenate`` must be a :class:`ParamSpec` or ellipsis (``...``). @@ -1136,8 +1139,9 @@ These can be used as types in annotations. They all support subscription using .. seealso:: * :pep:`612` -- Parameter Specification Variables (the PEP which introduced - ``ParamSpec`` and ``Concatenate``). - * :class:`ParamSpec` and :class:`Callable`. + ``ParamSpec`` and ``Concatenate``) + * :class:`ParamSpec` + * :ref:`annotating-callables` .. data:: Literal @@ -1893,8 +1897,9 @@ without the dedicated syntax, as documented below. .. seealso:: * :pep:`612` -- Parameter Specification Variables (the PEP which introduced - ``ParamSpec`` and ``Concatenate``). - * :class:`Callable` and :class:`Concatenate`. + ``ParamSpec`` and ``Concatenate``) + * :data:`Concatenate` + * :ref:`annotating-callables` .. data:: ParamSpecArgs .. data:: ParamSpecKwargs @@ -2179,7 +2184,7 @@ types. methods or attributes, not their type signatures or types. For example, :class:`ssl.SSLObject` is a class, therefore it passes an :func:`issubclass` - check against :data:`Callable`. However, the + check against :ref:`Callable `. However, the ``ssl.SSLObject.__init__`` method exists only to raise a :exc:`TypeError` with a more informative message, therefore making it impossible to call (instantiate) :class:`ssl.SSLObject`. @@ -3533,6 +3538,21 @@ Aliases to other ABCs in :mod:`collections.abc` :class:`collections.abc.Iterator` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. +.. data:: Callable + + Deprecated alias to :class:`collections.abc.Callable`. + + See :ref:`annotating-callables` for details on how to use + :class:`collections.abc.Callable` and ``typing.Callable`` in type annotations. + + .. deprecated:: 3.9 + :class:`collections.abc.Callable` now supports subscripting (``[]``). + See :pep:`585` and :ref:`types-genericalias`. + + .. versionchanged:: 3.10 + ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`. + See :pep:`612` for more details. + .. class:: Generator(Iterator[YieldType], Generic[YieldType, SendType, ReturnType]) Deprecated alias to :class:`collections.abc.Generator`. From webhook-mailer at python.org Sun Jul 9 17:21:58 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sun, 09 Jul 2023 21:21:58 -0000 Subject: [Python-checkins] [3.12] gh-106461: typing: Consolidate docs on `Callable` (GH-106462) (#106574) Message-ID: https://github.com/python/cpython/commit/128a962482ff29325e34b1400c87cc70d509e02a commit: 128a962482ff29325e34b1400c87cc70d509e02a branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-09T22:21:54+01:00 summary: [3.12] gh-106461: typing: Consolidate docs on `Callable` (GH-106462) (#106574) gh-106461: typing: Consolidate docs on `Callable` (GH-106462) (cherry picked from commit ca8b55c7f54b38e264056148075a8061a7082013) Co-authored-by: Alex Waygood files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 12a20ae543c77..2a712a4be5890 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -258,18 +258,21 @@ See :pep:`484` for more details. The performance of calling ``NewType`` has been restored to its level in Python 3.9. +.. _annotating-callables: -Callable -======== +Annotating callable objects +=========================== -Frameworks expecting callback functions of specific signatures might be -type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. +Functions -- or other :term:`callable` objects -- can be annotated using +:class:`collections.abc.Callable` or :data:`typing.Callable`. +``Callable[[int], str]`` signifies a function that takes a single parameter +of type :class:`int` and returns a :class:`str`. For example: .. testcode:: - from collections.abc import Callable + from collections.abc import Callable, Awaitable def feeder(get_next_item: Callable[[], str]) -> None: ... # Body @@ -283,9 +286,49 @@ For example: callback: Callable[[str], Awaitable[None]] = on_update -It is possible to declare the return type of a callable without specifying -the call signature by substituting a literal ellipsis -for the list of arguments in the type hint: ``Callable[..., ReturnType]``. +The subscription syntax must always be used with exactly two values: the +argument list and the return type. The argument list must be a list of types, +a :class:`ParamSpec`, :data:`Concatenate`, or an ellipsis. The return type must +be a single type. + +If a literal ellipsis ``...`` is given as the argument list, it indicates that +a callable with any arbitrary parameter list would be acceptable: + +.. testcode:: + + def concat(x: str, y: str) -> str: + return x + y + + x: Callable[..., str] + x = str # OK + x = concat # Also OK + +``Callable`` cannot express complex signatures such as functions that take a +variadic number of arguments, :func:`overloaded functions `, or +functions that have keyword-only parameters. However, these signatures can be +expressed by defining a :class:`Protocol` class with a +:meth:`~object.__call__` method: + +.. testcode:: + + from collections.abc import Iterable + from typing import Protocol + + class Combiner(Protocol): + def __call__(self, *vals: bytes, maxlen: int | None = None) -> list[bytes]: ... + + def batch_proc(data: Iterable[bytes], cb_results: Combiner) -> bytes: + for item in data: + ... + + def good_cb(*vals: bytes, maxlen: int | None = None) -> list[bytes]: + ... + def bad_cb(*vals: bytes, maxitems: int | None) -> list[bytes]: + ... + + batch_proc([], good_cb) # OK + batch_proc([], bad_cb) # Error! Argument 2 has incompatible type because of + # different name and kind in the callback Callables which take other callables as arguments may indicate that their parameter types are dependent on each other using :class:`ParamSpec`. @@ -1043,56 +1086,16 @@ These can be used as types in annotations. They all support subscription using Optional can now be written as ``X | None``. See :ref:`union type expressions`. -.. data:: Callable - - Deprecated alias to :class:`collections.abc.Callable`. - - ``Callable[[int], str]`` signifies a function that takes a single parameter - of type :class:`int` and returns a :class:`str`. - - The subscription syntax must always be used with exactly two - values: the argument list and the return type. The argument list - must be a list of types, a :class:`ParamSpec`, :data:`Concatenate`, - or an ellipsis. The return type must be a single type. - - There is no syntax to indicate optional or keyword arguments; - such function types are rarely used as callback types. - ``Callable[..., ReturnType]`` (literal ellipsis) can be used to - type hint a callable taking any number of arguments and returning - ``ReturnType``. A plain :data:`Callable` is equivalent to - ``Callable[..., Any]``, and in turn to - :class:`collections.abc.Callable`. - - Callables which take other callables as arguments may indicate that their - parameter types are dependent on each other using :class:`ParamSpec`. - Additionally, if that callable adds or removes arguments from other - callables, the :data:`Concatenate` operator may be used. They - take the form ``Callable[ParamSpecVariable, ReturnType]`` and - ``Callable[Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable], ReturnType]`` - respectively. - - .. deprecated:: 3.9 - :class:`collections.abc.Callable` now supports subscripting (``[]``). - See :pep:`585` and :ref:`types-genericalias`. - - .. versionchanged:: 3.10 - ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`. - See :pep:`612` for more details. - - .. seealso:: - The documentation for :class:`ParamSpec` and :class:`Concatenate` provide - examples of usage with ``Callable``. - .. data:: Concatenate Special form for annotating higher-order functions. - ``Concatenate`` can be used in conjunction with :data:`Callable` and + ``Concatenate`` can be used in conjunction with :ref:`Callable ` and :class:`ParamSpec` to annotate a higher-order callable which adds, removes, or transforms parameters of another callable. Usage is in the form ``Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable]``. ``Concatenate`` - is currently only valid when used as the first argument to a :data:`Callable`. + is currently only valid when used as the first argument to a :ref:`Callable `. The last parameter to ``Concatenate`` must be a :class:`ParamSpec` or ellipsis (``...``). @@ -1136,8 +1139,9 @@ These can be used as types in annotations. They all support subscription using .. seealso:: * :pep:`612` -- Parameter Specification Variables (the PEP which introduced - ``ParamSpec`` and ``Concatenate``). - * :class:`ParamSpec` and :class:`Callable`. + ``ParamSpec`` and ``Concatenate``) + * :class:`ParamSpec` + * :ref:`annotating-callables` .. data:: Literal @@ -1893,8 +1897,9 @@ without the dedicated syntax, as documented below. .. seealso:: * :pep:`612` -- Parameter Specification Variables (the PEP which introduced - ``ParamSpec`` and ``Concatenate``). - * :class:`Callable` and :class:`Concatenate`. + ``ParamSpec`` and ``Concatenate``) + * :data:`Concatenate` + * :ref:`annotating-callables` .. data:: ParamSpecArgs .. data:: ParamSpecKwargs @@ -2166,7 +2171,7 @@ types. methods or attributes, not their type signatures or types. For example, :class:`ssl.SSLObject` is a class, therefore it passes an :func:`issubclass` - check against :data:`Callable`. However, the + check against :ref:`Callable `. However, the ``ssl.SSLObject.__init__`` method exists only to raise a :exc:`TypeError` with a more informative message, therefore making it impossible to call (instantiate) :class:`ssl.SSLObject`. @@ -3496,6 +3501,21 @@ Aliases to other ABCs in :mod:`collections.abc` :class:`collections.abc.Iterator` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. +.. data:: Callable + + Deprecated alias to :class:`collections.abc.Callable`. + + See :ref:`annotating-callables` for details on how to use + :class:`collections.abc.Callable` and ``typing.Callable`` in type annotations. + + .. deprecated:: 3.9 + :class:`collections.abc.Callable` now supports subscripting (``[]``). + See :pep:`585` and :ref:`types-genericalias`. + + .. versionchanged:: 3.10 + ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`. + See :pep:`612` for more details. + .. class:: Generator(Iterator[YieldType], Generic[YieldType, SendType, ReturnType]) Deprecated alias to :class:`collections.abc.Generator`. From webhook-mailer at python.org Sun Jul 9 17:22:49 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sun, 09 Jul 2023 21:22:49 -0000 Subject: [Python-checkins] [3.11] gh-106461: typing: Consolidate docs on `Callable` (GH-106462) (#106575) Message-ID: https://github.com/python/cpython/commit/dd046973379cdc5cab47d3ddc878521b175f10c8 commit: dd046973379cdc5cab47d3ddc878521b175f10c8 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-09T22:22:46+01:00 summary: [3.11] gh-106461: typing: Consolidate docs on `Callable` (GH-106462) (#106575) gh-106461: typing: Consolidate docs on `Callable` (GH-106462) (cherry picked from commit ca8b55c7f54b38e264056148075a8061a7082013) Co-authored-by: Alex Waygood files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index da7c7f3b904ea..25b7cfe703a13 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -244,18 +244,21 @@ See :pep:`484` for more details. The performance of calling ``NewType`` has been restored to its level in Python 3.9. +.. _annotating-callables: -Callable -======== +Annotating callable objects +=========================== -Frameworks expecting callback functions of specific signatures might be -type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. +Functions -- or other :term:`callable` objects -- can be annotated using +:class:`collections.abc.Callable` or :data:`typing.Callable`. +``Callable[[int], str]`` signifies a function that takes a single parameter +of type :class:`int` and returns a :class:`str`. For example: .. testcode:: - from collections.abc import Callable + from collections.abc import Callable, Awaitable def feeder(get_next_item: Callable[[], str]) -> None: ... # Body @@ -269,9 +272,49 @@ For example: callback: Callable[[str], Awaitable[None]] = on_update -It is possible to declare the return type of a callable without specifying -the call signature by substituting a literal ellipsis -for the list of arguments in the type hint: ``Callable[..., ReturnType]``. +The subscription syntax must always be used with exactly two values: the +argument list and the return type. The argument list must be a list of types, +a :class:`ParamSpec`, :data:`Concatenate`, or an ellipsis. The return type must +be a single type. + +If a literal ellipsis ``...`` is given as the argument list, it indicates that +a callable with any arbitrary parameter list would be acceptable: + +.. testcode:: + + def concat(x: str, y: str) -> str: + return x + y + + x: Callable[..., str] + x = str # OK + x = concat # Also OK + +``Callable`` cannot express complex signatures such as functions that take a +variadic number of arguments, :func:`overloaded functions `, or +functions that have keyword-only parameters. However, these signatures can be +expressed by defining a :class:`Protocol` class with a +:meth:`~object.__call__` method: + +.. testcode:: + + from collections.abc import Iterable + from typing import Protocol + + class Combiner(Protocol): + def __call__(self, *vals: bytes, maxlen: int | None = None) -> list[bytes]: ... + + def batch_proc(data: Iterable[bytes], cb_results: Combiner) -> bytes: + for item in data: + ... + + def good_cb(*vals: bytes, maxlen: int | None = None) -> list[bytes]: + ... + def bad_cb(*vals: bytes, maxitems: int | None) -> list[bytes]: + ... + + batch_proc([], good_cb) # OK + batch_proc([], bad_cb) # Error! Argument 2 has incompatible type because of + # different name and kind in the callback Callables which take other callables as arguments may indicate that their parameter types are dependent on each other using :class:`ParamSpec`. @@ -985,56 +1028,16 @@ These can be used as types in annotations. They all support subscription using Optional can now be written as ``X | None``. See :ref:`union type expressions`. -.. data:: Callable - - Deprecated alias to :class:`collections.abc.Callable`. - - ``Callable[[int], str]`` signifies a function that takes a single parameter - of type :class:`int` and returns a :class:`str`. - - The subscription syntax must always be used with exactly two - values: the argument list and the return type. The argument list - must be a list of types, a :class:`ParamSpec`, :data:`Concatenate`, - or an ellipsis. The return type must be a single type. - - There is no syntax to indicate optional or keyword arguments; - such function types are rarely used as callback types. - ``Callable[..., ReturnType]`` (literal ellipsis) can be used to - type hint a callable taking any number of arguments and returning - ``ReturnType``. A plain :data:`Callable` is equivalent to - ``Callable[..., Any]``, and in turn to - :class:`collections.abc.Callable`. - - Callables which take other callables as arguments may indicate that their - parameter types are dependent on each other using :class:`ParamSpec`. - Additionally, if that callable adds or removes arguments from other - callables, the :data:`Concatenate` operator may be used. They - take the form ``Callable[ParamSpecVariable, ReturnType]`` and - ``Callable[Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable], ReturnType]`` - respectively. - - .. deprecated:: 3.9 - :class:`collections.abc.Callable` now supports subscripting (``[]``). - See :pep:`585` and :ref:`types-genericalias`. - - .. versionchanged:: 3.10 - ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`. - See :pep:`612` for more details. - - .. seealso:: - The documentation for :class:`ParamSpec` and :class:`Concatenate` provide - examples of usage with ``Callable``. - .. data:: Concatenate Special form for annotating higher-order functions. - ``Concatenate`` can be used in conjunction with :data:`Callable` and + ``Concatenate`` can be used in conjunction with :ref:`Callable ` and :class:`ParamSpec` to annotate a higher-order callable which adds, removes, or transforms parameters of another callable. Usage is in the form ``Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable]``. ``Concatenate`` - is currently only valid when used as the first argument to a :data:`Callable`. + is currently only valid when used as the first argument to a :ref:`Callable `. The last parameter to ``Concatenate`` must be a :class:`ParamSpec` or ellipsis (``...``). @@ -1078,8 +1081,9 @@ These can be used as types in annotations. They all support subscription using .. seealso:: * :pep:`612` -- Parameter Specification Variables (the PEP which introduced - ``ParamSpec`` and ``Concatenate``). - * :class:`ParamSpec` and :class:`Callable`. + ``ParamSpec`` and ``Concatenate``) + * :class:`ParamSpec` + * :ref:`annotating-callables` .. data:: Literal @@ -1734,8 +1738,9 @@ for creating generic types. .. seealso:: * :pep:`612` -- Parameter Specification Variables (the PEP which introduced - ``ParamSpec`` and ``Concatenate``). - * :class:`Callable` and :class:`Concatenate`. + ``ParamSpec`` and ``Concatenate``) + * :data:`Concatenate` + * :ref:`annotating-callables` .. data:: ParamSpecArgs .. data:: ParamSpecKwargs @@ -1929,7 +1934,7 @@ types. methods or attributes, not their type signatures or types. For example, :class:`ssl.SSLObject` is a class, therefore it passes an :func:`issubclass` - check against :data:`Callable`. However, the + check against :ref:`Callable `. However, the ``ssl.SSLObject.__init__`` method exists only to raise a :exc:`TypeError` with a more informative message, therefore making it impossible to call (instantiate) :class:`ssl.SSLObject`. @@ -3186,6 +3191,21 @@ Aliases to other ABCs in :mod:`collections.abc` :class:`collections.abc.Iterator` now supports subscripting (``[]``). See :pep:`585` and :ref:`types-genericalias`. +.. data:: Callable + + Deprecated alias to :class:`collections.abc.Callable`. + + See :ref:`annotating-callables` for details on how to use + :class:`collections.abc.Callable` and ``typing.Callable`` in type annotations. + + .. deprecated:: 3.9 + :class:`collections.abc.Callable` now supports subscripting (``[]``). + See :pep:`585` and :ref:`types-genericalias`. + + .. versionchanged:: 3.10 + ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`. + See :pep:`612` for more details. + .. class:: Generator(Iterator[YieldType], Generic[YieldType, SendType, ReturnType]) Deprecated alias to :class:`collections.abc.Generator`. From webhook-mailer at python.org Sun Jul 9 22:05:41 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 10 Jul 2023 02:05:41 -0000 Subject: [Python-checkins] gh-105733: Fix ctypes What's New entry (#106576) Message-ID: https://github.com/python/cpython/commit/970982e03d34655df262e14a5efcfdc0bddc0add commit: 970982e03d34655df262e14a5efcfdc0bddc0add branch: main author: Victor Stinner committer: vstinner date: 2023-07-10T04:05:38+02:00 summary: gh-105733: Fix ctypes What's New entry (#106576) files: M Doc/whatsnew/3.13.rst diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index a01fc3b34b6fe..767e78bf2470a 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -169,7 +169,7 @@ Deprecated * :mod:`ctypes`: Deprecate undocumented :func:`!ctypes.SetPointerType` and :func:`!ctypes.ARRAY` functions. - Replace ``ctypes.SetPointerType(item_type, size)`` with ``item_type * size``. + Replace ``ctypes.ARRAY(item_type, size)`` with ``item_type * size``. (Contributed by Victor Stinner in :gh:`105733`.) Pending Removal in Python 3.14 From webhook-mailer at python.org Sun Jul 9 23:41:35 2023 From: webhook-mailer at python.org (JelleZijlstra) Date: Mon, 10 Jul 2023 03:41:35 -0000 Subject: [Python-checkins] Clarify how topics.py gets created. (#106121) Message-ID: https://github.com/python/cpython/commit/dac1e364901d3668742e6eecc2ce63586330c11f commit: dac1e364901d3668742e6eecc2ce63586330c11f branch: main author: Ned Batchelder committer: JelleZijlstra date: 2023-07-09T20:41:31-07:00 summary: Clarify how topics.py gets created. (#106121) When changing docs, it was easy to find text in topics.py, and I wondered whether I was supposed to edit it. Thankfully, the top of the file says it's auto-generated, so I knew I didn't have to edit it. But I didn't know what started the auto-generation process. It's part of the release process, so I'll leave a note here for future editors. files: M Doc/tools/extensions/pyspecific.py diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py index 795aeed1007a7..8d99b0bfa4f38 100644 --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -559,6 +559,7 @@ def finish(self): try: f.write('# -*- coding: utf-8 -*-\n'.encode('utf-8')) f.write(('# Autogenerated by Sphinx on %s\n' % asctime()).encode('utf-8')) + f.write('# as part of the release process.\n'.encode('utf-8')) f.write(('topics = ' + pformat(self.topics) + '\n').encode('utf-8')) finally: f.close() From webhook-mailer at python.org Sun Jul 9 23:42:26 2023 From: webhook-mailer at python.org (JelleZijlstra) Date: Mon, 10 Jul 2023 03:42:26 -0000 Subject: [Python-checkins] [3.12] Clarify how topics.py gets created. (GH-106121) (#106579) Message-ID: https://github.com/python/cpython/commit/0481b805d6631063887fcbcc27684aa8a2576fae commit: 0481b805d6631063887fcbcc27684aa8a2576fae branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: JelleZijlstra date: 2023-07-09T20:42:22-07:00 summary: [3.12] Clarify how topics.py gets created. (GH-106121) (#106579) Clarify how topics.py gets created. (GH-106121) When changing docs, it was easy to find text in topics.py, and I wondered whether I was supposed to edit it. Thankfully, the top of the file says it's auto-generated, so I knew I didn't have to edit it. But I didn't know what started the auto-generation process. It's part of the release process, so I'll leave a note here for future editors. (cherry picked from commit dac1e364901d3668742e6eecc2ce63586330c11f) Co-authored-by: Ned Batchelder files: M Doc/tools/extensions/pyspecific.py diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py index 8a9e2fc61327e..003229d8a5975 100644 --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -559,6 +559,7 @@ def finish(self): try: f.write('# -*- coding: utf-8 -*-\n'.encode('utf-8')) f.write(('# Autogenerated by Sphinx on %s\n' % asctime()).encode('utf-8')) + f.write('# as part of the release process.\n'.encode('utf-8')) f.write(('topics = ' + pformat(self.topics) + '\n').encode('utf-8')) finally: f.close() From webhook-mailer at python.org Mon Jul 10 05:52:40 2023 From: webhook-mailer at python.org (hugovk) Date: Mon, 10 Jul 2023 09:52:40 -0000 Subject: [Python-checkins] gh-106487: Allow the 'count' argument of `str.replace` to be a keyword (#106488) Message-ID: https://github.com/python/cpython/commit/34c14147a2c52930b8b471905074509639e82d5b commit: 34c14147a2c52930b8b471905074509639e82d5b branch: main author: Hugo van Kemenade committer: hugovk date: 2023-07-10T12:52:36+03:00 summary: gh-106487: Allow the 'count' argument of `str.replace` to be a keyword (#106488) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-06-22-46-05.gh-issue-106487.u3KfAD.rst M Doc/library/stdtypes.rst M Doc/whatsnew/3.13.rst M Lib/test/string_tests.py M Objects/clinic/unicodeobject.c.h M Objects/unicodeobject.c diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 0ac727e0df38b..8e049d9645c0b 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -2014,11 +2014,14 @@ expression support in the :mod:`re` module). .. versionadded:: 3.9 -.. method:: str.replace(old, new[, count]) +.. method:: str.replace(old, new, count=-1) Return a copy of the string with all occurrences of substring *old* replaced by - *new*. If the optional argument *count* is given, only the first *count* - occurrences are replaced. + *new*. If *count* is given, only the first *count* occurrences are replaced. + If *count* is not specified or ``-1``, then all occurrences are replaced. + + .. versionchanged:: 3.13 + *count* is now supported as a keyword argument. .. method:: str.rfind(sub[, start[, end]]) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 767e78bf2470a..8fc809deca139 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -76,7 +76,8 @@ New Features Other Language Changes ====================== - +* Allow the *count* argument of :meth:`str.replace` to be a keyword. + (Contributed by Hugo van Kemenade in :gh:`106487`.) New Modules =========== diff --git a/Lib/test/string_tests.py b/Lib/test/string_tests.py index a6ea2f378b37a..8d210b198d248 100644 --- a/Lib/test/string_tests.py +++ b/Lib/test/string_tests.py @@ -154,6 +154,12 @@ def test_count(self): self.assertEqual(rem, 0, '%s != 0 for %s' % (rem, i)) self.assertEqual(r1, r2, '%s != %s for %s' % (r1, r2, i)) + def test_count_keyword(self): + self.assertEqual('aa'.replace('a', 'b', 0), 'aa'.replace('a', 'b', count=0)) + self.assertEqual('aa'.replace('a', 'b', 1), 'aa'.replace('a', 'b', count=1)) + self.assertEqual('aa'.replace('a', 'b', 2), 'aa'.replace('a', 'b', count=2)) + self.assertEqual('aa'.replace('a', 'b', 3), 'aa'.replace('a', 'b', count=3)) + def test_find(self): self.checkequal(0, 'abcdefghiabc', 'find', 'abc') self.checkequal(9, 'abcdefghiabc', 'find', 'abc', 1) diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-06-22-46-05.gh-issue-106487.u3KfAD.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-06-22-46-05.gh-issue-106487.u3KfAD.rst new file mode 100644 index 0000000000000..9e8100022bbd2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-06-22-46-05.gh-issue-106487.u3KfAD.rst @@ -0,0 +1,2 @@ +Allow the *count* argument of :meth:`str.replace` to be a keyword. Patch by +Hugo van Kemenade. diff --git a/Objects/clinic/unicodeobject.c.h b/Objects/clinic/unicodeobject.c.h index 70c7fe1750171..b9127e428f4c9 100644 --- a/Objects/clinic/unicodeobject.c.h +++ b/Objects/clinic/unicodeobject.c.h @@ -736,7 +736,7 @@ unicode_rstrip(PyObject *self, PyObject *const *args, Py_ssize_t nargs) } PyDoc_STRVAR(unicode_replace__doc__, -"replace($self, old, new, count=-1, /)\n" +"replace($self, old, new, /, count=-1)\n" "--\n" "\n" "Return a copy with all occurrences of substring old replaced by new.\n" @@ -749,21 +749,49 @@ PyDoc_STRVAR(unicode_replace__doc__, "replaced."); #define UNICODE_REPLACE_METHODDEF \ - {"replace", _PyCFunction_CAST(unicode_replace), METH_FASTCALL, unicode_replace__doc__}, + {"replace", _PyCFunction_CAST(unicode_replace), METH_FASTCALL|METH_KEYWORDS, unicode_replace__doc__}, static PyObject * unicode_replace_impl(PyObject *self, PyObject *old, PyObject *new, Py_ssize_t count); static PyObject * -unicode_replace(PyObject *self, PyObject *const *args, Py_ssize_t nargs) +unicode_replace(PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(count), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"", "", "count", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "replace", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[3]; + Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 2; PyObject *old; PyObject *new; Py_ssize_t count = -1; - if (!_PyArg_CheckPositional("replace", nargs, 2, 3)) { + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 2, 3, 0, argsbuf); + if (!args) { goto exit; } if (!PyUnicode_Check(args[0])) { @@ -776,8 +804,8 @@ unicode_replace(PyObject *self, PyObject *const *args, Py_ssize_t nargs) goto exit; } new = args[1]; - if (nargs < 3) { - goto skip_optional; + if (!noptargs) { + goto skip_optional_pos; } { Py_ssize_t ival = -1; @@ -791,7 +819,7 @@ unicode_replace(PyObject *self, PyObject *const *args, Py_ssize_t nargs) } count = ival; } -skip_optional: +skip_optional_pos: return_value = unicode_replace_impl(self, old, new, count); exit: @@ -1476,4 +1504,4 @@ unicode_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) exit: return return_value; } -/*[clinic end generated code: output=0a71c4aeffdf0bc5 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=ee76a1b49cd4cbb3 input=a9049054013a1b77]*/ diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 12e379d0c7e6c..f543c0a65b49f 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -12025,10 +12025,10 @@ str.replace as unicode_replace old: unicode new: unicode + / count: Py_ssize_t = -1 Maximum number of occurrences to replace. -1 (the default value) means replace all occurrences. - / Return a copy with all occurrences of substring old replaced by new. @@ -12039,7 +12039,7 @@ replaced. static PyObject * unicode_replace_impl(PyObject *self, PyObject *old, PyObject *new, Py_ssize_t count) -/*[clinic end generated code: output=b63f1a8b5eebf448 input=147d12206276ebeb]*/ +/*[clinic end generated code: output=b63f1a8b5eebf448 input=3345c455d60a5499]*/ { return replace(self, old, new, count); } From webhook-mailer at python.org Mon Jul 10 06:40:39 2023 From: webhook-mailer at python.org (markshannon) Date: Mon, 10 Jul 2023 10:40:39 -0000 Subject: [Python-checkins] GH-100288: Specialize LOAD_ATTR for simple class attributes. (#105990) Message-ID: https://github.com/python/cpython/commit/0c90e7561046a2deb358e6695148060a1c199b49 commit: 0c90e7561046a2deb358e6695148060a1c199b49 branch: main author: Mark Shannon committer: markshannon date: 2023-07-10T11:40:35+01:00 summary: GH-100288: Specialize LOAD_ATTR for simple class attributes. (#105990) * Add two more specializations of LOAD_ATTR. files: A Misc/NEWS.d/next/Core and Builtins/2023-07-04-04-50-14.gh-issue-100288.yNQ1ez.rst M Include/internal/pycore_opcode.h M Include/opcode.h M Lib/_opcode_metadata.py M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/opcode_metadata.h M Python/opcode_targets.h M Python/specialize.c diff --git a/Include/internal/pycore_opcode.h b/Include/internal/pycore_opcode.h index c7c2fdcc32788..d7f6b84e95c4f 100644 --- a/Include/internal/pycore_opcode.h +++ b/Include/internal/pycore_opcode.h @@ -158,6 +158,8 @@ const uint8_t _PyOpcode_Deopt[256] = { [LOAD_ATTR_METHOD_NO_DICT] = LOAD_ATTR, [LOAD_ATTR_METHOD_WITH_VALUES] = LOAD_ATTR, [LOAD_ATTR_MODULE] = LOAD_ATTR, + [LOAD_ATTR_NONDESCRIPTOR_NO_DICT] = LOAD_ATTR, + [LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES] = LOAD_ATTR, [LOAD_ATTR_PROPERTY] = LOAD_ATTR, [LOAD_ATTR_SLOT] = LOAD_ATTR, [LOAD_ATTR_WITH_HINT] = LOAD_ATTR, @@ -328,14 +330,14 @@ const char *const _PyOpcode_OpName[268] = { [LOAD_ATTR_METHOD_WITH_VALUES] = "LOAD_ATTR_METHOD_WITH_VALUES", [LOAD_ATTR_METHOD_NO_DICT] = "LOAD_ATTR_METHOD_NO_DICT", [LOAD_ATTR_METHOD_LAZY_DICT] = "LOAD_ATTR_METHOD_LAZY_DICT", - [COMPARE_OP_FLOAT] = "COMPARE_OP_FLOAT", - [COMPARE_OP_INT] = "COMPARE_OP_INT", + [LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES] = "LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES", + [LOAD_ATTR_NONDESCRIPTOR_NO_DICT] = "LOAD_ATTR_NONDESCRIPTOR_NO_DICT", [RETURN_VALUE] = "RETURN_VALUE", - [COMPARE_OP_STR] = "COMPARE_OP_STR", + [COMPARE_OP_FLOAT] = "COMPARE_OP_FLOAT", [SETUP_ANNOTATIONS] = "SETUP_ANNOTATIONS", - [FOR_ITER_LIST] = "FOR_ITER_LIST", + [COMPARE_OP_INT] = "COMPARE_OP_INT", [LOAD_LOCALS] = "LOAD_LOCALS", - [FOR_ITER_TUPLE] = "FOR_ITER_TUPLE", + [COMPARE_OP_STR] = "COMPARE_OP_STR", [POP_EXCEPT] = "POP_EXCEPT", [STORE_NAME] = "STORE_NAME", [DELETE_NAME] = "DELETE_NAME", @@ -358,9 +360,9 @@ const char *const _PyOpcode_OpName[268] = { [IMPORT_NAME] = "IMPORT_NAME", [IMPORT_FROM] = "IMPORT_FROM", [JUMP_FORWARD] = "JUMP_FORWARD", + [FOR_ITER_LIST] = "FOR_ITER_LIST", + [FOR_ITER_TUPLE] = "FOR_ITER_TUPLE", [FOR_ITER_RANGE] = "FOR_ITER_RANGE", - [FOR_ITER_GEN] = "FOR_ITER_GEN", - [CALL_BOUND_METHOD_EXACT_ARGS] = "CALL_BOUND_METHOD_EXACT_ARGS", [POP_JUMP_IF_FALSE] = "POP_JUMP_IF_FALSE", [POP_JUMP_IF_TRUE] = "POP_JUMP_IF_TRUE", [LOAD_GLOBAL] = "LOAD_GLOBAL", @@ -379,11 +381,11 @@ const char *const _PyOpcode_OpName[268] = { [POP_JUMP_IF_NONE] = "POP_JUMP_IF_NONE", [RAISE_VARARGS] = "RAISE_VARARGS", [GET_AWAITABLE] = "GET_AWAITABLE", - [CALL_PY_EXACT_ARGS] = "CALL_PY_EXACT_ARGS", + [FOR_ITER_GEN] = "FOR_ITER_GEN", [BUILD_SLICE] = "BUILD_SLICE", [JUMP_BACKWARD_NO_INTERRUPT] = "JUMP_BACKWARD_NO_INTERRUPT", [MAKE_CELL] = "MAKE_CELL", - [CALL_PY_WITH_DEFAULTS] = "CALL_PY_WITH_DEFAULTS", + [CALL_BOUND_METHOD_EXACT_ARGS] = "CALL_BOUND_METHOD_EXACT_ARGS", [LOAD_DEREF] = "LOAD_DEREF", [STORE_DEREF] = "STORE_DEREF", [DELETE_DEREF] = "DELETE_DEREF", @@ -395,26 +397,26 @@ const char *const _PyOpcode_OpName[268] = { [LIST_APPEND] = "LIST_APPEND", [SET_ADD] = "SET_ADD", [MAP_ADD] = "MAP_ADD", - [CALL_NO_KW_TYPE_1] = "CALL_NO_KW_TYPE_1", + [CALL_PY_EXACT_ARGS] = "CALL_PY_EXACT_ARGS", [COPY_FREE_VARS] = "COPY_FREE_VARS", [YIELD_VALUE] = "YIELD_VALUE", [RESUME] = "RESUME", [MATCH_CLASS] = "MATCH_CLASS", + [CALL_PY_WITH_DEFAULTS] = "CALL_PY_WITH_DEFAULTS", + [CALL_NO_KW_TYPE_1] = "CALL_NO_KW_TYPE_1", [CALL_NO_KW_STR_1] = "CALL_NO_KW_STR_1", - [CALL_NO_KW_TUPLE_1] = "CALL_NO_KW_TUPLE_1", - [CALL_BUILTIN_CLASS] = "CALL_BUILTIN_CLASS", [BUILD_CONST_KEY_MAP] = "BUILD_CONST_KEY_MAP", [BUILD_STRING] = "BUILD_STRING", [CONVERT_VALUE] = "CONVERT_VALUE", + [CALL_NO_KW_TUPLE_1] = "CALL_NO_KW_TUPLE_1", + [CALL_BUILTIN_CLASS] = "CALL_BUILTIN_CLASS", [CALL_NO_KW_BUILTIN_O] = "CALL_NO_KW_BUILTIN_O", - [CALL_NO_KW_BUILTIN_FAST] = "CALL_NO_KW_BUILTIN_FAST", - [CALL_BUILTIN_FAST_WITH_KEYWORDS] = "CALL_BUILTIN_FAST_WITH_KEYWORDS", [LIST_EXTEND] = "LIST_EXTEND", [SET_UPDATE] = "SET_UPDATE", [DICT_MERGE] = "DICT_MERGE", [DICT_UPDATE] = "DICT_UPDATE", - [CALL_NO_KW_LEN] = "CALL_NO_KW_LEN", - [CALL_NO_KW_ISINSTANCE] = "CALL_NO_KW_ISINSTANCE", + [CALL_NO_KW_BUILTIN_FAST] = "CALL_NO_KW_BUILTIN_FAST", + [CALL_BUILTIN_FAST_WITH_KEYWORDS] = "CALL_BUILTIN_FAST_WITH_KEYWORDS", [LOAD_FAST_LOAD_FAST] = "LOAD_FAST_LOAD_FAST", [STORE_FAST_LOAD_FAST] = "STORE_FAST_LOAD_FAST", [STORE_FAST_STORE_FAST] = "STORE_FAST_STORE_FAST", @@ -425,14 +427,14 @@ const char *const _PyOpcode_OpName[268] = { [LOAD_FROM_DICT_OR_GLOBALS] = "LOAD_FROM_DICT_OR_GLOBALS", [LOAD_FROM_DICT_OR_DEREF] = "LOAD_FROM_DICT_OR_DEREF", [SET_FUNCTION_ATTRIBUTE] = "SET_FUNCTION_ATTRIBUTE", + [CALL_NO_KW_LEN] = "CALL_NO_KW_LEN", + [CALL_NO_KW_ISINSTANCE] = "CALL_NO_KW_ISINSTANCE", [CALL_NO_KW_LIST_APPEND] = "CALL_NO_KW_LIST_APPEND", [CALL_NO_KW_METHOD_DESCRIPTOR_O] = "CALL_NO_KW_METHOD_DESCRIPTOR_O", [CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS] = "CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS", [CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS] = "CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS", [CALL_NO_KW_METHOD_DESCRIPTOR_FAST] = "CALL_NO_KW_METHOD_DESCRIPTOR_FAST", [CALL_NO_KW_ALLOC_AND_ENTER_INIT] = "CALL_NO_KW_ALLOC_AND_ENTER_INIT", - [184] = "<184>", - [185] = "<185>", [186] = "<186>", [187] = "<187>", [188] = "<188>", @@ -519,8 +521,6 @@ const char *const _PyOpcode_OpName[268] = { #endif // NEED_OPCODE_TABLES #define EXTRA_CASES \ - case 184: \ - case 185: \ case 186: \ case 187: \ case 188: \ diff --git a/Include/opcode.h b/Include/opcode.h index 6b855bdd99dfa..697520937d905 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -201,31 +201,33 @@ extern "C" { #define LOAD_ATTR_METHOD_WITH_VALUES 78 #define LOAD_ATTR_METHOD_NO_DICT 79 #define LOAD_ATTR_METHOD_LAZY_DICT 80 -#define COMPARE_OP_FLOAT 81 -#define COMPARE_OP_INT 82 -#define COMPARE_OP_STR 84 -#define FOR_ITER_LIST 86 -#define FOR_ITER_TUPLE 88 -#define FOR_ITER_RANGE 111 -#define FOR_ITER_GEN 112 -#define CALL_BOUND_METHOD_EXACT_ARGS 113 -#define CALL_PY_EXACT_ARGS 132 -#define CALL_PY_WITH_DEFAULTS 136 -#define CALL_NO_KW_TYPE_1 148 -#define CALL_NO_KW_STR_1 153 -#define CALL_NO_KW_TUPLE_1 154 -#define CALL_BUILTIN_CLASS 155 -#define CALL_NO_KW_BUILTIN_O 159 -#define CALL_NO_KW_BUILTIN_FAST 160 -#define CALL_BUILTIN_FAST_WITH_KEYWORDS 161 -#define CALL_NO_KW_LEN 166 -#define CALL_NO_KW_ISINSTANCE 167 -#define CALL_NO_KW_LIST_APPEND 178 -#define CALL_NO_KW_METHOD_DESCRIPTOR_O 179 -#define CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS 180 -#define CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 181 -#define CALL_NO_KW_METHOD_DESCRIPTOR_FAST 182 -#define CALL_NO_KW_ALLOC_AND_ENTER_INIT 183 +#define LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES 81 +#define LOAD_ATTR_NONDESCRIPTOR_NO_DICT 82 +#define COMPARE_OP_FLOAT 84 +#define COMPARE_OP_INT 86 +#define COMPARE_OP_STR 88 +#define FOR_ITER_LIST 111 +#define FOR_ITER_TUPLE 112 +#define FOR_ITER_RANGE 113 +#define FOR_ITER_GEN 132 +#define CALL_BOUND_METHOD_EXACT_ARGS 136 +#define CALL_PY_EXACT_ARGS 148 +#define CALL_PY_WITH_DEFAULTS 153 +#define CALL_NO_KW_TYPE_1 154 +#define CALL_NO_KW_STR_1 155 +#define CALL_NO_KW_TUPLE_1 159 +#define CALL_BUILTIN_CLASS 160 +#define CALL_NO_KW_BUILTIN_O 161 +#define CALL_NO_KW_BUILTIN_FAST 166 +#define CALL_BUILTIN_FAST_WITH_KEYWORDS 167 +#define CALL_NO_KW_LEN 178 +#define CALL_NO_KW_ISINSTANCE 179 +#define CALL_NO_KW_LIST_APPEND 180 +#define CALL_NO_KW_METHOD_DESCRIPTOR_O 181 +#define CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS 182 +#define CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 183 +#define CALL_NO_KW_METHOD_DESCRIPTOR_FAST 184 +#define CALL_NO_KW_ALLOC_AND_ENTER_INIT 185 #define NB_ADD 0 #define NB_AND 1 diff --git a/Lib/_opcode_metadata.py b/Lib/_opcode_metadata.py index b0fcd1a6b920f..fd8ecdb5c980f 100644 --- a/Lib/_opcode_metadata.py +++ b/Lib/_opcode_metadata.py @@ -63,6 +63,8 @@ "LOAD_ATTR_METHOD_WITH_VALUES", "LOAD_ATTR_METHOD_NO_DICT", "LOAD_ATTR_METHOD_LAZY_DICT", + "LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES", + "LOAD_ATTR_NONDESCRIPTOR_NO_DICT", ], "COMPARE_OP": [ "COMPARE_OP_FLOAT", diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-04-04-50-14.gh-issue-100288.yNQ1ez.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-04-04-50-14.gh-issue-100288.yNQ1ez.rst new file mode 100644 index 0000000000000..faf43bd0c2f48 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-04-04-50-14.gh-issue-100288.yNQ1ez.rst @@ -0,0 +1,3 @@ +Specialize :opcode:`LOAD_ATTR` for non-descriptors on the class. Adds +:opcode:`LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES` and :opcode +`LOAD_ATTR_NONDESCRIPTOR_NO_DICT`. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 89b077ac42834..0848bbfd203ec 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1809,6 +1809,8 @@ dummy_func( LOAD_ATTR_METHOD_WITH_VALUES, LOAD_ATTR_METHOD_NO_DICT, LOAD_ATTR_METHOD_LAZY_DICT, + LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES, + LOAD_ATTR_NONDESCRIPTOR_NO_DICT, }; inst(LOAD_ATTR, (unused/9, owner -- res2 if (oparg & 1), res)) { @@ -2657,7 +2659,8 @@ dummy_func( exc_info->exc_value = Py_NewRef(new_exc); } - inst(LOAD_ATTR_METHOD_WITH_VALUES, (unused/1, type_version/2, keys_version/2, descr/4, self -- res2 if (oparg & 1), res)) { + inst(LOAD_ATTR_METHOD_WITH_VALUES, (unused/1, type_version/2, keys_version/2, descr/4, self -- res2 if (1), res)) { + assert(oparg & 1); /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -2673,10 +2676,10 @@ dummy_func( res2 = Py_NewRef(descr); assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; - assert(oparg & 1); } - inst(LOAD_ATTR_METHOD_NO_DICT, (unused/1, type_version/2, unused/2, descr/4, self -- res2 if (oparg & 1), res)) { + inst(LOAD_ATTR_METHOD_NO_DICT, (unused/1, type_version/2, unused/2, descr/4, self -- res2 if (1), res)) { + assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); assert(self_cls->tp_dictoffset == 0); @@ -2685,10 +2688,39 @@ dummy_func( assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - assert(oparg & 1); } - inst(LOAD_ATTR_METHOD_LAZY_DICT, (unused/1, type_version/2, unused/2, descr/4, self -- res2 if (oparg & 1), res)) { + inst(LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES, (unused/1, type_version/2, keys_version/2, descr/4, self -- res2 if (0), res)) { + assert((oparg & 1) == 0); + PyTypeObject *self_cls = Py_TYPE(self); + assert(type_version != 0); + DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); + assert(self_cls->tp_flags & Py_TPFLAGS_MANAGED_DICT); + PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(self); + DEOPT_IF(!_PyDictOrValues_IsValues(dorv), LOAD_ATTR); + PyHeapTypeObject *self_heap_type = (PyHeapTypeObject *)self_cls; + DEOPT_IF(self_heap_type->ht_cached_keys->dk_version != + keys_version, LOAD_ATTR); + STAT_INC(LOAD_ATTR, hit); + assert(descr != NULL); + DECREF_INPUTS(); + res = Py_NewRef(descr); + } + + inst(LOAD_ATTR_NONDESCRIPTOR_NO_DICT, (unused/1, type_version/2, unused/2, descr/4, self -- res2 if (0), res)) { + assert((oparg & 1) == 0); + PyTypeObject *self_cls = Py_TYPE(self); + assert(type_version != 0); + DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); + assert(self_cls->tp_dictoffset == 0); + STAT_INC(LOAD_ATTR, hit); + assert(descr != NULL); + DECREF_INPUTS(); + res = Py_NewRef(descr); + } + + inst(LOAD_ATTR_METHOD_LAZY_DICT, (unused/1, type_version/2, unused/2, descr/4, self -- res2 if (1), res)) { + assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); Py_ssize_t dictoffset = self_cls->tp_dictoffset; @@ -2701,7 +2733,6 @@ dummy_func( assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - assert(oparg & 1); } inst(KW_NAMES, (--)) { diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 32efeb099d9b3..0ef752249ccb7 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1738,7 +1738,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2114 "Python/bytecodes.c" + #line 2116 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -1760,7 +1760,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2129 "Python/bytecodes.c" + #line 2131 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -1786,7 +1786,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2148 "Python/bytecodes.c" + #line 2150 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -1809,12 +1809,12 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2163 "Python/bytecodes.c" + #line 2165 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; #line 1815 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2165 "Python/bytecodes.c" + #line 2167 "Python/bytecodes.c" b = res ? Py_True : Py_False; #line 1820 "Python/executor_cases.c.h" STACK_SHRINK(1); @@ -1826,12 +1826,12 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2169 "Python/bytecodes.c" + #line 2171 "Python/bytecodes.c" int res = PySequence_Contains(right, left); #line 1832 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2171 "Python/bytecodes.c" + #line 2173 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; #line 1838 "Python/executor_cases.c.h" @@ -1845,12 +1845,12 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 2176 "Python/bytecodes.c" + #line 2178 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { #line 1851 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2178 "Python/bytecodes.c" + #line 2180 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -1861,7 +1861,7 @@ #line 1862 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2186 "Python/bytecodes.c" + #line 2188 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -1880,19 +1880,19 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2197 "Python/bytecodes.c" + #line 2199 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { #line 1887 "Python/executor_cases.c.h" Py_DECREF(right); - #line 2200 "Python/bytecodes.c" + #line 2202 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); #line 1894 "Python/executor_cases.c.h" Py_DECREF(right); - #line 2205 "Python/bytecodes.c" + #line 2207 "Python/bytecodes.c" b = res ? Py_True : Py_False; #line 1898 "Python/executor_cases.c.h" stack_pointer[-1] = b; @@ -1902,7 +1902,7 @@ case GET_LEN: { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2309 "Python/bytecodes.c" + #line 2311 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; @@ -1919,7 +1919,7 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2317 "Python/bytecodes.c" + #line 2319 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); @@ -1928,7 +1928,7 @@ Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2322 "Python/bytecodes.c" + #line 2324 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -1945,7 +1945,7 @@ case MATCH_MAPPING: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2332 "Python/bytecodes.c" + #line 2334 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; #line 1952 "Python/executor_cases.c.h" @@ -1957,7 +1957,7 @@ case MATCH_SEQUENCE: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2337 "Python/bytecodes.c" + #line 2339 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; #line 1964 "Python/executor_cases.c.h" @@ -1970,7 +1970,7 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2342 "Python/bytecodes.c" + #line 2344 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; @@ -1983,12 +1983,12 @@ case GET_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2348 "Python/bytecodes.c" + #line 2350 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); #line 1990 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2351 "Python/bytecodes.c" + #line 2353 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; #line 1994 "Python/executor_cases.c.h" stack_pointer[-1] = iter; @@ -1998,7 +1998,7 @@ case GET_YIELD_FROM_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2355 "Python/bytecodes.c" + #line 2357 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -2023,7 +2023,7 @@ } #line 2025 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2378 "Python/bytecodes.c" + #line 2380 "Python/bytecodes.c" } #line 2029 "Python/executor_cases.c.h" stack_pointer[-1] = iter; @@ -2035,7 +2035,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2610 "Python/bytecodes.c" + #line 2612 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -2065,7 +2065,7 @@ case PUSH_EXC_INFO: { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2649 "Python/bytecodes.c" + #line 2651 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -2084,7 +2084,7 @@ case EXIT_INIT_CHECK: { PyObject *should_be_none = stack_pointer[-1]; - #line 3019 "Python/bytecodes.c" + #line 3050 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -2100,7 +2100,7 @@ case MAKE_FUNCTION: { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3433 "Python/bytecodes.c" + #line 3464 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -2120,7 +2120,7 @@ case SET_FUNCTION_ATTRIBUTE: { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3447 "Python/bytecodes.c" + #line 3478 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -2156,13 +2156,13 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3497 "Python/bytecodes.c" + #line 3528 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); #line 2162 "Python/executor_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3499 "Python/bytecodes.c" + #line 3530 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } #line 2168 "Python/executor_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); @@ -2174,7 +2174,7 @@ case CONVERT_VALUE: { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3503 "Python/bytecodes.c" + #line 3534 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; @@ -2189,7 +2189,7 @@ case FORMAT_SIMPLE: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3512 "Python/bytecodes.c" + #line 3543 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -2209,7 +2209,7 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3525 "Python/bytecodes.c" + #line 3556 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); @@ -2223,7 +2223,7 @@ case COPY: { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3532 "Python/bytecodes.c" + #line 3563 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); #line 2230 "Python/executor_cases.c.h" @@ -2266,7 +2266,7 @@ case SWAP: { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3557 "Python/bytecodes.c" + #line 3588 "Python/bytecodes.c" assert(oparg >= 2); #line 2272 "Python/executor_cases.c.h" stack_pointer[-1] = bottom; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index eb8b50e5c905d..11823cf9cd293 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -2563,7 +2563,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1815 "Python/bytecodes.c" + #line 1817 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2599,7 +2599,7 @@ */ #line 2601 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1849 "Python/bytecodes.c" + #line 1851 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -2610,7 +2610,7 @@ res = PyObject_GetAttr(owner, name); #line 2612 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1858 "Python/bytecodes.c" + #line 1860 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } #line 2617 "Python/generated_cases.c.h" @@ -2627,7 +2627,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1867 "Python/bytecodes.c" + #line 1869 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2655,7 +2655,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1883 "Python/bytecodes.c" + #line 1885 "Python/bytecodes.c" DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; assert(dict != NULL); @@ -2683,7 +2683,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1899 "Python/bytecodes.c" + #line 1901 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2725,7 +2725,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1929 "Python/bytecodes.c" + #line 1931 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2750,7 +2750,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 1942 "Python/bytecodes.c" + #line 1944 "Python/bytecodes.c" DEOPT_IF(!PyType_Check(cls), LOAD_ATTR); DEOPT_IF(((PyTypeObject *)cls)->tp_version_tag != type_version, @@ -2776,7 +2776,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *fget = read_obj(&next_instr[5].cache); - #line 1957 "Python/bytecodes.c" + #line 1959 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); @@ -2808,7 +2808,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *getattribute = read_obj(&next_instr[5].cache); - #line 1983 "Python/bytecodes.c" + #line 1985 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); DEOPT_IF(cls->tp_version_tag != type_version, LOAD_ATTR); @@ -2842,7 +2842,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 2011 "Python/bytecodes.c" + #line 2013 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2871,7 +2871,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t hint = read_u16(&next_instr[3].cache); - #line 2031 "Python/bytecodes.c" + #line 2033 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2921,7 +2921,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 2072 "Python/bytecodes.c" + #line 2074 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2943,7 +2943,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2091 "Python/bytecodes.c" + #line 2093 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2959,7 +2959,7 @@ #line 2960 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2104 "Python/bytecodes.c" + #line 2106 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; if (oparg & 16) { int res_bool = PyObject_IsTrue(res); @@ -2978,7 +2978,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2114 "Python/bytecodes.c" + #line 2116 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -3001,7 +3001,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2129 "Python/bytecodes.c" + #line 2131 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -3028,7 +3028,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2148 "Python/bytecodes.c" + #line 2150 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -3052,12 +3052,12 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2163 "Python/bytecodes.c" + #line 2165 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; #line 3058 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2165 "Python/bytecodes.c" + #line 2167 "Python/bytecodes.c" b = res ? Py_True : Py_False; #line 3063 "Python/generated_cases.c.h" STACK_SHRINK(1); @@ -3069,12 +3069,12 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2169 "Python/bytecodes.c" + #line 2171 "Python/bytecodes.c" int res = PySequence_Contains(right, left); #line 3075 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2171 "Python/bytecodes.c" + #line 2173 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; #line 3081 "Python/generated_cases.c.h" @@ -3088,12 +3088,12 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 2176 "Python/bytecodes.c" + #line 2178 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { #line 3094 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2178 "Python/bytecodes.c" + #line 2180 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -3104,7 +3104,7 @@ #line 3105 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2186 "Python/bytecodes.c" + #line 2188 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -3123,19 +3123,19 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2197 "Python/bytecodes.c" + #line 2199 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { #line 3130 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2200 "Python/bytecodes.c" + #line 2202 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); #line 3137 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2205 "Python/bytecodes.c" + #line 2207 "Python/bytecodes.c" b = res ? Py_True : Py_False; #line 3141 "Python/generated_cases.c.h" stack_pointer[-1] = b; @@ -3146,13 +3146,13 @@ PyObject *fromlist = stack_pointer[-1]; PyObject *level = stack_pointer[-2]; PyObject *res; - #line 2209 "Python/bytecodes.c" + #line 2211 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_name(tstate, frame, name, fromlist, level); #line 3153 "Python/generated_cases.c.h" Py_DECREF(level); Py_DECREF(fromlist); - #line 2212 "Python/bytecodes.c" + #line 2214 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; #line 3158 "Python/generated_cases.c.h" STACK_SHRINK(1); @@ -3163,7 +3163,7 @@ TARGET(IMPORT_FROM) { PyObject *from = stack_pointer[-1]; PyObject *res; - #line 2216 "Python/bytecodes.c" + #line 2218 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_from(tstate, from, name); if (res == NULL) goto error; @@ -3174,14 +3174,14 @@ } TARGET(JUMP_FORWARD) { - #line 2222 "Python/bytecodes.c" + #line 2224 "Python/bytecodes.c" JUMPBY(oparg); #line 3180 "Python/generated_cases.c.h" DISPATCH(); } TARGET(JUMP_BACKWARD) { - #line 2226 "Python/bytecodes.c" + #line 2228 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); _Py_CODEUNIT *here = next_instr - 1; assert(oparg <= INSTR_OFFSET()); @@ -3205,7 +3205,7 @@ } TARGET(ENTER_EXECUTOR) { - #line 2257 "Python/bytecodes.c" + #line 2259 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); PyCodeObject *code = _PyFrame_GetCode(frame); @@ -3225,7 +3225,7 @@ TARGET(POP_JUMP_IF_FALSE) { PyObject *cond = stack_pointer[-1]; - #line 2274 "Python/bytecodes.c" + #line 2276 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsFalse(cond)); #line 3232 "Python/generated_cases.c.h" @@ -3235,7 +3235,7 @@ TARGET(POP_JUMP_IF_TRUE) { PyObject *cond = stack_pointer[-1]; - #line 2279 "Python/bytecodes.c" + #line 2281 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsTrue(cond)); #line 3242 "Python/generated_cases.c.h" @@ -3245,11 +3245,11 @@ TARGET(POP_JUMP_IF_NOT_NONE) { PyObject *value = stack_pointer[-1]; - #line 2284 "Python/bytecodes.c" + #line 2286 "Python/bytecodes.c" if (!Py_IsNone(value)) { #line 3251 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2286 "Python/bytecodes.c" + #line 2288 "Python/bytecodes.c" JUMPBY(oparg); } #line 3256 "Python/generated_cases.c.h" @@ -3259,14 +3259,14 @@ TARGET(POP_JUMP_IF_NONE) { PyObject *value = stack_pointer[-1]; - #line 2291 "Python/bytecodes.c" + #line 2293 "Python/bytecodes.c" if (Py_IsNone(value)) { JUMPBY(oparg); } else { #line 3268 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2296 "Python/bytecodes.c" + #line 2298 "Python/bytecodes.c" } #line 3272 "Python/generated_cases.c.h" STACK_SHRINK(1); @@ -3274,7 +3274,7 @@ } TARGET(JUMP_BACKWARD_NO_INTERRUPT) { - #line 2300 "Python/bytecodes.c" + #line 2302 "Python/bytecodes.c" /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost * generator or coroutine, so we deliberately do not check it here. @@ -3288,7 +3288,7 @@ TARGET(GET_LEN) { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2309 "Python/bytecodes.c" + #line 2311 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; @@ -3305,7 +3305,7 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2317 "Python/bytecodes.c" + #line 2319 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); @@ -3314,7 +3314,7 @@ Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2322 "Python/bytecodes.c" + #line 2324 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -3331,7 +3331,7 @@ TARGET(MATCH_MAPPING) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2332 "Python/bytecodes.c" + #line 2334 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; #line 3338 "Python/generated_cases.c.h" @@ -3343,7 +3343,7 @@ TARGET(MATCH_SEQUENCE) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2337 "Python/bytecodes.c" + #line 2339 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; #line 3350 "Python/generated_cases.c.h" @@ -3356,7 +3356,7 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2342 "Python/bytecodes.c" + #line 2344 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; @@ -3369,12 +3369,12 @@ TARGET(GET_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2348 "Python/bytecodes.c" + #line 2350 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); #line 3376 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2351 "Python/bytecodes.c" + #line 2353 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; #line 3380 "Python/generated_cases.c.h" stack_pointer[-1] = iter; @@ -3384,7 +3384,7 @@ TARGET(GET_YIELD_FROM_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2355 "Python/bytecodes.c" + #line 2357 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -3409,7 +3409,7 @@ } #line 3411 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2378 "Python/bytecodes.c" + #line 2380 "Python/bytecodes.c" } #line 3415 "Python/generated_cases.c.h" stack_pointer[-1] = iter; @@ -3421,7 +3421,7 @@ static_assert(INLINE_CACHE_ENTRIES_FOR_ITER == 1, "incorrect cache size"); PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2396 "Python/bytecodes.c" + #line 2398 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -3461,7 +3461,7 @@ } TARGET(INSTRUMENTED_FOR_ITER) { - #line 2430 "Python/bytecodes.c" + #line 2432 "Python/bytecodes.c" _Py_CODEUNIT *here = next_instr-1; _Py_CODEUNIT *target; PyObject *iter = TOP(); @@ -3494,7 +3494,7 @@ TARGET(FOR_ITER_LIST) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2458 "Python/bytecodes.c" + #line 2460 "Python/bytecodes.c" DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit); @@ -3525,7 +3525,7 @@ TARGET(FOR_ITER_TUPLE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2481 "Python/bytecodes.c" + #line 2483 "Python/bytecodes.c" _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3556,7 +3556,7 @@ TARGET(FOR_ITER_RANGE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2504 "Python/bytecodes.c" + #line 2506 "Python/bytecodes.c" _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3584,7 +3584,7 @@ TARGET(FOR_ITER_GEN) { PyObject *iter = stack_pointer[-1]; - #line 2525 "Python/bytecodes.c" + #line 2527 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER); @@ -3607,7 +3607,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2543 "Python/bytecodes.c" + #line 2545 "Python/bytecodes.c" PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { @@ -3632,7 +3632,7 @@ } #line 3634 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2566 "Python/bytecodes.c" + #line 2568 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { @@ -3650,7 +3650,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2575 "Python/bytecodes.c" + #line 2577 "Python/bytecodes.c" /* pop the context manager, push its __exit__ and the * value returned from calling its __enter__ */ @@ -3678,7 +3678,7 @@ } #line 3680 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2601 "Python/bytecodes.c" + #line 2603 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { @@ -3697,7 +3697,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2610 "Python/bytecodes.c" + #line 2612 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -3727,7 +3727,7 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2649 "Python/bytecodes.c" + #line 2651 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -3751,7 +3751,8 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2661 "Python/bytecodes.c" + #line 2663 "Python/bytecodes.c" + assert(oparg & 1); /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3767,11 +3768,10 @@ res2 = Py_NewRef(descr); assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; - assert(oparg & 1); #line 3772 "Python/generated_cases.c.h" - STACK_GROW(((oparg & 1) ? 1 : 0)); + STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; - if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } + if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } next_instr += 9; DISPATCH(); } @@ -3782,7 +3782,8 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2680 "Python/bytecodes.c" + #line 2682 "Python/bytecodes.c" + assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); assert(self_cls->tp_dictoffset == 0); @@ -3791,11 +3792,68 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - assert(oparg & 1); #line 3796 "Python/generated_cases.c.h" - STACK_GROW(((oparg & 1) ? 1 : 0)); + STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; - if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } + if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } + next_instr += 9; + DISPATCH(); + } + + TARGET(LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES) { + PyObject *self = stack_pointer[-1]; + PyObject *res2 = NULL; + PyObject *res; + uint32_t type_version = read_u32(&next_instr[1].cache); + uint32_t keys_version = read_u32(&next_instr[3].cache); + PyObject *descr = read_obj(&next_instr[5].cache); + #line 2694 "Python/bytecodes.c" + assert((oparg & 1) == 0); + PyTypeObject *self_cls = Py_TYPE(self); + assert(type_version != 0); + DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); + assert(self_cls->tp_flags & Py_TPFLAGS_MANAGED_DICT); + PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(self); + DEOPT_IF(!_PyDictOrValues_IsValues(dorv), LOAD_ATTR); + PyHeapTypeObject *self_heap_type = (PyHeapTypeObject *)self_cls; + DEOPT_IF(self_heap_type->ht_cached_keys->dk_version != + keys_version, LOAD_ATTR); + STAT_INC(LOAD_ATTR, hit); + assert(descr != NULL); + #line 3824 "Python/generated_cases.c.h" + Py_DECREF(self); + #line 2707 "Python/bytecodes.c" + res = Py_NewRef(descr); + #line 3828 "Python/generated_cases.c.h" + STACK_GROW((0 ? 1 : 0)); + stack_pointer[-1] = res; + if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } + next_instr += 9; + DISPATCH(); + } + + TARGET(LOAD_ATTR_NONDESCRIPTOR_NO_DICT) { + PyObject *self = stack_pointer[-1]; + PyObject *res2 = NULL; + PyObject *res; + uint32_t type_version = read_u32(&next_instr[1].cache); + PyObject *descr = read_obj(&next_instr[5].cache); + #line 2711 "Python/bytecodes.c" + assert((oparg & 1) == 0); + PyTypeObject *self_cls = Py_TYPE(self); + assert(type_version != 0); + DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); + assert(self_cls->tp_dictoffset == 0); + STAT_INC(LOAD_ATTR, hit); + assert(descr != NULL); + #line 3850 "Python/generated_cases.c.h" + Py_DECREF(self); + #line 2719 "Python/bytecodes.c" + res = Py_NewRef(descr); + #line 3854 "Python/generated_cases.c.h" + STACK_GROW((0 ? 1 : 0)); + stack_pointer[-1] = res; + if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } next_instr += 9; DISPATCH(); } @@ -3806,7 +3864,8 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2692 "Python/bytecodes.c" + #line 2723 "Python/bytecodes.c" + assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); Py_ssize_t dictoffset = self_cls->tp_dictoffset; @@ -3819,26 +3878,25 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - assert(oparg & 1); - #line 3824 "Python/generated_cases.c.h" - STACK_GROW(((oparg & 1) ? 1 : 0)); + #line 3882 "Python/generated_cases.c.h" + STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; - if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } + if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } next_instr += 9; DISPATCH(); } TARGET(KW_NAMES) { - #line 2708 "Python/bytecodes.c" + #line 2739 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(FRAME_CO_CONSTS)); kwnames = GETITEM(FRAME_CO_CONSTS, oparg); - #line 3837 "Python/generated_cases.c.h" + #line 3895 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_CALL) { - #line 2714 "Python/bytecodes.c" + #line 2745 "Python/bytecodes.c" int is_meth = PEEK(oparg+2) != NULL; int total_args = oparg + is_meth; PyObject *function = PEEK(total_args + 1); @@ -3851,7 +3909,7 @@ _PyCallCache *cache = (_PyCallCache *)next_instr; INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(CALL); - #line 3855 "Python/generated_cases.c.h" + #line 3913 "Python/generated_cases.c.h" } TARGET(CALL) { @@ -3861,7 +3919,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2760 "Python/bytecodes.c" + #line 2791 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3943,7 +4001,7 @@ Py_DECREF(args[i]); } if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3947 "Python/generated_cases.c.h" + #line 4005 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3955,7 +4013,7 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2848 "Python/bytecodes.c" + #line 2879 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit); @@ -3965,7 +4023,7 @@ PEEK(oparg + 2) = Py_NewRef(meth); // method Py_DECREF(callable); GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS); - #line 3969 "Python/generated_cases.c.h" + #line 4027 "Python/generated_cases.c.h" } TARGET(CALL_PY_EXACT_ARGS) { @@ -3974,7 +4032,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2860 "Python/bytecodes.c" + #line 2891 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4000,7 +4058,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4004 "Python/generated_cases.c.h" + #line 4062 "Python/generated_cases.c.h" } TARGET(CALL_PY_WITH_DEFAULTS) { @@ -4008,7 +4066,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2888 "Python/bytecodes.c" + #line 2919 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4044,7 +4102,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4048 "Python/generated_cases.c.h" + #line 4106 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_TYPE_1) { @@ -4052,7 +4110,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2926 "Python/bytecodes.c" + #line 2957 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4062,7 +4120,7 @@ res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(obj); Py_DECREF(&PyType_Type); // I.e., callable - #line 4066 "Python/generated_cases.c.h" + #line 4124 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4075,7 +4133,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2938 "Python/bytecodes.c" + #line 2969 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4086,7 +4144,7 @@ Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); // I.e., callable if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4090 "Python/generated_cases.c.h" + #line 4148 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4100,7 +4158,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2952 "Python/bytecodes.c" + #line 2983 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4111,7 +4169,7 @@ Py_DECREF(arg); Py_DECREF(&PyTuple_Type); // I.e., tuple if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4115 "Python/generated_cases.c.h" + #line 4173 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4124,7 +4182,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; - #line 2966 "Python/bytecodes.c" + #line 2997 "Python/bytecodes.c" /* This instruction does the following: * 1. Creates the object (by calling ``object.__new__``) * 2. Pushes a shim frame to the frame stack (to cleanup after ``__init__``) @@ -4175,12 +4233,12 @@ * as it will be checked after start_frame */ tstate->py_recursion_remaining--; goto start_frame; - #line 4179 "Python/generated_cases.c.h" + #line 4237 "Python/generated_cases.c.h" } TARGET(EXIT_INIT_CHECK) { PyObject *should_be_none = stack_pointer[-1]; - #line 3019 "Python/bytecodes.c" + #line 3050 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -4188,7 +4246,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 4192 "Python/generated_cases.c.h" + #line 4250 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -4198,7 +4256,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3029 "Python/bytecodes.c" + #line 3060 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4220,7 +4278,7 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4224 "Python/generated_cases.c.h" + #line 4282 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4234,7 +4292,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3054 "Python/bytecodes.c" + #line 3085 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4262,7 +4320,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4266 "Python/generated_cases.c.h" + #line 4324 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4276,7 +4334,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3085 "Python/bytecodes.c" + #line 3116 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4308,7 +4366,7 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 4312 "Python/generated_cases.c.h" + #line 4370 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4322,7 +4380,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3120 "Python/bytecodes.c" + #line 3151 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -4354,7 +4412,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4358 "Python/generated_cases.c.h" + #line 4416 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4368,7 +4426,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3155 "Python/bytecodes.c" + #line 3186 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -4393,7 +4451,7 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4397 "Python/generated_cases.c.h" + #line 4455 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4406,7 +4464,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3182 "Python/bytecodes.c" + #line 3213 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -4433,7 +4491,7 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4437 "Python/generated_cases.c.h" + #line 4495 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4445,7 +4503,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 3212 "Python/bytecodes.c" + #line 3243 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -4463,14 +4521,14 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 4467 "Python/generated_cases.c.h" + #line 4525 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3232 "Python/bytecodes.c" + #line 3263 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4501,7 +4559,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4505 "Python/generated_cases.c.h" + #line 4563 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4514,7 +4572,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3266 "Python/bytecodes.c" + #line 3297 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4543,7 +4601,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4547 "Python/generated_cases.c.h" + #line 4605 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4556,7 +4614,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3298 "Python/bytecodes.c" + #line 3329 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -4585,7 +4643,7 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4589 "Python/generated_cases.c.h" + #line 4647 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4598,7 +4656,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3330 "Python/bytecodes.c" + #line 3361 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4626,7 +4684,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4630 "Python/generated_cases.c.h" + #line 4688 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4636,9 +4694,9 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { - #line 3361 "Python/bytecodes.c" + #line 3392 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); - #line 4642 "Python/generated_cases.c.h" + #line 4700 "Python/generated_cases.c.h" } TARGET(CALL_FUNCTION_EX) { @@ -4647,7 +4705,7 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 3365 "Python/bytecodes.c" + #line 3396 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs)); @@ -4709,14 +4767,14 @@ } result = PyObject_Call(func, callargs, kwargs); } - #line 4713 "Python/generated_cases.c.h" + #line 4771 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 3427 "Python/bytecodes.c" + #line 3458 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4720 "Python/generated_cases.c.h" + #line 4778 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4727,7 +4785,7 @@ TARGET(MAKE_FUNCTION) { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3433 "Python/bytecodes.c" + #line 3464 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4739,7 +4797,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4743 "Python/generated_cases.c.h" + #line 4801 "Python/generated_cases.c.h" stack_pointer[-1] = func; DISPATCH(); } @@ -4747,7 +4805,7 @@ TARGET(SET_FUNCTION_ATTRIBUTE) { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3447 "Python/bytecodes.c" + #line 3478 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -4772,14 +4830,14 @@ default: Py_UNREACHABLE(); } - #line 4776 "Python/generated_cases.c.h" + #line 4834 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 3474 "Python/bytecodes.c" + #line 3505 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4800,7 +4858,7 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4804 "Python/generated_cases.c.h" + #line 4862 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4808,15 +4866,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3497 "Python/bytecodes.c" + #line 3528 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4814 "Python/generated_cases.c.h" + #line 4872 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3499 "Python/bytecodes.c" + #line 3530 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4820 "Python/generated_cases.c.h" + #line 4878 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4826,14 +4884,14 @@ TARGET(CONVERT_VALUE) { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3503 "Python/bytecodes.c" + #line 3534 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 4837 "Python/generated_cases.c.h" + #line 4895 "Python/generated_cases.c.h" stack_pointer[-1] = result; DISPATCH(); } @@ -4841,7 +4899,7 @@ TARGET(FORMAT_SIMPLE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3512 "Python/bytecodes.c" + #line 3543 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -4852,7 +4910,7 @@ else { res = value; } - #line 4856 "Python/generated_cases.c.h" + #line 4914 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -4861,12 +4919,12 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3525 "Python/bytecodes.c" + #line 3556 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 4870 "Python/generated_cases.c.h" + #line 4928 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -4875,10 +4933,10 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3532 "Python/bytecodes.c" + #line 3563 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4882 "Python/generated_cases.c.h" + #line 4940 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4890,7 +4948,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3537 "Python/bytecodes.c" + #line 3568 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -4905,12 +4963,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4909 "Python/generated_cases.c.h" + #line 4967 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3552 "Python/bytecodes.c" + #line 3583 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 4914 "Python/generated_cases.c.h" + #line 4972 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -4920,16 +4978,16 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3557 "Python/bytecodes.c" + #line 3588 "Python/bytecodes.c" assert(oparg >= 2); - #line 4926 "Python/generated_cases.c.h" + #line 4984 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } TARGET(INSTRUMENTED_INSTRUCTION) { - #line 3561 "Python/bytecodes.c" + #line 3592 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error; @@ -4941,48 +4999,48 @@ assert(next_opcode > 0 && next_opcode < 256); opcode = next_opcode; DISPATCH_GOTO(); - #line 4945 "Python/generated_cases.c.h" + #line 5003 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_JUMP_FORWARD) { - #line 3575 "Python/bytecodes.c" + #line 3606 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); - #line 4951 "Python/generated_cases.c.h" + #line 5009 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) { - #line 3579 "Python/bytecodes.c" + #line 3610 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); - #line 4959 "Python/generated_cases.c.h" + #line 5017 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { - #line 3584 "Python/bytecodes.c" + #line 3615 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsTrue(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4970 "Python/generated_cases.c.h" + #line 5028 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { - #line 3592 "Python/bytecodes.c" + #line 3623 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsFalse(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4981 "Python/generated_cases.c.h" + #line 5039 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { - #line 3600 "Python/bytecodes.c" + #line 3631 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -4994,12 +5052,12 @@ offset = 0; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4998 "Python/generated_cases.c.h" + #line 5056 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { - #line 3614 "Python/bytecodes.c" + #line 3645 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -5011,30 +5069,30 @@ offset = oparg; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5015 "Python/generated_cases.c.h" + #line 5073 "Python/generated_cases.c.h" DISPATCH(); } TARGET(EXTENDED_ARG) { - #line 3628 "Python/bytecodes.c" + #line 3659 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 5026 "Python/generated_cases.c.h" + #line 5084 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3636 "Python/bytecodes.c" + #line 3667 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); - #line 5033 "Python/generated_cases.c.h" + #line 5091 "Python/generated_cases.c.h" } TARGET(RESERVED) { - #line 3641 "Python/bytecodes.c" + #line 3672 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 5040 "Python/generated_cases.c.h" + #line 5098 "Python/generated_cases.c.h" } diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index 70e1ca44ce766..92768e60f62ab 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -378,6 +378,10 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return 1; case LOAD_ATTR_METHOD_NO_DICT: return 1; + case LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES: + return 1; + case LOAD_ATTR_NONDESCRIPTOR_NO_DICT: + return 1; case LOAD_ATTR_METHOD_LAZY_DICT: return 1; case KW_NAMES: @@ -815,11 +819,15 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { case PUSH_EXC_INFO: return 2; case LOAD_ATTR_METHOD_WITH_VALUES: - return ((oparg & 1) ? 1 : 0) + 1; + return (1 ? 1 : 0) + 1; case LOAD_ATTR_METHOD_NO_DICT: - return ((oparg & 1) ? 1 : 0) + 1; + return (1 ? 1 : 0) + 1; + case LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES: + return (0 ? 1 : 0) + 1; + case LOAD_ATTR_NONDESCRIPTOR_NO_DICT: + return (0 ? 1 : 0) + 1; case LOAD_ATTR_METHOD_LAZY_DICT: - return ((oparg & 1) ? 1 : 0) + 1; + return (1 ? 1 : 0) + 1; case KW_NAMES: return 0; case INSTRUMENTED_CALL: @@ -1120,6 +1128,8 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { [PUSH_EXC_INFO] = { true, INSTR_FMT_IX, 0 }, [LOAD_ATTR_METHOD_WITH_VALUES] = { true, INSTR_FMT_IBC00000000, HAS_ARG_FLAG }, [LOAD_ATTR_METHOD_NO_DICT] = { true, INSTR_FMT_IBC00000000, HAS_ARG_FLAG }, + [LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES] = { true, INSTR_FMT_IBC00000000, HAS_ARG_FLAG }, + [LOAD_ATTR_NONDESCRIPTOR_NO_DICT] = { true, INSTR_FMT_IBC00000000, HAS_ARG_FLAG }, [LOAD_ATTR_METHOD_LAZY_DICT] = { true, INSTR_FMT_IBC00000000, HAS_ARG_FLAG }, [KW_NAMES] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_CONST_FLAG }, [INSTRUMENTED_CALL] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index 1a1ad97e00135..d84d253c912a2 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -80,14 +80,14 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_ATTR_METHOD_WITH_VALUES, &&TARGET_LOAD_ATTR_METHOD_NO_DICT, &&TARGET_LOAD_ATTR_METHOD_LAZY_DICT, - &&TARGET_COMPARE_OP_FLOAT, - &&TARGET_COMPARE_OP_INT, + &&TARGET_LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES, + &&TARGET_LOAD_ATTR_NONDESCRIPTOR_NO_DICT, &&TARGET_RETURN_VALUE, - &&TARGET_COMPARE_OP_STR, + &&TARGET_COMPARE_OP_FLOAT, &&TARGET_SETUP_ANNOTATIONS, - &&TARGET_FOR_ITER_LIST, + &&TARGET_COMPARE_OP_INT, &&TARGET_LOAD_LOCALS, - &&TARGET_FOR_ITER_TUPLE, + &&TARGET_COMPARE_OP_STR, &&TARGET_POP_EXCEPT, &&TARGET_STORE_NAME, &&TARGET_DELETE_NAME, @@ -110,9 +110,9 @@ static void *opcode_targets[256] = { &&TARGET_IMPORT_NAME, &&TARGET_IMPORT_FROM, &&TARGET_JUMP_FORWARD, + &&TARGET_FOR_ITER_LIST, + &&TARGET_FOR_ITER_TUPLE, &&TARGET_FOR_ITER_RANGE, - &&TARGET_FOR_ITER_GEN, - &&TARGET_CALL_BOUND_METHOD_EXACT_ARGS, &&TARGET_POP_JUMP_IF_FALSE, &&TARGET_POP_JUMP_IF_TRUE, &&TARGET_LOAD_GLOBAL, @@ -131,11 +131,11 @@ static void *opcode_targets[256] = { &&TARGET_POP_JUMP_IF_NONE, &&TARGET_RAISE_VARARGS, &&TARGET_GET_AWAITABLE, - &&TARGET_CALL_PY_EXACT_ARGS, + &&TARGET_FOR_ITER_GEN, &&TARGET_BUILD_SLICE, &&TARGET_JUMP_BACKWARD_NO_INTERRUPT, &&TARGET_MAKE_CELL, - &&TARGET_CALL_PY_WITH_DEFAULTS, + &&TARGET_CALL_BOUND_METHOD_EXACT_ARGS, &&TARGET_LOAD_DEREF, &&TARGET_STORE_DEREF, &&TARGET_DELETE_DEREF, @@ -147,26 +147,26 @@ static void *opcode_targets[256] = { &&TARGET_LIST_APPEND, &&TARGET_SET_ADD, &&TARGET_MAP_ADD, - &&TARGET_CALL_NO_KW_TYPE_1, + &&TARGET_CALL_PY_EXACT_ARGS, &&TARGET_COPY_FREE_VARS, &&TARGET_YIELD_VALUE, &&TARGET_RESUME, &&TARGET_MATCH_CLASS, + &&TARGET_CALL_PY_WITH_DEFAULTS, + &&TARGET_CALL_NO_KW_TYPE_1, &&TARGET_CALL_NO_KW_STR_1, - &&TARGET_CALL_NO_KW_TUPLE_1, - &&TARGET_CALL_BUILTIN_CLASS, &&TARGET_BUILD_CONST_KEY_MAP, &&TARGET_BUILD_STRING, &&TARGET_CONVERT_VALUE, + &&TARGET_CALL_NO_KW_TUPLE_1, + &&TARGET_CALL_BUILTIN_CLASS, &&TARGET_CALL_NO_KW_BUILTIN_O, - &&TARGET_CALL_NO_KW_BUILTIN_FAST, - &&TARGET_CALL_BUILTIN_FAST_WITH_KEYWORDS, &&TARGET_LIST_EXTEND, &&TARGET_SET_UPDATE, &&TARGET_DICT_MERGE, &&TARGET_DICT_UPDATE, - &&TARGET_CALL_NO_KW_LEN, - &&TARGET_CALL_NO_KW_ISINSTANCE, + &&TARGET_CALL_NO_KW_BUILTIN_FAST, + &&TARGET_CALL_BUILTIN_FAST_WITH_KEYWORDS, &&TARGET_LOAD_FAST_LOAD_FAST, &&TARGET_STORE_FAST_LOAD_FAST, &&TARGET_STORE_FAST_STORE_FAST, @@ -177,6 +177,8 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_FROM_DICT_OR_GLOBALS, &&TARGET_LOAD_FROM_DICT_OR_DEREF, &&TARGET_SET_FUNCTION_ATTRIBUTE, + &&TARGET_CALL_NO_KW_LEN, + &&TARGET_CALL_NO_KW_ISINSTANCE, &&TARGET_CALL_NO_KW_LIST_APPEND, &&TARGET_CALL_NO_KW_METHOD_DESCRIPTOR_O, &&TARGET_CALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS, @@ -227,8 +229,6 @@ static void *opcode_targets[256] = { &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, - &&_unknown_opcode, - &&_unknown_opcode, &&TARGET_ENTER_EXECUTOR, &&_unknown_opcode, &&_unknown_opcode, diff --git a/Python/specialize.c b/Python/specialize.c index a3fce2e677591..dcf4be712db20 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -711,8 +711,8 @@ specialize_dict_access( return 1; } -static int specialize_attr_loadmethod(PyObject* owner, _Py_CODEUNIT* instr, PyObject* name, - PyObject* descr, DescriptorClassification kind); +static int specialize_attr_loadclassattr(PyObject* owner, _Py_CODEUNIT* instr, PyObject* name, + PyObject* descr, DescriptorClassification kind, bool is_method); static int specialize_class_load_attr(PyObject* owner, _Py_CODEUNIT* instr, PyObject* name); void @@ -753,7 +753,7 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name) { int oparg = instr->op.arg; if (oparg & 1) { - if (specialize_attr_loadmethod(owner, instr, name, descr, kind)) { + if (specialize_attr_loadclassattr(owner, instr, name, descr, kind, true)) { goto success; } } @@ -872,10 +872,14 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name) SPEC_FAIL_ATTR_NOT_MANAGED_DICT); goto fail; case NON_DESCRIPTOR: - SPECIALIZATION_FAIL(LOAD_ATTR, - (type->tp_flags & Py_TPFLAGS_MANAGED_DICT) ? - SPEC_FAIL_ATTR_CLASS_ATTR_SIMPLE : - SPEC_FAIL_ATTR_NOT_MANAGED_DICT); + if ((instr->op.arg & 1) == 0) { + if (specialize_attr_loadclassattr(owner, instr, name, descr, kind, false)) { + goto success; + } + } + else { + SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_CLASS_ATTR_SIMPLE); + } goto fail; case ABSENT: if (specialize_dict_access(owner, instr, type, kind, name, LOAD_ATTR, @@ -1064,13 +1068,14 @@ specialize_class_load_attr(PyObject *owner, _Py_CODEUNIT *instr, // can cause a significant drop in cache hits. A possible test is // python.exe -m test_typing test_re test_dis test_zlib. static int -specialize_attr_loadmethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, -PyObject *descr, DescriptorClassification kind) +specialize_attr_loadclassattr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, +PyObject *descr, DescriptorClassification kind, bool is_method) { _PyLoadMethodCache *cache = (_PyLoadMethodCache *)(instr + 1); PyTypeObject *owner_cls = Py_TYPE(owner); - assert(kind == METHOD && descr != NULL); + assert(descr != NULL); + assert((is_method && kind == METHOD) || (!is_method && kind == NON_DESCRIPTOR)); if (owner_cls->tp_flags & Py_TPFLAGS_MANAGED_DICT) { PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(owner); PyDictKeysObject *keys = ((PyHeapTypeObject *)owner_cls)->ht_cached_keys; @@ -1090,7 +1095,7 @@ PyObject *descr, DescriptorClassification kind) return 0; } write_u32(cache->keys_version, keys_version); - instr->op.code = LOAD_ATTR_METHOD_WITH_VALUES; + instr->op.code = is_method ? LOAD_ATTR_METHOD_WITH_VALUES : LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES; } else { Py_ssize_t dictoffset = owner_cls->tp_dictoffset; @@ -1099,9 +1104,9 @@ PyObject *descr, DescriptorClassification kind) return 0; } if (dictoffset == 0) { - instr->op.code = LOAD_ATTR_METHOD_NO_DICT; + instr->op.code = is_method ? LOAD_ATTR_METHOD_NO_DICT : LOAD_ATTR_NONDESCRIPTOR_NO_DICT; } - else { + else if (is_method) { PyObject *dict = *(PyObject **) ((char *)owner + dictoffset); if (dict) { SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_NOT_MANAGED_DICT); @@ -1111,6 +1116,10 @@ PyObject *descr, DescriptorClassification kind) assert(owner_cls->tp_dictoffset <= INT16_MAX); instr->op.code = LOAD_ATTR_METHOD_LAZY_DICT; } + else { + SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_CLASS_ATTR_SIMPLE); + return 0; + } } /* `descr` is borrowed. This is safe for methods (even inherited ones from * super classes!) as long as tp_version_tag is validated for two main reasons: From webhook-mailer at python.org Mon Jul 10 07:04:38 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Mon, 10 Jul 2023 11:04:38 -0000 Subject: [Python-checkins] gh-99593: Add tests for Unicode C API (part 3) (GH-104728) Message-ID: https://github.com/python/cpython/commit/51ea664d18938645521bdd128a3c55f9c197644c commit: 51ea664d18938645521bdd128a3c55f9c197644c branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-10T14:04:34+03:00 summary: gh-99593: Add tests for Unicode C API (part 3) (GH-104728) Add tests for codecs. files: M Lib/test/test_capi/test_codecs.py M Modules/_testcapi/unicode.c diff --git a/Lib/test/test_capi/test_codecs.py b/Lib/test/test_capi/test_codecs.py index e46726192aa05..682c56979c6df 100644 --- a/Lib/test/test_capi/test_codecs.py +++ b/Lib/test/test_capi/test_codecs.py @@ -1,10 +1,95 @@ import unittest +import sys from test.support import import_helper _testcapi = import_helper.import_module('_testcapi') +NULL = None + class CAPITest(unittest.TestCase): + # TODO: Test the following functions: + # + # PyUnicode_BuildEncodingMap + # PyUnicode_FSConverter + # PyUnicode_FSDecoder + # PyUnicode_DecodeMBCS + # PyUnicode_DecodeMBCSStateful + # PyUnicode_DecodeCodePageStateful + # PyUnicode_AsMBCSString + # PyUnicode_EncodeCodePage + # PyUnicode_DecodeLocaleAndSize + # PyUnicode_DecodeLocale + # PyUnicode_EncodeLocale + # PyUnicode_DecodeFSDefault + # PyUnicode_DecodeFSDefaultAndSize + # PyUnicode_EncodeFSDefault + + def test_fromencodedobject(self): + """Test PyUnicode_FromEncodedObject()""" + fromencodedobject = _testcapi.unicode_fromencodedobject + + self.assertEqual(fromencodedobject(b'abc', NULL), 'abc') + self.assertEqual(fromencodedobject(b'abc', 'ascii'), 'abc') + b = b'a\xc2\xa1\xe4\xbd\xa0\xf0\x9f\x98\x80' + s = 'a\xa1\u4f60\U0001f600' + self.assertEqual(fromencodedobject(b, NULL), s) + self.assertEqual(fromencodedobject(b, 'utf-8'), s) + self.assertEqual(fromencodedobject(b, 'latin1'), b.decode('latin1')) + self.assertRaises(UnicodeDecodeError, fromencodedobject, b, 'ascii') + self.assertEqual(fromencodedobject(b, 'ascii', 'replace'), + 'a' + '\ufffd'*9) + self.assertEqual(fromencodedobject(bytearray(b), NULL), s) + self.assertEqual(fromencodedobject(bytearray(b), 'utf-8'), s) + self.assertRaises(LookupError, fromencodedobject, b'abc', 'foo') + self.assertRaises(LookupError, fromencodedobject, b, 'ascii', 'foo') + self.assertRaises(TypeError, fromencodedobject, 'abc', NULL) + self.assertRaises(TypeError, fromencodedobject, 'abc', 'ascii') + self.assertRaises(TypeError, fromencodedobject, [], NULL) + self.assertRaises(TypeError, fromencodedobject, [], 'ascii') + self.assertRaises(SystemError, fromencodedobject, NULL, NULL) + self.assertRaises(SystemError, fromencodedobject, NULL, 'ascii') + + def test_decode(self): + """Test PyUnicode_Decode()""" + decode = _testcapi.unicode_decode + + self.assertEqual(decode(b'[\xe2\x82\xac]', 'utf-8'), '[\u20ac]') + self.assertEqual(decode(b'[\xa4]', 'iso8859-15'), '[\u20ac]') + self.assertEqual(decode(b'[\xa4]', 'iso8859-15', 'strict'), '[\u20ac]') + self.assertRaises(UnicodeDecodeError, decode, b'[\xa4]', 'utf-8') + self.assertEqual(decode(b'[\xa4]', 'utf-8', 'replace'), '[\ufffd]') + + self.assertEqual(decode(b'[\xe2\x82\xac]', NULL), '[\u20ac]') + self.assertEqual(decode(b'[\xa4]', NULL, 'replace'), '[\ufffd]') + + self.assertRaises(LookupError, decode, b'\xa4', 'foo') + self.assertRaises(LookupError, decode, b'\xa4', 'utf-8', 'foo') + # TODO: Test PyUnicode_Decode() with NULL as data and + # negative size. + + def test_asencodedstring(self): + """Test PyUnicode_AsEncodedString()""" + asencodedstring = _testcapi.unicode_asencodedstring + + self.assertEqual(asencodedstring('abc', NULL), b'abc') + self.assertEqual(asencodedstring('abc', 'ascii'), b'abc') + s = 'a\xa1\u4f60\U0001f600' + b = b'a\xc2\xa1\xe4\xbd\xa0\xf0\x9f\x98\x80' + self.assertEqual(asencodedstring(s, NULL), b) + self.assertEqual(asencodedstring(s, 'utf-8'), b) + self.assertEqual(asencodedstring('\xa1\xa2', 'latin1'), b'\xa1\xa2') + self.assertRaises(UnicodeEncodeError, asencodedstring, '\xa1\xa2', 'ascii') + self.assertEqual(asencodedstring(s, 'ascii', 'replace'), b'a???') + + self.assertRaises(LookupError, asencodedstring, 'abc', 'foo') + self.assertRaises(LookupError, asencodedstring, s, 'ascii', 'foo') + self.assertRaises(TypeError, asencodedstring, b'abc', NULL) + self.assertRaises(TypeError, asencodedstring, b'abc', 'ascii') + self.assertRaises(TypeError, asencodedstring, [], NULL) + self.assertRaises(TypeError, asencodedstring, [], 'ascii') + # CRASHES asencodedstring(NULL, NULL) + # CRASHES asencodedstring(NULL, 'ascii') def test_decodeutf8(self): """Test PyUnicode_DecodeUTF8()""" @@ -49,6 +134,387 @@ def test_decodeutf8stateful(self): # TODO: Test PyUnicode_DecodeUTF8Stateful() with NULL as the address of # "consumed". + def test_asutf8string(self): + """Test PyUnicode_AsUTF8String()""" + asutf8string = _testcapi.unicode_asutf8string + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + self.assertEqual(asutf8string(s), s.encode('utf-8')) + + self.assertRaises(UnicodeEncodeError, asutf8string, '\ud8ff') + self.assertRaises(TypeError, asutf8string, b'abc') + self.assertRaises(TypeError, asutf8string, []) + # CRASHES asutf8string(NULL) + + def test_decodeutf16(self): + """Test PyUnicode_DecodeUTF16()""" + decodeutf16 = _testcapi.unicode_decodeutf16 + + naturalbyteorder = -1 if sys.byteorder == 'little' else 1 + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-16') + self.assertEqual(decodeutf16(0, b), (naturalbyteorder, s)) + b = s.encode('utf-16le') + self.assertEqual(decodeutf16(-1, b), (-1, s)) + self.assertEqual(decodeutf16(0, b'\xff\xfe'+b), (-1, s)) + b = s.encode('utf-16be') + self.assertEqual(decodeutf16(1, b), (1, s)) + self.assertEqual(decodeutf16(0, b'\xfe\xff'+b), (1, s)) + + self.assertRaises(UnicodeDecodeError, decodeutf16, -1, b'a') + self.assertRaises(UnicodeDecodeError, decodeutf16, 1, b'a') + self.assertRaises(UnicodeDecodeError, decodeutf16, 0, b'\xff\xfea') + self.assertRaises(UnicodeDecodeError, decodeutf16, 0, b'\xfe\xffa') + + self.assertRaises(UnicodeDecodeError, decodeutf16, -1, b'\x00\xde') + self.assertRaises(UnicodeDecodeError, decodeutf16, 1, b'\xde\x00') + self.assertRaises(UnicodeDecodeError, decodeutf16, 0, b'\xde\xde') + self.assertEqual(decodeutf16(-1, b'\x00\xde', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf16(1, b'\xde\x00', 'replace'), (1, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xde\xde', 'replace'), (0, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xff\xfe\x00\xde', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xfe\xff\xde\x00', 'replace'), (1, '\ufffd')) + + self.assertRaises(UnicodeDecodeError, decodeutf16, -1, b'\x3d\xd8') + self.assertRaises(UnicodeDecodeError, decodeutf16, 1, b'\xd8\x3d') + self.assertRaises(UnicodeDecodeError, decodeutf16, 0, b'\xd8\xd8') + self.assertEqual(decodeutf16(-1, b'\x3d\xd8', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf16(1, b'\xd8\x3d', 'replace'), (1, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xd8\xd8', 'replace'), (0, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xff\xfe\x3d\xd8', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xfe\xff\xd8\x3d', 'replace'), (1, '\ufffd')) + + self.assertRaises(LookupError, decodeutf16, -1, b'\x00\xde', 'foo') + self.assertRaises(LookupError, decodeutf16, 1, b'\xde\x00', 'foo') + self.assertRaises(LookupError, decodeutf16, 0, b'\xde\xde', 'foo') + # TODO: Test PyUnicode_DecodeUTF16() with NULL as data and + # negative size. + + def test_decodeutf16stateful(self): + """Test PyUnicode_DecodeUTF16Stateful()""" + decodeutf16stateful = _testcapi.unicode_decodeutf16stateful + + naturalbyteorder = -1 if sys.byteorder == 'little' else 1 + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-16') + self.assertEqual(decodeutf16stateful(0, b), (naturalbyteorder, s, len(b))) + b = s.encode('utf-16le') + self.assertEqual(decodeutf16stateful(-1, b), (-1, s, len(b))) + self.assertEqual(decodeutf16stateful(0, b'\xff\xfe'+b), (-1, s, len(b)+2)) + b = s.encode('utf-16be') + self.assertEqual(decodeutf16stateful(1, b), (1, s, len(b))) + self.assertEqual(decodeutf16stateful(0, b'\xfe\xff'+b), (1, s, len(b)+2)) + + self.assertEqual(decodeutf16stateful(-1, b'\x61\x00\x3d'), (-1, 'a', 2)) + self.assertEqual(decodeutf16stateful(-1, b'\x61\x00\x3d\xd8'), (-1, 'a', 2)) + self.assertEqual(decodeutf16stateful(-1, b'\x61\x00\x3d\xd8\x00'), (-1, 'a', 2)) + self.assertEqual(decodeutf16stateful(1, b'\x00\x61\xd8'), (1, 'a', 2)) + self.assertEqual(decodeutf16stateful(1, b'\x00\x61\xd8\x3d'), (1, 'a', 2)) + self.assertEqual(decodeutf16stateful(1, b'\x00\x61\xd8\x3d\xde'), (1, 'a', 2)) + self.assertEqual(decodeutf16stateful(0, b'\xff\xfe\x61\x00\x3d\xd8\x00'), (-1, 'a', 4)) + self.assertEqual(decodeutf16stateful(0, b'\xfe\xff\x00\x61\xd8\x3d\xde'), (1, 'a', 4)) + + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, -1, b'\x00\xde') + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, 1, b'\xde\x00') + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, 0, b'\xde\xde') + self.assertEqual(decodeutf16stateful(-1, b'\x00\xde', 'replace'), (-1, '\ufffd', 2)) + self.assertEqual(decodeutf16stateful(1, b'\xde\x00', 'replace'), (1, '\ufffd', 2)) + self.assertEqual(decodeutf16stateful(0, b'\xde\xde', 'replace'), (0, '\ufffd', 2)) + self.assertEqual(decodeutf16stateful(0, b'\xff\xfe\x00\xde', 'replace'), (-1, '\ufffd', 4)) + self.assertEqual(decodeutf16stateful(0, b'\xfe\xff\xde\x00', 'replace'), (1, '\ufffd', 4)) + + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, -1, b'\x3d\xd8\x61\x00') + self.assertEqual(decodeutf16stateful(-1, b'\x3d\xd8\x61\x00', 'replace'), (-1, '\ufffda', 4)) + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, 1, b'\xd8\x3d\x00\x61') + self.assertEqual(decodeutf16stateful(1, b'\xd8\x3d\x00\x61', 'replace'), (1, '\ufffda', 4)) + + self.assertRaises(LookupError, decodeutf16stateful, -1, b'\x00\xde', 'foo') + self.assertRaises(LookupError, decodeutf16stateful, 1, b'\xde\x00', 'foo') + self.assertRaises(LookupError, decodeutf16stateful, 0, b'\xde\xde', 'foo') + # TODO: Test PyUnicode_DecodeUTF16Stateful() with NULL as data and + # negative size. + # TODO: Test PyUnicode_DecodeUTF16Stateful() with NULL as the address of + # "consumed". + + def test_asutf16string(self): + """Test PyUnicode_AsUTF16String()""" + asutf16string = _testcapi.unicode_asutf16string + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + self.assertEqual(asutf16string(s), s.encode('utf-16')) + + self.assertRaises(UnicodeEncodeError, asutf16string, '\ud8ff') + self.assertRaises(TypeError, asutf16string, b'abc') + self.assertRaises(TypeError, asutf16string, []) + # CRASHES asutf16string(NULL) + + def test_decodeutf32(self): + """Test PyUnicode_DecodeUTF8()""" + decodeutf32 = _testcapi.unicode_decodeutf32 + + naturalbyteorder = -1 if sys.byteorder == 'little' else 1 + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-32') + self.assertEqual(decodeutf32(0, b), (naturalbyteorder, s)) + b = s.encode('utf-32le') + self.assertEqual(decodeutf32(-1, b), (-1, s)) + self.assertEqual(decodeutf32(0, b'\xff\xfe\x00\x00'+b), (-1, s)) + b = s.encode('utf-32be') + self.assertEqual(decodeutf32(1, b), (1, s)) + self.assertEqual(decodeutf32(0, b'\x00\x00\xfe\xff'+b), (1, s)) + + self.assertRaises(UnicodeDecodeError, decodeutf32, -1, b'\x61\x00\x00\x00\x00') + self.assertRaises(UnicodeDecodeError, decodeutf32, 1, b'\x00\x00\x00\x61\x00') + self.assertRaises(UnicodeDecodeError, decodeutf32, 0, b'\xff\xfe\x00\x00\x61\x00\x00\x00\x00') + self.assertRaises(UnicodeDecodeError, decodeutf32, 0, b'\x00\x00\xfe\xff\x00\x00\x00\x61\x00') + + self.assertRaises(UnicodeDecodeError, decodeutf32, -1, b'\xff\xff\xff\xff') + self.assertRaises(UnicodeDecodeError, decodeutf32, 1, b'\xff\xff\xff\xff') + self.assertRaises(UnicodeDecodeError, decodeutf32, 0, b'\xff\xff\xff\xff') + self.assertEqual(decodeutf32(-1, b'\xff\xff\xff\xff', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf32(1, b'\xff\xff\xff\xff', 'replace'), (1, '\ufffd')) + self.assertEqual(decodeutf32(0, b'\xff\xff\xff\xff', 'replace'), (0, '\ufffd')) + self.assertEqual(decodeutf32(0, b'\xff\xfe\x00\x00\xff\xff\xff\xff', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf32(0, b'\x00\x00\xfe\xff\xff\xff\xff\xff', 'replace'), (1, '\ufffd')) + + self.assertRaises(UnicodeDecodeError, decodeutf32, -1, b'\x3d\xd8\x00\x00') + self.assertEqual(decodeutf32(-1, b'\x3d\xd8\x00\x00', 'replace'), (-1, '\ufffd')) + self.assertRaises(UnicodeDecodeError, decodeutf32, 1, b'\x00\x00\xd8\x3d') + self.assertEqual(decodeutf32(1, b'\x00\x00\xd8\x3d', 'replace'), (1, '\ufffd')) + + self.assertRaises(LookupError, decodeutf32, -1, b'\xff\xff\xff\xff', 'foo') + self.assertRaises(LookupError, decodeutf32, 1, b'\xff\xff\xff\xff', 'foo') + self.assertRaises(LookupError, decodeutf32, 0, b'\xff\xff\xff\xff', 'foo') + # TODO: Test PyUnicode_DecodeUTF32() with NULL as data and + # negative size. + + def test_decodeutf32stateful(self): + """Test PyUnicode_DecodeUTF32Stateful()""" + decodeutf32stateful = _testcapi.unicode_decodeutf32stateful + + naturalbyteorder = -1 if sys.byteorder == 'little' else 1 + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-32') + self.assertEqual(decodeutf32stateful(0, b), (naturalbyteorder, s, len(b))) + b = s.encode('utf-32le') + self.assertEqual(decodeutf32stateful(-1, b), (-1, s, len(b))) + self.assertEqual(decodeutf32stateful(0, b'\xff\xfe\x00\x00'+b), (-1, s, len(b)+4)) + b = s.encode('utf-32be') + self.assertEqual(decodeutf32stateful(1, b), (1, s, len(b))) + self.assertEqual(decodeutf32stateful(0, b'\x00\x00\xfe\xff'+b), (1, s, len(b)+4)) + + self.assertEqual(decodeutf32stateful(-1, b'\x61\x00\x00\x00\x00'), (-1, 'a', 4)) + self.assertEqual(decodeutf32stateful(-1, b'\x61\x00\x00\x00\x00\xf6'), (-1, 'a', 4)) + self.assertEqual(decodeutf32stateful(-1, b'\x61\x00\x00\x00\x00\xf6\x01'), (-1, 'a', 4)) + self.assertEqual(decodeutf32stateful(1, b'\x00\x00\x00\x61\x00'), (1, 'a', 4)) + self.assertEqual(decodeutf32stateful(1, b'\x00\x00\x00\x61\x00\x01'), (1, 'a', 4)) + self.assertEqual(decodeutf32stateful(1, b'\x00\x00\x00\x61\x00\x01\xf6'), (1, 'a', 4)) + self.assertEqual(decodeutf32stateful(0, b'\xff\xfe\x00\x00\x61\x00\x00\x00\x00\xf6\x01'), (-1, 'a', 8)) + self.assertEqual(decodeutf32stateful(0, b'\x00\x00\xfe\xff\x00\x00\x00\x61\x00\x01\xf6'), (1, 'a', 8)) + + for b in b'\xff', b'\xff\xff', b'\xff\xff\xff': + self.assertEqual(decodeutf32stateful(-1, b), (-1, '', 0)) + self.assertEqual(decodeutf32stateful(1, b), (1, '', 0)) + self.assertEqual(decodeutf32stateful(0, b), (0, '', 0)) + self.assertEqual(decodeutf32stateful(0, b'\xff\xfe\x00\x00'+b), (-1, '', 4)) + self.assertEqual(decodeutf32stateful(0, b'\x00\x00\xfe\xff'+b), (1, '', 4)) + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, -1, b'\xff\xff\xff\xff') + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, 1, b'\xff\xff\xff\xff') + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, 0, b'\xff\xff\xff\xff') + self.assertEqual(decodeutf32stateful(-1, b'\xff\xff\xff\xff', 'replace'), (-1, '\ufffd', 4)) + self.assertEqual(decodeutf32stateful(1, b'\xff\xff\xff\xff', 'replace'), (1, '\ufffd', 4)) + self.assertEqual(decodeutf32stateful(0, b'\xff\xff\xff\xff', 'replace'), (0, '\ufffd', 4)) + self.assertEqual(decodeutf32stateful(0, b'\xff\xfe\x00\x00\xff\xff\xff\xff', 'replace'), (-1, '\ufffd', 8)) + self.assertEqual(decodeutf32stateful(0, b'\x00\x00\xfe\xff\xff\xff\xff\xff', 'replace'), (1, '\ufffd', 8)) + + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, -1, b'\x3d\xd8\x00\x00') + self.assertEqual(decodeutf32stateful(-1, b'\x3d\xd8\x00\x00', 'replace'), (-1, '\ufffd', 4)) + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, 1, b'\x00\x00\xd8\x3d') + self.assertEqual(decodeutf32stateful(1, b'\x00\x00\xd8\x3d', 'replace'), (1, '\ufffd', 4)) + + self.assertRaises(LookupError, decodeutf32stateful, -1, b'\xff\xff\xff\xff', 'foo') + self.assertRaises(LookupError, decodeutf32stateful, 1, b'\xff\xff\xff\xff', 'foo') + self.assertRaises(LookupError, decodeutf32stateful, 0, b'\xff\xff\xff\xff', 'foo') + # TODO: Test PyUnicode_DecodeUTF32Stateful() with NULL as data and + # negative size. + # TODO: Test PyUnicode_DecodeUTF32Stateful() with NULL as the address of + # "consumed". + + def test_asutf32string(self): + """Test PyUnicode_AsUTF32String()""" + asutf32string = _testcapi.unicode_asutf32string + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + self.assertEqual(asutf32string(s), s.encode('utf-32')) + + self.assertRaises(UnicodeEncodeError, asutf32string, '\ud8ff') + self.assertRaises(TypeError, asutf32string, b'abc') + self.assertRaises(TypeError, asutf32string, []) + # CRASHES asutf32string(NULL) + + def test_decodelatin1(self): + """Test PyUnicode_DecodeLatin1()""" + decodelatin1 = _testcapi.unicode_decodelatin1 + + self.assertEqual(decodelatin1(b'abc'), 'abc') + self.assertEqual(decodelatin1(b'abc', 'strict'), 'abc') + self.assertEqual(decodelatin1(b'\xa1\xa2'), '\xa1\xa2') + self.assertEqual(decodelatin1(b'\xa1\xa2', 'strict'), '\xa1\xa2') + # TODO: Test PyUnicode_DecodeLatin1() with NULL as data and + # negative size. + + def test_aslatin1string(self): + """Test PyUnicode_AsLatin1String()""" + aslatin1string = _testcapi.unicode_aslatin1string + + self.assertEqual(aslatin1string('abc'), b'abc') + self.assertEqual(aslatin1string('\xa1\xa2'), b'\xa1\xa2') + + self.assertRaises(UnicodeEncodeError, aslatin1string, '\u4f60') + self.assertRaises(TypeError, aslatin1string, b'abc') + self.assertRaises(TypeError, aslatin1string, []) + # CRASHES aslatin1string(NULL) + + def test_decodeascii(self): + """Test PyUnicode_DecodeASCII()""" + decodeascii = _testcapi.unicode_decodeascii + + self.assertEqual(decodeascii(b'abc'), 'abc') + self.assertEqual(decodeascii(b'abc', 'strict'), 'abc') + + self.assertRaises(UnicodeDecodeError, decodeascii, b'\xff') + self.assertEqual(decodeascii(b'a\xff', 'replace'), 'a\ufffd') + self.assertEqual(decodeascii(b'a\xffb', 'replace'), 'a\ufffdb') + + self.assertRaises(LookupError, decodeascii, b'a\xff', 'foo') + # TODO: Test PyUnicode_DecodeASCII() with NULL as data and + # negative size. + + def test_asasciistring(self): + """Test PyUnicode_AsASCIIString()""" + asasciistring = _testcapi.unicode_asasciistring + + self.assertEqual(asasciistring('abc'), b'abc') + + self.assertRaises(UnicodeEncodeError, asasciistring, '\x80') + self.assertRaises(TypeError, asasciistring, b'abc') + self.assertRaises(TypeError, asasciistring, []) + # CRASHES asasciistring(NULL) + + def test_decodecharmap(self): + """Test PyUnicode_DecodeCharmap()""" + decodecharmap = _testcapi.unicode_decodecharmap + + self.assertEqual(decodecharmap(b'\3\0\7', {0: 'a', 3: 'b', 7: 'c'}), 'bac') + self.assertEqual(decodecharmap(b'\1\0\2', ['a', 'b', 'c']), 'bac') + self.assertEqual(decodecharmap(b'\1\0\2', 'abc'), 'bac') + self.assertEqual(decodecharmap(b'\1\0\2', ['\xa1', '\xa2', '\xa3']), '\xa2\xa1\xa3') + self.assertEqual(decodecharmap(b'\1\0\2', ['\u4f60', '\u597d', '\u4e16']), '\u597d\u4f60\u4e16') + self.assertEqual(decodecharmap(b'\1\0\2', ['\U0001f600', '\U0001f601', '\U0001f602']), '\U0001f601\U0001f600\U0001f602') + + self.assertEqual(decodecharmap(b'\1\0\2', [97, 98, 99]), 'bac') + self.assertEqual(decodecharmap(b'\1\0\2', ['', 'b', 'cd']), 'bcd') + + self.assertRaises(UnicodeDecodeError, decodecharmap, b'\0', {}) + self.assertRaises(UnicodeDecodeError, decodecharmap, b'\0', {0: None}) + self.assertEqual(decodecharmap(b'\1\0\2', [None, 'b', 'c'], 'replace'), 'b\ufffdc') + self.assertEqual(decodecharmap(b'\1\0\2\xff', NULL), '\1\0\2\xff') + self.assertRaises(TypeError, decodecharmap, b'\0', 42) + + # TODO: Test PyUnicode_DecodeCharmap() with NULL as data and + # negative size. + + def test_ascharmapstring(self): + """Test PyUnicode_AsCharmapString()""" + ascharmapstring = _testcapi.unicode_ascharmapstring + + self.assertEqual(ascharmapstring('abc', {97: 3, 98: 0, 99: 7}), b'\3\0\7') + self.assertEqual(ascharmapstring('\xa1\xa2\xa3', {0xa1: 3, 0xa2: 0, 0xa3: 7}), b'\3\0\7') + self.assertEqual(ascharmapstring('\u4f60\u597d\u4e16', {0x4f60: 3, 0x597d: 0, 0x4e16: 7}), b'\3\0\7') + self.assertEqual(ascharmapstring('\U0001f600\U0001f601\U0001f602', {0x1f600: 3, 0x1f601: 0, 0x1f602: 7}), b'\3\0\7') + self.assertEqual(ascharmapstring('abc', {97: 3, 98: b'', 99: b'spam'}), b'\3spam') + + self.assertRaises(UnicodeEncodeError, ascharmapstring, 'a', {}) + self.assertRaises(UnicodeEncodeError, ascharmapstring, 'a', {97: None}) + self.assertRaises(TypeError, ascharmapstring, b'a', {}) + self.assertRaises(TypeError, ascharmapstring, [], {}) + self.assertRaises(TypeError, ascharmapstring, 'a', NULL) + # CRASHES ascharmapstring(NULL, {}) + + def test_decodeunicodeescape(self): + """Test PyUnicode_DecodeUnicodeEscape()""" + decodeunicodeescape = _testcapi.unicode_decodeunicodeescape + + self.assertEqual(decodeunicodeescape(b'abc'), 'abc') + self.assertEqual(decodeunicodeescape(br'\t\n\r\x0b\x0c\x00\\'), '\t\n\r\v\f\0\\') + self.assertEqual(decodeunicodeescape(b'\t\n\r\x0b\x0c\x00'), '\t\n\r\v\f\0') + self.assertEqual(decodeunicodeescape(br'\xa1\xa2'), '\xa1\xa2') + self.assertEqual(decodeunicodeescape(b'\xa1\xa2'), '\xa1\xa2') + self.assertEqual(decodeunicodeescape(br'\u4f60\u597d'), '\u4f60\u597d') + self.assertEqual(decodeunicodeescape(br'\U0001f600'), '\U0001f600') + with self.assertWarns(DeprecationWarning): + self.assertEqual(decodeunicodeescape(br'\z'), r'\z') + + for b in b'\\', br'\xa', br'\u4f6', br'\U0001f60': + self.assertRaises(UnicodeDecodeError, decodeunicodeescape, b) + self.assertRaises(UnicodeDecodeError, decodeunicodeescape, b, 'strict') + self.assertEqual(decodeunicodeescape(br'x\U0001f60', 'replace'), 'x\ufffd') + self.assertEqual(decodeunicodeescape(br'x\U0001f60y', 'replace'), 'x\ufffdy') + + self.assertRaises(LookupError, decodeunicodeescape, b'\\', 'foo') + # TODO: Test PyUnicode_DecodeUnicodeEscape() with NULL as data and + # negative size. + + def test_asunicodeescapestring(self): + """Test PyUnicode_AsUnicodeEscapeString()""" + asunicodeescapestring = _testcapi.unicode_asunicodeescapestring + + self.assertEqual(asunicodeescapestring('abc'), b'abc') + self.assertEqual(asunicodeescapestring('\t\n\r\v\f\0\\'), br'\t\n\r\x0b\x0c\x00\\') + self.assertEqual(asunicodeescapestring('\xa1\xa2'), br'\xa1\xa2') + self.assertEqual(asunicodeescapestring('\u4f60\u597d'), br'\u4f60\u597d') + self.assertEqual(asunicodeescapestring('\U0001f600'), br'\U0001f600') + + self.assertRaises(TypeError, asunicodeescapestring, b'abc') + self.assertRaises(TypeError, asunicodeescapestring, []) + # CRASHES asunicodeescapestring(NULL) + + def test_decoderawunicodeescape(self): + """Test PyUnicode_DecodeRawUnicodeEscape()""" + decoderawunicodeescape = _testcapi.unicode_decoderawunicodeescape + + self.assertEqual(decoderawunicodeescape(b'abc'), 'abc') + self.assertEqual(decoderawunicodeescape(b'\t\n\r\v\f\0\\'), '\t\n\r\v\f\0\\') + self.assertEqual(decoderawunicodeescape(b'\xa1\xa2'), '\xa1\xa2') + self.assertEqual(decoderawunicodeescape(br'\u4f60\u597d'), '\u4f60\u597d') + self.assertEqual(decoderawunicodeescape(br'\U0001f600'), '\U0001f600') + self.assertEqual(decoderawunicodeescape(br'\xa1\xa2'), r'\xa1\xa2') + self.assertEqual(decoderawunicodeescape(br'\z'), r'\z') + + for b in br'\u4f6', br'\U0001f60': + self.assertRaises(UnicodeDecodeError, decoderawunicodeescape, b) + self.assertRaises(UnicodeDecodeError, decoderawunicodeescape, b, 'strict') + self.assertEqual(decoderawunicodeescape(br'x\U0001f60', 'replace'), 'x\ufffd') + self.assertEqual(decoderawunicodeescape(br'x\U0001f60y', 'replace'), 'x\ufffdy') + + self.assertRaises(LookupError, decoderawunicodeescape, br'\U0001f60', 'foo') + # TODO: Test PyUnicode_DecodeRawUnicodeEscape() with NULL as data and + # negative size. + + def test_asrawunicodeescapestring(self): + """Test PyUnicode_AsRawUnicodeEscapeString()""" + asrawunicodeescapestring = _testcapi.unicode_asrawunicodeescapestring + + self.assertEqual(asrawunicodeescapestring('abc'), b'abc') + self.assertEqual(asrawunicodeescapestring('\t\n\r\v\f\0\\'), b'\t\n\r\v\f\0\\') + self.assertEqual(asrawunicodeescapestring('\xa1\xa2'), b'\xa1\xa2') + self.assertEqual(asrawunicodeescapestring('\u4f60\u597d'), br'\u4f60\u597d') + self.assertEqual(asrawunicodeescapestring('\U0001f600'), br'\U0001f600') + + self.assertRaises(TypeError, asrawunicodeescapestring, b'abc') + self.assertRaises(TypeError, asrawunicodeescapestring, []) + # CRASHES asrawunicodeescapestring(NULL) + if __name__ == "__main__": unittest.main() diff --git a/Modules/_testcapi/unicode.c b/Modules/_testcapi/unicode.c index 51d741a6b5ff1..b4c6c4b5e3ce1 100644 --- a/Modules/_testcapi/unicode.c +++ b/Modules/_testcapi/unicode.c @@ -375,6 +375,22 @@ unicode_readchar(PyObject *self, PyObject *args) return PyLong_FromUnsignedLong(result); } +/* Test PyUnicode_FromEncodedObject() */ +static PyObject * +unicode_fromencodedobject(PyObject *self, PyObject *args) +{ + PyObject *obj; + const char *encoding; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "Oz|z", &obj, &encoding, &errors)) { + return NULL; + } + + NULLABLE(obj); + return PyUnicode_FromEncodedObject(obj, encoding, errors); +} + /* Test PyUnicode_FromObject() */ static PyObject * unicode_fromobject(PyObject *self, PyObject *arg) @@ -660,6 +676,78 @@ unicode_getdefaultencoding(PyObject *self, PyObject *Py_UNUSED(ignored)) return PyBytes_FromString(s); } +/* Test PyUnicode_Decode() */ +static PyObject * +unicode_decode(PyObject *self, PyObject *args) +{ + const char *s; + Py_ssize_t size; + const char *encoding; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#z|z", &s, &size, &encoding, &errors)) + return NULL; + + return PyUnicode_Decode(s, size, encoding, errors); +} + +/* Test PyUnicode_AsEncodedString() */ +static PyObject * +unicode_asencodedstring(PyObject *self, PyObject *args) +{ + PyObject *unicode; + const char *encoding; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "Oz|z", &unicode, &encoding, &errors)) + return NULL; + + NULLABLE(unicode); + return PyUnicode_AsEncodedString(unicode, encoding, errors); +} + +/* Test PyUnicode_BuildEncodingMap() */ +static PyObject * +unicode_buildencodingmap(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_BuildEncodingMap(arg); +} + +/* Test PyUnicode_DecodeUTF7() */ +static PyObject * +unicode_decodeutf7(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeUTF7(data, size, errors); +} + +/* Test PyUnicode_DecodeUTF7Stateful() */ +static PyObject * +unicode_decodeutf7stateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF7Stateful(data, size, errors, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(Nn)", result, consumed); +} + /* Test PyUnicode_DecodeUTF8() */ static PyObject * unicode_decodeutf8(PyObject *self, PyObject *args) @@ -694,6 +782,387 @@ unicode_decodeutf8stateful(PyObject *self, PyObject *args) return Py_BuildValue("(Nn)", result, consumed); } +/* Test PyUnicode_AsUTF8String() */ +static PyObject * +unicode_asutf8string(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsUTF8String(arg); +} + +/* Test PyUnicode_DecodeUTF32() */ +static PyObject * +unicode_decodeutf32(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + int byteorder; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &byteorder, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF32(data, size, errors, &byteorder); + if (!result) { + return NULL; + } + return Py_BuildValue("(iN)", byteorder, result); +} + +/* Test PyUnicode_DecodeUTF32Stateful() */ +static PyObject * +unicode_decodeutf32stateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + int byteorder; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &byteorder, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF32Stateful(data, size, errors, &byteorder, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(iNn)", byteorder, result, consumed); +} + +/* Test PyUnicode_AsUTF32String() */ +static PyObject * +unicode_asutf32string(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsUTF32String(arg); +} + +/* Test PyUnicode_DecodeUTF16() */ +static PyObject * +unicode_decodeutf16(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + int byteorder = 0; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &byteorder, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF16(data, size, errors, &byteorder); + if (!result) { + return NULL; + } + return Py_BuildValue("(iN)", byteorder, result); +} + +/* Test PyUnicode_DecodeUTF16Stateful() */ +static PyObject * +unicode_decodeutf16stateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + int byteorder; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &byteorder, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF16Stateful(data, size, errors, &byteorder, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(iNn)", byteorder, result, consumed); +} + +/* Test PyUnicode_AsUTF16String() */ +static PyObject * +unicode_asutf16string(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsUTF16String(arg); +} + +/* Test PyUnicode_DecodeUnicodeEscape() */ +static PyObject * +unicode_decodeunicodeescape(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeUnicodeEscape(data, size, errors); +} + +/* Test PyUnicode_AsUnicodeEscapeString() */ +static PyObject * +unicode_asunicodeescapestring(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsUnicodeEscapeString(arg); +} + +static PyObject * +unicode_decoderawunicodeescape(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeRawUnicodeEscape(data, size, errors); +} + +/* Test PyUnicode_AsRawUnicodeEscapeString() */ +static PyObject * +unicode_asrawunicodeescapestring(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsRawUnicodeEscapeString(arg); +} + +static PyObject * +unicode_decodelatin1(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeLatin1(data, size, errors); +} + +/* Test PyUnicode_AsLatin1String() */ +static PyObject * +unicode_aslatin1string(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsLatin1String(arg); +} + +/* Test PyUnicode_DecodeASCII() */ +static PyObject * +unicode_decodeascii(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeASCII(data, size, errors); +} + +/* Test PyUnicode_AsASCIIString() */ +static PyObject * +unicode_asasciistring(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsASCIIString(arg); +} + +/* Test PyUnicode_DecodeCharmap() */ +static PyObject * +unicode_decodecharmap(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + PyObject *mapping; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#O|z", &data, &size, &mapping, &errors)) + return NULL; + + NULLABLE(mapping); + return PyUnicode_DecodeCharmap(data, size, mapping, errors); +} + +/* Test PyUnicode_AsCharmapString() */ +static PyObject * +unicode_ascharmapstring(PyObject *self, PyObject *args) +{ + PyObject *unicode; + PyObject *mapping; + + if (!PyArg_ParseTuple(args, "OO", &unicode, &mapping)) + return NULL; + + NULLABLE(unicode); + NULLABLE(mapping); + return PyUnicode_AsCharmapString(unicode, mapping); +} + +#ifdef MS_WINDOWS + +/* Test PyUnicode_DecodeMBCS() */ +static PyObject * +unicode_decodembcs(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeMBCS(data, size, errors); +} + +/* Test PyUnicode_DecodeMBCSStateful() */ +static PyObject * +unicode_decodembcsstateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeMBCSStateful(data, size, errors, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(Nn)", result, consumed); +} + +/* Test PyUnicode_DecodeCodePageStateful() */ +static PyObject * +unicode_decodecodepagestateful(PyObject *self, PyObject *args) +{ + int code_page; + const char *data; + Py_ssize_t size; + const char *errors = NULL; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &code_page, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeCodePageStateful(code_page, data, size, errors, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(Nn)", result, consumed); +} + +/* Test PyUnicode_AsMBCSString() */ +static PyObject * +unicode_asmbcsstring(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsMBCSString(arg); +} + +/* Test PyUnicode_EncodeCodePage() */ +static PyObject * +unicode_encodecodepage(PyObject *self, PyObject *args) +{ + int code_page; + PyObject *unicode; + const char *errors; + + if (!PyArg_ParseTuple(args, "iO|z", &code_page, &unicode, &errors)) + return NULL; + + NULLABLE(unicode); + return PyUnicode_EncodeCodePage(code_page, unicode, errors); +} + +#endif /* MS_WINDOWS */ + +/* Test PyUnicode_DecodeLocaleAndSize() */ +static PyObject * +unicode_decodelocaleandsize(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeLocaleAndSize(data, size, errors); +} + +/* Test PyUnicode_DecodeLocale() */ +static PyObject * +unicode_decodelocale(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeLocale(data, errors); +} + +/* Test PyUnicode_EncodeLocale() */ +static PyObject * +unicode_encodelocale(PyObject *self, PyObject *args) +{ + PyObject *unicode; + const char *errors; + + if (!PyArg_ParseTuple(args, "O|z", &unicode, &errors)) + return NULL; + + NULLABLE(unicode); + return PyUnicode_EncodeLocale(unicode, errors); +} + +/* Test PyUnicode_DecodeFSDefault() */ +static PyObject * +unicode_decodefsdefault(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + + if (!PyArg_ParseTuple(args, "y#", &data, &size)) + return NULL; + + return PyUnicode_DecodeFSDefault(data); +} + +/* Test PyUnicode_DecodeFSDefaultAndSize() */ +static PyObject * +unicode_decodefsdefaultandsize(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + + if (!PyArg_ParseTuple(args, "y#|n", &data, &size, &size)) + return NULL; + + return PyUnicode_DecodeFSDefaultAndSize(data, size); +} + +/* Test PyUnicode_EncodeFSDefault() */ +static PyObject * +unicode_encodefsdefault(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_EncodeFSDefault(arg); +} + /* Test PyUnicode_Concat() */ static PyObject * unicode_concat(PyObject *self, PyObject *args) @@ -1519,6 +1988,7 @@ static PyMethodDef TestMethods[] = { {"unicode_substring", unicode_substring, METH_VARARGS}, {"unicode_getlength", unicode_getlength, METH_O}, {"unicode_readchar", unicode_readchar, METH_VARARGS}, + {"unicode_fromencodedobject",unicode_fromencodedobject, METH_VARARGS}, {"unicode_fromobject", unicode_fromobject, METH_O}, {"unicode_interninplace", unicode_interninplace, METH_O}, {"unicode_internfromstring", unicode_internfromstring, METH_O}, @@ -1533,9 +2003,44 @@ static PyMethodDef TestMethods[] = { {"unicode_asutf8", unicode_asutf8, METH_VARARGS}, {"unicode_asutf8andsize", unicode_asutf8andsize, METH_VARARGS}, {"unicode_asutf8andsize_null",unicode_asutf8andsize_null, METH_VARARGS}, + {"unicode_getdefaultencoding",unicode_getdefaultencoding, METH_NOARGS}, + {"unicode_decode", unicode_decode, METH_VARARGS}, + {"unicode_asencodedstring", unicode_asencodedstring, METH_VARARGS}, + {"unicode_buildencodingmap", unicode_buildencodingmap, METH_O}, + {"unicode_decodeutf7", unicode_decodeutf7, METH_VARARGS}, + {"unicode_decodeutf7stateful",unicode_decodeutf7stateful, METH_VARARGS}, {"unicode_decodeutf8", unicode_decodeutf8, METH_VARARGS}, {"unicode_decodeutf8stateful",unicode_decodeutf8stateful, METH_VARARGS}, - {"unicode_getdefaultencoding",unicode_getdefaultencoding, METH_NOARGS}, + {"unicode_asutf8string", unicode_asutf8string, METH_O}, + {"unicode_decodeutf16", unicode_decodeutf16, METH_VARARGS}, + {"unicode_decodeutf16stateful",unicode_decodeutf16stateful, METH_VARARGS}, + {"unicode_asutf16string", unicode_asutf16string, METH_O}, + {"unicode_decodeutf32", unicode_decodeutf32, METH_VARARGS}, + {"unicode_decodeutf32stateful",unicode_decodeutf32stateful, METH_VARARGS}, + {"unicode_asutf32string", unicode_asutf32string, METH_O}, + {"unicode_decodeunicodeescape",unicode_decodeunicodeescape, METH_VARARGS}, + {"unicode_asunicodeescapestring",unicode_asunicodeescapestring,METH_O}, + {"unicode_decoderawunicodeescape",unicode_decoderawunicodeescape,METH_VARARGS}, + {"unicode_asrawunicodeescapestring",unicode_asrawunicodeescapestring,METH_O}, + {"unicode_decodelatin1", unicode_decodelatin1, METH_VARARGS}, + {"unicode_aslatin1string", unicode_aslatin1string, METH_O}, + {"unicode_decodeascii", unicode_decodeascii, METH_VARARGS}, + {"unicode_asasciistring", unicode_asasciistring, METH_O}, + {"unicode_decodecharmap", unicode_decodecharmap, METH_VARARGS}, + {"unicode_ascharmapstring", unicode_ascharmapstring, METH_VARARGS}, +#ifdef MS_WINDOWS + {"unicode_decodembcs", unicode_decodembcs, METH_VARARGS}, + {"unicode_decodembcsstateful",unicode_decodembcsstateful, METH_VARARGS}, + {"unicode_decodecodepagestateful",unicode_decodecodepagestateful,METH_VARARGS}, + {"unicode_asmbcsstring", unicode_asmbcsstring, METH_O}, + {"unicode_encodecodepage", unicode_encodecodepage, METH_VARARGS}, +#endif /* MS_WINDOWS */ + {"unicode_decodelocaleandsize",unicode_decodelocaleandsize, METH_VARARGS}, + {"unicode_decodelocale", unicode_decodelocale, METH_VARARGS}, + {"unicode_encodelocale", unicode_encodelocale, METH_VARARGS}, + {"unicode_decodefsdefault", unicode_decodefsdefault, METH_VARARGS}, + {"unicode_decodefsdefaultandsize",unicode_decodefsdefaultandsize,METH_VARARGS}, + {"unicode_encodefsdefault", unicode_encodefsdefault, METH_O}, {"unicode_concat", unicode_concat, METH_VARARGS}, {"unicode_splitlines", unicode_splitlines, METH_VARARGS}, {"unicode_split", unicode_split, METH_VARARGS}, From webhook-mailer at python.org Mon Jul 10 07:48:07 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Mon, 10 Jul 2023 11:48:07 -0000 Subject: [Python-checkins] GH-104787: use managed weakrefs in `_asyncio` (#106516) Message-ID: https://github.com/python/cpython/commit/8fb6edf479b2cf58d503945d17467055a5eaf455 commit: 8fb6edf479b2cf58d503945d17467055a5eaf455 branch: main author: Kumar Aditya committer: kumaraditya303 date: 2023-07-10T17:18:03+05:30 summary: GH-104787: use managed weakrefs in `_asyncio` (#106516) files: M Modules/_asynciomodule.c diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 3843f9c45d723..3b0550279fb49 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -120,7 +120,6 @@ typedef enum { PyObject *prefix##_result; \ PyObject *prefix##_source_tb; \ PyObject *prefix##_cancel_msg; \ - PyObject *prefix##_weakreflist; \ PyObject *prefix##_cancelled_exc; \ fut_state prefix##_state; \ /* These bitfields need to be at the end of the struct @@ -1502,11 +1501,6 @@ static PyMethodDef FutureType_methods[] = { {NULL, NULL} /* Sentinel */ }; -static PyMemberDef FutureType_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(FutureObj, fut_weakreflist), READONLY}, - {NULL}, -}; - #define FUTURE_COMMON_GETSETLIST \ {"_state", (getter)FutureObj_get_state, NULL, NULL}, \ {"_asyncio_future_blocking", (getter)FutureObj_get_blocking, \ @@ -1537,7 +1531,6 @@ static PyType_Slot Future_slots[] = { {Py_tp_clear, (inquiry)FutureObj_clear}, {Py_tp_iter, (getiterfunc)future_new_iter}, {Py_tp_methods, FutureType_methods}, - {Py_tp_members, FutureType_members}, {Py_tp_getset, FutureType_getsetlist}, {Py_tp_init, (initproc)_asyncio_Future___init__}, {Py_tp_new, PyType_GenericNew}, @@ -1552,7 +1545,8 @@ static PyType_Spec Future_spec = { .name = "_asyncio.Future", .basicsize = sizeof(FutureObj), .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE | - Py_TPFLAGS_IMMUTABLETYPE | Py_TPFLAGS_MANAGED_DICT), + Py_TPFLAGS_IMMUTABLETYPE | Py_TPFLAGS_MANAGED_DICT | + Py_TPFLAGS_MANAGED_WEAKREF), .slots = Future_slots, }; @@ -1569,9 +1563,7 @@ FutureObj_dealloc(PyObject *self) PyTypeObject *tp = Py_TYPE(fut); PyObject_GC_UnTrack(self); - if (fut->fut_weakreflist != NULL) { - PyObject_ClearWeakRefs(self); - } + PyObject_ClearWeakRefs(self); (void)FutureObj_clear(fut); tp->tp_free(fut); @@ -2642,11 +2634,6 @@ static PyMethodDef TaskType_methods[] = { {NULL, NULL} /* Sentinel */ }; -static PyMemberDef TaskType_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(TaskObj, task_weakreflist), READONLY}, - {NULL}, -}; - static PyGetSetDef TaskType_getsetlist[] = { FUTURE_COMMON_GETSETLIST {"_log_destroy_pending", (getter)TaskObj_get_log_destroy_pending, @@ -2665,7 +2652,6 @@ static PyType_Slot Task_slots[] = { {Py_tp_clear, (inquiry)TaskObj_clear}, {Py_tp_iter, (getiterfunc)future_new_iter}, {Py_tp_methods, TaskType_methods}, - {Py_tp_members, TaskType_members}, {Py_tp_getset, TaskType_getsetlist}, {Py_tp_init, (initproc)_asyncio_Task___init__}, {Py_tp_new, PyType_GenericNew}, @@ -2680,7 +2666,8 @@ static PyType_Spec Task_spec = { .name = "_asyncio.Task", .basicsize = sizeof(TaskObj), .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE | - Py_TPFLAGS_IMMUTABLETYPE | Py_TPFLAGS_MANAGED_DICT), + Py_TPFLAGS_IMMUTABLETYPE | Py_TPFLAGS_MANAGED_DICT | + Py_TPFLAGS_MANAGED_WEAKREF), .slots = Task_slots, }; @@ -2697,9 +2684,7 @@ TaskObj_dealloc(PyObject *self) PyTypeObject *tp = Py_TYPE(task); PyObject_GC_UnTrack(self); - if (task->task_weakreflist != NULL) { - PyObject_ClearWeakRefs(self); - } + PyObject_ClearWeakRefs(self); (void)TaskObj_clear(task); tp->tp_free(task); From webhook-mailer at python.org Mon Jul 10 07:51:17 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Mon, 10 Jul 2023 11:51:17 -0000 Subject: [Python-checkins] update release schedule for 3.13 (#106577) Message-ID: https://github.com/python/cpython/commit/abe4ca5ecf3abb6ad72ba18ed732a0bc3f158335 commit: abe4ca5ecf3abb6ad72ba18ed732a0bc3f158335 branch: main author: sunmy2019 <59365878+sunmy2019 at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-10T17:21:14+05:30 summary: update release schedule for 3.13 (#106577) files: M README.rst diff --git a/README.rst b/README.rst index 4a4c8ab65dcef..4ab26565a13e0 100644 --- a/README.rst +++ b/README.rst @@ -238,7 +238,7 @@ All current PEPs, as well as guidelines for submitting a new PEP, are listed at Release Schedule ---------------- -See :pep:`XXX` for Python 3.13 release details. +See :pep:`719` for Python 3.13 release details. Copyright and License Information From webhook-mailer at python.org Mon Jul 10 07:52:45 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Mon, 10 Jul 2023 11:52:45 -0000 Subject: [Python-checkins] Fix typo in datamodel.rst (#106587) Message-ID: https://github.com/python/cpython/commit/43389e4a3a15daaa2c4ea7059637e2fce3f38966 commit: 43389e4a3a15daaa2c4ea7059637e2fce3f38966 branch: main author: Riahiamirreza <54557628+Riahiamirreza at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-10T17:22:41+05:30 summary: Fix typo in datamodel.rst (#106587) files: M Doc/reference/datamodel.rst diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 7f5edbbcadca6..8a10a34347c2d 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1625,7 +1625,7 @@ access (use of, assignment to, or deletion of ``x.name``) for class instances. :meth:`__getattr__` and :meth:`__setattr__`.) This is done both for efficiency reasons and because otherwise :meth:`__getattr__` would have no way to access other attributes of the instance. Note that at least for instance variables, - you can fake total control by not inserting any values in the instance attribute + you can take total control by not inserting any values in the instance attribute dictionary (but instead inserting them in another object). See the :meth:`__getattribute__` method below for a way to actually get total control over attribute access. From webhook-mailer at python.org Mon Jul 10 07:57:28 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Mon, 10 Jul 2023 11:57:28 -0000 Subject: [Python-checkins] gh-106078: Move static variables initialized once to decimal module global state (#106475) Message-ID: https://github.com/python/cpython/commit/93846657a35726358ef6714c6631a9f862090b04 commit: 93846657a35726358ef6714c6631a9f862090b04 branch: main author: Charlie Zhao committer: kumaraditya303 date: 2023-07-10T17:27:25+05:30 summary: gh-106078: Move static variables initialized once to decimal module global state (#106475) files: M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/globals-to-fix.tsv diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 7e1809cfb98b2..89924b205f99e 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -39,6 +39,12 @@ #include "docstrings.h" +#ifdef EXTRA_FUNCTIONALITY + #define _PY_DEC_ROUND_GUARD MPD_ROUND_GUARD +#else + #define _PY_DEC_ROUND_GUARD (MPD_ROUND_GUARD-1) +#endif + struct PyDecContextObject; typedef struct { @@ -68,6 +74,13 @@ typedef struct { /* Basic and extended context templates */ PyObject *basic_context_template; PyObject *extended_context_template; + + PyObject *round_map[_PY_DEC_ROUND_GUARD]; + + /* Convert rationals for comparison */ + PyObject *Rational; + + PyObject *SignalTuple; } decimal_state; static decimal_state global_state; @@ -216,13 +229,6 @@ static const char *dec_signal_string[MPD_NUM_FLAGS] = { "Underflow", }; -#ifdef EXTRA_FUNCTIONALITY - #define _PY_DEC_ROUND_GUARD MPD_ROUND_GUARD -#else - #define _PY_DEC_ROUND_GUARD (MPD_ROUND_GUARD-1) -#endif -static PyObject *round_map[_PY_DEC_ROUND_GUARD]; - static const char *invalid_rounding_err = "valid values for rounding are:\n\ [ROUND_CEILING, ROUND_FLOOR, ROUND_UP, ROUND_DOWN,\n\ @@ -520,15 +526,16 @@ static int getround(PyObject *v) { int i; + decimal_state *state = GLOBAL_STATE(); if (PyUnicode_Check(v)) { for (i = 0; i < _PY_DEC_ROUND_GUARD; i++) { - if (v == round_map[i]) { + if (v == state->round_map[i]) { return i; } } for (i = 0; i < _PY_DEC_ROUND_GUARD; i++) { - if (PyUnicode_Compare(v, round_map[i]) == 0) { + if (PyUnicode_Compare(v, state->round_map[i]) == 0) { return i; } } @@ -561,11 +568,11 @@ signaldict_len(PyObject *self UNUSED) return SIGNAL_MAP_LEN; } -static PyObject *SignalTuple; static PyObject * signaldict_iter(PyObject *self UNUSED) { - return PyTuple_Type.tp_iter(SignalTuple); + decimal_state *state = GLOBAL_STATE(); + return PyTuple_Type.tp_iter(state->SignalTuple); } static PyObject * @@ -754,8 +761,9 @@ static PyObject * context_getround(PyObject *self, void *closure UNUSED) { int i = mpd_getround(CTX(self)); + decimal_state *state = GLOBAL_STATE(); - return Py_NewRef(round_map[i]); + return Py_NewRef(state->round_map[i]); } static PyObject * @@ -2987,8 +2995,6 @@ convert_op(int type_err, PyObject **conv, PyObject *v, PyObject *context) /* Implicit conversions to Decimal for comparison */ /******************************************************************************/ -/* Convert rationals for comparison */ -static PyObject *Rational = NULL; static PyObject * multiply_by_denominator(PyObject *v, PyObject *r, PyObject *context) { @@ -3117,7 +3123,7 @@ convert_op_cmp(PyObject **vcmp, PyObject **wcmp, PyObject *v, PyObject *w, } } else { - int is_rational = PyObject_IsInstance(w, Rational); + int is_rational = PyObject_IsInstance(w, state->Rational); if (is_rational < 0) { *wcmp = NULL; } @@ -5865,7 +5871,7 @@ PyInit__decimal(void) (PyObject *)state->PyDec_Type)); Py_CLEAR(obj); /* Rational is a global variable used for fraction comparisons. */ - ASSIGN_PTR(Rational, PyObject_GetAttrString(numbers, "Rational")); + ASSIGN_PTR(state->Rational, PyObject_GetAttrString(numbers, "Rational")); /* Done with numbers, Number */ Py_CLEAR(numbers); Py_CLEAR(Number); @@ -5912,7 +5918,7 @@ PyInit__decimal(void) CHECK_INT(PyModule_AddType(m, (PyTypeObject *)state->DecimalException)); /* Create signal tuple */ - ASSIGN_PTR(SignalTuple, PyTuple_New(SIGNAL_MAP_LEN)); + ASSIGN_PTR(state->SignalTuple, PyTuple_New(SIGNAL_MAP_LEN)); /* Add exceptions that correspond to IEEE signals */ for (i = SIGNAL_MAP_LEN-1; i >= 0; i--) { @@ -5953,7 +5959,7 @@ PyInit__decimal(void) CHECK_INT(PyModule_AddObject(m, cm->name, Py_NewRef(cm->ex))); /* add to signal tuple */ - PyTuple_SET_ITEM(SignalTuple, i, Py_NewRef(cm->ex)); + PyTuple_SET_ITEM(state->SignalTuple, i, Py_NewRef(cm->ex)); } /* @@ -6029,8 +6035,8 @@ PyInit__decimal(void) /* Init string constants */ for (i = 0; i < _PY_DEC_ROUND_GUARD; i++) { - ASSIGN_PTR(round_map[i], PyUnicode_InternFromString(mpd_round_string[i])); - CHECK_INT(PyModule_AddObject(m, mpd_round_string[i], Py_NewRef(round_map[i]))); + ASSIGN_PTR(state->round_map[i], PyUnicode_InternFromString(mpd_round_string[i])); + CHECK_INT(PyModule_AddObject(m, mpd_round_string[i], Py_NewRef(state->round_map[i]))); } /* Add specification version number */ @@ -6045,11 +6051,11 @@ PyInit__decimal(void) Py_CLEAR(obj); /* GCOV_NOT_REACHED */ Py_CLEAR(numbers); /* GCOV_NOT_REACHED */ Py_CLEAR(Number); /* GCOV_NOT_REACHED */ - Py_CLEAR(Rational); /* GCOV_NOT_REACHED */ + Py_CLEAR(state->Rational); /* GCOV_NOT_REACHED */ Py_CLEAR(collections); /* GCOV_NOT_REACHED */ Py_CLEAR(collections_abc); /* GCOV_NOT_REACHED */ Py_CLEAR(MutableMapping); /* GCOV_NOT_REACHED */ - Py_CLEAR(SignalTuple); /* GCOV_NOT_REACHED */ + Py_CLEAR(state->SignalTuple); /* GCOV_NOT_REACHED */ Py_CLEAR(state->DecimalTuple); /* GCOV_NOT_REACHED */ Py_CLEAR(state->default_context_template); /* GCOV_NOT_REACHED */ #ifndef WITH_DECIMAL_CONTEXTVAR diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 12f91726b2d2f..38b47d06e4a5f 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -422,9 +422,6 @@ Modules/_datetimemodule.c - us_per_day - Modules/_datetimemodule.c - us_per_week - Modules/_datetimemodule.c - seconds_per_day - Modules/_decimal/_decimal.c - global_state - -Modules/_decimal/_decimal.c - round_map - -Modules/_decimal/_decimal.c - Rational - -Modules/_decimal/_decimal.c - SignalTuple - ## state Modules/_asynciomodule.c - fi_freelist - From webhook-mailer at python.org Mon Jul 10 08:31:30 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Mon, 10 Jul 2023 12:31:30 -0000 Subject: [Python-checkins] GH-100288: regen cases after #105990 (#106589) Message-ID: https://github.com/python/cpython/commit/3f9bc86c5a1b29fd636a53bf4150acacf60284d8 commit: 3f9bc86c5a1b29fd636a53bf4150acacf60284d8 branch: main author: Kumar Aditya committer: kumaraditya303 date: 2023-07-10T12:31:26Z summary: GH-100288: regen cases after #105990 (#106589) files: M Python/executor_cases.c.h diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 0ef752249ccb7..eccb30348173a 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1642,7 +1642,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1815 "Python/bytecodes.c" + #line 1817 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1678,7 +1678,7 @@ */ #line 1680 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1849 "Python/bytecodes.c" + #line 1851 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -1689,7 +1689,7 @@ res = PyObject_GetAttr(owner, name); #line 1691 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1858 "Python/bytecodes.c" + #line 1860 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } #line 1696 "Python/executor_cases.c.h" @@ -1704,7 +1704,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2091 "Python/bytecodes.c" + #line 2093 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1720,7 +1720,7 @@ #line 1721 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2104 "Python/bytecodes.c" + #line 2106 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; if (oparg & 16) { int res_bool = PyObject_IsTrue(res); @@ -2237,7 +2237,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3537 "Python/bytecodes.c" + #line 3568 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2255,7 +2255,7 @@ #line 2256 "Python/executor_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3552 "Python/bytecodes.c" + #line 3583 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; #line 2261 "Python/executor_cases.c.h" STACK_SHRINK(1); From webhook-mailer at python.org Mon Jul 10 09:35:57 2023 From: webhook-mailer at python.org (cjw296) Date: Mon, 10 Jul 2023 13:35:57 -0000 Subject: [Python-checkins] GH-61215: threadingmock: Remove unused branch for `timeout` (#106591) Message-ID: https://github.com/python/cpython/commit/3e23fa71f43fb225ca29a931644d1100e2f4d6b8 commit: 3e23fa71f43fb225ca29a931644d1100e2f4d6b8 branch: main author: Mario Corchero committer: cjw296 date: 2023-07-10T07:35:54-06:00 summary: GH-61215: threadingmock: Remove unused branch for `timeout` (#106591) threadingmock: Remove unused branch for `timeout` This is no longer needed as the mock does not hold a "timeout" parameter, the timeout is stored in `_mock_wait_timeout`. files: M Lib/unittest/mock.py diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py index 7ef7e7180b31c..3ed54b3ba230e 100644 --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -3012,9 +3012,7 @@ class ThreadingMixin(Base): DEFAULT_TIMEOUT = None def _get_child_mock(self, /, **kw): - if "timeout" in kw: - kw["timeout"] = kw.pop("timeout") - elif isinstance(kw.get("parent"), ThreadingMixin): + if isinstance(kw.get("parent"), ThreadingMixin): kw["timeout"] = kw["parent"]._mock_wait_timeout elif isinstance(kw.get("_new_parent"), ThreadingMixin): kw["timeout"] = kw["_new_parent"]._mock_wait_timeout From webhook-mailer at python.org Mon Jul 10 12:41:07 2023 From: webhook-mailer at python.org (encukou) Date: Mon, 10 Jul 2023 16:41:07 -0000 Subject: [Python-checkins] gh-105227: Add PyType_GetDict() (GH-105747) Message-ID: https://github.com/python/cpython/commit/a840806d338805fe74a9de01081d30da7605a29f commit: a840806d338805fe74a9de01081d30da7605a29f branch: main author: Eric Snow committer: encukou date: 2023-07-10T18:41:02+02:00 summary: gh-105227: Add PyType_GetDict() (GH-105747) This compensates for static builtin types having `tp_dict` set to `NULL`. Co-authored-by: Petr Viktorin files: A Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Include/cpython/object.h M Modules/_testcapimodule.c M Objects/typeobject.c diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index c99c7ef93a45d..a5f333e2a31e0 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -50,6 +50,23 @@ Type Objects The return type is now ``unsigned long`` rather than ``long``. +.. c:function:: PyObject* PyType_GetDict(PyTypeObject* type) + + Return the type object's internal namespace, which is otherwise only + exposed via a read-only proxy (``cls.__dict__``). This is a + replacement for accessing :c:member:`~PyTypeObject.tp_dict` directly. + The returned dictionary must be treated as read-only. + + This function is meant for specific embedding and language-binding cases, + where direct access to the dict is necessary and indirect access + (e.g. via the proxy or :c:func:`PyObject_GetAttr`) isn't adequate. + + Extension modules should continue to use ``tp_dict``, + directly or indirectly, when setting up their own types. + + .. versionadded:: 3.12 + + .. c:function:: void PyType_Modified(PyTypeObject *type) Invalidate the internal lookup cache for the type and all of its diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 239c191457f51..7249cfe79c32e 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1717,7 +1717,19 @@ and :c:type:`PyType_Type` effectively act as defaults.) called; it may also be initialized to a dictionary containing initial attributes for the type. Once :c:func:`PyType_Ready` has initialized the type, extra attributes for the type may be added to this dictionary only if they don't - correspond to overloaded operations (like :meth:`__add__`). + correspond to overloaded operations (like :meth:`__add__`). Once + initialization for the type has finished, this field should be + treated as read-only. + + Some types may not store their dictionary in this slot. + Use :c:func:`PyType_GetDict` to retreive the dictionary for an arbitrary + type. + + .. versionchanged:: 3.12 + + Internals detail: For static builtin types, this is always ``NULL``. + Instead, the dict for such types is stored on ``PyInterpreterState``. + Use :c:func:`PyType_GetDict` to get the dict for an arbitrary type. **Inheritance:** diff --git a/Include/cpython/object.h b/Include/cpython/object.h index d681435c84545..ef9006431bee2 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -284,6 +284,7 @@ PyAPI_FUNC(PyTypeObject *) _PyType_CalculateMetaclass(PyTypeObject *, PyObject * PyAPI_FUNC(PyObject *) _PyType_GetDocFromInternalDoc(const char *, const char *); PyAPI_FUNC(PyObject *) _PyType_GetTextSignatureFromInternalDoc(const char *, const char *); PyAPI_FUNC(PyObject *) PyType_GetModuleByDef(PyTypeObject *, PyModuleDef *); +PyAPI_FUNC(PyObject *) PyType_GetDict(PyTypeObject *); PyAPI_FUNC(int) PyObject_Print(PyObject *, FILE *, int); PyAPI_FUNC(void) _Py_BreakPoint(void); diff --git a/Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst b/Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst new file mode 100644 index 0000000000000..846663621e868 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst @@ -0,0 +1,5 @@ +The new :c:func:`PyType_GetDict` provides the dictionary for the given type +object that is normally exposed by ``cls.__dict__``. Normally it's +sufficient to use :c:member:`~PyTypeObject.tp_dict`, but for the static +builtin types :c:member:`!tp_dict` is now always ``NULL``. :c:func:`!PyType_GetDict()` +provides the correct dict object instead. diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 50eaff9917fd1..dd2c9c72e5378 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -638,6 +638,30 @@ test_get_type_qualname(PyObject *self, PyObject *Py_UNUSED(ignored)) Py_RETURN_NONE; } +static PyObject * +test_get_type_dict(PyObject *self, PyObject *Py_UNUSED(ignored)) +{ + /* Test for PyType_GetDict */ + + // Assert ints have a `to_bytes` method + PyObject *long_dict = PyType_GetDict(&PyLong_Type); + assert(long_dict); + assert(PyDict_GetItemString(long_dict, "to_bytes")); // borrowed ref + Py_DECREF(long_dict); + + // Make a new type, add an attribute to it and assert it's there + PyObject *HeapTypeNameType = PyType_FromSpec(&HeapTypeNameType_Spec); + assert(HeapTypeNameType); + assert(PyObject_SetAttrString( + HeapTypeNameType, "new_attr", Py_NewRef(Py_None)) >= 0); + PyObject *type_dict = PyType_GetDict((PyTypeObject*)HeapTypeNameType); + assert(type_dict); + assert(PyDict_GetItemString(type_dict, "new_attr")); // borrowed ref + Py_DECREF(HeapTypeNameType); + Py_DECREF(type_dict); + Py_RETURN_NONE; +} + static PyObject * pyobject_repr_from_null(PyObject *self, PyObject *Py_UNUSED(ignored)) { @@ -3472,6 +3496,7 @@ static PyMethodDef TestMethods[] = { {"test_get_statictype_slots", test_get_statictype_slots, METH_NOARGS}, {"test_get_type_name", test_get_type_name, METH_NOARGS}, {"test_get_type_qualname", test_get_type_qualname, METH_NOARGS}, + {"test_get_type_dict", test_get_type_dict, METH_NOARGS}, {"_test_thread_state", test_thread_state, METH_VARARGS}, #ifndef MS_WINDOWS {"_spawn_pthread_waiter", spawn_pthread_waiter, METH_NOARGS}, diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 87519efef081c..5430924e69dee 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -237,6 +237,13 @@ _PyType_GetDict(PyTypeObject *self) return lookup_tp_dict(self); } +PyObject * +PyType_GetDict(PyTypeObject *self) +{ + PyObject *dict = lookup_tp_dict(self); + return _Py_XNewRef(dict); +} + static inline void set_tp_dict(PyTypeObject *self, PyObject *dict) { From webhook-mailer at python.org Mon Jul 10 13:12:19 2023 From: webhook-mailer at python.org (encukou) Date: Mon, 10 Jul 2023 17:12:19 -0000 Subject: [Python-checkins] [3.12] gh-105227: Add PyType_GetDict() (GH-105747) (#106600) Message-ID: https://github.com/python/cpython/commit/41057b2ffeb5a8cb492e37e5503ab76ed1a3082d commit: 41057b2ffeb5a8cb492e37e5503ab76ed1a3082d branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: encukou date: 2023-07-10T17:12:15Z summary: [3.12] gh-105227: Add PyType_GetDict() (GH-105747) (#106600) gh-105227: Add PyType_GetDict() (GH-105747) This compensates for static builtin types having `tp_dict` set to `NULL`. (cherry picked from commit a840806d338805fe74a9de01081d30da7605a29f) Co-authored-by: Eric Snow Co-authored-by: Petr Viktorin files: A Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Include/cpython/object.h M Modules/_testcapimodule.c M Objects/typeobject.c diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index c99c7ef93a45d..a5f333e2a31e0 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -50,6 +50,23 @@ Type Objects The return type is now ``unsigned long`` rather than ``long``. +.. c:function:: PyObject* PyType_GetDict(PyTypeObject* type) + + Return the type object's internal namespace, which is otherwise only + exposed via a read-only proxy (``cls.__dict__``). This is a + replacement for accessing :c:member:`~PyTypeObject.tp_dict` directly. + The returned dictionary must be treated as read-only. + + This function is meant for specific embedding and language-binding cases, + where direct access to the dict is necessary and indirect access + (e.g. via the proxy or :c:func:`PyObject_GetAttr`) isn't adequate. + + Extension modules should continue to use ``tp_dict``, + directly or indirectly, when setting up their own types. + + .. versionadded:: 3.12 + + .. c:function:: void PyType_Modified(PyTypeObject *type) Invalidate the internal lookup cache for the type and all of its diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 239c191457f51..7249cfe79c32e 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1717,7 +1717,19 @@ and :c:type:`PyType_Type` effectively act as defaults.) called; it may also be initialized to a dictionary containing initial attributes for the type. Once :c:func:`PyType_Ready` has initialized the type, extra attributes for the type may be added to this dictionary only if they don't - correspond to overloaded operations (like :meth:`__add__`). + correspond to overloaded operations (like :meth:`__add__`). Once + initialization for the type has finished, this field should be + treated as read-only. + + Some types may not store their dictionary in this slot. + Use :c:func:`PyType_GetDict` to retreive the dictionary for an arbitrary + type. + + .. versionchanged:: 3.12 + + Internals detail: For static builtin types, this is always ``NULL``. + Instead, the dict for such types is stored on ``PyInterpreterState``. + Use :c:func:`PyType_GetDict` to get the dict for an arbitrary type. **Inheritance:** diff --git a/Include/cpython/object.h b/Include/cpython/object.h index d8eff691039d2..c5d0851a4b11b 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -283,6 +283,7 @@ PyAPI_FUNC(PyTypeObject *) _PyType_CalculateMetaclass(PyTypeObject *, PyObject * PyAPI_FUNC(PyObject *) _PyType_GetDocFromInternalDoc(const char *, const char *); PyAPI_FUNC(PyObject *) _PyType_GetTextSignatureFromInternalDoc(const char *, const char *); PyAPI_FUNC(PyObject *) PyType_GetModuleByDef(PyTypeObject *, PyModuleDef *); +PyAPI_FUNC(PyObject *) PyType_GetDict(PyTypeObject *); PyAPI_FUNC(int) PyObject_Print(PyObject *, FILE *, int); PyAPI_FUNC(void) _Py_BreakPoint(void); diff --git a/Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst b/Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst new file mode 100644 index 0000000000000..846663621e868 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst @@ -0,0 +1,5 @@ +The new :c:func:`PyType_GetDict` provides the dictionary for the given type +object that is normally exposed by ``cls.__dict__``. Normally it's +sufficient to use :c:member:`~PyTypeObject.tp_dict`, but for the static +builtin types :c:member:`!tp_dict` is now always ``NULL``. :c:func:`!PyType_GetDict()` +provides the correct dict object instead. diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 04a880021147e..6523734131c2d 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -640,6 +640,30 @@ test_get_type_qualname(PyObject *self, PyObject *Py_UNUSED(ignored)) Py_RETURN_NONE; } +static PyObject * +test_get_type_dict(PyObject *self, PyObject *Py_UNUSED(ignored)) +{ + /* Test for PyType_GetDict */ + + // Assert ints have a `to_bytes` method + PyObject *long_dict = PyType_GetDict(&PyLong_Type); + assert(long_dict); + assert(PyDict_GetItemString(long_dict, "to_bytes")); // borrowed ref + Py_DECREF(long_dict); + + // Make a new type, add an attribute to it and assert it's there + PyObject *HeapTypeNameType = PyType_FromSpec(&HeapTypeNameType_Spec); + assert(HeapTypeNameType); + assert(PyObject_SetAttrString( + HeapTypeNameType, "new_attr", Py_NewRef(Py_None)) >= 0); + PyObject *type_dict = PyType_GetDict((PyTypeObject*)HeapTypeNameType); + assert(type_dict); + assert(PyDict_GetItemString(type_dict, "new_attr")); // borrowed ref + Py_DECREF(HeapTypeNameType); + Py_DECREF(type_dict); + Py_RETURN_NONE; +} + static PyObject * pyobject_repr_from_null(PyObject *self, PyObject *Py_UNUSED(ignored)) { @@ -3347,6 +3371,7 @@ static PyMethodDef TestMethods[] = { {"test_get_statictype_slots", test_get_statictype_slots, METH_NOARGS}, {"test_get_type_name", test_get_type_name, METH_NOARGS}, {"test_get_type_qualname", test_get_type_qualname, METH_NOARGS}, + {"test_get_type_dict", test_get_type_dict, METH_NOARGS}, {"_test_thread_state", test_thread_state, METH_VARARGS}, #ifndef MS_WINDOWS {"_spawn_pthread_waiter", spawn_pthread_waiter, METH_NOARGS}, diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 966471e9341a8..6662379515f95 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -238,6 +238,13 @@ _PyType_GetDict(PyTypeObject *self) return lookup_tp_dict(self); } +PyObject * +PyType_GetDict(PyTypeObject *self) +{ + PyObject *dict = lookup_tp_dict(self); + return _Py_XNewRef(dict); +} + static inline void set_tp_dict(PyTypeObject *self, PyObject *dict) { From webhook-mailer at python.org Mon Jul 10 13:59:02 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Mon, 10 Jul 2023 17:59:02 -0000 Subject: [Python-checkins] [3.12] gh-99593: Add tests for Unicode C API (part 3) (GH-104728) (GH-106595) Message-ID: https://github.com/python/cpython/commit/2da967ea14a49d1ca3e2d22d83ce9f6ffd5e6186 commit: 2da967ea14a49d1ca3e2d22d83ce9f6ffd5e6186 branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-10T20:58:58+03:00 summary: [3.12] gh-99593: Add tests for Unicode C API (part 3) (GH-104728) (GH-106595) Add tests for codecs. (cherry picked from commit 51ea664d18938645521bdd128a3c55f9c197644c) files: M Lib/test/test_capi/test_codecs.py M Modules/_testcapi/unicode.c diff --git a/Lib/test/test_capi/test_codecs.py b/Lib/test/test_capi/test_codecs.py index e46726192aa05..682c56979c6df 100644 --- a/Lib/test/test_capi/test_codecs.py +++ b/Lib/test/test_capi/test_codecs.py @@ -1,10 +1,95 @@ import unittest +import sys from test.support import import_helper _testcapi = import_helper.import_module('_testcapi') +NULL = None + class CAPITest(unittest.TestCase): + # TODO: Test the following functions: + # + # PyUnicode_BuildEncodingMap + # PyUnicode_FSConverter + # PyUnicode_FSDecoder + # PyUnicode_DecodeMBCS + # PyUnicode_DecodeMBCSStateful + # PyUnicode_DecodeCodePageStateful + # PyUnicode_AsMBCSString + # PyUnicode_EncodeCodePage + # PyUnicode_DecodeLocaleAndSize + # PyUnicode_DecodeLocale + # PyUnicode_EncodeLocale + # PyUnicode_DecodeFSDefault + # PyUnicode_DecodeFSDefaultAndSize + # PyUnicode_EncodeFSDefault + + def test_fromencodedobject(self): + """Test PyUnicode_FromEncodedObject()""" + fromencodedobject = _testcapi.unicode_fromencodedobject + + self.assertEqual(fromencodedobject(b'abc', NULL), 'abc') + self.assertEqual(fromencodedobject(b'abc', 'ascii'), 'abc') + b = b'a\xc2\xa1\xe4\xbd\xa0\xf0\x9f\x98\x80' + s = 'a\xa1\u4f60\U0001f600' + self.assertEqual(fromencodedobject(b, NULL), s) + self.assertEqual(fromencodedobject(b, 'utf-8'), s) + self.assertEqual(fromencodedobject(b, 'latin1'), b.decode('latin1')) + self.assertRaises(UnicodeDecodeError, fromencodedobject, b, 'ascii') + self.assertEqual(fromencodedobject(b, 'ascii', 'replace'), + 'a' + '\ufffd'*9) + self.assertEqual(fromencodedobject(bytearray(b), NULL), s) + self.assertEqual(fromencodedobject(bytearray(b), 'utf-8'), s) + self.assertRaises(LookupError, fromencodedobject, b'abc', 'foo') + self.assertRaises(LookupError, fromencodedobject, b, 'ascii', 'foo') + self.assertRaises(TypeError, fromencodedobject, 'abc', NULL) + self.assertRaises(TypeError, fromencodedobject, 'abc', 'ascii') + self.assertRaises(TypeError, fromencodedobject, [], NULL) + self.assertRaises(TypeError, fromencodedobject, [], 'ascii') + self.assertRaises(SystemError, fromencodedobject, NULL, NULL) + self.assertRaises(SystemError, fromencodedobject, NULL, 'ascii') + + def test_decode(self): + """Test PyUnicode_Decode()""" + decode = _testcapi.unicode_decode + + self.assertEqual(decode(b'[\xe2\x82\xac]', 'utf-8'), '[\u20ac]') + self.assertEqual(decode(b'[\xa4]', 'iso8859-15'), '[\u20ac]') + self.assertEqual(decode(b'[\xa4]', 'iso8859-15', 'strict'), '[\u20ac]') + self.assertRaises(UnicodeDecodeError, decode, b'[\xa4]', 'utf-8') + self.assertEqual(decode(b'[\xa4]', 'utf-8', 'replace'), '[\ufffd]') + + self.assertEqual(decode(b'[\xe2\x82\xac]', NULL), '[\u20ac]') + self.assertEqual(decode(b'[\xa4]', NULL, 'replace'), '[\ufffd]') + + self.assertRaises(LookupError, decode, b'\xa4', 'foo') + self.assertRaises(LookupError, decode, b'\xa4', 'utf-8', 'foo') + # TODO: Test PyUnicode_Decode() with NULL as data and + # negative size. + + def test_asencodedstring(self): + """Test PyUnicode_AsEncodedString()""" + asencodedstring = _testcapi.unicode_asencodedstring + + self.assertEqual(asencodedstring('abc', NULL), b'abc') + self.assertEqual(asencodedstring('abc', 'ascii'), b'abc') + s = 'a\xa1\u4f60\U0001f600' + b = b'a\xc2\xa1\xe4\xbd\xa0\xf0\x9f\x98\x80' + self.assertEqual(asencodedstring(s, NULL), b) + self.assertEqual(asencodedstring(s, 'utf-8'), b) + self.assertEqual(asencodedstring('\xa1\xa2', 'latin1'), b'\xa1\xa2') + self.assertRaises(UnicodeEncodeError, asencodedstring, '\xa1\xa2', 'ascii') + self.assertEqual(asencodedstring(s, 'ascii', 'replace'), b'a???') + + self.assertRaises(LookupError, asencodedstring, 'abc', 'foo') + self.assertRaises(LookupError, asencodedstring, s, 'ascii', 'foo') + self.assertRaises(TypeError, asencodedstring, b'abc', NULL) + self.assertRaises(TypeError, asencodedstring, b'abc', 'ascii') + self.assertRaises(TypeError, asencodedstring, [], NULL) + self.assertRaises(TypeError, asencodedstring, [], 'ascii') + # CRASHES asencodedstring(NULL, NULL) + # CRASHES asencodedstring(NULL, 'ascii') def test_decodeutf8(self): """Test PyUnicode_DecodeUTF8()""" @@ -49,6 +134,387 @@ def test_decodeutf8stateful(self): # TODO: Test PyUnicode_DecodeUTF8Stateful() with NULL as the address of # "consumed". + def test_asutf8string(self): + """Test PyUnicode_AsUTF8String()""" + asutf8string = _testcapi.unicode_asutf8string + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + self.assertEqual(asutf8string(s), s.encode('utf-8')) + + self.assertRaises(UnicodeEncodeError, asutf8string, '\ud8ff') + self.assertRaises(TypeError, asutf8string, b'abc') + self.assertRaises(TypeError, asutf8string, []) + # CRASHES asutf8string(NULL) + + def test_decodeutf16(self): + """Test PyUnicode_DecodeUTF16()""" + decodeutf16 = _testcapi.unicode_decodeutf16 + + naturalbyteorder = -1 if sys.byteorder == 'little' else 1 + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-16') + self.assertEqual(decodeutf16(0, b), (naturalbyteorder, s)) + b = s.encode('utf-16le') + self.assertEqual(decodeutf16(-1, b), (-1, s)) + self.assertEqual(decodeutf16(0, b'\xff\xfe'+b), (-1, s)) + b = s.encode('utf-16be') + self.assertEqual(decodeutf16(1, b), (1, s)) + self.assertEqual(decodeutf16(0, b'\xfe\xff'+b), (1, s)) + + self.assertRaises(UnicodeDecodeError, decodeutf16, -1, b'a') + self.assertRaises(UnicodeDecodeError, decodeutf16, 1, b'a') + self.assertRaises(UnicodeDecodeError, decodeutf16, 0, b'\xff\xfea') + self.assertRaises(UnicodeDecodeError, decodeutf16, 0, b'\xfe\xffa') + + self.assertRaises(UnicodeDecodeError, decodeutf16, -1, b'\x00\xde') + self.assertRaises(UnicodeDecodeError, decodeutf16, 1, b'\xde\x00') + self.assertRaises(UnicodeDecodeError, decodeutf16, 0, b'\xde\xde') + self.assertEqual(decodeutf16(-1, b'\x00\xde', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf16(1, b'\xde\x00', 'replace'), (1, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xde\xde', 'replace'), (0, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xff\xfe\x00\xde', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xfe\xff\xde\x00', 'replace'), (1, '\ufffd')) + + self.assertRaises(UnicodeDecodeError, decodeutf16, -1, b'\x3d\xd8') + self.assertRaises(UnicodeDecodeError, decodeutf16, 1, b'\xd8\x3d') + self.assertRaises(UnicodeDecodeError, decodeutf16, 0, b'\xd8\xd8') + self.assertEqual(decodeutf16(-1, b'\x3d\xd8', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf16(1, b'\xd8\x3d', 'replace'), (1, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xd8\xd8', 'replace'), (0, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xff\xfe\x3d\xd8', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf16(0, b'\xfe\xff\xd8\x3d', 'replace'), (1, '\ufffd')) + + self.assertRaises(LookupError, decodeutf16, -1, b'\x00\xde', 'foo') + self.assertRaises(LookupError, decodeutf16, 1, b'\xde\x00', 'foo') + self.assertRaises(LookupError, decodeutf16, 0, b'\xde\xde', 'foo') + # TODO: Test PyUnicode_DecodeUTF16() with NULL as data and + # negative size. + + def test_decodeutf16stateful(self): + """Test PyUnicode_DecodeUTF16Stateful()""" + decodeutf16stateful = _testcapi.unicode_decodeutf16stateful + + naturalbyteorder = -1 if sys.byteorder == 'little' else 1 + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-16') + self.assertEqual(decodeutf16stateful(0, b), (naturalbyteorder, s, len(b))) + b = s.encode('utf-16le') + self.assertEqual(decodeutf16stateful(-1, b), (-1, s, len(b))) + self.assertEqual(decodeutf16stateful(0, b'\xff\xfe'+b), (-1, s, len(b)+2)) + b = s.encode('utf-16be') + self.assertEqual(decodeutf16stateful(1, b), (1, s, len(b))) + self.assertEqual(decodeutf16stateful(0, b'\xfe\xff'+b), (1, s, len(b)+2)) + + self.assertEqual(decodeutf16stateful(-1, b'\x61\x00\x3d'), (-1, 'a', 2)) + self.assertEqual(decodeutf16stateful(-1, b'\x61\x00\x3d\xd8'), (-1, 'a', 2)) + self.assertEqual(decodeutf16stateful(-1, b'\x61\x00\x3d\xd8\x00'), (-1, 'a', 2)) + self.assertEqual(decodeutf16stateful(1, b'\x00\x61\xd8'), (1, 'a', 2)) + self.assertEqual(decodeutf16stateful(1, b'\x00\x61\xd8\x3d'), (1, 'a', 2)) + self.assertEqual(decodeutf16stateful(1, b'\x00\x61\xd8\x3d\xde'), (1, 'a', 2)) + self.assertEqual(decodeutf16stateful(0, b'\xff\xfe\x61\x00\x3d\xd8\x00'), (-1, 'a', 4)) + self.assertEqual(decodeutf16stateful(0, b'\xfe\xff\x00\x61\xd8\x3d\xde'), (1, 'a', 4)) + + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, -1, b'\x00\xde') + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, 1, b'\xde\x00') + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, 0, b'\xde\xde') + self.assertEqual(decodeutf16stateful(-1, b'\x00\xde', 'replace'), (-1, '\ufffd', 2)) + self.assertEqual(decodeutf16stateful(1, b'\xde\x00', 'replace'), (1, '\ufffd', 2)) + self.assertEqual(decodeutf16stateful(0, b'\xde\xde', 'replace'), (0, '\ufffd', 2)) + self.assertEqual(decodeutf16stateful(0, b'\xff\xfe\x00\xde', 'replace'), (-1, '\ufffd', 4)) + self.assertEqual(decodeutf16stateful(0, b'\xfe\xff\xde\x00', 'replace'), (1, '\ufffd', 4)) + + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, -1, b'\x3d\xd8\x61\x00') + self.assertEqual(decodeutf16stateful(-1, b'\x3d\xd8\x61\x00', 'replace'), (-1, '\ufffda', 4)) + self.assertRaises(UnicodeDecodeError, decodeutf16stateful, 1, b'\xd8\x3d\x00\x61') + self.assertEqual(decodeutf16stateful(1, b'\xd8\x3d\x00\x61', 'replace'), (1, '\ufffda', 4)) + + self.assertRaises(LookupError, decodeutf16stateful, -1, b'\x00\xde', 'foo') + self.assertRaises(LookupError, decodeutf16stateful, 1, b'\xde\x00', 'foo') + self.assertRaises(LookupError, decodeutf16stateful, 0, b'\xde\xde', 'foo') + # TODO: Test PyUnicode_DecodeUTF16Stateful() with NULL as data and + # negative size. + # TODO: Test PyUnicode_DecodeUTF16Stateful() with NULL as the address of + # "consumed". + + def test_asutf16string(self): + """Test PyUnicode_AsUTF16String()""" + asutf16string = _testcapi.unicode_asutf16string + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + self.assertEqual(asutf16string(s), s.encode('utf-16')) + + self.assertRaises(UnicodeEncodeError, asutf16string, '\ud8ff') + self.assertRaises(TypeError, asutf16string, b'abc') + self.assertRaises(TypeError, asutf16string, []) + # CRASHES asutf16string(NULL) + + def test_decodeutf32(self): + """Test PyUnicode_DecodeUTF8()""" + decodeutf32 = _testcapi.unicode_decodeutf32 + + naturalbyteorder = -1 if sys.byteorder == 'little' else 1 + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-32') + self.assertEqual(decodeutf32(0, b), (naturalbyteorder, s)) + b = s.encode('utf-32le') + self.assertEqual(decodeutf32(-1, b), (-1, s)) + self.assertEqual(decodeutf32(0, b'\xff\xfe\x00\x00'+b), (-1, s)) + b = s.encode('utf-32be') + self.assertEqual(decodeutf32(1, b), (1, s)) + self.assertEqual(decodeutf32(0, b'\x00\x00\xfe\xff'+b), (1, s)) + + self.assertRaises(UnicodeDecodeError, decodeutf32, -1, b'\x61\x00\x00\x00\x00') + self.assertRaises(UnicodeDecodeError, decodeutf32, 1, b'\x00\x00\x00\x61\x00') + self.assertRaises(UnicodeDecodeError, decodeutf32, 0, b'\xff\xfe\x00\x00\x61\x00\x00\x00\x00') + self.assertRaises(UnicodeDecodeError, decodeutf32, 0, b'\x00\x00\xfe\xff\x00\x00\x00\x61\x00') + + self.assertRaises(UnicodeDecodeError, decodeutf32, -1, b'\xff\xff\xff\xff') + self.assertRaises(UnicodeDecodeError, decodeutf32, 1, b'\xff\xff\xff\xff') + self.assertRaises(UnicodeDecodeError, decodeutf32, 0, b'\xff\xff\xff\xff') + self.assertEqual(decodeutf32(-1, b'\xff\xff\xff\xff', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf32(1, b'\xff\xff\xff\xff', 'replace'), (1, '\ufffd')) + self.assertEqual(decodeutf32(0, b'\xff\xff\xff\xff', 'replace'), (0, '\ufffd')) + self.assertEqual(decodeutf32(0, b'\xff\xfe\x00\x00\xff\xff\xff\xff', 'replace'), (-1, '\ufffd')) + self.assertEqual(decodeutf32(0, b'\x00\x00\xfe\xff\xff\xff\xff\xff', 'replace'), (1, '\ufffd')) + + self.assertRaises(UnicodeDecodeError, decodeutf32, -1, b'\x3d\xd8\x00\x00') + self.assertEqual(decodeutf32(-1, b'\x3d\xd8\x00\x00', 'replace'), (-1, '\ufffd')) + self.assertRaises(UnicodeDecodeError, decodeutf32, 1, b'\x00\x00\xd8\x3d') + self.assertEqual(decodeutf32(1, b'\x00\x00\xd8\x3d', 'replace'), (1, '\ufffd')) + + self.assertRaises(LookupError, decodeutf32, -1, b'\xff\xff\xff\xff', 'foo') + self.assertRaises(LookupError, decodeutf32, 1, b'\xff\xff\xff\xff', 'foo') + self.assertRaises(LookupError, decodeutf32, 0, b'\xff\xff\xff\xff', 'foo') + # TODO: Test PyUnicode_DecodeUTF32() with NULL as data and + # negative size. + + def test_decodeutf32stateful(self): + """Test PyUnicode_DecodeUTF32Stateful()""" + decodeutf32stateful = _testcapi.unicode_decodeutf32stateful + + naturalbyteorder = -1 if sys.byteorder == 'little' else 1 + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-32') + self.assertEqual(decodeutf32stateful(0, b), (naturalbyteorder, s, len(b))) + b = s.encode('utf-32le') + self.assertEqual(decodeutf32stateful(-1, b), (-1, s, len(b))) + self.assertEqual(decodeutf32stateful(0, b'\xff\xfe\x00\x00'+b), (-1, s, len(b)+4)) + b = s.encode('utf-32be') + self.assertEqual(decodeutf32stateful(1, b), (1, s, len(b))) + self.assertEqual(decodeutf32stateful(0, b'\x00\x00\xfe\xff'+b), (1, s, len(b)+4)) + + self.assertEqual(decodeutf32stateful(-1, b'\x61\x00\x00\x00\x00'), (-1, 'a', 4)) + self.assertEqual(decodeutf32stateful(-1, b'\x61\x00\x00\x00\x00\xf6'), (-1, 'a', 4)) + self.assertEqual(decodeutf32stateful(-1, b'\x61\x00\x00\x00\x00\xf6\x01'), (-1, 'a', 4)) + self.assertEqual(decodeutf32stateful(1, b'\x00\x00\x00\x61\x00'), (1, 'a', 4)) + self.assertEqual(decodeutf32stateful(1, b'\x00\x00\x00\x61\x00\x01'), (1, 'a', 4)) + self.assertEqual(decodeutf32stateful(1, b'\x00\x00\x00\x61\x00\x01\xf6'), (1, 'a', 4)) + self.assertEqual(decodeutf32stateful(0, b'\xff\xfe\x00\x00\x61\x00\x00\x00\x00\xf6\x01'), (-1, 'a', 8)) + self.assertEqual(decodeutf32stateful(0, b'\x00\x00\xfe\xff\x00\x00\x00\x61\x00\x01\xf6'), (1, 'a', 8)) + + for b in b'\xff', b'\xff\xff', b'\xff\xff\xff': + self.assertEqual(decodeutf32stateful(-1, b), (-1, '', 0)) + self.assertEqual(decodeutf32stateful(1, b), (1, '', 0)) + self.assertEqual(decodeutf32stateful(0, b), (0, '', 0)) + self.assertEqual(decodeutf32stateful(0, b'\xff\xfe\x00\x00'+b), (-1, '', 4)) + self.assertEqual(decodeutf32stateful(0, b'\x00\x00\xfe\xff'+b), (1, '', 4)) + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, -1, b'\xff\xff\xff\xff') + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, 1, b'\xff\xff\xff\xff') + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, 0, b'\xff\xff\xff\xff') + self.assertEqual(decodeutf32stateful(-1, b'\xff\xff\xff\xff', 'replace'), (-1, '\ufffd', 4)) + self.assertEqual(decodeutf32stateful(1, b'\xff\xff\xff\xff', 'replace'), (1, '\ufffd', 4)) + self.assertEqual(decodeutf32stateful(0, b'\xff\xff\xff\xff', 'replace'), (0, '\ufffd', 4)) + self.assertEqual(decodeutf32stateful(0, b'\xff\xfe\x00\x00\xff\xff\xff\xff', 'replace'), (-1, '\ufffd', 8)) + self.assertEqual(decodeutf32stateful(0, b'\x00\x00\xfe\xff\xff\xff\xff\xff', 'replace'), (1, '\ufffd', 8)) + + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, -1, b'\x3d\xd8\x00\x00') + self.assertEqual(decodeutf32stateful(-1, b'\x3d\xd8\x00\x00', 'replace'), (-1, '\ufffd', 4)) + self.assertRaises(UnicodeDecodeError, decodeutf32stateful, 1, b'\x00\x00\xd8\x3d') + self.assertEqual(decodeutf32stateful(1, b'\x00\x00\xd8\x3d', 'replace'), (1, '\ufffd', 4)) + + self.assertRaises(LookupError, decodeutf32stateful, -1, b'\xff\xff\xff\xff', 'foo') + self.assertRaises(LookupError, decodeutf32stateful, 1, b'\xff\xff\xff\xff', 'foo') + self.assertRaises(LookupError, decodeutf32stateful, 0, b'\xff\xff\xff\xff', 'foo') + # TODO: Test PyUnicode_DecodeUTF32Stateful() with NULL as data and + # negative size. + # TODO: Test PyUnicode_DecodeUTF32Stateful() with NULL as the address of + # "consumed". + + def test_asutf32string(self): + """Test PyUnicode_AsUTF32String()""" + asutf32string = _testcapi.unicode_asutf32string + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + self.assertEqual(asutf32string(s), s.encode('utf-32')) + + self.assertRaises(UnicodeEncodeError, asutf32string, '\ud8ff') + self.assertRaises(TypeError, asutf32string, b'abc') + self.assertRaises(TypeError, asutf32string, []) + # CRASHES asutf32string(NULL) + + def test_decodelatin1(self): + """Test PyUnicode_DecodeLatin1()""" + decodelatin1 = _testcapi.unicode_decodelatin1 + + self.assertEqual(decodelatin1(b'abc'), 'abc') + self.assertEqual(decodelatin1(b'abc', 'strict'), 'abc') + self.assertEqual(decodelatin1(b'\xa1\xa2'), '\xa1\xa2') + self.assertEqual(decodelatin1(b'\xa1\xa2', 'strict'), '\xa1\xa2') + # TODO: Test PyUnicode_DecodeLatin1() with NULL as data and + # negative size. + + def test_aslatin1string(self): + """Test PyUnicode_AsLatin1String()""" + aslatin1string = _testcapi.unicode_aslatin1string + + self.assertEqual(aslatin1string('abc'), b'abc') + self.assertEqual(aslatin1string('\xa1\xa2'), b'\xa1\xa2') + + self.assertRaises(UnicodeEncodeError, aslatin1string, '\u4f60') + self.assertRaises(TypeError, aslatin1string, b'abc') + self.assertRaises(TypeError, aslatin1string, []) + # CRASHES aslatin1string(NULL) + + def test_decodeascii(self): + """Test PyUnicode_DecodeASCII()""" + decodeascii = _testcapi.unicode_decodeascii + + self.assertEqual(decodeascii(b'abc'), 'abc') + self.assertEqual(decodeascii(b'abc', 'strict'), 'abc') + + self.assertRaises(UnicodeDecodeError, decodeascii, b'\xff') + self.assertEqual(decodeascii(b'a\xff', 'replace'), 'a\ufffd') + self.assertEqual(decodeascii(b'a\xffb', 'replace'), 'a\ufffdb') + + self.assertRaises(LookupError, decodeascii, b'a\xff', 'foo') + # TODO: Test PyUnicode_DecodeASCII() with NULL as data and + # negative size. + + def test_asasciistring(self): + """Test PyUnicode_AsASCIIString()""" + asasciistring = _testcapi.unicode_asasciistring + + self.assertEqual(asasciistring('abc'), b'abc') + + self.assertRaises(UnicodeEncodeError, asasciistring, '\x80') + self.assertRaises(TypeError, asasciistring, b'abc') + self.assertRaises(TypeError, asasciistring, []) + # CRASHES asasciistring(NULL) + + def test_decodecharmap(self): + """Test PyUnicode_DecodeCharmap()""" + decodecharmap = _testcapi.unicode_decodecharmap + + self.assertEqual(decodecharmap(b'\3\0\7', {0: 'a', 3: 'b', 7: 'c'}), 'bac') + self.assertEqual(decodecharmap(b'\1\0\2', ['a', 'b', 'c']), 'bac') + self.assertEqual(decodecharmap(b'\1\0\2', 'abc'), 'bac') + self.assertEqual(decodecharmap(b'\1\0\2', ['\xa1', '\xa2', '\xa3']), '\xa2\xa1\xa3') + self.assertEqual(decodecharmap(b'\1\0\2', ['\u4f60', '\u597d', '\u4e16']), '\u597d\u4f60\u4e16') + self.assertEqual(decodecharmap(b'\1\0\2', ['\U0001f600', '\U0001f601', '\U0001f602']), '\U0001f601\U0001f600\U0001f602') + + self.assertEqual(decodecharmap(b'\1\0\2', [97, 98, 99]), 'bac') + self.assertEqual(decodecharmap(b'\1\0\2', ['', 'b', 'cd']), 'bcd') + + self.assertRaises(UnicodeDecodeError, decodecharmap, b'\0', {}) + self.assertRaises(UnicodeDecodeError, decodecharmap, b'\0', {0: None}) + self.assertEqual(decodecharmap(b'\1\0\2', [None, 'b', 'c'], 'replace'), 'b\ufffdc') + self.assertEqual(decodecharmap(b'\1\0\2\xff', NULL), '\1\0\2\xff') + self.assertRaises(TypeError, decodecharmap, b'\0', 42) + + # TODO: Test PyUnicode_DecodeCharmap() with NULL as data and + # negative size. + + def test_ascharmapstring(self): + """Test PyUnicode_AsCharmapString()""" + ascharmapstring = _testcapi.unicode_ascharmapstring + + self.assertEqual(ascharmapstring('abc', {97: 3, 98: 0, 99: 7}), b'\3\0\7') + self.assertEqual(ascharmapstring('\xa1\xa2\xa3', {0xa1: 3, 0xa2: 0, 0xa3: 7}), b'\3\0\7') + self.assertEqual(ascharmapstring('\u4f60\u597d\u4e16', {0x4f60: 3, 0x597d: 0, 0x4e16: 7}), b'\3\0\7') + self.assertEqual(ascharmapstring('\U0001f600\U0001f601\U0001f602', {0x1f600: 3, 0x1f601: 0, 0x1f602: 7}), b'\3\0\7') + self.assertEqual(ascharmapstring('abc', {97: 3, 98: b'', 99: b'spam'}), b'\3spam') + + self.assertRaises(UnicodeEncodeError, ascharmapstring, 'a', {}) + self.assertRaises(UnicodeEncodeError, ascharmapstring, 'a', {97: None}) + self.assertRaises(TypeError, ascharmapstring, b'a', {}) + self.assertRaises(TypeError, ascharmapstring, [], {}) + self.assertRaises(TypeError, ascharmapstring, 'a', NULL) + # CRASHES ascharmapstring(NULL, {}) + + def test_decodeunicodeescape(self): + """Test PyUnicode_DecodeUnicodeEscape()""" + decodeunicodeescape = _testcapi.unicode_decodeunicodeescape + + self.assertEqual(decodeunicodeescape(b'abc'), 'abc') + self.assertEqual(decodeunicodeescape(br'\t\n\r\x0b\x0c\x00\\'), '\t\n\r\v\f\0\\') + self.assertEqual(decodeunicodeescape(b'\t\n\r\x0b\x0c\x00'), '\t\n\r\v\f\0') + self.assertEqual(decodeunicodeescape(br'\xa1\xa2'), '\xa1\xa2') + self.assertEqual(decodeunicodeescape(b'\xa1\xa2'), '\xa1\xa2') + self.assertEqual(decodeunicodeescape(br'\u4f60\u597d'), '\u4f60\u597d') + self.assertEqual(decodeunicodeescape(br'\U0001f600'), '\U0001f600') + with self.assertWarns(DeprecationWarning): + self.assertEqual(decodeunicodeescape(br'\z'), r'\z') + + for b in b'\\', br'\xa', br'\u4f6', br'\U0001f60': + self.assertRaises(UnicodeDecodeError, decodeunicodeescape, b) + self.assertRaises(UnicodeDecodeError, decodeunicodeescape, b, 'strict') + self.assertEqual(decodeunicodeescape(br'x\U0001f60', 'replace'), 'x\ufffd') + self.assertEqual(decodeunicodeescape(br'x\U0001f60y', 'replace'), 'x\ufffdy') + + self.assertRaises(LookupError, decodeunicodeescape, b'\\', 'foo') + # TODO: Test PyUnicode_DecodeUnicodeEscape() with NULL as data and + # negative size. + + def test_asunicodeescapestring(self): + """Test PyUnicode_AsUnicodeEscapeString()""" + asunicodeescapestring = _testcapi.unicode_asunicodeescapestring + + self.assertEqual(asunicodeescapestring('abc'), b'abc') + self.assertEqual(asunicodeescapestring('\t\n\r\v\f\0\\'), br'\t\n\r\x0b\x0c\x00\\') + self.assertEqual(asunicodeescapestring('\xa1\xa2'), br'\xa1\xa2') + self.assertEqual(asunicodeescapestring('\u4f60\u597d'), br'\u4f60\u597d') + self.assertEqual(asunicodeescapestring('\U0001f600'), br'\U0001f600') + + self.assertRaises(TypeError, asunicodeescapestring, b'abc') + self.assertRaises(TypeError, asunicodeescapestring, []) + # CRASHES asunicodeescapestring(NULL) + + def test_decoderawunicodeescape(self): + """Test PyUnicode_DecodeRawUnicodeEscape()""" + decoderawunicodeescape = _testcapi.unicode_decoderawunicodeescape + + self.assertEqual(decoderawunicodeescape(b'abc'), 'abc') + self.assertEqual(decoderawunicodeescape(b'\t\n\r\v\f\0\\'), '\t\n\r\v\f\0\\') + self.assertEqual(decoderawunicodeescape(b'\xa1\xa2'), '\xa1\xa2') + self.assertEqual(decoderawunicodeescape(br'\u4f60\u597d'), '\u4f60\u597d') + self.assertEqual(decoderawunicodeescape(br'\U0001f600'), '\U0001f600') + self.assertEqual(decoderawunicodeescape(br'\xa1\xa2'), r'\xa1\xa2') + self.assertEqual(decoderawunicodeescape(br'\z'), r'\z') + + for b in br'\u4f6', br'\U0001f60': + self.assertRaises(UnicodeDecodeError, decoderawunicodeescape, b) + self.assertRaises(UnicodeDecodeError, decoderawunicodeescape, b, 'strict') + self.assertEqual(decoderawunicodeescape(br'x\U0001f60', 'replace'), 'x\ufffd') + self.assertEqual(decoderawunicodeescape(br'x\U0001f60y', 'replace'), 'x\ufffdy') + + self.assertRaises(LookupError, decoderawunicodeescape, br'\U0001f60', 'foo') + # TODO: Test PyUnicode_DecodeRawUnicodeEscape() with NULL as data and + # negative size. + + def test_asrawunicodeescapestring(self): + """Test PyUnicode_AsRawUnicodeEscapeString()""" + asrawunicodeescapestring = _testcapi.unicode_asrawunicodeescapestring + + self.assertEqual(asrawunicodeescapestring('abc'), b'abc') + self.assertEqual(asrawunicodeescapestring('\t\n\r\v\f\0\\'), b'\t\n\r\v\f\0\\') + self.assertEqual(asrawunicodeescapestring('\xa1\xa2'), b'\xa1\xa2') + self.assertEqual(asrawunicodeescapestring('\u4f60\u597d'), br'\u4f60\u597d') + self.assertEqual(asrawunicodeescapestring('\U0001f600'), br'\U0001f600') + + self.assertRaises(TypeError, asrawunicodeescapestring, b'abc') + self.assertRaises(TypeError, asrawunicodeescapestring, []) + # CRASHES asrawunicodeescapestring(NULL) + if __name__ == "__main__": unittest.main() diff --git a/Modules/_testcapi/unicode.c b/Modules/_testcapi/unicode.c index 73929eaffc676..b4d7bf82d73c8 100644 --- a/Modules/_testcapi/unicode.c +++ b/Modules/_testcapi/unicode.c @@ -376,6 +376,22 @@ unicode_readchar(PyObject *self, PyObject *args) return PyLong_FromUnsignedLong(result); } +/* Test PyUnicode_FromEncodedObject() */ +static PyObject * +unicode_fromencodedobject(PyObject *self, PyObject *args) +{ + PyObject *obj; + const char *encoding; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "Oz|z", &obj, &encoding, &errors)) { + return NULL; + } + + NULLABLE(obj); + return PyUnicode_FromEncodedObject(obj, encoding, errors); +} + /* Test PyUnicode_FromObject() */ static PyObject * unicode_fromobject(PyObject *self, PyObject *arg) @@ -669,6 +685,78 @@ unicode_transformdecimalandspacetoascii(PyObject *self, PyObject *arg) return _PyUnicode_TransformDecimalAndSpaceToASCII(arg); } +/* Test PyUnicode_Decode() */ +static PyObject * +unicode_decode(PyObject *self, PyObject *args) +{ + const char *s; + Py_ssize_t size; + const char *encoding; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#z|z", &s, &size, &encoding, &errors)) + return NULL; + + return PyUnicode_Decode(s, size, encoding, errors); +} + +/* Test PyUnicode_AsEncodedString() */ +static PyObject * +unicode_asencodedstring(PyObject *self, PyObject *args) +{ + PyObject *unicode; + const char *encoding; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "Oz|z", &unicode, &encoding, &errors)) + return NULL; + + NULLABLE(unicode); + return PyUnicode_AsEncodedString(unicode, encoding, errors); +} + +/* Test PyUnicode_BuildEncodingMap() */ +static PyObject * +unicode_buildencodingmap(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_BuildEncodingMap(arg); +} + +/* Test PyUnicode_DecodeUTF7() */ +static PyObject * +unicode_decodeutf7(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeUTF7(data, size, errors); +} + +/* Test PyUnicode_DecodeUTF7Stateful() */ +static PyObject * +unicode_decodeutf7stateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF7Stateful(data, size, errors, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(Nn)", result, consumed); +} + /* Test PyUnicode_DecodeUTF8() */ static PyObject * unicode_decodeutf8(PyObject *self, PyObject *args) @@ -703,6 +791,387 @@ unicode_decodeutf8stateful(PyObject *self, PyObject *args) return Py_BuildValue("(Nn)", result, consumed); } +/* Test PyUnicode_AsUTF8String() */ +static PyObject * +unicode_asutf8string(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsUTF8String(arg); +} + +/* Test PyUnicode_DecodeUTF32() */ +static PyObject * +unicode_decodeutf32(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + int byteorder; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &byteorder, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF32(data, size, errors, &byteorder); + if (!result) { + return NULL; + } + return Py_BuildValue("(iN)", byteorder, result); +} + +/* Test PyUnicode_DecodeUTF32Stateful() */ +static PyObject * +unicode_decodeutf32stateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + int byteorder; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &byteorder, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF32Stateful(data, size, errors, &byteorder, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(iNn)", byteorder, result, consumed); +} + +/* Test PyUnicode_AsUTF32String() */ +static PyObject * +unicode_asutf32string(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsUTF32String(arg); +} + +/* Test PyUnicode_DecodeUTF16() */ +static PyObject * +unicode_decodeutf16(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + int byteorder = 0; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &byteorder, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF16(data, size, errors, &byteorder); + if (!result) { + return NULL; + } + return Py_BuildValue("(iN)", byteorder, result); +} + +/* Test PyUnicode_DecodeUTF16Stateful() */ +static PyObject * +unicode_decodeutf16stateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + int byteorder; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &byteorder, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF16Stateful(data, size, errors, &byteorder, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(iNn)", byteorder, result, consumed); +} + +/* Test PyUnicode_AsUTF16String() */ +static PyObject * +unicode_asutf16string(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsUTF16String(arg); +} + +/* Test PyUnicode_DecodeUnicodeEscape() */ +static PyObject * +unicode_decodeunicodeescape(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeUnicodeEscape(data, size, errors); +} + +/* Test PyUnicode_AsUnicodeEscapeString() */ +static PyObject * +unicode_asunicodeescapestring(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsUnicodeEscapeString(arg); +} + +static PyObject * +unicode_decoderawunicodeescape(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeRawUnicodeEscape(data, size, errors); +} + +/* Test PyUnicode_AsRawUnicodeEscapeString() */ +static PyObject * +unicode_asrawunicodeescapestring(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsRawUnicodeEscapeString(arg); +} + +static PyObject * +unicode_decodelatin1(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeLatin1(data, size, errors); +} + +/* Test PyUnicode_AsLatin1String() */ +static PyObject * +unicode_aslatin1string(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsLatin1String(arg); +} + +/* Test PyUnicode_DecodeASCII() */ +static PyObject * +unicode_decodeascii(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeASCII(data, size, errors); +} + +/* Test PyUnicode_AsASCIIString() */ +static PyObject * +unicode_asasciistring(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsASCIIString(arg); +} + +/* Test PyUnicode_DecodeCharmap() */ +static PyObject * +unicode_decodecharmap(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + PyObject *mapping; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#O|z", &data, &size, &mapping, &errors)) + return NULL; + + NULLABLE(mapping); + return PyUnicode_DecodeCharmap(data, size, mapping, errors); +} + +/* Test PyUnicode_AsCharmapString() */ +static PyObject * +unicode_ascharmapstring(PyObject *self, PyObject *args) +{ + PyObject *unicode; + PyObject *mapping; + + if (!PyArg_ParseTuple(args, "OO", &unicode, &mapping)) + return NULL; + + NULLABLE(unicode); + NULLABLE(mapping); + return PyUnicode_AsCharmapString(unicode, mapping); +} + +#ifdef MS_WINDOWS + +/* Test PyUnicode_DecodeMBCS() */ +static PyObject * +unicode_decodembcs(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeMBCS(data, size, errors); +} + +/* Test PyUnicode_DecodeMBCSStateful() */ +static PyObject * +unicode_decodembcsstateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeMBCSStateful(data, size, errors, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(Nn)", result, consumed); +} + +/* Test PyUnicode_DecodeCodePageStateful() */ +static PyObject * +unicode_decodecodepagestateful(PyObject *self, PyObject *args) +{ + int code_page; + const char *data; + Py_ssize_t size; + const char *errors = NULL; + Py_ssize_t consumed; + PyObject *result; + + if (!PyArg_ParseTuple(args, "iy#|z", &code_page, &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeCodePageStateful(code_page, data, size, errors, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(Nn)", result, consumed); +} + +/* Test PyUnicode_AsMBCSString() */ +static PyObject * +unicode_asmbcsstring(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_AsMBCSString(arg); +} + +/* Test PyUnicode_EncodeCodePage() */ +static PyObject * +unicode_encodecodepage(PyObject *self, PyObject *args) +{ + int code_page; + PyObject *unicode; + const char *errors; + + if (!PyArg_ParseTuple(args, "iO|z", &code_page, &unicode, &errors)) + return NULL; + + NULLABLE(unicode); + return PyUnicode_EncodeCodePage(code_page, unicode, errors); +} + +#endif /* MS_WINDOWS */ + +/* Test PyUnicode_DecodeLocaleAndSize() */ +static PyObject * +unicode_decodelocaleandsize(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeLocaleAndSize(data, size, errors); +} + +/* Test PyUnicode_DecodeLocale() */ +static PyObject * +unicode_decodelocale(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeLocale(data, errors); +} + +/* Test PyUnicode_EncodeLocale() */ +static PyObject * +unicode_encodelocale(PyObject *self, PyObject *args) +{ + PyObject *unicode; + const char *errors; + + if (!PyArg_ParseTuple(args, "O|z", &unicode, &errors)) + return NULL; + + NULLABLE(unicode); + return PyUnicode_EncodeLocale(unicode, errors); +} + +/* Test PyUnicode_DecodeFSDefault() */ +static PyObject * +unicode_decodefsdefault(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + + if (!PyArg_ParseTuple(args, "y#", &data, &size)) + return NULL; + + return PyUnicode_DecodeFSDefault(data); +} + +/* Test PyUnicode_DecodeFSDefaultAndSize() */ +static PyObject * +unicode_decodefsdefaultandsize(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + + if (!PyArg_ParseTuple(args, "y#|n", &data, &size, &size)) + return NULL; + + return PyUnicode_DecodeFSDefaultAndSize(data, size); +} + +/* Test PyUnicode_EncodeFSDefault() */ +static PyObject * +unicode_encodefsdefault(PyObject *self, PyObject *arg) +{ + NULLABLE(arg); + return PyUnicode_EncodeFSDefault(arg); +} + /* Test PyUnicode_Concat() */ static PyObject * unicode_concat(PyObject *self, PyObject *args) @@ -1528,6 +1997,7 @@ static PyMethodDef TestMethods[] = { {"unicode_substring", unicode_substring, METH_VARARGS}, {"unicode_getlength", unicode_getlength, METH_O}, {"unicode_readchar", unicode_readchar, METH_VARARGS}, + {"unicode_fromencodedobject",unicode_fromencodedobject, METH_VARARGS}, {"unicode_fromobject", unicode_fromobject, METH_O}, {"unicode_interninplace", unicode_interninplace, METH_O}, {"unicode_internfromstring", unicode_internfromstring, METH_O}, @@ -1542,9 +2012,44 @@ static PyMethodDef TestMethods[] = { {"unicode_asutf8", unicode_asutf8, METH_VARARGS}, {"unicode_asutf8andsize", unicode_asutf8andsize, METH_VARARGS}, {"unicode_asutf8andsize_null",unicode_asutf8andsize_null, METH_VARARGS}, + {"unicode_getdefaultencoding",unicode_getdefaultencoding, METH_NOARGS}, + {"unicode_decode", unicode_decode, METH_VARARGS}, + {"unicode_asencodedstring", unicode_asencodedstring, METH_VARARGS}, + {"unicode_buildencodingmap", unicode_buildencodingmap, METH_O}, + {"unicode_decodeutf7", unicode_decodeutf7, METH_VARARGS}, + {"unicode_decodeutf7stateful",unicode_decodeutf7stateful, METH_VARARGS}, {"unicode_decodeutf8", unicode_decodeutf8, METH_VARARGS}, {"unicode_decodeutf8stateful",unicode_decodeutf8stateful, METH_VARARGS}, - {"unicode_getdefaultencoding",unicode_getdefaultencoding, METH_NOARGS}, + {"unicode_asutf8string", unicode_asutf8string, METH_O}, + {"unicode_decodeutf16", unicode_decodeutf16, METH_VARARGS}, + {"unicode_decodeutf16stateful",unicode_decodeutf16stateful, METH_VARARGS}, + {"unicode_asutf16string", unicode_asutf16string, METH_O}, + {"unicode_decodeutf32", unicode_decodeutf32, METH_VARARGS}, + {"unicode_decodeutf32stateful",unicode_decodeutf32stateful, METH_VARARGS}, + {"unicode_asutf32string", unicode_asutf32string, METH_O}, + {"unicode_decodeunicodeescape",unicode_decodeunicodeescape, METH_VARARGS}, + {"unicode_asunicodeescapestring",unicode_asunicodeescapestring,METH_O}, + {"unicode_decoderawunicodeescape",unicode_decoderawunicodeescape,METH_VARARGS}, + {"unicode_asrawunicodeescapestring",unicode_asrawunicodeescapestring,METH_O}, + {"unicode_decodelatin1", unicode_decodelatin1, METH_VARARGS}, + {"unicode_aslatin1string", unicode_aslatin1string, METH_O}, + {"unicode_decodeascii", unicode_decodeascii, METH_VARARGS}, + {"unicode_asasciistring", unicode_asasciistring, METH_O}, + {"unicode_decodecharmap", unicode_decodecharmap, METH_VARARGS}, + {"unicode_ascharmapstring", unicode_ascharmapstring, METH_VARARGS}, +#ifdef MS_WINDOWS + {"unicode_decodembcs", unicode_decodembcs, METH_VARARGS}, + {"unicode_decodembcsstateful",unicode_decodembcsstateful, METH_VARARGS}, + {"unicode_decodecodepagestateful",unicode_decodecodepagestateful,METH_VARARGS}, + {"unicode_asmbcsstring", unicode_asmbcsstring, METH_O}, + {"unicode_encodecodepage", unicode_encodecodepage, METH_VARARGS}, +#endif /* MS_WINDOWS */ + {"unicode_decodelocaleandsize",unicode_decodelocaleandsize, METH_VARARGS}, + {"unicode_decodelocale", unicode_decodelocale, METH_VARARGS}, + {"unicode_encodelocale", unicode_encodelocale, METH_VARARGS}, + {"unicode_decodefsdefault", unicode_decodefsdefault, METH_VARARGS}, + {"unicode_decodefsdefaultandsize",unicode_decodefsdefaultandsize,METH_VARARGS}, + {"unicode_encodefsdefault", unicode_encodefsdefault, METH_O}, {"unicode_transformdecimalandspacetoascii", unicode_transformdecimalandspacetoascii, METH_O}, {"unicode_concat", unicode_concat, METH_VARARGS}, {"unicode_splitlines", unicode_splitlines, METH_VARARGS}, From webhook-mailer at python.org Mon Jul 10 16:29:07 2023 From: webhook-mailer at python.org (terryjreedy) Date: Mon, 10 Jul 2023 20:29:07 -0000 Subject: [Python-checkins] gh-103186: Fix or catch 'extra' stderr output from unittests (#103196) Message-ID: https://github.com/python/cpython/commit/9d582250d8fde240b8e7299b74ba888c574f74a3 commit: 9d582250d8fde240b8e7299b74ba888c574f74a3 branch: main author: Ijtaba Hussain committer: terryjreedy date: 2023-07-10T16:29:03-04:00 summary: gh-103186: Fix or catch 'extra' stderr output from unittests (#103196) Reduce test noise by fixing or catching and testing stderr messages from individual tests. test_cmd_line_script.test_script_as_dev_fd calls spawn_python and hence subprocess.Popen with incompatible arguments. On POSIX, pass_fds forces close_fds to be True (subprocess.py line 848). Correct the call. test_uuid.test_cli_namespace_required_for_uuid3: when the namespace is omitted, uuid.main calls argparse.Argument_Parser.error, which prints to stderr before calling sys.exit, which raises SystemExit. Unittest assertRaises catches the exception but not the previous output. Catch the output and test it. test_warnings.test_catchwarnings_with_simplefilter_error similarly prints before raising. Catch the output and test it. --------- Co-authored-by: Oleg Iarygin files: M Lib/test/test_cmd_line_script.py M Lib/test/test_uuid.py M Lib/test/test_warnings/__init__.py diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index 8bf299382e9ca..1b58882601071 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -777,7 +777,7 @@ def test_script_as_dev_fd(self): with os_helper.temp_dir() as work_dir: script_name = _make_test_script(work_dir, 'script.py', script) with open(script_name, "r") as fp: - p = spawn_python(f"/dev/fd/{fp.fileno()}", close_fds=False, pass_fds=(0,1,2,fp.fileno())) + p = spawn_python(f"/dev/fd/{fp.fileno()}", close_fds=True, pass_fds=(0,1,2,fp.fileno())) out, err = p.communicate() self.assertEqual(out, b"12345678912345678912345\n") diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py index a178e942ecda0..9cec1e87fd3c2 100755 --- a/Lib/test/test_uuid.py +++ b/Lib/test/test_uuid.py @@ -706,20 +706,23 @@ def test_uuid_weakref(self): self.assertIs(strong, weak()) @mock.patch.object(sys, "argv", ["", "-u", "uuid3", "-n", "@dns"]) - def test_cli_namespace_required_for_uuid3(self): + @mock.patch('sys.stderr', new_callable=io.StringIO) + def test_cli_namespace_required_for_uuid3(self, mock_err): with self.assertRaises(SystemExit) as cm: self.uuid.main() # Check that exception code is the same as argparse.ArgumentParser.error self.assertEqual(cm.exception.code, 2) + self.assertIn("error: Incorrect number of arguments", mock_err.getvalue()) @mock.patch.object(sys, "argv", ["", "-u", "uuid3", "-N", "python.org"]) - def test_cli_name_required_for_uuid3(self): + @mock.patch('sys.stderr', new_callable=io.StringIO) + def test_cli_name_required_for_uuid3(self, mock_err): with self.assertRaises(SystemExit) as cm: self.uuid.main() - # Check that exception code is the same as argparse.ArgumentParser.error self.assertEqual(cm.exception.code, 2) + self.assertIn("error: Incorrect number of arguments", mock_err.getvalue()) @mock.patch.object(sys, "argv", [""]) def test_cli_uuid4_outputted_with_no_args(self): diff --git a/Lib/test/test_warnings/__init__.py b/Lib/test/test_warnings/__init__.py index 9e680c847dab7..83237f5fe0d1b 100644 --- a/Lib/test/test_warnings/__init__.py +++ b/Lib/test/test_warnings/__init__.py @@ -387,9 +387,13 @@ def test_catchwarnings_with_simplefilter_error(self): with self.module.catch_warnings( module=self.module, action="error", category=FutureWarning ): - self.module.warn("Other types of warnings are not errors") - self.assertRaises(FutureWarning, - self.module.warn, FutureWarning("msg")) + with support.captured_stderr() as stderr: + error_msg = "Other types of warnings are not errors" + self.module.warn(error_msg) + self.assertRaises(FutureWarning, + self.module.warn, FutureWarning("msg")) + stderr = stderr.getvalue() + self.assertIn(error_msg, stderr) class CFilterTests(FilterTests, unittest.TestCase): module = c_warnings From webhook-mailer at python.org Mon Jul 10 17:03:36 2023 From: webhook-mailer at python.org (terryjreedy) Date: Mon, 10 Jul 2023 21:03:36 -0000 Subject: [Python-checkins] [3.12] gh-103186: Fix or catch 'extra' stderr output from unittests (GH-103196) (#106605) Message-ID: https://github.com/python/cpython/commit/68ca19061d19345a72574a6c3849981213de212b commit: 68ca19061d19345a72574a6c3849981213de212b branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-10T21:03:32Z summary: [3.12] gh-103186: Fix or catch 'extra' stderr output from unittests (GH-103196) (#106605) gh-103186: Fix or catch 'extra' stderr output from unittests (GH-103196) Reduce test noise by fixing or catching and testing stderr messages from individual tests. test_cmd_line_script.test_script_as_dev_fd calls spawn_python and hence subprocess.Popen with incompatible arguments. On POSIX, pass_fds forces close_fds to be True (subprocess.py line 848). Correct the call. test_uuid.test_cli_namespace_required_for_uuid3: when the namespace is omitted, uuid.main calls argparse.Argument_Parser.error, which prints to stderr before calling sys.exit, which raises SystemExit. Unittest assertRaises catches the exception but not the previous output. Catch the output and test it. test_warnings.test_catchwarnings_with_simplefilter_error similarly prints before raising. Catch the output and test it. --------- (cherry picked from commit 9d582250d8fde240b8e7299b74ba888c574f74a3) Co-authored-by: Ijtaba Hussain Co-authored-by: Oleg Iarygin files: M Lib/test/test_cmd_line_script.py M Lib/test/test_uuid.py M Lib/test/test_warnings/__init__.py diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index 8bf299382e9ca..1b58882601071 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -777,7 +777,7 @@ def test_script_as_dev_fd(self): with os_helper.temp_dir() as work_dir: script_name = _make_test_script(work_dir, 'script.py', script) with open(script_name, "r") as fp: - p = spawn_python(f"/dev/fd/{fp.fileno()}", close_fds=False, pass_fds=(0,1,2,fp.fileno())) + p = spawn_python(f"/dev/fd/{fp.fileno()}", close_fds=True, pass_fds=(0,1,2,fp.fileno())) out, err = p.communicate() self.assertEqual(out, b"12345678912345678912345\n") diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py index a178e942ecda0..9cec1e87fd3c2 100755 --- a/Lib/test/test_uuid.py +++ b/Lib/test/test_uuid.py @@ -706,20 +706,23 @@ def test_uuid_weakref(self): self.assertIs(strong, weak()) @mock.patch.object(sys, "argv", ["", "-u", "uuid3", "-n", "@dns"]) - def test_cli_namespace_required_for_uuid3(self): + @mock.patch('sys.stderr', new_callable=io.StringIO) + def test_cli_namespace_required_for_uuid3(self, mock_err): with self.assertRaises(SystemExit) as cm: self.uuid.main() # Check that exception code is the same as argparse.ArgumentParser.error self.assertEqual(cm.exception.code, 2) + self.assertIn("error: Incorrect number of arguments", mock_err.getvalue()) @mock.patch.object(sys, "argv", ["", "-u", "uuid3", "-N", "python.org"]) - def test_cli_name_required_for_uuid3(self): + @mock.patch('sys.stderr', new_callable=io.StringIO) + def test_cli_name_required_for_uuid3(self, mock_err): with self.assertRaises(SystemExit) as cm: self.uuid.main() - # Check that exception code is the same as argparse.ArgumentParser.error self.assertEqual(cm.exception.code, 2) + self.assertIn("error: Incorrect number of arguments", mock_err.getvalue()) @mock.patch.object(sys, "argv", [""]) def test_cli_uuid4_outputted_with_no_args(self): diff --git a/Lib/test/test_warnings/__init__.py b/Lib/test/test_warnings/__init__.py index 9e680c847dab7..83237f5fe0d1b 100644 --- a/Lib/test/test_warnings/__init__.py +++ b/Lib/test/test_warnings/__init__.py @@ -387,9 +387,13 @@ def test_catchwarnings_with_simplefilter_error(self): with self.module.catch_warnings( module=self.module, action="error", category=FutureWarning ): - self.module.warn("Other types of warnings are not errors") - self.assertRaises(FutureWarning, - self.module.warn, FutureWarning("msg")) + with support.captured_stderr() as stderr: + error_msg = "Other types of warnings are not errors" + self.module.warn(error_msg) + self.assertRaises(FutureWarning, + self.module.warn, FutureWarning("msg")) + stderr = stderr.getvalue() + self.assertIn(error_msg, stderr) class CFilterTests(FilterTests, unittest.TestCase): module = c_warnings From webhook-mailer at python.org Mon Jul 10 17:13:30 2023 From: webhook-mailer at python.org (terryjreedy) Date: Mon, 10 Jul 2023 21:13:30 -0000 Subject: [Python-checkins] [3.11] gh-103186: Fix or catch 'extra' stderr output from unittests (#103196) (#106606) Message-ID: https://github.com/python/cpython/commit/563829df559332eaa169c5cd84ce87e4f962dfaa commit: 563829df559332eaa169c5cd84ce87e4f962dfaa branch: 3.11 author: Terry Jan Reedy committer: terryjreedy date: 2023-07-10T21:13:26Z summary: [3.11] gh-103186: Fix or catch 'extra' stderr output from unittests (#103196) (#106606) Reduce test noise by fixing or catching and testing stderr messages from individual tests. test_cmd_line_script.test_script_as_dev_fd calls spawn_python and hence subprocess.Popen with incompatible arguments. On POSIX, pass_fds forces close_fds to be True (subprocess.py line 848). Correct the call. test_uuid.test_cli_namespace_required_for_uuid3: when the namespace is omitted, uuid.main calls argparse.Argument_Parser.error, which prints to stderr before calling sys.exit, which raises SystemExit. Unittest assertRaises catches the exception but not the previous output. Catch the output and test it. test_warnings.test_catchwarnings_with_simplefilter_error similarly prints before raising. Catch the output and test it. --------- Co-authored-by: Oleg Iarygin (cherry picked from commit 9d582250d8fde240b8e7299b74ba888c574f74a3) files: M Lib/test/test_cmd_line_script.py M Lib/test/test_warnings/__init__.py diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index d10012759c370..7fcd563be271a 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -777,7 +777,7 @@ def test_script_as_dev_fd(self): with os_helper.temp_dir() as work_dir: script_name = _make_test_script(work_dir, 'script.py', script) with open(script_name, "r") as fp: - p = spawn_python(f"/dev/fd/{fp.fileno()}", close_fds=False, pass_fds=(0,1,2,fp.fileno())) + p = spawn_python(f"/dev/fd/{fp.fileno()}", close_fds=True, pass_fds=(0,1,2,fp.fileno())) out, err = p.communicate() self.assertEqual(out, b"12345678912345678912345\n") diff --git a/Lib/test/test_warnings/__init__.py b/Lib/test/test_warnings/__init__.py index 61a644406a6c2..29ee120f40af5 100644 --- a/Lib/test/test_warnings/__init__.py +++ b/Lib/test/test_warnings/__init__.py @@ -388,9 +388,13 @@ def test_catchwarnings_with_simplefilter_error(self): with self.module.catch_warnings( module=self.module, action="error", category=FutureWarning ): - self.module.warn("Other types of warnings are not errors") - self.assertRaises(FutureWarning, - self.module.warn, FutureWarning("msg")) + with support.captured_stderr() as stderr: + error_msg = "Other types of warnings are not errors" + self.module.warn(error_msg) + self.assertRaises(FutureWarning, + self.module.warn, FutureWarning("msg")) + stderr = stderr.getvalue() + self.assertIn(error_msg, stderr) class CFilterTests(FilterTests, unittest.TestCase): module = c_warnings From webhook-mailer at python.org Mon Jul 10 17:45:30 2023 From: webhook-mailer at python.org (gpshead) Date: Mon, 10 Jul 2023 21:45:30 -0000 Subject: [Python-checkins] gh-94777: Fix deadlock in ProcessPoolExecutor (#94784) Message-ID: https://github.com/python/cpython/commit/6782fc050281205734700a1c3e13b123961ed15b commit: 6782fc050281205734700a1c3e13b123961ed15b branch: main author: Louis Paulot <55740424+lpaulot at users.noreply.github.com> committer: gpshead date: 2023-07-10T21:45:27Z summary: gh-94777: Fix deadlock in ProcessPoolExecutor (#94784) Fixes a hang in multiprocessing process pool executor when a child process crashes and code could otherwise block on writing to the pipe. See GH-94777 for more details. files: A Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst M Lib/concurrent/futures/process.py M Lib/test/test_concurrent_futures.py diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py index 816edab99f63e..301207f59de37 100644 --- a/Lib/concurrent/futures/process.py +++ b/Lib/concurrent/futures/process.py @@ -499,6 +499,10 @@ def terminate_broken(self, cause): for p in self.processes.values(): p.terminate() + # Prevent queue writing to a pipe which is no longer read. + # https://github.com/python/cpython/issues/94777 + self.call_queue._reader.close() + # clean up resources self.join_executor_internals() diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py index a20cb844a293c..39dbe234e765e 100644 --- a/Lib/test/test_concurrent_futures.py +++ b/Lib/test/test_concurrent_futures.py @@ -1172,6 +1172,11 @@ def _crash(delay=None): faulthandler._sigsegv() +def _crash_with_data(data): + """Induces a segfault with dummy data in input.""" + _crash() + + def _exit(): """Induces a sys exit with exitcode 1.""" sys.exit(1) @@ -1371,6 +1376,19 @@ def test_shutdown_deadlock_pickle(self): # dangling threads executor_manager.join() + def test_crash_big_data(self): + # Test that there is a clean exception instad of a deadlock when a + # child process crashes while some data is being written into the + # queue. + # https://github.com/python/cpython/issues/94777 + self.executor.shutdown(wait=True) + data = "a" * support.PIPE_MAX_SIZE + with self.executor_type(max_workers=2, + mp_context=self.get_context()) as executor: + self.executor = executor # Allow clean up in fail_on_deadlock + with self.assertRaises(BrokenProcessPool): + list(executor.map(_crash_with_data, [data] * 10)) + create_executor_tests(ExecutorDeadlockTest, executor_mixins=(ProcessPoolForkMixin, diff --git a/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst b/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst new file mode 100644 index 0000000000000..2c04a35fbfce1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst @@ -0,0 +1 @@ +Fix hanging :mod:`multiprocessing` ``ProcessPoolExecutor`` when a child process crashes while data is being written in the call queue. From webhook-mailer at python.org Mon Jul 10 18:21:08 2023 From: webhook-mailer at python.org (carljm) Date: Mon, 10 Jul 2023 22:21:08 -0000 Subject: [Python-checkins] [3.11] gh-94777: Fix deadlock in ProcessPoolExecutor (GH-94784) (#106607) Message-ID: https://github.com/python/cpython/commit/d63953d0f532fd4ed9615171b1f5699d5fff7db9 commit: d63953d0f532fd4ed9615171b1f5699d5fff7db9 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: carljm date: 2023-07-10T16:21:04-06:00 summary: [3.11] gh-94777: Fix deadlock in ProcessPoolExecutor (GH-94784) (#106607) gh-94777: Fix deadlock in ProcessPoolExecutor (GH-94784) Fixes a hang in multiprocessing process pool executor when a child process crashes and code could otherwise block on writing to the pipe. See GH-94777 for more details. (cherry picked from commit 6782fc050281205734700a1c3e13b123961ed15b) Co-authored-by: Louis Paulot <55740424+lpaulot at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst M Lib/concurrent/futures/process.py M Lib/test/test_concurrent_futures.py diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py index 392d1960e0595..321608ab174cf 100644 --- a/Lib/concurrent/futures/process.py +++ b/Lib/concurrent/futures/process.py @@ -497,6 +497,10 @@ def terminate_broken(self, cause): for p in self.processes.values(): p.terminate() + # Prevent queue writing to a pipe which is no longer read. + # https://github.com/python/cpython/issues/94777 + self.call_queue._reader.close() + # clean up resources self.join_executor_internals() diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py index 1bb6156a18ddb..4558560e1a992 100644 --- a/Lib/test/test_concurrent_futures.py +++ b/Lib/test/test_concurrent_futures.py @@ -1167,6 +1167,11 @@ def _crash(delay=None): faulthandler._sigsegv() +def _crash_with_data(data): + """Induces a segfault with dummy data in input.""" + _crash() + + def _exit(): """Induces a sys exit with exitcode 1.""" sys.exit(1) @@ -1366,6 +1371,19 @@ def test_shutdown_deadlock_pickle(self): # dangling threads executor_manager.join() + def test_crash_big_data(self): + # Test that there is a clean exception instad of a deadlock when a + # child process crashes while some data is being written into the + # queue. + # https://github.com/python/cpython/issues/94777 + self.executor.shutdown(wait=True) + data = "a" * support.PIPE_MAX_SIZE + with self.executor_type(max_workers=2, + mp_context=self.get_context()) as executor: + self.executor = executor # Allow clean up in fail_on_deadlock + with self.assertRaises(BrokenProcessPool): + list(executor.map(_crash_with_data, [data] * 10)) + create_executor_tests(ExecutorDeadlockTest, executor_mixins=(ProcessPoolForkMixin, diff --git a/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst b/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst new file mode 100644 index 0000000000000..2c04a35fbfce1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst @@ -0,0 +1 @@ +Fix hanging :mod:`multiprocessing` ``ProcessPoolExecutor`` when a child process crashes while data is being written in the call queue. From webhook-mailer at python.org Mon Jul 10 18:49:11 2023 From: webhook-mailer at python.org (carljm) Date: Mon, 10 Jul 2023 22:49:11 -0000 Subject: [Python-checkins] [3.12] gh-94777: Fix deadlock in ProcessPoolExecutor (GH-94784) (#106609) Message-ID: https://github.com/python/cpython/commit/90ea3be6fbee1520dd3e9344c71123e15fcbe6f2 commit: 90ea3be6fbee1520dd3e9344c71123e15fcbe6f2 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: carljm date: 2023-07-10T22:49:07Z summary: [3.12] gh-94777: Fix deadlock in ProcessPoolExecutor (GH-94784) (#106609) gh-94777: Fix deadlock in ProcessPoolExecutor (GH-94784) Fixes a hang in multiprocessing process pool executor when a child process crashes and code could otherwise block on writing to the pipe. See GH-94777 for more details. (cherry picked from commit 6782fc050281205734700a1c3e13b123961ed15b) Co-authored-by: Louis Paulot <55740424+lpaulot at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst M Lib/concurrent/futures/process.py M Lib/test/test_concurrent_futures.py diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py index 816edab99f63e..301207f59de37 100644 --- a/Lib/concurrent/futures/process.py +++ b/Lib/concurrent/futures/process.py @@ -499,6 +499,10 @@ def terminate_broken(self, cause): for p in self.processes.values(): p.terminate() + # Prevent queue writing to a pipe which is no longer read. + # https://github.com/python/cpython/issues/94777 + self.call_queue._reader.close() + # clean up resources self.join_executor_internals() diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py index a20cb844a293c..39dbe234e765e 100644 --- a/Lib/test/test_concurrent_futures.py +++ b/Lib/test/test_concurrent_futures.py @@ -1172,6 +1172,11 @@ def _crash(delay=None): faulthandler._sigsegv() +def _crash_with_data(data): + """Induces a segfault with dummy data in input.""" + _crash() + + def _exit(): """Induces a sys exit with exitcode 1.""" sys.exit(1) @@ -1371,6 +1376,19 @@ def test_shutdown_deadlock_pickle(self): # dangling threads executor_manager.join() + def test_crash_big_data(self): + # Test that there is a clean exception instad of a deadlock when a + # child process crashes while some data is being written into the + # queue. + # https://github.com/python/cpython/issues/94777 + self.executor.shutdown(wait=True) + data = "a" * support.PIPE_MAX_SIZE + with self.executor_type(max_workers=2, + mp_context=self.get_context()) as executor: + self.executor = executor # Allow clean up in fail_on_deadlock + with self.assertRaises(BrokenProcessPool): + list(executor.map(_crash_with_data, [data] * 10)) + create_executor_tests(ExecutorDeadlockTest, executor_mixins=(ProcessPoolForkMixin, diff --git a/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst b/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst new file mode 100644 index 0000000000000..2c04a35fbfce1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst @@ -0,0 +1 @@ +Fix hanging :mod:`multiprocessing` ``ProcessPoolExecutor`` when a child process crashes while data is being written in the call queue. From webhook-mailer at python.org Mon Jul 10 19:00:59 2023 From: webhook-mailer at python.org (gpshead) Date: Mon, 10 Jul 2023 23:00:59 -0000 Subject: [Python-checkins] gh-102988: Detect email address parsing errors and return empty tuple to indicate the parsing error (old API) (#105127) Message-ID: https://github.com/python/cpython/commit/18dfbd035775c15533d13a98e56b1d2bf5c65f00 commit: 18dfbd035775c15533d13a98e56b1d2bf5c65f00 branch: main author: Thomas Dwyer committer: gpshead date: 2023-07-10T23:00:55Z summary: gh-102988: Detect email address parsing errors and return empty tuple to indicate the parsing error (old API) (#105127) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). This fixes or at least ameliorates CVE-2023-27043. --------- Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst M Doc/library/email.utils.rst M Doc/whatsnew/3.12.rst M Lib/email/utils.py M Lib/test/test_email/test_email.py diff --git a/Doc/library/email.utils.rst b/Doc/library/email.utils.rst index 345b64001c1ac..a87a0bd2e7de6 100644 --- a/Doc/library/email.utils.rst +++ b/Doc/library/email.utils.rst @@ -65,6 +65,11 @@ of the new API. *email address* parts. Returns a tuple of that information, unless the parse fails, in which case a 2-tuple of ``('', '')`` is returned. + .. versionchanged:: 3.12 + For security reasons, addresses that were ambiguous and could parse into + multiple different addresses now cause ``('', '')`` to be returned + instead of only one of the *potential* addresses. + .. function:: formataddr(pair, charset='utf-8') @@ -87,7 +92,7 @@ of the new API. This method returns a list of 2-tuples of the form returned by ``parseaddr()``. *fieldvalues* is a sequence of header field values as might be returned by :meth:`Message.get_all `. Here's a simple - example that gets all the recipients of a message:: + example that gets all the recipients of a message: from email.utils import getaddresses @@ -97,6 +102,25 @@ of the new API. resent_ccs = msg.get_all('resent-cc', []) all_recipients = getaddresses(tos + ccs + resent_tos + resent_ccs) + When parsing fails for a single fieldvalue, a 2-tuple of ``('', '')`` + is returned in its place. Other errors in parsing the list of + addresses such as a fieldvalue seemingly parsing into multiple + addresses may result in a list containing a single empty 2-tuple + ``[('', '')]`` being returned rather than returning potentially + invalid output. + + Example malformed input parsing: + + .. doctest:: + + >>> from email.utils import getaddresses + >>> getaddresses(['alice at example.com ', 'me at example.com']) + [('', '')] + + .. versionchanged:: 3.12 + The 2-tuple of ``('', '')`` in the returned values when parsing + fails were added as to address a security issue. + .. function:: parsedate(date) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index a892f92dd2628..766f468cf9cfb 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -570,6 +570,14 @@ dis :data:`~dis.hasarg` collection instead. (Contributed by Irit Katriel in :gh:`94216`.) +email +----- + +* :func:`email.utils.getaddresses` and :func:`email.utils.parseaddr` now return + ``('', '')`` 2-tuples in more situations where invalid email addresses are + encountered instead of potentially inaccurate values. + (Contributed by Thomas Dwyer for :gh:`102988` to ameliorate CVE-2023-27043.) + fractions --------- diff --git a/Lib/email/utils.py b/Lib/email/utils.py index 81da5394ea169..11ad75e94e934 100644 --- a/Lib/email/utils.py +++ b/Lib/email/utils.py @@ -106,12 +106,54 @@ def formataddr(pair, charset='utf-8'): return address +def _pre_parse_validation(email_header_fields): + accepted_values = [] + for v in email_header_fields: + s = v.replace('\\(', '').replace('\\)', '') + if s.count('(') != s.count(')'): + v = "('', '')" + accepted_values.append(v) + + return accepted_values + + +def _post_parse_validation(parsed_email_header_tuples): + accepted_values = [] + # The parser would have parsed a correctly formatted domain-literal + # The existence of an [ after parsing indicates a parsing failure + for v in parsed_email_header_tuples: + if '[' in v[1]: + v = ('', '') + accepted_values.append(v) + + return accepted_values + def getaddresses(fieldvalues): - """Return a list of (REALNAME, EMAIL) for each fieldvalue.""" - all = COMMASPACE.join(str(v) for v in fieldvalues) + """Return a list of (REALNAME, EMAIL) or ('','') for each fieldvalue. + + When parsing fails for a fieldvalue, a 2-tuple of ('', '') is returned in + its place. + + If the resulting list of parsed address is not the same as the number of + fieldvalues in the input list a parsing error has occurred. A list + containing a single empty 2-tuple [('', '')] is returned in its place. + This is done to avoid invalid output. + """ + fieldvalues = [str(v) for v in fieldvalues] + fieldvalues = _pre_parse_validation(fieldvalues) + all = COMMASPACE.join(v for v in fieldvalues) a = _AddressList(all) - return a.addresslist + result = _post_parse_validation(a.addresslist) + + n = 0 + for v in fieldvalues: + n += v.count(',') + 1 + + if len(result) != n: + return [('', '')] + + return result def _format_timetuple_and_zone(timetuple, zone): @@ -212,9 +254,18 @@ def parseaddr(addr): Return a tuple of realname and email address, unless the parse fails, in which case return a 2-tuple of ('', ''). """ - addrs = _AddressList(addr).addresslist - if not addrs: - return '', '' + if isinstance(addr, list): + addr = addr[0] + + if not isinstance(addr, str): + return ('', '') + + addr = _pre_parse_validation([addr])[0] + addrs = _post_parse_validation(_AddressList(addr).addresslist) + + if not addrs or len(addrs) > 1: + return ('', '') + return addrs[0] diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index 44b405740c440..5238944d6b478 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3319,15 +3319,90 @@ def test_getaddresses(self): [('Al Person', 'aperson at dom.ain'), ('Bud Person', 'bperson at dom.ain')]) + def test_getaddresses_parsing_errors(self): + """Test for parsing errors from CVE-2023-27043""" + eq = self.assertEqual + eq(utils.getaddresses(['alice at example.org(']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org)']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org<']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org>']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org@']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org,']), + [('', 'alice at example.org'), ('', 'bob at example.com')]) + eq(utils.getaddresses(['alice at example.org;']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org:']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org.']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org"']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org[']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org]']), + [('', '')]) + + def test_parseaddr_parsing_errors(self): + """Test for parsing errors from CVE-2023-27043""" + eq = self.assertEqual + eq(utils.parseaddr(['alice at example.org(']), + ('', '')) + eq(utils.parseaddr(['alice at example.org)']), + ('', '')) + eq(utils.parseaddr(['alice at example.org<']), + ('', '')) + eq(utils.parseaddr(['alice at example.org>']), + ('', '')) + eq(utils.parseaddr(['alice at example.org@']), + ('', '')) + eq(utils.parseaddr(['alice at example.org,']), + ('', '')) + eq(utils.parseaddr(['alice at example.org;']), + ('', '')) + eq(utils.parseaddr(['alice at example.org:']), + ('', '')) + eq(utils.parseaddr(['alice at example.org.']), + ('', '')) + eq(utils.parseaddr(['alice at example.org"']), + ('', '')) + eq(utils.parseaddr(['alice at example.org[']), + ('', '')) + eq(utils.parseaddr(['alice at example.org]']), + ('', '')) + def test_getaddresses_nasty(self): eq = self.assertEqual eq(utils.getaddresses(['foo: ;']), [('', '')]) - eq(utils.getaddresses( - ['[]*-- =~$']), - [('', ''), ('', ''), ('', '*--')]) + eq(utils.getaddresses(['[]*-- =~$']), [('', '')]) eq(utils.getaddresses( ['foo: ;', '"Jason R. Mastaler" ']), [('', ''), ('Jason R. Mastaler', 'jason at dom.ain')]) + eq(utils.getaddresses( + [r'Pete(A nice \) chap) ']), + [('Pete (A nice ) chap his account his host)', 'pete at silly.test')]) + eq(utils.getaddresses( + ['(Empty list)(start)Undisclosed recipients :(nobody(I know))']), + [('', '')]) + eq(utils.getaddresses( + ['Mary <@machine.tld:mary at example.net>, , jdoe at test . example']), + [('Mary', 'mary at example.net'), ('', ''), ('', 'jdoe at test.example')]) + eq(utils.getaddresses( + ['John Doe ']), + [('John Doe (comment)', 'jdoe at machine.example')]) + eq(utils.getaddresses( + ['"Mary Smith: Personal Account" ']), + [('Mary Smith: Personal Account', 'smith at home.example')]) + eq(utils.getaddresses( + ['Undisclosed recipients:;']), + [('', '')]) + eq(utils.getaddresses( + [r', "Giant; \"Big\" Box" ']), + [('', 'boss at nil.test'), ('Giant; "Big" Box', 'bob at example.net')]) def test_getaddresses_embedded_comment(self): """Test proper handling of a nested comment""" diff --git a/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst b/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst new file mode 100644 index 0000000000000..e0434ccd2ccab --- /dev/null +++ b/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst @@ -0,0 +1,4 @@ +CVE-2023-27043: Prevent :func:`email.utils.parseaddr` +and :func:`email.utils.getaddresses` from returning the realname portion of an +invalid RFC2822 email header in the email address portion of the 2-tuple +returned after being parsed by :class:`email._parseaddr.AddressList`. From webhook-mailer at python.org Mon Jul 10 19:04:29 2023 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 10 Jul 2023 23:04:29 -0000 Subject: [Python-checkins] gh-106529: Implement POP_JUMP_IF_XXX uops (#106551) Message-ID: https://github.com/python/cpython/commit/22988c323ad621b9f47b6cb640b80ac806e26368 commit: 22988c323ad621b9f47b6cb640b80ac806e26368 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-10T16:04:26-07:00 summary: gh-106529: Implement POP_JUMP_IF_XXX uops (#106551) - Hand-written uops JUMP_IF_{TRUE,FALSE}. These peek at the top of the stack. The jump target (in superblock space) is absolute. - Hand-written translation for POP_JUMP_IF_{TRUE,FALSE}, assuming the jump is unlikely. Once we implement jump-likelihood profiling, we can implement the jump-unlikely case (in another PR). - Tests (including some test cleanup). - Improvements to len(ex) and ex[i] to expose the whole trace. files: M Lib/test/test_capi/test_misc.py M Python/ceval.c M Python/opcode_metadata.h M Python/optimizer.c M Tools/cases_generator/generate_cases.py diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 5f39f23401c3f..fd27bd06097f6 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2347,11 +2347,12 @@ def func(): @contextlib.contextmanager def temporary_optimizer(opt): + old_opt = _testinternalcapi.get_optimizer() _testinternalcapi.set_optimizer(opt) try: yield finally: - _testinternalcapi.set_optimizer(None) + _testinternalcapi.set_optimizer(old_opt) @contextlib.contextmanager @@ -2420,8 +2421,8 @@ def long_loop(): self.assertEqual(opt.get_count(), 10) - -def get_first_executor(code): +def get_first_executor(func): + code = func.__code__ co_code = code.co_code JUMP_BACKWARD = opcode.opmap["JUMP_BACKWARD"] for i in range(0, len(co_code), 2): @@ -2446,13 +2447,7 @@ def testfunc(x): with temporary_optimizer(opt): testfunc(1000) - ex = None - for offset in range(0, len(testfunc.__code__.co_code), 2): - try: - ex = _testinternalcapi.get_executor(testfunc.__code__, offset) - break - except ValueError: - pass + ex = get_first_executor(testfunc) self.assertIsNotNone(ex) uops = {opname for opname, _ in ex} self.assertIn("SAVE_IP", uops) @@ -2493,11 +2488,13 @@ def many_vars(): opt = _testinternalcapi.get_uop_optimizer() with temporary_optimizer(opt): - ex = get_first_executor(many_vars.__code__) + ex = get_first_executor(many_vars) self.assertIsNone(ex) many_vars() - ex = get_first_executor(many_vars.__code__) - self.assertIn(("LOAD_FAST", 259), list(ex)) + + ex = get_first_executor(many_vars) + self.assertIsNotNone(ex) + self.assertIn(("LOAD_FAST", 259), list(ex)) def test_unspecialized_unpack(self): # An example of an unspecialized opcode @@ -2516,17 +2513,43 @@ def testfunc(x): with temporary_optimizer(opt): testfunc(10) - ex = None - for offset in range(0, len(testfunc.__code__.co_code), 2): - try: - ex = _testinternalcapi.get_executor(testfunc.__code__, offset) - break - except ValueError: - pass + ex = get_first_executor(testfunc) self.assertIsNotNone(ex) uops = {opname for opname, _ in ex} self.assertIn("UNPACK_SEQUENCE", uops) + def test_pop_jump_if_false(self): + def testfunc(n): + i = 0 + while i < n: + i += 1 + + opt = _testinternalcapi.get_uop_optimizer() + + with temporary_optimizer(opt): + testfunc(10) + + ex = get_first_executor(testfunc) + self.assertIsNotNone(ex) + uops = {opname for opname, _ in ex} + self.assertIn("_POP_JUMP_IF_FALSE", uops) + + def test_pop_jump_if_true(self): + def testfunc(n): + i = 0 + while not i >= n: + i += 1 + + opt = _testinternalcapi.get_uop_optimizer() + + with temporary_optimizer(opt): + testfunc(10) + + ex = get_first_executor(testfunc) + self.assertIsNotNone(ex) + uops = {opname for opname, _ in ex} + self.assertIn("_POP_JUMP_IF_TRUE", uops) + if __name__ == "__main__": unittest.main() diff --git a/Python/ceval.c b/Python/ceval.c index da1135549a255..866acd2dd69c7 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2751,7 +2751,8 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject operand = self->trace[pc].operand; oparg = (int)operand; DPRINTF(3, - " uop %s, operand %" PRIu64 ", stack_level %d\n", + "%4d: uop %s, operand %" PRIu64 ", stack_level %d\n", + pc, opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode], operand, (int)(stack_pointer - _PyFrame_Stackbase(frame))); @@ -2763,6 +2764,25 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject #define ENABLE_SPECIALIZATION 0 #include "executor_cases.c.h" + // NOTE: These pop-jumps move the uop pc, not the bytecode ip + case _POP_JUMP_IF_FALSE: + { + if (Py_IsFalse(stack_pointer[-1])) { + pc = oparg; + } + stack_pointer--; + break; + } + + case _POP_JUMP_IF_TRUE: + { + if (Py_IsTrue(stack_pointer[-1])) { + pc = oparg; + } + stack_pointer--; + break; + } + case SAVE_IP: { frame->prev_instr = ip_offset + oparg; diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index 92768e60f62ab..64923e61fa459 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -21,18 +21,20 @@ #define EXIT_TRACE 300 #define SAVE_IP 301 -#define _GUARD_BOTH_INT 302 -#define _BINARY_OP_MULTIPLY_INT 303 -#define _BINARY_OP_ADD_INT 304 -#define _BINARY_OP_SUBTRACT_INT 305 -#define _GUARD_BOTH_FLOAT 306 -#define _BINARY_OP_MULTIPLY_FLOAT 307 -#define _BINARY_OP_ADD_FLOAT 308 -#define _BINARY_OP_SUBTRACT_FLOAT 309 -#define _GUARD_BOTH_UNICODE 310 -#define _BINARY_OP_ADD_UNICODE 311 -#define _LOAD_LOCALS 312 -#define _LOAD_FROM_DICT_OR_GLOBALS 313 +#define _POP_JUMP_IF_FALSE 302 +#define _POP_JUMP_IF_TRUE 303 +#define _GUARD_BOTH_INT 304 +#define _BINARY_OP_MULTIPLY_INT 305 +#define _BINARY_OP_ADD_INT 306 +#define _BINARY_OP_SUBTRACT_INT 307 +#define _GUARD_BOTH_FLOAT 308 +#define _BINARY_OP_MULTIPLY_FLOAT 309 +#define _BINARY_OP_ADD_FLOAT 310 +#define _BINARY_OP_SUBTRACT_FLOAT 311 +#define _GUARD_BOTH_UNICODE 312 +#define _BINARY_OP_ADD_UNICODE 313 +#define _LOAD_LOCALS 314 +#define _LOAD_FROM_DICT_OR_GLOBALS 315 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -1294,18 +1296,20 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { const char * const _PyOpcode_uop_name[512] = { [300] = "EXIT_TRACE", [301] = "SAVE_IP", - [302] = "_GUARD_BOTH_INT", - [303] = "_BINARY_OP_MULTIPLY_INT", - [304] = "_BINARY_OP_ADD_INT", - [305] = "_BINARY_OP_SUBTRACT_INT", - [306] = "_GUARD_BOTH_FLOAT", - [307] = "_BINARY_OP_MULTIPLY_FLOAT", - [308] = "_BINARY_OP_ADD_FLOAT", - [309] = "_BINARY_OP_SUBTRACT_FLOAT", - [310] = "_GUARD_BOTH_UNICODE", - [311] = "_BINARY_OP_ADD_UNICODE", - [312] = "_LOAD_LOCALS", - [313] = "_LOAD_FROM_DICT_OR_GLOBALS", + [302] = "_POP_JUMP_IF_FALSE", + [303] = "_POP_JUMP_IF_TRUE", + [304] = "_GUARD_BOTH_INT", + [305] = "_BINARY_OP_MULTIPLY_INT", + [306] = "_BINARY_OP_ADD_INT", + [307] = "_BINARY_OP_SUBTRACT_INT", + [308] = "_GUARD_BOTH_FLOAT", + [309] = "_BINARY_OP_MULTIPLY_FLOAT", + [310] = "_BINARY_OP_ADD_FLOAT", + [311] = "_BINARY_OP_SUBTRACT_FLOAT", + [312] = "_GUARD_BOTH_UNICODE", + [313] = "_BINARY_OP_ADD_UNICODE", + [314] = "_LOAD_LOCALS", + [315] = "_LOAD_FROM_DICT_OR_GLOBALS", }; #endif // NEED_OPCODE_METADATA #endif diff --git a/Python/optimizer.c b/Python/optimizer.c index 1d731ed2309c3..48c29f55bee46 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -307,7 +307,7 @@ uop_dealloc(_PyUOpExecutorObject *self) { static const char * uop_name(int index) { - if (index < EXIT_TRACE) { + if (index < 256) { return _PyOpcode_OpName[index]; } return _PyOpcode_uop_name[index]; @@ -316,9 +316,9 @@ uop_name(int index) { static Py_ssize_t uop_len(_PyUOpExecutorObject *self) { - int count = 1; + int count = 0; for (; count < _Py_UOP_MAX_TRACE_LENGTH; count++) { - if (self->trace[count-1].opcode == EXIT_TRACE) { + if (self->trace[count].opcode == 0) { break; } } @@ -328,28 +328,26 @@ uop_len(_PyUOpExecutorObject *self) static PyObject * uop_item(_PyUOpExecutorObject *self, Py_ssize_t index) { - for (int i = 0; i < _Py_UOP_MAX_TRACE_LENGTH; i++) { - if (self->trace[i].opcode == EXIT_TRACE) { - break; - } - if (i != index) { - continue; - } - const char *name = uop_name(self->trace[i].opcode); - PyObject *oname = _PyUnicode_FromASCII(name, strlen(name)); - if (oname == NULL) { - return NULL; - } - PyObject *operand = PyLong_FromUnsignedLongLong(self->trace[i].operand); - if (operand == NULL) { - Py_DECREF(oname); - return NULL; - } - PyObject *args[2] = { oname, operand }; - return _PyTuple_FromArraySteal(args, 2); + Py_ssize_t len = uop_len(self); + if (index < 0 || index >= len) { + PyErr_SetNone(PyExc_IndexError); + return NULL; } - PyErr_SetNone(PyExc_IndexError); - return NULL; + const char *name = uop_name(self->trace[index].opcode); + if (name == NULL) { + name = ""; + } + PyObject *oname = _PyUnicode_FromASCII(name, strlen(name)); + if (oname == NULL) { + return NULL; + } + PyObject *operand = PyLong_FromUnsignedLongLong(self->trace[index].operand); + if (operand == NULL) { + Py_DECREF(oname); + return NULL; + } + PyObject *args[2] = { oname, operand }; + return _PyTuple_FromArraySteal(args, 2); } PySequenceMethods uop_as_sequence = { @@ -372,12 +370,13 @@ translate_bytecode_to_trace( PyCodeObject *code, _Py_CODEUNIT *instr, _PyUOpInstruction *trace, - int max_length) + int buffer_size) { #ifdef Py_DEBUG _Py_CODEUNIT *initial_instr = instr; #endif int trace_length = 0; + int max_length = buffer_size; #ifdef Py_DEBUG char *uop_debug = Py_GETENV("PYTHONUOPSDEBUG"); @@ -401,6 +400,14 @@ translate_bytecode_to_trace( trace[trace_length].operand = (OPERAND); \ trace_length++; +#define ADD_TO_STUB(INDEX, OPCODE, OPERAND) \ + DPRINTF(2, " ADD_TO_STUB(%d, %s, %" PRIu64 ")\n", \ + (INDEX), \ + (OPCODE) < 256 ? _PyOpcode_OpName[(OPCODE)] : _PyOpcode_uop_name[(OPCODE)], \ + (uint64_t)(OPERAND)); \ + trace[(INDEX)].opcode = (OPCODE); \ + trace[(INDEX)].operand = (OPERAND); + DPRINTF(4, "Optimizing %s (%s:%d) at byte offset %ld\n", PyUnicode_AsUTF8(code->co_qualname), @@ -409,7 +416,7 @@ translate_bytecode_to_trace( 2 * (long)(initial_instr - (_Py_CODEUNIT *)code->co_code_adaptive)); for (;;) { - ADD_TO_TRACE(SAVE_IP, (int)(instr - (_Py_CODEUNIT *)code->co_code_adaptive)); + ADD_TO_TRACE(SAVE_IP, instr - (_Py_CODEUNIT *)code->co_code_adaptive); int opcode = instr->op.code; int oparg = instr->op.arg; int extras = 0; @@ -420,12 +427,35 @@ translate_bytecode_to_trace( oparg = (oparg << 8) | instr->op.arg; } if (opcode == ENTER_EXECUTOR) { - _PyExecutorObject *executor = (_PyExecutorObject *)code->co_executors->executors[oparg&255]; + _PyExecutorObject *executor = + (_PyExecutorObject *)code->co_executors->executors[oparg&255]; opcode = executor->vm_data.opcode; DPRINTF(2, " * ENTER_EXECUTOR -> %s\n", _PyOpcode_OpName[opcode]); oparg = (oparg & 0xffffff00) | executor->vm_data.oparg; } switch (opcode) { + + case POP_JUMP_IF_FALSE: + case POP_JUMP_IF_TRUE: + { + // Assume jump unlikely (TODO: handle jump likely case) + // Reserve 5 entries (1 here, 2 stub, plus SAVE_IP + EXIT_TRACE) + if (trace_length + 5 > max_length) { + DPRINTF(1, "Ran out of space for POP_JUMP_IF_FALSE\n"); + goto done; + } + _Py_CODEUNIT *target_instr = + instr + 1 + _PyOpcode_Caches[_PyOpcode_Deopt[opcode]] + oparg; + max_length -= 2; // Really the start of the stubs + int uopcode = opcode == POP_JUMP_IF_TRUE ? + _POP_JUMP_IF_TRUE : _POP_JUMP_IF_FALSE; + ADD_TO_TRACE(uopcode, max_length); + ADD_TO_STUB(max_length, SAVE_IP, + target_instr - (_Py_CODEUNIT *)code->co_code_adaptive); + ADD_TO_STUB(max_length + 1, EXIT_TRACE, 0); + break; + } + default: { const struct opcode_macro_expansion *expansion = &_PyOpcode_macro_expansion[opcode]; @@ -503,6 +533,30 @@ translate_bytecode_to_trace( code->co_firstlineno, 2 * (long)(initial_instr - (_Py_CODEUNIT *)code->co_code_adaptive), trace_length); + if (max_length < buffer_size && trace_length < max_length) { + // Move the stubs back to be immediately after the main trace + // (which ends at trace_length) + DPRINTF(2, + "Moving %d stub uops back by %d\n", + buffer_size - max_length, + max_length - trace_length); + memmove(trace + trace_length, + trace + max_length, + (buffer_size - max_length) * sizeof(_PyUOpInstruction)); + // Patch up the jump targets + for (int i = 0; i < trace_length; i++) { + if (trace[i].opcode == _POP_JUMP_IF_FALSE || + trace[i].opcode == _POP_JUMP_IF_TRUE) + { + int target = trace[i].operand; + if (target >= max_length) { + target += trace_length - max_length; + trace[i].operand = target; + } + } + } + trace_length += buffer_size - max_length; + } return trace_length; } else { @@ -539,6 +593,9 @@ uop_optimize( } executor->base.execute = _PyUopExecute; memcpy(executor->trace, trace, trace_length * sizeof(_PyUOpInstruction)); + if (trace_length < _Py_UOP_MAX_TRACE_LENGTH) { + executor->trace[trace_length].opcode = 0; // Sentinel + } *exec_ptr = (_PyExecutorObject *)executor; return 1; } diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 14269ca8cbe75..932d0c14d398a 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -1338,12 +1338,17 @@ def write_pseudo_instrs(self) -> None: def write_uop_items(self, make_text: typing.Callable[[str, int], str]) -> None: """Write '#define XXX NNN' for each uop""" counter = 300 # TODO: Avoid collision with pseudo instructions + def add(name: str) -> None: nonlocal counter self.out.emit(make_text(name, counter)) counter += 1 + add("EXIT_TRACE") add("SAVE_IP") + add("_POP_JUMP_IF_FALSE") + add("_POP_JUMP_IF_TRUE") + for instr in self.instrs.values(): if instr.kind == "op" and instr.is_viable_uop(): add(instr.name) From webhook-mailer at python.org Mon Jul 10 19:38:45 2023 From: webhook-mailer at python.org (gpshead) Date: Mon, 10 Jul 2023 23:38:45 -0000 Subject: [Python-checkins] [3.12] gh-102988: Detect email address parsing errors and return empty tuple to indicate the parsing error (old API) (GH-105127) (#106612) Message-ID: https://github.com/python/cpython/commit/30870c834ce0e47cc689b56aed7ac468f6e1c1ad commit: 30870c834ce0e47cc689b56aed7ac468f6e1c1ad branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gpshead date: 2023-07-10T23:38:42Z summary: [3.12] gh-102988: Detect email address parsing errors and return empty tuple to indicate the parsing error (old API) (GH-105127) (#106612) gh-102988: Detect email address parsing errors and return empty tuple to indicate the parsing error (old API) (GH-105127) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). This fixes or at least ameliorates CVE-2023-27043. --------- (cherry picked from commit 18dfbd035775c15533d13a98e56b1d2bf5c65f00) Co-authored-by: Thomas Dwyer Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst M Doc/library/email.utils.rst M Doc/whatsnew/3.12.rst M Lib/email/utils.py M Lib/test/test_email/test_email.py diff --git a/Doc/library/email.utils.rst b/Doc/library/email.utils.rst index 345b64001c1ac..a87a0bd2e7de6 100644 --- a/Doc/library/email.utils.rst +++ b/Doc/library/email.utils.rst @@ -65,6 +65,11 @@ of the new API. *email address* parts. Returns a tuple of that information, unless the parse fails, in which case a 2-tuple of ``('', '')`` is returned. + .. versionchanged:: 3.12 + For security reasons, addresses that were ambiguous and could parse into + multiple different addresses now cause ``('', '')`` to be returned + instead of only one of the *potential* addresses. + .. function:: formataddr(pair, charset='utf-8') @@ -87,7 +92,7 @@ of the new API. This method returns a list of 2-tuples of the form returned by ``parseaddr()``. *fieldvalues* is a sequence of header field values as might be returned by :meth:`Message.get_all `. Here's a simple - example that gets all the recipients of a message:: + example that gets all the recipients of a message: from email.utils import getaddresses @@ -97,6 +102,25 @@ of the new API. resent_ccs = msg.get_all('resent-cc', []) all_recipients = getaddresses(tos + ccs + resent_tos + resent_ccs) + When parsing fails for a single fieldvalue, a 2-tuple of ``('', '')`` + is returned in its place. Other errors in parsing the list of + addresses such as a fieldvalue seemingly parsing into multiple + addresses may result in a list containing a single empty 2-tuple + ``[('', '')]`` being returned rather than returning potentially + invalid output. + + Example malformed input parsing: + + .. doctest:: + + >>> from email.utils import getaddresses + >>> getaddresses(['alice at example.com ', 'me at example.com']) + [('', '')] + + .. versionchanged:: 3.12 + The 2-tuple of ``('', '')`` in the returned values when parsing + fails were added as to address a security issue. + .. function:: parsedate(date) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index eba3e226fa372..ef00f097b8f0e 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -570,6 +570,14 @@ dis :data:`~dis.hasarg` collection instead. (Contributed by Irit Katriel in :gh:`94216`.) +email +----- + +* :func:`email.utils.getaddresses` and :func:`email.utils.parseaddr` now return + ``('', '')`` 2-tuples in more situations where invalid email addresses are + encountered instead of potentially inaccurate values. + (Contributed by Thomas Dwyer for :gh:`102988` to ameliorate CVE-2023-27043.) + fractions --------- diff --git a/Lib/email/utils.py b/Lib/email/utils.py index 81da5394ea169..11ad75e94e934 100644 --- a/Lib/email/utils.py +++ b/Lib/email/utils.py @@ -106,12 +106,54 @@ def formataddr(pair, charset='utf-8'): return address +def _pre_parse_validation(email_header_fields): + accepted_values = [] + for v in email_header_fields: + s = v.replace('\\(', '').replace('\\)', '') + if s.count('(') != s.count(')'): + v = "('', '')" + accepted_values.append(v) + + return accepted_values + + +def _post_parse_validation(parsed_email_header_tuples): + accepted_values = [] + # The parser would have parsed a correctly formatted domain-literal + # The existence of an [ after parsing indicates a parsing failure + for v in parsed_email_header_tuples: + if '[' in v[1]: + v = ('', '') + accepted_values.append(v) + + return accepted_values + def getaddresses(fieldvalues): - """Return a list of (REALNAME, EMAIL) for each fieldvalue.""" - all = COMMASPACE.join(str(v) for v in fieldvalues) + """Return a list of (REALNAME, EMAIL) or ('','') for each fieldvalue. + + When parsing fails for a fieldvalue, a 2-tuple of ('', '') is returned in + its place. + + If the resulting list of parsed address is not the same as the number of + fieldvalues in the input list a parsing error has occurred. A list + containing a single empty 2-tuple [('', '')] is returned in its place. + This is done to avoid invalid output. + """ + fieldvalues = [str(v) for v in fieldvalues] + fieldvalues = _pre_parse_validation(fieldvalues) + all = COMMASPACE.join(v for v in fieldvalues) a = _AddressList(all) - return a.addresslist + result = _post_parse_validation(a.addresslist) + + n = 0 + for v in fieldvalues: + n += v.count(',') + 1 + + if len(result) != n: + return [('', '')] + + return result def _format_timetuple_and_zone(timetuple, zone): @@ -212,9 +254,18 @@ def parseaddr(addr): Return a tuple of realname and email address, unless the parse fails, in which case return a 2-tuple of ('', ''). """ - addrs = _AddressList(addr).addresslist - if not addrs: - return '', '' + if isinstance(addr, list): + addr = addr[0] + + if not isinstance(addr, str): + return ('', '') + + addr = _pre_parse_validation([addr])[0] + addrs = _post_parse_validation(_AddressList(addr).addresslist) + + if not addrs or len(addrs) > 1: + return ('', '') + return addrs[0] diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index 44b405740c440..5238944d6b478 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3319,15 +3319,90 @@ def test_getaddresses(self): [('Al Person', 'aperson at dom.ain'), ('Bud Person', 'bperson at dom.ain')]) + def test_getaddresses_parsing_errors(self): + """Test for parsing errors from CVE-2023-27043""" + eq = self.assertEqual + eq(utils.getaddresses(['alice at example.org(']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org)']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org<']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org>']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org@']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org,']), + [('', 'alice at example.org'), ('', 'bob at example.com')]) + eq(utils.getaddresses(['alice at example.org;']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org:']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org.']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org"']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org[']), + [('', '')]) + eq(utils.getaddresses(['alice at example.org]']), + [('', '')]) + + def test_parseaddr_parsing_errors(self): + """Test for parsing errors from CVE-2023-27043""" + eq = self.assertEqual + eq(utils.parseaddr(['alice at example.org(']), + ('', '')) + eq(utils.parseaddr(['alice at example.org)']), + ('', '')) + eq(utils.parseaddr(['alice at example.org<']), + ('', '')) + eq(utils.parseaddr(['alice at example.org>']), + ('', '')) + eq(utils.parseaddr(['alice at example.org@']), + ('', '')) + eq(utils.parseaddr(['alice at example.org,']), + ('', '')) + eq(utils.parseaddr(['alice at example.org;']), + ('', '')) + eq(utils.parseaddr(['alice at example.org:']), + ('', '')) + eq(utils.parseaddr(['alice at example.org.']), + ('', '')) + eq(utils.parseaddr(['alice at example.org"']), + ('', '')) + eq(utils.parseaddr(['alice at example.org[']), + ('', '')) + eq(utils.parseaddr(['alice at example.org]']), + ('', '')) + def test_getaddresses_nasty(self): eq = self.assertEqual eq(utils.getaddresses(['foo: ;']), [('', '')]) - eq(utils.getaddresses( - ['[]*-- =~$']), - [('', ''), ('', ''), ('', '*--')]) + eq(utils.getaddresses(['[]*-- =~$']), [('', '')]) eq(utils.getaddresses( ['foo: ;', '"Jason R. Mastaler" ']), [('', ''), ('Jason R. Mastaler', 'jason at dom.ain')]) + eq(utils.getaddresses( + [r'Pete(A nice \) chap) ']), + [('Pete (A nice ) chap his account his host)', 'pete at silly.test')]) + eq(utils.getaddresses( + ['(Empty list)(start)Undisclosed recipients :(nobody(I know))']), + [('', '')]) + eq(utils.getaddresses( + ['Mary <@machine.tld:mary at example.net>, , jdoe at test . example']), + [('Mary', 'mary at example.net'), ('', ''), ('', 'jdoe at test.example')]) + eq(utils.getaddresses( + ['John Doe ']), + [('John Doe (comment)', 'jdoe at machine.example')]) + eq(utils.getaddresses( + ['"Mary Smith: Personal Account" ']), + [('Mary Smith: Personal Account', 'smith at home.example')]) + eq(utils.getaddresses( + ['Undisclosed recipients:;']), + [('', '')]) + eq(utils.getaddresses( + [r', "Giant; \"Big\" Box" ']), + [('', 'boss at nil.test'), ('Giant; "Big" Box', 'bob at example.net')]) def test_getaddresses_embedded_comment(self): """Test proper handling of a nested comment""" diff --git a/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst b/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst new file mode 100644 index 0000000000000..e0434ccd2ccab --- /dev/null +++ b/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst @@ -0,0 +1,4 @@ +CVE-2023-27043: Prevent :func:`email.utils.parseaddr` +and :func:`email.utils.getaddresses` from returning the realname portion of an +invalid RFC2822 email header in the email address portion of the 2-tuple +returned after being parsed by :class:`email._parseaddr.AddressList`. From webhook-mailer at python.org Mon Jul 10 21:14:56 2023 From: webhook-mailer at python.org (corona10) Date: Tue, 11 Jul 2023 01:14:56 -0000 Subject: [Python-checkins] gh-104635: Add a test case for variables that have a dependency. (gh-106583) Message-ID: https://github.com/python/cpython/commit/115df8491a6633ced3cc3f2343b349869de30b8c commit: 115df8491a6633ced3cc3f2343b349869de30b8c branch: main author: Dong-hee Na committer: corona10 date: 2023-07-11T10:14:53+09:00 summary: gh-104635: Add a test case for variables that have a dependency. (gh-106583) files: M Lib/test/test_compile.py diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py index 784c0550cc09b..85ce0a4b39d85 100644 --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -1186,6 +1186,15 @@ def f(x, y, z): return a self.assertEqual(f("x", "y", "z"), "y") + def test_variable_dependent(self): + # gh-104635: Since the value of b is dependent on the value of a + # the first STORE_FAST for a should not be skipped. (e.g POP_TOP). + # This test case is added to prevent potential regression from aggressive optimization. + def f(): + a = 42; b = a + 54; a = 54 + return a, b + self.assertEqual(f(), (54, 96)) + @requires_debug_ranges() class TestSourcePositions(unittest.TestCase): From webhook-mailer at python.org Mon Jul 10 22:12:35 2023 From: webhook-mailer at python.org (gvanrossum) Date: Tue, 11 Jul 2023 02:12:35 -0000 Subject: [Python-checkins] gh-106529: Silence compiler warning in jump target patching (#106613) Message-ID: https://github.com/python/cpython/commit/4bd8320dd7922d529eab51753dd524e8bf9c47b2 commit: 4bd8320dd7922d529eab51753dd524e8bf9c47b2 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-10T19:12:32-07:00 summary: gh-106529: Silence compiler warning in jump target patching (#106613) (gh-106551 caused a compiler warning about on Windows.) files: M Python/optimizer.c diff --git a/Python/optimizer.c b/Python/optimizer.c index 48c29f55bee46..08073193c0228 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -548,8 +548,8 @@ translate_bytecode_to_trace( if (trace[i].opcode == _POP_JUMP_IF_FALSE || trace[i].opcode == _POP_JUMP_IF_TRUE) { - int target = trace[i].operand; - if (target >= max_length) { + uint64_t target = trace[i].operand; + if (target >= (uint64_t)max_length) { target += trace_length - max_length; trace[i].operand = target; } From webhook-mailer at python.org Tue Jul 11 04:52:16 2023 From: webhook-mailer at python.org (cjw296) Date: Tue, 11 Jul 2023 08:52:16 -0000 Subject: [Python-checkins] Remove unused branches from mock module (#106617) Message-ID: https://github.com/python/cpython/commit/e6379f72cbc60f6b3c5676f9e225d4f145d5693f commit: e6379f72cbc60f6b3c5676f9e225d4f145d5693f branch: main author: Chris Withers committer: cjw296 date: 2023-07-11T08:52:12Z summary: Remove unused branches from mock module (#106617) * lambda has a name of __none__, but no async lambda so this branch is not needed * _get_signature_object only returns None for bound builtins. There are no async builtins so this branch isn't needed * Exclude a couple of methods from coverage checking in the downstream rolling backport of mock files: M Lib/test/test_unittest/testmock/testthreadingmock.py M Lib/unittest/mock.py diff --git a/Lib/test/test_unittest/testmock/testthreadingmock.py b/Lib/test/test_unittest/testmock/testthreadingmock.py index c6f179490d01f..b6e12bcb3cda9 100644 --- a/Lib/test/test_unittest/testmock/testthreadingmock.py +++ b/Lib/test/test_unittest/testmock/testthreadingmock.py @@ -11,10 +11,10 @@ class Something: def method_1(self): - pass + pass # pragma: no cover def method_2(self): - pass + pass # pragma: no cover class TestThreadingMock(unittest.TestCase): diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py index 3ed54b3ba230e..e8c8360e0ae13 100644 --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -212,17 +212,12 @@ def _set_async_signature(mock, original, instance=False, is_async_mock=False): # signature as the original. skipfirst = isinstance(original, type) - result = _get_signature_object(original, instance, skipfirst) - if result is None: - return mock - func, sig = result + func, sig = _get_signature_object(original, instance, skipfirst) def checksig(*args, **kwargs): sig.bind(*args, **kwargs) _copy_func_details(func, checksig) name = original.__name__ - if not name.isidentifier(): - name = 'funcopy' context = {'_checksig_': checksig, 'mock': mock} src = """async def %s(*args, **kwargs): _checksig_(*args, **kwargs) From webhook-mailer at python.org Tue Jul 11 05:38:26 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 11 Jul 2023 09:38:26 -0000 Subject: [Python-checkins] gh-106572: Convert PyObject_DelAttr() to a function (#106611) Message-ID: https://github.com/python/cpython/commit/1f2921b72c369b19c2e32aaedb9f8c63e0cb8b48 commit: 1f2921b72c369b19c2e32aaedb9f8c63e0cb8b48 branch: main author: Victor Stinner committer: vstinner date: 2023-07-11T11:38:22+02:00 summary: gh-106572: Convert PyObject_DelAttr() to a function (#106611) * Convert PyObject_DelAttr() and PyObject_DelAttrString() macros to functions. * Add PyObject_DelAttr() and PyObject_DelAttrString() functions to the stable ABI. * Replace PyObject_SetAttr(obj, name, NULL) with PyObject_DelAttr(obj, name). files: A Misc/NEWS.d/next/C API/2023-07-11-01-07-39.gh-issue-106572.y1b35X.rst M Doc/data/stable_abi.dat M Include/abstract.h M Include/object.h M Lib/test/test_stable_abi_ctypes.py M Misc/stable_abi.toml M Objects/object.c M PC/python3dll.c M Python/bltinmodule.c M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index 7fb002cd80369..9a61ddc39a353 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -490,6 +490,8 @@ function,PyObject_Calloc,3.7,, function,PyObject_CheckBuffer,3.11,, function,PyObject_ClearWeakRefs,3.2,, function,PyObject_CopyData,3.11,, +function,PyObject_DelAttr,3.13,, +function,PyObject_DelAttrString,3.13,, function,PyObject_DelItem,3.2,, function,PyObject_DelItemString,3.2,, function,PyObject_Dir,3.2,, diff --git a/Include/abstract.h b/Include/abstract.h index 016ace9bc89e9..c84d2c704e960 100644 --- a/Include/abstract.h +++ b/Include/abstract.h @@ -80,7 +80,7 @@ extern "C" { This is the equivalent of the Python statement o.attr_name=v. */ -/* Implemented as a macro: +/* Implemented elsewhere: int PyObject_DelAttrString(PyObject *o, const char *attr_name); @@ -88,17 +88,15 @@ extern "C" { -1 on failure. This is the equivalent of the Python statement: del o.attr_name. */ -#define PyObject_DelAttrString(O, A) PyObject_SetAttrString((O), (A), NULL) -/* Implemented as a macro: +/* Implemented elsewhere: int PyObject_DelAttr(PyObject *o, PyObject *attr_name); Delete attribute named attr_name, for object o. Returns -1 on failure. This is the equivalent of the Python statement: del o.attr_name. */ -#define PyObject_DelAttr(O, A) PyObject_SetAttr((O), (A), NULL) /* Implemented elsewhere: diff --git a/Include/object.h b/Include/object.h index 3ef64511399c6..dccab07e5f2c6 100644 --- a/Include/object.h +++ b/Include/object.h @@ -391,9 +391,11 @@ PyAPI_FUNC(PyObject *) PyObject_RichCompare(PyObject *, PyObject *, int); PyAPI_FUNC(int) PyObject_RichCompareBool(PyObject *, PyObject *, int); PyAPI_FUNC(PyObject *) PyObject_GetAttrString(PyObject *, const char *); PyAPI_FUNC(int) PyObject_SetAttrString(PyObject *, const char *, PyObject *); +PyAPI_FUNC(int) PyObject_DelAttrString(PyObject *v, const char *name); PyAPI_FUNC(int) PyObject_HasAttrString(PyObject *, const char *); PyAPI_FUNC(PyObject *) PyObject_GetAttr(PyObject *, PyObject *); PyAPI_FUNC(int) PyObject_SetAttr(PyObject *, PyObject *, PyObject *); +PyAPI_FUNC(int) PyObject_DelAttr(PyObject *v, PyObject *name); PyAPI_FUNC(int) PyObject_HasAttr(PyObject *, PyObject *); PyAPI_FUNC(PyObject *) PyObject_SelfIter(PyObject *); PyAPI_FUNC(PyObject *) PyObject_GenericGetAttr(PyObject *, PyObject *); diff --git a/Lib/test/test_stable_abi_ctypes.py b/Lib/test/test_stable_abi_ctypes.py index 038c978e7bbd0..20bc2624c8136 100644 --- a/Lib/test/test_stable_abi_ctypes.py +++ b/Lib/test/test_stable_abi_ctypes.py @@ -509,6 +509,8 @@ def test_windows_feature_macros(self): "PyObject_CheckReadBuffer", "PyObject_ClearWeakRefs", "PyObject_CopyData", + "PyObject_DelAttr", + "PyObject_DelAttrString", "PyObject_DelItem", "PyObject_DelItemString", "PyObject_Dir", diff --git a/Misc/NEWS.d/next/C API/2023-07-11-01-07-39.gh-issue-106572.y1b35X.rst b/Misc/NEWS.d/next/C API/2023-07-11-01-07-39.gh-issue-106572.y1b35X.rst new file mode 100644 index 0000000000000..140e9fe7b9abf --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-11-01-07-39.gh-issue-106572.y1b35X.rst @@ -0,0 +1,2 @@ +Convert :c:func:`PyObject_DelAttr` and :c:func:`PyObject_DelAttrString` +macros to functions. Patch by Victor Stinner. diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml index bc7259f11816f..c61fedf8390e2 100644 --- a/Misc/stable_abi.toml +++ b/Misc/stable_abi.toml @@ -2432,3 +2432,7 @@ added = '3.13' [function.PyWeakref_GetRef] added = '3.13' +[function.PyObject_DelAttr] + added = '3.13' +[function.PyObject_DelAttrString] + added = '3.13' diff --git a/Objects/object.c b/Objects/object.c index c27b13e9e0c31..540ba5d07427c 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -942,6 +942,12 @@ PyObject_SetAttrString(PyObject *v, const char *name, PyObject *w) return res; } +int +PyObject_DelAttrString(PyObject *v, const char *name) +{ + return PyObject_SetAttrString(v, name, NULL); +} + int _PyObject_IsAbstract(PyObject *obj) { @@ -1185,6 +1191,12 @@ PyObject_SetAttr(PyObject *v, PyObject *name, PyObject *value) return -1; } +int +PyObject_DelAttr(PyObject *v, PyObject *name) +{ + return PyObject_SetAttr(v, name, NULL); +} + PyObject ** _PyObject_ComputedDictPointer(PyObject *obj) { diff --git a/PC/python3dll.c b/PC/python3dll.c index 65bdf326ffbc7..a7173911c7c1e 100755 --- a/PC/python3dll.c +++ b/PC/python3dll.c @@ -447,6 +447,8 @@ EXPORT_FUNC(PyObject_CheckBuffer) EXPORT_FUNC(PyObject_CheckReadBuffer) EXPORT_FUNC(PyObject_ClearWeakRefs) EXPORT_FUNC(PyObject_CopyData) +EXPORT_FUNC(PyObject_DelAttr) +EXPORT_FUNC(PyObject_DelAttrString) EXPORT_FUNC(PyObject_DelItem) EXPORT_FUNC(PyObject_DelItemString) EXPORT_FUNC(PyObject_Dir) diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c index 49efafc07f424..20a86fc6f583d 100644 --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -1567,8 +1567,9 @@ static PyObject * builtin_delattr_impl(PyObject *module, PyObject *obj, PyObject *name) /*[clinic end generated code: output=85134bc58dff79fa input=164865623abe7216]*/ { - if (PyObject_SetAttr(obj, name, (PyObject *)NULL) != 0) + if (PyObject_DelAttr(obj, name) < 0) { return NULL; + } Py_RETURN_NONE; } diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 0848bbfd203ec..88444293aa228 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1240,7 +1240,7 @@ dummy_func( inst(DELETE_ATTR, (owner --)) { PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); - int err = PyObject_SetAttr(owner, name, (PyObject *)NULL); + int err = PyObject_DelAttr(owner, name); DECREF_INPUTS(); ERROR_IF(err, error); } diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index eccb30348173a..030d7ac15ef06 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1018,7 +1018,7 @@ PyObject *owner = stack_pointer[-1]; #line 1242 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); - int err = PyObject_SetAttr(owner, name, (PyObject *)NULL); + int err = PyObject_DelAttr(owner, name); #line 1023 "Python/executor_cases.c.h" Py_DECREF(owner); #line 1245 "Python/bytecodes.c" diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 11823cf9cd293..ecbf8ee59da1d 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -1696,7 +1696,7 @@ PyObject *owner = stack_pointer[-1]; #line 1242 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); - int err = PyObject_SetAttr(owner, name, (PyObject *)NULL); + int err = PyObject_DelAttr(owner, name); #line 1701 "Python/generated_cases.c.h" Py_DECREF(owner); #line 1245 "Python/bytecodes.c" From webhook-mailer at python.org Tue Jul 11 06:34:03 2023 From: webhook-mailer at python.org (markshannon) Date: Tue, 11 Jul 2023 10:34:03 -0000 Subject: [Python-checkins] GH-106529: Define POP_JUMP_IF_NONE in terms of POP_JUMP_IF_TRUE (GH-106599) Message-ID: https://github.com/python/cpython/commit/c0c041a31ba6a8d2da993a475a56b7d8211fdbf2 commit: c0c041a31ba6a8d2da993a475a56b7d8211fdbf2 branch: main author: Mark Shannon committer: markshannon date: 2023-07-11T11:33:59+01:00 summary: GH-106529: Define POP_JUMP_IF_NONE in terms of POP_JUMP_IF_TRUE (GH-106599) files: M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/opcode_metadata.h diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 88444293aa228..3ba0d0f7e292f 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2282,22 +2282,20 @@ dummy_func( JUMPBY(oparg * Py_IsTrue(cond)); } - inst(POP_JUMP_IF_NOT_NONE, (value -- )) { - if (!Py_IsNone(value)) { - DECREF_INPUTS(); - JUMPBY(oparg); - } - } - - inst(POP_JUMP_IF_NONE, (value -- )) { + op(IS_NONE, (value -- b)) { if (Py_IsNone(value)) { - JUMPBY(oparg); + b = Py_True; } else { + b = Py_False; DECREF_INPUTS(); } } + macro(POP_JUMP_IF_NONE) = IS_NONE + POP_JUMP_IF_TRUE; + + macro(POP_JUMP_IF_NOT_NONE) = IS_NONE + POP_JUMP_IF_FALSE; + inst(JUMP_BACKWARD_NO_INTERRUPT, (--)) { /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 030d7ac15ef06..ba854be7f6c05 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1899,16 +1899,34 @@ break; } + case IS_NONE: { + PyObject *value = stack_pointer[-1]; + PyObject *b; + #line 2286 "Python/bytecodes.c" + if (Py_IsNone(value)) { + b = Py_True; + } + else { + b = Py_False; + #line 1912 "Python/executor_cases.c.h" + Py_DECREF(value); + #line 2292 "Python/bytecodes.c" + } + #line 1916 "Python/executor_cases.c.h" + stack_pointer[-1] = b; + break; + } + case GET_LEN: { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2311 "Python/bytecodes.c" + #line 2309 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 1912 "Python/executor_cases.c.h" + #line 1930 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; break; @@ -1919,16 +1937,16 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2319 "Python/bytecodes.c" + #line 2317 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 1928 "Python/executor_cases.c.h" + #line 1946 "Python/executor_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2324 "Python/bytecodes.c" + #line 2322 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -1936,7 +1954,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 1940 "Python/executor_cases.c.h" + #line 1958 "Python/executor_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; break; @@ -1945,10 +1963,10 @@ case MATCH_MAPPING: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2334 "Python/bytecodes.c" + #line 2332 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 1952 "Python/executor_cases.c.h" + #line 1970 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1957,10 +1975,10 @@ case MATCH_SEQUENCE: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2339 "Python/bytecodes.c" + #line 2337 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 1964 "Python/executor_cases.c.h" + #line 1982 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1970,11 +1988,11 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2344 "Python/bytecodes.c" + #line 2342 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 1978 "Python/executor_cases.c.h" + #line 1996 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; break; @@ -1983,14 +2001,14 @@ case GET_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2350 "Python/bytecodes.c" + #line 2348 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 1990 "Python/executor_cases.c.h" + #line 2008 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2353 "Python/bytecodes.c" + #line 2351 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 1994 "Python/executor_cases.c.h" + #line 2012 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -1998,7 +2016,7 @@ case GET_YIELD_FROM_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2357 "Python/bytecodes.c" + #line 2355 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -2021,11 +2039,11 @@ if (iter == NULL) { goto error; } - #line 2025 "Python/executor_cases.c.h" + #line 2043 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2380 "Python/bytecodes.c" + #line 2378 "Python/bytecodes.c" } - #line 2029 "Python/executor_cases.c.h" + #line 2047 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -2035,7 +2053,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2612 "Python/bytecodes.c" + #line 2610 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -2056,7 +2074,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 2060 "Python/executor_cases.c.h" + #line 2078 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -2065,7 +2083,7 @@ case PUSH_EXC_INFO: { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2651 "Python/bytecodes.c" + #line 2649 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -2075,7 +2093,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 2079 "Python/executor_cases.c.h" + #line 2097 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -2084,7 +2102,7 @@ case EXIT_INIT_CHECK: { PyObject *should_be_none = stack_pointer[-1]; - #line 3050 "Python/bytecodes.c" + #line 3048 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -2092,7 +2110,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 2096 "Python/executor_cases.c.h" + #line 2114 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -2100,7 +2118,7 @@ case MAKE_FUNCTION: { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3464 "Python/bytecodes.c" + #line 3462 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -2112,7 +2130,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 2116 "Python/executor_cases.c.h" + #line 2134 "Python/executor_cases.c.h" stack_pointer[-1] = func; break; } @@ -2120,7 +2138,7 @@ case SET_FUNCTION_ATTRIBUTE: { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3478 "Python/bytecodes.c" + #line 3476 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -2145,7 +2163,7 @@ default: Py_UNREACHABLE(); } - #line 2149 "Python/executor_cases.c.h" + #line 2167 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; break; @@ -2156,15 +2174,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3528 "Python/bytecodes.c" + #line 3526 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 2162 "Python/executor_cases.c.h" + #line 2180 "Python/executor_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3530 "Python/bytecodes.c" + #line 3528 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 2168 "Python/executor_cases.c.h" + #line 2186 "Python/executor_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -2174,14 +2192,14 @@ case CONVERT_VALUE: { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3534 "Python/bytecodes.c" + #line 3532 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 2185 "Python/executor_cases.c.h" + #line 2203 "Python/executor_cases.c.h" stack_pointer[-1] = result; break; } @@ -2189,7 +2207,7 @@ case FORMAT_SIMPLE: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3543 "Python/bytecodes.c" + #line 3541 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -2200,7 +2218,7 @@ else { res = value; } - #line 2204 "Python/executor_cases.c.h" + #line 2222 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -2209,12 +2227,12 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3556 "Python/bytecodes.c" + #line 3554 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 2218 "Python/executor_cases.c.h" + #line 2236 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -2223,10 +2241,10 @@ case COPY: { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3563 "Python/bytecodes.c" + #line 3561 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 2230 "Python/executor_cases.c.h" + #line 2248 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; break; @@ -2237,7 +2255,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3568 "Python/bytecodes.c" + #line 3566 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2252,12 +2270,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 2256 "Python/executor_cases.c.h" + #line 2274 "Python/executor_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3583 "Python/bytecodes.c" + #line 3581 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 2261 "Python/executor_cases.c.h" + #line 2279 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -2266,9 +2284,9 @@ case SWAP: { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3588 "Python/bytecodes.c" + #line 3586 "Python/bytecodes.c" assert(oparg >= 2); - #line 2272 "Python/executor_cases.c.h" + #line 2290 "Python/executor_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; break; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index ecbf8ee59da1d..325090a60f9d5 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3243,58 +3243,86 @@ DISPATCH(); } - TARGET(POP_JUMP_IF_NOT_NONE) { - PyObject *value = stack_pointer[-1]; - #line 2286 "Python/bytecodes.c" - if (!Py_IsNone(value)) { - #line 3251 "Python/generated_cases.c.h" - Py_DECREF(value); - #line 2288 "Python/bytecodes.c" - JUMPBY(oparg); + TARGET(POP_JUMP_IF_NONE) { + PyObject *_tmp_1 = stack_pointer[-1]; + { + PyObject *value = _tmp_1; + PyObject *b; + #line 2286 "Python/bytecodes.c" + if (Py_IsNone(value)) { + b = Py_True; + } + else { + b = Py_False; + #line 3258 "Python/generated_cases.c.h" + Py_DECREF(value); + #line 2292 "Python/bytecodes.c" + } + #line 3262 "Python/generated_cases.c.h" + _tmp_1 = b; + } + { + PyObject *cond = _tmp_1; + #line 2281 "Python/bytecodes.c" + assert(PyBool_Check(cond)); + JUMPBY(oparg * Py_IsTrue(cond)); + #line 3270 "Python/generated_cases.c.h" } - #line 3256 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } - TARGET(POP_JUMP_IF_NONE) { - PyObject *value = stack_pointer[-1]; - #line 2293 "Python/bytecodes.c" - if (Py_IsNone(value)) { - JUMPBY(oparg); + TARGET(POP_JUMP_IF_NOT_NONE) { + PyObject *_tmp_1 = stack_pointer[-1]; + { + PyObject *value = _tmp_1; + PyObject *b; + #line 2286 "Python/bytecodes.c" + if (Py_IsNone(value)) { + b = Py_True; + } + else { + b = Py_False; + #line 3287 "Python/generated_cases.c.h" + Py_DECREF(value); + #line 2292 "Python/bytecodes.c" + } + #line 3291 "Python/generated_cases.c.h" + _tmp_1 = b; } - else { - #line 3268 "Python/generated_cases.c.h" - Py_DECREF(value); - #line 2298 "Python/bytecodes.c" + { + PyObject *cond = _tmp_1; + #line 2276 "Python/bytecodes.c" + assert(PyBool_Check(cond)); + JUMPBY(oparg * Py_IsFalse(cond)); + #line 3299 "Python/generated_cases.c.h" } - #line 3272 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(JUMP_BACKWARD_NO_INTERRUPT) { - #line 2302 "Python/bytecodes.c" + #line 2300 "Python/bytecodes.c" /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost * generator or coroutine, so we deliberately do not check it here. * (see bpo-30039). */ JUMPBY(-oparg); - #line 3285 "Python/generated_cases.c.h" + #line 3313 "Python/generated_cases.c.h" DISPATCH(); } TARGET(GET_LEN) { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2311 "Python/bytecodes.c" + #line 2309 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 3298 "Python/generated_cases.c.h" + #line 3326 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; DISPATCH(); @@ -3305,16 +3333,16 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2319 "Python/bytecodes.c" + #line 2317 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 3314 "Python/generated_cases.c.h" + #line 3342 "Python/generated_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2324 "Python/bytecodes.c" + #line 2322 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -3322,7 +3350,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 3326 "Python/generated_cases.c.h" + #line 3354 "Python/generated_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; DISPATCH(); @@ -3331,10 +3359,10 @@ TARGET(MATCH_MAPPING) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2334 "Python/bytecodes.c" + #line 2332 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 3338 "Python/generated_cases.c.h" + #line 3366 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3343,10 +3371,10 @@ TARGET(MATCH_SEQUENCE) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2339 "Python/bytecodes.c" + #line 2337 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 3350 "Python/generated_cases.c.h" + #line 3378 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3356,11 +3384,11 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2344 "Python/bytecodes.c" + #line 2342 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 3364 "Python/generated_cases.c.h" + #line 3392 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; DISPATCH(); @@ -3369,14 +3397,14 @@ TARGET(GET_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2350 "Python/bytecodes.c" + #line 2348 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 3376 "Python/generated_cases.c.h" + #line 3404 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2353 "Python/bytecodes.c" + #line 2351 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 3380 "Python/generated_cases.c.h" + #line 3408 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3384,7 +3412,7 @@ TARGET(GET_YIELD_FROM_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2357 "Python/bytecodes.c" + #line 2355 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -3407,11 +3435,11 @@ if (iter == NULL) { goto error; } - #line 3411 "Python/generated_cases.c.h" + #line 3439 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2380 "Python/bytecodes.c" + #line 2378 "Python/bytecodes.c" } - #line 3415 "Python/generated_cases.c.h" + #line 3443 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3421,7 +3449,7 @@ static_assert(INLINE_CACHE_ENTRIES_FOR_ITER == 1, "incorrect cache size"); PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2398 "Python/bytecodes.c" + #line 2396 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -3453,7 +3481,7 @@ DISPATCH(); } // Common case: no jump, leave it to the code generator - #line 3457 "Python/generated_cases.c.h" + #line 3485 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3461,7 +3489,7 @@ } TARGET(INSTRUMENTED_FOR_ITER) { - #line 2432 "Python/bytecodes.c" + #line 2430 "Python/bytecodes.c" _Py_CODEUNIT *here = next_instr-1; _Py_CODEUNIT *target; PyObject *iter = TOP(); @@ -3487,14 +3515,14 @@ target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER + oparg + 1; } INSTRUMENTED_JUMP(here, target, PY_MONITORING_EVENT_BRANCH); - #line 3491 "Python/generated_cases.c.h" + #line 3519 "Python/generated_cases.c.h" DISPATCH(); } TARGET(FOR_ITER_LIST) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2460 "Python/bytecodes.c" + #line 2458 "Python/bytecodes.c" DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit); @@ -3515,7 +3543,7 @@ DISPATCH(); end_for_iter_list: // Common case: no jump, leave it to the code generator - #line 3519 "Python/generated_cases.c.h" + #line 3547 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3525,7 +3553,7 @@ TARGET(FOR_ITER_TUPLE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2483 "Python/bytecodes.c" + #line 2481 "Python/bytecodes.c" _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3546,7 +3574,7 @@ DISPATCH(); end_for_iter_tuple: // Common case: no jump, leave it to the code generator - #line 3550 "Python/generated_cases.c.h" + #line 3578 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3556,7 +3584,7 @@ TARGET(FOR_ITER_RANGE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2506 "Python/bytecodes.c" + #line 2504 "Python/bytecodes.c" _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3575,7 +3603,7 @@ if (next == NULL) { goto error; } - #line 3579 "Python/generated_cases.c.h" + #line 3607 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3584,7 +3612,7 @@ TARGET(FOR_ITER_GEN) { PyObject *iter = stack_pointer[-1]; - #line 2527 "Python/bytecodes.c" + #line 2525 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER); @@ -3600,14 +3628,14 @@ assert(next_instr[oparg].op.code == END_FOR || next_instr[oparg].op.code == INSTRUMENTED_END_FOR); DISPATCH_INLINED(gen_frame); - #line 3604 "Python/generated_cases.c.h" + #line 3632 "Python/generated_cases.c.h" } TARGET(BEFORE_ASYNC_WITH) { PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2545 "Python/bytecodes.c" + #line 2543 "Python/bytecodes.c" PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { @@ -3630,16 +3658,16 @@ Py_DECREF(enter); goto error; } - #line 3634 "Python/generated_cases.c.h" + #line 3662 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2568 "Python/bytecodes.c" + #line 2566 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3643 "Python/generated_cases.c.h" + #line 3671 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3650,7 +3678,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2577 "Python/bytecodes.c" + #line 2575 "Python/bytecodes.c" /* pop the context manager, push its __exit__ and the * value returned from calling its __enter__ */ @@ -3676,16 +3704,16 @@ Py_DECREF(enter); goto error; } - #line 3680 "Python/generated_cases.c.h" + #line 3708 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2603 "Python/bytecodes.c" + #line 2601 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3689 "Python/generated_cases.c.h" + #line 3717 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3697,7 +3725,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2612 "Python/bytecodes.c" + #line 2610 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -3718,7 +3746,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 3722 "Python/generated_cases.c.h" + #line 3750 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3727,7 +3755,7 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2651 "Python/bytecodes.c" + #line 2649 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -3737,7 +3765,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 3741 "Python/generated_cases.c.h" + #line 3769 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -3751,7 +3779,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2663 "Python/bytecodes.c" + #line 2661 "Python/bytecodes.c" assert(oparg & 1); /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); @@ -3768,7 +3796,7 @@ res2 = Py_NewRef(descr); assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; - #line 3772 "Python/generated_cases.c.h" + #line 3800 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3782,7 +3810,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2682 "Python/bytecodes.c" + #line 2680 "Python/bytecodes.c" assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -3792,7 +3820,7 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - #line 3796 "Python/generated_cases.c.h" + #line 3824 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3807,7 +3835,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2694 "Python/bytecodes.c" + #line 2692 "Python/bytecodes.c" assert((oparg & 1) == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3820,11 +3848,11 @@ keys_version, LOAD_ATTR); STAT_INC(LOAD_ATTR, hit); assert(descr != NULL); - #line 3824 "Python/generated_cases.c.h" + #line 3852 "Python/generated_cases.c.h" Py_DECREF(self); - #line 2707 "Python/bytecodes.c" + #line 2705 "Python/bytecodes.c" res = Py_NewRef(descr); - #line 3828 "Python/generated_cases.c.h" + #line 3856 "Python/generated_cases.c.h" STACK_GROW((0 ? 1 : 0)); stack_pointer[-1] = res; if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } @@ -3838,7 +3866,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2711 "Python/bytecodes.c" + #line 2709 "Python/bytecodes.c" assert((oparg & 1) == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3846,11 +3874,11 @@ assert(self_cls->tp_dictoffset == 0); STAT_INC(LOAD_ATTR, hit); assert(descr != NULL); - #line 3850 "Python/generated_cases.c.h" + #line 3878 "Python/generated_cases.c.h" Py_DECREF(self); - #line 2719 "Python/bytecodes.c" + #line 2717 "Python/bytecodes.c" res = Py_NewRef(descr); - #line 3854 "Python/generated_cases.c.h" + #line 3882 "Python/generated_cases.c.h" STACK_GROW((0 ? 1 : 0)); stack_pointer[-1] = res; if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } @@ -3864,7 +3892,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2723 "Python/bytecodes.c" + #line 2721 "Python/bytecodes.c" assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -3878,7 +3906,7 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - #line 3882 "Python/generated_cases.c.h" + #line 3910 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3887,16 +3915,16 @@ } TARGET(KW_NAMES) { - #line 2739 "Python/bytecodes.c" + #line 2737 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(FRAME_CO_CONSTS)); kwnames = GETITEM(FRAME_CO_CONSTS, oparg); - #line 3895 "Python/generated_cases.c.h" + #line 3923 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_CALL) { - #line 2745 "Python/bytecodes.c" + #line 2743 "Python/bytecodes.c" int is_meth = PEEK(oparg+2) != NULL; int total_args = oparg + is_meth; PyObject *function = PEEK(total_args + 1); @@ -3909,7 +3937,7 @@ _PyCallCache *cache = (_PyCallCache *)next_instr; INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(CALL); - #line 3913 "Python/generated_cases.c.h" + #line 3941 "Python/generated_cases.c.h" } TARGET(CALL) { @@ -3919,7 +3947,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2791 "Python/bytecodes.c" + #line 2789 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4001,7 +4029,7 @@ Py_DECREF(args[i]); } if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4005 "Python/generated_cases.c.h" + #line 4033 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4013,7 +4041,7 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2879 "Python/bytecodes.c" + #line 2877 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit); @@ -4023,7 +4051,7 @@ PEEK(oparg + 2) = Py_NewRef(meth); // method Py_DECREF(callable); GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS); - #line 4027 "Python/generated_cases.c.h" + #line 4055 "Python/generated_cases.c.h" } TARGET(CALL_PY_EXACT_ARGS) { @@ -4032,7 +4060,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2891 "Python/bytecodes.c" + #line 2889 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4058,7 +4086,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4062 "Python/generated_cases.c.h" + #line 4090 "Python/generated_cases.c.h" } TARGET(CALL_PY_WITH_DEFAULTS) { @@ -4066,7 +4094,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2919 "Python/bytecodes.c" + #line 2917 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4102,7 +4130,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4106 "Python/generated_cases.c.h" + #line 4134 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_TYPE_1) { @@ -4110,7 +4138,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2957 "Python/bytecodes.c" + #line 2955 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4120,7 +4148,7 @@ res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(obj); Py_DECREF(&PyType_Type); // I.e., callable - #line 4124 "Python/generated_cases.c.h" + #line 4152 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4133,7 +4161,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2969 "Python/bytecodes.c" + #line 2967 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4144,7 +4172,7 @@ Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); // I.e., callable if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4148 "Python/generated_cases.c.h" + #line 4176 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4158,7 +4186,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2983 "Python/bytecodes.c" + #line 2981 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4169,7 +4197,7 @@ Py_DECREF(arg); Py_DECREF(&PyTuple_Type); // I.e., tuple if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4173 "Python/generated_cases.c.h" + #line 4201 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4182,7 +4210,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; - #line 2997 "Python/bytecodes.c" + #line 2995 "Python/bytecodes.c" /* This instruction does the following: * 1. Creates the object (by calling ``object.__new__``) * 2. Pushes a shim frame to the frame stack (to cleanup after ``__init__``) @@ -4233,12 +4261,12 @@ * as it will be checked after start_frame */ tstate->py_recursion_remaining--; goto start_frame; - #line 4237 "Python/generated_cases.c.h" + #line 4265 "Python/generated_cases.c.h" } TARGET(EXIT_INIT_CHECK) { PyObject *should_be_none = stack_pointer[-1]; - #line 3050 "Python/bytecodes.c" + #line 3048 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -4246,7 +4274,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 4250 "Python/generated_cases.c.h" + #line 4278 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -4256,7 +4284,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3060 "Python/bytecodes.c" + #line 3058 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4278,7 +4306,7 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4282 "Python/generated_cases.c.h" + #line 4310 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4292,7 +4320,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3085 "Python/bytecodes.c" + #line 3083 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4320,7 +4348,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4324 "Python/generated_cases.c.h" + #line 4352 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4334,7 +4362,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3116 "Python/bytecodes.c" + #line 3114 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4366,7 +4394,7 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 4370 "Python/generated_cases.c.h" + #line 4398 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4380,7 +4408,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3151 "Python/bytecodes.c" + #line 3149 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -4412,7 +4440,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4416 "Python/generated_cases.c.h" + #line 4444 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4426,7 +4454,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3186 "Python/bytecodes.c" + #line 3184 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -4451,7 +4479,7 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4455 "Python/generated_cases.c.h" + #line 4483 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4464,7 +4492,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3213 "Python/bytecodes.c" + #line 3211 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -4491,7 +4519,7 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4495 "Python/generated_cases.c.h" + #line 4523 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4503,7 +4531,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 3243 "Python/bytecodes.c" + #line 3241 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -4521,14 +4549,14 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 4525 "Python/generated_cases.c.h" + #line 4553 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3263 "Python/bytecodes.c" + #line 3261 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4559,7 +4587,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4563 "Python/generated_cases.c.h" + #line 4591 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4572,7 +4600,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3297 "Python/bytecodes.c" + #line 3295 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4601,7 +4629,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4605 "Python/generated_cases.c.h" + #line 4633 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4614,7 +4642,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3329 "Python/bytecodes.c" + #line 3327 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -4643,7 +4671,7 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4647 "Python/generated_cases.c.h" + #line 4675 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4656,7 +4684,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3361 "Python/bytecodes.c" + #line 3359 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4684,7 +4712,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4688 "Python/generated_cases.c.h" + #line 4716 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4694,9 +4722,9 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { - #line 3392 "Python/bytecodes.c" + #line 3390 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); - #line 4700 "Python/generated_cases.c.h" + #line 4728 "Python/generated_cases.c.h" } TARGET(CALL_FUNCTION_EX) { @@ -4705,7 +4733,7 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 3396 "Python/bytecodes.c" + #line 3394 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs)); @@ -4767,14 +4795,14 @@ } result = PyObject_Call(func, callargs, kwargs); } - #line 4771 "Python/generated_cases.c.h" + #line 4799 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 3458 "Python/bytecodes.c" + #line 3456 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4778 "Python/generated_cases.c.h" + #line 4806 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4785,7 +4813,7 @@ TARGET(MAKE_FUNCTION) { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3464 "Python/bytecodes.c" + #line 3462 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4797,7 +4825,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4801 "Python/generated_cases.c.h" + #line 4829 "Python/generated_cases.c.h" stack_pointer[-1] = func; DISPATCH(); } @@ -4805,7 +4833,7 @@ TARGET(SET_FUNCTION_ATTRIBUTE) { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3478 "Python/bytecodes.c" + #line 3476 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -4830,14 +4858,14 @@ default: Py_UNREACHABLE(); } - #line 4834 "Python/generated_cases.c.h" + #line 4862 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 3505 "Python/bytecodes.c" + #line 3503 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4858,7 +4886,7 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4862 "Python/generated_cases.c.h" + #line 4890 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4866,15 +4894,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3528 "Python/bytecodes.c" + #line 3526 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4872 "Python/generated_cases.c.h" + #line 4900 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3530 "Python/bytecodes.c" + #line 3528 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4878 "Python/generated_cases.c.h" + #line 4906 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4884,14 +4912,14 @@ TARGET(CONVERT_VALUE) { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3534 "Python/bytecodes.c" + #line 3532 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 4895 "Python/generated_cases.c.h" + #line 4923 "Python/generated_cases.c.h" stack_pointer[-1] = result; DISPATCH(); } @@ -4899,7 +4927,7 @@ TARGET(FORMAT_SIMPLE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3543 "Python/bytecodes.c" + #line 3541 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -4910,7 +4938,7 @@ else { res = value; } - #line 4914 "Python/generated_cases.c.h" + #line 4942 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -4919,12 +4947,12 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3556 "Python/bytecodes.c" + #line 3554 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 4928 "Python/generated_cases.c.h" + #line 4956 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -4933,10 +4961,10 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3563 "Python/bytecodes.c" + #line 3561 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4940 "Python/generated_cases.c.h" + #line 4968 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4948,7 +4976,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3568 "Python/bytecodes.c" + #line 3566 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -4963,12 +4991,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4967 "Python/generated_cases.c.h" + #line 4995 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3583 "Python/bytecodes.c" + #line 3581 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 4972 "Python/generated_cases.c.h" + #line 5000 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -4978,16 +5006,16 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3588 "Python/bytecodes.c" + #line 3586 "Python/bytecodes.c" assert(oparg >= 2); - #line 4984 "Python/generated_cases.c.h" + #line 5012 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } TARGET(INSTRUMENTED_INSTRUCTION) { - #line 3592 "Python/bytecodes.c" + #line 3590 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error; @@ -4999,48 +5027,48 @@ assert(next_opcode > 0 && next_opcode < 256); opcode = next_opcode; DISPATCH_GOTO(); - #line 5003 "Python/generated_cases.c.h" + #line 5031 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_JUMP_FORWARD) { - #line 3606 "Python/bytecodes.c" + #line 3604 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); - #line 5009 "Python/generated_cases.c.h" + #line 5037 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) { - #line 3610 "Python/bytecodes.c" + #line 3608 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); - #line 5017 "Python/generated_cases.c.h" + #line 5045 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { - #line 3615 "Python/bytecodes.c" + #line 3613 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsTrue(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5028 "Python/generated_cases.c.h" + #line 5056 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { - #line 3623 "Python/bytecodes.c" + #line 3621 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsFalse(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5039 "Python/generated_cases.c.h" + #line 5067 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { - #line 3631 "Python/bytecodes.c" + #line 3629 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -5052,12 +5080,12 @@ offset = 0; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5056 "Python/generated_cases.c.h" + #line 5084 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { - #line 3645 "Python/bytecodes.c" + #line 3643 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -5069,30 +5097,30 @@ offset = oparg; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5073 "Python/generated_cases.c.h" + #line 5101 "Python/generated_cases.c.h" DISPATCH(); } TARGET(EXTENDED_ARG) { - #line 3659 "Python/bytecodes.c" + #line 3657 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 5084 "Python/generated_cases.c.h" + #line 5112 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3667 "Python/bytecodes.c" + #line 3665 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); - #line 5091 "Python/generated_cases.c.h" + #line 5119 "Python/generated_cases.c.h" } TARGET(RESERVED) { - #line 3672 "Python/bytecodes.c" + #line 3670 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 5098 "Python/generated_cases.c.h" + #line 5126 "Python/generated_cases.c.h" } diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index 64923e61fa459..ce2384ee2e483 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -35,6 +35,7 @@ #define _BINARY_OP_ADD_UNICODE 313 #define _LOAD_LOCALS 314 #define _LOAD_FROM_DICT_OR_GLOBALS 315 +#define IS_NONE 316 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -328,10 +329,10 @@ _PyOpcode_num_popped(int opcode, int oparg, bool jump) { return 1; case POP_JUMP_IF_TRUE: return 1; - case POP_JUMP_IF_NOT_NONE: - return 1; case POP_JUMP_IF_NONE: return 1; + case POP_JUMP_IF_NOT_NONE: + return 1; case JUMP_BACKWARD_NO_INTERRUPT: return 0; case GET_LEN: @@ -772,10 +773,10 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { return 0; case POP_JUMP_IF_TRUE: return 0; - case POP_JUMP_IF_NOT_NONE: - return 0; case POP_JUMP_IF_NONE: return 0; + case POP_JUMP_IF_NOT_NONE: + return 0; case JUMP_BACKWARD_NO_INTERRUPT: return 0; case GET_LEN: @@ -1104,8 +1105,8 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { [ENTER_EXECUTOR] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [POP_JUMP_IF_FALSE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [POP_JUMP_IF_TRUE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, - [POP_JUMP_IF_NOT_NONE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [POP_JUMP_IF_NONE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, + [POP_JUMP_IF_NOT_NONE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [JUMP_BACKWARD_NO_INTERRUPT] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_JUMP_FLAG }, [GET_LEN] = { true, INSTR_FMT_IX, 0 }, [MATCH_CLASS] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, @@ -1310,6 +1311,7 @@ const char * const _PyOpcode_uop_name[512] = { [313] = "_BINARY_OP_ADD_UNICODE", [314] = "_LOAD_LOCALS", [315] = "_LOAD_FROM_DICT_OR_GLOBALS", + [316] = "IS_NONE", }; #endif // NEED_OPCODE_METADATA #endif From webhook-mailer at python.org Tue Jul 11 07:06:46 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Tue, 11 Jul 2023 11:06:46 -0000 Subject: [Python-checkins] gh-106078: Move external C-API functions to decimal module global state (#106616) Message-ID: https://github.com/python/cpython/commit/e5c32a811c248a9b052fc63236da58f81f488b44 commit: e5c32a811c248a9b052fc63236da58f81f488b44 branch: main author: Charlie Zhao committer: kumaraditya303 date: 2023-07-11T16:36:42+05:30 summary: gh-106078: Move external C-API functions to decimal module global state (#106616) files: M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/globals-to-fix.tsv diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 89924b205f99e..e3dc304066b45 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -81,6 +81,14 @@ typedef struct { PyObject *Rational; PyObject *SignalTuple; + + /* External C-API functions */ + binaryfunc _py_long_multiply; + binaryfunc _py_long_floor_divide; + ternaryfunc _py_long_power; + unaryfunc _py_float_abs; + PyCFunction _py_long_bit_length; + PyCFunction _py_float_as_integer_ratio; } decimal_state; static decimal_state global_state; @@ -2299,14 +2307,6 @@ PyDecType_FromLongExact(PyTypeObject *type, PyObject *v, return dec; } -/* External C-API functions */ -static binaryfunc _py_long_multiply; -static binaryfunc _py_long_floor_divide; -static ternaryfunc _py_long_power; -static unaryfunc _py_float_abs; -static PyCFunction _py_long_bit_length; -static PyCFunction _py_float_as_integer_ratio; - /* Return a PyDecObject or a subtype from a PyFloatObject. Conversion is exact. */ static PyObject * @@ -2322,8 +2322,8 @@ PyDecType_FromFloatExact(PyTypeObject *type, PyObject *v, uint32_t status = 0; mpd_context_t maxctx; -#ifdef Py_DEBUG decimal_state *state = GLOBAL_STATE(); +#ifdef Py_DEBUG assert(PyType_IsSubtype(type, state->PyDec_Type)); #endif if (PyLong_Check(v)) { @@ -2358,13 +2358,13 @@ PyDecType_FromFloatExact(PyTypeObject *type, PyObject *v, } /* absolute value of the float */ - tmp = _py_float_abs(v); + tmp = state->_py_float_abs(v); if (tmp == NULL) { return NULL; } /* float as integer ratio: numerator/denominator */ - n_d = _py_float_as_integer_ratio(tmp, NULL); + n_d = state->_py_float_as_integer_ratio(tmp, NULL); Py_DECREF(tmp); if (n_d == NULL) { return NULL; @@ -2372,7 +2372,7 @@ PyDecType_FromFloatExact(PyTypeObject *type, PyObject *v, n = PyTuple_GET_ITEM(n_d, 0); d = PyTuple_GET_ITEM(n_d, 1); - tmp = _py_long_bit_length(d, NULL); + tmp = state->_py_long_bit_length(d, NULL); if (tmp == NULL) { Py_DECREF(n_d); return NULL; @@ -3660,14 +3660,14 @@ dec_as_integer_ratio(PyObject *self, PyObject *args UNUSED) goto error; } - Py_SETREF(exponent, _py_long_power(tmp, exponent, Py_None)); + Py_SETREF(exponent, state->_py_long_power(tmp, exponent, Py_None)); Py_DECREF(tmp); if (exponent == NULL) { goto error; } if (exp >= 0) { - Py_SETREF(numerator, _py_long_multiply(numerator, exponent)); + Py_SETREF(numerator, state->_py_long_multiply(numerator, exponent)); if (numerator == NULL) { goto error; } @@ -3683,12 +3683,12 @@ dec_as_integer_ratio(PyObject *self, PyObject *args UNUSED) if (tmp == NULL) { goto error; } - Py_SETREF(numerator, _py_long_floor_divide(numerator, tmp)); + Py_SETREF(numerator, state->_py_long_floor_divide(numerator, tmp)); if (numerator == NULL) { Py_DECREF(tmp); goto error; } - Py_SETREF(denominator, _py_long_floor_divide(denominator, tmp)); + Py_SETREF(denominator, state->_py_long_floor_divide(denominator, tmp)); Py_DECREF(tmp); if (denominator == NULL) { goto error; @@ -5834,13 +5834,14 @@ PyInit__decimal(void) decimal_state *state = GLOBAL_STATE(); /* Init external C-API functions */ - _py_long_multiply = PyLong_Type.tp_as_number->nb_multiply; - _py_long_floor_divide = PyLong_Type.tp_as_number->nb_floor_divide; - _py_long_power = PyLong_Type.tp_as_number->nb_power; - _py_float_abs = PyFloat_Type.tp_as_number->nb_absolute; - ASSIGN_PTR(_py_float_as_integer_ratio, cfunc_noargs(&PyFloat_Type, - "as_integer_ratio")); - ASSIGN_PTR(_py_long_bit_length, cfunc_noargs(&PyLong_Type, "bit_length")); + state->_py_long_multiply = PyLong_Type.tp_as_number->nb_multiply; + state->_py_long_floor_divide = PyLong_Type.tp_as_number->nb_floor_divide; + state->_py_long_power = PyLong_Type.tp_as_number->nb_power; + state->_py_float_abs = PyFloat_Type.tp_as_number->nb_absolute; + ASSIGN_PTR(state->_py_float_as_integer_ratio, + cfunc_noargs(&PyFloat_Type, "as_integer_ratio")); + ASSIGN_PTR(state->_py_long_bit_length, + cfunc_noargs(&PyLong_Type, "bit_length")); /* Init types */ diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 38b47d06e4a5f..90bbc8928b428 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -449,12 +449,6 @@ Modules/_cursesmodule.c - initialised - Modules/_cursesmodule.c - initialised_setupterm - Modules/_cursesmodule.c - initialisedcolors - Modules/_cursesmodule.c - screen_encoding - -Modules/_decimal/_decimal.c - _py_long_multiply - -Modules/_decimal/_decimal.c - _py_long_floor_divide - -Modules/_decimal/_decimal.c - _py_long_power - -Modules/_decimal/_decimal.c - _py_float_abs - -Modules/_decimal/_decimal.c - _py_long_bit_length - -Modules/_decimal/_decimal.c - _py_float_as_integer_ratio - Modules/_elementtree.c - expat_capi - Modules/readline.c - libedit_append_replace_history_offset - Modules/readline.c - using_libedit_emulation - From webhook-mailer at python.org Tue Jul 11 07:20:33 2023 From: webhook-mailer at python.org (encukou) Date: Tue, 11 Jul 2023 11:20:33 -0000 Subject: [Python-checkins] gh-103968: What's New: Add porting hints for PyType_From with metaclasses (GH-105698) Message-ID: https://github.com/python/cpython/commit/af5cf1e75136fcef967d4ebe1bc45f29e6dc1bcf commit: af5cf1e75136fcef967d4ebe1bc45f29e6dc1bcf branch: main author: Petr Viktorin committer: encukou date: 2023-07-11T13:20:29+02:00 summary: gh-103968: What's New: Add porting hints for PyType_From with metaclasses (GH-105698) files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 766f468cf9cfb..a6d101bdb9f7a 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1837,7 +1837,31 @@ Porting to Python 3.12 allowing incomplete initialization. Note that :c:func:`PyType_FromMetaclass` (added in Python 3.12) - already disallows creating classes whose metaclass overrides ``tp_new``. + already disallows creating classes whose metaclass overrides ``tp_new`` + (:meth:`~object.__new__` in Python). + + Since ``tp_new`` overrides almost everything ``PyType_From*`` functions do, + the two are incompatible with each other. + The existing behavior -- ignoring the metaclass for several steps + of type creation -- is unsafe in general, since (meta)classes assume that + ``tp_new`` was called. + There is no simple general workaround. One of the following may work for you: + + - If you control the metaclass, avoid using ``tp_new`` in it: + + - If initialization can be skipped, it can be done in + :c:member:`~PyTypeObject.tp_init` instead. + - If the metaclass doesn't need to be instantiated from Python, + set its ``tp_new`` to ``NULL`` using + the :const:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. + This makes it acceptable for ``PyType_From*`` functions. + + - Avoid ``PyType_From*`` functions: if you don't need C-specific features + (slots or setting the instance size), create types by :ref:`calling ` + the metaclass. + + - If you *know* the ``tp_new`` can be skipped safely, filter the deprecation + warning out using :func:`warnings.catch_warnings` from Python. * :c:var:`PyOS_InputHook` and :c:var:`PyOS_ReadlineFunctionPointer` are no longer called in :ref:`subinterpreters `. This is From webhook-mailer at python.org Tue Jul 11 07:35:58 2023 From: webhook-mailer at python.org (ambv) Date: Tue, 11 Jul 2023 11:35:58 -0000 Subject: [Python-checkins] gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (#106468) Message-ID: https://github.com/python/cpython/commit/95b7426f45edb570869a5513c142f29ed9f851a1 commit: 95b7426f45edb570869a5513c142f29ed9f851a1 branch: main author: Ethan Furman committer: ambv date: 2023-07-11T13:35:54+02:00 summary: gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (#106468) For example: class Flag(enum.Flag): A = 0x01 B = 0x02 MASK = 0xff ~Flag.MASK is Flag(0) files: A Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 47e31b17c2a49..202f0da028bdf 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1515,14 +1515,10 @@ def __xor__(self, other): def __invert__(self): if self._inverted_ is None: - if self._boundary_ is KEEP: - # use all bits + if self._boundary_ in (EJECT, KEEP): self._inverted_ = self.__class__(~self._value_) else: - # use canonical bits (i.e. calculate flags not in this member) - self._inverted_ = self.__class__(self._singles_mask_ ^ self._value_) - if isinstance(self._inverted_, self.__class__): - self._inverted_._inverted_ = self + self._inverted_ = self.__class__(self._singles_mask_ & ~self._value_) return self._inverted_ __rand__ = __and__ diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 54b7d18d45415..adb1e0e52c548 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -818,6 +818,89 @@ def test_default_missing_with_wrong_type_value(self): self.MainEnum('RED') self.assertIs(ctx.exception.__context__, None) + def test_closed_invert_expectations(self): + class ClosedAB(self.enum_type): + A = 1 + B = 2 + MASK = 3 + A, B = ClosedAB + AB_MASK = ClosedAB.MASK + # + self.assertIs(~A, B) + self.assertIs(~B, A) + self.assertIs(~(A|B), ClosedAB(0)) + self.assertIs(~AB_MASK, ClosedAB(0)) + self.assertIs(~ClosedAB(0), (A|B)) + # + class ClosedXYZ(self.enum_type): + X = 4 + Y = 2 + Z = 1 + MASK = 7 + X, Y, Z = ClosedXYZ + XYZ_MASK = ClosedXYZ.MASK + # + self.assertIs(~X, Y|Z) + self.assertIs(~Y, X|Z) + self.assertIs(~Z, X|Y) + self.assertIs(~(X|Y), Z) + self.assertIs(~(X|Z), Y) + self.assertIs(~(Y|Z), X) + self.assertIs(~(X|Y|Z), ClosedXYZ(0)) + self.assertIs(~XYZ_MASK, ClosedXYZ(0)) + self.assertIs(~ClosedXYZ(0), (X|Y|Z)) + + def test_open_invert_expectations(self): + class OpenAB(self.enum_type): + A = 1 + B = 2 + MASK = 255 + A, B = OpenAB + AB_MASK = OpenAB.MASK + # + if OpenAB._boundary_ in (EJECT, KEEP): + self.assertIs(~A, OpenAB(254)) + self.assertIs(~B, OpenAB(253)) + self.assertIs(~(A|B), OpenAB(252)) + self.assertIs(~AB_MASK, OpenAB(0)) + self.assertIs(~OpenAB(0), AB_MASK) + else: + self.assertIs(~A, B) + self.assertIs(~B, A) + self.assertIs(~(A|B), OpenAB(0)) + self.assertIs(~AB_MASK, OpenAB(0)) + self.assertIs(~OpenAB(0), (A|B)) + # + class OpenXYZ(self.enum_type): + X = 4 + Y = 2 + Z = 1 + MASK = 31 + X, Y, Z = OpenXYZ + XYZ_MASK = OpenXYZ.MASK + # + if OpenXYZ._boundary_ in (EJECT, KEEP): + self.assertIs(~X, OpenXYZ(27)) + self.assertIs(~Y, OpenXYZ(29)) + self.assertIs(~Z, OpenXYZ(30)) + self.assertIs(~(X|Y), OpenXYZ(25)) + self.assertIs(~(X|Z), OpenXYZ(26)) + self.assertIs(~(Y|Z), OpenXYZ(28)) + self.assertIs(~(X|Y|Z), OpenXYZ(24)) + self.assertIs(~XYZ_MASK, OpenXYZ(0)) + self.assertTrue(~OpenXYZ(0), XYZ_MASK) + else: + self.assertIs(~X, Y|Z) + self.assertIs(~Y, X|Z) + self.assertIs(~Z, X|Y) + self.assertIs(~(X|Y), Z) + self.assertIs(~(X|Z), Y) + self.assertIs(~(Y|Z), X) + self.assertIs(~(X|Y|Z), OpenXYZ(0)) + self.assertIs(~XYZ_MASK, OpenXYZ(0)) + self.assertTrue(~OpenXYZ(0), (X|Y|Z)) + + class TestPlainEnum(_EnumTests, _PlainOutputTests, unittest.TestCase): enum_type = Enum @@ -3045,33 +3128,6 @@ class Color(Flag): WHITE = RED|GREEN|BLUE BLANCO = RED|GREEN|BLUE - class Complete(Flag): - A = 0x01 - B = 0x02 - - class Partial(Flag): - A = 0x01 - B = 0x02 - MASK = 0xff - - class CompleteInt(IntFlag): - A = 0x01 - B = 0x02 - - class PartialInt(IntFlag): - A = 0x01 - B = 0x02 - MASK = 0xff - - class CompleteIntStrict(IntFlag, boundary=STRICT): - A = 0x01 - B = 0x02 - - class PartialIntStrict(IntFlag, boundary=STRICT): - A = 0x01 - B = 0x02 - MASK = 0xff - def test_or(self): Perm = self.Perm for i in Perm: @@ -3115,34 +3171,6 @@ def test_xor(self): self.assertIs(Open.RO ^ Open.CE, Open.CE) self.assertIs(Open.CE ^ Open.CE, Open.RO) - def test_invert(self): - Perm = self.Perm - RW = Perm.R | Perm.W - RX = Perm.R | Perm.X - WX = Perm.W | Perm.X - RWX = Perm.R | Perm.W | Perm.X - values = list(Perm) + [RW, RX, WX, RWX, Perm(0)] - for i in values: - self.assertIs(type(~i), Perm) - self.assertEqual(~~i, i) - for i in Perm: - self.assertIs(~~i, i) - Open = self.Open - self.assertIs(Open.WO & ~Open.WO, Open.RO) - self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) - Complete = self.Complete - self.assertIs(~Complete.A, Complete.B) - Partial = self.Partial - self.assertIs(~Partial.A, Partial.B) - CompleteInt = self.CompleteInt - self.assertIs(~CompleteInt.A, CompleteInt.B) - PartialInt = self.PartialInt - self.assertIs(~PartialInt.A, PartialInt(254)) - CompleteIntStrict = self.CompleteIntStrict - self.assertIs(~CompleteIntStrict.A, CompleteIntStrict.B) - PartialIntStrict = self.PartialIntStrict - self.assertIs(~PartialIntStrict.A, PartialIntStrict.B) - def test_bool(self): Perm = self.Perm for f in Perm: diff --git a/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst b/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst new file mode 100644 index 0000000000000..f4f2db08f73f5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst @@ -0,0 +1 @@ +Fix flag mask inversion when unnamed flags exist. From webhook-mailer at python.org Tue Jul 11 07:53:44 2023 From: webhook-mailer at python.org (encukou) Date: Tue, 11 Jul 2023 11:53:44 -0000 Subject: [Python-checkins] [3.12] gh-103968: What's New: Add porting hints for PyType_From with metaclasses (GH-105698) (GH-106619) Message-ID: https://github.com/python/cpython/commit/2eb9fe92b4180af0a9e51b1232df5a3d7d93788f commit: 2eb9fe92b4180af0a9e51b1232df5a3d7d93788f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: encukou date: 2023-07-11T13:53:40+02:00 summary: [3.12] gh-103968: What's New: Add porting hints for PyType_From with metaclasses (GH-105698) (GH-106619) gh-103968: What's New: Add porting hints for PyType_From with metaclasses (GH-105698) (cherry picked from commit af5cf1e75136fcef967d4ebe1bc45f29e6dc1bcf) Co-authored-by: Petr Viktorin files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index ef00f097b8f0e..d6d7b7dac9a31 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1836,7 +1836,31 @@ Porting to Python 3.12 allowing incomplete initialization. Note that :c:func:`PyType_FromMetaclass` (added in Python 3.12) - already disallows creating classes whose metaclass overrides ``tp_new``. + already disallows creating classes whose metaclass overrides ``tp_new`` + (:meth:`~object.__new__` in Python). + + Since ``tp_new`` overrides almost everything ``PyType_From*`` functions do, + the two are incompatible with each other. + The existing behavior -- ignoring the metaclass for several steps + of type creation -- is unsafe in general, since (meta)classes assume that + ``tp_new`` was called. + There is no simple general workaround. One of the following may work for you: + + - If you control the metaclass, avoid using ``tp_new`` in it: + + - If initialization can be skipped, it can be done in + :c:member:`~PyTypeObject.tp_init` instead. + - If the metaclass doesn't need to be instantiated from Python, + set its ``tp_new`` to ``NULL`` using + the :const:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. + This makes it acceptable for ``PyType_From*`` functions. + + - Avoid ``PyType_From*`` functions: if you don't need C-specific features + (slots or setting the instance size), create types by :ref:`calling ` + the metaclass. + + - If you *know* the ``tp_new`` can be skipped safely, filter the deprecation + warning out using :func:`warnings.catch_warnings` from Python. * :c:var:`PyOS_InputHook` and :c:var:`PyOS_ReadlineFunctionPointer` are no longer called in :ref:`subinterpreters `. This is From webhook-mailer at python.org Tue Jul 11 08:09:44 2023 From: webhook-mailer at python.org (Yhg1s) Date: Tue, 11 Jul 2023 12:09:44 -0000 Subject: [Python-checkins] [3.12] gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468) (#106620) Message-ID: https://github.com/python/cpython/commit/6968f9e4d3593610b60c1140f04de275ff40cd41 commit: 6968f9e4d3593610b60c1140f04de275ff40cd41 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Yhg1s date: 2023-07-11T14:09:41+02:00 summary: [3.12] gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468) (#106620) gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468) For example: class Flag(enum.Flag): A = 0x01 B = 0x02 MASK = 0xff ~Flag.MASK is Flag(0) (cherry picked from commit 95b7426f45edb570869a5513c142f29ed9f851a1) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 92a9632dc892d..56ebfbd1a61f7 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1539,14 +1539,10 @@ def __xor__(self, other): def __invert__(self): if self._inverted_ is None: - if self._boundary_ is KEEP: - # use all bits + if self._boundary_ in (EJECT, KEEP): self._inverted_ = self.__class__(~self._value_) else: - # use canonical bits (i.e. calculate flags not in this member) - self._inverted_ = self.__class__(self._singles_mask_ ^ self._value_) - if isinstance(self._inverted_, self.__class__): - self._inverted_._inverted_ = self + self._inverted_ = self.__class__(self._singles_mask_ & ~self._value_) return self._inverted_ __rand__ = __and__ diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 291213a13f36b..592deb837914c 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -818,6 +818,89 @@ def test_default_missing_with_wrong_type_value(self): self.MainEnum('RED') self.assertIs(ctx.exception.__context__, None) + def test_closed_invert_expectations(self): + class ClosedAB(self.enum_type): + A = 1 + B = 2 + MASK = 3 + A, B = ClosedAB + AB_MASK = ClosedAB.MASK + # + self.assertIs(~A, B) + self.assertIs(~B, A) + self.assertIs(~(A|B), ClosedAB(0)) + self.assertIs(~AB_MASK, ClosedAB(0)) + self.assertIs(~ClosedAB(0), (A|B)) + # + class ClosedXYZ(self.enum_type): + X = 4 + Y = 2 + Z = 1 + MASK = 7 + X, Y, Z = ClosedXYZ + XYZ_MASK = ClosedXYZ.MASK + # + self.assertIs(~X, Y|Z) + self.assertIs(~Y, X|Z) + self.assertIs(~Z, X|Y) + self.assertIs(~(X|Y), Z) + self.assertIs(~(X|Z), Y) + self.assertIs(~(Y|Z), X) + self.assertIs(~(X|Y|Z), ClosedXYZ(0)) + self.assertIs(~XYZ_MASK, ClosedXYZ(0)) + self.assertIs(~ClosedXYZ(0), (X|Y|Z)) + + def test_open_invert_expectations(self): + class OpenAB(self.enum_type): + A = 1 + B = 2 + MASK = 255 + A, B = OpenAB + AB_MASK = OpenAB.MASK + # + if OpenAB._boundary_ in (EJECT, KEEP): + self.assertIs(~A, OpenAB(254)) + self.assertIs(~B, OpenAB(253)) + self.assertIs(~(A|B), OpenAB(252)) + self.assertIs(~AB_MASK, OpenAB(0)) + self.assertIs(~OpenAB(0), AB_MASK) + else: + self.assertIs(~A, B) + self.assertIs(~B, A) + self.assertIs(~(A|B), OpenAB(0)) + self.assertIs(~AB_MASK, OpenAB(0)) + self.assertIs(~OpenAB(0), (A|B)) + # + class OpenXYZ(self.enum_type): + X = 4 + Y = 2 + Z = 1 + MASK = 31 + X, Y, Z = OpenXYZ + XYZ_MASK = OpenXYZ.MASK + # + if OpenXYZ._boundary_ in (EJECT, KEEP): + self.assertIs(~X, OpenXYZ(27)) + self.assertIs(~Y, OpenXYZ(29)) + self.assertIs(~Z, OpenXYZ(30)) + self.assertIs(~(X|Y), OpenXYZ(25)) + self.assertIs(~(X|Z), OpenXYZ(26)) + self.assertIs(~(Y|Z), OpenXYZ(28)) + self.assertIs(~(X|Y|Z), OpenXYZ(24)) + self.assertIs(~XYZ_MASK, OpenXYZ(0)) + self.assertTrue(~OpenXYZ(0), XYZ_MASK) + else: + self.assertIs(~X, Y|Z) + self.assertIs(~Y, X|Z) + self.assertIs(~Z, X|Y) + self.assertIs(~(X|Y), Z) + self.assertIs(~(X|Z), Y) + self.assertIs(~(Y|Z), X) + self.assertIs(~(X|Y|Z), OpenXYZ(0)) + self.assertIs(~XYZ_MASK, OpenXYZ(0)) + self.assertTrue(~OpenXYZ(0), (X|Y|Z)) + + class TestPlainEnum(_EnumTests, _PlainOutputTests, unittest.TestCase): enum_type = Enum @@ -3045,33 +3128,6 @@ class Color(Flag): WHITE = RED|GREEN|BLUE BLANCO = RED|GREEN|BLUE - class Complete(Flag): - A = 0x01 - B = 0x02 - - class Partial(Flag): - A = 0x01 - B = 0x02 - MASK = 0xff - - class CompleteInt(IntFlag): - A = 0x01 - B = 0x02 - - class PartialInt(IntFlag): - A = 0x01 - B = 0x02 - MASK = 0xff - - class CompleteIntStrict(IntFlag, boundary=STRICT): - A = 0x01 - B = 0x02 - - class PartialIntStrict(IntFlag, boundary=STRICT): - A = 0x01 - B = 0x02 - MASK = 0xff - def test_or(self): Perm = self.Perm for i in Perm: @@ -3115,34 +3171,6 @@ def test_xor(self): self.assertIs(Open.RO ^ Open.CE, Open.CE) self.assertIs(Open.CE ^ Open.CE, Open.RO) - def test_invert(self): - Perm = self.Perm - RW = Perm.R | Perm.W - RX = Perm.R | Perm.X - WX = Perm.W | Perm.X - RWX = Perm.R | Perm.W | Perm.X - values = list(Perm) + [RW, RX, WX, RWX, Perm(0)] - for i in values: - self.assertIs(type(~i), Perm) - self.assertEqual(~~i, i) - for i in Perm: - self.assertIs(~~i, i) - Open = self.Open - self.assertIs(Open.WO & ~Open.WO, Open.RO) - self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) - Complete = self.Complete - self.assertIs(~Complete.A, Complete.B) - Partial = self.Partial - self.assertIs(~Partial.A, Partial.B) - CompleteInt = self.CompleteInt - self.assertIs(~CompleteInt.A, CompleteInt.B) - PartialInt = self.PartialInt - self.assertIs(~PartialInt.A, PartialInt(254)) - CompleteIntStrict = self.CompleteIntStrict - self.assertIs(~CompleteIntStrict.A, CompleteIntStrict.B) - PartialIntStrict = self.PartialIntStrict - self.assertIs(~PartialIntStrict.A, PartialIntStrict.B) - def test_bool(self): Perm = self.Perm for f in Perm: diff --git a/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst b/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst new file mode 100644 index 0000000000000..f4f2db08f73f5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst @@ -0,0 +1 @@ +Fix flag mask inversion when unnamed flags exist. From webhook-mailer at python.org Tue Jul 11 08:10:06 2023 From: webhook-mailer at python.org (Yhg1s) Date: Tue, 11 Jul 2023 12:10:06 -0000 Subject: [Python-checkins] [3.11] gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468) (#106621) Message-ID: https://github.com/python/cpython/commit/318f6ae1cd3c5936062ff2d16fa4f3d1f38ea802 commit: 318f6ae1cd3c5936062ff2d16fa4f3d1f38ea802 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Yhg1s date: 2023-07-11T14:10:02+02:00 summary: [3.11] gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468) (#106621) gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468) For example: class Flag(enum.Flag): A = 0x01 B = 0x02 MASK = 0xff ~Flag.MASK is Flag(0) (cherry picked from commit 95b7426f45edb570869a5513c142f29ed9f851a1) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index d59026162bdc6..1184b5b13e6d7 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1522,14 +1522,10 @@ def __xor__(self, other): def __invert__(self): if self._inverted_ is None: - if self._boundary_ is KEEP: - # use all bits + if self._boundary_ in (EJECT, KEEP): self._inverted_ = self.__class__(~self._value_) else: - # use canonical bits (i.e. calculate flags not in this member) - self._inverted_ = self.__class__(self._singles_mask_ ^ self._value_) - if isinstance(self._inverted_, self.__class__): - self._inverted_._inverted_ = self + self._inverted_ = self.__class__(self._singles_mask_ & ~self._value_) return self._inverted_ __rand__ = __and__ diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index ce8182e789b9a..442802823e7d0 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -789,6 +789,89 @@ def test_default_missing_with_wrong_type_value(self): self.MainEnum('RED') self.assertIs(ctx.exception.__context__, None) + def test_closed_invert_expectations(self): + class ClosedAB(self.enum_type): + A = 1 + B = 2 + MASK = 3 + A, B = ClosedAB + AB_MASK = ClosedAB.MASK + # + self.assertIs(~A, B) + self.assertIs(~B, A) + self.assertIs(~(A|B), ClosedAB(0)) + self.assertIs(~AB_MASK, ClosedAB(0)) + self.assertIs(~ClosedAB(0), (A|B)) + # + class ClosedXYZ(self.enum_type): + X = 4 + Y = 2 + Z = 1 + MASK = 7 + X, Y, Z = ClosedXYZ + XYZ_MASK = ClosedXYZ.MASK + # + self.assertIs(~X, Y|Z) + self.assertIs(~Y, X|Z) + self.assertIs(~Z, X|Y) + self.assertIs(~(X|Y), Z) + self.assertIs(~(X|Z), Y) + self.assertIs(~(Y|Z), X) + self.assertIs(~(X|Y|Z), ClosedXYZ(0)) + self.assertIs(~XYZ_MASK, ClosedXYZ(0)) + self.assertIs(~ClosedXYZ(0), (X|Y|Z)) + + def test_open_invert_expectations(self): + class OpenAB(self.enum_type): + A = 1 + B = 2 + MASK = 255 + A, B = OpenAB + AB_MASK = OpenAB.MASK + # + if OpenAB._boundary_ in (EJECT, KEEP): + self.assertIs(~A, OpenAB(254)) + self.assertIs(~B, OpenAB(253)) + self.assertIs(~(A|B), OpenAB(252)) + self.assertIs(~AB_MASK, OpenAB(0)) + self.assertIs(~OpenAB(0), AB_MASK) + else: + self.assertIs(~A, B) + self.assertIs(~B, A) + self.assertIs(~(A|B), OpenAB(0)) + self.assertIs(~AB_MASK, OpenAB(0)) + self.assertIs(~OpenAB(0), (A|B)) + # + class OpenXYZ(self.enum_type): + X = 4 + Y = 2 + Z = 1 + MASK = 31 + X, Y, Z = OpenXYZ + XYZ_MASK = OpenXYZ.MASK + # + if OpenXYZ._boundary_ in (EJECT, KEEP): + self.assertIs(~X, OpenXYZ(27)) + self.assertIs(~Y, OpenXYZ(29)) + self.assertIs(~Z, OpenXYZ(30)) + self.assertIs(~(X|Y), OpenXYZ(25)) + self.assertIs(~(X|Z), OpenXYZ(26)) + self.assertIs(~(Y|Z), OpenXYZ(28)) + self.assertIs(~(X|Y|Z), OpenXYZ(24)) + self.assertIs(~XYZ_MASK, OpenXYZ(0)) + self.assertTrue(~OpenXYZ(0), XYZ_MASK) + else: + self.assertIs(~X, Y|Z) + self.assertIs(~Y, X|Z) + self.assertIs(~Z, X|Y) + self.assertIs(~(X|Y), Z) + self.assertIs(~(X|Z), Y) + self.assertIs(~(Y|Z), X) + self.assertIs(~(X|Y|Z), OpenXYZ(0)) + self.assertIs(~XYZ_MASK, OpenXYZ(0)) + self.assertTrue(~OpenXYZ(0), (X|Y|Z)) + + class TestPlainEnum(_EnumTests, _PlainOutputTests, unittest.TestCase): enum_type = Enum @@ -2913,33 +2996,6 @@ class Color(Flag): WHITE = RED|GREEN|BLUE BLANCO = RED|GREEN|BLUE - class Complete(Flag): - A = 0x01 - B = 0x02 - - class Partial(Flag): - A = 0x01 - B = 0x02 - MASK = 0xff - - class CompleteInt(IntFlag): - A = 0x01 - B = 0x02 - - class PartialInt(IntFlag): - A = 0x01 - B = 0x02 - MASK = 0xff - - class CompleteIntStrict(IntFlag, boundary=STRICT): - A = 0x01 - B = 0x02 - - class PartialIntStrict(IntFlag, boundary=STRICT): - A = 0x01 - B = 0x02 - MASK = 0xff - def test_or(self): Perm = self.Perm for i in Perm: @@ -2983,34 +3039,6 @@ def test_xor(self): self.assertIs(Open.RO ^ Open.CE, Open.CE) self.assertIs(Open.CE ^ Open.CE, Open.RO) - def test_invert(self): - Perm = self.Perm - RW = Perm.R | Perm.W - RX = Perm.R | Perm.X - WX = Perm.W | Perm.X - RWX = Perm.R | Perm.W | Perm.X - values = list(Perm) + [RW, RX, WX, RWX, Perm(0)] - for i in values: - self.assertIs(type(~i), Perm) - self.assertEqual(~~i, i) - for i in Perm: - self.assertIs(~~i, i) - Open = self.Open - self.assertIs(Open.WO & ~Open.WO, Open.RO) - self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) - Complete = self.Complete - self.assertIs(~Complete.A, Complete.B) - Partial = self.Partial - self.assertIs(~Partial.A, Partial.B) - CompleteInt = self.CompleteInt - self.assertIs(~CompleteInt.A, CompleteInt.B) - PartialInt = self.PartialInt - self.assertIs(~PartialInt.A, PartialInt(254)) - CompleteIntStrict = self.CompleteIntStrict - self.assertIs(~CompleteIntStrict.A, CompleteIntStrict.B) - PartialIntStrict = self.PartialIntStrict - self.assertIs(~PartialIntStrict.A, PartialIntStrict.B) - def test_bool(self): Perm = self.Perm for f in Perm: diff --git a/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst b/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst new file mode 100644 index 0000000000000..f4f2db08f73f5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst @@ -0,0 +1 @@ +Fix flag mask inversion when unnamed flags exist. From webhook-mailer at python.org Tue Jul 11 08:41:54 2023 From: webhook-mailer at python.org (iritkatriel) Date: Tue, 11 Jul 2023 12:41:54 -0000 Subject: [Python-checkins] gh-106360: remove redundant #ifdef (#106622) Message-ID: https://github.com/python/cpython/commit/388b5daa523b828dc0f7e2a1a6886bebc20833ba commit: 388b5daa523b828dc0f7e2a1a6886bebc20833ba branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-11T13:41:50+01:00 summary: gh-106360: remove redundant #ifdef (#106622) files: M Python/opcode_metadata.h M Tools/cases_generator/generate_cases.py diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index ce2384ee2e483..34ac85d6517a8 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -960,7 +960,7 @@ struct opcode_macro_expansion { extern const struct opcode_metadata _PyOpcode_opcode_metadata[512]; extern const struct opcode_macro_expansion _PyOpcode_macro_expansion[256]; extern const char * const _PyOpcode_uop_name[512]; -#else +#else // if NEED_OPCODE_METADATA const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { [NOP] = { true, INSTR_FMT_IX, 0 }, [RESUME] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, @@ -1293,7 +1293,6 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [BINARY_OP] = { .nuops = 1, .uops = { { BINARY_OP, 0, 0 } } }, [SWAP] = { .nuops = 1, .uops = { { SWAP, 0, 0 } } }, }; -#ifdef NEED_OPCODE_METADATA const char * const _PyOpcode_uop_name[512] = { [300] = "EXIT_TRACE", [301] = "SAVE_IP", @@ -1314,4 +1313,3 @@ const char * const _PyOpcode_uop_name[512] = { [316] = "IS_NONE", }; #endif // NEED_OPCODE_METADATA -#endif diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 932d0c14d398a..641a327b06aa2 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -1240,7 +1240,7 @@ def write_metadata(self) -> None: self.out.emit("extern const struct opcode_metadata _PyOpcode_opcode_metadata[512];") self.out.emit("extern const struct opcode_macro_expansion _PyOpcode_macro_expansion[256];") self.out.emit("extern const char * const _PyOpcode_uop_name[512];") - self.out.emit("#else") + self.out.emit("#else // if NEED_OPCODE_METADATA") self.out.emit("const struct opcode_metadata _PyOpcode_opcode_metadata[512] = {") @@ -1289,12 +1289,10 @@ def write_metadata(self) -> None: case _: typing.assert_never(thing) - self.out.emit("#ifdef NEED_OPCODE_METADATA") with self.out.block("const char * const _PyOpcode_uop_name[512] =", ";"): self.write_uop_items(lambda name, counter: f"[{counter}] = \"{name}\",") - self.out.emit("#endif // NEED_OPCODE_METADATA") - self.out.emit("#endif") + self.out.emit("#endif // NEED_OPCODE_METADATA") with open(self.pymetadata_filename, "w") as f: # Create formatter From webhook-mailer at python.org Tue Jul 11 10:55:42 2023 From: webhook-mailer at python.org (methane) Date: Tue, 11 Jul 2023 14:55:42 -0000 Subject: [Python-checkins] wasm: do not use inline comment in .editorconfig (#106610) Message-ID: https://github.com/python/cpython/commit/64c0890b697783db9b3f67e3bb4dcee1165a0b9b commit: 64c0890b697783db9b3f67e3bb4dcee1165a0b9b branch: main author: Eisuke Kawashima committer: methane date: 2023-07-11T23:55:37+09:00 summary: wasm: do not use inline comment in .editorconfig (#106610) It is no longer valid since 0.15.0 https://github.com/editorconfig/specification/blob/v0.15/index.rst#no-inline-comments files: M Tools/wasm/.editorconfig diff --git a/Tools/wasm/.editorconfig b/Tools/wasm/.editorconfig index da1aa6acaccc7..4de5fe5954d84 100644 --- a/Tools/wasm/.editorconfig +++ b/Tools/wasm/.editorconfig @@ -1,4 +1,5 @@ -root = false # This extends the root .editorconfig +# This extends the root .editorconfig +root = false [*.{html,js}] trim_trailing_whitespace = true From webhook-mailer at python.org Tue Jul 11 11:07:25 2023 From: webhook-mailer at python.org (terryjreedy) Date: Tue, 11 Jul 2023 15:07:25 -0000 Subject: [Python-checkins] gh-106498: Revert incorrect colorsys.rgb_to_hls change (#106627) Message-ID: https://github.com/python/cpython/commit/a2d54d4e8ab12f967a220be88bde8ac8227c5ab3 commit: a2d54d4e8ab12f967a220be88bde8ac8227c5ab3 branch: main author: Terry Jan Reedy committer: terryjreedy date: 2023-07-11T11:07:20-04:00 summary: gh-106498: Revert incorrect colorsys.rgb_to_hls change (#106627) gh-86618 assumed a-b-c = a-(b+c) = a-d where d = b+d. For floats 2.0, 1.0, and 0.9999999999999999, this assumption is false. The net change of 1.1102230246251565e-16 to 0.0 results in division by 0. Revert the replacement. Add test. files: A Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst M Lib/colorsys.py M Lib/test/test_colorsys.py diff --git a/Lib/colorsys.py b/Lib/colorsys.py index 9bdc83e377260..bc897bd0f9929 100644 --- a/Lib/colorsys.py +++ b/Lib/colorsys.py @@ -83,7 +83,7 @@ def rgb_to_hls(r, g, b): if l <= 0.5: s = rangec / sumc else: - s = rangec / (2.0-sumc) + s = rangec / (2.0-maxc-minc) # Not always 2.0-sumc: gh-106498. rc = (maxc-r) / rangec gc = (maxc-g) / rangec bc = (maxc-b) / rangec diff --git a/Lib/test/test_colorsys.py b/Lib/test/test_colorsys.py index a24e3adcb4b84..74d76294b0b4d 100644 --- a/Lib/test/test_colorsys.py +++ b/Lib/test/test_colorsys.py @@ -69,6 +69,16 @@ def test_hls_values(self): self.assertTripleEqual(hls, colorsys.rgb_to_hls(*rgb)) self.assertTripleEqual(rgb, colorsys.hls_to_rgb(*hls)) + def test_hls_nearwhite(self): # gh-106498 + values = ( + # rgb, hls: these do not work in reverse + ((0.9999999999999999, 1, 1), (0.5, 1.0, 1.0)), + ((1, 0.9999999999999999, 0.9999999999999999), (0.0, 1.0, 1.0)), + ) + for rgb, hls in values: + self.assertTripleEqual(hls, colorsys.rgb_to_hls(*rgb)) + self.assertTripleEqual((1.0, 1.0, 1.0), colorsys.hls_to_rgb(*hls)) + def test_yiq_roundtrip(self): for r in frange(0.0, 1.0, 0.2): for g in frange(0.0, 1.0, 0.2): diff --git a/Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst b/Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst new file mode 100644 index 0000000000000..09fc647cc01d2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst @@ -0,0 +1,2 @@ +Revert a change to :func:`colorsys.rgb_to_hls` that caused division by zero +for certain almost-white inputs. Patch by Terry Jan Reedy. From webhook-mailer at python.org Tue Jul 11 11:43:29 2023 From: webhook-mailer at python.org (JelleZijlstra) Date: Tue, 11 Jul 2023 15:43:29 -0000 Subject: [Python-checkins] gh-106403: Restore weakref support for TypeVar and friends (#106418) Message-ID: https://github.com/python/cpython/commit/945d3cbf2e8e756ed16c3ec51106e6157abb2698 commit: 945d3cbf2e8e756ed16c3ec51106e6157abb2698 branch: main author: Jelle Zijlstra committer: JelleZijlstra date: 2023-07-11T08:43:24-07:00 summary: gh-106403: Restore weakref support for TypeVar and friends (#106418) files: A Misc/NEWS.d/next/Library/2023-07-04-07-25-30.gh-issue-106403.GmefbV.rst M Lib/test/test_type_params.py M Objects/typevarobject.c diff --git a/Lib/test/test_type_params.py b/Lib/test/test_type_params.py index 3026cc2247661..bced641a9661f 100644 --- a/Lib/test/test_type_params.py +++ b/Lib/test/test_type_params.py @@ -3,6 +3,7 @@ import types import unittest import pickle +import weakref from test.support import requires_working_socket, check_syntax_error, run_code from typing import Generic, Sequence, TypeVar, TypeVarTuple, ParamSpec, get_args @@ -921,3 +922,33 @@ def test_pickling_classes(self): # These instances are not equal, # but class check is good enough: self.assertIsInstance(pickle.loads(pickled), real_class) + + +class TypeParamsWeakRefTest(unittest.TestCase): + def test_weakrefs(self): + T = TypeVar('T') + P = ParamSpec('P') + class OldStyle(Generic[T]): + pass + + class NewStyle[T]: + pass + + cases = [ + T, + TypeVar('T', bound=int), + P, + P.args, + P.kwargs, + TypeVarTuple('Ts'), + OldStyle, + OldStyle[int], + OldStyle(), + NewStyle, + NewStyle[int], + NewStyle(), + Generic[T], + ] + for case in cases: + with self.subTest(case=case): + weakref.ref(case) diff --git a/Misc/NEWS.d/next/Library/2023-07-04-07-25-30.gh-issue-106403.GmefbV.rst b/Misc/NEWS.d/next/Library/2023-07-04-07-25-30.gh-issue-106403.GmefbV.rst new file mode 100644 index 0000000000000..4fea45f16c4f8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-04-07-25-30.gh-issue-106403.GmefbV.rst @@ -0,0 +1,4 @@ +Instances of :class:`typing.TypeVar`, :class:`typing.ParamSpec`, +:class:`typing.ParamSpecArgs`, :class:`typing.ParamSpecKwargs`, and +:class:`typing.TypeVarTuple` once again support weak references, fixing a +regression introduced in Python 3.12.0 beta 1. Patch by Jelle Zijlstra. diff --git a/Objects/typevarobject.c b/Objects/typevarobject.c index 0b44f84b6f01d..5605662f0e6d5 100644 --- a/Objects/typevarobject.c +++ b/Objects/typevarobject.c @@ -500,7 +500,7 @@ PyType_Spec typevar_spec = { .name = "typing.TypeVar", .basicsize = sizeof(typevarobject), .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE - | Py_TPFLAGS_MANAGED_DICT, + | Py_TPFLAGS_MANAGED_DICT | Py_TPFLAGS_MANAGED_WEAKREF, .slots = typevar_slots, }; @@ -647,7 +647,8 @@ static PyType_Slot paramspecargs_slots[] = { PyType_Spec paramspecargs_spec = { .name = "typing.ParamSpecArgs", .basicsize = sizeof(paramspecattrobject), - .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE, + .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE + | Py_TPFLAGS_MANAGED_WEAKREF, .slots = paramspecargs_slots, }; @@ -726,7 +727,8 @@ static PyType_Slot paramspeckwargs_slots[] = { PyType_Spec paramspeckwargs_spec = { .name = "typing.ParamSpecKwargs", .basicsize = sizeof(paramspecattrobject), - .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE, + .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE + | Py_TPFLAGS_MANAGED_WEAKREF, .slots = paramspeckwargs_slots, }; @@ -1007,7 +1009,7 @@ PyType_Spec paramspec_spec = { .name = "typing.ParamSpec", .basicsize = sizeof(paramspecobject), .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE - | Py_TPFLAGS_MANAGED_DICT, + | Py_TPFLAGS_MANAGED_DICT | Py_TPFLAGS_MANAGED_WEAKREF, .slots = paramspec_slots, }; @@ -1228,7 +1230,7 @@ PyType_Spec typevartuple_spec = { .name = "typing.TypeVarTuple", .basicsize = sizeof(typevartupleobject), .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_IMMUTABLETYPE | Py_TPFLAGS_MANAGED_DICT - | Py_TPFLAGS_HAVE_GC, + | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_MANAGED_WEAKREF, .slots = typevartuple_slots, }; From webhook-mailer at python.org Tue Jul 11 11:47:19 2023 From: webhook-mailer at python.org (terryjreedy) Date: Tue, 11 Jul 2023 15:47:19 -0000 Subject: [Python-checkins] [3.12] gh-106498: Revert incorrect colorsys.rgb_to_hls change (GH-106627) (#106632) Message-ID: https://github.com/python/cpython/commit/c594e25cd76b9d2be04eaebc13df2becbdda7aed commit: c594e25cd76b9d2be04eaebc13df2becbdda7aed branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-11T15:47:15Z summary: [3.12] gh-106498: Revert incorrect colorsys.rgb_to_hls change (GH-106627) (#106632) gh-106498: Revert incorrect colorsys.rgb_to_hls change (GH-106627) gh-86618 assumed a-b-c = a-(b+c) = a-d where d = b+d. For floats 2.0, 1.0, and 0.9999999999999999, this assumption is false. The net change of 1.1102230246251565e-16 to 0.0 results in division by 0. Revert the replacement. Add test. (cherry picked from commit a2d54d4e8ab12f967a220be88bde8ac8227c5ab3) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst M Lib/colorsys.py M Lib/test/test_colorsys.py diff --git a/Lib/colorsys.py b/Lib/colorsys.py index 9bdc83e377260..bc897bd0f9929 100644 --- a/Lib/colorsys.py +++ b/Lib/colorsys.py @@ -83,7 +83,7 @@ def rgb_to_hls(r, g, b): if l <= 0.5: s = rangec / sumc else: - s = rangec / (2.0-sumc) + s = rangec / (2.0-maxc-minc) # Not always 2.0-sumc: gh-106498. rc = (maxc-r) / rangec gc = (maxc-g) / rangec bc = (maxc-b) / rangec diff --git a/Lib/test/test_colorsys.py b/Lib/test/test_colorsys.py index a24e3adcb4b84..74d76294b0b4d 100644 --- a/Lib/test/test_colorsys.py +++ b/Lib/test/test_colorsys.py @@ -69,6 +69,16 @@ def test_hls_values(self): self.assertTripleEqual(hls, colorsys.rgb_to_hls(*rgb)) self.assertTripleEqual(rgb, colorsys.hls_to_rgb(*hls)) + def test_hls_nearwhite(self): # gh-106498 + values = ( + # rgb, hls: these do not work in reverse + ((0.9999999999999999, 1, 1), (0.5, 1.0, 1.0)), + ((1, 0.9999999999999999, 0.9999999999999999), (0.0, 1.0, 1.0)), + ) + for rgb, hls in values: + self.assertTripleEqual(hls, colorsys.rgb_to_hls(*rgb)) + self.assertTripleEqual((1.0, 1.0, 1.0), colorsys.hls_to_rgb(*hls)) + def test_yiq_roundtrip(self): for r in frange(0.0, 1.0, 0.2): for g in frange(0.0, 1.0, 0.2): diff --git a/Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst b/Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst new file mode 100644 index 0000000000000..09fc647cc01d2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst @@ -0,0 +1,2 @@ +Revert a change to :func:`colorsys.rgb_to_hls` that caused division by zero +for certain almost-white inputs. Patch by Terry Jan Reedy. From webhook-mailer at python.org Tue Jul 11 11:48:04 2023 From: webhook-mailer at python.org (terryjreedy) Date: Tue, 11 Jul 2023 15:48:04 -0000 Subject: [Python-checkins] [3.11] gh-106498: Revert incorrect colorsys.rgb_to_hls change (GH-106627) (#106633) Message-ID: https://github.com/python/cpython/commit/da15b12d8d7f4fec89d4e56f61799048b0590c95 commit: da15b12d8d7f4fec89d4e56f61799048b0590c95 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-11T15:48:00Z summary: [3.11] gh-106498: Revert incorrect colorsys.rgb_to_hls change (GH-106627) (#106633) gh-106498: Revert incorrect colorsys.rgb_to_hls change (GH-106627) gh-86618 assumed a-b-c = a-(b+c) = a-d where d = b+d. For floats 2.0, 1.0, and 0.9999999999999999, this assumption is false. The net change of 1.1102230246251565e-16 to 0.0 results in division by 0. Revert the replacement. Add test. (cherry picked from commit a2d54d4e8ab12f967a220be88bde8ac8227c5ab3) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst M Lib/colorsys.py M Lib/test/test_colorsys.py diff --git a/Lib/colorsys.py b/Lib/colorsys.py index 9bdc83e377260..bc897bd0f9929 100644 --- a/Lib/colorsys.py +++ b/Lib/colorsys.py @@ -83,7 +83,7 @@ def rgb_to_hls(r, g, b): if l <= 0.5: s = rangec / sumc else: - s = rangec / (2.0-sumc) + s = rangec / (2.0-maxc-minc) # Not always 2.0-sumc: gh-106498. rc = (maxc-r) / rangec gc = (maxc-g) / rangec bc = (maxc-b) / rangec diff --git a/Lib/test/test_colorsys.py b/Lib/test/test_colorsys.py index a24e3adcb4b84..74d76294b0b4d 100644 --- a/Lib/test/test_colorsys.py +++ b/Lib/test/test_colorsys.py @@ -69,6 +69,16 @@ def test_hls_values(self): self.assertTripleEqual(hls, colorsys.rgb_to_hls(*rgb)) self.assertTripleEqual(rgb, colorsys.hls_to_rgb(*hls)) + def test_hls_nearwhite(self): # gh-106498 + values = ( + # rgb, hls: these do not work in reverse + ((0.9999999999999999, 1, 1), (0.5, 1.0, 1.0)), + ((1, 0.9999999999999999, 0.9999999999999999), (0.0, 1.0, 1.0)), + ) + for rgb, hls in values: + self.assertTripleEqual(hls, colorsys.rgb_to_hls(*rgb)) + self.assertTripleEqual((1.0, 1.0, 1.0), colorsys.hls_to_rgb(*hls)) + def test_yiq_roundtrip(self): for r in frange(0.0, 1.0, 0.2): for g in frange(0.0, 1.0, 0.2): diff --git a/Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst b/Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst new file mode 100644 index 0000000000000..09fc647cc01d2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-11-09-25-40.gh-issue-106530.VgXrMx.rst @@ -0,0 +1,2 @@ +Revert a change to :func:`colorsys.rgb_to_hls` that caused division by zero +for certain almost-white inputs. Patch by Terry Jan Reedy. From webhook-mailer at python.org Tue Jul 11 12:06:39 2023 From: webhook-mailer at python.org (Yhg1s) Date: Tue, 11 Jul 2023 16:06:39 -0000 Subject: [Python-checkins] Python 3.12.0b4 Message-ID: https://github.com/python/cpython/commit/97a6a418167f1c8bbb014fab813e440b88cf2221 commit: 97a6a418167f1c8bbb014fab813e440b88cf2221 branch: 3.12 author: Thomas Wouters committer: Yhg1s date: 2023-07-11T14:23:30+02:00 summary: Python 3.12.0b4 files: A Misc/NEWS.d/3.12.0b4.rst D Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst D Misc/NEWS.d/next/Build/2023-06-26-21-56-29.gh-issue-106118.0cCfhl.rst D Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst D Misc/NEWS.d/next/Core and Builtins/2023-01-13-11-37-41.gh-issue-101006.fuLvn2.rst D Misc/NEWS.d/next/Core and Builtins/2023-06-08-09-10-15.gh-issue-105486.dev-WS.rst D Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst D Misc/NEWS.d/next/Core and Builtins/2023-06-19-11-04-01.gh-issue-105908.7oanny.rst D Misc/NEWS.d/next/Core and Builtins/2023-06-22-14-19-17.gh-issue-98931.PPgvSF.rst D Misc/NEWS.d/next/Core and Builtins/2023-06-22-19-16-24.gh-issue-105979.TDP2CU.rst D Misc/NEWS.d/next/Core and Builtins/2023-06-29-09-46-41.gh-issue-106145.QC6-Kq.rst D Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst D Misc/NEWS.d/next/Documentation/2023-06-30-19-28-59.gh-issue-106232.hQ4-tz.rst D Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst D Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst D Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst D Misc/NEWS.d/next/Library/2023-06-20-23-18-45.gh-issue-96145.o5dTRM.rst D Misc/NEWS.d/next/Library/2023-06-21-19-04-27.gh-issue-105974.M47n3t.rst D Misc/NEWS.d/next/Library/2023-06-22-15-21-11.gh-issue-105987.T7Kzrb.rst D Misc/NEWS.d/next/Library/2023-06-25-12-28-55.gh-issue-106075.W7tMRb.rst D Misc/NEWS.d/next/Library/2023-06-27-23-22-37.gh-issue-106152.ya5jBT.rst D Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst D Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst D Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst D Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst D Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst D Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst D Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst D Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst D Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst D Misc/NEWS.d/next/Tests/2023-06-28-02-51-08.gh-issue-101634.Rayczr.rst D Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst M Include/patchlevel.h M Lib/pydoc_data/topics.py M README.rst diff --git a/Include/patchlevel.h b/Include/patchlevel.h index aa35c24a9fa30..62d7e77bdaa8f 100644 --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 12 #define PY_MICRO_VERSION 0 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_BETA -#define PY_RELEASE_SERIAL 3 +#define PY_RELEASE_SERIAL 4 /* Version as a string */ -#define PY_VERSION "3.12.0b3+" +#define PY_VERSION "3.12.0b4" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py index 9a6a1b3754572..9603975729cfa 100644 --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,6 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Mon Jun 19 20:55:48 2023 +# Autogenerated by Sphinx on Tue Jul 11 14:22:58 2023 +# as part of the release process. topics = {'assert': 'The "assert" statement\n' '**********************\n' '\n' @@ -208,7 +209,7 @@ 'the\n' ' subscript must have a type compatible with the mapping?s key ' 'type,\n' - ' and the mapping is then asked to create a key/datum pair ' + ' and the mapping is then asked to create a key/value pair ' 'which maps\n' ' the subscript to the assigned object. This can either ' 'replace an\n' @@ -5687,30 +5688,31 @@ 'dict': 'Dictionary displays\n' '*******************\n' '\n' - 'A dictionary display is a possibly empty series of key/datum pairs\n' - 'enclosed in curly braces:\n' + 'A dictionary display is a possibly empty series of dict items\n' + '(key/value pairs) enclosed in curly braces:\n' '\n' - ' dict_display ::= "{" [key_datum_list | dict_comprehension] ' + ' dict_display ::= "{" [dict_item_list | dict_comprehension] ' '"}"\n' - ' key_datum_list ::= key_datum ("," key_datum)* [","]\n' - ' key_datum ::= expression ":" expression | "**" or_expr\n' + ' dict_item_list ::= dict_item ("," dict_item)* [","]\n' + ' dict_item ::= expression ":" expression | "**" or_expr\n' ' dict_comprehension ::= expression ":" expression comp_for\n' '\n' 'A dictionary display yields a new dictionary object.\n' '\n' - 'If a comma-separated sequence of key/datum pairs is given, they are\n' + 'If a comma-separated sequence of dict items is given, they are\n' 'evaluated from left to right to define the entries of the ' 'dictionary:\n' 'each key object is used as a key into the dictionary to store the\n' - 'corresponding datum. This means that you can specify the same key\n' - 'multiple times in the key/datum list, and the final dictionary?s ' + 'corresponding value. This means that you can specify the same key\n' + 'multiple times in the dict item list, and the final dictionary?s ' 'value\n' 'for that key will be the last one given.\n' '\n' 'A double asterisk "**" denotes *dictionary unpacking*. Its operand\n' 'must be a *mapping*. Each mapping item is added to the new\n' - 'dictionary. Later values replace values already set by earlier\n' - 'key/datum pairs and earlier dictionary unpackings.\n' + 'dictionary. Later values replace values already set by earlier ' + 'dict\n' + 'items and earlier dictionary unpackings.\n' '\n' 'New in version 3.5: Unpacking into dictionary displays, originally\n' 'proposed by **PEP 448**.\n' @@ -5726,7 +5728,7 @@ 'Restrictions on the types of the key values are listed earlier in\n' 'section The standard type hierarchy. (To summarize, the key type\n' 'should be *hashable*, which excludes all mutable objects.) Clashes\n' - 'between duplicate keys are not detected; the last datum (textually\n' + 'between duplicate keys are not detected; the last value (textually\n' 'rightmost in the display) stored for a given key value prevails.\n' '\n' 'Changed in version 3.8: Prior to Python 3.8, in dict ' @@ -13256,7 +13258,7 @@ 'are\n' 'most of the built-in objects considered false:\n' '\n' - '* constants defined to be false: "None" and "False".\n' + '* constants defined to be false: "None" and "False"\n' '\n' '* zero of any numeric type: "0", "0.0", "0j", "Decimal(0)",\n' ' "Fraction(0, 1)"\n' diff --git a/Misc/NEWS.d/3.12.0b4.rst b/Misc/NEWS.d/3.12.0b4.rst new file mode 100644 index 0000000000000..5212f359ec506 --- /dev/null +++ b/Misc/NEWS.d/3.12.0b4.rst @@ -0,0 +1,318 @@ +.. date: 2023-06-13-20-52-24 +.. gh-issue: 102988 +.. nonce: Kei7Vf +.. release date: 2023-07-11 +.. section: Security + +CVE-2023-27043: Prevent :func:`email.utils.parseaddr` and +:func:`email.utils.getaddresses` from returning the realname portion of an +invalid RFC2822 email header in the email address portion of the 2-tuple +returned after being parsed by :class:`email._parseaddr.AddressList`. + +.. + +.. date: 2023-07-04-09-51-45 +.. gh-issue: 106396 +.. nonce: DmYp7x +.. section: Core and Builtins + +When the format specification of an f-string expression is empty, the parser +now generates an empty :class:`ast.JoinedStr` node for it instead of an +one-element :class:`ast.JoinedStr` with an empty string +:class:`ast.Constant`. + +.. + +.. date: 2023-06-29-09-46-41 +.. gh-issue: 106145 +.. nonce: QC6-Kq +.. section: Core and Builtins + +Make ``end_lineno`` and ``end_col_offset`` required on ``type_param`` ast +nodes. + +.. + +.. date: 2023-06-22-19-16-24 +.. gh-issue: 105979 +.. nonce: TDP2CU +.. section: Core and Builtins + +Fix crash in :func:`!_imp.get_frozen_object` due to improper exception +handling. + +.. + +.. date: 2023-06-22-14-19-17 +.. gh-issue: 98931 +.. nonce: PPgvSF +.. section: Core and Builtins + +Ensure custom :exc:`SyntaxError` error messages are raised for invalid +imports with multiple targets. Patch by Pablo Galindo + +.. + +.. date: 2023-06-19-11-04-01 +.. gh-issue: 105908 +.. nonce: 7oanny +.. section: Core and Builtins + +Fixed bug where :gh:`99111` breaks future import ``barry_as_FLUFL`` in the +Python REPL. + +.. + +.. date: 2023-06-12-16-38-31 +.. gh-issue: 105340 +.. nonce: _jRHXe +.. section: Core and Builtins + +Include the comprehension iteration variable in ``locals()`` inside a +module- or class-scope comprehension. + +.. + +.. date: 2023-06-08-09-10-15 +.. gh-issue: 105486 +.. nonce: dev-WS +.. section: Core and Builtins + +Change the repr of ``ParamSpec`` list of args in ``types.GenericAlias``. + +.. + +.. date: 2023-01-13-11-37-41 +.. gh-issue: 101006 +.. nonce: fuLvn2 +.. section: Core and Builtins + +Improve error handling when read :mod:`marshal` data. + +.. + +.. date: 2023-07-07-17-44-03 +.. gh-issue: 106524 +.. nonce: XkBV8h +.. section: Library + +Fix crash in :func:`!_sre.template` with templates containing invalid group +indices. + +.. + +.. date: 2023-07-07-13-47-28 +.. gh-issue: 106510 +.. nonce: 9n5BdC +.. section: Library + +Improve debug output for atomic groups in regular expressions. + +.. + +.. date: 2023-07-07-03-05-58 +.. gh-issue: 106503 +.. nonce: ltfeiH +.. section: Library + +Fix ref cycle in :class:`!asyncio._SelectorSocketTransport` by removing +``_write_ready`` in ``close``. + +.. + +.. date: 2023-07-05-14-34-10 +.. gh-issue: 105497 +.. nonce: HU5u89 +.. section: Library + +Fix flag mask inversion when unnamed flags exist. + +.. + +.. date: 2023-07-05-13-08-23 +.. gh-issue: 90876 +.. nonce: Qvlkfl +.. section: Library + +Prevent :mod:`multiprocessing.spawn` from failing to *import* in +environments where ``sys.executable`` is ``None``. This regressed in 3.11 +with the addition of support for path-like objects in multiprocessing. + +.. + +.. date: 2023-07-03-15-09-44 +.. gh-issue: 106292 +.. nonce: 3npldV +.. section: Library + +Check for an instance-dict cached value in the :meth:`__get__` method of +:func:`functools.cached_property`. This better matches the pre-3.12 behavior +and improves compatibility for users subclassing +:func:`functools.cached_property` and adding a :meth:`__set__` method. + +.. + +.. date: 2023-07-02-10-56-41 +.. gh-issue: 106330 +.. nonce: QSkIUH +.. section: Library + +Fix incorrect matching of empty paths in :meth:`pathlib.PurePath.match`. +This bug was introduced in Python 3.12.0 beta 1. + +.. + +.. date: 2023-07-01-16-40-54 +.. gh-issue: 102541 +.. nonce: C1ahtk +.. section: Library + +Make pydoc.doc catch bad module ImportError when output stream is not None. + +.. + +.. date: 2023-06-27-23-22-37 +.. gh-issue: 106152 +.. nonce: ya5jBT +.. section: Library + +Added PY_THROW event hook for :mod:`cProfile` for generators + +.. + +.. date: 2023-06-25-12-28-55 +.. gh-issue: 106075 +.. nonce: W7tMRb +.. section: Library + +Added `asyncio.taskgroups.__all__` to `asyncio.__all__` for export in star +imports. + +.. + +.. date: 2023-06-22-15-21-11 +.. gh-issue: 105987 +.. nonce: T7Kzrb +.. section: Library + +Fix crash due to improper reference counting in :mod:`asyncio` eager task +factory internal routines. + +.. + +.. date: 2023-06-21-19-04-27 +.. gh-issue: 105974 +.. nonce: M47n3t +.. section: Library + +Fix bug where a :class:`typing.Protocol` class that had one or more +non-callable members would raise :exc:`TypeError` when :func:`issubclass` +was called against it, even if it defined a custom ``__subclasshook__`` +method. The behaviour in Python 3.11 and lower -- which has now been +restored -- was not to raise :exc:`TypeError` in these situations if a +custom ``__subclasshook__`` method was defined. Patch by Alex Waygood. + +.. + +.. date: 2023-06-20-23-18-45 +.. gh-issue: 96145 +.. nonce: o5dTRM +.. section: Library + +Reverted addition of ``json.AttrDict``. + +.. + +.. date: 2023-06-08-17-49-46 +.. gh-issue: 105497 +.. nonce: K6Q8nU +.. section: Library + +Fix flag inversion when alias/mask members exist. + +.. + +.. date: 2023-06-05-14-43-56 +.. gh-issue: 104554 +.. nonce: pwfKIo +.. section: Library + +Add RTSPS scheme support in urllib.parse + +.. + +.. date: 2022-07-12-18-45-13 +.. gh-issue: 94777 +.. nonce: mOybx7 +.. section: Library + +Fix hanging :mod:`multiprocessing` ``ProcessPoolExecutor`` when a child +process crashes while data is being written in the call queue. + +.. + +.. date: 2023-06-30-19-28-59 +.. gh-issue: 106232 +.. nonce: hQ4-tz +.. section: Documentation + +Make timeit doc command lines compatible with Windows by using double quotes +for arguments. This works on linux and macOS also. + +.. + +.. date: 2023-06-28-02-51-08 +.. gh-issue: 101634 +.. nonce: Rayczr +.. section: Tests + +When running the Python test suite with ``-jN`` option, if a worker stdout +cannot be decoded from the locale encoding report a failed testn so the +exitcode is non-zero. Patch by Victor Stinner. + +.. + +.. date: 2023-06-26-21-56-29 +.. gh-issue: 106118 +.. nonce: 0cCfhl +.. section: Build + +Fix compilation for platforms without :data:`!O_CLOEXEC`. The issue was +introduced with Python 3.12b1 in :gh:`103295`. Patch by Erlend Aasland. + +.. + +.. date: 2023-05-20-23-49-30 +.. gh-issue: 104692 +.. nonce: s5UIu5 +.. section: Build + +Include ``commoninstall`` as a prerequisite for ``bininstall`` + +This ensures that ``commoninstall`` is completed before ``bininstall`` is +started when parallel builds are used (``make -j install``), and so the +``python3`` symlink is only installed after all standard library modules are +installed. + +.. + +.. date: 2023-07-03-14-06-19 +.. gh-issue: 106359 +.. nonce: RfJuR0 +.. section: Tools/Demos + +Argument Clinic now explicitly forbids "kwarg splats" in function calls used +as annotations. + +.. + +.. date: 2023-06-13-14-24-55 +.. gh-issue: 105227 +.. nonce: HDL9aF +.. section: C API + +The new :c:func:`PyType_GetDict` provides the dictionary for the given type +object that is normally exposed by ``cls.__dict__``. Normally it's +sufficient to use :c:member:`~PyTypeObject.tp_dict`, but for the static +builtin types :c:member:`!tp_dict` is now always ``NULL``. +:c:func:`!PyType_GetDict()` provides the correct dict object instead. diff --git a/Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst b/Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst deleted file mode 100644 index 2936990999e1a..0000000000000 --- a/Misc/NEWS.d/next/Build/2023-05-20-23-49-30.gh-issue-104692.s5UIu5.rst +++ /dev/null @@ -1,6 +0,0 @@ -Include ``commoninstall`` as a prerequisite for ``bininstall`` - -This ensures that ``commoninstall`` is completed before ``bininstall`` -is started when parallel builds are used (``make -j install``), and so -the ``python3`` symlink is only installed after all standard library -modules are installed. diff --git a/Misc/NEWS.d/next/Build/2023-06-26-21-56-29.gh-issue-106118.0cCfhl.rst b/Misc/NEWS.d/next/Build/2023-06-26-21-56-29.gh-issue-106118.0cCfhl.rst deleted file mode 100644 index f93cae5d03b53..0000000000000 --- a/Misc/NEWS.d/next/Build/2023-06-26-21-56-29.gh-issue-106118.0cCfhl.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix compilation for platforms without :data:`!O_CLOEXEC`. The issue was -introduced with Python 3.12b1 in :gh:`103295`. Patch by Erlend Aasland. diff --git a/Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst b/Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst deleted file mode 100644 index 846663621e868..0000000000000 --- a/Misc/NEWS.d/next/C API/2023-06-13-14-24-55.gh-issue-105227.HDL9aF.rst +++ /dev/null @@ -1,5 +0,0 @@ -The new :c:func:`PyType_GetDict` provides the dictionary for the given type -object that is normally exposed by ``cls.__dict__``. Normally it's -sufficient to use :c:member:`~PyTypeObject.tp_dict`, but for the static -builtin types :c:member:`!tp_dict` is now always ``NULL``. :c:func:`!PyType_GetDict()` -provides the correct dict object instead. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-01-13-11-37-41.gh-issue-101006.fuLvn2.rst b/Misc/NEWS.d/next/Core and Builtins/2023-01-13-11-37-41.gh-issue-101006.fuLvn2.rst deleted file mode 100644 index c98670d8c4963..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2023-01-13-11-37-41.gh-issue-101006.fuLvn2.rst +++ /dev/null @@ -1 +0,0 @@ -Improve error handling when read :mod:`marshal` data. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-08-09-10-15.gh-issue-105486.dev-WS.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-08-09-10-15.gh-issue-105486.dev-WS.rst deleted file mode 100644 index 9f735db3dc89c..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2023-06-08-09-10-15.gh-issue-105486.dev-WS.rst +++ /dev/null @@ -1 +0,0 @@ -Change the repr of ``ParamSpec`` list of args in ``types.GenericAlias``. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst deleted file mode 100644 index f6d4fa8fc4d74..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2023-06-12-16-38-31.gh-issue-105340._jRHXe.rst +++ /dev/null @@ -1,2 +0,0 @@ -Include the comprehension iteration variable in ``locals()`` inside a -module- or class-scope comprehension. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-19-11-04-01.gh-issue-105908.7oanny.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-19-11-04-01.gh-issue-105908.7oanny.rst deleted file mode 100644 index 03db3f064f503..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2023-06-19-11-04-01.gh-issue-105908.7oanny.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed bug where :gh:`99111` breaks future import ``barry_as_FLUFL`` in the Python REPL. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-22-14-19-17.gh-issue-98931.PPgvSF.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-22-14-19-17.gh-issue-98931.PPgvSF.rst deleted file mode 100644 index 611660d628626..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2023-06-22-14-19-17.gh-issue-98931.PPgvSF.rst +++ /dev/null @@ -1,2 +0,0 @@ -Ensure custom :exc:`SyntaxError` error messages are raised for invalid -imports with multiple targets. Patch by Pablo Galindo diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-22-19-16-24.gh-issue-105979.TDP2CU.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-22-19-16-24.gh-issue-105979.TDP2CU.rst deleted file mode 100644 index be6962afd0c78..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2023-06-22-19-16-24.gh-issue-105979.TDP2CU.rst +++ /dev/null @@ -1 +0,0 @@ -Fix crash in :func:`!_imp.get_frozen_object` due to improper exception handling. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-29-09-46-41.gh-issue-106145.QC6-Kq.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-29-09-46-41.gh-issue-106145.QC6-Kq.rst deleted file mode 100644 index 4f9445bbcbe55..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2023-06-29-09-46-41.gh-issue-106145.QC6-Kq.rst +++ /dev/null @@ -1,2 +0,0 @@ -Make ``end_lineno`` and ``end_col_offset`` required on ``type_param`` ast -nodes. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst deleted file mode 100644 index c5767e97271d9..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2023-07-04-09-51-45.gh-issue-106396.DmYp7x.rst +++ /dev/null @@ -1,3 +0,0 @@ -When the format specification of an f-string expression is empty, the parser now -generates an empty :class:`ast.JoinedStr` node for it instead of an one-element -:class:`ast.JoinedStr` with an empty string :class:`ast.Constant`. diff --git a/Misc/NEWS.d/next/Documentation/2023-06-30-19-28-59.gh-issue-106232.hQ4-tz.rst b/Misc/NEWS.d/next/Documentation/2023-06-30-19-28-59.gh-issue-106232.hQ4-tz.rst deleted file mode 100644 index bc16f92b7d647..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2023-06-30-19-28-59.gh-issue-106232.hQ4-tz.rst +++ /dev/null @@ -1,2 +0,0 @@ -Make timeit doc command lines compatible with Windows by using double quotes -for arguments. This works on linux and macOS also. diff --git a/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst b/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst deleted file mode 100644 index 2c04a35fbfce1..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-07-12-18-45-13.gh-issue-94777.mOybx7.rst +++ /dev/null @@ -1 +0,0 @@ -Fix hanging :mod:`multiprocessing` ``ProcessPoolExecutor`` when a child process crashes while data is being written in the call queue. diff --git a/Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst b/Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst deleted file mode 100644 index 9ef8c67459c40..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-06-05-14-43-56.gh-issue-104554.pwfKIo.rst +++ /dev/null @@ -1 +0,0 @@ -Add RTSPS scheme support in urllib.parse diff --git a/Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst b/Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst deleted file mode 100644 index 2d4e2091b5071..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-06-08-17-49-46.gh-issue-105497.K6Q8nU.rst +++ /dev/null @@ -1 +0,0 @@ -Fix flag inversion when alias/mask members exist. diff --git a/Misc/NEWS.d/next/Library/2023-06-20-23-18-45.gh-issue-96145.o5dTRM.rst b/Misc/NEWS.d/next/Library/2023-06-20-23-18-45.gh-issue-96145.o5dTRM.rst deleted file mode 100644 index f4fb0e46ce5e5..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-06-20-23-18-45.gh-issue-96145.o5dTRM.rst +++ /dev/null @@ -1 +0,0 @@ -Reverted addition of ``json.AttrDict``. diff --git a/Misc/NEWS.d/next/Library/2023-06-21-19-04-27.gh-issue-105974.M47n3t.rst b/Misc/NEWS.d/next/Library/2023-06-21-19-04-27.gh-issue-105974.M47n3t.rst deleted file mode 100644 index 982192e59e3a5..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-06-21-19-04-27.gh-issue-105974.M47n3t.rst +++ /dev/null @@ -1,6 +0,0 @@ -Fix bug where a :class:`typing.Protocol` class that had one or more -non-callable members would raise :exc:`TypeError` when :func:`issubclass` -was called against it, even if it defined a custom ``__subclasshook__`` -method. The behaviour in Python 3.11 and lower -- which has now been -restored -- was not to raise :exc:`TypeError` in these situations if a -custom ``__subclasshook__`` method was defined. Patch by Alex Waygood. diff --git a/Misc/NEWS.d/next/Library/2023-06-22-15-21-11.gh-issue-105987.T7Kzrb.rst b/Misc/NEWS.d/next/Library/2023-06-22-15-21-11.gh-issue-105987.T7Kzrb.rst deleted file mode 100644 index 0bc97da4edf0f..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-06-22-15-21-11.gh-issue-105987.T7Kzrb.rst +++ /dev/null @@ -1 +0,0 @@ -Fix crash due to improper reference counting in :mod:`asyncio` eager task factory internal routines. diff --git a/Misc/NEWS.d/next/Library/2023-06-25-12-28-55.gh-issue-106075.W7tMRb.rst b/Misc/NEWS.d/next/Library/2023-06-25-12-28-55.gh-issue-106075.W7tMRb.rst deleted file mode 100644 index d2687154a5859..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-06-25-12-28-55.gh-issue-106075.W7tMRb.rst +++ /dev/null @@ -1 +0,0 @@ -Added `asyncio.taskgroups.__all__` to `asyncio.__all__` for export in star imports. diff --git a/Misc/NEWS.d/next/Library/2023-06-27-23-22-37.gh-issue-106152.ya5jBT.rst b/Misc/NEWS.d/next/Library/2023-06-27-23-22-37.gh-issue-106152.ya5jBT.rst deleted file mode 100644 index da9d2605f4629..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-06-27-23-22-37.gh-issue-106152.ya5jBT.rst +++ /dev/null @@ -1 +0,0 @@ -Added PY_THROW event hook for :mod:`cProfile` for generators diff --git a/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst b/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst deleted file mode 100644 index efaf5db10f3e1..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-07-01-16-40-54.gh-issue-102541.C1ahtk.rst +++ /dev/null @@ -1 +0,0 @@ -Make pydoc.doc catch bad module ImportError when output stream is not None. diff --git a/Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst b/Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst deleted file mode 100644 index c1f55ab658b51..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-07-02-10-56-41.gh-issue-106330.QSkIUH.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix incorrect matching of empty paths in :meth:`pathlib.PurePath.match`. -This bug was introduced in Python 3.12.0 beta 1. diff --git a/Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst b/Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst deleted file mode 100644 index 233509344d509..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-07-03-15-09-44.gh-issue-106292.3npldV.rst +++ /dev/null @@ -1,4 +0,0 @@ -Check for an instance-dict cached value in the :meth:`__get__` method of -:func:`functools.cached_property`. This better matches the pre-3.12 behavior -and improves compatibility for users subclassing -:func:`functools.cached_property` and adding a :meth:`__set__` method. diff --git a/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst b/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst deleted file mode 100644 index 3e062b5add6d8..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-07-05-13-08-23.gh-issue-90876.Qvlkfl.rst +++ /dev/null @@ -1,3 +0,0 @@ -Prevent :mod:`multiprocessing.spawn` from failing to *import* in environments -where ``sys.executable`` is ``None``. This regressed in 3.11 with the addition -of support for path-like objects in multiprocessing. diff --git a/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst b/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst deleted file mode 100644 index f4f2db08f73f5..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-07-05-14-34-10.gh-issue-105497.HU5u89.rst +++ /dev/null @@ -1 +0,0 @@ -Fix flag mask inversion when unnamed flags exist. diff --git a/Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst b/Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst deleted file mode 100644 index b8dd850386e86..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-07-07-03-05-58.gh-issue-106503.ltfeiH.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix ref cycle in :class:`!asyncio._SelectorSocketTransport` by removing -``_write_ready`` in ``close``. diff --git a/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst b/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst deleted file mode 100644 index e0646fa9bc021..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-07-07-13-47-28.gh-issue-106510.9n5BdC.rst +++ /dev/null @@ -1 +0,0 @@ -Improve debug output for atomic groups in regular expressions. diff --git a/Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst b/Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst deleted file mode 100644 index f3fd070e391a6..0000000000000 --- a/Misc/NEWS.d/next/Library/2023-07-07-17-44-03.gh-issue-106524.XkBV8h.rst +++ /dev/null @@ -1 +0,0 @@ -Fix crash in :func:`!_sre.template` with templates containing invalid group indices. diff --git a/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst b/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst deleted file mode 100644 index e0434ccd2ccab..0000000000000 --- a/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst +++ /dev/null @@ -1,4 +0,0 @@ -CVE-2023-27043: Prevent :func:`email.utils.parseaddr` -and :func:`email.utils.getaddresses` from returning the realname portion of an -invalid RFC2822 email header in the email address portion of the 2-tuple -returned after being parsed by :class:`email._parseaddr.AddressList`. diff --git a/Misc/NEWS.d/next/Tests/2023-06-28-02-51-08.gh-issue-101634.Rayczr.rst b/Misc/NEWS.d/next/Tests/2023-06-28-02-51-08.gh-issue-101634.Rayczr.rst deleted file mode 100644 index 6fbfc84c19e1b..0000000000000 --- a/Misc/NEWS.d/next/Tests/2023-06-28-02-51-08.gh-issue-101634.Rayczr.rst +++ /dev/null @@ -1,3 +0,0 @@ -When running the Python test suite with ``-jN`` option, if a worker stdout -cannot be decoded from the locale encoding report a failed testn so the -exitcode is non-zero. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst b/Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst deleted file mode 100644 index 600c265391ec5..0000000000000 --- a/Misc/NEWS.d/next/Tools-Demos/2023-07-03-14-06-19.gh-issue-106359.RfJuR0.rst +++ /dev/null @@ -1,2 +0,0 @@ -Argument Clinic now explicitly forbids "kwarg splats" in function calls used as -annotations. diff --git a/README.rst b/README.rst index daa30b1a65dfb..5257beacd3185 100644 --- a/README.rst +++ b/README.rst @@ -1,4 +1,4 @@ -This is Python version 3.12.0 beta 3 +This is Python version 3.12.0 beta 4 ===================================== .. image:: https://github.com/python/cpython/workflows/Tests/badge.svg From webhook-mailer at python.org Tue Jul 11 12:15:34 2023 From: webhook-mailer at python.org (JelleZijlstra) Date: Tue, 11 Jul 2023 16:15:34 -0000 Subject: [Python-checkins] [3.12] gh-106403: Restore weakref support for TypeVar and friends (GH-106418) (#106635) Message-ID: https://github.com/python/cpython/commit/58f9c8889d8fa79d617fddf6cb48942d3003c7fd commit: 58f9c8889d8fa79d617fddf6cb48942d3003c7fd branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: JelleZijlstra date: 2023-07-11T16:15:30Z summary: [3.12] gh-106403: Restore weakref support for TypeVar and friends (GH-106418) (#106635) gh-106403: Restore weakref support for TypeVar and friends (GH-106418) (cherry picked from commit 945d3cbf2e8e756ed16c3ec51106e6157abb2698) Co-authored-by: Jelle Zijlstra files: A Misc/NEWS.d/next/Library/2023-07-04-07-25-30.gh-issue-106403.GmefbV.rst M Lib/test/test_type_params.py M Objects/typevarobject.c diff --git a/Lib/test/test_type_params.py b/Lib/test/test_type_params.py index 3026cc2247661..bced641a9661f 100644 --- a/Lib/test/test_type_params.py +++ b/Lib/test/test_type_params.py @@ -3,6 +3,7 @@ import types import unittest import pickle +import weakref from test.support import requires_working_socket, check_syntax_error, run_code from typing import Generic, Sequence, TypeVar, TypeVarTuple, ParamSpec, get_args @@ -921,3 +922,33 @@ def test_pickling_classes(self): # These instances are not equal, # but class check is good enough: self.assertIsInstance(pickle.loads(pickled), real_class) + + +class TypeParamsWeakRefTest(unittest.TestCase): + def test_weakrefs(self): + T = TypeVar('T') + P = ParamSpec('P') + class OldStyle(Generic[T]): + pass + + class NewStyle[T]: + pass + + cases = [ + T, + TypeVar('T', bound=int), + P, + P.args, + P.kwargs, + TypeVarTuple('Ts'), + OldStyle, + OldStyle[int], + OldStyle(), + NewStyle, + NewStyle[int], + NewStyle(), + Generic[T], + ] + for case in cases: + with self.subTest(case=case): + weakref.ref(case) diff --git a/Misc/NEWS.d/next/Library/2023-07-04-07-25-30.gh-issue-106403.GmefbV.rst b/Misc/NEWS.d/next/Library/2023-07-04-07-25-30.gh-issue-106403.GmefbV.rst new file mode 100644 index 0000000000000..4fea45f16c4f8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-04-07-25-30.gh-issue-106403.GmefbV.rst @@ -0,0 +1,4 @@ +Instances of :class:`typing.TypeVar`, :class:`typing.ParamSpec`, +:class:`typing.ParamSpecArgs`, :class:`typing.ParamSpecKwargs`, and +:class:`typing.TypeVarTuple` once again support weak references, fixing a +regression introduced in Python 3.12.0 beta 1. Patch by Jelle Zijlstra. diff --git a/Objects/typevarobject.c b/Objects/typevarobject.c index 406a6eb76e3a8..97f39136996fc 100644 --- a/Objects/typevarobject.c +++ b/Objects/typevarobject.c @@ -500,7 +500,7 @@ PyType_Spec typevar_spec = { .name = "typing.TypeVar", .basicsize = sizeof(typevarobject), .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE - | Py_TPFLAGS_MANAGED_DICT, + | Py_TPFLAGS_MANAGED_DICT | Py_TPFLAGS_MANAGED_WEAKREF, .slots = typevar_slots, }; @@ -647,7 +647,8 @@ static PyType_Slot paramspecargs_slots[] = { PyType_Spec paramspecargs_spec = { .name = "typing.ParamSpecArgs", .basicsize = sizeof(paramspecattrobject), - .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE, + .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE + | Py_TPFLAGS_MANAGED_WEAKREF, .slots = paramspecargs_slots, }; @@ -726,7 +727,8 @@ static PyType_Slot paramspeckwargs_slots[] = { PyType_Spec paramspeckwargs_spec = { .name = "typing.ParamSpecKwargs", .basicsize = sizeof(paramspecattrobject), - .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE, + .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE + | Py_TPFLAGS_MANAGED_WEAKREF, .slots = paramspeckwargs_slots, }; @@ -1007,7 +1009,7 @@ PyType_Spec paramspec_spec = { .name = "typing.ParamSpec", .basicsize = sizeof(paramspecobject), .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE - | Py_TPFLAGS_MANAGED_DICT, + | Py_TPFLAGS_MANAGED_DICT | Py_TPFLAGS_MANAGED_WEAKREF, .slots = paramspec_slots, }; @@ -1228,7 +1230,7 @@ PyType_Spec typevartuple_spec = { .name = "typing.TypeVarTuple", .basicsize = sizeof(typevartupleobject), .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_IMMUTABLETYPE | Py_TPFLAGS_MANAGED_DICT - | Py_TPFLAGS_HAVE_GC, + | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_MANAGED_WEAKREF, .slots = typevartuple_slots, }; From webhook-mailer at python.org Tue Jul 11 13:22:11 2023 From: webhook-mailer at python.org (terryjreedy) Date: Tue, 11 Jul 2023 17:22:11 -0000 Subject: [Python-checkins] gh-106625 : Add missing code to tutorial 4.6 example (#106623) Message-ID: https://github.com/python/cpython/commit/d0b7e18262e69dd4b8252e804e4f98fc9533bcd6 commit: d0b7e18262e69dd4b8252e804e4f98fc9533bcd6 branch: main author: RustyNail committer: terryjreedy date: 2023-07-11T13:22:07-04:00 summary: gh-106625 : Add missing code to tutorial 4.6 example (#106623) * Added missing import statement. * Update Doc/tutorial/controlflow.rst * Update Doc/tutorial/controlflow.rst * Update controlflow.rst * Make point regular class with __init__. --------- Co-authored-by: Terry Jan Reedy files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index c9b3d982c31c9..4336bf50df40a 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -307,8 +307,9 @@ you can use the class name followed by an argument list resembling a constructor, but with the ability to capture attributes into variables:: class Point: - x: int - y: int + def __init__(self, x, y): + self.x = x + self.y = y def where_is(point): match point: From webhook-mailer at python.org Tue Jul 11 13:40:21 2023 From: webhook-mailer at python.org (terryjreedy) Date: Tue, 11 Jul 2023 17:40:21 -0000 Subject: [Python-checkins] [3.11] gh-106625 : Add missing code to tutorial 4.6 example (GH-106623) (#106637) Message-ID: https://github.com/python/cpython/commit/549b528abeb7a8594967c367ce8ac5e79f96949c commit: 549b528abeb7a8594967c367ce8ac5e79f96949c branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-11T13:40:17-04:00 summary: [3.11] gh-106625 : Add missing code to tutorial 4.6 example (GH-106623) (#106637) (cherry picked from commit d0b7e18262e69dd4b8252e804e4f98fc9533bcd6) Co-authored-by: RustyNail Co-authored-by: Terry Jan Reedy files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index c9b3d982c31c9..4336bf50df40a 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -307,8 +307,9 @@ you can use the class name followed by an argument list resembling a constructor, but with the ability to capture attributes into variables:: class Point: - x: int - y: int + def __init__(self, x, y): + self.x = x + self.y = y def where_is(point): match point: From webhook-mailer at python.org Tue Jul 11 13:41:02 2023 From: webhook-mailer at python.org (terryjreedy) Date: Tue, 11 Jul 2023 17:41:02 -0000 Subject: [Python-checkins] [3.12] gh-106625 : Add missing code to tutorial 4.6 example (GH-106623) (#106636) Message-ID: https://github.com/python/cpython/commit/139e7ac7c2a798820554abb6c84b9feaf810c493 commit: 139e7ac7c2a798820554abb6c84b9feaf810c493 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-11T13:40:59-04:00 summary: [3.12] gh-106625 : Add missing code to tutorial 4.6 example (GH-106623) (#106636) (cherry picked from commit d0b7e18262e69dd4b8252e804e4f98fc9533bcd6) Co-authored-by: RustyNail Co-authored-by: Terry Jan Reedy files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index c9b3d982c31c9..4336bf50df40a 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -307,8 +307,9 @@ you can use the class name followed by an argument list resembling a constructor, but with the ability to capture attributes into variables:: class Point: - x: int - y: int + def __init__(self, x, y): + self.x = x + self.y = y def where_is(point): match point: From webhook-mailer at python.org Tue Jul 11 14:04:13 2023 From: webhook-mailer at python.org (terryjreedy) Date: Tue, 11 Jul 2023 18:04:13 -0000 Subject: [Python-checkins] gh-102541: Add test case for help() for non_existent_module (#106340) Message-ID: https://github.com/python/cpython/commit/292ac4bfe92768140c2d383fd329cfa1949869b2 commit: 292ac4bfe92768140c2d383fd329cfa1949869b2 branch: main author: Kirill Podoprigora committer: terryjreedy date: 2023-07-11T14:04:09-04:00 summary: gh-102541: Add test case for help() for non_existent_module (#106340) Test fix for when one enters, for instance, 'abd' at the 'help>' prompt. --------- Co-authored-by: Terry Jan Reedy files: M Lib/test/test_pydoc.py diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py index 115ffd3c29d4d..ddb5187f90da9 100644 --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -24,7 +24,8 @@ from urllib.request import urlopen, urlcleanup from test.support import import_helper from test.support import os_helper -from test.support.script_helper import assert_python_ok, assert_python_failure +from test.support.script_helper import (assert_python_ok, + assert_python_failure, spawn_python) from test.support import threading_helper from test.support import (reap_children, captured_output, captured_stdout, captured_stderr, is_emscripten, is_wasi, @@ -631,6 +632,14 @@ def test_builtin_on_metaclasses(self): # Testing that the subclasses section does not appear self.assertNotIn('Built-in subclasses', text) + def test_fail_help_cli(self): + elines = (missing_pattern % 'abd').splitlines() + with spawn_python("-c" "help()") as proc: + out, _ = proc.communicate(b"abd") + olines = out.decode().splitlines()[-9:-6] + olines[0] = olines[0].removeprefix('help> ') + self.assertEqual(elines, olines) + def test_fail_help_output_redirect(self): with StringIO() as buf: helper = pydoc.Helper(output=buf) From webhook-mailer at python.org Tue Jul 11 14:08:14 2023 From: webhook-mailer at python.org (gvanrossum) Date: Tue, 11 Jul 2023 18:08:14 -0000 Subject: [Python-checkins] gh-106529: Support JUMP_BACKWARD in Tier 2 (uops) (#106543) Message-ID: https://github.com/python/cpython/commit/cabd6e8a107127ff02f0b514148f648fb2472a58 commit: cabd6e8a107127ff02f0b514148f648fb2472a58 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-11T18:08:10Z summary: gh-106529: Support JUMP_BACKWARD in Tier 2 (uops) (#106543) During superblock generation, a JUMP_BACKWARD instruction is translated to either a JUMP_TO_TOP micro-op (when the target of the jump is exactly the beginning of the superblock, closing the loop), or a SAVE_IP + EXIT_TRACE pair, when the jump goes elsewhere. The new JUMP_TO_TOP instruction includes a CHECK_EVAL_BREAKER() call, so a closed loop can still be interrupted. files: M Lib/test/test_capi/test_misc.py M Python/ceval.c M Python/opcode_metadata.h M Python/optimizer.c M Tools/cases_generator/generate_cases.py diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index fd27bd06097f6..9cbd5506bf6de 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2525,7 +2525,6 @@ def testfunc(n): i += 1 opt = _testinternalcapi.get_uop_optimizer() - with temporary_optimizer(opt): testfunc(10) @@ -2541,7 +2540,6 @@ def testfunc(n): i += 1 opt = _testinternalcapi.get_uop_optimizer() - with temporary_optimizer(opt): testfunc(10) @@ -2550,6 +2548,20 @@ def testfunc(n): uops = {opname for opname, _ in ex} self.assertIn("_POP_JUMP_IF_TRUE", uops) + def test_jump_backward(self): + def testfunc(n): + i = 0 + while i < n: + i += 1 + opt = _testinternalcapi.get_uop_optimizer() + with temporary_optimizer(opt): + testfunc(10) + + ex = get_first_executor(testfunc) + self.assertIsNotNone(ex) + uops = {opname for opname, _ in ex} + self.assertIn("JUMP_TO_TOP", uops) + if __name__ == "__main__": unittest.main() diff --git a/Python/ceval.c b/Python/ceval.c index 866acd2dd69c7..57e478c1313f6 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2783,6 +2783,13 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject break; } + case JUMP_TO_TOP: + { + pc = 0; + CHECK_EVAL_BREAKER(); + break; + } + case SAVE_IP: { frame->prev_instr = ip_offset + oparg; diff --git a/Python/opcode_metadata.h b/Python/opcode_metadata.h index 34ac85d6517a8..4a41cd86a4287 100644 --- a/Python/opcode_metadata.h +++ b/Python/opcode_metadata.h @@ -23,19 +23,20 @@ #define SAVE_IP 301 #define _POP_JUMP_IF_FALSE 302 #define _POP_JUMP_IF_TRUE 303 -#define _GUARD_BOTH_INT 304 -#define _BINARY_OP_MULTIPLY_INT 305 -#define _BINARY_OP_ADD_INT 306 -#define _BINARY_OP_SUBTRACT_INT 307 -#define _GUARD_BOTH_FLOAT 308 -#define _BINARY_OP_MULTIPLY_FLOAT 309 -#define _BINARY_OP_ADD_FLOAT 310 -#define _BINARY_OP_SUBTRACT_FLOAT 311 -#define _GUARD_BOTH_UNICODE 312 -#define _BINARY_OP_ADD_UNICODE 313 -#define _LOAD_LOCALS 314 -#define _LOAD_FROM_DICT_OR_GLOBALS 315 -#define IS_NONE 316 +#define JUMP_TO_TOP 304 +#define _GUARD_BOTH_INT 305 +#define _BINARY_OP_MULTIPLY_INT 306 +#define _BINARY_OP_ADD_INT 307 +#define _BINARY_OP_SUBTRACT_INT 308 +#define _GUARD_BOTH_FLOAT 309 +#define _BINARY_OP_MULTIPLY_FLOAT 310 +#define _BINARY_OP_ADD_FLOAT 311 +#define _BINARY_OP_SUBTRACT_FLOAT 312 +#define _GUARD_BOTH_UNICODE 313 +#define _BINARY_OP_ADD_UNICODE 314 +#define _LOAD_LOCALS 315 +#define _LOAD_FROM_DICT_OR_GLOBALS 316 +#define IS_NONE 317 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -1298,18 +1299,19 @@ const char * const _PyOpcode_uop_name[512] = { [301] = "SAVE_IP", [302] = "_POP_JUMP_IF_FALSE", [303] = "_POP_JUMP_IF_TRUE", - [304] = "_GUARD_BOTH_INT", - [305] = "_BINARY_OP_MULTIPLY_INT", - [306] = "_BINARY_OP_ADD_INT", - [307] = "_BINARY_OP_SUBTRACT_INT", - [308] = "_GUARD_BOTH_FLOAT", - [309] = "_BINARY_OP_MULTIPLY_FLOAT", - [310] = "_BINARY_OP_ADD_FLOAT", - [311] = "_BINARY_OP_SUBTRACT_FLOAT", - [312] = "_GUARD_BOTH_UNICODE", - [313] = "_BINARY_OP_ADD_UNICODE", - [314] = "_LOAD_LOCALS", - [315] = "_LOAD_FROM_DICT_OR_GLOBALS", - [316] = "IS_NONE", + [304] = "JUMP_TO_TOP", + [305] = "_GUARD_BOTH_INT", + [306] = "_BINARY_OP_MULTIPLY_INT", + [307] = "_BINARY_OP_ADD_INT", + [308] = "_BINARY_OP_SUBTRACT_INT", + [309] = "_GUARD_BOTH_FLOAT", + [310] = "_BINARY_OP_MULTIPLY_FLOAT", + [311] = "_BINARY_OP_ADD_FLOAT", + [312] = "_BINARY_OP_SUBTRACT_FLOAT", + [313] = "_GUARD_BOTH_UNICODE", + [314] = "_BINARY_OP_ADD_UNICODE", + [315] = "_LOAD_LOCALS", + [316] = "_LOAD_FROM_DICT_OR_GLOBALS", + [317] = "IS_NONE", }; #endif // NEED_OPCODE_METADATA diff --git a/Python/optimizer.c b/Python/optimizer.c index 08073193c0228..8d4162b44c2f3 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -372,9 +372,7 @@ translate_bytecode_to_trace( _PyUOpInstruction *trace, int buffer_size) { -#ifdef Py_DEBUG _Py_CODEUNIT *initial_instr = instr; -#endif int trace_length = 0; int max_length = buffer_size; @@ -456,6 +454,19 @@ translate_bytecode_to_trace( break; } + case JUMP_BACKWARD: + { + if (instr + 2 - oparg == initial_instr + && trace_length + 3 <= max_length) + { + ADD_TO_TRACE(JUMP_TO_TOP, 0); + } + else { + DPRINTF(2, "JUMP_BACKWARD not to top ends trace\n"); + } + goto done; + } + default: { const struct opcode_macro_expansion *expansion = &_PyOpcode_macro_expansion[opcode]; diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 641a327b06aa2..8c77c1f08335d 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -1346,6 +1346,7 @@ def add(name: str) -> None: add("SAVE_IP") add("_POP_JUMP_IF_FALSE") add("_POP_JUMP_IF_TRUE") + add("JUMP_TO_TOP") for instr in self.instrs.values(): if instr.kind == "op" and instr.is_viable_uop(): From webhook-mailer at python.org Tue Jul 11 14:28:32 2023 From: webhook-mailer at python.org (terryjreedy) Date: Tue, 11 Jul 2023 18:28:32 -0000 Subject: [Python-checkins] [3.11] gh-102541: Add test case for help() for non_existent_module (GH-106340) (#106640) Message-ID: https://github.com/python/cpython/commit/3e61e20043ed8613edad681a2041564040756648 commit: 3e61e20043ed8613edad681a2041564040756648 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-11T18:28:28Z summary: [3.11] gh-102541: Add test case for help() for non_existent_module (GH-106340) (#106640) gh-102541: Add test case for help() for non_existent_module (GH-106340) Test fix for when one enters, for instance, 'abd' at the 'help>' prompt. --------- (cherry picked from commit 292ac4bfe92768140c2d383fd329cfa1949869b2) Co-authored-by: Kirill Podoprigora Co-authored-by: Terry Jan Reedy files: M Lib/test/test_pydoc.py diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py index 71be6d510addd..be224feca872e 100644 --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -24,7 +24,8 @@ from urllib.request import urlopen, urlcleanup from test.support import import_helper from test.support import os_helper -from test.support.script_helper import assert_python_ok, assert_python_failure +from test.support.script_helper import (assert_python_ok, + assert_python_failure, spawn_python) from test.support import threading_helper from test.support import (reap_children, captured_output, captured_stdout, captured_stderr, is_emscripten, is_wasi, @@ -631,6 +632,14 @@ def test_builtin_on_metaclasses(self): # Testing that the subclasses section does not appear self.assertNotIn('Built-in subclasses', text) + def test_fail_help_cli(self): + elines = (missing_pattern % 'abd').splitlines() + with spawn_python("-c" "help()") as proc: + out, _ = proc.communicate(b"abd") + olines = out.decode().splitlines()[-9:-6] + olines[0] = olines[0].removeprefix('help> ') + self.assertEqual(elines, olines) + def test_fail_help_output_redirect(self): with StringIO() as buf: helper = pydoc.Helper(output=buf) From webhook-mailer at python.org Tue Jul 11 14:35:17 2023 From: webhook-mailer at python.org (terryjreedy) Date: Tue, 11 Jul 2023 18:35:17 -0000 Subject: [Python-checkins] [3.12] gh-102541: Add test case for help() for non_existent_module (GH-106340) (#106639) Message-ID: https://github.com/python/cpython/commit/46a21f5c5e99aa3017618c7bb52d3254a6c0257b commit: 46a21f5c5e99aa3017618c7bb52d3254a6c0257b branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-11T18:35:13Z summary: [3.12] gh-102541: Add test case for help() for non_existent_module (GH-106340) (#106639) gh-102541: Add test case for help() for non_existent_module (GH-106340) Test fix for when one enters, for instance, 'abd' at the 'help>' prompt. --------- (cherry picked from commit 292ac4bfe92768140c2d383fd329cfa1949869b2) Co-authored-by: Kirill Podoprigora Co-authored-by: Terry Jan Reedy files: M Lib/test/test_pydoc.py diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py index 115ffd3c29d4d..ddb5187f90da9 100644 --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -24,7 +24,8 @@ from urllib.request import urlopen, urlcleanup from test.support import import_helper from test.support import os_helper -from test.support.script_helper import assert_python_ok, assert_python_failure +from test.support.script_helper import (assert_python_ok, + assert_python_failure, spawn_python) from test.support import threading_helper from test.support import (reap_children, captured_output, captured_stdout, captured_stderr, is_emscripten, is_wasi, @@ -631,6 +632,14 @@ def test_builtin_on_metaclasses(self): # Testing that the subclasses section does not appear self.assertNotIn('Built-in subclasses', text) + def test_fail_help_cli(self): + elines = (missing_pattern % 'abd').splitlines() + with spawn_python("-c" "help()") as proc: + out, _ = proc.communicate(b"abd") + olines = out.decode().splitlines()[-9:-6] + olines[0] = olines[0].removeprefix('help> ') + self.assertEqual(elines, olines) + def test_fail_help_output_redirect(self): with StringIO() as buf: helper = pydoc.Helper(output=buf) From webhook-mailer at python.org Tue Jul 11 15:13:31 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 11 Jul 2023 19:13:31 -0000 Subject: [Python-checkins] gh-106521: Add PyObject_GetOptionalAttr() function (GH-106522) Message-ID: https://github.com/python/cpython/commit/579aa89e68a6607398317a50586af781981e89fb commit: 579aa89e68a6607398317a50586af781981e89fb branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-11T22:13:27+03:00 summary: gh-106521: Add PyObject_GetOptionalAttr() function (GH-106522) It is a new name of former _PyObject_LookupAttr(). Add also PyObject_GetOptionalAttrString(). files: A Misc/NEWS.d/next/C API/2023-07-07-19-14-00.gh-issue-106521.Veh9f3.rst M Doc/c-api/object.rst M Doc/data/stable_abi.dat M Doc/whatsnew/3.13.rst M Include/abstract.h M Include/cpython/object.h M Include/object.h M Lib/test/test_stable_abi_ctypes.py M Misc/stable_abi.toml M Objects/object.c M PC/python3dll.c diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 22e7458013fb3..6fc5b2d14dd32 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -37,7 +37,8 @@ Object Protocol Exceptions that occur when this calls :meth:`~object.__getattr__` and :meth:`~object.__getattribute__` methods are silently ignored. - For proper error handling, use :c:func:`PyObject_GetAttr` instead. + For proper error handling, use :c:func:`PyObject_GetOptionalAttr` or + :c:func:`PyObject_GetAttr` instead. .. c:function:: int PyObject_HasAttrString(PyObject *o, const char *attr_name) @@ -51,7 +52,8 @@ Object Protocol Exceptions that occur when this calls :meth:`~object.__getattr__` and :meth:`~object.__getattribute__` methods or while creating the temporary :class:`str` object are silently ignored. - For proper error handling, use :c:func:`PyObject_GetAttrString` instead. + For proper error handling, use :c:func:`PyObject_GetOptionalAttrString` + or :c:func:`PyObject_GetAttrString` instead. .. c:function:: PyObject* PyObject_GetAttr(PyObject *o, PyObject *attr_name) @@ -60,6 +62,9 @@ Object Protocol value on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``o.attr_name``. + If the missing attribute should not be treated as a failure, you can use + :c:func:`PyObject_GetOptionalAttr` instead. + .. c:function:: PyObject* PyObject_GetAttrString(PyObject *o, const char *attr_name) @@ -67,6 +72,38 @@ Object Protocol value on success, or ``NULL`` on failure. This is the equivalent of the Python expression ``o.attr_name``. + If the missing attribute should not be treated as a failure, you can use + :c:func:`PyObject_GetOptionalAttrString` instead. + + +.. c:function:: int PyObject_GetOptionalAttr(PyObject *obj, PyObject *attr_name, PyObject **result); + + Variant of :c:func:`PyObject_GetAttr` which doesn't raise + :exc:`AttributeError` if the attribute is not found. + + If the attribute is found, return ``1`` and set *\*result* to a new + :term:`strong reference` to the attribute. + If the attribute is not found, return ``0`` and set *\*result* to ``NULL``; + the :exc:`AttributeError` is silenced. + If an error other than :exc:`AttributeError` is raised, return ``-1`` and + set *\*result* to ``NULL``. + + .. versionadded:: 3.13 + + +.. c:function:: int PyObject_GetOptionalAttrString(PyObject *obj, const char *attr_name, PyObject **result); + + Variant of :c:func:`PyObject_GetAttrString` which doesn't raise + :exc:`AttributeError` if the attribute is not found. + + If the attribute is found, return ``1`` and set *\*result* to a new + :term:`strong reference` to the attribute. + If the attribute is not found, return ``0`` and set *\*result* to ``NULL``; + the :exc:`AttributeError` is silenced. + If an error other than :exc:`AttributeError` is raised, return ``-1`` and + set *\*result* to ``NULL``. + + .. versionadded:: 3.13 .. c:function:: PyObject* PyObject_GenericGetAttr(PyObject *o, PyObject *name) diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index 9a61ddc39a353..d7e8b5b464a71 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -512,6 +512,8 @@ function,PyObject_GetAttrString,3.2,, function,PyObject_GetBuffer,3.11,, function,PyObject_GetItem,3.2,, function,PyObject_GetIter,3.2,, +function,PyObject_GetOptionalAttr,3.13,, +function,PyObject_GetOptionalAttrString,3.13,, function,PyObject_GetTypeData,3.12,, function,PyObject_HasAttr,3.2,, function,PyObject_HasAttrString,3.2,, diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 8fc809deca139..62981673140cf 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -734,6 +734,14 @@ New Features ``NULL`` if the referent is no longer live. (Contributed by Victor Stinner in :gh:`105927`.) +* Add :c:func:`PyObject_GetOptionalAttr` and + :c:func:`PyObject_GetOptionalAttrString`, variants of + :c:func:`PyObject_GetAttr` and :c:func:`PyObject_GetAttrString` which + don't raise :exc:`AttributeError` if the attribute is not found. + These variants are more convenient and faster if the missing attribute + should not be treated as a failure. + (Contributed by Serhiy Storchaka in :gh:`106521`.) + * If Python is built in :ref:`debug mode ` or :option:`with assertions <--with-assertions>`, :c:func:`PyTuple_SET_ITEM` and :c:func:`PyList_SET_ITEM` now check the index argument with an assertion. diff --git a/Include/abstract.h b/Include/abstract.h index c84d2c704e960..f47ea1ae3210e 100644 --- a/Include/abstract.h +++ b/Include/abstract.h @@ -60,6 +60,38 @@ extern "C" { This is the equivalent of the Python expression: o.attr_name. */ +/* Implemented elsewhere: + + int PyObject_GetOptionalAttr(PyObject *obj, PyObject *attr_name, PyObject **result); + + Variant of PyObject_GetAttr() which doesn't raise AttributeError + if the attribute is not found. + + If the attribute is found, return 1 and set *result to a new strong + reference to the attribute. + If the attribute is not found, return 0 and set *result to NULL; + the AttributeError is silenced. + If an error other than AttributeError is raised, return -1 and + set *result to NULL. +*/ + + +/* Implemented elsewhere: + + int PyObject_GetOptionalAttrString(PyObject *obj, const char *attr_name, PyObject **result); + + Variant of PyObject_GetAttrString() which doesn't raise AttributeError + if the attribute is not found. + + If the attribute is found, return 1 and set *result to a new strong + reference to the attribute. + If the attribute is not found, return 0 and set *result to NULL; + the AttributeError is silenced. + If an error other than AttributeError is raised, return -1 and + set *result to NULL. +*/ + + /* Implemented elsewhere: int PyObject_SetAttrString(PyObject *o, const char *attr_name, PyObject *v); diff --git a/Include/cpython/object.h b/Include/cpython/object.h index ef9006431bee2..ba30c567c40eb 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -294,17 +294,7 @@ PyAPI_FUNC(int) _PyObject_IsFreed(PyObject *); PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *); PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, _Py_Identifier *); PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, _Py_Identifier *, PyObject *); -/* Replacements of PyObject_GetAttr() and _PyObject_GetAttrId() which - don't raise AttributeError. - - Return 1 and set *result != NULL if an attribute is found. - Return 0 and set *result == NULL if an attribute is not found; - an AttributeError is silenced. - Return -1 and set *result == NULL if an error other than AttributeError - is raised. -*/ -PyAPI_FUNC(int) _PyObject_LookupAttr(PyObject *, PyObject *, PyObject **); -PyAPI_FUNC(int) _PyObject_LookupAttrId(PyObject *, _Py_Identifier *, PyObject **); +#define _PyObject_LookupAttr PyObject_GetOptionalAttr PyAPI_FUNC(int) _PyObject_GetMethod(PyObject *obj, PyObject *name, PyObject **method); diff --git a/Include/object.h b/Include/object.h index dccab07e5f2c6..7f2e4e90615e7 100644 --- a/Include/object.h +++ b/Include/object.h @@ -394,6 +394,10 @@ PyAPI_FUNC(int) PyObject_SetAttrString(PyObject *, const char *, PyObject *); PyAPI_FUNC(int) PyObject_DelAttrString(PyObject *v, const char *name); PyAPI_FUNC(int) PyObject_HasAttrString(PyObject *, const char *); PyAPI_FUNC(PyObject *) PyObject_GetAttr(PyObject *, PyObject *); +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030d0000 +PyAPI_FUNC(int) PyObject_GetOptionalAttr(PyObject *, PyObject *, PyObject **); +PyAPI_FUNC(int) PyObject_GetOptionalAttrString(PyObject *, const char *, PyObject **); +#endif PyAPI_FUNC(int) PyObject_SetAttr(PyObject *, PyObject *, PyObject *); PyAPI_FUNC(int) PyObject_DelAttr(PyObject *v, PyObject *name); PyAPI_FUNC(int) PyObject_HasAttr(PyObject *, PyObject *); diff --git a/Lib/test/test_stable_abi_ctypes.py b/Lib/test/test_stable_abi_ctypes.py index 20bc2624c8136..6aa3cf382f9c6 100644 --- a/Lib/test/test_stable_abi_ctypes.py +++ b/Lib/test/test_stable_abi_ctypes.py @@ -531,6 +531,8 @@ def test_windows_feature_macros(self): "PyObject_GetBuffer", "PyObject_GetItem", "PyObject_GetIter", + "PyObject_GetOptionalAttr", + "PyObject_GetOptionalAttrString", "PyObject_GetTypeData", "PyObject_HasAttr", "PyObject_HasAttrString", diff --git a/Misc/NEWS.d/next/C API/2023-07-07-19-14-00.gh-issue-106521.Veh9f3.rst b/Misc/NEWS.d/next/C API/2023-07-07-19-14-00.gh-issue-106521.Veh9f3.rst new file mode 100644 index 0000000000000..f38fd271e8a44 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-07-19-14-00.gh-issue-106521.Veh9f3.rst @@ -0,0 +1 @@ +Add :c:func:`PyObject_GetOptionalAttr` and :c:func:`PyObject_GetOptionalAttrString` functions. diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml index c61fedf8390e2..1f6519cb1e1b2 100644 --- a/Misc/stable_abi.toml +++ b/Misc/stable_abi.toml @@ -2436,3 +2436,7 @@ added = '3.13' [function.PyObject_DelAttrString] added = '3.13' +[function.PyObject_GetOptionalAttr] + added = '3.13' +[function.PyObject_GetOptionalAttrString] + added = '3.13' diff --git a/Objects/object.c b/Objects/object.c index 540ba5d07427c..d30e048335ab6 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -692,7 +692,7 @@ _PyObject_FunctionStr(PyObject *x) { assert(!PyErr_Occurred()); PyObject *qualname; - int ret = _PyObject_LookupAttr(x, &_Py_ID(__qualname__), &qualname); + int ret = PyObject_GetOptionalAttr(x, &_Py_ID(__qualname__), &qualname); if (qualname == NULL) { if (ret < 0) { return NULL; @@ -701,7 +701,7 @@ _PyObject_FunctionStr(PyObject *x) } PyObject *module; PyObject *result = NULL; - ret = _PyObject_LookupAttr(x, &_Py_ID(__module__), &module); + ret = PyObject_GetOptionalAttr(x, &_Py_ID(__module__), &module); if (module != NULL && module != Py_None) { ret = PyObject_RichCompareBool(module, &_Py_ID(builtins), Py_NE); if (ret < 0) { @@ -957,7 +957,7 @@ _PyObject_IsAbstract(PyObject *obj) if (obj == NULL) return 0; - res = _PyObject_LookupAttr(obj, &_Py_ID(__isabstractmethod__), &isabstract); + res = PyObject_GetOptionalAttr(obj, &_Py_ID(__isabstractmethod__), &isabstract); if (res > 0) { res = PyObject_IsTrue(isabstract); Py_DECREF(isabstract); @@ -1049,7 +1049,7 @@ PyObject_GetAttr(PyObject *v, PyObject *name) } int -_PyObject_LookupAttr(PyObject *v, PyObject *name, PyObject **result) +PyObject_GetOptionalAttr(PyObject *v, PyObject *name, PyObject **result) { PyTypeObject *tp = Py_TYPE(v); @@ -1117,21 +1117,35 @@ _PyObject_LookupAttr(PyObject *v, PyObject *name, PyObject **result) } int -_PyObject_LookupAttrId(PyObject *v, _Py_Identifier *name, PyObject **result) +PyObject_GetOptionalAttrString(PyObject *obj, const char *name, PyObject **result) { - PyObject *oname = _PyUnicode_FromId(name); /* borrowed */ - if (!oname) { - *result = NULL; + if (Py_TYPE(obj)->tp_getattr == NULL) { + PyObject *oname = PyUnicode_FromString(name); + if (oname == NULL) { + *result = NULL; + return -1; + } + int rc = PyObject_GetOptionalAttr(obj, oname, result); + Py_DECREF(oname); + return rc; + } + + *result = (*Py_TYPE(obj)->tp_getattr)(obj, (char*)name); + if (*result != NULL) { + return 1; + } + if (!PyErr_ExceptionMatches(PyExc_AttributeError)) { return -1; } - return _PyObject_LookupAttr(v, oname, result); + PyErr_Clear(); + return 0; } int PyObject_HasAttr(PyObject *v, PyObject *name) { PyObject *res; - if (_PyObject_LookupAttr(v, name, &res) < 0) { + if (PyObject_GetOptionalAttr(v, name, &res) < 0) { PyErr_Clear(); return 0; } diff --git a/PC/python3dll.c b/PC/python3dll.c index a7173911c7c1e..7d7192958eeb9 100755 --- a/PC/python3dll.c +++ b/PC/python3dll.c @@ -469,6 +469,8 @@ EXPORT_FUNC(PyObject_GetAttrString) EXPORT_FUNC(PyObject_GetBuffer) EXPORT_FUNC(PyObject_GetItem) EXPORT_FUNC(PyObject_GetIter) +EXPORT_FUNC(PyObject_GetOptionalAttr) +EXPORT_FUNC(PyObject_GetOptionalAttrString) EXPORT_FUNC(PyObject_GetTypeData) EXPORT_FUNC(PyObject_HasAttr) EXPORT_FUNC(PyObject_HasAttrString) From webhook-mailer at python.org Tue Jul 11 15:35:45 2023 From: webhook-mailer at python.org (pablogsal) Date: Tue, 11 Jul 2023 19:35:45 -0000 Subject: [Python-checkins] gh-106597: Add debugging struct with offsets for out-of-process tools (#106598) Message-ID: https://github.com/python/cpython/commit/b444bfb0a325dea8c29f7b1828233b00fbf4a1cb commit: b444bfb0a325dea8c29f7b1828233b00fbf4a1cb branch: main author: Pablo Galindo Salgado committer: pablogsal date: 2023-07-11T20:35:41+01:00 summary: gh-106597: Add debugging struct with offsets for out-of-process tools (#106598) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-10-15-30-45.gh-issue-106597.WAZ14y.rst M Include/internal/pycore_runtime.h M Include/internal/pycore_runtime_init.h diff --git a/Include/internal/pycore_runtime.h b/Include/internal/pycore_runtime.h index 5ed97e9715b2b..a16d4202b616d 100644 --- a/Include/internal/pycore_runtime.h +++ b/Include/internal/pycore_runtime.h @@ -53,12 +53,101 @@ typedef struct _Py_AuditHookEntry { void *userData; } _Py_AuditHookEntry; +typedef struct _Py_DebugOffsets { + char cookie[8]; + uint64_t version; + // Runtime state offset; + struct _runtime_state { + off_t finalizing; + off_t interpreters_head; + } runtime_state; + + // Interpreter state offset; + struct _interpreter_state { + off_t next; + off_t threads_head; + off_t gc; + off_t imports_modules; + off_t sysdict; + off_t builtins; + off_t ceval_gil; + off_t gil_runtime_state_locked; + off_t gil_runtime_state_holder; + } interpreter_state; + + // Thread state offset; + struct _thread_state{ + off_t prev; + off_t next; + off_t interp; + off_t cframe; + off_t thread_id; + off_t native_thread_id; + } thread_state; + + // InterpreterFrame offset; + struct _interpreter_frame { + off_t previous; + off_t executable; + off_t prev_instr; + off_t localsplus; + off_t owner; + } interpreter_frame; + + // CFrame offset; + struct _cframe { + off_t current_frame; + off_t previous; + } cframe; + + // Code object offset; + struct _code_object { + off_t filename; + off_t name; + off_t linetable; + off_t firstlineno; + off_t argcount; + off_t localsplusnames; + off_t localspluskinds; + off_t co_code_adaptive; + } code_object; + + // PyObject offset; + struct _pyobject { + off_t ob_type; + } pyobject; + + // PyTypeObject object offset; + struct _type_object { + off_t tp_name; + } type_object; + + // PyTuple object offset; + struct _tuple_object { + off_t ob_item; + } tuple_object; +} _Py_DebugOffsets; + /* Full Python runtime state */ /* _PyRuntimeState holds the global state for the CPython runtime. That data is exposed in the internal API as a static variable (_PyRuntime). */ typedef struct pyruntimestate { + /* This field must be first to facilitate locating it by out of process + * debuggers. Out of process debuggers will use the offsets contained in this + * field to be able to locate other fields in several interpreter structures + * in a way that doesn't require them to know the exact layout of those + * structures. + * + * IMPORTANT: + * This struct is **NOT** backwards compatible between minor version of the + * interpreter and the members, order of members and size can change between + * minor versions. This struct is only guaranteed to be stable between patch + * versions for a given minor version of the interpreter. + */ + _Py_DebugOffsets debug_offsets; + /* Has been initialized to a safe state. In order to be effective, this must be set to 0 during or right diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h index b507de0437d9a..e72e7422c7207 100644 --- a/Include/internal/pycore_runtime_init.h +++ b/Include/internal/pycore_runtime_init.h @@ -21,9 +21,65 @@ extern PyTypeObject _PyExc_MemoryError; /* The static initializers defined here should only be used in the runtime init code (in pystate.c and pylifecycle.c). */ - #define _PyRuntimeState_INIT(runtime) \ { \ + .debug_offsets = { \ + .cookie = "xdebugpy", \ + .version = PY_VERSION_HEX, \ + .runtime_state = { \ + .finalizing = offsetof(_PyRuntimeState, _finalizing), \ + .interpreters_head = offsetof(_PyRuntimeState, interpreters.head), \ + }, \ + .interpreter_state = { \ + .next = offsetof(PyInterpreterState, next), \ + .threads_head = offsetof(PyInterpreterState, threads.head), \ + .gc = offsetof(PyInterpreterState, gc), \ + .imports_modules = offsetof(PyInterpreterState, imports.modules), \ + .sysdict = offsetof(PyInterpreterState, sysdict), \ + .builtins = offsetof(PyInterpreterState, builtins), \ + .ceval_gil = offsetof(PyInterpreterState, ceval.gil), \ + .gil_runtime_state_locked = offsetof(PyInterpreterState, _gil.locked), \ + .gil_runtime_state_holder = offsetof(PyInterpreterState, _gil.last_holder), \ + }, \ + .thread_state = { \ + .prev = offsetof(PyThreadState, prev), \ + .next = offsetof(PyThreadState, next), \ + .interp = offsetof(PyThreadState, interp), \ + .cframe = offsetof(PyThreadState, cframe), \ + .thread_id = offsetof(PyThreadState, thread_id), \ + .native_thread_id = offsetof(PyThreadState, native_thread_id), \ + }, \ + .interpreter_frame = { \ + .previous = offsetof(_PyInterpreterFrame, previous), \ + .executable = offsetof(_PyInterpreterFrame, f_executable), \ + .prev_instr = offsetof(_PyInterpreterFrame, prev_instr), \ + .localsplus = offsetof(_PyInterpreterFrame, localsplus), \ + .owner = offsetof(_PyInterpreterFrame, owner), \ + }, \ + .cframe = { \ + .current_frame = offsetof(_PyCFrame, current_frame), \ + .previous = offsetof(_PyCFrame, previous), \ + }, \ + .code_object = { \ + .filename = offsetof(PyCodeObject, co_filename), \ + .name = offsetof(PyCodeObject, co_name), \ + .linetable = offsetof(PyCodeObject, co_linetable), \ + .firstlineno = offsetof(PyCodeObject, co_firstlineno), \ + .argcount = offsetof(PyCodeObject, co_argcount), \ + .localsplusnames = offsetof(PyCodeObject, co_localsplusnames), \ + .localspluskinds = offsetof(PyCodeObject, co_localspluskinds), \ + .co_code_adaptive = offsetof(PyCodeObject, co_code_adaptive), \ + }, \ + .pyobject = { \ + .ob_type = offsetof(PyObject, ob_type), \ + }, \ + .type_object = { \ + .tp_name = offsetof(PyTypeObject, tp_name), \ + }, \ + .tuple_object = { \ + .ob_item = offsetof(PyTupleObject, ob_item), \ + }, \ + }, \ .allocators = { \ .standard = _pymem_allocators_standard_INIT(runtime), \ .debug = _pymem_allocators_debug_INIT, \ diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-10-15-30-45.gh-issue-106597.WAZ14y.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-10-15-30-45.gh-issue-106597.WAZ14y.rst new file mode 100644 index 0000000000000..bbe455d652f50 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-10-15-30-45.gh-issue-106597.WAZ14y.rst @@ -0,0 +1,5 @@ +A new debug structure of offsets has been added to the ``_PyRuntimeState`` +that will help out-of-process debuggers and profilers to obtain the offsets +to relevant interpreter structures in a way that is agnostic of how Python +was compiled and that doesn't require copying the headers. Patch by Pablo +Galindo From webhook-mailer at python.org Tue Jul 11 16:04:17 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 11 Jul 2023 20:04:17 -0000 Subject: [Python-checkins] gh-106307: C API: Add PyMapping_GetOptionalItem() function (GH-106308) Message-ID: https://github.com/python/cpython/commit/4bf43710d1e1f19cc46b116b5d8524f6c75dabfa commit: 4bf43710d1e1f19cc46b116b5d8524f6c75dabfa branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-11T23:04:12+03:00 summary: gh-106307: C API: Add PyMapping_GetOptionalItem() function (GH-106308) Also add PyMapping_GetOptionalItemString() function. files: A Misc/NEWS.d/next/C API/2023-07-08-12-24-17.gh-issue-106307.FVnkBw.rst M Doc/c-api/mapping.rst M Doc/data/stable_abi.dat M Doc/whatsnew/3.13.rst M Include/abstract.h M Lib/test/test_stable_abi_ctypes.py M Misc/stable_abi.toml M Modules/_lzmamodule.c M Modules/_pickle.c M Objects/abstract.c M PC/python3dll.c M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/import.c diff --git a/Doc/c-api/mapping.rst b/Doc/c-api/mapping.rst index cffb0ed50fb77..9176a4652cbf2 100644 --- a/Doc/c-api/mapping.rst +++ b/Doc/c-api/mapping.rst @@ -33,6 +33,36 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and See also :c:func:`PyObject_GetItem`. +.. c:function:: int PyMapping_GetOptionalItem(PyObject *obj, PyObject *key, PyObject **result) + + Variant of :c:func:`PyObject_GetItem` which doesn't raise + :exc:`KeyError` if the key is not found. + + If the key is found, return ``1`` and set *\*result* to a new + :term:`strong reference` to the corresponding value. + If the key is not found, return ``0`` and set *\*result* to ``NULL``; + the :exc:`KeyError` is silenced. + If an error other than :exc:`KeyError` is raised, return ``-1`` and + set *\*result* to ``NULL``. + + .. versionadded:: 3.13 + + +.. c:function:: int PyMapping_GetOptionalItemString(PyObject *obj, const char *key, PyObject **result) + + Variant of :c:func:`PyMapping_GetItemString` which doesn't raise + :exc:`KeyError` if the key is not found. + + If the key is found, return ``1`` and set *\*result* to a new + :term:`strong reference` to the corresponding value. + If the key is not found, return ``0`` and set *\*result* to ``NULL``; + the :exc:`KeyError` is silenced. + If an error other than :exc:`KeyError` is raised, return ``-1`` and + set *\*result* to ``NULL``. + + .. versionadded:: 3.13 + + .. c:function:: int PyMapping_SetItemString(PyObject *o, const char *key, PyObject *v) Map the string *key* to the value *v* in object *o*. Returns ``-1`` on diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index d7e8b5b464a71..e3dd3dab27a03 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -370,6 +370,8 @@ var,PyLong_Type,3.2,, var,PyMap_Type,3.2,, function,PyMapping_Check,3.2,, function,PyMapping_GetItemString,3.2,, +function,PyMapping_GetOptionalItem,3.13,, +function,PyMapping_GetOptionalItemString,3.13,, function,PyMapping_HasKey,3.2,, function,PyMapping_HasKeyString,3.2,, function,PyMapping_Items,3.2,, diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 62981673140cf..dc020682ac1ed 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -742,6 +742,14 @@ New Features should not be treated as a failure. (Contributed by Serhiy Storchaka in :gh:`106521`.) +* Add :c:func:`PyMapping_GetOptionalItem` and + :c:func:`PyMapping_GetOptionalItemString`: variants of + :c:func:`PyObject_GetItem` and :c:func:`PyMapping_GetItemString` which don't + raise :exc:`KeyError` if the key is not found. + These variants are more convenient and faster if the missing key should not + be treated as a failure. + (Contributed by Serhiy Storchaka in :gh:`106307`.) + * If Python is built in :ref:`debug mode ` or :option:`with assertions <--with-assertions>`, :c:func:`PyTuple_SET_ITEM` and :c:func:`PyList_SET_ITEM` now check the index argument with an assertion. diff --git a/Include/abstract.h b/Include/abstract.h index f47ea1ae3210e..dd915004e7834 100644 --- a/Include/abstract.h +++ b/Include/abstract.h @@ -840,6 +840,21 @@ PyAPI_FUNC(PyObject *) PyMapping_Items(PyObject *o); PyAPI_FUNC(PyObject *) PyMapping_GetItemString(PyObject *o, const char *key); +/* Variants of PyObject_GetItem() and PyMapping_GetItemString() which don't + raise KeyError if the key is not found. + + If the key is found, return 1 and set *result to a new strong + reference to the corresponding value. + If the key is not found, return 0 and set *result to NULL; + the KeyError is silenced. + If an error other than KeyError is raised, return -1 and + set *result to NULL. +*/ +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030d0000 +PyAPI_FUNC(int) PyMapping_GetOptionalItem(PyObject *, PyObject *, PyObject **); +PyAPI_FUNC(int) PyMapping_GetOptionalItemString(PyObject *, const char *, PyObject **); +#endif + /* Map the string 'key' to the value 'v' in the mapping 'o'. Returns -1 on failure. diff --git a/Lib/test/test_stable_abi_ctypes.py b/Lib/test/test_stable_abi_ctypes.py index 6aa3cf382f9c6..e92e986b29337 100644 --- a/Lib/test/test_stable_abi_ctypes.py +++ b/Lib/test/test_stable_abi_ctypes.py @@ -398,6 +398,8 @@ def test_windows_feature_macros(self): "PyMap_Type", "PyMapping_Check", "PyMapping_GetItemString", + "PyMapping_GetOptionalItem", + "PyMapping_GetOptionalItemString", "PyMapping_HasKey", "PyMapping_HasKeyString", "PyMapping_Items", diff --git a/Misc/NEWS.d/next/C API/2023-07-08-12-24-17.gh-issue-106307.FVnkBw.rst b/Misc/NEWS.d/next/C API/2023-07-08-12-24-17.gh-issue-106307.FVnkBw.rst new file mode 100644 index 0000000000000..dc1ab8d3e3fb8 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-08-12-24-17.gh-issue-106307.FVnkBw.rst @@ -0,0 +1 @@ +Add :c:func:`PyMapping_GetOptionalItem` function. diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml index 1f6519cb1e1b2..8ea8fde68b833 100644 --- a/Misc/stable_abi.toml +++ b/Misc/stable_abi.toml @@ -2440,3 +2440,7 @@ added = '3.13' [function.PyObject_GetOptionalAttrString] added = '3.13' +[function.PyMapping_GetOptionalItem] + added = '3.13' +[function.PyMapping_GetOptionalItemString] + added = '3.13' diff --git a/Modules/_lzmamodule.c b/Modules/_lzmamodule.c index ba0987ee0abca..02a32ddccb3ed 100644 --- a/Modules/_lzmamodule.c +++ b/Modules/_lzmamodule.c @@ -240,15 +240,10 @@ parse_filter_spec_lzma(_lzma_state *state, PyObject *spec) /* First, fill in default values for all the options using a preset. Then, override the defaults with any values given by the caller. */ - preset_obj = PyMapping_GetItemString(spec, "preset"); - if (preset_obj == NULL) { - if (PyErr_ExceptionMatches(PyExc_KeyError)) { - PyErr_Clear(); - } - else { - return NULL; - } - } else { + if (PyMapping_GetOptionalItemString(spec, "preset", &preset_obj) < 0) { + return NULL; + } + if (preset_obj != NULL) { int ok = uint32_converter(preset_obj, &preset); Py_DECREF(preset_obj); if (!ok) { @@ -345,11 +340,12 @@ lzma_filter_converter(_lzma_state *state, PyObject *spec, void *ptr) "Filter specifier must be a dict or dict-like object"); return 0; } - id_obj = PyMapping_GetItemString(spec, "id"); + if (PyMapping_GetOptionalItemString(spec, "id", &id_obj) < 0) { + return 0; + } if (id_obj == NULL) { - if (PyErr_ExceptionMatches(PyExc_KeyError)) - PyErr_SetString(PyExc_ValueError, - "Filter specifier must have an \"id\" entry"); + PyErr_SetString(PyExc_ValueError, + "Filter specifier must have an \"id\" entry"); return 0; } f->id = PyLong_AsUnsignedLongLong(id_obj); diff --git a/Modules/_pickle.c b/Modules/_pickle.c index a68a0aaa64c2b..5ecf7ca7cb9bc 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -4438,16 +4438,13 @@ save(PickleState *st, PicklerObject *self, PyObject *obj, int pers_save) PyObject_GetItem and _PyObject_GetAttrId used below. */ Py_INCREF(reduce_func); } - } else { - reduce_func = PyObject_GetItem(self->dispatch_table, - (PyObject *)type); - if (reduce_func == NULL) { - if (PyErr_ExceptionMatches(PyExc_KeyError)) - PyErr_Clear(); - else - goto error; - } } + else if (PyMapping_GetOptionalItem(self->dispatch_table, (PyObject *)type, + &reduce_func) < 0) + { + goto error; + } + if (reduce_func != NULL) { reduce_value = _Pickle_FastCall(reduce_func, Py_NewRef(obj)); } diff --git a/Objects/abstract.c b/Objects/abstract.c index 80a40944d6d21..e595bc91c220f 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -199,6 +199,30 @@ PyObject_GetItem(PyObject *o, PyObject *key) return type_error("'%.200s' object is not subscriptable", o); } +int +PyMapping_GetOptionalItem(PyObject *obj, PyObject *key, PyObject **result) +{ + if (PyDict_CheckExact(obj)) { + *result = PyDict_GetItemWithError(obj, key); /* borrowed */ + if (*result) { + Py_INCREF(*result); + return 1; + } + return PyErr_Occurred() ? -1 : 0; + } + + *result = PyObject_GetItem(obj, key); + if (*result) { + return 1; + } + assert(PyErr_Occurred()); + if (!PyErr_ExceptionMatches(PyExc_KeyError)) { + return -1; + } + PyErr_Clear(); + return 0; +} + int PyObject_SetItem(PyObject *o, PyObject *key, PyObject *value) { @@ -2366,6 +2390,22 @@ PyMapping_GetItemString(PyObject *o, const char *key) return r; } +int +PyMapping_GetOptionalItemString(PyObject *obj, const char *key, PyObject **result) +{ + if (key == NULL) { + null_error(); + return -1; + } + PyObject *okey = PyUnicode_FromString(key); + if (okey == NULL) { + return -1; + } + int rc = PyMapping_GetOptionalItem(obj, okey, result); + Py_DECREF(okey); + return rc; +} + int PyMapping_SetItemString(PyObject *o, const char *key, PyObject *value) { diff --git a/PC/python3dll.c b/PC/python3dll.c index 7d7192958eeb9..8f2df6950cfc0 100755 --- a/PC/python3dll.c +++ b/PC/python3dll.c @@ -352,6 +352,8 @@ EXPORT_FUNC(PyLong_FromVoidPtr) EXPORT_FUNC(PyLong_GetInfo) EXPORT_FUNC(PyMapping_Check) EXPORT_FUNC(PyMapping_GetItemString) +EXPORT_FUNC(PyMapping_GetOptionalItem) +EXPORT_FUNC(PyMapping_GetOptionalItemString) EXPORT_FUNC(PyMapping_HasKey) EXPORT_FUNC(PyMapping_HasKeyString) EXPORT_FUNC(PyMapping_Items) diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 3ba0d0f7e292f..2f6b8c5ae2f9c 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1086,26 +1086,11 @@ dummy_func( } inst(LOAD_BUILD_CLASS, ( -- bc)) { - if (PyDict_CheckExact(BUILTINS())) { - bc = _PyDict_GetItemWithError(BUILTINS(), - &_Py_ID(__build_class__)); - if (bc == NULL) { - if (!_PyErr_Occurred(tstate)) { - _PyErr_SetString(tstate, PyExc_NameError, - "__build_class__ not found"); - } - ERROR_IF(true, error); - } - Py_INCREF(bc); - } - else { - bc = PyObject_GetItem(BUILTINS(), &_Py_ID(__build_class__)); - if (bc == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) - _PyErr_SetString(tstate, PyExc_NameError, - "__build_class__ not found"); - ERROR_IF(true, error); - } + ERROR_IF(PyMapping_GetOptionalItem(BUILTINS(), &_Py_ID(__build_class__), &bc) < 0, error); + if (bc == NULL) { + _PyErr_SetString(tstate, PyExc_NameError, + "__build_class__ not found"); + ERROR_IF(true, error); } } @@ -1280,25 +1265,9 @@ dummy_func( op(_LOAD_FROM_DICT_OR_GLOBALS, (mod_or_class_dict -- v)) { PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); - if (PyDict_CheckExact(mod_or_class_dict)) { - v = PyDict_GetItemWithError(mod_or_class_dict, name); - if (v != NULL) { - Py_INCREF(v); - } - else if (_PyErr_Occurred(tstate)) { - Py_DECREF(mod_or_class_dict); - goto error; - } - } - else { - v = PyObject_GetItem(mod_or_class_dict, name); - if (v == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - Py_DECREF(mod_or_class_dict); - goto error; - } - _PyErr_Clear(tstate); - } + if (PyMapping_GetOptionalItem(mod_or_class_dict, name, &v) < 0) { + Py_DECREF(mod_or_class_dict); + goto error; } Py_DECREF(mod_or_class_dict); if (v == NULL) { @@ -1310,28 +1279,14 @@ dummy_func( goto error; } else { - if (PyDict_CheckExact(BUILTINS())) { - v = PyDict_GetItemWithError(BUILTINS(), name); - if (v == NULL) { - if (!_PyErr_Occurred(tstate)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } - goto error; - } - Py_INCREF(v); + if (PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0) { + goto error; } - else { - v = PyObject_GetItem(BUILTINS(), name); - if (v == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } - goto error; - } + if (v == NULL) { + format_exc_check_arg( + tstate, PyExc_NameError, + NAME_ERROR_MSG, name); + goto error; } } } @@ -1381,19 +1336,14 @@ dummy_func( /* Slow-path if globals or builtins is not a dict */ /* namespace 1: globals */ - v = PyObject_GetItem(GLOBALS(), name); + ERROR_IF(PyMapping_GetOptionalItem(GLOBALS(), name, &v) < 0, error); if (v == NULL) { - ERROR_IF(!_PyErr_ExceptionMatches(tstate, PyExc_KeyError), error); - _PyErr_Clear(tstate); - /* namespace 2: builtins */ - v = PyObject_GetItem(BUILTINS(), name); + ERROR_IF(PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0, error); if (v == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } + format_exc_check_arg( + tstate, PyExc_NameError, + NAME_ERROR_MSG, name); ERROR_IF(true, error); } } @@ -1466,25 +1416,9 @@ dummy_func( assert(class_dict); assert(oparg >= 0 && oparg < _PyFrame_GetCode(frame)->co_nlocalsplus); name = PyTuple_GET_ITEM(_PyFrame_GetCode(frame)->co_localsplusnames, oparg); - if (PyDict_CheckExact(class_dict)) { - value = PyDict_GetItemWithError(class_dict, name); - if (value != NULL) { - Py_INCREF(value); - } - else if (_PyErr_Occurred(tstate)) { - Py_DECREF(class_dict); - goto error; - } - } - else { - value = PyObject_GetItem(class_dict, name); - if (value == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - Py_DECREF(class_dict); - goto error; - } - _PyErr_Clear(tstate); - } + if (PyMapping_GetOptionalItem(class_dict, name, &value) < 0) { + Py_DECREF(class_dict); + goto error; } Py_DECREF(class_dict); if (!value) { @@ -1622,10 +1556,8 @@ dummy_func( } else { /* do the same if locals() is not a dict */ - ann_dict = PyObject_GetItem(LOCALS(), &_Py_ID(__annotations__)); + ERROR_IF(PyMapping_GetOptionalItem(LOCALS(), &_Py_ID(__annotations__), &ann_dict) < 0, error); if (ann_dict == NULL) { - ERROR_IF(!_PyErr_ExceptionMatches(tstate, PyExc_KeyError), error); - _PyErr_Clear(tstate); ann_dict = PyDict_New(); ERROR_IF(ann_dict == NULL, error); err = PyObject_SetItem(LOCALS(), &_Py_ID(__annotations__), diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index ba854be7f6c05..681efb97d89a5 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -844,28 +844,13 @@ case LOAD_BUILD_CLASS: { PyObject *bc; #line 1089 "Python/bytecodes.c" - if (PyDict_CheckExact(BUILTINS())) { - bc = _PyDict_GetItemWithError(BUILTINS(), - &_Py_ID(__build_class__)); - if (bc == NULL) { - if (!_PyErr_Occurred(tstate)) { - _PyErr_SetString(tstate, PyExc_NameError, - "__build_class__ not found"); - } - if (true) goto error; - } - Py_INCREF(bc); - } - else { - bc = PyObject_GetItem(BUILTINS(), &_Py_ID(__build_class__)); - if (bc == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) - _PyErr_SetString(tstate, PyExc_NameError, - "__build_class__ not found"); - if (true) goto error; - } + if (PyMapping_GetOptionalItem(BUILTINS(), &_Py_ID(__build_class__), &bc) < 0) goto error; + if (bc == NULL) { + _PyErr_SetString(tstate, PyExc_NameError, + "__build_class__ not found"); + if (true) goto error; } - #line 869 "Python/executor_cases.c.h" + #line 854 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = bc; break; @@ -873,33 +858,33 @@ case STORE_NAME: { PyObject *v = stack_pointer[-1]; - #line 1114 "Python/bytecodes.c" + #line 1099 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err; if (ns == NULL) { _PyErr_Format(tstate, PyExc_SystemError, "no locals found when storing %R", name); - #line 884 "Python/executor_cases.c.h" + #line 869 "Python/executor_cases.c.h" Py_DECREF(v); - #line 1121 "Python/bytecodes.c" + #line 1106 "Python/bytecodes.c" if (true) goto pop_1_error; } if (PyDict_CheckExact(ns)) err = PyDict_SetItem(ns, name, v); else err = PyObject_SetItem(ns, name, v); - #line 893 "Python/executor_cases.c.h" + #line 878 "Python/executor_cases.c.h" Py_DECREF(v); - #line 1128 "Python/bytecodes.c" + #line 1113 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 897 "Python/executor_cases.c.h" + #line 882 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case DELETE_NAME: { - #line 1132 "Python/bytecodes.c" + #line 1117 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err; @@ -916,14 +901,14 @@ name); goto error; } - #line 920 "Python/executor_cases.c.h" + #line 905 "Python/executor_cases.c.h" break; } case UNPACK_SEQUENCE: { static_assert(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE == 1, "incorrect cache size"); PyObject *seq = stack_pointer[-1]; - #line 1158 "Python/bytecodes.c" + #line 1143 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -936,11 +921,11 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject **top = stack_pointer + oparg - 1; int res = unpack_iterable(tstate, seq, oparg, -1, top); - #line 940 "Python/executor_cases.c.h" + #line 925 "Python/executor_cases.c.h" Py_DECREF(seq); - #line 1171 "Python/bytecodes.c" + #line 1156 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 944 "Python/executor_cases.c.h" + #line 929 "Python/executor_cases.c.h" STACK_SHRINK(1); STACK_GROW(oparg); break; @@ -949,14 +934,14 @@ case UNPACK_SEQUENCE_TWO_TUPLE: { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1175 "Python/bytecodes.c" + #line 1160 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != 2, UNPACK_SEQUENCE); assert(oparg == 2); STAT_INC(UNPACK_SEQUENCE, hit); values[0] = Py_NewRef(PyTuple_GET_ITEM(seq, 1)); values[1] = Py_NewRef(PyTuple_GET_ITEM(seq, 0)); - #line 960 "Python/executor_cases.c.h" + #line 945 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -966,7 +951,7 @@ case UNPACK_SEQUENCE_TUPLE: { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1185 "Python/bytecodes.c" + #line 1170 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -974,7 +959,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 978 "Python/executor_cases.c.h" + #line 963 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -984,7 +969,7 @@ case UNPACK_SEQUENCE_LIST: { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1196 "Python/bytecodes.c" + #line 1181 "Python/bytecodes.c" DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyList_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -992,7 +977,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 996 "Python/executor_cases.c.h" + #line 981 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1001,49 +986,49 @@ case UNPACK_EX: { PyObject *seq = stack_pointer[-1]; - #line 1207 "Python/bytecodes.c" + #line 1192 "Python/bytecodes.c" int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); - #line 1009 "Python/executor_cases.c.h" + #line 994 "Python/executor_cases.c.h" Py_DECREF(seq); - #line 1211 "Python/bytecodes.c" + #line 1196 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 1013 "Python/executor_cases.c.h" + #line 998 "Python/executor_cases.c.h" STACK_GROW((oparg & 0xFF) + (oparg >> 8)); break; } case DELETE_ATTR: { PyObject *owner = stack_pointer[-1]; - #line 1242 "Python/bytecodes.c" + #line 1227 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_DelAttr(owner, name); - #line 1023 "Python/executor_cases.c.h" + #line 1008 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1245 "Python/bytecodes.c" + #line 1230 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1027 "Python/executor_cases.c.h" + #line 1012 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case STORE_GLOBAL: { PyObject *v = stack_pointer[-1]; - #line 1249 "Python/bytecodes.c" + #line 1234 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyDict_SetItem(GLOBALS(), name, v); - #line 1037 "Python/executor_cases.c.h" + #line 1022 "Python/executor_cases.c.h" Py_DECREF(v); - #line 1252 "Python/bytecodes.c" + #line 1237 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1041 "Python/executor_cases.c.h" + #line 1026 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case DELETE_GLOBAL: { - #line 1256 "Python/bytecodes.c" + #line 1241 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err; err = PyDict_DelItem(GLOBALS(), name); @@ -1055,13 +1040,13 @@ } goto error; } - #line 1059 "Python/executor_cases.c.h" + #line 1044 "Python/executor_cases.c.h" break; } case _LOAD_LOCALS: { PyObject *locals; - #line 1270 "Python/bytecodes.c" + #line 1255 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1069,7 +1054,7 @@ if (true) goto error; } Py_INCREF(locals); - #line 1073 "Python/executor_cases.c.h" + #line 1058 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = locals; break; @@ -1078,27 +1063,11 @@ case _LOAD_FROM_DICT_OR_GLOBALS: { PyObject *mod_or_class_dict = stack_pointer[-1]; PyObject *v; - #line 1282 "Python/bytecodes.c" + #line 1267 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); - if (PyDict_CheckExact(mod_or_class_dict)) { - v = PyDict_GetItemWithError(mod_or_class_dict, name); - if (v != NULL) { - Py_INCREF(v); - } - else if (_PyErr_Occurred(tstate)) { - Py_DECREF(mod_or_class_dict); - goto error; - } - } - else { - v = PyObject_GetItem(mod_or_class_dict, name); - if (v == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - Py_DECREF(mod_or_class_dict); - goto error; - } - _PyErr_Clear(tstate); - } + if (PyMapping_GetOptionalItem(mod_or_class_dict, name, &v) < 0) { + Py_DECREF(mod_or_class_dict); + goto error; } Py_DECREF(mod_or_class_dict); if (v == NULL) { @@ -1110,32 +1079,18 @@ goto error; } else { - if (PyDict_CheckExact(BUILTINS())) { - v = PyDict_GetItemWithError(BUILTINS(), name); - if (v == NULL) { - if (!_PyErr_Occurred(tstate)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } - goto error; - } - Py_INCREF(v); + if (PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0) { + goto error; } - else { - v = PyObject_GetItem(BUILTINS(), name); - if (v == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } - goto error; - } + if (v == NULL) { + format_exc_check_arg( + tstate, PyExc_NameError, + NAME_ERROR_MSG, name); + goto error; } } } - #line 1139 "Python/executor_cases.c.h" + #line 1094 "Python/executor_cases.c.h" stack_pointer[-1] = v; break; } @@ -1144,7 +1099,7 @@ static_assert(INLINE_CACHE_ENTRIES_LOAD_GLOBAL == 4, "incorrect cache size"); PyObject *null = NULL; PyObject *v; - #line 1351 "Python/bytecodes.c" + #line 1306 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1178,25 +1133,20 @@ /* Slow-path if globals or builtins is not a dict */ /* namespace 1: globals */ - v = PyObject_GetItem(GLOBALS(), name); + if (PyMapping_GetOptionalItem(GLOBALS(), name, &v) < 0) goto error; if (v == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) goto error; - _PyErr_Clear(tstate); - /* namespace 2: builtins */ - v = PyObject_GetItem(BUILTINS(), name); + if (PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0) goto error; if (v == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } + format_exc_check_arg( + tstate, PyExc_NameError, + NAME_ERROR_MSG, name); if (true) goto error; } } } null = NULL; - #line 1200 "Python/executor_cases.c.h" + #line 1150 "Python/executor_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = v; @@ -1205,16 +1155,16 @@ } case DELETE_FAST: { - #line 1435 "Python/bytecodes.c" + #line 1385 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL); - #line 1213 "Python/executor_cases.c.h" + #line 1163 "Python/executor_cases.c.h" break; } case DELETE_DEREF: { - #line 1452 "Python/bytecodes.c" + #line 1402 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); // Can't use ERROR_IF here. @@ -1225,37 +1175,21 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 1229 "Python/executor_cases.c.h" + #line 1179 "Python/executor_cases.c.h" break; } case LOAD_FROM_DICT_OR_DEREF: { PyObject *class_dict = stack_pointer[-1]; PyObject *value; - #line 1465 "Python/bytecodes.c" + #line 1415 "Python/bytecodes.c" PyObject *name; assert(class_dict); assert(oparg >= 0 && oparg < _PyFrame_GetCode(frame)->co_nlocalsplus); name = PyTuple_GET_ITEM(_PyFrame_GetCode(frame)->co_localsplusnames, oparg); - if (PyDict_CheckExact(class_dict)) { - value = PyDict_GetItemWithError(class_dict, name); - if (value != NULL) { - Py_INCREF(value); - } - else if (_PyErr_Occurred(tstate)) { - Py_DECREF(class_dict); - goto error; - } - } - else { - value = PyObject_GetItem(class_dict, name); - if (value == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - Py_DECREF(class_dict); - goto error; - } - _PyErr_Clear(tstate); - } + if (PyMapping_GetOptionalItem(class_dict, name, &value) < 0) { + Py_DECREF(class_dict); + goto error; } Py_DECREF(class_dict); if (!value) { @@ -1267,14 +1201,14 @@ } Py_INCREF(value); } - #line 1271 "Python/executor_cases.c.h" + #line 1205 "Python/executor_cases.c.h" stack_pointer[-1] = value; break; } case LOAD_DEREF: { PyObject *value; - #line 1502 "Python/bytecodes.c" + #line 1436 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { @@ -1282,7 +1216,7 @@ if (true) goto error; } Py_INCREF(value); - #line 1286 "Python/executor_cases.c.h" + #line 1220 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -1290,18 +1224,18 @@ case STORE_DEREF: { PyObject *v = stack_pointer[-1]; - #line 1512 "Python/bytecodes.c" + #line 1446 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 1299 "Python/executor_cases.c.h" + #line 1233 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case COPY_FREE_VARS: { - #line 1519 "Python/bytecodes.c" + #line 1453 "Python/bytecodes.c" /* Copy closure variables to free variables */ PyCodeObject *co = _PyFrame_GetCode(frame); assert(PyFunction_Check(frame->f_funcobj)); @@ -1312,22 +1246,22 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 1316 "Python/executor_cases.c.h" + #line 1250 "Python/executor_cases.c.h" break; } case BUILD_STRING: { PyObject **pieces = (stack_pointer - oparg); PyObject *str; - #line 1532 "Python/bytecodes.c" + #line 1466 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 1325 "Python/executor_cases.c.h" + #line 1259 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } - #line 1534 "Python/bytecodes.c" + #line 1468 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1331 "Python/executor_cases.c.h" + #line 1265 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -1337,10 +1271,10 @@ case BUILD_TUPLE: { PyObject **values = (stack_pointer - oparg); PyObject *tup; - #line 1538 "Python/bytecodes.c" + #line 1472 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1344 "Python/executor_cases.c.h" + #line 1278 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -1350,10 +1284,10 @@ case BUILD_LIST: { PyObject **values = (stack_pointer - oparg); PyObject *list; - #line 1543 "Python/bytecodes.c" + #line 1477 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1357 "Python/executor_cases.c.h" + #line 1291 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -1363,7 +1297,7 @@ case LIST_EXTEND: { PyObject *iterable = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 1548 "Python/bytecodes.c" + #line 1482 "Python/bytecodes.c" PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable); if (none_val == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) && @@ -1374,13 +1308,13 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 1378 "Python/executor_cases.c.h" + #line 1312 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 1559 "Python/bytecodes.c" + #line 1493 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val)); - #line 1384 "Python/executor_cases.c.h" + #line 1318 "Python/executor_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); break; @@ -1389,13 +1323,13 @@ case SET_UPDATE: { PyObject *iterable = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 1566 "Python/bytecodes.c" + #line 1500 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 1395 "Python/executor_cases.c.h" + #line 1329 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 1568 "Python/bytecodes.c" + #line 1502 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 1399 "Python/executor_cases.c.h" + #line 1333 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -1403,7 +1337,7 @@ case BUILD_SET: { PyObject **values = (stack_pointer - oparg); PyObject *set; - #line 1572 "Python/bytecodes.c" + #line 1506 "Python/bytecodes.c" set = PySet_New(NULL); if (set == NULL) goto error; @@ -1418,7 +1352,7 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 1422 "Python/executor_cases.c.h" + #line 1356 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -1428,7 +1362,7 @@ case BUILD_MAP: { PyObject **values = (stack_pointer - oparg*2); PyObject *map; - #line 1589 "Python/bytecodes.c" + #line 1523 "Python/bytecodes.c" map = _PyDict_FromItems( values, 2, values+1, 2, @@ -1436,13 +1370,13 @@ if (map == NULL) goto error; - #line 1440 "Python/executor_cases.c.h" + #line 1374 "Python/executor_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } - #line 1597 "Python/bytecodes.c" + #line 1531 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 1446 "Python/executor_cases.c.h" + #line 1380 "Python/executor_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -1450,7 +1384,7 @@ } case SETUP_ANNOTATIONS: { - #line 1601 "Python/bytecodes.c" + #line 1535 "Python/bytecodes.c" int err; PyObject *ann_dict; if (LOCALS() == NULL) { @@ -1475,10 +1409,8 @@ } else { /* do the same if locals() is not a dict */ - ann_dict = PyObject_GetItem(LOCALS(), &_Py_ID(__annotations__)); + if (PyMapping_GetOptionalItem(LOCALS(), &_Py_ID(__annotations__), &ann_dict) < 0) goto error; if (ann_dict == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) goto error; - _PyErr_Clear(tstate); ann_dict = PyDict_New(); if (ann_dict == NULL) goto error; err = PyObject_SetItem(LOCALS(), &_Py_ID(__annotations__), @@ -1490,7 +1422,7 @@ Py_DECREF(ann_dict); } } - #line 1494 "Python/executor_cases.c.h" + #line 1426 "Python/executor_cases.c.h" break; } @@ -1498,7 +1430,7 @@ PyObject *keys = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); PyObject *map; - #line 1643 "Python/bytecodes.c" + #line 1575 "Python/bytecodes.c" if (!PyTuple_CheckExact(keys) || PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1508,14 +1440,14 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 1512 "Python/executor_cases.c.h" + #line 1444 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); - #line 1653 "Python/bytecodes.c" + #line 1585 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 1519 "Python/executor_cases.c.h" + #line 1451 "Python/executor_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; break; @@ -1523,7 +1455,7 @@ case DICT_UPDATE: { PyObject *update = stack_pointer[-1]; - #line 1657 "Python/bytecodes.c" + #line 1589 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (PyDict_Update(dict, update) < 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { @@ -1531,12 +1463,12 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 1535 "Python/executor_cases.c.h" + #line 1467 "Python/executor_cases.c.h" Py_DECREF(update); - #line 1665 "Python/bytecodes.c" + #line 1597 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1540 "Python/executor_cases.c.h" + #line 1472 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1544,17 +1476,17 @@ case DICT_MERGE: { PyObject *update = stack_pointer[-1]; - #line 1671 "Python/bytecodes.c" + #line 1603 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 1553 "Python/executor_cases.c.h" + #line 1485 "Python/executor_cases.c.h" Py_DECREF(update); - #line 1676 "Python/bytecodes.c" + #line 1608 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1558 "Python/executor_cases.c.h" + #line 1490 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1563,13 +1495,13 @@ case MAP_ADD: { PyObject *value = stack_pointer[-1]; PyObject *key = stack_pointer[-2]; - #line 1682 "Python/bytecodes.c" + #line 1614 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack assert(PyDict_CheckExact(dict)); /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 1573 "Python/executor_cases.c.h" + #line 1505 "Python/executor_cases.c.h" STACK_SHRINK(2); break; } @@ -1580,20 +1512,20 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1765 "Python/bytecodes.c" + #line 1697 "Python/bytecodes.c" assert(!(oparg & 1)); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); STAT_INC(LOAD_SUPER_ATTR, hit); PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL); - #line 1591 "Python/executor_cases.c.h" + #line 1523 "Python/executor_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1772 "Python/bytecodes.c" + #line 1704 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; - #line 1597 "Python/executor_cases.c.h" + #line 1529 "Python/executor_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -1607,7 +1539,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2; PyObject *res; - #line 1776 "Python/bytecodes.c" + #line 1708 "Python/bytecodes.c" assert(oparg & 1); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); @@ -1630,7 +1562,7 @@ res = res2; res2 = NULL; } - #line 1634 "Python/executor_cases.c.h" + #line 1566 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; stack_pointer[-2] = res2; @@ -1642,7 +1574,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1817 "Python/bytecodes.c" + #line 1749 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1676,9 +1608,9 @@ NULL | meth | arg1 | ... | argN */ - #line 1680 "Python/executor_cases.c.h" + #line 1612 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1851 "Python/bytecodes.c" + #line 1783 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -1687,12 +1619,12 @@ else { /* Classic, pushes one value. */ res = PyObject_GetAttr(owner, name); - #line 1691 "Python/executor_cases.c.h" + #line 1623 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1860 "Python/bytecodes.c" + #line 1792 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } - #line 1696 "Python/executor_cases.c.h" + #line 1628 "Python/executor_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -1704,7 +1636,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2093 "Python/bytecodes.c" + #line 2025 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1717,10 +1649,10 @@ #endif /* ENABLE_SPECIALIZATION */ assert((oparg >> 5) <= Py_GE); res = PyObject_RichCompare(left, right, oparg >> 5); - #line 1721 "Python/executor_cases.c.h" + #line 1653 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2106 "Python/bytecodes.c" + #line 2038 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; if (oparg & 16) { int res_bool = PyObject_IsTrue(res); @@ -1728,7 +1660,7 @@ if (res_bool < 0) goto pop_2_error; res = res_bool ? Py_True : Py_False; } - #line 1732 "Python/executor_cases.c.h" + #line 1664 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1738,7 +1670,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2116 "Python/bytecodes.c" + #line 2048 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -1750,7 +1682,7 @@ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1754 "Python/executor_cases.c.h" + #line 1686 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1760,7 +1692,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2131 "Python/bytecodes.c" + #line 2063 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -1776,7 +1708,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1780 "Python/executor_cases.c.h" + #line 1712 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1786,7 +1718,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2150 "Python/bytecodes.c" + #line 2082 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -1799,7 +1731,7 @@ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1803 "Python/executor_cases.c.h" + #line 1735 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1809,14 +1741,14 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2165 "Python/bytecodes.c" + #line 2097 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 1815 "Python/executor_cases.c.h" + #line 1747 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2167 "Python/bytecodes.c" + #line 2099 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1820 "Python/executor_cases.c.h" + #line 1752 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1826,15 +1758,15 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2171 "Python/bytecodes.c" + #line 2103 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 1832 "Python/executor_cases.c.h" + #line 1764 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2173 "Python/bytecodes.c" + #line 2105 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; - #line 1838 "Python/executor_cases.c.h" + #line 1770 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1845,12 +1777,12 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 2178 "Python/bytecodes.c" + #line 2110 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 1851 "Python/executor_cases.c.h" + #line 1783 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2180 "Python/bytecodes.c" + #line 2112 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -1858,10 +1790,10 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 1862 "Python/executor_cases.c.h" + #line 1794 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2188 "Python/bytecodes.c" + #line 2120 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -1870,7 +1802,7 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 1874 "Python/executor_cases.c.h" + #line 1806 "Python/executor_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; break; @@ -1880,21 +1812,21 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2199 "Python/bytecodes.c" + #line 2131 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 1887 "Python/executor_cases.c.h" + #line 1819 "Python/executor_cases.c.h" Py_DECREF(right); - #line 2202 "Python/bytecodes.c" + #line 2134 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 1894 "Python/executor_cases.c.h" + #line 1826 "Python/executor_cases.c.h" Py_DECREF(right); - #line 2207 "Python/bytecodes.c" + #line 2139 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1898 "Python/executor_cases.c.h" + #line 1830 "Python/executor_cases.c.h" stack_pointer[-1] = b; break; } @@ -1902,17 +1834,17 @@ case IS_NONE: { PyObject *value = stack_pointer[-1]; PyObject *b; - #line 2286 "Python/bytecodes.c" + #line 2218 "Python/bytecodes.c" if (Py_IsNone(value)) { b = Py_True; } else { b = Py_False; - #line 1912 "Python/executor_cases.c.h" + #line 1844 "Python/executor_cases.c.h" Py_DECREF(value); - #line 2292 "Python/bytecodes.c" + #line 2224 "Python/bytecodes.c" } - #line 1916 "Python/executor_cases.c.h" + #line 1848 "Python/executor_cases.c.h" stack_pointer[-1] = b; break; } @@ -1920,13 +1852,13 @@ case GET_LEN: { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2309 "Python/bytecodes.c" + #line 2241 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 1930 "Python/executor_cases.c.h" + #line 1862 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; break; @@ -1937,16 +1869,16 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2317 "Python/bytecodes.c" + #line 2249 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 1946 "Python/executor_cases.c.h" + #line 1878 "Python/executor_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2322 "Python/bytecodes.c" + #line 2254 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -1954,7 +1886,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 1958 "Python/executor_cases.c.h" + #line 1890 "Python/executor_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; break; @@ -1963,10 +1895,10 @@ case MATCH_MAPPING: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2332 "Python/bytecodes.c" + #line 2264 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 1970 "Python/executor_cases.c.h" + #line 1902 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1975,10 +1907,10 @@ case MATCH_SEQUENCE: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2337 "Python/bytecodes.c" + #line 2269 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 1982 "Python/executor_cases.c.h" + #line 1914 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1988,11 +1920,11 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2342 "Python/bytecodes.c" + #line 2274 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 1996 "Python/executor_cases.c.h" + #line 1928 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; break; @@ -2001,14 +1933,14 @@ case GET_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2348 "Python/bytecodes.c" + #line 2280 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 2008 "Python/executor_cases.c.h" + #line 1940 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2351 "Python/bytecodes.c" + #line 2283 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 2012 "Python/executor_cases.c.h" + #line 1944 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -2016,7 +1948,7 @@ case GET_YIELD_FROM_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2355 "Python/bytecodes.c" + #line 2287 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -2039,11 +1971,11 @@ if (iter == NULL) { goto error; } - #line 2043 "Python/executor_cases.c.h" + #line 1975 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2378 "Python/bytecodes.c" + #line 2310 "Python/bytecodes.c" } - #line 2047 "Python/executor_cases.c.h" + #line 1979 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -2053,7 +1985,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2610 "Python/bytecodes.c" + #line 2542 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -2074,7 +2006,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 2078 "Python/executor_cases.c.h" + #line 2010 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -2083,7 +2015,7 @@ case PUSH_EXC_INFO: { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2649 "Python/bytecodes.c" + #line 2581 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -2093,7 +2025,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 2097 "Python/executor_cases.c.h" + #line 2029 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -2102,7 +2034,7 @@ case EXIT_INIT_CHECK: { PyObject *should_be_none = stack_pointer[-1]; - #line 3048 "Python/bytecodes.c" + #line 2980 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -2110,7 +2042,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 2114 "Python/executor_cases.c.h" + #line 2046 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -2118,7 +2050,7 @@ case MAKE_FUNCTION: { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3462 "Python/bytecodes.c" + #line 3394 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -2130,7 +2062,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 2134 "Python/executor_cases.c.h" + #line 2066 "Python/executor_cases.c.h" stack_pointer[-1] = func; break; } @@ -2138,7 +2070,7 @@ case SET_FUNCTION_ATTRIBUTE: { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3476 "Python/bytecodes.c" + #line 3408 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -2163,7 +2095,7 @@ default: Py_UNREACHABLE(); } - #line 2167 "Python/executor_cases.c.h" + #line 2099 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; break; @@ -2174,15 +2106,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3526 "Python/bytecodes.c" + #line 3458 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 2180 "Python/executor_cases.c.h" + #line 2112 "Python/executor_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3528 "Python/bytecodes.c" + #line 3460 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 2186 "Python/executor_cases.c.h" + #line 2118 "Python/executor_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -2192,14 +2124,14 @@ case CONVERT_VALUE: { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3532 "Python/bytecodes.c" + #line 3464 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 2203 "Python/executor_cases.c.h" + #line 2135 "Python/executor_cases.c.h" stack_pointer[-1] = result; break; } @@ -2207,7 +2139,7 @@ case FORMAT_SIMPLE: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3541 "Python/bytecodes.c" + #line 3473 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -2218,7 +2150,7 @@ else { res = value; } - #line 2222 "Python/executor_cases.c.h" + #line 2154 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -2227,12 +2159,12 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3554 "Python/bytecodes.c" + #line 3486 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 2236 "Python/executor_cases.c.h" + #line 2168 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -2241,10 +2173,10 @@ case COPY: { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3561 "Python/bytecodes.c" + #line 3493 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 2248 "Python/executor_cases.c.h" + #line 2180 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; break; @@ -2255,7 +2187,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3566 "Python/bytecodes.c" + #line 3498 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2270,12 +2202,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 2274 "Python/executor_cases.c.h" + #line 2206 "Python/executor_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3581 "Python/bytecodes.c" + #line 3513 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 2279 "Python/executor_cases.c.h" + #line 2211 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -2284,9 +2216,9 @@ case SWAP: { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3586 "Python/bytecodes.c" + #line 3518 "Python/bytecodes.c" assert(oparg >= 2); - #line 2290 "Python/executor_cases.c.h" + #line 2222 "Python/executor_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; break; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 325090a60f9d5..eb3de5e5bca97 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -1484,28 +1484,13 @@ TARGET(LOAD_BUILD_CLASS) { PyObject *bc; #line 1089 "Python/bytecodes.c" - if (PyDict_CheckExact(BUILTINS())) { - bc = _PyDict_GetItemWithError(BUILTINS(), - &_Py_ID(__build_class__)); - if (bc == NULL) { - if (!_PyErr_Occurred(tstate)) { - _PyErr_SetString(tstate, PyExc_NameError, - "__build_class__ not found"); - } - if (true) goto error; - } - Py_INCREF(bc); - } - else { - bc = PyObject_GetItem(BUILTINS(), &_Py_ID(__build_class__)); - if (bc == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) - _PyErr_SetString(tstate, PyExc_NameError, - "__build_class__ not found"); - if (true) goto error; - } + if (PyMapping_GetOptionalItem(BUILTINS(), &_Py_ID(__build_class__), &bc) < 0) goto error; + if (bc == NULL) { + _PyErr_SetString(tstate, PyExc_NameError, + "__build_class__ not found"); + if (true) goto error; } - #line 1509 "Python/generated_cases.c.h" + #line 1494 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = bc; DISPATCH(); @@ -1513,33 +1498,33 @@ TARGET(STORE_NAME) { PyObject *v = stack_pointer[-1]; - #line 1114 "Python/bytecodes.c" + #line 1099 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err; if (ns == NULL) { _PyErr_Format(tstate, PyExc_SystemError, "no locals found when storing %R", name); - #line 1524 "Python/generated_cases.c.h" + #line 1509 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1121 "Python/bytecodes.c" + #line 1106 "Python/bytecodes.c" if (true) goto pop_1_error; } if (PyDict_CheckExact(ns)) err = PyDict_SetItem(ns, name, v); else err = PyObject_SetItem(ns, name, v); - #line 1533 "Python/generated_cases.c.h" + #line 1518 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1128 "Python/bytecodes.c" + #line 1113 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1537 "Python/generated_cases.c.h" + #line 1522 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(DELETE_NAME) { - #line 1132 "Python/bytecodes.c" + #line 1117 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err; @@ -1556,7 +1541,7 @@ name); goto error; } - #line 1560 "Python/generated_cases.c.h" + #line 1545 "Python/generated_cases.c.h" DISPATCH(); } @@ -1564,7 +1549,7 @@ PREDICTED(UNPACK_SEQUENCE); static_assert(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE == 1, "incorrect cache size"); PyObject *seq = stack_pointer[-1]; - #line 1158 "Python/bytecodes.c" + #line 1143 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1577,11 +1562,11 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject **top = stack_pointer + oparg - 1; int res = unpack_iterable(tstate, seq, oparg, -1, top); - #line 1581 "Python/generated_cases.c.h" + #line 1566 "Python/generated_cases.c.h" Py_DECREF(seq); - #line 1171 "Python/bytecodes.c" + #line 1156 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 1585 "Python/generated_cases.c.h" + #line 1570 "Python/generated_cases.c.h" STACK_SHRINK(1); STACK_GROW(oparg); next_instr += 1; @@ -1591,14 +1576,14 @@ TARGET(UNPACK_SEQUENCE_TWO_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1175 "Python/bytecodes.c" + #line 1160 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != 2, UNPACK_SEQUENCE); assert(oparg == 2); STAT_INC(UNPACK_SEQUENCE, hit); values[0] = Py_NewRef(PyTuple_GET_ITEM(seq, 1)); values[1] = Py_NewRef(PyTuple_GET_ITEM(seq, 0)); - #line 1602 "Python/generated_cases.c.h" + #line 1587 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1609,7 +1594,7 @@ TARGET(UNPACK_SEQUENCE_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1185 "Python/bytecodes.c" + #line 1170 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -1617,7 +1602,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 1621 "Python/generated_cases.c.h" + #line 1606 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1628,7 +1613,7 @@ TARGET(UNPACK_SEQUENCE_LIST) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1196 "Python/bytecodes.c" + #line 1181 "Python/bytecodes.c" DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyList_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -1636,7 +1621,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 1640 "Python/generated_cases.c.h" + #line 1625 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1646,15 +1631,15 @@ TARGET(UNPACK_EX) { PyObject *seq = stack_pointer[-1]; - #line 1207 "Python/bytecodes.c" + #line 1192 "Python/bytecodes.c" int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); - #line 1654 "Python/generated_cases.c.h" + #line 1639 "Python/generated_cases.c.h" Py_DECREF(seq); - #line 1211 "Python/bytecodes.c" + #line 1196 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 1658 "Python/generated_cases.c.h" + #line 1643 "Python/generated_cases.c.h" STACK_GROW((oparg & 0xFF) + (oparg >> 8)); DISPATCH(); } @@ -1665,7 +1650,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *v = stack_pointer[-2]; uint16_t counter = read_u16(&next_instr[0].cache); - #line 1222 "Python/bytecodes.c" + #line 1207 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); @@ -1681,12 +1666,12 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_SetAttr(owner, name, v); - #line 1685 "Python/generated_cases.c.h" + #line 1670 "Python/generated_cases.c.h" Py_DECREF(v); Py_DECREF(owner); - #line 1238 "Python/bytecodes.c" + #line 1223 "Python/bytecodes.c" if (err) goto pop_2_error; - #line 1690 "Python/generated_cases.c.h" + #line 1675 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -1694,34 +1679,34 @@ TARGET(DELETE_ATTR) { PyObject *owner = stack_pointer[-1]; - #line 1242 "Python/bytecodes.c" + #line 1227 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_DelAttr(owner, name); - #line 1701 "Python/generated_cases.c.h" + #line 1686 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1245 "Python/bytecodes.c" + #line 1230 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1705 "Python/generated_cases.c.h" + #line 1690 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(STORE_GLOBAL) { PyObject *v = stack_pointer[-1]; - #line 1249 "Python/bytecodes.c" + #line 1234 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyDict_SetItem(GLOBALS(), name, v); - #line 1715 "Python/generated_cases.c.h" + #line 1700 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1252 "Python/bytecodes.c" + #line 1237 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1719 "Python/generated_cases.c.h" + #line 1704 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(DELETE_GLOBAL) { - #line 1256 "Python/bytecodes.c" + #line 1241 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err; err = PyDict_DelItem(GLOBALS(), name); @@ -1733,7 +1718,7 @@ } goto error; } - #line 1737 "Python/generated_cases.c.h" + #line 1722 "Python/generated_cases.c.h" DISPATCH(); } @@ -1741,7 +1726,7 @@ PyObject *_tmp_1; { PyObject *locals; - #line 1270 "Python/bytecodes.c" + #line 1255 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1749,7 +1734,7 @@ if (true) goto error; } Py_INCREF(locals); - #line 1753 "Python/generated_cases.c.h" + #line 1738 "Python/generated_cases.c.h" _tmp_1 = locals; } STACK_GROW(1); @@ -1761,7 +1746,7 @@ PyObject *_tmp_1; { PyObject *locals; - #line 1270 "Python/bytecodes.c" + #line 1255 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1769,33 +1754,17 @@ if (true) goto error; } Py_INCREF(locals); - #line 1773 "Python/generated_cases.c.h" + #line 1758 "Python/generated_cases.c.h" _tmp_1 = locals; } { PyObject *mod_or_class_dict = _tmp_1; PyObject *v; - #line 1282 "Python/bytecodes.c" + #line 1267 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); - if (PyDict_CheckExact(mod_or_class_dict)) { - v = PyDict_GetItemWithError(mod_or_class_dict, name); - if (v != NULL) { - Py_INCREF(v); - } - else if (_PyErr_Occurred(tstate)) { - Py_DECREF(mod_or_class_dict); - goto error; - } - } - else { - v = PyObject_GetItem(mod_or_class_dict, name); - if (v == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - Py_DECREF(mod_or_class_dict); - goto error; - } - _PyErr_Clear(tstate); - } + if (PyMapping_GetOptionalItem(mod_or_class_dict, name, &v) < 0) { + Py_DECREF(mod_or_class_dict); + goto error; } Py_DECREF(mod_or_class_dict); if (v == NULL) { @@ -1807,32 +1776,18 @@ goto error; } else { - if (PyDict_CheckExact(BUILTINS())) { - v = PyDict_GetItemWithError(BUILTINS(), name); - if (v == NULL) { - if (!_PyErr_Occurred(tstate)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } - goto error; - } - Py_INCREF(v); + if (PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0) { + goto error; } - else { - v = PyObject_GetItem(BUILTINS(), name); - if (v == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } - goto error; - } + if (v == NULL) { + format_exc_check_arg( + tstate, PyExc_NameError, + NAME_ERROR_MSG, name); + goto error; } } } - #line 1836 "Python/generated_cases.c.h" + #line 1791 "Python/generated_cases.c.h" _tmp_1 = v; } STACK_GROW(1); @@ -1845,27 +1800,11 @@ { PyObject *mod_or_class_dict = _tmp_1; PyObject *v; - #line 1282 "Python/bytecodes.c" + #line 1267 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); - if (PyDict_CheckExact(mod_or_class_dict)) { - v = PyDict_GetItemWithError(mod_or_class_dict, name); - if (v != NULL) { - Py_INCREF(v); - } - else if (_PyErr_Occurred(tstate)) { - Py_DECREF(mod_or_class_dict); - goto error; - } - } - else { - v = PyObject_GetItem(mod_or_class_dict, name); - if (v == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - Py_DECREF(mod_or_class_dict); - goto error; - } - _PyErr_Clear(tstate); - } + if (PyMapping_GetOptionalItem(mod_or_class_dict, name, &v) < 0) { + Py_DECREF(mod_or_class_dict); + goto error; } Py_DECREF(mod_or_class_dict); if (v == NULL) { @@ -1877,32 +1816,18 @@ goto error; } else { - if (PyDict_CheckExact(BUILTINS())) { - v = PyDict_GetItemWithError(BUILTINS(), name); - if (v == NULL) { - if (!_PyErr_Occurred(tstate)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } - goto error; - } - Py_INCREF(v); + if (PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0) { + goto error; } - else { - v = PyObject_GetItem(BUILTINS(), name); - if (v == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } - goto error; - } + if (v == NULL) { + format_exc_check_arg( + tstate, PyExc_NameError, + NAME_ERROR_MSG, name); + goto error; } } } - #line 1906 "Python/generated_cases.c.h" + #line 1831 "Python/generated_cases.c.h" _tmp_1 = v; } stack_pointer[-1] = _tmp_1; @@ -1914,7 +1839,7 @@ static_assert(INLINE_CACHE_ENTRIES_LOAD_GLOBAL == 4, "incorrect cache size"); PyObject *null = NULL; PyObject *v; - #line 1351 "Python/bytecodes.c" + #line 1306 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1948,25 +1873,20 @@ /* Slow-path if globals or builtins is not a dict */ /* namespace 1: globals */ - v = PyObject_GetItem(GLOBALS(), name); + if (PyMapping_GetOptionalItem(GLOBALS(), name, &v) < 0) goto error; if (v == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) goto error; - _PyErr_Clear(tstate); - /* namespace 2: builtins */ - v = PyObject_GetItem(BUILTINS(), name); + if (PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0) goto error; if (v == NULL) { - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg( - tstate, PyExc_NameError, - NAME_ERROR_MSG, name); - } + format_exc_check_arg( + tstate, PyExc_NameError, + NAME_ERROR_MSG, name); if (true) goto error; } } } null = NULL; - #line 1970 "Python/generated_cases.c.h" + #line 1890 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = v; @@ -1980,7 +1900,7 @@ PyObject *res; uint16_t index = read_u16(&next_instr[1].cache); uint16_t version = read_u16(&next_instr[2].cache); - #line 1405 "Python/bytecodes.c" + #line 1355 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); PyDictObject *dict = (PyDictObject *)GLOBALS(); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); @@ -1991,7 +1911,7 @@ Py_INCREF(res); STAT_INC(LOAD_GLOBAL, hit); null = NULL; - #line 1995 "Python/generated_cases.c.h" + #line 1915 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2006,7 +1926,7 @@ uint16_t index = read_u16(&next_instr[1].cache); uint16_t mod_version = read_u16(&next_instr[2].cache); uint16_t bltn_version = read_u16(&next_instr[3].cache); - #line 1418 "Python/bytecodes.c" + #line 1368 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); DEOPT_IF(!PyDict_CheckExact(BUILTINS()), LOAD_GLOBAL); PyDictObject *mdict = (PyDictObject *)GLOBALS(); @@ -2021,7 +1941,7 @@ Py_INCREF(res); STAT_INC(LOAD_GLOBAL, hit); null = NULL; - #line 2025 "Python/generated_cases.c.h" + #line 1945 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2031,16 +1951,16 @@ } TARGET(DELETE_FAST) { - #line 1435 "Python/bytecodes.c" + #line 1385 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL); - #line 2039 "Python/generated_cases.c.h" + #line 1959 "Python/generated_cases.c.h" DISPATCH(); } TARGET(MAKE_CELL) { - #line 1441 "Python/bytecodes.c" + #line 1391 "Python/bytecodes.c" // "initial" is probably NULL but not if it's an arg (or set // via PyFrame_LocalsToFast() before MAKE_CELL has run). PyObject *initial = GETLOCAL(oparg); @@ -2049,12 +1969,12 @@ goto resume_with_error; } SETLOCAL(oparg, cell); - #line 2053 "Python/generated_cases.c.h" + #line 1973 "Python/generated_cases.c.h" DISPATCH(); } TARGET(DELETE_DEREF) { - #line 1452 "Python/bytecodes.c" + #line 1402 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); // Can't use ERROR_IF here. @@ -2065,37 +1985,21 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 2069 "Python/generated_cases.c.h" + #line 1989 "Python/generated_cases.c.h" DISPATCH(); } TARGET(LOAD_FROM_DICT_OR_DEREF) { PyObject *class_dict = stack_pointer[-1]; PyObject *value; - #line 1465 "Python/bytecodes.c" + #line 1415 "Python/bytecodes.c" PyObject *name; assert(class_dict); assert(oparg >= 0 && oparg < _PyFrame_GetCode(frame)->co_nlocalsplus); name = PyTuple_GET_ITEM(_PyFrame_GetCode(frame)->co_localsplusnames, oparg); - if (PyDict_CheckExact(class_dict)) { - value = PyDict_GetItemWithError(class_dict, name); - if (value != NULL) { - Py_INCREF(value); - } - else if (_PyErr_Occurred(tstate)) { - Py_DECREF(class_dict); - goto error; - } - } - else { - value = PyObject_GetItem(class_dict, name); - if (value == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - Py_DECREF(class_dict); - goto error; - } - _PyErr_Clear(tstate); - } + if (PyMapping_GetOptionalItem(class_dict, name, &value) < 0) { + Py_DECREF(class_dict); + goto error; } Py_DECREF(class_dict); if (!value) { @@ -2107,14 +2011,14 @@ } Py_INCREF(value); } - #line 2111 "Python/generated_cases.c.h" + #line 2015 "Python/generated_cases.c.h" stack_pointer[-1] = value; DISPATCH(); } TARGET(LOAD_DEREF) { PyObject *value; - #line 1502 "Python/bytecodes.c" + #line 1436 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { @@ -2122,7 +2026,7 @@ if (true) goto error; } Py_INCREF(value); - #line 2126 "Python/generated_cases.c.h" + #line 2030 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -2130,18 +2034,18 @@ TARGET(STORE_DEREF) { PyObject *v = stack_pointer[-1]; - #line 1512 "Python/bytecodes.c" + #line 1446 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 2139 "Python/generated_cases.c.h" + #line 2043 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(COPY_FREE_VARS) { - #line 1519 "Python/bytecodes.c" + #line 1453 "Python/bytecodes.c" /* Copy closure variables to free variables */ PyCodeObject *co = _PyFrame_GetCode(frame); assert(PyFunction_Check(frame->f_funcobj)); @@ -2152,22 +2056,22 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 2156 "Python/generated_cases.c.h" + #line 2060 "Python/generated_cases.c.h" DISPATCH(); } TARGET(BUILD_STRING) { PyObject **pieces = (stack_pointer - oparg); PyObject *str; - #line 1532 "Python/bytecodes.c" + #line 1466 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 2165 "Python/generated_cases.c.h" + #line 2069 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } - #line 1534 "Python/bytecodes.c" + #line 1468 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2171 "Python/generated_cases.c.h" + #line 2075 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -2177,10 +2081,10 @@ TARGET(BUILD_TUPLE) { PyObject **values = (stack_pointer - oparg); PyObject *tup; - #line 1538 "Python/bytecodes.c" + #line 1472 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2184 "Python/generated_cases.c.h" + #line 2088 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -2190,10 +2094,10 @@ TARGET(BUILD_LIST) { PyObject **values = (stack_pointer - oparg); PyObject *list; - #line 1543 "Python/bytecodes.c" + #line 1477 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2197 "Python/generated_cases.c.h" + #line 2101 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -2203,7 +2107,7 @@ TARGET(LIST_EXTEND) { PyObject *iterable = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 1548 "Python/bytecodes.c" + #line 1482 "Python/bytecodes.c" PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable); if (none_val == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) && @@ -2214,13 +2118,13 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 2218 "Python/generated_cases.c.h" + #line 2122 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1559 "Python/bytecodes.c" + #line 1493 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val)); - #line 2224 "Python/generated_cases.c.h" + #line 2128 "Python/generated_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); DISPATCH(); @@ -2229,13 +2133,13 @@ TARGET(SET_UPDATE) { PyObject *iterable = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 1566 "Python/bytecodes.c" + #line 1500 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 2235 "Python/generated_cases.c.h" + #line 2139 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1568 "Python/bytecodes.c" + #line 1502 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 2239 "Python/generated_cases.c.h" + #line 2143 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -2243,7 +2147,7 @@ TARGET(BUILD_SET) { PyObject **values = (stack_pointer - oparg); PyObject *set; - #line 1572 "Python/bytecodes.c" + #line 1506 "Python/bytecodes.c" set = PySet_New(NULL); if (set == NULL) goto error; @@ -2258,7 +2162,7 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 2262 "Python/generated_cases.c.h" + #line 2166 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -2268,7 +2172,7 @@ TARGET(BUILD_MAP) { PyObject **values = (stack_pointer - oparg*2); PyObject *map; - #line 1589 "Python/bytecodes.c" + #line 1523 "Python/bytecodes.c" map = _PyDict_FromItems( values, 2, values+1, 2, @@ -2276,13 +2180,13 @@ if (map == NULL) goto error; - #line 2280 "Python/generated_cases.c.h" + #line 2184 "Python/generated_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } - #line 1597 "Python/bytecodes.c" + #line 1531 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 2286 "Python/generated_cases.c.h" + #line 2190 "Python/generated_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -2290,7 +2194,7 @@ } TARGET(SETUP_ANNOTATIONS) { - #line 1601 "Python/bytecodes.c" + #line 1535 "Python/bytecodes.c" int err; PyObject *ann_dict; if (LOCALS() == NULL) { @@ -2315,10 +2219,8 @@ } else { /* do the same if locals() is not a dict */ - ann_dict = PyObject_GetItem(LOCALS(), &_Py_ID(__annotations__)); + if (PyMapping_GetOptionalItem(LOCALS(), &_Py_ID(__annotations__), &ann_dict) < 0) goto error; if (ann_dict == NULL) { - if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) goto error; - _PyErr_Clear(tstate); ann_dict = PyDict_New(); if (ann_dict == NULL) goto error; err = PyObject_SetItem(LOCALS(), &_Py_ID(__annotations__), @@ -2330,7 +2232,7 @@ Py_DECREF(ann_dict); } } - #line 2334 "Python/generated_cases.c.h" + #line 2236 "Python/generated_cases.c.h" DISPATCH(); } @@ -2338,7 +2240,7 @@ PyObject *keys = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); PyObject *map; - #line 1643 "Python/bytecodes.c" + #line 1575 "Python/bytecodes.c" if (!PyTuple_CheckExact(keys) || PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -2348,14 +2250,14 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 2352 "Python/generated_cases.c.h" + #line 2254 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); - #line 1653 "Python/bytecodes.c" + #line 1585 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 2359 "Python/generated_cases.c.h" + #line 2261 "Python/generated_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; DISPATCH(); @@ -2363,7 +2265,7 @@ TARGET(DICT_UPDATE) { PyObject *update = stack_pointer[-1]; - #line 1657 "Python/bytecodes.c" + #line 1589 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (PyDict_Update(dict, update) < 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { @@ -2371,12 +2273,12 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 2375 "Python/generated_cases.c.h" + #line 2277 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1665 "Python/bytecodes.c" + #line 1597 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 2380 "Python/generated_cases.c.h" + #line 2282 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); DISPATCH(); @@ -2384,17 +2286,17 @@ TARGET(DICT_MERGE) { PyObject *update = stack_pointer[-1]; - #line 1671 "Python/bytecodes.c" + #line 1603 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 2393 "Python/generated_cases.c.h" + #line 2295 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1676 "Python/bytecodes.c" + #line 1608 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 2398 "Python/generated_cases.c.h" + #line 2300 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); DISPATCH(); @@ -2403,25 +2305,25 @@ TARGET(MAP_ADD) { PyObject *value = stack_pointer[-1]; PyObject *key = stack_pointer[-2]; - #line 1682 "Python/bytecodes.c" + #line 1614 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack assert(PyDict_CheckExact(dict)); /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 2413 "Python/generated_cases.c.h" + #line 2315 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } TARGET(INSTRUMENTED_LOAD_SUPER_ATTR) { - #line 1690 "Python/bytecodes.c" + #line 1622 "Python/bytecodes.c" _PySuperAttrCache *cache = (_PySuperAttrCache *)next_instr; // cancel out the decrement that will happen in LOAD_SUPER_ATTR; we // don't want to specialize instrumented instructions INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(LOAD_SUPER_ATTR); - #line 2425 "Python/generated_cases.c.h" + #line 2327 "Python/generated_cases.c.h" } TARGET(LOAD_SUPER_ATTR) { @@ -2432,7 +2334,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1704 "Python/bytecodes.c" + #line 1636 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); int load_method = oparg & 1; #if ENABLE_SPECIALIZATION @@ -2474,16 +2376,16 @@ } } } - #line 2478 "Python/generated_cases.c.h" + #line 2380 "Python/generated_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1746 "Python/bytecodes.c" + #line 1678 "Python/bytecodes.c" if (super == NULL) goto pop_3_error; res = PyObject_GetAttr(super, name); Py_DECREF(super); if (res == NULL) goto pop_3_error; - #line 2487 "Python/generated_cases.c.h" + #line 2389 "Python/generated_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2498,20 +2400,20 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1765 "Python/bytecodes.c" + #line 1697 "Python/bytecodes.c" assert(!(oparg & 1)); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); STAT_INC(LOAD_SUPER_ATTR, hit); PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL); - #line 2509 "Python/generated_cases.c.h" + #line 2411 "Python/generated_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1772 "Python/bytecodes.c" + #line 1704 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; - #line 2515 "Python/generated_cases.c.h" + #line 2417 "Python/generated_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2526,7 +2428,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2; PyObject *res; - #line 1776 "Python/bytecodes.c" + #line 1708 "Python/bytecodes.c" assert(oparg & 1); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); @@ -2549,7 +2451,7 @@ res = res2; res2 = NULL; } - #line 2553 "Python/generated_cases.c.h" + #line 2455 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; stack_pointer[-2] = res2; @@ -2563,7 +2465,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1817 "Python/bytecodes.c" + #line 1749 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2597,9 +2499,9 @@ NULL | meth | arg1 | ... | argN */ - #line 2601 "Python/generated_cases.c.h" + #line 2503 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1851 "Python/bytecodes.c" + #line 1783 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -2608,12 +2510,12 @@ else { /* Classic, pushes one value. */ res = PyObject_GetAttr(owner, name); - #line 2612 "Python/generated_cases.c.h" + #line 2514 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1860 "Python/bytecodes.c" + #line 1792 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } - #line 2617 "Python/generated_cases.c.h" + #line 2519 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -2627,7 +2529,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1869 "Python/bytecodes.c" + #line 1801 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2640,7 +2542,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2644 "Python/generated_cases.c.h" + #line 2546 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2655,7 +2557,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1885 "Python/bytecodes.c" + #line 1817 "Python/bytecodes.c" DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; assert(dict != NULL); @@ -2668,7 +2570,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2672 "Python/generated_cases.c.h" + #line 2574 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2683,7 +2585,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1901 "Python/bytecodes.c" + #line 1833 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2710,7 +2612,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2714 "Python/generated_cases.c.h" + #line 2616 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2725,7 +2627,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1931 "Python/bytecodes.c" + #line 1863 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2735,7 +2637,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2739 "Python/generated_cases.c.h" + #line 2641 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2750,7 +2652,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 1944 "Python/bytecodes.c" + #line 1876 "Python/bytecodes.c" DEOPT_IF(!PyType_Check(cls), LOAD_ATTR); DEOPT_IF(((PyTypeObject *)cls)->tp_version_tag != type_version, @@ -2762,7 +2664,7 @@ res = descr; assert(res != NULL); Py_INCREF(res); - #line 2766 "Python/generated_cases.c.h" + #line 2668 "Python/generated_cases.c.h" Py_DECREF(cls); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2776,7 +2678,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *fget = read_obj(&next_instr[5].cache); - #line 1959 "Python/bytecodes.c" + #line 1891 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); @@ -2800,7 +2702,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_LOAD_ATTR); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 2804 "Python/generated_cases.c.h" + #line 2706 "Python/generated_cases.c.h" } TARGET(LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN) { @@ -2808,7 +2710,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *getattribute = read_obj(&next_instr[5].cache); - #line 1985 "Python/bytecodes.c" + #line 1917 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); DEOPT_IF(cls->tp_version_tag != type_version, LOAD_ATTR); @@ -2834,7 +2736,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_LOAD_ATTR); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 2838 "Python/generated_cases.c.h" + #line 2740 "Python/generated_cases.c.h" } TARGET(STORE_ATTR_INSTANCE_VALUE) { @@ -2842,7 +2744,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 2013 "Python/bytecodes.c" + #line 1945 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2860,7 +2762,7 @@ Py_DECREF(old_value); } Py_DECREF(owner); - #line 2864 "Python/generated_cases.c.h" + #line 2766 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2871,7 +2773,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t hint = read_u16(&next_instr[3].cache); - #line 2033 "Python/bytecodes.c" + #line 1965 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2910,7 +2812,7 @@ /* PEP 509 */ dict->ma_version_tag = new_version; Py_DECREF(owner); - #line 2914 "Python/generated_cases.c.h" + #line 2816 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2921,7 +2823,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 2074 "Python/bytecodes.c" + #line 2006 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2931,7 +2833,7 @@ *(PyObject **)addr = value; Py_XDECREF(old_value); Py_DECREF(owner); - #line 2935 "Python/generated_cases.c.h" + #line 2837 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2943,7 +2845,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2093 "Python/bytecodes.c" + #line 2025 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2956,10 +2858,10 @@ #endif /* ENABLE_SPECIALIZATION */ assert((oparg >> 5) <= Py_GE); res = PyObject_RichCompare(left, right, oparg >> 5); - #line 2960 "Python/generated_cases.c.h" + #line 2862 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2106 "Python/bytecodes.c" + #line 2038 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; if (oparg & 16) { int res_bool = PyObject_IsTrue(res); @@ -2967,7 +2869,7 @@ if (res_bool < 0) goto pop_2_error; res = res_bool ? Py_True : Py_False; } - #line 2971 "Python/generated_cases.c.h" + #line 2873 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2978,7 +2880,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2116 "Python/bytecodes.c" + #line 2048 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2990,7 +2892,7 @@ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 2994 "Python/generated_cases.c.h" + #line 2896 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -3001,7 +2903,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2131 "Python/bytecodes.c" + #line 2063 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -3017,7 +2919,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 3021 "Python/generated_cases.c.h" + #line 2923 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -3028,7 +2930,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2150 "Python/bytecodes.c" + #line 2082 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -3041,7 +2943,7 @@ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 3045 "Python/generated_cases.c.h" + #line 2947 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -3052,14 +2954,14 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2165 "Python/bytecodes.c" + #line 2097 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 3058 "Python/generated_cases.c.h" + #line 2960 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2167 "Python/bytecodes.c" + #line 2099 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 3063 "Python/generated_cases.c.h" + #line 2965 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -3069,15 +2971,15 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2171 "Python/bytecodes.c" + #line 2103 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 3075 "Python/generated_cases.c.h" + #line 2977 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2173 "Python/bytecodes.c" + #line 2105 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; - #line 3081 "Python/generated_cases.c.h" + #line 2983 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -3088,12 +2990,12 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 2178 "Python/bytecodes.c" + #line 2110 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 3094 "Python/generated_cases.c.h" + #line 2996 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2180 "Python/bytecodes.c" + #line 2112 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -3101,10 +3003,10 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 3105 "Python/generated_cases.c.h" + #line 3007 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2188 "Python/bytecodes.c" + #line 2120 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -3113,7 +3015,7 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 3117 "Python/generated_cases.c.h" + #line 3019 "Python/generated_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; DISPATCH(); @@ -3123,21 +3025,21 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2199 "Python/bytecodes.c" + #line 2131 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 3130 "Python/generated_cases.c.h" + #line 3032 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2202 "Python/bytecodes.c" + #line 2134 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 3137 "Python/generated_cases.c.h" + #line 3039 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2207 "Python/bytecodes.c" + #line 2139 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 3141 "Python/generated_cases.c.h" + #line 3043 "Python/generated_cases.c.h" stack_pointer[-1] = b; DISPATCH(); } @@ -3146,15 +3048,15 @@ PyObject *fromlist = stack_pointer[-1]; PyObject *level = stack_pointer[-2]; PyObject *res; - #line 2211 "Python/bytecodes.c" + #line 2143 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_name(tstate, frame, name, fromlist, level); - #line 3153 "Python/generated_cases.c.h" + #line 3055 "Python/generated_cases.c.h" Py_DECREF(level); Py_DECREF(fromlist); - #line 2214 "Python/bytecodes.c" + #line 2146 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 3158 "Python/generated_cases.c.h" + #line 3060 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -3163,25 +3065,25 @@ TARGET(IMPORT_FROM) { PyObject *from = stack_pointer[-1]; PyObject *res; - #line 2218 "Python/bytecodes.c" + #line 2150 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_from(tstate, from, name); if (res == NULL) goto error; - #line 3171 "Python/generated_cases.c.h" + #line 3073 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); } TARGET(JUMP_FORWARD) { - #line 2224 "Python/bytecodes.c" + #line 2156 "Python/bytecodes.c" JUMPBY(oparg); - #line 3180 "Python/generated_cases.c.h" + #line 3082 "Python/generated_cases.c.h" DISPATCH(); } TARGET(JUMP_BACKWARD) { - #line 2228 "Python/bytecodes.c" + #line 2160 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); _Py_CODEUNIT *here = next_instr - 1; assert(oparg <= INSTR_OFFSET()); @@ -3200,12 +3102,12 @@ goto resume_frame; } #endif /* ENABLE_SPECIALIZATION */ - #line 3204 "Python/generated_cases.c.h" + #line 3106 "Python/generated_cases.c.h" DISPATCH(); } TARGET(ENTER_EXECUTOR) { - #line 2259 "Python/bytecodes.c" + #line 2191 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); PyCodeObject *code = _PyFrame_GetCode(frame); @@ -3220,25 +3122,25 @@ goto resume_with_error; } goto resume_frame; - #line 3224 "Python/generated_cases.c.h" + #line 3126 "Python/generated_cases.c.h" } TARGET(POP_JUMP_IF_FALSE) { PyObject *cond = stack_pointer[-1]; - #line 2276 "Python/bytecodes.c" + #line 2208 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsFalse(cond)); - #line 3232 "Python/generated_cases.c.h" + #line 3134 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_TRUE) { PyObject *cond = stack_pointer[-1]; - #line 2281 "Python/bytecodes.c" + #line 2213 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsTrue(cond)); - #line 3242 "Python/generated_cases.c.h" + #line 3144 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -3248,25 +3150,25 @@ { PyObject *value = _tmp_1; PyObject *b; - #line 2286 "Python/bytecodes.c" + #line 2218 "Python/bytecodes.c" if (Py_IsNone(value)) { b = Py_True; } else { b = Py_False; - #line 3258 "Python/generated_cases.c.h" + #line 3160 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2292 "Python/bytecodes.c" + #line 2224 "Python/bytecodes.c" } - #line 3262 "Python/generated_cases.c.h" + #line 3164 "Python/generated_cases.c.h" _tmp_1 = b; } { PyObject *cond = _tmp_1; - #line 2281 "Python/bytecodes.c" + #line 2213 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsTrue(cond)); - #line 3270 "Python/generated_cases.c.h" + #line 3172 "Python/generated_cases.c.h" } STACK_SHRINK(1); DISPATCH(); @@ -3277,52 +3179,52 @@ { PyObject *value = _tmp_1; PyObject *b; - #line 2286 "Python/bytecodes.c" + #line 2218 "Python/bytecodes.c" if (Py_IsNone(value)) { b = Py_True; } else { b = Py_False; - #line 3287 "Python/generated_cases.c.h" + #line 3189 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2292 "Python/bytecodes.c" + #line 2224 "Python/bytecodes.c" } - #line 3291 "Python/generated_cases.c.h" + #line 3193 "Python/generated_cases.c.h" _tmp_1 = b; } { PyObject *cond = _tmp_1; - #line 2276 "Python/bytecodes.c" + #line 2208 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsFalse(cond)); - #line 3299 "Python/generated_cases.c.h" + #line 3201 "Python/generated_cases.c.h" } STACK_SHRINK(1); DISPATCH(); } TARGET(JUMP_BACKWARD_NO_INTERRUPT) { - #line 2300 "Python/bytecodes.c" + #line 2232 "Python/bytecodes.c" /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost * generator or coroutine, so we deliberately do not check it here. * (see bpo-30039). */ JUMPBY(-oparg); - #line 3313 "Python/generated_cases.c.h" + #line 3215 "Python/generated_cases.c.h" DISPATCH(); } TARGET(GET_LEN) { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2309 "Python/bytecodes.c" + #line 2241 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 3326 "Python/generated_cases.c.h" + #line 3228 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; DISPATCH(); @@ -3333,16 +3235,16 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2317 "Python/bytecodes.c" + #line 2249 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 3342 "Python/generated_cases.c.h" + #line 3244 "Python/generated_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2322 "Python/bytecodes.c" + #line 2254 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -3350,7 +3252,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 3354 "Python/generated_cases.c.h" + #line 3256 "Python/generated_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; DISPATCH(); @@ -3359,10 +3261,10 @@ TARGET(MATCH_MAPPING) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2332 "Python/bytecodes.c" + #line 2264 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 3366 "Python/generated_cases.c.h" + #line 3268 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3371,10 +3273,10 @@ TARGET(MATCH_SEQUENCE) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2337 "Python/bytecodes.c" + #line 2269 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 3378 "Python/generated_cases.c.h" + #line 3280 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3384,11 +3286,11 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2342 "Python/bytecodes.c" + #line 2274 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 3392 "Python/generated_cases.c.h" + #line 3294 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; DISPATCH(); @@ -3397,14 +3299,14 @@ TARGET(GET_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2348 "Python/bytecodes.c" + #line 2280 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 3404 "Python/generated_cases.c.h" + #line 3306 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2351 "Python/bytecodes.c" + #line 2283 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 3408 "Python/generated_cases.c.h" + #line 3310 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3412,7 +3314,7 @@ TARGET(GET_YIELD_FROM_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2355 "Python/bytecodes.c" + #line 2287 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -3435,11 +3337,11 @@ if (iter == NULL) { goto error; } - #line 3439 "Python/generated_cases.c.h" + #line 3341 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2378 "Python/bytecodes.c" + #line 2310 "Python/bytecodes.c" } - #line 3443 "Python/generated_cases.c.h" + #line 3345 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3449,7 +3351,7 @@ static_assert(INLINE_CACHE_ENTRIES_FOR_ITER == 1, "incorrect cache size"); PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2396 "Python/bytecodes.c" + #line 2328 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -3481,7 +3383,7 @@ DISPATCH(); } // Common case: no jump, leave it to the code generator - #line 3485 "Python/generated_cases.c.h" + #line 3387 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3489,7 +3391,7 @@ } TARGET(INSTRUMENTED_FOR_ITER) { - #line 2430 "Python/bytecodes.c" + #line 2362 "Python/bytecodes.c" _Py_CODEUNIT *here = next_instr-1; _Py_CODEUNIT *target; PyObject *iter = TOP(); @@ -3515,14 +3417,14 @@ target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER + oparg + 1; } INSTRUMENTED_JUMP(here, target, PY_MONITORING_EVENT_BRANCH); - #line 3519 "Python/generated_cases.c.h" + #line 3421 "Python/generated_cases.c.h" DISPATCH(); } TARGET(FOR_ITER_LIST) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2458 "Python/bytecodes.c" + #line 2390 "Python/bytecodes.c" DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit); @@ -3543,7 +3445,7 @@ DISPATCH(); end_for_iter_list: // Common case: no jump, leave it to the code generator - #line 3547 "Python/generated_cases.c.h" + #line 3449 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3553,7 +3455,7 @@ TARGET(FOR_ITER_TUPLE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2481 "Python/bytecodes.c" + #line 2413 "Python/bytecodes.c" _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3574,7 +3476,7 @@ DISPATCH(); end_for_iter_tuple: // Common case: no jump, leave it to the code generator - #line 3578 "Python/generated_cases.c.h" + #line 3480 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3584,7 +3486,7 @@ TARGET(FOR_ITER_RANGE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2504 "Python/bytecodes.c" + #line 2436 "Python/bytecodes.c" _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3603,7 +3505,7 @@ if (next == NULL) { goto error; } - #line 3607 "Python/generated_cases.c.h" + #line 3509 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3612,7 +3514,7 @@ TARGET(FOR_ITER_GEN) { PyObject *iter = stack_pointer[-1]; - #line 2525 "Python/bytecodes.c" + #line 2457 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER); @@ -3628,14 +3530,14 @@ assert(next_instr[oparg].op.code == END_FOR || next_instr[oparg].op.code == INSTRUMENTED_END_FOR); DISPATCH_INLINED(gen_frame); - #line 3632 "Python/generated_cases.c.h" + #line 3534 "Python/generated_cases.c.h" } TARGET(BEFORE_ASYNC_WITH) { PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2543 "Python/bytecodes.c" + #line 2475 "Python/bytecodes.c" PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { @@ -3658,16 +3560,16 @@ Py_DECREF(enter); goto error; } - #line 3662 "Python/generated_cases.c.h" + #line 3564 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2566 "Python/bytecodes.c" + #line 2498 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3671 "Python/generated_cases.c.h" + #line 3573 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3678,7 +3580,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2575 "Python/bytecodes.c" + #line 2507 "Python/bytecodes.c" /* pop the context manager, push its __exit__ and the * value returned from calling its __enter__ */ @@ -3704,16 +3606,16 @@ Py_DECREF(enter); goto error; } - #line 3708 "Python/generated_cases.c.h" + #line 3610 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2601 "Python/bytecodes.c" + #line 2533 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3717 "Python/generated_cases.c.h" + #line 3619 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3725,7 +3627,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2610 "Python/bytecodes.c" + #line 2542 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -3746,7 +3648,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 3750 "Python/generated_cases.c.h" + #line 3652 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3755,7 +3657,7 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2649 "Python/bytecodes.c" + #line 2581 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -3765,7 +3667,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 3769 "Python/generated_cases.c.h" + #line 3671 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -3779,7 +3681,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2661 "Python/bytecodes.c" + #line 2593 "Python/bytecodes.c" assert(oparg & 1); /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); @@ -3796,7 +3698,7 @@ res2 = Py_NewRef(descr); assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; - #line 3800 "Python/generated_cases.c.h" + #line 3702 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3810,7 +3712,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2680 "Python/bytecodes.c" + #line 2612 "Python/bytecodes.c" assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -3820,7 +3722,7 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - #line 3824 "Python/generated_cases.c.h" + #line 3726 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3835,7 +3737,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2692 "Python/bytecodes.c" + #line 2624 "Python/bytecodes.c" assert((oparg & 1) == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3848,11 +3750,11 @@ keys_version, LOAD_ATTR); STAT_INC(LOAD_ATTR, hit); assert(descr != NULL); - #line 3852 "Python/generated_cases.c.h" + #line 3754 "Python/generated_cases.c.h" Py_DECREF(self); - #line 2705 "Python/bytecodes.c" + #line 2637 "Python/bytecodes.c" res = Py_NewRef(descr); - #line 3856 "Python/generated_cases.c.h" + #line 3758 "Python/generated_cases.c.h" STACK_GROW((0 ? 1 : 0)); stack_pointer[-1] = res; if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } @@ -3866,7 +3768,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2709 "Python/bytecodes.c" + #line 2641 "Python/bytecodes.c" assert((oparg & 1) == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3874,11 +3776,11 @@ assert(self_cls->tp_dictoffset == 0); STAT_INC(LOAD_ATTR, hit); assert(descr != NULL); - #line 3878 "Python/generated_cases.c.h" + #line 3780 "Python/generated_cases.c.h" Py_DECREF(self); - #line 2717 "Python/bytecodes.c" + #line 2649 "Python/bytecodes.c" res = Py_NewRef(descr); - #line 3882 "Python/generated_cases.c.h" + #line 3784 "Python/generated_cases.c.h" STACK_GROW((0 ? 1 : 0)); stack_pointer[-1] = res; if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } @@ -3892,7 +3794,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2721 "Python/bytecodes.c" + #line 2653 "Python/bytecodes.c" assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -3906,7 +3808,7 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - #line 3910 "Python/generated_cases.c.h" + #line 3812 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3915,16 +3817,16 @@ } TARGET(KW_NAMES) { - #line 2737 "Python/bytecodes.c" + #line 2669 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(FRAME_CO_CONSTS)); kwnames = GETITEM(FRAME_CO_CONSTS, oparg); - #line 3923 "Python/generated_cases.c.h" + #line 3825 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_CALL) { - #line 2743 "Python/bytecodes.c" + #line 2675 "Python/bytecodes.c" int is_meth = PEEK(oparg+2) != NULL; int total_args = oparg + is_meth; PyObject *function = PEEK(total_args + 1); @@ -3937,7 +3839,7 @@ _PyCallCache *cache = (_PyCallCache *)next_instr; INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(CALL); - #line 3941 "Python/generated_cases.c.h" + #line 3843 "Python/generated_cases.c.h" } TARGET(CALL) { @@ -3947,7 +3849,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2789 "Python/bytecodes.c" + #line 2721 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4029,7 +3931,7 @@ Py_DECREF(args[i]); } if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4033 "Python/generated_cases.c.h" + #line 3935 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4041,7 +3943,7 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2877 "Python/bytecodes.c" + #line 2809 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit); @@ -4051,7 +3953,7 @@ PEEK(oparg + 2) = Py_NewRef(meth); // method Py_DECREF(callable); GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS); - #line 4055 "Python/generated_cases.c.h" + #line 3957 "Python/generated_cases.c.h" } TARGET(CALL_PY_EXACT_ARGS) { @@ -4060,7 +3962,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2889 "Python/bytecodes.c" + #line 2821 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4086,7 +3988,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4090 "Python/generated_cases.c.h" + #line 3992 "Python/generated_cases.c.h" } TARGET(CALL_PY_WITH_DEFAULTS) { @@ -4094,7 +3996,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2917 "Python/bytecodes.c" + #line 2849 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4130,7 +4032,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4134 "Python/generated_cases.c.h" + #line 4036 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_TYPE_1) { @@ -4138,7 +4040,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2955 "Python/bytecodes.c" + #line 2887 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4148,7 +4050,7 @@ res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(obj); Py_DECREF(&PyType_Type); // I.e., callable - #line 4152 "Python/generated_cases.c.h" + #line 4054 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4161,7 +4063,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2967 "Python/bytecodes.c" + #line 2899 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4172,7 +4074,7 @@ Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); // I.e., callable if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4176 "Python/generated_cases.c.h" + #line 4078 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4186,7 +4088,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2981 "Python/bytecodes.c" + #line 2913 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4197,7 +4099,7 @@ Py_DECREF(arg); Py_DECREF(&PyTuple_Type); // I.e., tuple if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4201 "Python/generated_cases.c.h" + #line 4103 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4210,7 +4112,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; - #line 2995 "Python/bytecodes.c" + #line 2927 "Python/bytecodes.c" /* This instruction does the following: * 1. Creates the object (by calling ``object.__new__``) * 2. Pushes a shim frame to the frame stack (to cleanup after ``__init__``) @@ -4261,12 +4163,12 @@ * as it will be checked after start_frame */ tstate->py_recursion_remaining--; goto start_frame; - #line 4265 "Python/generated_cases.c.h" + #line 4167 "Python/generated_cases.c.h" } TARGET(EXIT_INIT_CHECK) { PyObject *should_be_none = stack_pointer[-1]; - #line 3048 "Python/bytecodes.c" + #line 2980 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -4274,7 +4176,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 4278 "Python/generated_cases.c.h" + #line 4180 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -4284,7 +4186,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3058 "Python/bytecodes.c" + #line 2990 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4306,7 +4208,7 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4310 "Python/generated_cases.c.h" + #line 4212 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4320,7 +4222,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3083 "Python/bytecodes.c" + #line 3015 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4348,7 +4250,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4352 "Python/generated_cases.c.h" + #line 4254 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4362,7 +4264,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3114 "Python/bytecodes.c" + #line 3046 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4394,7 +4296,7 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 4398 "Python/generated_cases.c.h" + #line 4300 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4408,7 +4310,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3149 "Python/bytecodes.c" + #line 3081 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -4440,7 +4342,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4444 "Python/generated_cases.c.h" + #line 4346 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4454,7 +4356,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3184 "Python/bytecodes.c" + #line 3116 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -4479,7 +4381,7 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4483 "Python/generated_cases.c.h" + #line 4385 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4492,7 +4394,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3211 "Python/bytecodes.c" + #line 3143 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -4519,7 +4421,7 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4523 "Python/generated_cases.c.h" + #line 4425 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4531,7 +4433,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 3241 "Python/bytecodes.c" + #line 3173 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -4549,14 +4451,14 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 4553 "Python/generated_cases.c.h" + #line 4455 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3261 "Python/bytecodes.c" + #line 3193 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4587,7 +4489,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4591 "Python/generated_cases.c.h" + #line 4493 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4600,7 +4502,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3295 "Python/bytecodes.c" + #line 3227 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4629,7 +4531,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4633 "Python/generated_cases.c.h" + #line 4535 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4642,7 +4544,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3327 "Python/bytecodes.c" + #line 3259 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -4671,7 +4573,7 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4675 "Python/generated_cases.c.h" + #line 4577 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4684,7 +4586,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3359 "Python/bytecodes.c" + #line 3291 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4712,7 +4614,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4716 "Python/generated_cases.c.h" + #line 4618 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4722,9 +4624,9 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { - #line 3390 "Python/bytecodes.c" + #line 3322 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); - #line 4728 "Python/generated_cases.c.h" + #line 4630 "Python/generated_cases.c.h" } TARGET(CALL_FUNCTION_EX) { @@ -4733,7 +4635,7 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 3394 "Python/bytecodes.c" + #line 3326 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs)); @@ -4795,14 +4697,14 @@ } result = PyObject_Call(func, callargs, kwargs); } - #line 4799 "Python/generated_cases.c.h" + #line 4701 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 3456 "Python/bytecodes.c" + #line 3388 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4806 "Python/generated_cases.c.h" + #line 4708 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4813,7 +4715,7 @@ TARGET(MAKE_FUNCTION) { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3462 "Python/bytecodes.c" + #line 3394 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4825,7 +4727,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4829 "Python/generated_cases.c.h" + #line 4731 "Python/generated_cases.c.h" stack_pointer[-1] = func; DISPATCH(); } @@ -4833,7 +4735,7 @@ TARGET(SET_FUNCTION_ATTRIBUTE) { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3476 "Python/bytecodes.c" + #line 3408 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -4858,14 +4760,14 @@ default: Py_UNREACHABLE(); } - #line 4862 "Python/generated_cases.c.h" + #line 4764 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 3503 "Python/bytecodes.c" + #line 3435 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4886,7 +4788,7 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4890 "Python/generated_cases.c.h" + #line 4792 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4894,15 +4796,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3526 "Python/bytecodes.c" + #line 3458 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4900 "Python/generated_cases.c.h" + #line 4802 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3528 "Python/bytecodes.c" + #line 3460 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4906 "Python/generated_cases.c.h" + #line 4808 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4912,14 +4814,14 @@ TARGET(CONVERT_VALUE) { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3532 "Python/bytecodes.c" + #line 3464 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 4923 "Python/generated_cases.c.h" + #line 4825 "Python/generated_cases.c.h" stack_pointer[-1] = result; DISPATCH(); } @@ -4927,7 +4829,7 @@ TARGET(FORMAT_SIMPLE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3541 "Python/bytecodes.c" + #line 3473 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -4938,7 +4840,7 @@ else { res = value; } - #line 4942 "Python/generated_cases.c.h" + #line 4844 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -4947,12 +4849,12 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3554 "Python/bytecodes.c" + #line 3486 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 4956 "Python/generated_cases.c.h" + #line 4858 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -4961,10 +4863,10 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3561 "Python/bytecodes.c" + #line 3493 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4968 "Python/generated_cases.c.h" + #line 4870 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4976,7 +4878,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3566 "Python/bytecodes.c" + #line 3498 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -4991,12 +4893,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4995 "Python/generated_cases.c.h" + #line 4897 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3581 "Python/bytecodes.c" + #line 3513 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 5000 "Python/generated_cases.c.h" + #line 4902 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -5006,16 +4908,16 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3586 "Python/bytecodes.c" + #line 3518 "Python/bytecodes.c" assert(oparg >= 2); - #line 5012 "Python/generated_cases.c.h" + #line 4914 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } TARGET(INSTRUMENTED_INSTRUCTION) { - #line 3590 "Python/bytecodes.c" + #line 3522 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error; @@ -5027,48 +4929,48 @@ assert(next_opcode > 0 && next_opcode < 256); opcode = next_opcode; DISPATCH_GOTO(); - #line 5031 "Python/generated_cases.c.h" + #line 4933 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_JUMP_FORWARD) { - #line 3604 "Python/bytecodes.c" + #line 3536 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); - #line 5037 "Python/generated_cases.c.h" + #line 4939 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) { - #line 3608 "Python/bytecodes.c" + #line 3540 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); - #line 5045 "Python/generated_cases.c.h" + #line 4947 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { - #line 3613 "Python/bytecodes.c" + #line 3545 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsTrue(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5056 "Python/generated_cases.c.h" + #line 4958 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { - #line 3621 "Python/bytecodes.c" + #line 3553 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsFalse(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5067 "Python/generated_cases.c.h" + #line 4969 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { - #line 3629 "Python/bytecodes.c" + #line 3561 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -5080,12 +4982,12 @@ offset = 0; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5084 "Python/generated_cases.c.h" + #line 4986 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { - #line 3643 "Python/bytecodes.c" + #line 3575 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -5097,30 +4999,30 @@ offset = oparg; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5101 "Python/generated_cases.c.h" + #line 5003 "Python/generated_cases.c.h" DISPATCH(); } TARGET(EXTENDED_ARG) { - #line 3657 "Python/bytecodes.c" + #line 3589 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 5112 "Python/generated_cases.c.h" + #line 5014 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3665 "Python/bytecodes.c" + #line 3597 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); - #line 5119 "Python/generated_cases.c.h" + #line 5021 "Python/generated_cases.c.h" } TARGET(RESERVED) { - #line 3670 "Python/bytecodes.c" + #line 3602 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 5126 "Python/generated_cases.c.h" + #line 5028 "Python/generated_cases.c.h" } diff --git a/Python/import.c b/Python/import.c index fcc0346e7f15a..7ce89cdf9a91a 100644 --- a/Python/import.c +++ b/Python/import.c @@ -249,16 +249,7 @@ import_get_module(PyThreadState *tstate, PyObject *name) PyObject *m; Py_INCREF(modules); - if (PyDict_CheckExact(modules)) { - m = PyDict_GetItemWithError(modules, name); /* borrowed */ - Py_XINCREF(m); - } - else { - m = PyObject_GetItem(modules, name); - if (m == NULL && _PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - _PyErr_Clear(tstate); - } - } + (void)PyMapping_GetOptionalItem(modules, name, &m); Py_DECREF(modules); return m; } @@ -322,18 +313,7 @@ import_add_module(PyThreadState *tstate, PyObject *name) } PyObject *m; - if (PyDict_CheckExact(modules)) { - m = Py_XNewRef(PyDict_GetItemWithError(modules, name)); - } - else { - m = PyObject_GetItem(modules, name); - // For backward-compatibility we copy the behavior - // of PyDict_GetItemWithError(). - if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - _PyErr_Clear(tstate); - } - } - if (_PyErr_Occurred(tstate)) { + if (PyMapping_GetOptionalItem(modules, name, &m) < 0) { return NULL; } if (m != NULL && PyModule_Check(m)) { From webhook-mailer at python.org Tue Jul 11 16:09:08 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 11 Jul 2023 20:09:08 -0000 Subject: [Python-checkins] gh-96165: Clarify omitting the FROM clause in SQLite queries (#106513) Message-ID: https://github.com/python/cpython/commit/fc7ff1af457e27b7d9752600b3436641be90f598 commit: fc7ff1af457e27b7d9752600b3436641be90f598 branch: main author: Mariusz Felisiak committer: erlend-aasland date: 2023-07-11T22:09:04+02:00 summary: gh-96165: Clarify omitting the FROM clause in SQLite queries (#106513) files: M Doc/library/sqlite3.rst diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index bc69387fca8a9..f6a8714519f59 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -2519,6 +2519,13 @@ Queries now return :class:`!Row` objects: >>> row["RADIUS"] # Column names are case-insensitive. 6378 +.. note:: + + The ``FROM`` clause can be omitted in the ``SELECT`` statement, as in the + above example. In such cases, SQLite returns a single row with columns + defined by expressions, e.g. literals, with the given aliases + ``expr AS alias``. + You can create a custom :attr:`~Cursor.row_factory` that returns each row as a :class:`dict`, with column names mapped to values: From webhook-mailer at python.org Tue Jul 11 16:18:23 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 11 Jul 2023 20:18:23 -0000 Subject: [Python-checkins] gh-96165: Clarify passing ":memory:" in sqlite3.connect() (#106451) Message-ID: https://github.com/python/cpython/commit/f520804b039df0d87fb9df6f1fed2a9bc9df8d61 commit: f520804b039df0d87fb9df6f1fed2a9bc9df8d61 branch: main author: Mariusz Felisiak committer: erlend-aasland date: 2023-07-11T22:18:19+02:00 summary: gh-96165: Clarify passing ":memory:" in sqlite3.connect() (#106451) files: M Doc/library/sqlite3.rst diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index f6a8714519f59..356888d64b887 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -266,8 +266,9 @@ Module functions :param database: The path to the database file to be opened. - Pass ``":memory:"`` to open a connection to a database that is - in RAM instead of on disk. + You can pass ``":memory:"`` to create an `SQLite database existing only + in memory `_, and open a connection + to it. :type database: :term:`path-like object` :param float timeout: From webhook-mailer at python.org Tue Jul 11 16:25:45 2023 From: webhook-mailer at python.org (iritkatriel) Date: Tue, 11 Jul 2023 20:25:45 -0000 Subject: [Python-checkins] gh-104584: readability improvements in optimizer.c (#106641) Message-ID: https://github.com/python/cpython/commit/3590c45a3d564b3182ae21d899bae81c49d685a2 commit: 3590c45a3d564b3182ae21d899bae81c49d685a2 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-11T21:25:41+01:00 summary: gh-104584: readability improvements in optimizer.c (#106641) files: M Python/optimizer.c diff --git a/Python/optimizer.c b/Python/optimizer.c index 8d4162b44c2f3..e2b92e68d39e1 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -3,6 +3,7 @@ #include "pycore_interp.h" #include "pycore_opcode.h" #include "opcode_metadata.h" +#include "pycore_opcode_utils.h" #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_uops.h" #include "cpython/optimizer.h" @@ -10,6 +11,8 @@ #include #include +#define MAX_EXECUTORS_SIZE 256 + static bool has_space_for_executor(PyCodeObject *code, _Py_CODEUNIT *instr) { @@ -19,7 +22,7 @@ has_space_for_executor(PyCodeObject *code, _Py_CODEUNIT *instr) if (code->co_executors == NULL) { return true; } - return code->co_executors->size < 256; + return code->co_executors->size < MAX_EXECUTORS_SIZE; } static int32_t @@ -34,7 +37,7 @@ get_index_for_executor(PyCodeObject *code, _Py_CODEUNIT *instr) if (old != NULL) { size = old->size; capacity = old->capacity; - assert(size < 256); + assert(size < MAX_EXECUTORS_SIZE); } assert(size <= capacity); if (size == capacity) { @@ -74,7 +77,7 @@ insert_executor(PyCodeObject *code, _Py_CODEUNIT *instr, int index, _PyExecutorO executor->vm_data.opcode = instr->op.code; executor->vm_data.oparg = instr->op.arg; code->co_executors->executors[index] = executor; - assert(index < 256); + assert(index < MAX_EXECUTORS_SIZE); instr->op.code = ENTER_EXECUTOR; instr->op.arg = index; code->co_executors->size++; @@ -307,7 +310,7 @@ uop_dealloc(_PyUOpExecutorObject *self) { static const char * uop_name(int index) { - if (index < 256) { + if (index <= MAX_REAL_OPCODE) { return _PyOpcode_OpName[index]; } return _PyOpcode_uop_name[index]; @@ -391,17 +394,20 @@ translate_bytecode_to_trace( #define ADD_TO_TRACE(OPCODE, OPERAND) \ DPRINTF(2, \ " ADD_TO_TRACE(%s, %" PRIu64 ")\n", \ - (OPCODE) < 256 ? _PyOpcode_OpName[(OPCODE)] : _PyOpcode_uop_name[(OPCODE)], \ + uop_name(OPCODE), \ (uint64_t)(OPERAND)); \ assert(trace_length < max_length); \ trace[trace_length].opcode = (OPCODE); \ trace[trace_length].operand = (OPERAND); \ trace_length++; +#define INSTR_IP(INSTR, CODE) \ + ((long)((INSTR) - ((_Py_CODEUNIT *)(CODE)->co_code_adaptive))) + #define ADD_TO_STUB(INDEX, OPCODE, OPERAND) \ DPRINTF(2, " ADD_TO_STUB(%d, %s, %" PRIu64 ")\n", \ (INDEX), \ - (OPCODE) < 256 ? _PyOpcode_OpName[(OPCODE)] : _PyOpcode_uop_name[(OPCODE)], \ + uop_name(OPCODE), \ (uint64_t)(OPERAND)); \ trace[(INDEX)].opcode = (OPCODE); \ trace[(INDEX)].operand = (OPERAND); @@ -411,10 +417,10 @@ translate_bytecode_to_trace( PyUnicode_AsUTF8(code->co_qualname), PyUnicode_AsUTF8(code->co_filename), code->co_firstlineno, - 2 * (long)(initial_instr - (_Py_CODEUNIT *)code->co_code_adaptive)); + 2 * INSTR_IP(initial_instr, code)); for (;;) { - ADD_TO_TRACE(SAVE_IP, instr - (_Py_CODEUNIT *)code->co_code_adaptive); + ADD_TO_TRACE(SAVE_IP, INSTR_IP(instr, code)); int opcode = instr->op.code; int oparg = instr->op.arg; int extras = 0; @@ -448,8 +454,7 @@ translate_bytecode_to_trace( int uopcode = opcode == POP_JUMP_IF_TRUE ? _POP_JUMP_IF_TRUE : _POP_JUMP_IF_FALSE; ADD_TO_TRACE(uopcode, max_length); - ADD_TO_STUB(max_length, SAVE_IP, - target_instr - (_Py_CODEUNIT *)code->co_code_adaptive); + ADD_TO_STUB(max_length, SAVE_IP, INSTR_IP(target_instr, code)); ADD_TO_STUB(max_length + 1, EXIT_TRACE, 0); break; } @@ -474,9 +479,7 @@ translate_bytecode_to_trace( // Reserve space for nuops (+ SAVE_IP + EXIT_TRACE) int nuops = expansion->nuops; if (trace_length + nuops + 2 > max_length) { - DPRINTF(1, - "Ran out of space for %s\n", - opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode]); + DPRINTF(1, "Ran out of space for %s\n", uop_name(opcode)); goto done; } for (int i = 0; i < nuops; i++) { @@ -522,9 +525,7 @@ translate_bytecode_to_trace( } break; } - DPRINTF(2, - "Unsupported opcode %s\n", - opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode]); + DPRINTF(2, "Unsupported opcode %s\n", uop_name(opcode)); goto done; // Break out of loop } } @@ -542,7 +543,7 @@ translate_bytecode_to_trace( PyUnicode_AsUTF8(code->co_qualname), PyUnicode_AsUTF8(code->co_filename), code->co_firstlineno, - 2 * (long)(initial_instr - (_Py_CODEUNIT *)code->co_code_adaptive), + 2 * INSTR_IP(initial_instr, code), trace_length); if (max_length < buffer_size && trace_length < max_length) { // Move the stubs back to be immediately after the main trace @@ -576,7 +577,7 @@ translate_bytecode_to_trace( PyUnicode_AsUTF8(code->co_qualname), PyUnicode_AsUTF8(code->co_filename), code->co_firstlineno, - 2 * (long)(initial_instr - (_Py_CODEUNIT *)code->co_code_adaptive)); + 2 * INSTR_IP(initial_instr, code)); } return 0; From webhook-mailer at python.org Tue Jul 11 16:30:56 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 11 Jul 2023 20:30:56 -0000 Subject: [Python-checkins] [3.11] gh-96165: Clarify omitting the FROM clause in SQLite queries (GH-106513) (#106646) Message-ID: https://github.com/python/cpython/commit/6d8662381d73982f53d3cc71bf435aa82def8eab commit: 6d8662381d73982f53d3cc71bf435aa82def8eab branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-11T22:30:52+02:00 summary: [3.11] gh-96165: Clarify omitting the FROM clause in SQLite queries (GH-106513) (#106646) (cherry picked from commit fc7ff1af457e27b7d9752600b3436641be90f598) Co-authored-by: Mariusz Felisiak files: M Doc/library/sqlite3.rst diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index c99c01b0e5c79..a50be741ce8bc 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -2311,6 +2311,13 @@ Queries now return :class:`!Row` objects: >>> row["RADIUS"] # Column names are case-insensitive. 6378 +.. note:: + + The ``FROM`` clause can be omitted in the ``SELECT`` statement, as in the + above example. In such cases, SQLite returns a single row with columns + defined by expressions, e.g. literals, with the given aliases + ``expr AS alias``. + You can create a custom :attr:`~Cursor.row_factory` that returns each row as a :class:`dict`, with column names mapped to values: From webhook-mailer at python.org Tue Jul 11 16:31:20 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 11 Jul 2023 20:31:20 -0000 Subject: [Python-checkins] [3.12] gh-96165: Clarify omitting the FROM clause in SQLite queries (GH-106513) (#106645) Message-ID: https://github.com/python/cpython/commit/bf7e92583dd33aa7e83d5929f2138c5d6957f571 commit: bf7e92583dd33aa7e83d5929f2138c5d6957f571 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-11T22:31:16+02:00 summary: [3.12] gh-96165: Clarify omitting the FROM clause in SQLite queries (GH-106513) (#106645) (cherry picked from commit fc7ff1af457e27b7d9752600b3436641be90f598) Co-authored-by: Mariusz Felisiak files: M Doc/library/sqlite3.rst diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 3ca8ea9011c7e..26f4bfd06f5bb 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -2522,6 +2522,13 @@ Queries now return :class:`!Row` objects: >>> row["RADIUS"] # Column names are case-insensitive. 6378 +.. note:: + + The ``FROM`` clause can be omitted in the ``SELECT`` statement, as in the + above example. In such cases, SQLite returns a single row with columns + defined by expressions, e.g. literals, with the given aliases + ``expr AS alias``. + You can create a custom :attr:`~Cursor.row_factory` that returns each row as a :class:`dict`, with column names mapped to values: From webhook-mailer at python.org Tue Jul 11 16:31:52 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 11 Jul 2023 20:31:52 -0000 Subject: [Python-checkins] [3.12] gh-96165: Clarify passing ":memory:" in sqlite3.connect() (GH-106451) (#106647) Message-ID: https://github.com/python/cpython/commit/e248ba7b72adad794399d7b55116862fa31471fd commit: e248ba7b72adad794399d7b55116862fa31471fd branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-11T22:31:48+02:00 summary: [3.12] gh-96165: Clarify passing ":memory:" in sqlite3.connect() (GH-106451) (#106647) (cherry picked from commit f520804b039df0d87fb9df6f1fed2a9bc9df8d61) Co-authored-by: Mariusz Felisiak files: M Doc/library/sqlite3.rst diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 26f4bfd06f5bb..2e2c28b1e5fe7 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -266,8 +266,9 @@ Module functions :param database: The path to the database file to be opened. - Pass ``":memory:"`` to open a connection to a database that is - in RAM instead of on disk. + You can pass ``":memory:"`` to create an `SQLite database existing only + in memory `_, and open a connection + to it. :type database: :term:`path-like object` :param float timeout: From webhook-mailer at python.org Tue Jul 11 16:32:12 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 11 Jul 2023 20:32:12 -0000 Subject: [Python-checkins] [3.11] gh-96165: Clarify passing ":memory:" in sqlite3.connect() (GH-106451) (#106648) Message-ID: https://github.com/python/cpython/commit/247f0500b88bb881f563792c7db535e91259b90c commit: 247f0500b88bb881f563792c7db535e91259b90c branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-11T22:32:08+02:00 summary: [3.11] gh-96165: Clarify passing ":memory:" in sqlite3.connect() (GH-106451) (#106648) cherry picked from commit f520804b039df0d87fb9df6f1fed2a9bc9df8d61) Co-authored-by: Mariusz Felisiak files: M Doc/library/sqlite3.rst diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index a50be741ce8bc..06f5293fbdbd1 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -265,8 +265,9 @@ Module functions :param database: The path to the database file to be opened. - Pass ``":memory:"`` to open a connection to a database that is - in RAM instead of on disk. + You can pass ``":memory:"`` to create an `SQLite database existing only + in memory `_, and open a connection + to it. :type database: :term:`path-like object` :param float timeout: From webhook-mailer at python.org Tue Jul 11 17:21:18 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Tue, 11 Jul 2023 21:21:18 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Annotate the `Block` class (#106519) Message-ID: https://github.com/python/cpython/commit/d0972c77aa1cd5fe27618e82c10141a2bf157476 commit: d0972c77aa1cd5fe27618e82c10141a2bf157476 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-11T21:21:14Z summary: gh-104050: Argument Clinic: Annotate the `Block` class (#106519) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 19c5f573920cb..12ab633388485 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -759,7 +759,7 @@ def parse_line(self, line: str) -> None: def render( self, clinic: Clinic | None, - signatures: Iterable[Function] + signatures: Iterable[Module | Class | Function] ) -> str: function = None for o in signatures: @@ -1633,6 +1633,7 @@ def create_regex( return re.compile(pattern) + at dc.dataclass(slots=True, repr=False) class Block: r""" Represents a single block of text embedded in @@ -1679,18 +1680,16 @@ class Block: "preindent" would be "____" and "indent" would be "__". """ - def __init__(self, input, dsl_name=None, signatures=None, output=None, indent='', preindent=''): - assert isinstance(input, str) - self.input = input - self.dsl_name = dsl_name - self.signatures = signatures or [] - self.output = output - self.indent = indent - self.preindent = preindent + input: str + dsl_name: str | None = None + signatures: list[Module | Class | Function] = dc.field(default_factory=list) + output: Any = None # TODO: Very dynamic; probably untypeable in its current form? + indent: str = '' + preindent: str = '' - def __repr__(self): + def __repr__(self) -> str: dsl_name = self.dsl_name or "text" - def summarize(s): + def summarize(s: object) -> str: s = repr(s) if len(s) > 30: return s[:26] + "..." + s[0] @@ -4619,10 +4618,13 @@ def state_modulename_name(self, line: str | None) -> None: if not (existing_function.kind == self.kind and existing_function.coexist == self.coexist): fail("'kind' of function and cloned function don't match! (@classmethod/@staticmethod/@coexist)") - self.function = existing_function.copy(name=function_name, full_name=full_name, module=module, cls=cls, c_basename=c_basename, docstring='') - - self.block.signatures.append(self.function) - (cls or module).functions.append(self.function) + function = existing_function.copy( + name=function_name, full_name=full_name, module=module, + cls=cls, c_basename=c_basename, docstring='' + ) + self.function = function + self.block.signatures.append(function) + (cls or module).functions.append(function) self.next(self.state_function_docstring) return From webhook-mailer at python.org Tue Jul 11 18:14:01 2023 From: webhook-mailer at python.org (gvanrossum) Date: Tue, 11 Jul 2023 22:14:01 -0000 Subject: [Python-checkins] gh-106529: Implement JUMP_FORWARD in uops (with test) (#106651) Message-ID: https://github.com/python/cpython/commit/da86db56cb595fdbeda8b57a1ec03b1dd80ad1f0 commit: da86db56cb595fdbeda8b57a1ec03b1dd80ad1f0 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-11T15:13:57-07:00 summary: gh-106529: Implement JUMP_FORWARD in uops (with test) (#106651) Note that this may generate two SAVE_IP uops in a row. Removing unneeded SAVE_IP uops is the optimizer's job. files: M Lib/test/test_capi/test_misc.py M Python/optimizer.c diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 9cbd5506bf6de..9c14a501875b6 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2553,6 +2553,7 @@ def testfunc(n): i = 0 while i < n: i += 1 + opt = _testinternalcapi.get_uop_optimizer() with temporary_optimizer(opt): testfunc(10) @@ -2562,6 +2563,30 @@ def testfunc(n): uops = {opname for opname, _ in ex} self.assertIn("JUMP_TO_TOP", uops) + def test_jump_forward(self): + def testfunc(n): + a = 0 + while a < n: + if a < 0: + a = -a + else: + a = +a + a += 1 + return a + + opt = _testinternalcapi.get_uop_optimizer() + with temporary_optimizer(opt): + testfunc(10) + + ex = get_first_executor(testfunc) + self.assertIsNotNone(ex) + # for i, (opname, oparg) in enumerate(ex): + # print(f"{i:4d}: {opname:<20s} {oparg:4d}") + uops = {opname for opname, _ in ex} + # Since there is no JUMP_FORWARD instruction, + # look for indirect evidence: the += operator + self.assertIn("_BINARY_OP_ADD_INT", uops) + if __name__ == "__main__": unittest.main() diff --git a/Python/optimizer.c b/Python/optimizer.c index e2b92e68d39e1..7fc40e66057d3 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -472,6 +472,13 @@ translate_bytecode_to_trace( goto done; } + case JUMP_FORWARD: + { + // This will emit two SAVE_IP instructions; leave it to the optimizer + instr += oparg; + break; + } + default: { const struct opcode_macro_expansion *expansion = &_PyOpcode_macro_expansion[opcode]; From webhook-mailer at python.org Tue Jul 11 18:22:21 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 11 Jul 2023 22:22:21 -0000 Subject: [Python-checkins] gh-103186: In test_tools.freeze, fetch CONFIG_ARGS from original source directory (#103213) Message-ID: https://github.com/python/cpython/commit/de827322ca47e51d52ff44536a7c3fd44648383a commit: de827322ca47e51d52ff44536a7c3fd44648383a branch: main author: Ijtaba Hussain committer: erlend-aasland date: 2023-07-11T22:22:18Z summary: gh-103186: In test_tools.freeze, fetch CONFIG_ARGS from original source directory (#103213) Fetch CONFIG_ARGS from the original source directory, instead of from the copied source tree. When "make clean" is executed in the copied source tree, the build directory is cleared and the configure argument lookup fails. However, the original source directory still contains this information. files: A Misc/NEWS.d/next/Tools-Demos/2023-04-05-07-19-36.gh-issue-103186.yEozgK.rst M Tools/freeze/test/freeze.py diff --git a/Misc/NEWS.d/next/Tools-Demos/2023-04-05-07-19-36.gh-issue-103186.yEozgK.rst b/Misc/NEWS.d/next/Tools-Demos/2023-04-05-07-19-36.gh-issue-103186.yEozgK.rst new file mode 100644 index 0000000000000..7e28ba6963216 --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2023-04-05-07-19-36.gh-issue-103186.yEozgK.rst @@ -0,0 +1,2 @@ +``freeze`` now fetches ``CONFIG_ARGS`` from the original CPython instance +the Makefile uses to call utility scripts. Patch by Ijtaba Hussain. diff --git a/Tools/freeze/test/freeze.py b/Tools/freeze/test/freeze.py index f6a5adb4519fd..92e97cb261719 100644 --- a/Tools/freeze/test/freeze.py +++ b/Tools/freeze/test/freeze.py @@ -153,7 +153,7 @@ def prepare(script=None, outdir=None): print(f'configuring python in {builddir}...') cmd = [ os.path.join(srcdir, 'configure'), - *shlex.split(get_config_var(srcdir, 'CONFIG_ARGS') or ''), + *shlex.split(get_config_var(SRCDIR, 'CONFIG_ARGS') or ''), ] ensure_opt(cmd, 'cache-file', os.path.join(outdir, 'python-config.cache')) prefix = os.path.join(outdir, 'python-installation') From webhook-mailer at python.org Tue Jul 11 19:08:31 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Tue, 11 Jul 2023 23:08:31 -0000 Subject: [Python-checkins] gh-104683: Argument clinic: Minor readability improvements for `Destination.__init__` (#106652) Message-ID: https://github.com/python/cpython/commit/7ce3ea4906986c2bab784c878d31c57b14ee1945 commit: 7ce3ea4906986c2bab784c878d31c57b14ee1945 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-11T23:08:28Z summary: gh-104683: Argument clinic: Minor readability improvements for `Destination.__init__` (#106652) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 12ab633388485..a0cf50b29c978 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1959,14 +1959,19 @@ def __init__(self, name, type, clinic, *args): self.name = name self.type = type self.clinic = clinic + self.buffers = BufferSeries() + valid_types = ('buffer', 'file', 'suppress') if type not in valid_types: - fail("Invalid destination type " + repr(type) + " for " + name + " , must be " + ', '.join(valid_types)) + fail( + f"Invalid destination type {type!r} for {name}, " + f"must be {', '.join(valid_types)}" + ) extra_arguments = 1 if type == "file" else 0 if len(args) < extra_arguments: - fail("Not enough arguments for destination " + name + " new " + type) + fail(f"Not enough arguments for destination {name} new {type}") if len(args) > extra_arguments: - fail("Too many arguments for destination " + name + " new " + type) + fail(f"Too many arguments for destination {name} new {type}") if type =='file': d = {} filename = clinic.filename @@ -1979,8 +1984,6 @@ def __init__(self, name, type, clinic, *args): d['basename_root'], d['basename_extension'] = os.path.splitext(filename) self.filename = args[0].format_map(d) - self.buffers = BufferSeries() - def __repr__(self): if self.type == 'file': file_repr = " " + repr(self.filename) From webhook-mailer at python.org Tue Jul 11 22:35:27 2023 From: webhook-mailer at python.org (hugovk) Date: Wed, 12 Jul 2023 02:35:27 -0000 Subject: [Python-checkins] Add Plausible for docs metrics (#106644) Message-ID: https://github.com/python/cpython/commit/e8ab0096a583184fe24dfbc39eff70d270c8e6f4 commit: e8ab0096a583184fe24dfbc39eff70d270c8e6f4 branch: main author: Hugo van Kemenade committer: hugovk date: 2023-07-12T05:35:24+03:00 summary: Add Plausible for docs metrics (#106644) files: M Doc/tools/templates/layout.html diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index 9632ad50a51bf..18a49271df5f2 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -26,6 +26,7 @@ {% endblock %} {% block extrahead %} + {% if builder != "htmlhelp" %} {% if pagename == 'whatsnew/changelog' and not embedded %} From webhook-mailer at python.org Tue Jul 11 22:45:27 2023 From: webhook-mailer at python.org (hugovk) Date: Wed, 12 Jul 2023 02:45:27 -0000 Subject: [Python-checkins] [3.11] Add Plausible for docs metrics (GH-106644) (#106662) Message-ID: https://github.com/python/cpython/commit/81a5bcf7ba14fc1f006968b1c9b5065b70b8229b commit: 81a5bcf7ba14fc1f006968b1c9b5065b70b8229b branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-12T05:45:23+03:00 summary: [3.11] Add Plausible for docs metrics (GH-106644) (#106662) Add Plausible for docs metrics (GH-106644) (cherry picked from commit e8ab0096a583184fe24dfbc39eff70d270c8e6f4) Co-authored-by: Hugo van Kemenade files: M Doc/tools/templates/layout.html diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index b91f8138553e6..10cb6fd880fa1 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -26,6 +26,7 @@ {% endblock %} {% block extrahead %} + {% if builder != "htmlhelp" %} {% if pagename == 'whatsnew/changelog' and not embedded %} From webhook-mailer at python.org Tue Jul 11 22:46:15 2023 From: webhook-mailer at python.org (hugovk) Date: Wed, 12 Jul 2023 02:46:15 -0000 Subject: [Python-checkins] [3.12] Add Plausible for docs metrics (GH-106644) (#106661) Message-ID: https://github.com/python/cpython/commit/ae315991431df5172799ef7ccc0202ac7c0841c9 commit: ae315991431df5172799ef7ccc0202ac7c0841c9 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-12T05:46:12+03:00 summary: [3.12] Add Plausible for docs metrics (GH-106644) (#106661) Add Plausible for docs metrics (GH-106644) (cherry picked from commit e8ab0096a583184fe24dfbc39eff70d270c8e6f4) Co-authored-by: Hugo van Kemenade files: M Doc/tools/templates/layout.html diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index b91f8138553e6..10cb6fd880fa1 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -26,6 +26,7 @@ {% endblock %} {% block extrahead %} + {% if builder != "htmlhelp" %} {% if pagename == 'whatsnew/changelog' and not embedded %} From webhook-mailer at python.org Wed Jul 12 01:57:14 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 12 Jul 2023 05:57:14 -0000 Subject: [Python-checkins] gh-106521: Remove _PyObject_LookupAttr() function (GH-106642) Message-ID: https://github.com/python/cpython/commit/be1b968dc1e63c3c68e161ddc5a05eb064833440 commit: be1b968dc1e63c3c68e161ddc5a05eb064833440 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-12T08:57:10+03:00 summary: gh-106521: Remove _PyObject_LookupAttr() function (GH-106642) files: M Include/cpython/object.h M Modules/_abc.c M Modules/_asynciomodule.c M Modules/_csv.c M Modules/_ctypes/_ctypes.c M Modules/_ctypes/callproc.c M Modules/_ctypes/stgdict.c M Modules/_datetimemodule.c M Modules/_elementtree.c M Modules/_io/bufferedio.c M Modules/_io/fileio.c M Modules/_io/iobase.c M Modules/_io/textio.c M Modules/_pickle.c M Modules/_sqlite/microprotocols.c M Modules/_threadmodule.c M Modules/arraymodule.c M Modules/itertoolsmodule.c M Modules/pyexpat.c M Objects/abstract.c M Objects/call.c M Objects/classobject.c M Objects/descrobject.c M Objects/dictobject.c M Objects/exceptions.c M Objects/fileobject.c M Objects/funcobject.c M Objects/genericaliasobject.c M Objects/genobject.c M Objects/moduleobject.c M Objects/odictobject.c M Objects/typeobject.c M Objects/unionobject.c M Parser/asdl_c.py M Python/Python-ast.c M Python/_warnings.c M Python/bltinmodule.c M Python/ceval.c M Python/codecs.c M Python/errors.c M Python/import.c M Python/intrinsics.c M Python/pythonrun.c M Python/suggestions.c M Python/sysmodule.c diff --git a/Include/cpython/object.h b/Include/cpython/object.h index ba30c567c40eb..cd421b4f7e0d4 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -294,7 +294,6 @@ PyAPI_FUNC(int) _PyObject_IsFreed(PyObject *); PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *); PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, _Py_Identifier *); PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, _Py_Identifier *, PyObject *); -#define _PyObject_LookupAttr PyObject_GetOptionalAttr PyAPI_FUNC(int) _PyObject_GetMethod(PyObject *obj, PyObject *name, PyObject **method); diff --git a/Modules/_abc.c b/Modules/_abc.c index 93c0a93c442cd..8a3aa9cb88880 100644 --- a/Modules/_abc.c +++ b/Modules/_abc.c @@ -361,7 +361,7 @@ compute_abstract_methods(PyObject *self) PyObject *item = PyTuple_GET_ITEM(bases, pos); // borrowed PyObject *base_abstracts, *iter; - if (_PyObject_LookupAttr(item, &_Py_ID(__abstractmethods__), + if (PyObject_GetOptionalAttr(item, &_Py_ID(__abstractmethods__), &base_abstracts) < 0) { goto error; } @@ -375,7 +375,7 @@ compute_abstract_methods(PyObject *self) Py_DECREF(base_abstracts); PyObject *key, *value; while ((key = PyIter_Next(iter))) { - if (_PyObject_LookupAttr(self, key, &value) < 0) { + if (PyObject_GetOptionalAttr(self, key, &value) < 0) { Py_DECREF(key); Py_DECREF(iter); goto error; diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 3b0550279fb49..ef9f7f8902e09 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -248,7 +248,7 @@ get_future_loop(asyncio_state *state, PyObject *fut) return Py_NewRef(loop); } - if (_PyObject_LookupAttr(fut, &_Py_ID(get_loop), &getloop) < 0) { + if (PyObject_GetOptionalAttr(fut, &_Py_ID(get_loop), &getloop) < 0) { return NULL; } if (getloop != NULL) { @@ -2966,7 +2966,7 @@ task_step_handle_result_impl(asyncio_state *state, TaskObj *task, PyObject *resu } /* Check if `result` is a Future-compatible object */ - if (_PyObject_LookupAttr(result, &_Py_ID(_asyncio_future_blocking), &o) < 0) { + if (PyObject_GetOptionalAttr(result, &_Py_ID(_asyncio_future_blocking), &o) < 0) { goto fail; } if (o != NULL && o != Py_None) { diff --git a/Modules/_csv.c b/Modules/_csv.c index 5b501af449f18..c36d9805a1284 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -1450,7 +1450,7 @@ csv_writer(PyObject *module, PyObject *args, PyObject *keyword_args) Py_DECREF(self); return NULL; } - if (_PyObject_LookupAttr(output_file, + if (PyObject_GetOptionalAttr(output_file, module_state->str_write, &self->write) < 0) { Py_DECREF(self); diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index a9d8a2b9cb4ad..7624c15ac522d 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -851,7 +851,7 @@ CDataType_from_param(PyObject *type, PyObject *value) return NULL; } - if (_PyObject_LookupAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { + if (PyObject_GetOptionalAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { return NULL; } if (as_parameter) { @@ -1495,7 +1495,7 @@ PyCArrayType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) stgdict = NULL; type_attr = NULL; - if (_PyObject_LookupAttr((PyObject *)result, &_Py_ID(_length_), &length_attr) < 0) { + if (PyObject_GetOptionalAttr((PyObject *)result, &_Py_ID(_length_), &length_attr) < 0) { goto error; } if (!length_attr) { @@ -1528,7 +1528,7 @@ PyCArrayType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) goto error; } - if (_PyObject_LookupAttr((PyObject *)result, &_Py_ID(_type_), &type_attr) < 0) { + if (PyObject_GetOptionalAttr((PyObject *)result, &_Py_ID(_type_), &type_attr) < 0) { goto error; } if (!type_attr) { @@ -1720,7 +1720,7 @@ c_wchar_p_from_param(PyObject *type, PyObject *value) } } - if (_PyObject_LookupAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { + if (PyObject_GetOptionalAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { return NULL; } if (as_parameter) { @@ -1784,7 +1784,7 @@ c_char_p_from_param(PyObject *type, PyObject *value) } } - if (_PyObject_LookupAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { + if (PyObject_GetOptionalAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { return NULL; } if (as_parameter) { @@ -1919,7 +1919,7 @@ c_void_p_from_param(PyObject *type, PyObject *value) } } - if (_PyObject_LookupAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { + if (PyObject_GetOptionalAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { return NULL; } if (as_parameter) { @@ -2054,7 +2054,7 @@ PyCSimpleType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) if (result == NULL) return NULL; - if (_PyObject_LookupAttr((PyObject *)result, &_Py_ID(_type_), &proto) < 0) { + if (PyObject_GetOptionalAttr((PyObject *)result, &_Py_ID(_type_), &proto) < 0) { return NULL; } if (!proto) { @@ -2266,7 +2266,7 @@ PyCSimpleType_from_param(PyObject *type, PyObject *value) PyObject *exc = PyErr_GetRaisedException(); Py_DECREF(parg); - if (_PyObject_LookupAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { + if (PyObject_GetOptionalAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) { Py_XDECREF(exc); return NULL; } @@ -2429,7 +2429,7 @@ converters_from_argtypes(PyObject *ob) } */ - if (_PyObject_LookupAttr(tp, &_Py_ID(from_param), &cnv) <= 0) { + if (PyObject_GetOptionalAttr(tp, &_Py_ID(from_param), &cnv) <= 0) { Py_DECREF(converters); Py_DECREF(ob); if (!PyErr_Occurred()) { @@ -2489,7 +2489,7 @@ make_funcptrtype_dict(StgDictObject *stgdict) return -1; } stgdict->restype = Py_NewRef(ob); - if (_PyObject_LookupAttr(ob, &_Py_ID(_check_retval_), + if (PyObject_GetOptionalAttr(ob, &_Py_ID(_check_retval_), &stgdict->checker) < 0) { return -1; @@ -3275,7 +3275,7 @@ PyCFuncPtr_set_restype(PyCFuncPtrObject *self, PyObject *ob, void *Py_UNUSED(ign "restype must be a type, a callable, or None"); return -1; } - if (_PyObject_LookupAttr(ob, &_Py_ID(_check_retval_), &checker) < 0) { + if (PyObject_GetOptionalAttr(ob, &_Py_ID(_check_retval_), &checker) < 0) { return -1; } oldchecker = self->checker; diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index d2fe525dd4d39..b3831ae7119a5 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -727,7 +727,7 @@ static int ConvParam(PyObject *obj, Py_ssize_t index, struct argument *pa) { PyObject *arg; - if (_PyObject_LookupAttr(obj, &_Py_ID(_as_parameter_), &arg) < 0) { + if (PyObject_GetOptionalAttr(obj, &_Py_ID(_as_parameter_), &arg) < 0) { return -1; } /* Which types should we exactly allow here? diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c index b1b2bac1455e6..3348ebd6593d2 100644 --- a/Modules/_ctypes/stgdict.c +++ b/Modules/_ctypes/stgdict.c @@ -295,7 +295,7 @@ MakeAnonFields(PyObject *type) PyObject *anon_names; Py_ssize_t i; - if (_PyObject_LookupAttr(type, &_Py_ID(_anonymous_), &anon) < 0) { + if (PyObject_GetOptionalAttr(type, &_Py_ID(_anonymous_), &anon) < 0) { return -1; } if (anon == NULL) { @@ -385,7 +385,7 @@ PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct if (fields == NULL) return 0; - if (_PyObject_LookupAttr(type, &_Py_ID(_swappedbytes_), &tmp) < 0) { + if (PyObject_GetOptionalAttr(type, &_Py_ID(_swappedbytes_), &tmp) < 0) { return -1; } if (tmp) { @@ -396,7 +396,7 @@ PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct big_endian = PY_BIG_ENDIAN; } - if (_PyObject_LookupAttr(type, &_Py_ID(_pack_), &tmp) < 0) { + if (PyObject_GetOptionalAttr(type, &_Py_ID(_pack_), &tmp) < 0) { return -1; } if (tmp) { diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index db2d339fc36db..b8cb0c012fd53 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3791,7 +3791,7 @@ tzinfo_reduce(PyObject *self, PyObject *Py_UNUSED(ignored)) PyObject *args, *state; PyObject *getinitargs; - if (_PyObject_LookupAttr(self, &_Py_ID(__getinitargs__), &getinitargs) < 0) { + if (PyObject_GetOptionalAttr(self, &_Py_ID(__getinitargs__), &getinitargs) < 0) { return NULL; } if (getinitargs != NULL) { diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 3e742e067e7db..a8d68d68420d3 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3530,7 +3530,7 @@ expat_start_doctype_handler(XMLParserObject *self, sysid_obj, NULL); Py_XDECREF(res); } - else if (_PyObject_LookupAttr((PyObject *)self, st->str_doctype, &res) > 0) { + else if (PyObject_GetOptionalAttr((PyObject *)self, st->str_doctype, &res) > 0) { (void)PyErr_WarnEx(PyExc_RuntimeWarning, "The doctype() method of XMLParser is ignored. " "Define doctype() method on the TreeBuilder target.", diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c index e58e87926f673..bfc3d2558c9e3 100644 --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -1463,7 +1463,7 @@ buffered_repr(buffered *self) { PyObject *nameobj, *res; - if (_PyObject_LookupAttr((PyObject *) self, &_Py_ID(name), &nameobj) < 0) { + if (PyObject_GetOptionalAttr((PyObject *) self, &_Py_ID(name), &nameobj) < 0) { if (!PyErr_ExceptionMatches(PyExc_ValueError)) { return NULL; } @@ -1630,7 +1630,7 @@ _bufferedreader_read_all(buffered *self) } _bufferedreader_reset_buf(self); - if (_PyObject_LookupAttr(self->raw, &_Py_ID(readall), &readall) < 0) { + if (PyObject_GetOptionalAttr(self->raw, &_Py_ID(readall), &readall) < 0) { goto cleanup; } if (readall) { diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c index 1a5b61301dec5..39709fd293131 100644 --- a/Modules/_io/fileio.c +++ b/Modules/_io/fileio.c @@ -1099,7 +1099,7 @@ fileio_repr(fileio *self) if (self->fd < 0) return PyUnicode_FromFormat("<_io.FileIO [closed]>"); - if (_PyObject_LookupAttr((PyObject *) self, &_Py_ID(name), &nameobj) < 0) { + if (PyObject_GetOptionalAttr((PyObject *) self, &_Py_ID(name), &nameobj) < 0) { return NULL; } if (nameobj == NULL) { diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c index 729a708a3fc40..e2e8ef46adf90 100644 --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -148,7 +148,7 @@ iobase_is_closed(PyObject *self) int ret; /* This gets the derived attribute, which is *not* __IOBase_closed in most cases! */ - ret = _PyObject_LookupAttr(self, &_Py_ID(__IOBase_closed), &res); + ret = PyObject_GetOptionalAttr(self, &_Py_ID(__IOBase_closed), &res); Py_XDECREF(res); return ret; } @@ -196,7 +196,7 @@ iobase_check_closed(PyObject *self) int closed; /* This gets the derived attribute, which is *not* __IOBase_closed in most cases! */ - closed = _PyObject_LookupAttr(self, &_Py_ID(closed), &res); + closed = PyObject_GetOptionalAttr(self, &_Py_ID(closed), &res); if (closed > 0) { closed = PyObject_IsTrue(res); Py_DECREF(res); @@ -303,7 +303,7 @@ iobase_finalize(PyObject *self) /* If `closed` doesn't exist or can't be evaluated as bool, then the object is probably in an unusable state, so ignore. */ - if (_PyObject_LookupAttr(self, &_Py_ID(closed), &res) <= 0) { + if (PyObject_GetOptionalAttr(self, &_Py_ID(closed), &res) <= 0) { PyErr_Clear(); closed = -1; } @@ -571,7 +571,7 @@ _io__IOBase_readline_impl(PyObject *self, Py_ssize_t limit) PyObject *peek, *buffer, *result; Py_ssize_t old_size = -1; - if (_PyObject_LookupAttr(self, &_Py_ID(peek), &peek) < 0) { + if (PyObject_GetOptionalAttr(self, &_Py_ID(peek), &peek) < 0) { return NULL; } diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c index f704875280da3..a5cf9fc397f5f 100644 --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -946,7 +946,7 @@ _textiowrapper_set_encoder(textio *self, PyObject *codec_info, return -1; /* Get the normalized named of the codec */ - if (_PyObject_LookupAttr(codec_info, &_Py_ID(name), &res) < 0) { + if (PyObject_GetOptionalAttr(codec_info, &_Py_ID(name), &res) < 0) { return -1; } if (res != NULL && PyUnicode_Check(res)) { @@ -1202,7 +1202,7 @@ _io_TextIOWrapper___init___impl(textio *self, PyObject *buffer, Py_IS_TYPE(buffer, state->PyBufferedWriter_Type) || Py_IS_TYPE(buffer, state->PyBufferedRandom_Type)) { - if (_PyObject_LookupAttr(buffer, &_Py_ID(raw), &raw) < 0) + if (PyObject_GetOptionalAttr(buffer, &_Py_ID(raw), &raw) < 0) goto error; /* Cache the raw FileIO object to speed up 'closed' checks */ if (raw != NULL) { @@ -1222,7 +1222,7 @@ _io_TextIOWrapper___init___impl(textio *self, PyObject *buffer, goto error; self->seekable = self->telling = r; - r = _PyObject_LookupAttr(buffer, &_Py_ID(read1), &res); + r = PyObject_GetOptionalAttr(buffer, &_Py_ID(read1), &res); if (r < 0) { goto error; } @@ -2897,7 +2897,7 @@ textiowrapper_repr(textio *self) } goto error; } - if (_PyObject_LookupAttr((PyObject *) self, &_Py_ID(name), &nameobj) < 0) { + if (PyObject_GetOptionalAttr((PyObject *) self, &_Py_ID(name), &nameobj) < 0) { if (!PyErr_ExceptionMatches(PyExc_ValueError)) { goto error; } @@ -2913,7 +2913,7 @@ textiowrapper_repr(textio *self) if (res == NULL) goto error; } - if (_PyObject_LookupAttr((PyObject *) self, &_Py_ID(mode), &modeobj) < 0) { + if (PyObject_GetOptionalAttr((PyObject *) self, &_Py_ID(mode), &modeobj) < 0) { goto error; } if (modeobj != NULL) { @@ -3130,7 +3130,7 @@ textiowrapper_newlines_get(textio *self, void *context) PyObject *res; CHECK_ATTACHED(self); if (self->decoder == NULL || - _PyObject_LookupAttr(self->decoder, &_Py_ID(newlines), &res) == 0) + PyObject_GetOptionalAttr(self->decoder, &_Py_ID(newlines), &res) == 0) { Py_RETURN_NONE; } diff --git a/Modules/_pickle.c b/Modules/_pickle.c index 5ecf7ca7cb9bc..ea44b494cdd7c 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -406,7 +406,7 @@ init_method_ref(PyObject *self, PyObject *name, /* *method_func and *method_self should be consistent. All refcount decrements should be occurred after setting *method_self and *method_func. */ - ret = _PyObject_LookupAttr(self, name, &func); + ret = PyObject_GetOptionalAttr(self, name, &func); if (func == NULL) { *method_self = NULL; Py_CLEAR(*method_func); @@ -1224,7 +1224,7 @@ static int _Pickler_SetOutputStream(PicklerObject *self, PyObject *file) { assert(file != NULL); - if (_PyObject_LookupAttr(file, &_Py_ID(write), &self->write) < 0) { + if (PyObject_GetOptionalAttr(file, &_Py_ID(write), &self->write) < 0) { return -1; } if (self->write == NULL) { @@ -1694,16 +1694,16 @@ static int _Unpickler_SetInputStream(UnpicklerObject *self, PyObject *file) { /* Optional file methods */ - if (_PyObject_LookupAttr(file, &_Py_ID(peek), &self->peek) < 0) { + if (PyObject_GetOptionalAttr(file, &_Py_ID(peek), &self->peek) < 0) { goto error; } - if (_PyObject_LookupAttr(file, &_Py_ID(readinto), &self->readinto) < 0) { + if (PyObject_GetOptionalAttr(file, &_Py_ID(readinto), &self->readinto) < 0) { goto error; } - if (_PyObject_LookupAttr(file, &_Py_ID(read), &self->read) < 0) { + if (PyObject_GetOptionalAttr(file, &_Py_ID(read), &self->read) < 0) { goto error; } - if (_PyObject_LookupAttr(file, &_Py_ID(readline), &self->readline) < 0) { + if (PyObject_GetOptionalAttr(file, &_Py_ID(readline), &self->readline) < 0) { goto error; } if (!self->readline || !self->read) { @@ -1900,7 +1900,7 @@ get_deep_attribute(PyObject *obj, PyObject *names, PyObject **pparent) for (i = 0; i < n; i++) { PyObject *name = PyList_GET_ITEM(names, i); Py_XSETREF(parent, obj); - (void)_PyObject_LookupAttr(parent, name, &obj); + (void)PyObject_GetOptionalAttr(parent, name, &obj); if (obj == NULL) { Py_DECREF(parent); return NULL; @@ -1927,7 +1927,7 @@ getattribute(PyObject *obj, PyObject *name, int allow_qualname) Py_DECREF(dotted_path); } else { - (void)_PyObject_LookupAttr(obj, name, &attr); + (void)PyObject_GetOptionalAttr(obj, name, &attr); } if (attr == NULL && !PyErr_Occurred()) { PyErr_Format(PyExc_AttributeError, @@ -1968,7 +1968,7 @@ whichmodule(PyObject *global, PyObject *dotted_path) Py_ssize_t i; PyObject *modules; - if (_PyObject_LookupAttr(global, &_Py_ID(__module__), &module_name) < 0) { + if (PyObject_GetOptionalAttr(global, &_Py_ID(__module__), &module_name) < 0) { return NULL; } if (module_name) { @@ -3656,7 +3656,7 @@ save_global(PickleState *st, PicklerObject *self, PyObject *obj, global_name = Py_NewRef(name); } else { - if (_PyObject_LookupAttr(obj, &_Py_ID(__qualname__), &global_name) < 0) + if (PyObject_GetOptionalAttr(obj, &_Py_ID(__qualname__), &global_name) < 0) goto error; if (global_name == NULL) { global_name = PyObject_GetAttr(obj, &_Py_ID(__name__)); @@ -3979,7 +3979,7 @@ get_class(PyObject *obj) { PyObject *cls; - if (_PyObject_LookupAttr(obj, &_Py_ID(__class__), &cls) == 0) { + if (PyObject_GetOptionalAttr(obj, &_Py_ID(__class__), &cls) == 0) { cls = Py_NewRef(Py_TYPE(obj)); } return cls; @@ -4062,7 +4062,7 @@ save_reduce(PickleState *st, PicklerObject *self, PyObject *args, if (self->proto >= 2) { PyObject *name; - if (_PyObject_LookupAttr(callable, &_Py_ID(__name__), &name) < 0) { + if (PyObject_GetOptionalAttr(callable, &_Py_ID(__name__), &name) < 0) { return -1; } if (name != NULL && PyUnicode_Check(name)) { @@ -4462,7 +4462,7 @@ save(PickleState *st, PicklerObject *self, PyObject *obj, int pers_save) don't actually have to check for a __reduce__ method. */ /* Check for a __reduce_ex__ method. */ - if (_PyObject_LookupAttr(obj, &_Py_ID(__reduce_ex__), &reduce_func) < 0) { + if (PyObject_GetOptionalAttr(obj, &_Py_ID(__reduce_ex__), &reduce_func) < 0) { goto error; } if (reduce_func != NULL) { @@ -4474,7 +4474,7 @@ save(PickleState *st, PicklerObject *self, PyObject *obj, int pers_save) } else { /* Check for a __reduce__ method. */ - if (_PyObject_LookupAttr(obj, &_Py_ID(__reduce__), &reduce_func) < 0) { + if (PyObject_GetOptionalAttr(obj, &_Py_ID(__reduce__), &reduce_func) < 0) { goto error; } if (reduce_func != NULL) { @@ -4526,7 +4526,7 @@ dump(PickleState *state, PicklerObject *self, PyObject *obj) int status = -1; PyObject *tmp; - if (_PyObject_LookupAttr((PyObject *)self, &_Py_ID(reducer_override), + if (PyObject_GetOptionalAttr((PyObject *)self, &_Py_ID(reducer_override), &tmp) < 0) { goto error; } @@ -4796,7 +4796,7 @@ _pickle_Pickler___init___impl(PicklerObject *self, PyObject *file, if (self->dispatch_table != NULL) { return 0; } - if (_PyObject_LookupAttr((PyObject *)self, &_Py_ID(dispatch_table), + if (PyObject_GetOptionalAttr((PyObject *)self, &_Py_ID(dispatch_table), &self->dispatch_table) < 0) { return -1; } @@ -5797,7 +5797,7 @@ instantiate(PyObject *cls, PyObject *args) assert(PyTuple_Check(args)); if (!PyTuple_GET_SIZE(args) && PyType_Check(cls)) { PyObject *func; - if (_PyObject_LookupAttr(cls, &_Py_ID(__getinitargs__), &func) < 0) { + if (PyObject_GetOptionalAttr(cls, &_Py_ID(__getinitargs__), &func) < 0) { return NULL; } if (func == NULL) { @@ -6451,7 +6451,7 @@ do_append(PickleState *state, UnpicklerObject *self, Py_ssize_t x) else { PyObject *extend_func; - if (_PyObject_LookupAttr(list, &_Py_ID(extend), &extend_func) < 0) { + if (PyObject_GetOptionalAttr(list, &_Py_ID(extend), &extend_func) < 0) { return -1; } if (extend_func != NULL) { @@ -6637,7 +6637,7 @@ load_build(PickleState *st, UnpicklerObject *self) inst = self->stack->data[Py_SIZE(self->stack) - 1]; - if (_PyObject_LookupAttr(inst, &_Py_ID(__setstate__), &setstate) < 0) { + if (PyObject_GetOptionalAttr(inst, &_Py_ID(__setstate__), &setstate) < 0) { Py_DECREF(state); return -1; } diff --git a/Modules/_sqlite/microprotocols.c b/Modules/_sqlite/microprotocols.c index 148220d0f91f9..92f0148bd4c21 100644 --- a/Modules/_sqlite/microprotocols.c +++ b/Modules/_sqlite/microprotocols.c @@ -98,7 +98,7 @@ pysqlite_microprotocols_adapt(pysqlite_state *state, PyObject *obj, } /* try to have the protocol adapt this object */ - if (_PyObject_LookupAttr(proto, state->str___adapt__, &adapter) < 0) { + if (PyObject_GetOptionalAttr(proto, state->str___adapt__, &adapter) < 0) { return NULL; } if (adapter) { @@ -117,7 +117,7 @@ pysqlite_microprotocols_adapt(pysqlite_state *state, PyObject *obj, } /* and finally try to have the object adapt itself */ - if (_PyObject_LookupAttr(obj, state->str___conform__, &adapter) < 0) { + if (PyObject_GetOptionalAttr(obj, state->str___conform__, &adapter) < 0) { return NULL; } if (adapter) { diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index f570b4e700715..82393309b6703 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -1438,7 +1438,7 @@ thread_excepthook_file(PyObject *file, PyObject *exc_type, PyObject *exc_value, PyObject *name = NULL; if (thread != Py_None) { - if (_PyObject_LookupAttr(thread, &_Py_ID(name), &name) < 0) { + if (PyObject_GetOptionalAttr(thread, &_Py_ID(name), &name) < 0) { return -1; } } diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c index f43a23467976b..0000a8d637eb5 100644 --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -2268,7 +2268,7 @@ array_array___reduce_ex___impl(arrayobject *self, PyTypeObject *cls, if (protocol == -1 && PyErr_Occurred()) return NULL; - if (_PyObject_LookupAttr((PyObject *)self, state->str___dict__, &dict) < 0) { + if (PyObject_GetOptionalAttr((PyObject *)self, state->str___dict__, &dict) < 0) { return NULL; } if (dict == NULL) { diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index 13ff253f560b0..f5f7bf33bf8f4 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -1146,7 +1146,7 @@ itertools_tee_impl(PyObject *module, PyObject *iterable, Py_ssize_t n) return NULL; } - if (_PyObject_LookupAttr(it, &_Py_ID(__copy__), ©func) < 0) { + if (PyObject_GetOptionalAttr(it, &_Py_ID(__copy__), ©func) < 0) { Py_DECREF(it); Py_DECREF(result); return NULL; diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index 28915359fb49e..5721ed4412be5 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -832,7 +832,7 @@ pyexpat_xmlparser_ParseFile_impl(xmlparseobject *self, PyTypeObject *cls, pyexpat_state *state = PyType_GetModuleState(cls); - if (_PyObject_LookupAttr(file, state->str_read, &readmethod) < 0) { + if (PyObject_GetOptionalAttr(file, state->str_read, &readmethod) < 0) { return NULL; } if (readmethod == NULL) { diff --git a/Objects/abstract.c b/Objects/abstract.c index e595bc91c220f..b4edcec600771 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -182,7 +182,7 @@ PyObject_GetItem(PyObject *o, PyObject *key) return Py_GenericAlias(o, key); } - if (_PyObject_LookupAttr(o, &_Py_ID(__class_getitem__), &meth) < 0) { + if (PyObject_GetOptionalAttr(o, &_Py_ID(__class_getitem__), &meth) < 0) { return NULL; } if (meth && meth != Py_None) { @@ -2552,7 +2552,7 @@ abstract_get_bases(PyObject *cls) { PyObject *bases; - (void)_PyObject_LookupAttr(cls, &_Py_ID(__bases__), &bases); + (void)PyObject_GetOptionalAttr(cls, &_Py_ID(__bases__), &bases); if (bases != NULL && !PyTuple_Check(bases)) { Py_DECREF(bases); return NULL; @@ -2636,7 +2636,7 @@ object_isinstance(PyObject *inst, PyObject *cls) if (PyType_Check(cls)) { retval = PyObject_TypeCheck(inst, (PyTypeObject *)cls); if (retval == 0) { - retval = _PyObject_LookupAttr(inst, &_Py_ID(__class__), &icls); + retval = PyObject_GetOptionalAttr(inst, &_Py_ID(__class__), &icls); if (icls != NULL) { if (icls != (PyObject *)(Py_TYPE(inst)) && PyType_Check(icls)) { retval = PyType_IsSubtype( @@ -2654,7 +2654,7 @@ object_isinstance(PyObject *inst, PyObject *cls) if (!check_class(cls, "isinstance() arg 2 must be a type, a tuple of types, or a union")) return -1; - retval = _PyObject_LookupAttr(inst, &_Py_ID(__class__), &icls); + retval = PyObject_GetOptionalAttr(inst, &_Py_ID(__class__), &icls); if (icls != NULL) { retval = abstract_issubclass(icls, cls); Py_DECREF(icls); diff --git a/Objects/call.c b/Objects/call.c index 5045c0dc92843..396552d85cfca 100644 --- a/Objects/call.c +++ b/Objects/call.c @@ -173,7 +173,7 @@ object_is_not_callable(PyThreadState *tstate, PyObject *callable) goto basic_type_error; } PyObject *attr; - int res = _PyObject_LookupAttr(callable, name, &attr); + int res = PyObject_GetOptionalAttr(callable, name, &attr); if (res < 0) { _PyErr_Clear(tstate); } diff --git a/Objects/classobject.c b/Objects/classobject.c index 71c4a4e5d0f8a..548b8672f8632 100644 --- a/Objects/classobject.c +++ b/Objects/classobject.c @@ -275,9 +275,9 @@ method_repr(PyMethodObject *a) PyObject *funcname, *result; const char *defname = "?"; - if (_PyObject_LookupAttr(func, &_Py_ID(__qualname__), &funcname) < 0 || + if (PyObject_GetOptionalAttr(func, &_Py_ID(__qualname__), &funcname) < 0 || (funcname == NULL && - _PyObject_LookupAttr(func, &_Py_ID(__name__), &funcname) < 0)) + PyObject_GetOptionalAttr(func, &_Py_ID(__name__), &funcname) < 0)) { return NULL; } @@ -479,7 +479,7 @@ instancemethod_repr(PyObject *self) return NULL; } - if (_PyObject_LookupAttr(func, &_Py_ID(__name__), &funcname) < 0) { + if (PyObject_GetOptionalAttr(func, &_Py_ID(__name__), &funcname) < 0) { return NULL; } if (funcname != NULL && !PyUnicode_Check(funcname)) { diff --git a/Objects/descrobject.c b/Objects/descrobject.c index a81490285eed4..810bd196e8f7e 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -1792,7 +1792,7 @@ property_init_impl(propertyobject *self, PyObject *fget, PyObject *fset, } /* if no docstring given and the getter has one, use that one */ else if (fget != NULL) { - int rc = _PyObject_LookupAttr(fget, &_Py_ID(__doc__), &prop_doc); + int rc = PyObject_GetOptionalAttr(fget, &_Py_ID(__doc__), &prop_doc); if (rc <= 0) { return rc; } diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 194081db2f290..013c21884032a 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -2664,7 +2664,7 @@ dict_update_arg(PyObject *self, PyObject *arg) return PyDict_Merge(self, arg, 1); } PyObject *func; - if (_PyObject_LookupAttr(arg, &_Py_ID(keys), &func) < 0) { + if (PyObject_GetOptionalAttr(arg, &_Py_ID(keys), &func) < 0) { return -1; } if (func != NULL) { diff --git a/Objects/exceptions.c b/Objects/exceptions.c index 85cf2cca16516..42c5317d83d0c 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -208,7 +208,7 @@ BaseException_add_note(PyObject *self, PyObject *note) } PyObject *notes; - if (_PyObject_LookupAttr(self, &_Py_ID(__notes__), ¬es) < 0) { + if (PyObject_GetOptionalAttr(self, &_Py_ID(__notes__), ¬es) < 0) { return NULL; } if (notes == NULL) { @@ -941,7 +941,7 @@ exceptiongroup_subset( PyException_SetCause(eg, PyException_GetCause(orig)); PyObject *notes; - if (_PyObject_LookupAttr(orig, &_Py_ID(__notes__), ¬es) < 0) { + if (PyObject_GetOptionalAttr(orig, &_Py_ID(__notes__), ¬es) < 0) { goto error; } if (notes) { diff --git a/Objects/fileobject.c b/Objects/fileobject.c index 6d980a1b37916..751fb69d0891c 100644 --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -176,7 +176,7 @@ PyObject_AsFileDescriptor(PyObject *o) if (PyLong_Check(o)) { fd = _PyLong_AsInt(o); } - else if (_PyObject_LookupAttr(o, &_Py_ID(fileno), &meth) < 0) { + else if (PyObject_GetOptionalAttr(o, &_Py_ID(fileno), &meth) < 0) { return -1; } else if (meth != NULL) { diff --git a/Objects/funcobject.c b/Objects/funcobject.c index a8a9ee2b9bad4..0c69bf4ebcfed 100644 --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -943,7 +943,7 @@ static int functools_copy_attr(PyObject *wrapper, PyObject *wrapped, PyObject *name) { PyObject *value; - int res = _PyObject_LookupAttr(wrapped, name, &value); + int res = PyObject_GetOptionalAttr(wrapped, name, &value); if (value != NULL) { res = PyObject_SetAttr(wrapper, name, value); Py_DECREF(value); diff --git a/Objects/genericaliasobject.c b/Objects/genericaliasobject.c index 117b4e8dfb960..0c478f3717e03 100644 --- a/Objects/genericaliasobject.c +++ b/Objects/genericaliasobject.c @@ -63,12 +63,12 @@ ga_repr_item(_PyUnicodeWriter *writer, PyObject *p) goto done; } - if (_PyObject_LookupAttr(p, &_Py_ID(__origin__), &tmp) < 0) { + if (PyObject_GetOptionalAttr(p, &_Py_ID(__origin__), &tmp) < 0) { goto done; } if (tmp != NULL) { Py_DECREF(tmp); - if (_PyObject_LookupAttr(p, &_Py_ID(__args__), &tmp) < 0) { + if (PyObject_GetOptionalAttr(p, &_Py_ID(__args__), &tmp) < 0) { goto done; } if (tmp != NULL) { @@ -78,13 +78,13 @@ ga_repr_item(_PyUnicodeWriter *writer, PyObject *p) } } - if (_PyObject_LookupAttr(p, &_Py_ID(__qualname__), &qualname) < 0) { + if (PyObject_GetOptionalAttr(p, &_Py_ID(__qualname__), &qualname) < 0) { goto done; } if (qualname == NULL) { goto use_repr; } - if (_PyObject_LookupAttr(p, &_Py_ID(__module__), &module) < 0) { + if (PyObject_GetOptionalAttr(p, &_Py_ID(__module__), &module) < 0) { goto done; } if (module == NULL || module == Py_None) { @@ -257,7 +257,7 @@ _Py_make_parameters(PyObject *args) if (PyType_Check(t)) { continue; } - if (_PyObject_LookupAttr(t, &_Py_ID(__typing_subst__), &subst) < 0) { + if (PyObject_GetOptionalAttr(t, &_Py_ID(__typing_subst__), &subst) < 0) { Py_DECREF(parameters); return NULL; } @@ -267,7 +267,7 @@ _Py_make_parameters(PyObject *args) } else { PyObject *subparams; - if (_PyObject_LookupAttr(t, &_Py_ID(__parameters__), + if (PyObject_GetOptionalAttr(t, &_Py_ID(__parameters__), &subparams) < 0) { Py_DECREF(parameters); return NULL; @@ -310,7 +310,7 @@ subs_tvars(PyObject *obj, PyObject *params, PyObject **argitems, Py_ssize_t nargs) { PyObject *subparams; - if (_PyObject_LookupAttr(obj, &_Py_ID(__parameters__), &subparams) < 0) { + if (PyObject_GetOptionalAttr(obj, &_Py_ID(__parameters__), &subparams) < 0) { return NULL; } if (subparams && PyTuple_Check(subparams) && PyTuple_GET_SIZE(subparams)) { @@ -361,7 +361,7 @@ _is_unpacked_typevartuple(PyObject *arg) if (PyType_Check(arg)) { // TODO: Add test return 0; } - int res = _PyObject_LookupAttr(arg, &_Py_ID(__typing_is_unpacked_typevartuple__), &tmp); + int res = PyObject_GetOptionalAttr(arg, &_Py_ID(__typing_is_unpacked_typevartuple__), &tmp); if (res > 0) { res = PyObject_IsTrue(tmp); Py_DECREF(tmp); @@ -383,7 +383,7 @@ _unpacked_tuple_args(PyObject *arg) return Py_NewRef(result); } - if (_PyObject_LookupAttr(arg, &_Py_ID(__typing_unpacked_tuple_args__), &result) > 0) { + if (PyObject_GetOptionalAttr(arg, &_Py_ID(__typing_unpacked_tuple_args__), &result) > 0) { if (result == Py_None) { Py_DECREF(result); return NULL; @@ -448,7 +448,7 @@ _Py_subs_parameters(PyObject *self, PyObject *args, PyObject *parameters, PyObje for (Py_ssize_t i = 0; i < nparams; i++) { PyObject *param = PyTuple_GET_ITEM(parameters, i); PyObject *prepare, *tmp; - if (_PyObject_LookupAttr(param, &_Py_ID(__typing_prepare_subst__), &prepare) < 0) { + if (PyObject_GetOptionalAttr(param, &_Py_ID(__typing_prepare_subst__), &prepare) < 0) { Py_DECREF(item); return NULL; } @@ -503,7 +503,7 @@ _Py_subs_parameters(PyObject *self, PyObject *args, PyObject *parameters, PyObje return NULL; } PyObject *subst; - if (_PyObject_LookupAttr(arg, &_Py_ID(__typing_subst__), &subst) < 0) { + if (PyObject_GetOptionalAttr(arg, &_Py_ID(__typing_subst__), &subst) < 0) { Py_DECREF(newargs); Py_DECREF(item); return NULL; diff --git a/Objects/genobject.c b/Objects/genobject.c index 6f0f02c9a16e5..103e8b8bb882f 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -317,7 +317,7 @@ gen_close_iter(PyObject *yf) } else { PyObject *meth; - if (_PyObject_LookupAttr(yf, &_Py_ID(close), &meth) < 0) { + if (PyObject_GetOptionalAttr(yf, &_Py_ID(close), &meth) < 0) { PyErr_WriteUnraisable(yf); } if (meth) { @@ -492,7 +492,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, } else { /* `yf` is an iterator or a coroutine-like object. */ PyObject *meth; - if (_PyObject_LookupAttr(yf, &_Py_ID(throw), &meth) < 0) { + if (PyObject_GetOptionalAttr(yf, &_Py_ID(throw), &meth) < 0) { Py_DECREF(yf); return NULL; } diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index 3e500b555e70f..4071b5a3f1a62 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -742,7 +742,7 @@ _PyModuleSpec_IsInitializing(PyObject *spec) { if (spec != NULL) { PyObject *value; - int ok = _PyObject_LookupAttr(spec, &_Py_ID(_initializing), &value); + int ok = PyObject_GetOptionalAttr(spec, &_Py_ID(_initializing), &value); if (ok == 0) { return 0; } diff --git a/Objects/odictobject.c b/Objects/odictobject.c index 39b0f68451057..e76d2ded61cf7 100644 --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -2154,7 +2154,7 @@ mutablemapping_update_arg(PyObject *self, PyObject *arg) return res; } PyObject *func; - if (_PyObject_LookupAttr(arg, &_Py_ID(keys), &func) < 0) { + if (PyObject_GetOptionalAttr(arg, &_Py_ID(keys), &func) < 0) { return -1; } if (func != NULL) { @@ -2186,7 +2186,7 @@ mutablemapping_update_arg(PyObject *self, PyObject *arg) } return 0; } - if (_PyObject_LookupAttr(arg, &_Py_ID(items), &func) < 0) { + if (PyObject_GetOptionalAttr(arg, &_Py_ID(items), &func) < 0) { return -1; } if (func != NULL) { diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 5430924e69dee..b1f9f1280fd04 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -2359,7 +2359,7 @@ static PyObject * class_name(PyObject *cls) { PyObject *name; - if (_PyObject_LookupAttr(cls, &_Py_ID(__name__), &name) == 0) { + if (PyObject_GetOptionalAttr(cls, &_Py_ID(__name__), &name) == 0) { name = PyObject_Repr(cls); } return name; @@ -3865,7 +3865,7 @@ type_new_get_bases(type_new_ctx *ctx, PyObject **type) continue; } PyObject *mro_entries; - if (_PyObject_LookupAttr(base, &_Py_ID(__mro_entries__), + if (PyObject_GetOptionalAttr(base, &_Py_ID(__mro_entries__), &mro_entries) < 0) { return -1; } @@ -5147,7 +5147,7 @@ merge_class_dict(PyObject *dict, PyObject *aclass) assert(aclass); /* Merge in the type's dict (if any). */ - if (_PyObject_LookupAttr(aclass, &_Py_ID(__dict__), &classdict) < 0) { + if (PyObject_GetOptionalAttr(aclass, &_Py_ID(__dict__), &classdict) < 0) { return -1; } if (classdict != NULL) { @@ -5158,7 +5158,7 @@ merge_class_dict(PyObject *dict, PyObject *aclass) } /* Recursively merge in the base types' (if any) dicts. */ - if (_PyObject_LookupAttr(aclass, &_Py_ID(__bases__), &bases) < 0) { + if (PyObject_GetOptionalAttr(aclass, &_Py_ID(__bases__), &bases) < 0) { return -1; } if (bases != NULL) { @@ -5984,7 +5984,7 @@ object_getstate_default(PyObject *obj, int required) PyObject *name, *value; name = Py_NewRef(PyList_GET_ITEM(slotnames, i)); - if (_PyObject_LookupAttr(obj, name, &value) < 0) { + if (PyObject_GetOptionalAttr(obj, name, &value) < 0) { Py_DECREF(name); goto error; } @@ -6381,7 +6381,7 @@ object___reduce_ex___impl(PyObject *self, int protocol) } } - if (_PyObject_LookupAttr(self, &_Py_ID(__reduce__), &reduce) < 0) { + if (PyObject_GetOptionalAttr(self, &_Py_ID(__reduce__), &reduce) < 0) { return NULL; } if (reduce != NULL) { @@ -6500,7 +6500,7 @@ object___dir___impl(PyObject *self) PyObject *itsclass = NULL; /* Get __dict__ (which may or may not be a real dict...) */ - if (_PyObject_LookupAttr(self, &_Py_ID(__dict__), &dict) < 0) { + if (PyObject_GetOptionalAttr(self, &_Py_ID(__dict__), &dict) < 0) { return NULL; } if (dict == NULL) { @@ -6520,7 +6520,7 @@ object___dir___impl(PyObject *self) goto error; /* Merge in attrs reachable from its class. */ - if (_PyObject_LookupAttr(self, &_Py_ID(__class__), &itsclass) < 0) { + if (PyObject_GetOptionalAttr(self, &_Py_ID(__class__), &itsclass) < 0) { goto error; } /* XXX(tomer): Perhaps fall back to Py_TYPE(obj) if no @@ -8393,7 +8393,7 @@ method_is_overloaded(PyObject *left, PyObject *right, PyObject *name) PyObject *a, *b; int ok; - if (_PyObject_LookupAttr((PyObject *)(Py_TYPE(right)), name, &b) < 0) { + if (PyObject_GetOptionalAttr((PyObject *)(Py_TYPE(right)), name, &b) < 0) { return -1; } if (b == NULL) { @@ -8401,7 +8401,7 @@ method_is_overloaded(PyObject *left, PyObject *right, PyObject *name) return 0; } - if (_PyObject_LookupAttr((PyObject *)(Py_TYPE(left)), name, &a) < 0) { + if (PyObject_GetOptionalAttr((PyObject *)(Py_TYPE(left)), name, &a) < 0) { Py_DECREF(b); return -1; } @@ -10373,7 +10373,7 @@ supercheck(PyTypeObject *type, PyObject *obj) /* Try the slow way */ PyObject *class_attr; - if (_PyObject_LookupAttr(obj, &_Py_ID(__class__), &class_attr) < 0) { + if (PyObject_GetOptionalAttr(obj, &_Py_ID(__class__), &class_attr) < 0) { return NULL; } if (class_attr != NULL && diff --git a/Objects/unionobject.c b/Objects/unionobject.c index f509a161bb956..269f46914f263 100644 --- a/Objects/unionobject.c +++ b/Objects/unionobject.c @@ -194,13 +194,13 @@ union_repr_item(_PyUnicodeWriter *writer, PyObject *p) return _PyUnicodeWriter_WriteASCIIString(writer, "None", 4); } - if (_PyObject_LookupAttr(p, &_Py_ID(__origin__), &tmp) < 0) { + if (PyObject_GetOptionalAttr(p, &_Py_ID(__origin__), &tmp) < 0) { goto exit; } if (tmp) { Py_DECREF(tmp); - if (_PyObject_LookupAttr(p, &_Py_ID(__args__), &tmp) < 0) { + if (PyObject_GetOptionalAttr(p, &_Py_ID(__args__), &tmp) < 0) { goto exit; } if (tmp) { @@ -210,13 +210,13 @@ union_repr_item(_PyUnicodeWriter *writer, PyObject *p) } } - if (_PyObject_LookupAttr(p, &_Py_ID(__qualname__), &qualname) < 0) { + if (PyObject_GetOptionalAttr(p, &_Py_ID(__qualname__), &qualname) < 0) { goto exit; } if (qualname == NULL) { goto use_repr; } - if (_PyObject_LookupAttr(p, &_Py_ID(__module__), &module) < 0) { + if (PyObject_GetOptionalAttr(p, &_Py_ID(__module__), &module) < 0) { goto exit; } if (module == NULL || module == Py_None) { diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py index d4763ea260a5a..e9665dd808af3 100755 --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -629,7 +629,7 @@ def isSimpleType(self, field): def visitField(self, field, name, sum=None, prod=None, depth=0): ctype = get_c_type(field.type) - line = "if (_PyObject_LookupAttr(obj, state->%s, &tmp) < 0) {" + line = "if (PyObject_GetOptionalAttr(obj, state->%s, &tmp) < 0) {" self.emit(line % field.name, depth) self.emit("return 1;", depth+1) self.emit("}", depth) @@ -813,7 +813,7 @@ def visitModule(self, mod): Py_ssize_t i, numfields = 0; int res = -1; PyObject *key, *value, *fields; - if (_PyObject_LookupAttr((PyObject*)Py_TYPE(self), state->_fields, &fields) < 0) { + if (PyObject_GetOptionalAttr((PyObject*)Py_TYPE(self), state->_fields, &fields) < 0) { goto cleanup; } if (fields) { @@ -887,7 +887,7 @@ def visitModule(self, mod): } PyObject *dict; - if (_PyObject_LookupAttr(self, state->__dict__, &dict) < 0) { + if (PyObject_GetOptionalAttr(self, state->__dict__, &dict) < 0) { return NULL; } if (dict) { diff --git a/Python/Python-ast.c b/Python/Python-ast.c index 5db9ade3af454..55a1370fbd038 100644 --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -839,7 +839,7 @@ ast_type_init(PyObject *self, PyObject *args, PyObject *kw) Py_ssize_t i, numfields = 0; int res = -1; PyObject *key, *value, *fields; - if (_PyObject_LookupAttr((PyObject*)Py_TYPE(self), state->_fields, &fields) < 0) { + if (PyObject_GetOptionalAttr((PyObject*)Py_TYPE(self), state->_fields, &fields) < 0) { goto cleanup; } if (fields) { @@ -913,7 +913,7 @@ ast_type_reduce(PyObject *self, PyObject *unused) } PyObject *dict; - if (_PyObject_LookupAttr(self, state->__dict__, &dict) < 0) { + if (PyObject_GetOptionalAttr(self, state->__dict__, &dict) < 0) { return NULL; } if (dict) { @@ -5750,7 +5750,7 @@ obj2ast_mod(struct ast_state *state, PyObject* obj, mod_ty* out, PyArena* arena) asdl_stmt_seq* body; asdl_type_ignore_seq* type_ignores; - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -5788,7 +5788,7 @@ obj2ast_mod(struct ast_state *state, PyObject* obj, mod_ty* out, PyArena* arena) } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_ignores, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_ignores, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -5838,7 +5838,7 @@ obj2ast_mod(struct ast_state *state, PyObject* obj, mod_ty* out, PyArena* arena) if (isinstance) { asdl_stmt_seq* body; - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -5888,7 +5888,7 @@ obj2ast_mod(struct ast_state *state, PyObject* obj, mod_ty* out, PyArena* arena) if (isinstance) { expr_ty body; - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -5918,7 +5918,7 @@ obj2ast_mod(struct ast_state *state, PyObject* obj, mod_ty* out, PyArena* arena) asdl_expr_seq* argtypes; expr_ty returns; - if (_PyObject_LookupAttr(obj, state->argtypes, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->argtypes, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -5956,7 +5956,7 @@ obj2ast_mod(struct ast_state *state, PyObject* obj, mod_ty* out, PyArena* arena) } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->returns, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->returns, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6001,7 +6001,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* *out = NULL; return 0; } - if (_PyObject_LookupAttr(obj, state->lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6018,7 +6018,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6035,7 +6035,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_lineno, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -6052,7 +6052,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_col_offset, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -6083,7 +6083,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* string type_comment; asdl_type_param_seq* type_params; - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6100,7 +6100,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->args, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->args, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6117,7 +6117,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6155,7 +6155,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->decorator_list, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->decorator_list, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6193,7 +6193,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->returns, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->returns, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -6210,7 +6210,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_comment, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_comment, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -6227,7 +6227,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_params, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_params, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6286,7 +6286,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* string type_comment; asdl_type_param_seq* type_params; - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6303,7 +6303,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->args, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->args, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6320,7 +6320,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6358,7 +6358,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->decorator_list, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->decorator_list, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6396,7 +6396,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->returns, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->returns, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -6413,7 +6413,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_comment, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_comment, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -6430,7 +6430,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_params, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_params, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6488,7 +6488,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_expr_seq* decorator_list; asdl_type_param_seq* type_params; - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6505,7 +6505,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->bases, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->bases, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6543,7 +6543,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->keywords, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->keywords, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6581,7 +6581,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6619,7 +6619,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->decorator_list, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->decorator_list, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6657,7 +6657,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_params, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_params, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6709,7 +6709,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (isinstance) { expr_ty value; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -6739,7 +6739,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (isinstance) { asdl_expr_seq* targets; - if (_PyObject_LookupAttr(obj, state->targets, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->targets, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6792,7 +6792,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* expr_ty value; string type_comment; - if (_PyObject_LookupAttr(obj, state->targets, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->targets, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6830,7 +6830,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6847,7 +6847,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_comment, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_comment, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -6879,7 +6879,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_type_param_seq* type_params; expr_ty value; - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6896,7 +6896,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_params, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_params, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6934,7 +6934,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6966,7 +6966,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* operator_ty op; expr_ty value; - if (_PyObject_LookupAttr(obj, state->target, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->target, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -6983,7 +6983,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->op, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->op, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7000,7 +7000,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7033,7 +7033,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* expr_ty value; int simple; - if (_PyObject_LookupAttr(obj, state->target, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->target, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7050,7 +7050,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->annotation, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->annotation, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7067,7 +7067,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -7084,7 +7084,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->simple, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->simple, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7118,7 +7118,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_stmt_seq* orelse; string type_comment; - if (_PyObject_LookupAttr(obj, state->target, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->target, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7135,7 +7135,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->iter, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->iter, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7152,7 +7152,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7190,7 +7190,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->orelse, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->orelse, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7228,7 +7228,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_comment, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_comment, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -7262,7 +7262,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_stmt_seq* orelse; string type_comment; - if (_PyObject_LookupAttr(obj, state->target, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->target, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7279,7 +7279,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->iter, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->iter, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7296,7 +7296,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7334,7 +7334,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->orelse, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->orelse, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7372,7 +7372,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_comment, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_comment, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -7405,7 +7405,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_stmt_seq* body; asdl_stmt_seq* orelse; - if (_PyObject_LookupAttr(obj, state->test, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->test, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7422,7 +7422,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7460,7 +7460,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->orelse, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->orelse, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7513,7 +7513,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_stmt_seq* body; asdl_stmt_seq* orelse; - if (_PyObject_LookupAttr(obj, state->test, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->test, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7530,7 +7530,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7568,7 +7568,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->orelse, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->orelse, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7621,7 +7621,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_stmt_seq* body; string type_comment; - if (_PyObject_LookupAttr(obj, state->items, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->items, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7659,7 +7659,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7697,7 +7697,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_comment, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_comment, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -7729,7 +7729,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_stmt_seq* body; string type_comment; - if (_PyObject_LookupAttr(obj, state->items, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->items, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7767,7 +7767,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7805,7 +7805,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_comment, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_comment, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -7836,7 +7836,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* expr_ty subject; asdl_match_case_seq* cases; - if (_PyObject_LookupAttr(obj, state->subject, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->subject, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7853,7 +7853,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->cases, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->cases, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7905,7 +7905,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* expr_ty exc; expr_ty cause; - if (_PyObject_LookupAttr(obj, state->exc, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->exc, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -7922,7 +7922,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->cause, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->cause, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -7955,7 +7955,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_stmt_seq* orelse; asdl_stmt_seq* finalbody; - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -7993,7 +7993,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->handlers, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->handlers, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8031,7 +8031,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->orelse, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->orelse, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8069,7 +8069,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->finalbody, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->finalbody, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8123,7 +8123,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_stmt_seq* orelse; asdl_stmt_seq* finalbody; - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8161,7 +8161,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->handlers, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->handlers, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8199,7 +8199,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->orelse, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->orelse, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8237,7 +8237,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->finalbody, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->finalbody, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8289,7 +8289,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* expr_ty test; expr_ty msg; - if (_PyObject_LookupAttr(obj, state->test, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->test, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8306,7 +8306,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->msg, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->msg, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -8336,7 +8336,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (isinstance) { asdl_alias_seq* names; - if (_PyObject_LookupAttr(obj, state->names, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->names, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8389,7 +8389,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* asdl_alias_seq* names; int level; - if (_PyObject_LookupAttr(obj, state->module, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->module, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -8406,7 +8406,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->names, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->names, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8444,7 +8444,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->level, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->level, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -8474,7 +8474,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (isinstance) { asdl_identifier_seq* names; - if (_PyObject_LookupAttr(obj, state->names, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->names, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8525,7 +8525,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (isinstance) { asdl_identifier_seq* names; - if (_PyObject_LookupAttr(obj, state->names, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->names, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8576,7 +8576,7 @@ obj2ast_stmt(struct ast_state *state, PyObject* obj, stmt_ty* out, PyArena* if (isinstance) { expr_ty value; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8658,7 +8658,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* *out = NULL; return 0; } - if (_PyObject_LookupAttr(obj, state->lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8675,7 +8675,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8692,7 +8692,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_lineno, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -8709,7 +8709,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_col_offset, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -8735,7 +8735,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* boolop_ty op; asdl_expr_seq* values; - if (_PyObject_LookupAttr(obj, state->op, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->op, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8752,7 +8752,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->values, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->values, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8804,7 +8804,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* expr_ty target; expr_ty value; - if (_PyObject_LookupAttr(obj, state->target, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->target, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8821,7 +8821,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8853,7 +8853,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* operator_ty op; expr_ty right; - if (_PyObject_LookupAttr(obj, state->left, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->left, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8870,7 +8870,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->op, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->op, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8887,7 +8887,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->right, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->right, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8918,7 +8918,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* unaryop_ty op; expr_ty operand; - if (_PyObject_LookupAttr(obj, state->op, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->op, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8935,7 +8935,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->operand, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->operand, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8966,7 +8966,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* arguments_ty args; expr_ty body; - if (_PyObject_LookupAttr(obj, state->args, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->args, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -8983,7 +8983,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9015,7 +9015,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* expr_ty body; expr_ty orelse; - if (_PyObject_LookupAttr(obj, state->test, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->test, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9032,7 +9032,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9049,7 +9049,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->orelse, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->orelse, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9080,7 +9080,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* asdl_expr_seq* keys; asdl_expr_seq* values; - if (_PyObject_LookupAttr(obj, state->keys, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->keys, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9118,7 +9118,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->values, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->values, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9169,7 +9169,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (isinstance) { asdl_expr_seq* elts; - if (_PyObject_LookupAttr(obj, state->elts, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->elts, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9221,7 +9221,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* expr_ty elt; asdl_comprehension_seq* generators; - if (_PyObject_LookupAttr(obj, state->elt, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->elt, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9238,7 +9238,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->generators, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->generators, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9290,7 +9290,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* expr_ty elt; asdl_comprehension_seq* generators; - if (_PyObject_LookupAttr(obj, state->elt, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->elt, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9307,7 +9307,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->generators, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->generators, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9360,7 +9360,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* expr_ty value; asdl_comprehension_seq* generators; - if (_PyObject_LookupAttr(obj, state->key, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->key, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9377,7 +9377,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9394,7 +9394,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->generators, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->generators, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9446,7 +9446,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* expr_ty elt; asdl_comprehension_seq* generators; - if (_PyObject_LookupAttr(obj, state->elt, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->elt, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9463,7 +9463,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->generators, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->generators, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9514,7 +9514,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (isinstance) { expr_ty value; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9544,7 +9544,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (isinstance) { expr_ty value; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -9574,7 +9574,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (isinstance) { expr_ty value; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9606,7 +9606,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* asdl_int_seq* ops; asdl_expr_seq* comparators; - if (_PyObject_LookupAttr(obj, state->left, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->left, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9623,7 +9623,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->ops, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->ops, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9661,7 +9661,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->comparators, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->comparators, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9714,7 +9714,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* asdl_expr_seq* args; asdl_keyword_seq* keywords; - if (_PyObject_LookupAttr(obj, state->func, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->func, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9731,7 +9731,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->args, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->args, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9769,7 +9769,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->keywords, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->keywords, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9822,7 +9822,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* int conversion; expr_ty format_spec; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9839,7 +9839,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->conversion, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->conversion, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9856,7 +9856,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->format_spec, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->format_spec, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -9887,7 +9887,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (isinstance) { asdl_expr_seq* values; - if (_PyObject_LookupAttr(obj, state->values, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->values, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9939,7 +9939,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* constant value; string kind; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -9956,7 +9956,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->kind, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->kind, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -9988,7 +9988,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* identifier attr; expr_context_ty ctx; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10005,7 +10005,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->attr, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->attr, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10022,7 +10022,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->ctx, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->ctx, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10054,7 +10054,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* expr_ty slice; expr_context_ty ctx; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10071,7 +10071,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->slice, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->slice, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10088,7 +10088,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->ctx, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->ctx, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10119,7 +10119,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* expr_ty value; expr_context_ty ctx; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10136,7 +10136,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->ctx, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->ctx, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10167,7 +10167,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* identifier id; expr_context_ty ctx; - if (_PyObject_LookupAttr(obj, state->id, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->id, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10184,7 +10184,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->ctx, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->ctx, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10215,7 +10215,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* asdl_expr_seq* elts; expr_context_ty ctx; - if (_PyObject_LookupAttr(obj, state->elts, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->elts, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10253,7 +10253,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->ctx, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->ctx, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10284,7 +10284,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* asdl_expr_seq* elts; expr_context_ty ctx; - if (_PyObject_LookupAttr(obj, state->elts, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->elts, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10322,7 +10322,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->ctx, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->ctx, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10354,7 +10354,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* expr_ty upper; expr_ty step; - if (_PyObject_LookupAttr(obj, state->lower, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lower, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -10371,7 +10371,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->upper, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->upper, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -10388,7 +10388,7 @@ obj2ast_expr(struct ast_state *state, PyObject* obj, expr_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->step, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->step, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -10738,7 +10738,7 @@ obj2ast_comprehension(struct ast_state *state, PyObject* obj, comprehension_ty* asdl_expr_seq* ifs; int is_async; - if (_PyObject_LookupAttr(obj, state->target, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->target, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10755,7 +10755,7 @@ obj2ast_comprehension(struct ast_state *state, PyObject* obj, comprehension_ty* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->iter, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->iter, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10772,7 +10772,7 @@ obj2ast_comprehension(struct ast_state *state, PyObject* obj, comprehension_ty* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->ifs, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->ifs, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10810,7 +10810,7 @@ obj2ast_comprehension(struct ast_state *state, PyObject* obj, comprehension_ty* } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->is_async, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->is_async, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10852,7 +10852,7 @@ obj2ast_excepthandler(struct ast_state *state, PyObject* obj, excepthandler_ty* *out = NULL; return 0; } - if (_PyObject_LookupAttr(obj, state->lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10869,7 +10869,7 @@ obj2ast_excepthandler(struct ast_state *state, PyObject* obj, excepthandler_ty* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -10886,7 +10886,7 @@ obj2ast_excepthandler(struct ast_state *state, PyObject* obj, excepthandler_ty* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_lineno, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -10903,7 +10903,7 @@ obj2ast_excepthandler(struct ast_state *state, PyObject* obj, excepthandler_ty* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_col_offset, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -10930,7 +10930,7 @@ obj2ast_excepthandler(struct ast_state *state, PyObject* obj, excepthandler_ty* identifier name; asdl_stmt_seq* body; - if (_PyObject_LookupAttr(obj, state->type, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -10947,7 +10947,7 @@ obj2ast_excepthandler(struct ast_state *state, PyObject* obj, excepthandler_ty* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -10964,7 +10964,7 @@ obj2ast_excepthandler(struct ast_state *state, PyObject* obj, excepthandler_ty* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11027,7 +11027,7 @@ obj2ast_arguments(struct ast_state *state, PyObject* obj, arguments_ty* out, arg_ty kwarg; asdl_expr_seq* defaults; - if (_PyObject_LookupAttr(obj, state->posonlyargs, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->posonlyargs, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11065,7 +11065,7 @@ obj2ast_arguments(struct ast_state *state, PyObject* obj, arguments_ty* out, } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->args, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->args, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11103,7 +11103,7 @@ obj2ast_arguments(struct ast_state *state, PyObject* obj, arguments_ty* out, } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->vararg, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->vararg, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11120,7 +11120,7 @@ obj2ast_arguments(struct ast_state *state, PyObject* obj, arguments_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->kwonlyargs, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->kwonlyargs, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11158,7 +11158,7 @@ obj2ast_arguments(struct ast_state *state, PyObject* obj, arguments_ty* out, } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->kw_defaults, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->kw_defaults, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11196,7 +11196,7 @@ obj2ast_arguments(struct ast_state *state, PyObject* obj, arguments_ty* out, } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->kwarg, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->kwarg, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11213,7 +11213,7 @@ obj2ast_arguments(struct ast_state *state, PyObject* obj, arguments_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->defaults, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->defaults, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11272,7 +11272,7 @@ obj2ast_arg(struct ast_state *state, PyObject* obj, arg_ty* out, PyArena* arena) int end_lineno; int end_col_offset; - if (_PyObject_LookupAttr(obj, state->arg, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->arg, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11289,7 +11289,7 @@ obj2ast_arg(struct ast_state *state, PyObject* obj, arg_ty* out, PyArena* arena) if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->annotation, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->annotation, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11306,7 +11306,7 @@ obj2ast_arg(struct ast_state *state, PyObject* obj, arg_ty* out, PyArena* arena) if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->type_comment, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->type_comment, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11323,7 +11323,7 @@ obj2ast_arg(struct ast_state *state, PyObject* obj, arg_ty* out, PyArena* arena) if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11340,7 +11340,7 @@ obj2ast_arg(struct ast_state *state, PyObject* obj, arg_ty* out, PyArena* arena) if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11357,7 +11357,7 @@ obj2ast_arg(struct ast_state *state, PyObject* obj, arg_ty* out, PyArena* arena) if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_lineno, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11374,7 +11374,7 @@ obj2ast_arg(struct ast_state *state, PyObject* obj, arg_ty* out, PyArena* arena) if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_col_offset, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11412,7 +11412,7 @@ obj2ast_keyword(struct ast_state *state, PyObject* obj, keyword_ty* out, int end_lineno; int end_col_offset; - if (_PyObject_LookupAttr(obj, state->arg, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->arg, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11429,7 +11429,7 @@ obj2ast_keyword(struct ast_state *state, PyObject* obj, keyword_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11446,7 +11446,7 @@ obj2ast_keyword(struct ast_state *state, PyObject* obj, keyword_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11463,7 +11463,7 @@ obj2ast_keyword(struct ast_state *state, PyObject* obj, keyword_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11480,7 +11480,7 @@ obj2ast_keyword(struct ast_state *state, PyObject* obj, keyword_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_lineno, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11497,7 +11497,7 @@ obj2ast_keyword(struct ast_state *state, PyObject* obj, keyword_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_col_offset, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11535,7 +11535,7 @@ obj2ast_alias(struct ast_state *state, PyObject* obj, alias_ty* out, PyArena* int end_lineno; int end_col_offset; - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11552,7 +11552,7 @@ obj2ast_alias(struct ast_state *state, PyObject* obj, alias_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->asname, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->asname, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11569,7 +11569,7 @@ obj2ast_alias(struct ast_state *state, PyObject* obj, alias_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11586,7 +11586,7 @@ obj2ast_alias(struct ast_state *state, PyObject* obj, alias_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11603,7 +11603,7 @@ obj2ast_alias(struct ast_state *state, PyObject* obj, alias_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_lineno, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11620,7 +11620,7 @@ obj2ast_alias(struct ast_state *state, PyObject* obj, alias_ty* out, PyArena* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_col_offset, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11654,7 +11654,7 @@ obj2ast_withitem(struct ast_state *state, PyObject* obj, withitem_ty* out, expr_ty context_expr; expr_ty optional_vars; - if (_PyObject_LookupAttr(obj, state->context_expr, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->context_expr, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11671,7 +11671,7 @@ obj2ast_withitem(struct ast_state *state, PyObject* obj, withitem_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->optional_vars, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->optional_vars, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11705,7 +11705,7 @@ obj2ast_match_case(struct ast_state *state, PyObject* obj, match_case_ty* out, expr_ty guard; asdl_stmt_seq* body; - if (_PyObject_LookupAttr(obj, state->pattern, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->pattern, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11722,7 +11722,7 @@ obj2ast_match_case(struct ast_state *state, PyObject* obj, match_case_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->guard, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->guard, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -11739,7 +11739,7 @@ obj2ast_match_case(struct ast_state *state, PyObject* obj, match_case_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->body, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11802,7 +11802,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, *out = NULL; return 0; } - if (_PyObject_LookupAttr(obj, state->lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11819,7 +11819,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11836,7 +11836,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11853,7 +11853,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11878,7 +11878,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (isinstance) { expr_ty value; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11908,7 +11908,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (isinstance) { constant value; - if (_PyObject_LookupAttr(obj, state->value, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->value, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11938,7 +11938,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (isinstance) { asdl_pattern_seq* patterns; - if (_PyObject_LookupAttr(obj, state->patterns, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->patterns, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -11991,7 +11991,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, asdl_pattern_seq* patterns; identifier rest; - if (_PyObject_LookupAttr(obj, state->keys, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->keys, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12029,7 +12029,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->patterns, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->patterns, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12067,7 +12067,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->rest, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->rest, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -12100,7 +12100,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, asdl_identifier_seq* kwd_attrs; asdl_pattern_seq* kwd_patterns; - if (_PyObject_LookupAttr(obj, state->cls, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->cls, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12117,7 +12117,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->patterns, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->patterns, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12155,7 +12155,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->kwd_attrs, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->kwd_attrs, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12193,7 +12193,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, } Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->kwd_patterns, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->kwd_patterns, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12245,7 +12245,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (isinstance) { identifier name; - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -12276,7 +12276,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, pattern_ty pattern; identifier name; - if (_PyObject_LookupAttr(obj, state->pattern, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->pattern, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -12293,7 +12293,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -12323,7 +12323,7 @@ obj2ast_pattern(struct ast_state *state, PyObject* obj, pattern_ty* out, if (isinstance) { asdl_pattern_seq* patterns; - if (_PyObject_LookupAttr(obj, state->patterns, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->patterns, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12395,7 +12395,7 @@ obj2ast_type_ignore(struct ast_state *state, PyObject* obj, type_ignore_ty* int lineno; string tag; - if (_PyObject_LookupAttr(obj, state->lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12412,7 +12412,7 @@ obj2ast_type_ignore(struct ast_state *state, PyObject* obj, type_ignore_ty* if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->tag, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->tag, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12457,7 +12457,7 @@ obj2ast_type_param(struct ast_state *state, PyObject* obj, type_param_ty* out, *out = NULL; return 0; } - if (_PyObject_LookupAttr(obj, state->lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12474,7 +12474,7 @@ obj2ast_type_param(struct ast_state *state, PyObject* obj, type_param_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12491,7 +12491,7 @@ obj2ast_type_param(struct ast_state *state, PyObject* obj, type_param_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_lineno, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_lineno, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12508,7 +12508,7 @@ obj2ast_type_param(struct ast_state *state, PyObject* obj, type_param_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->end_col_offset, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->end_col_offset, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12534,7 +12534,7 @@ obj2ast_type_param(struct ast_state *state, PyObject* obj, type_param_ty* out, identifier name; expr_ty bound; - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12551,7 +12551,7 @@ obj2ast_type_param(struct ast_state *state, PyObject* obj, type_param_ty* out, if (res != 0) goto failed; Py_CLEAR(tmp); } - if (_PyObject_LookupAttr(obj, state->bound, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->bound, &tmp) < 0) { return 1; } if (tmp == NULL || tmp == Py_None) { @@ -12581,7 +12581,7 @@ obj2ast_type_param(struct ast_state *state, PyObject* obj, type_param_ty* out, if (isinstance) { identifier name; - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL) { @@ -12611,7 +12611,7 @@ obj2ast_type_param(struct ast_state *state, PyObject* obj, type_param_ty* out, if (isinstance) { identifier name; - if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { + if (PyObject_GetOptionalAttr(obj, state->name, &tmp) < 0) { return 1; } if (tmp == NULL) { diff --git a/Python/_warnings.c b/Python/_warnings.c index e0580f01d9361..82e621243a0c1 100644 --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -223,7 +223,7 @@ get_warnings_attr(PyInterpreterState *interp, PyObject *attr, int try_import) return NULL; } - (void)_PyObject_LookupAttr(warnings_module, attr, &obj); + (void)PyObject_GetOptionalAttr(warnings_module, attr, &obj); Py_DECREF(warnings_module); return obj; } @@ -1069,7 +1069,7 @@ get_source_line(PyInterpreterState *interp, PyObject *module_globals, int lineno Py_INCREF(module_name); /* Make sure the loader implements the optional get_source() method. */ - (void)_PyObject_LookupAttr(loader, &_Py_ID(get_source), &get_source); + (void)PyObject_GetOptionalAttr(loader, &_Py_ID(get_source), &get_source); Py_DECREF(loader); if (!get_source) { Py_DECREF(module_name); diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c index 20a86fc6f583d..7d77fd5c0c328 100644 --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -34,7 +34,7 @@ update_bases(PyObject *bases, PyObject *const *args, Py_ssize_t nargs) } continue; } - if (_PyObject_LookupAttr(base, &_Py_ID(__mro_entries__), &meth) < 0) { + if (PyObject_GetOptionalAttr(base, &_Py_ID(__mro_entries__), &meth) < 0) { goto error; } if (!meth) { @@ -175,7 +175,7 @@ builtin___build_class__(PyObject *self, PyObject *const *args, Py_ssize_t nargs, } /* else: meta is not a class, so we cannot do the metaclass calculation, so we will use the explicitly given object as it is */ - if (_PyObject_LookupAttr(meta, &_Py_ID(__prepare__), &prep) < 0) { + if (PyObject_GetOptionalAttr(meta, &_Py_ID(__prepare__), &prep) < 0) { ns = NULL; } else if (prep == NULL) { @@ -1160,7 +1160,7 @@ builtin_getattr_impl(PyObject *module, PyObject *object, PyObject *name, PyObject *result; if (default_value != NULL) { - if (_PyObject_LookupAttr(object, name, &result) == 0) { + if (PyObject_GetOptionalAttr(object, name, &result) == 0) { return Py_NewRef(default_value); } } @@ -1209,7 +1209,7 @@ builtin_hasattr_impl(PyObject *module, PyObject *obj, PyObject *name) { PyObject *v; - if (_PyObject_LookupAttr(obj, name, &v) < 0) { + if (PyObject_GetOptionalAttr(obj, name, &v) < 0) { return NULL; } if (v == NULL) { @@ -2465,7 +2465,7 @@ builtin_vars_impl(PyObject *module, PyObject *object) d = _PyEval_GetFrameLocals(); } else { - if (_PyObject_LookupAttr(object, &_Py_ID(__dict__), &d) == 0) { + if (PyObject_GetOptionalAttr(object, &_Py_ID(__dict__), &d) == 0) { PyErr_SetString(PyExc_TypeError, "vars() argument must have __dict__ attribute"); } diff --git a/Python/ceval.c b/Python/ceval.c index 57e478c1313f6..63150a9456bbc 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -419,7 +419,7 @@ match_class_attr(PyThreadState *tstate, PyObject *subject, PyObject *type, return NULL; } PyObject *attr; - (void)_PyObject_LookupAttr(subject, name, &attr); + (void)PyObject_GetOptionalAttr(subject, name, &attr); return attr; } @@ -454,7 +454,7 @@ match_class(PyThreadState *tstate, PyObject *subject, PyObject *type, // First, the positional subpatterns: if (nargs) { int match_self = 0; - if (_PyObject_LookupAttr(type, &_Py_ID(__match_args__), &match_args) < 0) { + if (PyObject_GetOptionalAttr(type, &_Py_ID(__match_args__), &match_args) < 0) { goto fail; } if (match_args) { @@ -2414,7 +2414,7 @@ import_from(PyThreadState *tstate, PyObject *v, PyObject *name) PyObject *x; PyObject *fullmodname, *pkgname, *pkgpath, *pkgname_or_unknown, *errmsg; - if (_PyObject_LookupAttr(v, name, &x) != 0) { + if (PyObject_GetOptionalAttr(v, name, &x) != 0) { return x; } /* Issue #17636: in case this failed because of a circular relative diff --git a/Python/codecs.c b/Python/codecs.c index f9f23005debb1..4e47ff93a3691 100644 --- a/Python/codecs.c +++ b/Python/codecs.c @@ -516,7 +516,7 @@ PyObject * _PyCodec_LookupTextEncoding(const char *encoding, * attribute. */ if (!PyTuple_CheckExact(codec)) { - if (_PyObject_LookupAttr(codec, &_Py_ID(_is_text_encoding), &attr) < 0) { + if (PyObject_GetOptionalAttr(codec, &_Py_ID(_is_text_encoding), &attr) < 0) { Py_DECREF(codec); return NULL; } diff --git a/Python/errors.c b/Python/errors.c index b1a9858d82f7d..916958c9a0ab0 100644 --- a/Python/errors.c +++ b/Python/errors.c @@ -1748,7 +1748,7 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, } } if ((PyObject *)Py_TYPE(exc) != PyExc_SyntaxError) { - if (_PyObject_LookupAttr(exc, &_Py_ID(msg), &tmp) < 0) { + if (PyObject_GetOptionalAttr(exc, &_Py_ID(msg), &tmp) < 0) { _PyErr_Clear(tstate); } else if (tmp) { @@ -1767,7 +1767,7 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, } } - if (_PyObject_LookupAttr(exc, &_Py_ID(print_file_and_line), &tmp) < 0) { + if (PyObject_GetOptionalAttr(exc, &_Py_ID(print_file_and_line), &tmp) < 0) { _PyErr_Clear(tstate); } else if (tmp) { diff --git a/Python/import.c b/Python/import.c index 7ce89cdf9a91a..3e52a4e4eb145 100644 --- a/Python/import.c +++ b/Python/import.c @@ -2878,7 +2878,7 @@ PyImport_ImportModuleLevelObject(PyObject *name, PyObject *globals, } else { PyObject *path; - if (_PyObject_LookupAttr(mod, &_Py_ID(__path__), &path) < 0) { + if (PyObject_GetOptionalAttr(mod, &_Py_ID(__path__), &path) < 0) { goto error; } if (path) { diff --git a/Python/intrinsics.c b/Python/intrinsics.c index 82f8cce7f92a7..037b74ca820fa 100644 --- a/Python/intrinsics.c +++ b/Python/intrinsics.c @@ -40,11 +40,11 @@ import_all_from(PyThreadState *tstate, PyObject *locals, PyObject *v) int skip_leading_underscores = 0; int pos, err; - if (_PyObject_LookupAttr(v, &_Py_ID(__all__), &all) < 0) { + if (PyObject_GetOptionalAttr(v, &_Py_ID(__all__), &all) < 0) { return -1; /* Unexpected error */ } if (all == NULL) { - if (_PyObject_LookupAttr(v, &_Py_ID(__dict__), &dict) < 0) { + if (PyObject_GetOptionalAttr(v, &_Py_ID(__dict__), &dict) < 0) { return -1; } if (dict == NULL) { diff --git a/Python/pythonrun.c b/Python/pythonrun.c index 60bf66c8a6bf8..721c527745c44 100644 --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -953,7 +953,7 @@ print_exception_file_and_line(struct exception_print_context *ctx, PyObject *f = ctx->file; PyObject *tmp; - int res = _PyObject_LookupAttr(*value_p, &_Py_ID(print_file_and_line), &tmp); + int res = PyObject_GetOptionalAttr(*value_p, &_Py_ID(print_file_and_line), &tmp); if (res <= 0) { if (res < 0) { PyErr_Clear(); @@ -1132,7 +1132,7 @@ print_exception_notes(struct exception_print_context *ctx, PyObject *value) } PyObject *notes; - int res = _PyObject_LookupAttr(value, &_Py_ID(__notes__), ¬es); + int res = PyObject_GetOptionalAttr(value, &_Py_ID(__notes__), ¬es); if (res <= 0) { return res; } diff --git a/Python/suggestions.c b/Python/suggestions.c index ad58393490efc..47aeb08180f6b 100644 --- a/Python/suggestions.c +++ b/Python/suggestions.c @@ -247,7 +247,7 @@ get_suggestions_for_name_error(PyObject* name, PyFrameObject* frame) } PyObject *value; - res = _PyObject_LookupAttr(self, name, &value); + res = PyObject_GetOptionalAttr(self, name, &value); Py_DECREF(locals); if (res < 0) { goto error; diff --git a/Python/sysmodule.c b/Python/sysmodule.c index 0ac6edc5a16f8..fea3f61ee0176 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -257,7 +257,7 @@ sys_audit_tstate(PyThreadState *ts, const char *event, PyThreadState_EnterTracing(ts); while ((hook = PyIter_Next(hooks)) != NULL) { PyObject *o; - int canTrace = _PyObject_LookupAttr(hook, &_Py_ID(__cantrace__), &o); + int canTrace = PyObject_GetOptionalAttr(hook, &_Py_ID(__cantrace__), &o); if (o) { canTrace = PyObject_IsTrue(o); Py_DECREF(o); @@ -657,7 +657,7 @@ sys_displayhook_unencodable(PyObject *outf, PyObject *o) if (encoded == NULL) goto error; - if (_PyObject_LookupAttr(outf, &_Py_ID(buffer), &buffer) < 0) { + if (PyObject_GetOptionalAttr(outf, &_Py_ID(buffer), &buffer) < 0) { Py_DECREF(encoded); goto error; } From webhook-mailer at python.org Wed Jul 12 03:32:54 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Wed, 12 Jul 2023 07:32:54 -0000 Subject: [Python-checkins] [3.12] gh-103186: In test_tools.freeze, fetch CONFIG_ARGS from original source directory (GH-103213) (#106667) Message-ID: https://github.com/python/cpython/commit/af06a8ad4d94f78d86d59a6268b3f38543921beb commit: af06a8ad4d94f78d86d59a6268b3f38543921beb branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-12T07:32:51Z summary: [3.12] gh-103186: In test_tools.freeze, fetch CONFIG_ARGS from original source directory (GH-103213) (#106667) gh-103186: In test_tools.freeze, fetch CONFIG_ARGS from original source directory (GH-103213) Fetch CONFIG_ARGS from the original source directory, instead of from the copied source tree. When "make clean" is executed in the copied source tree, the build directory is cleared and the configure argument lookup fails. However, the original source directory still contains this information. (cherry picked from commit de827322ca47e51d52ff44536a7c3fd44648383a) Co-authored-by: Ijtaba Hussain files: A Misc/NEWS.d/next/Tools-Demos/2023-04-05-07-19-36.gh-issue-103186.yEozgK.rst M Tools/freeze/test/freeze.py diff --git a/Misc/NEWS.d/next/Tools-Demos/2023-04-05-07-19-36.gh-issue-103186.yEozgK.rst b/Misc/NEWS.d/next/Tools-Demos/2023-04-05-07-19-36.gh-issue-103186.yEozgK.rst new file mode 100644 index 0000000000000..7e28ba6963216 --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2023-04-05-07-19-36.gh-issue-103186.yEozgK.rst @@ -0,0 +1,2 @@ +``freeze`` now fetches ``CONFIG_ARGS`` from the original CPython instance +the Makefile uses to call utility scripts. Patch by Ijtaba Hussain. diff --git a/Tools/freeze/test/freeze.py b/Tools/freeze/test/freeze.py index f6a5adb4519fd..92e97cb261719 100644 --- a/Tools/freeze/test/freeze.py +++ b/Tools/freeze/test/freeze.py @@ -153,7 +153,7 @@ def prepare(script=None, outdir=None): print(f'configuring python in {builddir}...') cmd = [ os.path.join(srcdir, 'configure'), - *shlex.split(get_config_var(srcdir, 'CONFIG_ARGS') or ''), + *shlex.split(get_config_var(SRCDIR, 'CONFIG_ARGS') or ''), ] ensure_opt(cmd, 'cache-file', os.path.join(outdir, 'python-config.cache')) prefix = os.path.join(outdir, 'python-installation') From webhook-mailer at python.org Wed Jul 12 06:28:15 2023 From: webhook-mailer at python.org (vstinner) Date: Wed, 12 Jul 2023 10:28:15 -0000 Subject: [Python-checkins] gh-105373: Elaborate Pending Removal in What's New in Python 3.13 (#106675) Message-ID: https://github.com/python/cpython/commit/42bc485a24f0f67751474bd1697e1beb4a783b9a commit: 42bc485a24f0f67751474bd1697e1beb4a783b9a branch: main author: Victor Stinner committer: vstinner date: 2023-07-12T12:28:10+02:00 summary: gh-105373: Elaborate Pending Removal in What's New in Python 3.13 (#106675) Co-authored-by: Hugo van Kemenade files: M Doc/whatsnew/3.13.rst diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index dc020682ac1ed..b7c436fc15161 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -270,7 +270,9 @@ Pending Removal in Python 3.14 * :mod:`typing`: :class:`~typing.ByteString`, deprecated since Python 3.9, now causes a :exc:`DeprecationWarning` to be emitted when it is used. -* :class:`!urllib.parse.Quoter`. +* :class:`!urllib.parse.Quoter` is deprecated: it was not intended to be a + public API. + (Contributed by Gregory P. Smith in :gh:`88168`.) * :mod:`xml.etree.ElementTree`: Testing the truth value of an :class:`~xml.etree.ElementTree.Element` is deprecated and will raise an @@ -397,7 +399,12 @@ although there is currently no date scheduled for their removal. * :class:`!pydoc.ErrorDuringImport`: A tuple value for *exc_info* parameter is deprecated, use an exception instance. -* :mod:`re`: bad character in group name. +* :mod:`re`: More strict rules are now applied for numerical group references + and group names in regular expressions. Only sequence of ASCII digits is now + accepted as a numerical reference. The group name in bytes patterns and + replacement strings can now only contain ASCII letters and digits and + underscore. + (Contributed by Serhiy Storchaka in :gh:`91760`.) * :mod:`ssl` options and protocols: From webhook-mailer at python.org Wed Jul 12 06:30:29 2023 From: webhook-mailer at python.org (iritkatriel) Date: Wed, 12 Jul 2023 10:30:29 -0000 Subject: [Python-checkins] gh-105481: move Python/opcode_metadata.h to Include/internal/pycore_opcode_metadata.h (#106673) Message-ID: https://github.com/python/cpython/commit/2ca008e2b738b8c08b4bf46b2b23f315d6510d92 commit: 2ca008e2b738b8c08b4bf46b2b23f315d6510d92 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-12T11:30:25+01:00 summary: gh-105481: move Python/opcode_metadata.h to Include/internal/pycore_opcode_metadata.h (#106673) files: A Include/internal/pycore_opcode_metadata.h D Python/opcode_metadata.h M .gitattributes M Makefile.pre.in M Python/assemble.c M Python/ceval.c M Python/compile.c M Python/flowgraph.c M Python/optimizer.c M Tools/cases_generator/generate_cases.py diff --git a/.gitattributes b/.gitattributes index 5e4ce963b63e5..2616da74b48c0 100644 --- a/.gitattributes +++ b/.gitattributes @@ -87,7 +87,7 @@ Programs/test_frozenmain.h generated Python/Python-ast.c generated Python/executor_cases.c.h generated Python/generated_cases.c.h generated -Python/opcode_metadata.h generated +Include/internal/pycore_opcode_metadata.h generated Python/opcode_targets.h generated Python/stdlib_module_names.h generated Tools/peg_generator/pegen/grammar_parser.py generated diff --git a/Python/opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h similarity index 100% rename from Python/opcode_metadata.h rename to Include/internal/pycore_opcode_metadata.h diff --git a/Makefile.pre.in b/Makefile.pre.in index 073b4bcc271ff..a1ceedbd4812c 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1548,30 +1548,30 @@ regen-cases: $(srcdir)/Tools/cases_generator/generate_cases.py \ --emit-line-directives \ -o $(srcdir)/Python/generated_cases.c.h.new \ - -m $(srcdir)/Python/opcode_metadata.h.new \ + -m $(srcdir)/Include/internal/pycore_opcode_metadata.h.new \ -e $(srcdir)/Python/executor_cases.c.h.new \ -p $(srcdir)/Lib/_opcode_metadata.py.new \ $(srcdir)/Python/bytecodes.c $(UPDATE_FILE) $(srcdir)/Python/generated_cases.c.h $(srcdir)/Python/generated_cases.c.h.new - $(UPDATE_FILE) $(srcdir)/Python/opcode_metadata.h $(srcdir)/Python/opcode_metadata.h.new + $(UPDATE_FILE) $(srcdir)/Include/internal/pycore_opcode_metadata.h $(srcdir)/Include/internal/pycore_opcode_metadata.h.new $(UPDATE_FILE) $(srcdir)/Python/executor_cases.c.h $(srcdir)/Python/executor_cases.c.h.new $(UPDATE_FILE) $(srcdir)/Lib/_opcode_metadata.py $(srcdir)/Lib/_opcode_metadata.py.new -Python/compile.o: $(srcdir)/Python/opcode_metadata.h +Python/compile.o: $(srcdir)/Include/internal/pycore_opcode_metadata.h Python/ceval.o: \ $(srcdir)/Python/ceval_macros.h \ $(srcdir)/Python/condvar.h \ $(srcdir)/Python/generated_cases.c.h \ - $(srcdir)/Python/opcode_metadata.h \ + $(srcdir)/Include/internal/pycore_opcode_metadata.h \ $(srcdir)/Python/opcode_targets.h Python/flowgraph.o: \ - $(srcdir)/Python/opcode_metadata.h + $(srcdir)/Include/internal/pycore_opcode_metadata.h Python/optimizer.o: \ $(srcdir)/Python/executor_cases.c.h \ - $(srcdir)/Python/opcode_metadata.h + $(srcdir)/Include/internal/pycore_opcode_metadata.h Python/frozen.o: $(FROZEN_FILES_OUT) diff --git a/Python/assemble.c b/Python/assemble.c index ff7bca22286cd..b7012534d6cc4 100644 --- a/Python/assemble.c +++ b/Python/assemble.c @@ -1,11 +1,11 @@ #include #include "Python.h" -#include "pycore_code.h" // write_location_entry_start() +#include "pycore_code.h" // write_location_entry_start() #include "pycore_compile.h" -#include "pycore_opcode.h" // _PyOpcode_Caches[] and opcode category macros -#include "pycore_opcode_utils.h" // IS_BACKWARDS_JUMP_OPCODE -#include "opcode_metadata.h" // IS_PSEUDO_INSTR +#include "pycore_opcode.h" // _PyOpcode_Caches[] and opcode category macros +#include "pycore_opcode_utils.h" // IS_BACKWARDS_JUMP_OPCODE +#include "pycore_opcode_metadata.h" // IS_PSEUDO_INSTR #define DEFAULT_CODE_SIZE 128 diff --git a/Python/ceval.c b/Python/ceval.c index 63150a9456bbc..de44085d732cf 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -14,6 +14,7 @@ #include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_moduleobject.h" // PyModuleObject #include "pycore_opcode.h" // EXTRA_CASES +#include "pycore_opcode_metadata.h" #include "pycore_opcode_utils.h" // MAKE_FUNCTION_* #include "pycore_pyerrors.h" // _PyErr_GetRaisedException() #include "pycore_pystate.h" // _PyInterpreterState_GET() @@ -30,7 +31,6 @@ #include "pycore_frame.h" #include "frameobject.h" // _PyInterpreterFrame_GetLine #include "opcode.h" -#include "opcode_metadata.h" #include "pydtrace.h" #include "setobject.h" #include "structmember.h" // struct PyMemberDef, T_OFFSET_EX diff --git a/Python/compile.c b/Python/compile.c index 29ea2742fad0c..d9e38cfdefaf2 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -36,7 +36,7 @@ #include "pycore_pystate.h" // _Py_GetConfig() #include "pycore_symtable.h" // PySTEntryObject, _PyFuture_FromAST() -#include "opcode_metadata.h" // _PyOpcode_opcode_metadata, _PyOpcode_num_popped/pushed +#include "pycore_opcode_metadata.h" // _PyOpcode_opcode_metadata, _PyOpcode_num_popped/pushed #define COMP_GENEXP 0 #define COMP_LISTCOMP 1 diff --git a/Python/flowgraph.c b/Python/flowgraph.c index e159a4356dfe4..04f269c585383 100644 --- a/Python/flowgraph.c +++ b/Python/flowgraph.c @@ -8,7 +8,7 @@ #include "pycore_opcode_utils.h" #define NEED_OPCODE_METADATA -#include "opcode_metadata.h" // _PyOpcode_opcode_metadata, _PyOpcode_num_popped/pushed +#include "pycore_opcode_metadata.h" // _PyOpcode_opcode_metadata, _PyOpcode_num_popped/pushed #undef NEED_OPCODE_METADATA diff --git a/Python/optimizer.c b/Python/optimizer.c index 7fc40e66057d3..c3fdee63a7ed4 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -2,7 +2,7 @@ #include "opcode.h" #include "pycore_interp.h" #include "pycore_opcode.h" -#include "opcode_metadata.h" +#include "pycore_opcode_metadata.h" #include "pycore_opcode_utils.h" #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_uops.h" diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 8c77c1f08335d..a20abcde85b7c 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -24,7 +24,7 @@ DEFAULT_INPUT = os.path.relpath(os.path.join(ROOT, "Python/bytecodes.c")) DEFAULT_OUTPUT = os.path.relpath(os.path.join(ROOT, "Python/generated_cases.c.h")) DEFAULT_METADATA_OUTPUT = os.path.relpath( - os.path.join(ROOT, "Python/opcode_metadata.h") + os.path.join(ROOT, "Include/internal/pycore_opcode_metadata.h") ) DEFAULT_PYMETADATA_OUTPUT = os.path.relpath( os.path.join(ROOT, "Lib/_opcode_metadata.py") From webhook-mailer at python.org Wed Jul 12 08:46:34 2023 From: webhook-mailer at python.org (zooba) Date: Wed, 12 Jul 2023 12:46:34 -0000 Subject: [Python-checkins] gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649) Message-ID: https://github.com/python/cpython/commit/e2d7366fb3df44e7434132636d49f22d6d25cc9f commit: e2d7366fb3df44e7434132636d49f22d6d25cc9f branch: main author: Steve Dower committer: zooba date: 2023-07-12T13:46:30+01:00 summary: gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649) files: A Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst M PCbuild/get_externals.bat M PCbuild/openssl.props M PCbuild/python.props M PCbuild/readme.txt M PCbuild/regen.targets diff --git a/Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst b/Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst new file mode 100644 index 0000000000000..11f411be0f17c --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst @@ -0,0 +1 @@ +Update Windows build to use OpenSSL 3.0.9 diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat index 3a6f616f27922..257b360ba3506 100644 --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -53,7 +53,7 @@ echo.Fetching external libraries... set libraries= set libraries=%libraries% bzip2-1.0.8 if NOT "%IncludeLibffiSrc%"=="false" set libraries=%libraries% libffi-3.4.4 -if NOT "%IncludeSSLSrc%"=="false" set libraries=%libraries% openssl-1.1.1u +if NOT "%IncludeSSLSrc%"=="false" set libraries=%libraries% openssl-3.0.9 set libraries=%libraries% sqlite-3.42.0.0 if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tcl-core-8.6.13.0 if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tk-8.6.13.0 @@ -76,7 +76,7 @@ echo.Fetching external binaries... set binaries= if NOT "%IncludeLibffi%"=="false" set binaries=%binaries% libffi-3.4.4 -if NOT "%IncludeSSL%"=="false" set binaries=%binaries% openssl-bin-1.1.1u +if NOT "%IncludeSSL%"=="false" set binaries=%binaries% openssl-bin-3.0.9 if NOT "%IncludeTkinter%"=="false" set binaries=%binaries% tcltk-8.6.13.0 if NOT "%IncludeSSLSrc%"=="false" set binaries=%binaries% nasm-2.11.06 diff --git a/PCbuild/openssl.props b/PCbuild/openssl.props index 7071bb57c06cd..5fd708b211e42 100644 --- a/PCbuild/openssl.props +++ b/PCbuild/openssl.props @@ -10,10 +10,10 @@ - <_DLLSuffix>-1_1 + <_DLLSuffix>-3 <_DLLSuffix Condition="$(Platform) == 'ARM'">$(_DLLSuffix)-arm <_DLLSuffix Condition="$(Platform) == 'ARM64'">$(_DLLSuffix)-arm64 - $(_DLLSuffix) + $(_DLLSuffix) <_SSLDLL Include="$(opensslOutDir)\libcrypto$(_DLLSuffix).dll" /> diff --git a/PCbuild/python.props b/PCbuild/python.props index 68052ef668aa6..d3586235c8265 100644 --- a/PCbuild/python.props +++ b/PCbuild/python.props @@ -74,8 +74,8 @@ $(ExternalsDir)libffi-3.4.4\ $(libffiDir)$(ArchName)\ $(libffiOutDir)include - $(ExternalsDir)openssl-1.1.1u\ - $(ExternalsDir)openssl-bin-1.1.1u\$(ArchName)\ + $(ExternalsDir)openssl-3.0.9\ + $(ExternalsDir)openssl-bin-3.0.9\$(ArchName)\ $(opensslOutDir)include $(ExternalsDir)\nasm-2.11.06\ $(ExternalsDir)\zlib-1.2.13\ diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt index 88b04a749e3b2..f0de142f0573b 100644 --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -168,7 +168,7 @@ _lzma Homepage: https://tukaani.org/xz/ _ssl - Python wrapper for version 1.1.1u of the OpenSSL secure sockets + Python wrapper for version 3.0 of the OpenSSL secure sockets library, which is downloaded from our binaries repository at https://github.com/python/cpython-bin-deps. diff --git a/PCbuild/regen.targets b/PCbuild/regen.targets index e9e16a15f94cc..2dd786e5e82e3 100644 --- a/PCbuild/regen.targets +++ b/PCbuild/regen.targets @@ -104,8 +104,9 @@ <_LicenseSources Include="$(PySourcePath)LICENSE; $(PySourcePath)PC\crtlicense.txt; $(bz2Dir)LICENSE; - $(opensslOutDir)LICENSE; $(libffiDir)LICENSE;" /> + <_LicenseSources Include="$(opensslOutDir)LICENSE.txt" Condition="Exists('$(opensslOutDir)LICENSE.txt')" /> + <_LicenseSources Include="$(opensslOutDir)LICENSE" Condition="!Exists('$(opensslOutDir)LICENSE.txt')" /> <_LicenseSources Include="$(tcltkDir)tcllicense.terms; $(tcltkDir)tklicense.terms" Condition="$(IncludeTkinter)" /> From webhook-mailer at python.org Wed Jul 12 09:34:18 2023 From: webhook-mailer at python.org (markshannon) Date: Wed, 12 Jul 2023 13:34:18 -0000 Subject: [Python-checkins] GH-104909: Break LOAD_GLOBAL specializations in micro-ops. (GH-106677) Message-ID: https://github.com/python/cpython/commit/b03755a2347325a89a48b08fc158419000513bcb commit: b03755a2347325a89a48b08fc158419000513bcb branch: main author: Mark Shannon committer: markshannon date: 2023-07-12T14:34:14+01:00 summary: GH-104909: Break LOAD_GLOBAL specializations in micro-ops. (GH-106677) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-12-10-48-08.gh-issue-104909.sWjcr2.rst M Include/internal/pycore_code.h M Include/internal/pycore_opcode_metadata.h M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index d1829eb3245d2..b6b1aeca6e5c5 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -19,9 +19,9 @@ extern "C" { typedef struct { uint16_t counter; - uint16_t index; uint16_t module_keys_version; uint16_t builtin_keys_version; + uint16_t index; } _PyLoadGlobalCache; #define INLINE_CACHE_ENTRIES_LOAD_GLOBAL CACHE_ENTRIES(_PyLoadGlobalCache) diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index 4a41cd86a4287..317f42afea804 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -36,7 +36,10 @@ #define _BINARY_OP_ADD_UNICODE 314 #define _LOAD_LOCALS 315 #define _LOAD_FROM_DICT_OR_GLOBALS 316 -#define IS_NONE 317 +#define _SKIP_CACHE 317 +#define _GUARD_GLOBALS_VERSION 318 +#define _GUARD_BUILTINS_VERSION 319 +#define IS_NONE 320 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -1312,6 +1315,9 @@ const char * const _PyOpcode_uop_name[512] = { [314] = "_BINARY_OP_ADD_UNICODE", [315] = "_LOAD_LOCALS", [316] = "_LOAD_FROM_DICT_OR_GLOBALS", - [317] = "IS_NONE", + [317] = "_SKIP_CACHE", + [318] = "_GUARD_GLOBALS_VERSION", + [319] = "_GUARD_BUILTINS_VERSION", + [320] = "IS_NONE", }; #endif // NEED_OPCODE_METADATA diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-12-10-48-08.gh-issue-104909.sWjcr2.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-12-10-48-08.gh-issue-104909.sWjcr2.rst new file mode 100644 index 0000000000000..f20226e5c54d1 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-12-10-48-08.gh-issue-104909.sWjcr2.rst @@ -0,0 +1 @@ +Split :opcode:`LOAD_GLOBAL` specializations into micro-ops. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 2f6b8c5ae2f9c..f5ce2e72d2676 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1351,11 +1351,25 @@ dummy_func( null = NULL; } - inst(LOAD_GLOBAL_MODULE, (unused/1, index/1, version/1, unused/1 -- null if (oparg & 1), res)) { - DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); + op(_SKIP_CACHE, (unused/1 -- )) { + } + + op(_GUARD_GLOBALS_VERSION, (version/1 --)) { PyDictObject *dict = (PyDictObject *)GLOBALS(); + DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); + DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); + assert(DK_IS_UNICODE(dict->ma_keys)); + } + + op(_GUARD_BUILTINS_VERSION, (version/1 --)) { + PyDictObject *dict = (PyDictObject *)BUILTINS(); + DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); assert(DK_IS_UNICODE(dict->ma_keys)); + } + + op(_LOAD_GLOBAL_MODULE, (index/1 -- null if (oparg & 1), res)) { + PyDictObject *dict = (PyDictObject *)GLOBALS(); PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(dict->ma_keys); res = entries[index].me_value; DEOPT_IF(res == NULL, LOAD_GLOBAL); @@ -1364,15 +1378,8 @@ dummy_func( null = NULL; } - inst(LOAD_GLOBAL_BUILTIN, (unused/1, index/1, mod_version/1, bltn_version/1 -- null if (oparg & 1), res)) { - DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); - DEOPT_IF(!PyDict_CheckExact(BUILTINS()), LOAD_GLOBAL); - PyDictObject *mdict = (PyDictObject *)GLOBALS(); + op(_LOAD_GLOBAL_BUILTINS, (index/1 -- null if (oparg & 1), res)) { PyDictObject *bdict = (PyDictObject *)BUILTINS(); - assert(opcode == LOAD_GLOBAL_BUILTIN); - DEOPT_IF(mdict->ma_keys->dk_version != mod_version, LOAD_GLOBAL); - DEOPT_IF(bdict->ma_keys->dk_version != bltn_version, LOAD_GLOBAL); - assert(DK_IS_UNICODE(bdict->ma_keys)); PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(bdict->ma_keys); res = entries[index].me_value; DEOPT_IF(res == NULL, LOAD_GLOBAL); @@ -1381,6 +1388,18 @@ dummy_func( null = NULL; } + macro(LOAD_GLOBAL_MODULE) = + _SKIP_CACHE + // Skip over the counter + _GUARD_GLOBALS_VERSION + + _SKIP_CACHE + // Skip over the builtins version + _LOAD_GLOBAL_MODULE; + + macro(LOAD_GLOBAL_BUILTIN) = + _SKIP_CACHE + // Skip over the counter + _GUARD_GLOBALS_VERSION + + _GUARD_BUILTINS_VERSION + + _LOAD_GLOBAL_BUILTINS; + inst(DELETE_FAST, (--)) { PyObject *v = GETLOCAL(oparg); ERROR_IF(v == NULL, unbound_local_error); diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 681efb97d89a5..94536ed8ce5a1 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1154,17 +1154,43 @@ break; } + case _SKIP_CACHE: { + break; + } + + case _GUARD_GLOBALS_VERSION: { + uint16_t version = (uint16_t)operand; + #line 1358 "Python/bytecodes.c" + PyDictObject *dict = (PyDictObject *)GLOBALS(); + DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); + DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); + assert(DK_IS_UNICODE(dict->ma_keys)); + #line 1169 "Python/executor_cases.c.h" + break; + } + + case _GUARD_BUILTINS_VERSION: { + uint16_t version = (uint16_t)operand; + #line 1365 "Python/bytecodes.c" + PyDictObject *dict = (PyDictObject *)BUILTINS(); + DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); + DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); + assert(DK_IS_UNICODE(dict->ma_keys)); + #line 1180 "Python/executor_cases.c.h" + break; + } + case DELETE_FAST: { - #line 1385 "Python/bytecodes.c" + #line 1404 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL); - #line 1163 "Python/executor_cases.c.h" + #line 1189 "Python/executor_cases.c.h" break; } case DELETE_DEREF: { - #line 1402 "Python/bytecodes.c" + #line 1421 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); // Can't use ERROR_IF here. @@ -1175,14 +1201,14 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 1179 "Python/executor_cases.c.h" + #line 1205 "Python/executor_cases.c.h" break; } case LOAD_FROM_DICT_OR_DEREF: { PyObject *class_dict = stack_pointer[-1]; PyObject *value; - #line 1415 "Python/bytecodes.c" + #line 1434 "Python/bytecodes.c" PyObject *name; assert(class_dict); assert(oparg >= 0 && oparg < _PyFrame_GetCode(frame)->co_nlocalsplus); @@ -1201,14 +1227,14 @@ } Py_INCREF(value); } - #line 1205 "Python/executor_cases.c.h" + #line 1231 "Python/executor_cases.c.h" stack_pointer[-1] = value; break; } case LOAD_DEREF: { PyObject *value; - #line 1436 "Python/bytecodes.c" + #line 1455 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { @@ -1216,7 +1242,7 @@ if (true) goto error; } Py_INCREF(value); - #line 1220 "Python/executor_cases.c.h" + #line 1246 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -1224,18 +1250,18 @@ case STORE_DEREF: { PyObject *v = stack_pointer[-1]; - #line 1446 "Python/bytecodes.c" + #line 1465 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 1233 "Python/executor_cases.c.h" + #line 1259 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case COPY_FREE_VARS: { - #line 1453 "Python/bytecodes.c" + #line 1472 "Python/bytecodes.c" /* Copy closure variables to free variables */ PyCodeObject *co = _PyFrame_GetCode(frame); assert(PyFunction_Check(frame->f_funcobj)); @@ -1246,22 +1272,22 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 1250 "Python/executor_cases.c.h" + #line 1276 "Python/executor_cases.c.h" break; } case BUILD_STRING: { PyObject **pieces = (stack_pointer - oparg); PyObject *str; - #line 1466 "Python/bytecodes.c" + #line 1485 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 1259 "Python/executor_cases.c.h" + #line 1285 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } - #line 1468 "Python/bytecodes.c" + #line 1487 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1265 "Python/executor_cases.c.h" + #line 1291 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -1271,10 +1297,10 @@ case BUILD_TUPLE: { PyObject **values = (stack_pointer - oparg); PyObject *tup; - #line 1472 "Python/bytecodes.c" + #line 1491 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1278 "Python/executor_cases.c.h" + #line 1304 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -1284,10 +1310,10 @@ case BUILD_LIST: { PyObject **values = (stack_pointer - oparg); PyObject *list; - #line 1477 "Python/bytecodes.c" + #line 1496 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1291 "Python/executor_cases.c.h" + #line 1317 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -1297,7 +1323,7 @@ case LIST_EXTEND: { PyObject *iterable = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 1482 "Python/bytecodes.c" + #line 1501 "Python/bytecodes.c" PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable); if (none_val == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) && @@ -1308,13 +1334,13 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 1312 "Python/executor_cases.c.h" + #line 1338 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 1493 "Python/bytecodes.c" + #line 1512 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val)); - #line 1318 "Python/executor_cases.c.h" + #line 1344 "Python/executor_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); break; @@ -1323,13 +1349,13 @@ case SET_UPDATE: { PyObject *iterable = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 1500 "Python/bytecodes.c" + #line 1519 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 1329 "Python/executor_cases.c.h" + #line 1355 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 1502 "Python/bytecodes.c" + #line 1521 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 1333 "Python/executor_cases.c.h" + #line 1359 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -1337,7 +1363,7 @@ case BUILD_SET: { PyObject **values = (stack_pointer - oparg); PyObject *set; - #line 1506 "Python/bytecodes.c" + #line 1525 "Python/bytecodes.c" set = PySet_New(NULL); if (set == NULL) goto error; @@ -1352,7 +1378,7 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 1356 "Python/executor_cases.c.h" + #line 1382 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -1362,7 +1388,7 @@ case BUILD_MAP: { PyObject **values = (stack_pointer - oparg*2); PyObject *map; - #line 1523 "Python/bytecodes.c" + #line 1542 "Python/bytecodes.c" map = _PyDict_FromItems( values, 2, values+1, 2, @@ -1370,13 +1396,13 @@ if (map == NULL) goto error; - #line 1374 "Python/executor_cases.c.h" + #line 1400 "Python/executor_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } - #line 1531 "Python/bytecodes.c" + #line 1550 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 1380 "Python/executor_cases.c.h" + #line 1406 "Python/executor_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -1384,7 +1410,7 @@ } case SETUP_ANNOTATIONS: { - #line 1535 "Python/bytecodes.c" + #line 1554 "Python/bytecodes.c" int err; PyObject *ann_dict; if (LOCALS() == NULL) { @@ -1422,7 +1448,7 @@ Py_DECREF(ann_dict); } } - #line 1426 "Python/executor_cases.c.h" + #line 1452 "Python/executor_cases.c.h" break; } @@ -1430,7 +1456,7 @@ PyObject *keys = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); PyObject *map; - #line 1575 "Python/bytecodes.c" + #line 1594 "Python/bytecodes.c" if (!PyTuple_CheckExact(keys) || PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1440,14 +1466,14 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 1444 "Python/executor_cases.c.h" + #line 1470 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); - #line 1585 "Python/bytecodes.c" + #line 1604 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 1451 "Python/executor_cases.c.h" + #line 1477 "Python/executor_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; break; @@ -1455,7 +1481,7 @@ case DICT_UPDATE: { PyObject *update = stack_pointer[-1]; - #line 1589 "Python/bytecodes.c" + #line 1608 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (PyDict_Update(dict, update) < 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { @@ -1463,12 +1489,12 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 1467 "Python/executor_cases.c.h" + #line 1493 "Python/executor_cases.c.h" Py_DECREF(update); - #line 1597 "Python/bytecodes.c" + #line 1616 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1472 "Python/executor_cases.c.h" + #line 1498 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1476,17 +1502,17 @@ case DICT_MERGE: { PyObject *update = stack_pointer[-1]; - #line 1603 "Python/bytecodes.c" + #line 1622 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 1485 "Python/executor_cases.c.h" + #line 1511 "Python/executor_cases.c.h" Py_DECREF(update); - #line 1608 "Python/bytecodes.c" + #line 1627 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1490 "Python/executor_cases.c.h" + #line 1516 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1495,13 +1521,13 @@ case MAP_ADD: { PyObject *value = stack_pointer[-1]; PyObject *key = stack_pointer[-2]; - #line 1614 "Python/bytecodes.c" + #line 1633 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack assert(PyDict_CheckExact(dict)); /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 1505 "Python/executor_cases.c.h" + #line 1531 "Python/executor_cases.c.h" STACK_SHRINK(2); break; } @@ -1512,20 +1538,20 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1697 "Python/bytecodes.c" + #line 1716 "Python/bytecodes.c" assert(!(oparg & 1)); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); STAT_INC(LOAD_SUPER_ATTR, hit); PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL); - #line 1523 "Python/executor_cases.c.h" + #line 1549 "Python/executor_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1704 "Python/bytecodes.c" + #line 1723 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; - #line 1529 "Python/executor_cases.c.h" + #line 1555 "Python/executor_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -1539,7 +1565,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2; PyObject *res; - #line 1708 "Python/bytecodes.c" + #line 1727 "Python/bytecodes.c" assert(oparg & 1); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); @@ -1562,7 +1588,7 @@ res = res2; res2 = NULL; } - #line 1566 "Python/executor_cases.c.h" + #line 1592 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; stack_pointer[-2] = res2; @@ -1574,7 +1600,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1749 "Python/bytecodes.c" + #line 1768 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1608,9 +1634,9 @@ NULL | meth | arg1 | ... | argN */ - #line 1612 "Python/executor_cases.c.h" + #line 1638 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1783 "Python/bytecodes.c" + #line 1802 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -1619,12 +1645,12 @@ else { /* Classic, pushes one value. */ res = PyObject_GetAttr(owner, name); - #line 1623 "Python/executor_cases.c.h" + #line 1649 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1792 "Python/bytecodes.c" + #line 1811 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } - #line 1628 "Python/executor_cases.c.h" + #line 1654 "Python/executor_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -1636,7 +1662,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2025 "Python/bytecodes.c" + #line 2044 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1649,10 +1675,10 @@ #endif /* ENABLE_SPECIALIZATION */ assert((oparg >> 5) <= Py_GE); res = PyObject_RichCompare(left, right, oparg >> 5); - #line 1653 "Python/executor_cases.c.h" + #line 1679 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2038 "Python/bytecodes.c" + #line 2057 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; if (oparg & 16) { int res_bool = PyObject_IsTrue(res); @@ -1660,7 +1686,7 @@ if (res_bool < 0) goto pop_2_error; res = res_bool ? Py_True : Py_False; } - #line 1664 "Python/executor_cases.c.h" + #line 1690 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1670,7 +1696,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2048 "Python/bytecodes.c" + #line 2067 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -1682,7 +1708,7 @@ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1686 "Python/executor_cases.c.h" + #line 1712 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1692,7 +1718,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2063 "Python/bytecodes.c" + #line 2082 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -1708,7 +1734,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1712 "Python/executor_cases.c.h" + #line 1738 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1718,7 +1744,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2082 "Python/bytecodes.c" + #line 2101 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -1731,7 +1757,7 @@ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1735 "Python/executor_cases.c.h" + #line 1761 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1741,14 +1767,14 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2097 "Python/bytecodes.c" + #line 2116 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 1747 "Python/executor_cases.c.h" + #line 1773 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2099 "Python/bytecodes.c" + #line 2118 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1752 "Python/executor_cases.c.h" + #line 1778 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1758,15 +1784,15 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2103 "Python/bytecodes.c" + #line 2122 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 1764 "Python/executor_cases.c.h" + #line 1790 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2105 "Python/bytecodes.c" + #line 2124 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; - #line 1770 "Python/executor_cases.c.h" + #line 1796 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1777,12 +1803,12 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 2110 "Python/bytecodes.c" + #line 2129 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 1783 "Python/executor_cases.c.h" + #line 1809 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2112 "Python/bytecodes.c" + #line 2131 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -1790,10 +1816,10 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 1794 "Python/executor_cases.c.h" + #line 1820 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2120 "Python/bytecodes.c" + #line 2139 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -1802,7 +1828,7 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 1806 "Python/executor_cases.c.h" + #line 1832 "Python/executor_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; break; @@ -1812,21 +1838,21 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2131 "Python/bytecodes.c" + #line 2150 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 1819 "Python/executor_cases.c.h" + #line 1845 "Python/executor_cases.c.h" Py_DECREF(right); - #line 2134 "Python/bytecodes.c" + #line 2153 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 1826 "Python/executor_cases.c.h" + #line 1852 "Python/executor_cases.c.h" Py_DECREF(right); - #line 2139 "Python/bytecodes.c" + #line 2158 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1830 "Python/executor_cases.c.h" + #line 1856 "Python/executor_cases.c.h" stack_pointer[-1] = b; break; } @@ -1834,17 +1860,17 @@ case IS_NONE: { PyObject *value = stack_pointer[-1]; PyObject *b; - #line 2218 "Python/bytecodes.c" + #line 2237 "Python/bytecodes.c" if (Py_IsNone(value)) { b = Py_True; } else { b = Py_False; - #line 1844 "Python/executor_cases.c.h" + #line 1870 "Python/executor_cases.c.h" Py_DECREF(value); - #line 2224 "Python/bytecodes.c" + #line 2243 "Python/bytecodes.c" } - #line 1848 "Python/executor_cases.c.h" + #line 1874 "Python/executor_cases.c.h" stack_pointer[-1] = b; break; } @@ -1852,13 +1878,13 @@ case GET_LEN: { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2241 "Python/bytecodes.c" + #line 2260 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 1862 "Python/executor_cases.c.h" + #line 1888 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; break; @@ -1869,16 +1895,16 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2249 "Python/bytecodes.c" + #line 2268 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 1878 "Python/executor_cases.c.h" + #line 1904 "Python/executor_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2254 "Python/bytecodes.c" + #line 2273 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -1886,7 +1912,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 1890 "Python/executor_cases.c.h" + #line 1916 "Python/executor_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; break; @@ -1895,10 +1921,10 @@ case MATCH_MAPPING: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2264 "Python/bytecodes.c" + #line 2283 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 1902 "Python/executor_cases.c.h" + #line 1928 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1907,10 +1933,10 @@ case MATCH_SEQUENCE: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2269 "Python/bytecodes.c" + #line 2288 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 1914 "Python/executor_cases.c.h" + #line 1940 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1920,11 +1946,11 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2274 "Python/bytecodes.c" + #line 2293 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 1928 "Python/executor_cases.c.h" + #line 1954 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; break; @@ -1933,14 +1959,14 @@ case GET_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2280 "Python/bytecodes.c" + #line 2299 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 1940 "Python/executor_cases.c.h" + #line 1966 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2283 "Python/bytecodes.c" + #line 2302 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 1944 "Python/executor_cases.c.h" + #line 1970 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -1948,7 +1974,7 @@ case GET_YIELD_FROM_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2287 "Python/bytecodes.c" + #line 2306 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -1971,11 +1997,11 @@ if (iter == NULL) { goto error; } - #line 1975 "Python/executor_cases.c.h" + #line 2001 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2310 "Python/bytecodes.c" + #line 2329 "Python/bytecodes.c" } - #line 1979 "Python/executor_cases.c.h" + #line 2005 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -1985,7 +2011,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2542 "Python/bytecodes.c" + #line 2561 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -2006,7 +2032,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 2010 "Python/executor_cases.c.h" + #line 2036 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -2015,7 +2041,7 @@ case PUSH_EXC_INFO: { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2581 "Python/bytecodes.c" + #line 2600 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -2025,7 +2051,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 2029 "Python/executor_cases.c.h" + #line 2055 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -2034,7 +2060,7 @@ case EXIT_INIT_CHECK: { PyObject *should_be_none = stack_pointer[-1]; - #line 2980 "Python/bytecodes.c" + #line 2999 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -2042,7 +2068,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 2046 "Python/executor_cases.c.h" + #line 2072 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -2050,7 +2076,7 @@ case MAKE_FUNCTION: { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3394 "Python/bytecodes.c" + #line 3413 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -2062,7 +2088,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 2066 "Python/executor_cases.c.h" + #line 2092 "Python/executor_cases.c.h" stack_pointer[-1] = func; break; } @@ -2070,7 +2096,7 @@ case SET_FUNCTION_ATTRIBUTE: { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3408 "Python/bytecodes.c" + #line 3427 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -2095,7 +2121,7 @@ default: Py_UNREACHABLE(); } - #line 2099 "Python/executor_cases.c.h" + #line 2125 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; break; @@ -2106,15 +2132,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3458 "Python/bytecodes.c" + #line 3477 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 2112 "Python/executor_cases.c.h" + #line 2138 "Python/executor_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3460 "Python/bytecodes.c" + #line 3479 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 2118 "Python/executor_cases.c.h" + #line 2144 "Python/executor_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -2124,14 +2150,14 @@ case CONVERT_VALUE: { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3464 "Python/bytecodes.c" + #line 3483 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 2135 "Python/executor_cases.c.h" + #line 2161 "Python/executor_cases.c.h" stack_pointer[-1] = result; break; } @@ -2139,7 +2165,7 @@ case FORMAT_SIMPLE: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3473 "Python/bytecodes.c" + #line 3492 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -2150,7 +2176,7 @@ else { res = value; } - #line 2154 "Python/executor_cases.c.h" + #line 2180 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -2159,12 +2185,12 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3486 "Python/bytecodes.c" + #line 3505 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 2168 "Python/executor_cases.c.h" + #line 2194 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -2173,10 +2199,10 @@ case COPY: { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3493 "Python/bytecodes.c" + #line 3512 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 2180 "Python/executor_cases.c.h" + #line 2206 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; break; @@ -2187,7 +2213,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3498 "Python/bytecodes.c" + #line 3517 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2202,12 +2228,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 2206 "Python/executor_cases.c.h" + #line 2232 "Python/executor_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3513 "Python/bytecodes.c" + #line 3532 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 2211 "Python/executor_cases.c.h" + #line 2237 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -2216,9 +2242,9 @@ case SWAP: { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3518 "Python/bytecodes.c" + #line 3537 "Python/bytecodes.c" assert(oparg >= 2); - #line 2222 "Python/executor_cases.c.h" + #line 2248 "Python/executor_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; break; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index eb3de5e5bca97..fcdf732315830 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -1896,71 +1896,103 @@ } TARGET(LOAD_GLOBAL_MODULE) { - PyObject *null = NULL; - PyObject *res; - uint16_t index = read_u16(&next_instr[1].cache); - uint16_t version = read_u16(&next_instr[2].cache); - #line 1355 "Python/bytecodes.c" - DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); - PyDictObject *dict = (PyDictObject *)GLOBALS(); - DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); - assert(DK_IS_UNICODE(dict->ma_keys)); - PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(dict->ma_keys); - res = entries[index].me_value; - DEOPT_IF(res == NULL, LOAD_GLOBAL); - Py_INCREF(res); - STAT_INC(LOAD_GLOBAL, hit); - null = NULL; - #line 1915 "Python/generated_cases.c.h" + PyObject *_tmp_1; + PyObject *_tmp_2; + { + } + { + uint16_t version = read_u16(&next_instr[1].cache); + #line 1358 "Python/bytecodes.c" + PyDictObject *dict = (PyDictObject *)GLOBALS(); + DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); + DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); + assert(DK_IS_UNICODE(dict->ma_keys)); + #line 1911 "Python/generated_cases.c.h" + } + { + } + { + PyObject *null = NULL; + PyObject *res; + uint16_t index = read_u16(&next_instr[3].cache); + #line 1372 "Python/bytecodes.c" + PyDictObject *dict = (PyDictObject *)GLOBALS(); + PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(dict->ma_keys); + res = entries[index].me_value; + DEOPT_IF(res == NULL, LOAD_GLOBAL); + Py_INCREF(res); + STAT_INC(LOAD_GLOBAL, hit); + null = NULL; + #line 1927 "Python/generated_cases.c.h" + if (oparg & 1) { _tmp_2 = null; } + _tmp_1 = res; + } + next_instr += 4; STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); - stack_pointer[-1] = res; - if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = null; } - next_instr += 4; + stack_pointer[-1] = _tmp_1; + if (oparg & 1) { stack_pointer[-2] = _tmp_2; } DISPATCH(); } TARGET(LOAD_GLOBAL_BUILTIN) { - PyObject *null = NULL; - PyObject *res; - uint16_t index = read_u16(&next_instr[1].cache); - uint16_t mod_version = read_u16(&next_instr[2].cache); - uint16_t bltn_version = read_u16(&next_instr[3].cache); - #line 1368 "Python/bytecodes.c" - DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); - DEOPT_IF(!PyDict_CheckExact(BUILTINS()), LOAD_GLOBAL); - PyDictObject *mdict = (PyDictObject *)GLOBALS(); - PyDictObject *bdict = (PyDictObject *)BUILTINS(); - assert(opcode == LOAD_GLOBAL_BUILTIN); - DEOPT_IF(mdict->ma_keys->dk_version != mod_version, LOAD_GLOBAL); - DEOPT_IF(bdict->ma_keys->dk_version != bltn_version, LOAD_GLOBAL); - assert(DK_IS_UNICODE(bdict->ma_keys)); - PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(bdict->ma_keys); - res = entries[index].me_value; - DEOPT_IF(res == NULL, LOAD_GLOBAL); - Py_INCREF(res); - STAT_INC(LOAD_GLOBAL, hit); - null = NULL; - #line 1945 "Python/generated_cases.c.h" + PyObject *_tmp_1; + PyObject *_tmp_2; + { + } + { + uint16_t version = read_u16(&next_instr[1].cache); + #line 1358 "Python/bytecodes.c" + PyDictObject *dict = (PyDictObject *)GLOBALS(); + DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); + DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); + assert(DK_IS_UNICODE(dict->ma_keys)); + #line 1951 "Python/generated_cases.c.h" + } + { + uint16_t version = read_u16(&next_instr[2].cache); + #line 1365 "Python/bytecodes.c" + PyDictObject *dict = (PyDictObject *)BUILTINS(); + DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); + DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); + assert(DK_IS_UNICODE(dict->ma_keys)); + #line 1960 "Python/generated_cases.c.h" + } + { + PyObject *null = NULL; + PyObject *res; + uint16_t index = read_u16(&next_instr[3].cache); + #line 1382 "Python/bytecodes.c" + PyDictObject *bdict = (PyDictObject *)BUILTINS(); + PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(bdict->ma_keys); + res = entries[index].me_value; + DEOPT_IF(res == NULL, LOAD_GLOBAL); + Py_INCREF(res); + STAT_INC(LOAD_GLOBAL, hit); + null = NULL; + #line 1974 "Python/generated_cases.c.h" + if (oparg & 1) { _tmp_2 = null; } + _tmp_1 = res; + } + next_instr += 4; STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); - stack_pointer[-1] = res; - if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = null; } - next_instr += 4; + stack_pointer[-1] = _tmp_1; + if (oparg & 1) { stack_pointer[-2] = _tmp_2; } DISPATCH(); } TARGET(DELETE_FAST) { - #line 1385 "Python/bytecodes.c" + #line 1404 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL); - #line 1959 "Python/generated_cases.c.h" + #line 1991 "Python/generated_cases.c.h" DISPATCH(); } TARGET(MAKE_CELL) { - #line 1391 "Python/bytecodes.c" + #line 1410 "Python/bytecodes.c" // "initial" is probably NULL but not if it's an arg (or set // via PyFrame_LocalsToFast() before MAKE_CELL has run). PyObject *initial = GETLOCAL(oparg); @@ -1969,12 +2001,12 @@ goto resume_with_error; } SETLOCAL(oparg, cell); - #line 1973 "Python/generated_cases.c.h" + #line 2005 "Python/generated_cases.c.h" DISPATCH(); } TARGET(DELETE_DEREF) { - #line 1402 "Python/bytecodes.c" + #line 1421 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); // Can't use ERROR_IF here. @@ -1985,14 +2017,14 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 1989 "Python/generated_cases.c.h" + #line 2021 "Python/generated_cases.c.h" DISPATCH(); } TARGET(LOAD_FROM_DICT_OR_DEREF) { PyObject *class_dict = stack_pointer[-1]; PyObject *value; - #line 1415 "Python/bytecodes.c" + #line 1434 "Python/bytecodes.c" PyObject *name; assert(class_dict); assert(oparg >= 0 && oparg < _PyFrame_GetCode(frame)->co_nlocalsplus); @@ -2011,14 +2043,14 @@ } Py_INCREF(value); } - #line 2015 "Python/generated_cases.c.h" + #line 2047 "Python/generated_cases.c.h" stack_pointer[-1] = value; DISPATCH(); } TARGET(LOAD_DEREF) { PyObject *value; - #line 1436 "Python/bytecodes.c" + #line 1455 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { @@ -2026,7 +2058,7 @@ if (true) goto error; } Py_INCREF(value); - #line 2030 "Python/generated_cases.c.h" + #line 2062 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -2034,18 +2066,18 @@ TARGET(STORE_DEREF) { PyObject *v = stack_pointer[-1]; - #line 1446 "Python/bytecodes.c" + #line 1465 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 2043 "Python/generated_cases.c.h" + #line 2075 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(COPY_FREE_VARS) { - #line 1453 "Python/bytecodes.c" + #line 1472 "Python/bytecodes.c" /* Copy closure variables to free variables */ PyCodeObject *co = _PyFrame_GetCode(frame); assert(PyFunction_Check(frame->f_funcobj)); @@ -2056,22 +2088,22 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 2060 "Python/generated_cases.c.h" + #line 2092 "Python/generated_cases.c.h" DISPATCH(); } TARGET(BUILD_STRING) { PyObject **pieces = (stack_pointer - oparg); PyObject *str; - #line 1466 "Python/bytecodes.c" + #line 1485 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 2069 "Python/generated_cases.c.h" + #line 2101 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } - #line 1468 "Python/bytecodes.c" + #line 1487 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2075 "Python/generated_cases.c.h" + #line 2107 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -2081,10 +2113,10 @@ TARGET(BUILD_TUPLE) { PyObject **values = (stack_pointer - oparg); PyObject *tup; - #line 1472 "Python/bytecodes.c" + #line 1491 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2088 "Python/generated_cases.c.h" + #line 2120 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -2094,10 +2126,10 @@ TARGET(BUILD_LIST) { PyObject **values = (stack_pointer - oparg); PyObject *list; - #line 1477 "Python/bytecodes.c" + #line 1496 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2101 "Python/generated_cases.c.h" + #line 2133 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -2107,7 +2139,7 @@ TARGET(LIST_EXTEND) { PyObject *iterable = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 1482 "Python/bytecodes.c" + #line 1501 "Python/bytecodes.c" PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable); if (none_val == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) && @@ -2118,13 +2150,13 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 2122 "Python/generated_cases.c.h" + #line 2154 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1493 "Python/bytecodes.c" + #line 1512 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val)); - #line 2128 "Python/generated_cases.c.h" + #line 2160 "Python/generated_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); DISPATCH(); @@ -2133,13 +2165,13 @@ TARGET(SET_UPDATE) { PyObject *iterable = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 1500 "Python/bytecodes.c" + #line 1519 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 2139 "Python/generated_cases.c.h" + #line 2171 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1502 "Python/bytecodes.c" + #line 1521 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 2143 "Python/generated_cases.c.h" + #line 2175 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -2147,7 +2179,7 @@ TARGET(BUILD_SET) { PyObject **values = (stack_pointer - oparg); PyObject *set; - #line 1506 "Python/bytecodes.c" + #line 1525 "Python/bytecodes.c" set = PySet_New(NULL); if (set == NULL) goto error; @@ -2162,7 +2194,7 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 2166 "Python/generated_cases.c.h" + #line 2198 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -2172,7 +2204,7 @@ TARGET(BUILD_MAP) { PyObject **values = (stack_pointer - oparg*2); PyObject *map; - #line 1523 "Python/bytecodes.c" + #line 1542 "Python/bytecodes.c" map = _PyDict_FromItems( values, 2, values+1, 2, @@ -2180,13 +2212,13 @@ if (map == NULL) goto error; - #line 2184 "Python/generated_cases.c.h" + #line 2216 "Python/generated_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } - #line 1531 "Python/bytecodes.c" + #line 1550 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 2190 "Python/generated_cases.c.h" + #line 2222 "Python/generated_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -2194,7 +2226,7 @@ } TARGET(SETUP_ANNOTATIONS) { - #line 1535 "Python/bytecodes.c" + #line 1554 "Python/bytecodes.c" int err; PyObject *ann_dict; if (LOCALS() == NULL) { @@ -2232,7 +2264,7 @@ Py_DECREF(ann_dict); } } - #line 2236 "Python/generated_cases.c.h" + #line 2268 "Python/generated_cases.c.h" DISPATCH(); } @@ -2240,7 +2272,7 @@ PyObject *keys = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); PyObject *map; - #line 1575 "Python/bytecodes.c" + #line 1594 "Python/bytecodes.c" if (!PyTuple_CheckExact(keys) || PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -2250,14 +2282,14 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 2254 "Python/generated_cases.c.h" + #line 2286 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); - #line 1585 "Python/bytecodes.c" + #line 1604 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 2261 "Python/generated_cases.c.h" + #line 2293 "Python/generated_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; DISPATCH(); @@ -2265,7 +2297,7 @@ TARGET(DICT_UPDATE) { PyObject *update = stack_pointer[-1]; - #line 1589 "Python/bytecodes.c" + #line 1608 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (PyDict_Update(dict, update) < 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { @@ -2273,12 +2305,12 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 2277 "Python/generated_cases.c.h" + #line 2309 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1597 "Python/bytecodes.c" + #line 1616 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 2282 "Python/generated_cases.c.h" + #line 2314 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); DISPATCH(); @@ -2286,17 +2318,17 @@ TARGET(DICT_MERGE) { PyObject *update = stack_pointer[-1]; - #line 1603 "Python/bytecodes.c" + #line 1622 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 2295 "Python/generated_cases.c.h" + #line 2327 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1608 "Python/bytecodes.c" + #line 1627 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 2300 "Python/generated_cases.c.h" + #line 2332 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); DISPATCH(); @@ -2305,25 +2337,25 @@ TARGET(MAP_ADD) { PyObject *value = stack_pointer[-1]; PyObject *key = stack_pointer[-2]; - #line 1614 "Python/bytecodes.c" + #line 1633 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack assert(PyDict_CheckExact(dict)); /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 2315 "Python/generated_cases.c.h" + #line 2347 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } TARGET(INSTRUMENTED_LOAD_SUPER_ATTR) { - #line 1622 "Python/bytecodes.c" + #line 1641 "Python/bytecodes.c" _PySuperAttrCache *cache = (_PySuperAttrCache *)next_instr; // cancel out the decrement that will happen in LOAD_SUPER_ATTR; we // don't want to specialize instrumented instructions INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(LOAD_SUPER_ATTR); - #line 2327 "Python/generated_cases.c.h" + #line 2359 "Python/generated_cases.c.h" } TARGET(LOAD_SUPER_ATTR) { @@ -2334,7 +2366,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1636 "Python/bytecodes.c" + #line 1655 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); int load_method = oparg & 1; #if ENABLE_SPECIALIZATION @@ -2376,16 +2408,16 @@ } } } - #line 2380 "Python/generated_cases.c.h" + #line 2412 "Python/generated_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1678 "Python/bytecodes.c" + #line 1697 "Python/bytecodes.c" if (super == NULL) goto pop_3_error; res = PyObject_GetAttr(super, name); Py_DECREF(super); if (res == NULL) goto pop_3_error; - #line 2389 "Python/generated_cases.c.h" + #line 2421 "Python/generated_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2400,20 +2432,20 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1697 "Python/bytecodes.c" + #line 1716 "Python/bytecodes.c" assert(!(oparg & 1)); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); STAT_INC(LOAD_SUPER_ATTR, hit); PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL); - #line 2411 "Python/generated_cases.c.h" + #line 2443 "Python/generated_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1704 "Python/bytecodes.c" + #line 1723 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; - #line 2417 "Python/generated_cases.c.h" + #line 2449 "Python/generated_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2428,7 +2460,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2; PyObject *res; - #line 1708 "Python/bytecodes.c" + #line 1727 "Python/bytecodes.c" assert(oparg & 1); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); @@ -2451,7 +2483,7 @@ res = res2; res2 = NULL; } - #line 2455 "Python/generated_cases.c.h" + #line 2487 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; stack_pointer[-2] = res2; @@ -2465,7 +2497,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1749 "Python/bytecodes.c" + #line 1768 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2499,9 +2531,9 @@ NULL | meth | arg1 | ... | argN */ - #line 2503 "Python/generated_cases.c.h" + #line 2535 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1783 "Python/bytecodes.c" + #line 1802 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -2510,12 +2542,12 @@ else { /* Classic, pushes one value. */ res = PyObject_GetAttr(owner, name); - #line 2514 "Python/generated_cases.c.h" + #line 2546 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1792 "Python/bytecodes.c" + #line 1811 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } - #line 2519 "Python/generated_cases.c.h" + #line 2551 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -2529,7 +2561,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1801 "Python/bytecodes.c" + #line 1820 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2542,7 +2574,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2546 "Python/generated_cases.c.h" + #line 2578 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2557,7 +2589,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1817 "Python/bytecodes.c" + #line 1836 "Python/bytecodes.c" DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; assert(dict != NULL); @@ -2570,7 +2602,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2574 "Python/generated_cases.c.h" + #line 2606 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2585,7 +2617,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1833 "Python/bytecodes.c" + #line 1852 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2612,7 +2644,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2616 "Python/generated_cases.c.h" + #line 2648 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2627,7 +2659,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1863 "Python/bytecodes.c" + #line 1882 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2637,7 +2669,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2641 "Python/generated_cases.c.h" + #line 2673 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2652,7 +2684,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 1876 "Python/bytecodes.c" + #line 1895 "Python/bytecodes.c" DEOPT_IF(!PyType_Check(cls), LOAD_ATTR); DEOPT_IF(((PyTypeObject *)cls)->tp_version_tag != type_version, @@ -2664,7 +2696,7 @@ res = descr; assert(res != NULL); Py_INCREF(res); - #line 2668 "Python/generated_cases.c.h" + #line 2700 "Python/generated_cases.c.h" Py_DECREF(cls); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2678,7 +2710,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *fget = read_obj(&next_instr[5].cache); - #line 1891 "Python/bytecodes.c" + #line 1910 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); @@ -2702,7 +2734,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_LOAD_ATTR); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 2706 "Python/generated_cases.c.h" + #line 2738 "Python/generated_cases.c.h" } TARGET(LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN) { @@ -2710,7 +2742,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *getattribute = read_obj(&next_instr[5].cache); - #line 1917 "Python/bytecodes.c" + #line 1936 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); DEOPT_IF(cls->tp_version_tag != type_version, LOAD_ATTR); @@ -2736,7 +2768,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_LOAD_ATTR); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 2740 "Python/generated_cases.c.h" + #line 2772 "Python/generated_cases.c.h" } TARGET(STORE_ATTR_INSTANCE_VALUE) { @@ -2744,7 +2776,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1945 "Python/bytecodes.c" + #line 1964 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2762,7 +2794,7 @@ Py_DECREF(old_value); } Py_DECREF(owner); - #line 2766 "Python/generated_cases.c.h" + #line 2798 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2773,7 +2805,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t hint = read_u16(&next_instr[3].cache); - #line 1965 "Python/bytecodes.c" + #line 1984 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2812,7 +2844,7 @@ /* PEP 509 */ dict->ma_version_tag = new_version; Py_DECREF(owner); - #line 2816 "Python/generated_cases.c.h" + #line 2848 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2823,7 +2855,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 2006 "Python/bytecodes.c" + #line 2025 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2833,7 +2865,7 @@ *(PyObject **)addr = value; Py_XDECREF(old_value); Py_DECREF(owner); - #line 2837 "Python/generated_cases.c.h" + #line 2869 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2845,7 +2877,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2025 "Python/bytecodes.c" + #line 2044 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2858,10 +2890,10 @@ #endif /* ENABLE_SPECIALIZATION */ assert((oparg >> 5) <= Py_GE); res = PyObject_RichCompare(left, right, oparg >> 5); - #line 2862 "Python/generated_cases.c.h" + #line 2894 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2038 "Python/bytecodes.c" + #line 2057 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; if (oparg & 16) { int res_bool = PyObject_IsTrue(res); @@ -2869,7 +2901,7 @@ if (res_bool < 0) goto pop_2_error; res = res_bool ? Py_True : Py_False; } - #line 2873 "Python/generated_cases.c.h" + #line 2905 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2880,7 +2912,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2048 "Python/bytecodes.c" + #line 2067 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2892,7 +2924,7 @@ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 2896 "Python/generated_cases.c.h" + #line 2928 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2903,7 +2935,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2063 "Python/bytecodes.c" + #line 2082 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -2919,7 +2951,7 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 2923 "Python/generated_cases.c.h" + #line 2955 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2930,7 +2962,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2082 "Python/bytecodes.c" + #line 2101 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2943,7 +2975,7 @@ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 2947 "Python/generated_cases.c.h" + #line 2979 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2954,14 +2986,14 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2097 "Python/bytecodes.c" + #line 2116 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 2960 "Python/generated_cases.c.h" + #line 2992 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2099 "Python/bytecodes.c" + #line 2118 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 2965 "Python/generated_cases.c.h" + #line 2997 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -2971,15 +3003,15 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2103 "Python/bytecodes.c" + #line 2122 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 2977 "Python/generated_cases.c.h" + #line 3009 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2105 "Python/bytecodes.c" + #line 2124 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; - #line 2983 "Python/generated_cases.c.h" + #line 3015 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -2990,12 +3022,12 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 2110 "Python/bytecodes.c" + #line 2129 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 2996 "Python/generated_cases.c.h" + #line 3028 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2112 "Python/bytecodes.c" + #line 2131 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -3003,10 +3035,10 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 3007 "Python/generated_cases.c.h" + #line 3039 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2120 "Python/bytecodes.c" + #line 2139 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -3015,7 +3047,7 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 3019 "Python/generated_cases.c.h" + #line 3051 "Python/generated_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; DISPATCH(); @@ -3025,21 +3057,21 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2131 "Python/bytecodes.c" + #line 2150 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 3032 "Python/generated_cases.c.h" + #line 3064 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2134 "Python/bytecodes.c" + #line 2153 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 3039 "Python/generated_cases.c.h" + #line 3071 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2139 "Python/bytecodes.c" + #line 2158 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 3043 "Python/generated_cases.c.h" + #line 3075 "Python/generated_cases.c.h" stack_pointer[-1] = b; DISPATCH(); } @@ -3048,15 +3080,15 @@ PyObject *fromlist = stack_pointer[-1]; PyObject *level = stack_pointer[-2]; PyObject *res; - #line 2143 "Python/bytecodes.c" + #line 2162 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_name(tstate, frame, name, fromlist, level); - #line 3055 "Python/generated_cases.c.h" + #line 3087 "Python/generated_cases.c.h" Py_DECREF(level); Py_DECREF(fromlist); - #line 2146 "Python/bytecodes.c" + #line 2165 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 3060 "Python/generated_cases.c.h" + #line 3092 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -3065,25 +3097,25 @@ TARGET(IMPORT_FROM) { PyObject *from = stack_pointer[-1]; PyObject *res; - #line 2150 "Python/bytecodes.c" + #line 2169 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_from(tstate, from, name); if (res == NULL) goto error; - #line 3073 "Python/generated_cases.c.h" + #line 3105 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); } TARGET(JUMP_FORWARD) { - #line 2156 "Python/bytecodes.c" + #line 2175 "Python/bytecodes.c" JUMPBY(oparg); - #line 3082 "Python/generated_cases.c.h" + #line 3114 "Python/generated_cases.c.h" DISPATCH(); } TARGET(JUMP_BACKWARD) { - #line 2160 "Python/bytecodes.c" + #line 2179 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); _Py_CODEUNIT *here = next_instr - 1; assert(oparg <= INSTR_OFFSET()); @@ -3102,12 +3134,12 @@ goto resume_frame; } #endif /* ENABLE_SPECIALIZATION */ - #line 3106 "Python/generated_cases.c.h" + #line 3138 "Python/generated_cases.c.h" DISPATCH(); } TARGET(ENTER_EXECUTOR) { - #line 2191 "Python/bytecodes.c" + #line 2210 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); PyCodeObject *code = _PyFrame_GetCode(frame); @@ -3122,25 +3154,25 @@ goto resume_with_error; } goto resume_frame; - #line 3126 "Python/generated_cases.c.h" + #line 3158 "Python/generated_cases.c.h" } TARGET(POP_JUMP_IF_FALSE) { PyObject *cond = stack_pointer[-1]; - #line 2208 "Python/bytecodes.c" + #line 2227 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsFalse(cond)); - #line 3134 "Python/generated_cases.c.h" + #line 3166 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_TRUE) { PyObject *cond = stack_pointer[-1]; - #line 2213 "Python/bytecodes.c" + #line 2232 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsTrue(cond)); - #line 3144 "Python/generated_cases.c.h" + #line 3176 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -3150,25 +3182,25 @@ { PyObject *value = _tmp_1; PyObject *b; - #line 2218 "Python/bytecodes.c" + #line 2237 "Python/bytecodes.c" if (Py_IsNone(value)) { b = Py_True; } else { b = Py_False; - #line 3160 "Python/generated_cases.c.h" + #line 3192 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2224 "Python/bytecodes.c" + #line 2243 "Python/bytecodes.c" } - #line 3164 "Python/generated_cases.c.h" + #line 3196 "Python/generated_cases.c.h" _tmp_1 = b; } { PyObject *cond = _tmp_1; - #line 2213 "Python/bytecodes.c" + #line 2232 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsTrue(cond)); - #line 3172 "Python/generated_cases.c.h" + #line 3204 "Python/generated_cases.c.h" } STACK_SHRINK(1); DISPATCH(); @@ -3179,52 +3211,52 @@ { PyObject *value = _tmp_1; PyObject *b; - #line 2218 "Python/bytecodes.c" + #line 2237 "Python/bytecodes.c" if (Py_IsNone(value)) { b = Py_True; } else { b = Py_False; - #line 3189 "Python/generated_cases.c.h" + #line 3221 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2224 "Python/bytecodes.c" + #line 2243 "Python/bytecodes.c" } - #line 3193 "Python/generated_cases.c.h" + #line 3225 "Python/generated_cases.c.h" _tmp_1 = b; } { PyObject *cond = _tmp_1; - #line 2208 "Python/bytecodes.c" + #line 2227 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsFalse(cond)); - #line 3201 "Python/generated_cases.c.h" + #line 3233 "Python/generated_cases.c.h" } STACK_SHRINK(1); DISPATCH(); } TARGET(JUMP_BACKWARD_NO_INTERRUPT) { - #line 2232 "Python/bytecodes.c" + #line 2251 "Python/bytecodes.c" /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost * generator or coroutine, so we deliberately do not check it here. * (see bpo-30039). */ JUMPBY(-oparg); - #line 3215 "Python/generated_cases.c.h" + #line 3247 "Python/generated_cases.c.h" DISPATCH(); } TARGET(GET_LEN) { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2241 "Python/bytecodes.c" + #line 2260 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 3228 "Python/generated_cases.c.h" + #line 3260 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; DISPATCH(); @@ -3235,16 +3267,16 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2249 "Python/bytecodes.c" + #line 2268 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 3244 "Python/generated_cases.c.h" + #line 3276 "Python/generated_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2254 "Python/bytecodes.c" + #line 2273 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -3252,7 +3284,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 3256 "Python/generated_cases.c.h" + #line 3288 "Python/generated_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; DISPATCH(); @@ -3261,10 +3293,10 @@ TARGET(MATCH_MAPPING) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2264 "Python/bytecodes.c" + #line 2283 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 3268 "Python/generated_cases.c.h" + #line 3300 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3273,10 +3305,10 @@ TARGET(MATCH_SEQUENCE) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2269 "Python/bytecodes.c" + #line 2288 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 3280 "Python/generated_cases.c.h" + #line 3312 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3286,11 +3318,11 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2274 "Python/bytecodes.c" + #line 2293 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 3294 "Python/generated_cases.c.h" + #line 3326 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; DISPATCH(); @@ -3299,14 +3331,14 @@ TARGET(GET_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2280 "Python/bytecodes.c" + #line 2299 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 3306 "Python/generated_cases.c.h" + #line 3338 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2283 "Python/bytecodes.c" + #line 2302 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 3310 "Python/generated_cases.c.h" + #line 3342 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3314,7 +3346,7 @@ TARGET(GET_YIELD_FROM_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2287 "Python/bytecodes.c" + #line 2306 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -3337,11 +3369,11 @@ if (iter == NULL) { goto error; } - #line 3341 "Python/generated_cases.c.h" + #line 3373 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2310 "Python/bytecodes.c" + #line 2329 "Python/bytecodes.c" } - #line 3345 "Python/generated_cases.c.h" + #line 3377 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3351,7 +3383,7 @@ static_assert(INLINE_CACHE_ENTRIES_FOR_ITER == 1, "incorrect cache size"); PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2328 "Python/bytecodes.c" + #line 2347 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -3383,7 +3415,7 @@ DISPATCH(); } // Common case: no jump, leave it to the code generator - #line 3387 "Python/generated_cases.c.h" + #line 3419 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3391,7 +3423,7 @@ } TARGET(INSTRUMENTED_FOR_ITER) { - #line 2362 "Python/bytecodes.c" + #line 2381 "Python/bytecodes.c" _Py_CODEUNIT *here = next_instr-1; _Py_CODEUNIT *target; PyObject *iter = TOP(); @@ -3417,14 +3449,14 @@ target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER + oparg + 1; } INSTRUMENTED_JUMP(here, target, PY_MONITORING_EVENT_BRANCH); - #line 3421 "Python/generated_cases.c.h" + #line 3453 "Python/generated_cases.c.h" DISPATCH(); } TARGET(FOR_ITER_LIST) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2390 "Python/bytecodes.c" + #line 2409 "Python/bytecodes.c" DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit); @@ -3445,7 +3477,7 @@ DISPATCH(); end_for_iter_list: // Common case: no jump, leave it to the code generator - #line 3449 "Python/generated_cases.c.h" + #line 3481 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3455,7 +3487,7 @@ TARGET(FOR_ITER_TUPLE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2413 "Python/bytecodes.c" + #line 2432 "Python/bytecodes.c" _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3476,7 +3508,7 @@ DISPATCH(); end_for_iter_tuple: // Common case: no jump, leave it to the code generator - #line 3480 "Python/generated_cases.c.h" + #line 3512 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3486,7 +3518,7 @@ TARGET(FOR_ITER_RANGE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2436 "Python/bytecodes.c" + #line 2455 "Python/bytecodes.c" _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3505,7 +3537,7 @@ if (next == NULL) { goto error; } - #line 3509 "Python/generated_cases.c.h" + #line 3541 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3514,7 +3546,7 @@ TARGET(FOR_ITER_GEN) { PyObject *iter = stack_pointer[-1]; - #line 2457 "Python/bytecodes.c" + #line 2476 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER); @@ -3530,14 +3562,14 @@ assert(next_instr[oparg].op.code == END_FOR || next_instr[oparg].op.code == INSTRUMENTED_END_FOR); DISPATCH_INLINED(gen_frame); - #line 3534 "Python/generated_cases.c.h" + #line 3566 "Python/generated_cases.c.h" } TARGET(BEFORE_ASYNC_WITH) { PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2475 "Python/bytecodes.c" + #line 2494 "Python/bytecodes.c" PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { @@ -3560,16 +3592,16 @@ Py_DECREF(enter); goto error; } - #line 3564 "Python/generated_cases.c.h" + #line 3596 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2498 "Python/bytecodes.c" + #line 2517 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3573 "Python/generated_cases.c.h" + #line 3605 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3580,7 +3612,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2507 "Python/bytecodes.c" + #line 2526 "Python/bytecodes.c" /* pop the context manager, push its __exit__ and the * value returned from calling its __enter__ */ @@ -3606,16 +3638,16 @@ Py_DECREF(enter); goto error; } - #line 3610 "Python/generated_cases.c.h" + #line 3642 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2533 "Python/bytecodes.c" + #line 2552 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3619 "Python/generated_cases.c.h" + #line 3651 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3627,7 +3659,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2542 "Python/bytecodes.c" + #line 2561 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -3648,7 +3680,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 3652 "Python/generated_cases.c.h" + #line 3684 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3657,7 +3689,7 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2581 "Python/bytecodes.c" + #line 2600 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -3667,7 +3699,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 3671 "Python/generated_cases.c.h" + #line 3703 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -3681,7 +3713,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2593 "Python/bytecodes.c" + #line 2612 "Python/bytecodes.c" assert(oparg & 1); /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); @@ -3698,7 +3730,7 @@ res2 = Py_NewRef(descr); assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; - #line 3702 "Python/generated_cases.c.h" + #line 3734 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3712,7 +3744,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2612 "Python/bytecodes.c" + #line 2631 "Python/bytecodes.c" assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -3722,7 +3754,7 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - #line 3726 "Python/generated_cases.c.h" + #line 3758 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3737,7 +3769,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2624 "Python/bytecodes.c" + #line 2643 "Python/bytecodes.c" assert((oparg & 1) == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3750,11 +3782,11 @@ keys_version, LOAD_ATTR); STAT_INC(LOAD_ATTR, hit); assert(descr != NULL); - #line 3754 "Python/generated_cases.c.h" + #line 3786 "Python/generated_cases.c.h" Py_DECREF(self); - #line 2637 "Python/bytecodes.c" + #line 2656 "Python/bytecodes.c" res = Py_NewRef(descr); - #line 3758 "Python/generated_cases.c.h" + #line 3790 "Python/generated_cases.c.h" STACK_GROW((0 ? 1 : 0)); stack_pointer[-1] = res; if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } @@ -3768,7 +3800,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2641 "Python/bytecodes.c" + #line 2660 "Python/bytecodes.c" assert((oparg & 1) == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3776,11 +3808,11 @@ assert(self_cls->tp_dictoffset == 0); STAT_INC(LOAD_ATTR, hit); assert(descr != NULL); - #line 3780 "Python/generated_cases.c.h" + #line 3812 "Python/generated_cases.c.h" Py_DECREF(self); - #line 2649 "Python/bytecodes.c" + #line 2668 "Python/bytecodes.c" res = Py_NewRef(descr); - #line 3784 "Python/generated_cases.c.h" + #line 3816 "Python/generated_cases.c.h" STACK_GROW((0 ? 1 : 0)); stack_pointer[-1] = res; if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } @@ -3794,7 +3826,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2653 "Python/bytecodes.c" + #line 2672 "Python/bytecodes.c" assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -3808,7 +3840,7 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - #line 3812 "Python/generated_cases.c.h" + #line 3844 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3817,16 +3849,16 @@ } TARGET(KW_NAMES) { - #line 2669 "Python/bytecodes.c" + #line 2688 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(FRAME_CO_CONSTS)); kwnames = GETITEM(FRAME_CO_CONSTS, oparg); - #line 3825 "Python/generated_cases.c.h" + #line 3857 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_CALL) { - #line 2675 "Python/bytecodes.c" + #line 2694 "Python/bytecodes.c" int is_meth = PEEK(oparg+2) != NULL; int total_args = oparg + is_meth; PyObject *function = PEEK(total_args + 1); @@ -3839,7 +3871,7 @@ _PyCallCache *cache = (_PyCallCache *)next_instr; INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(CALL); - #line 3843 "Python/generated_cases.c.h" + #line 3875 "Python/generated_cases.c.h" } TARGET(CALL) { @@ -3849,7 +3881,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2721 "Python/bytecodes.c" + #line 2740 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3931,7 +3963,7 @@ Py_DECREF(args[i]); } if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3935 "Python/generated_cases.c.h" + #line 3967 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3943,7 +3975,7 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2809 "Python/bytecodes.c" + #line 2828 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit); @@ -3953,7 +3985,7 @@ PEEK(oparg + 2) = Py_NewRef(meth); // method Py_DECREF(callable); GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS); - #line 3957 "Python/generated_cases.c.h" + #line 3989 "Python/generated_cases.c.h" } TARGET(CALL_PY_EXACT_ARGS) { @@ -3962,7 +3994,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2821 "Python/bytecodes.c" + #line 2840 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -3988,7 +4020,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 3992 "Python/generated_cases.c.h" + #line 4024 "Python/generated_cases.c.h" } TARGET(CALL_PY_WITH_DEFAULTS) { @@ -3996,7 +4028,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2849 "Python/bytecodes.c" + #line 2868 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4032,7 +4064,7 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4036 "Python/generated_cases.c.h" + #line 4068 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_TYPE_1) { @@ -4040,7 +4072,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2887 "Python/bytecodes.c" + #line 2906 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4050,7 +4082,7 @@ res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(obj); Py_DECREF(&PyType_Type); // I.e., callable - #line 4054 "Python/generated_cases.c.h" + #line 4086 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4063,7 +4095,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2899 "Python/bytecodes.c" + #line 2918 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4074,7 +4106,7 @@ Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); // I.e., callable if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4078 "Python/generated_cases.c.h" + #line 4110 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4088,7 +4120,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2913 "Python/bytecodes.c" + #line 2932 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4099,7 +4131,7 @@ Py_DECREF(arg); Py_DECREF(&PyTuple_Type); // I.e., tuple if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4103 "Python/generated_cases.c.h" + #line 4135 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4112,7 +4144,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; - #line 2927 "Python/bytecodes.c" + #line 2946 "Python/bytecodes.c" /* This instruction does the following: * 1. Creates the object (by calling ``object.__new__``) * 2. Pushes a shim frame to the frame stack (to cleanup after ``__init__``) @@ -4163,12 +4195,12 @@ * as it will be checked after start_frame */ tstate->py_recursion_remaining--; goto start_frame; - #line 4167 "Python/generated_cases.c.h" + #line 4199 "Python/generated_cases.c.h" } TARGET(EXIT_INIT_CHECK) { PyObject *should_be_none = stack_pointer[-1]; - #line 2980 "Python/bytecodes.c" + #line 2999 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -4176,7 +4208,7 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 4180 "Python/generated_cases.c.h" + #line 4212 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -4186,7 +4218,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2990 "Python/bytecodes.c" + #line 3009 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4208,7 +4240,7 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4212 "Python/generated_cases.c.h" + #line 4244 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4222,7 +4254,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3015 "Python/bytecodes.c" + #line 3034 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4250,7 +4282,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4254 "Python/generated_cases.c.h" + #line 4286 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4264,7 +4296,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3046 "Python/bytecodes.c" + #line 3065 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4296,7 +4328,7 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 4300 "Python/generated_cases.c.h" + #line 4332 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4310,7 +4342,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3081 "Python/bytecodes.c" + #line 3100 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -4342,7 +4374,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4346 "Python/generated_cases.c.h" + #line 4378 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4356,7 +4388,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3116 "Python/bytecodes.c" + #line 3135 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -4381,7 +4413,7 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4385 "Python/generated_cases.c.h" + #line 4417 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4394,7 +4426,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3143 "Python/bytecodes.c" + #line 3162 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -4421,7 +4453,7 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4425 "Python/generated_cases.c.h" + #line 4457 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4433,7 +4465,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 3173 "Python/bytecodes.c" + #line 3192 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -4451,14 +4483,14 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 4455 "Python/generated_cases.c.h" + #line 4487 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3193 "Python/bytecodes.c" + #line 3212 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4489,7 +4521,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4493 "Python/generated_cases.c.h" + #line 4525 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4502,7 +4534,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3227 "Python/bytecodes.c" + #line 3246 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4531,7 +4563,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4535 "Python/generated_cases.c.h" + #line 4567 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4544,7 +4576,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3259 "Python/bytecodes.c" + #line 3278 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -4573,7 +4605,7 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4577 "Python/generated_cases.c.h" + #line 4609 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4586,7 +4618,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3291 "Python/bytecodes.c" + #line 3310 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4614,7 +4646,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4618 "Python/generated_cases.c.h" + #line 4650 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4624,9 +4656,9 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { - #line 3322 "Python/bytecodes.c" + #line 3341 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); - #line 4630 "Python/generated_cases.c.h" + #line 4662 "Python/generated_cases.c.h" } TARGET(CALL_FUNCTION_EX) { @@ -4635,7 +4667,7 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 3326 "Python/bytecodes.c" + #line 3345 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs)); @@ -4697,14 +4729,14 @@ } result = PyObject_Call(func, callargs, kwargs); } - #line 4701 "Python/generated_cases.c.h" + #line 4733 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 3388 "Python/bytecodes.c" + #line 3407 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4708 "Python/generated_cases.c.h" + #line 4740 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4715,7 +4747,7 @@ TARGET(MAKE_FUNCTION) { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3394 "Python/bytecodes.c" + #line 3413 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4727,7 +4759,7 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4731 "Python/generated_cases.c.h" + #line 4763 "Python/generated_cases.c.h" stack_pointer[-1] = func; DISPATCH(); } @@ -4735,7 +4767,7 @@ TARGET(SET_FUNCTION_ATTRIBUTE) { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3408 "Python/bytecodes.c" + #line 3427 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -4760,14 +4792,14 @@ default: Py_UNREACHABLE(); } - #line 4764 "Python/generated_cases.c.h" + #line 4796 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 3435 "Python/bytecodes.c" + #line 3454 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4788,7 +4820,7 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4792 "Python/generated_cases.c.h" + #line 4824 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4796,15 +4828,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3458 "Python/bytecodes.c" + #line 3477 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4802 "Python/generated_cases.c.h" + #line 4834 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3460 "Python/bytecodes.c" + #line 3479 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4808 "Python/generated_cases.c.h" + #line 4840 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4814,14 +4846,14 @@ TARGET(CONVERT_VALUE) { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3464 "Python/bytecodes.c" + #line 3483 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 4825 "Python/generated_cases.c.h" + #line 4857 "Python/generated_cases.c.h" stack_pointer[-1] = result; DISPATCH(); } @@ -4829,7 +4861,7 @@ TARGET(FORMAT_SIMPLE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3473 "Python/bytecodes.c" + #line 3492 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -4840,7 +4872,7 @@ else { res = value; } - #line 4844 "Python/generated_cases.c.h" + #line 4876 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -4849,12 +4881,12 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3486 "Python/bytecodes.c" + #line 3505 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 4858 "Python/generated_cases.c.h" + #line 4890 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -4863,10 +4895,10 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3493 "Python/bytecodes.c" + #line 3512 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4870 "Python/generated_cases.c.h" + #line 4902 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4878,7 +4910,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3498 "Python/bytecodes.c" + #line 3517 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -4893,12 +4925,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4897 "Python/generated_cases.c.h" + #line 4929 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3513 "Python/bytecodes.c" + #line 3532 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 4902 "Python/generated_cases.c.h" + #line 4934 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -4908,16 +4940,16 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3518 "Python/bytecodes.c" + #line 3537 "Python/bytecodes.c" assert(oparg >= 2); - #line 4914 "Python/generated_cases.c.h" + #line 4946 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } TARGET(INSTRUMENTED_INSTRUCTION) { - #line 3522 "Python/bytecodes.c" + #line 3541 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error; @@ -4929,48 +4961,48 @@ assert(next_opcode > 0 && next_opcode < 256); opcode = next_opcode; DISPATCH_GOTO(); - #line 4933 "Python/generated_cases.c.h" + #line 4965 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_JUMP_FORWARD) { - #line 3536 "Python/bytecodes.c" + #line 3555 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); - #line 4939 "Python/generated_cases.c.h" + #line 4971 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) { - #line 3540 "Python/bytecodes.c" + #line 3559 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); - #line 4947 "Python/generated_cases.c.h" + #line 4979 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { - #line 3545 "Python/bytecodes.c" + #line 3564 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsTrue(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4958 "Python/generated_cases.c.h" + #line 4990 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { - #line 3553 "Python/bytecodes.c" + #line 3572 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsFalse(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4969 "Python/generated_cases.c.h" + #line 5001 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { - #line 3561 "Python/bytecodes.c" + #line 3580 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -4982,12 +5014,12 @@ offset = 0; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4986 "Python/generated_cases.c.h" + #line 5018 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { - #line 3575 "Python/bytecodes.c" + #line 3594 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -4999,30 +5031,30 @@ offset = oparg; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5003 "Python/generated_cases.c.h" + #line 5035 "Python/generated_cases.c.h" DISPATCH(); } TARGET(EXTENDED_ARG) { - #line 3589 "Python/bytecodes.c" + #line 3608 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 5014 "Python/generated_cases.c.h" + #line 5046 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3597 "Python/bytecodes.c" + #line 3616 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); - #line 5021 "Python/generated_cases.c.h" + #line 5053 "Python/generated_cases.c.h" } TARGET(RESERVED) { - #line 3602 "Python/bytecodes.c" + #line 3621 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 5028 "Python/generated_cases.c.h" + #line 5060 "Python/generated_cases.c.h" } From webhook-mailer at python.org Wed Jul 12 10:30:56 2023 From: webhook-mailer at python.org (Yhg1s) Date: Wed, 12 Jul 2023 14:30:56 -0000 Subject: [Python-checkins] [3.12] gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649) (#106680) Message-ID: https://github.com/python/cpython/commit/6de9cffcd38b7ce0f6b914643d6df08b6fc750d3 commit: 6de9cffcd38b7ce0f6b914643d6df08b6fc750d3 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Yhg1s date: 2023-07-12T16:30:52+02:00 summary: [3.12] gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649) (#106680) gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649) (cherry picked from commit e2d7366fb3df44e7434132636d49f22d6d25cc9f) Co-authored-by: Steve Dower files: A Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst M PCbuild/get_externals.bat M PCbuild/openssl.props M PCbuild/python.props M PCbuild/readme.txt M PCbuild/regen.targets diff --git a/Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst b/Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst new file mode 100644 index 0000000000000..11f411be0f17c --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst @@ -0,0 +1 @@ +Update Windows build to use OpenSSL 3.0.9 diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat index 534a9ebcfd151..77b9594799ea7 100644 --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -53,7 +53,7 @@ echo.Fetching external libraries... set libraries= set libraries=%libraries% bzip2-1.0.8 if NOT "%IncludeLibffiSrc%"=="false" set libraries=%libraries% libffi-3.4.4 -if NOT "%IncludeSSLSrc%"=="false" set libraries=%libraries% openssl-1.1.1u +if NOT "%IncludeSSLSrc%"=="false" set libraries=%libraries% openssl-3.0.9 set libraries=%libraries% sqlite-3.42.0.0 if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tcl-core-8.6.13.0 if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tk-8.6.13.0 @@ -77,7 +77,7 @@ echo.Fetching external binaries... set binaries= if NOT "%IncludeLibffi%"=="false" set binaries=%binaries% libffi-3.4.4 -if NOT "%IncludeSSL%"=="false" set binaries=%binaries% openssl-bin-1.1.1u +if NOT "%IncludeSSL%"=="false" set binaries=%binaries% openssl-bin-3.0.9 if NOT "%IncludeTkinter%"=="false" set binaries=%binaries% tcltk-8.6.13.0 if NOT "%IncludeSSLSrc%"=="false" set binaries=%binaries% nasm-2.11.06 diff --git a/PCbuild/openssl.props b/PCbuild/openssl.props index 7071bb57c06cd..5fd708b211e42 100644 --- a/PCbuild/openssl.props +++ b/PCbuild/openssl.props @@ -10,10 +10,10 @@ - <_DLLSuffix>-1_1 + <_DLLSuffix>-3 <_DLLSuffix Condition="$(Platform) == 'ARM'">$(_DLLSuffix)-arm <_DLLSuffix Condition="$(Platform) == 'ARM64'">$(_DLLSuffix)-arm64 - $(_DLLSuffix) + $(_DLLSuffix) <_SSLDLL Include="$(opensslOutDir)\libcrypto$(_DLLSuffix).dll" /> diff --git a/PCbuild/python.props b/PCbuild/python.props index 68052ef668aa6..d3586235c8265 100644 --- a/PCbuild/python.props +++ b/PCbuild/python.props @@ -74,8 +74,8 @@ $(ExternalsDir)libffi-3.4.4\ $(libffiDir)$(ArchName)\ $(libffiOutDir)include - $(ExternalsDir)openssl-1.1.1u\ - $(ExternalsDir)openssl-bin-1.1.1u\$(ArchName)\ + $(ExternalsDir)openssl-3.0.9\ + $(ExternalsDir)openssl-bin-3.0.9\$(ArchName)\ $(opensslOutDir)include $(ExternalsDir)\nasm-2.11.06\ $(ExternalsDir)\zlib-1.2.13\ diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt index 784d3ce6a6066..2ab4537b99882 100644 --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -169,7 +169,7 @@ _lzma Homepage: https://tukaani.org/xz/ _ssl - Python wrapper for version 1.1.1u of the OpenSSL secure sockets + Python wrapper for version 3.0 of the OpenSSL secure sockets library, which is downloaded from our binaries repository at https://github.com/python/cpython-bin-deps. diff --git a/PCbuild/regen.targets b/PCbuild/regen.targets index 107066817ba6b..af07adeca8812 100644 --- a/PCbuild/regen.targets +++ b/PCbuild/regen.targets @@ -104,8 +104,9 @@ <_LicenseSources Include="$(PySourcePath)LICENSE; $(PySourcePath)PC\crtlicense.txt; $(bz2Dir)LICENSE; - $(opensslOutDir)LICENSE; $(libffiDir)LICENSE;" /> + <_LicenseSources Include="$(opensslOutDir)LICENSE.txt" Condition="Exists('$(opensslOutDir)LICENSE.txt')" /> + <_LicenseSources Include="$(opensslOutDir)LICENSE" Condition="!Exists('$(opensslOutDir)LICENSE.txt')" /> <_LicenseSources Include="$(tcltkDir)tcllicense.terms; $(tcltkDir)tklicense.terms; $(tcltkDir)tixlicense.terms" Condition="$(IncludeTkinter)" /> From webhook-mailer at python.org Wed Jul 12 12:12:42 2023 From: webhook-mailer at python.org (gvanrossum) Date: Wed, 12 Jul 2023 16:12:42 -0000 Subject: [Python-checkins] gh-106656: Remove --emit-line-directives from regen-cases (#106657) Message-ID: https://github.com/python/cpython/commit/7f55f58b6c97306da350f5b441d26f859e9d8f16 commit: 7f55f58b6c97306da350f5b441d26f859e9d8f16 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-12T16:12:39Z summary: gh-106656: Remove --emit-line-directives from regen-cases (#106657) If you prefer to see `#line` directives in generated_cases.c.h, run ``` make regen-cases CASESFLAG=-l ``` But please don't commit the result. files: M Makefile.pre.in M Python/executor_cases.c.h M Python/generated_cases.c.h diff --git a/Makefile.pre.in b/Makefile.pre.in index a1ceedbd4812c..4b655513f656d 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1543,10 +1543,11 @@ regen-opcode-targets: .PHONY: regen-cases regen-cases: # Regenerate various files from Python/bytecodes.c + # Pass CASESFLAG=-l to insert #line directives in the output PYTHONPATH=$(srcdir)/Tools/cases_generator \ $(PYTHON_FOR_REGEN) \ $(srcdir)/Tools/cases_generator/generate_cases.py \ - --emit-line-directives \ + $(CASESFLAG) \ -o $(srcdir)/Python/generated_cases.c.h.new \ -m $(srcdir)/Include/internal/pycore_opcode_metadata.h.new \ -e $(srcdir)/Python/executor_cases.c.h.new \ diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 94536ed8ce5a1..1df8feba352cf 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -9,11 +9,9 @@ case LOAD_FAST_CHECK: { PyObject *value; - #line 182 "Python/bytecodes.c" value = GETLOCAL(oparg); if (value == NULL) goto unbound_local_error; Py_INCREF(value); - #line 17 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -21,11 +19,9 @@ case LOAD_FAST: { PyObject *value; - #line 188 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value); - #line 29 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -33,11 +29,9 @@ case LOAD_FAST_AND_CLEAR: { PyObject *value; - #line 194 "Python/bytecodes.c" value = GETLOCAL(oparg); // do not use SETLOCAL here, it decrefs the old value GETLOCAL(oparg) = NULL; - #line 41 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -45,10 +39,8 @@ case LOAD_CONST: { PyObject *value; - #line 209 "Python/bytecodes.c" value = GETITEM(FRAME_CO_CONSTS, oparg); Py_INCREF(value); - #line 52 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -56,17 +48,13 @@ case STORE_FAST: { PyObject *value = stack_pointer[-1]; - #line 214 "Python/bytecodes.c" SETLOCAL(oparg, value); - #line 62 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case POP_TOP: { PyObject *value = stack_pointer[-1]; - #line 237 "Python/bytecodes.c" - #line 70 "Python/executor_cases.c.h" Py_DECREF(value); STACK_SHRINK(1); break; @@ -74,9 +62,7 @@ case PUSH_NULL: { PyObject *res; - #line 241 "Python/bytecodes.c" res = NULL; - #line 80 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -85,9 +71,7 @@ case END_SEND: { PyObject *value = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; - #line 260 "Python/bytecodes.c" Py_DECREF(receiver); - #line 91 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = value; break; @@ -96,13 +80,9 @@ case UNARY_NEGATIVE: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 275 "Python/bytecodes.c" res = PyNumber_Negative(value); - #line 102 "Python/executor_cases.c.h" Py_DECREF(value); - #line 277 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 106 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -110,10 +90,8 @@ case UNARY_NOT: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 281 "Python/bytecodes.c" assert(PyBool_Check(value)); res = Py_IsFalse(value) ? Py_True : Py_False; - #line 117 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -122,7 +100,6 @@ static_assert(INLINE_CACHE_ENTRIES_TO_BOOL == 3, "incorrect cache size"); PyObject *value = stack_pointer[-1]; PyObject *res; - #line 296 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyToBoolCache *cache = (_PyToBoolCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -134,29 +111,23 @@ DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ int err = PyObject_IsTrue(value); - #line 138 "Python/executor_cases.c.h" Py_DECREF(value); - #line 308 "Python/bytecodes.c" if (err < 0) goto pop_1_error; res = err ? Py_True : Py_False; - #line 143 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } case TO_BOOL_BOOL: { PyObject *value = stack_pointer[-1]; - #line 313 "Python/bytecodes.c" DEOPT_IF(!PyBool_Check(value), TO_BOOL); STAT_INC(TO_BOOL, hit); - #line 153 "Python/executor_cases.c.h" break; } case TO_BOOL_INT: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 318 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(value), TO_BOOL); STAT_INC(TO_BOOL, hit); if (_PyLong_IsZero((PyLongObject *)value)) { @@ -164,12 +135,9 @@ res = Py_False; } else { - #line 168 "Python/executor_cases.c.h" Py_DECREF(value); - #line 326 "Python/bytecodes.c" res = Py_True; } - #line 173 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -177,11 +145,9 @@ case TO_BOOL_LIST: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 331 "Python/bytecodes.c" DEOPT_IF(!PyList_CheckExact(value), TO_BOOL); STAT_INC(TO_BOOL, hit); res = Py_SIZE(value) ? Py_True : Py_False; - #line 185 "Python/executor_cases.c.h" Py_DECREF(value); stack_pointer[-1] = res; break; @@ -190,12 +156,10 @@ case TO_BOOL_NONE: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 338 "Python/bytecodes.c" // This one is a bit weird, because we expect *some* failures: DEOPT_IF(!Py_IsNone(value), TO_BOOL); STAT_INC(TO_BOOL, hit); res = Py_False; - #line 199 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -203,7 +167,6 @@ case TO_BOOL_STR: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 345 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(value), TO_BOOL); STAT_INC(TO_BOOL, hit); if (value == &_Py_STR(empty)) { @@ -212,12 +175,9 @@ } else { assert(Py_SIZE(value)); - #line 216 "Python/executor_cases.c.h" Py_DECREF(value); - #line 354 "Python/bytecodes.c" res = Py_True; } - #line 221 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -226,16 +186,12 @@ PyObject *value = stack_pointer[-1]; PyObject *res; uint32_t version = (uint32_t)operand; - #line 359 "Python/bytecodes.c" // This one is a bit weird, because we expect *some* failures: assert(version); DEOPT_IF(Py_TYPE(value)->tp_version_tag != version, TO_BOOL); STAT_INC(TO_BOOL, hit); - #line 235 "Python/executor_cases.c.h" Py_DECREF(value); - #line 364 "Python/bytecodes.c" res = Py_True; - #line 239 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -243,13 +199,9 @@ case UNARY_INVERT: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 368 "Python/bytecodes.c" res = PyNumber_Invert(value); - #line 249 "Python/executor_cases.c.h" Py_DECREF(value); - #line 370 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 253 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -257,10 +209,8 @@ case _GUARD_BOTH_INT: { PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; - #line 386 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); - #line 264 "Python/executor_cases.c.h" break; } @@ -268,13 +218,11 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 391 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = _PyLong_Multiply((PyLongObject *)left, (PyLongObject *)right); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 278 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -284,13 +232,11 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 399 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = _PyLong_Add((PyLongObject *)left, (PyLongObject *)right); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 294 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -300,13 +246,11 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 407 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = _PyLong_Subtract((PyLongObject *)left, (PyLongObject *)right); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 310 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -315,10 +259,8 @@ case _GUARD_BOTH_FLOAT: { PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; - #line 422 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); - #line 322 "Python/executor_cases.c.h" break; } @@ -326,13 +268,11 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 427 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); double dres = ((PyFloatObject *)left)->ob_fval * ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 336 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -342,13 +282,11 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 435 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); double dres = ((PyFloatObject *)left)->ob_fval + ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 352 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -358,13 +296,11 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 443 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); double dres = ((PyFloatObject *)left)->ob_fval - ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 368 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -373,10 +309,8 @@ case _GUARD_BOTH_UNICODE: { PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; - #line 458 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(!PyUnicode_CheckExact(right), BINARY_OP); - #line 380 "Python/executor_cases.c.h" break; } @@ -384,13 +318,11 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 463 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = PyUnicode_Concat(left, right); _Py_DECREF_SPECIALIZED(left, _PyUnicode_ExactDealloc); _Py_DECREF_SPECIALIZED(right, _PyUnicode_ExactDealloc); if (res == NULL) goto pop_2_error; - #line 394 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -401,7 +333,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; PyObject *res; - #line 517 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinarySubscrCache *cache = (_PyBinarySubscrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -413,12 +344,9 @@ DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ res = PyObject_GetItem(container, sub); - #line 417 "Python/executor_cases.c.h" Py_DECREF(container); Py_DECREF(sub); - #line 529 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 422 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -429,7 +357,6 @@ PyObject *start = stack_pointer[-2]; PyObject *container = stack_pointer[-3]; PyObject *res; - #line 533 "Python/bytecodes.c" PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop); // Can't use ERROR_IF() here, because we haven't // DECREF'ed container yet, and we still own slice. @@ -442,7 +369,6 @@ } Py_DECREF(container); if (res == NULL) goto pop_3_error; - #line 446 "Python/executor_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = res; break; @@ -453,7 +379,6 @@ PyObject *start = stack_pointer[-2]; PyObject *container = stack_pointer[-3]; PyObject *v = stack_pointer[-4]; - #line 548 "Python/bytecodes.c" PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop); int err; if (slice == NULL) { @@ -466,7 +391,6 @@ Py_DECREF(v); Py_DECREF(container); if (err) goto pop_4_error; - #line 470 "Python/executor_cases.c.h" STACK_SHRINK(4); break; } @@ -475,7 +399,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *list = stack_pointer[-2]; PyObject *res; - #line 563 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), BINARY_SUBSCR); @@ -489,7 +412,6 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 493 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -499,7 +421,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *tuple = stack_pointer[-2]; PyObject *res; - #line 579 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyTuple_CheckExact(tuple), BINARY_SUBSCR); @@ -513,7 +434,6 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(tuple); - #line 517 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -523,7 +443,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *dict = stack_pointer[-2]; PyObject *res; - #line 595 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(dict), BINARY_SUBSCR); STAT_INC(BINARY_SUBSCR, hit); res = PyDict_GetItemWithError(dict, sub); @@ -531,14 +450,11 @@ if (!_PyErr_Occurred(tstate)) { _PyErr_SetKeyError(sub); } - #line 535 "Python/executor_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); - #line 603 "Python/bytecodes.c" if (true) goto pop_2_error; } Py_INCREF(res); // Do this before DECREF'ing dict, sub - #line 542 "Python/executor_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); STACK_SHRINK(1); @@ -549,9 +465,7 @@ case LIST_APPEND: { PyObject *v = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 635 "Python/bytecodes.c" if (_PyList_AppendTakeRef((PyListObject *)list, v) < 0) goto pop_1_error; - #line 555 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -559,13 +473,9 @@ case SET_ADD: { PyObject *v = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 639 "Python/bytecodes.c" int err = PySet_Add(set, v); - #line 565 "Python/executor_cases.c.h" Py_DECREF(v); - #line 641 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 569 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -576,7 +486,6 @@ PyObject *container = stack_pointer[-2]; PyObject *v = stack_pointer[-3]; uint16_t counter = (uint16_t)operand; - #line 651 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { next_instr--; @@ -591,13 +500,10 @@ #endif /* ENABLE_SPECIALIZATION */ /* container[sub] = v */ int err = PyObject_SetItem(container, sub, v); - #line 595 "Python/executor_cases.c.h" Py_DECREF(v); Py_DECREF(container); Py_DECREF(sub); - #line 666 "Python/bytecodes.c" if (err) goto pop_3_error; - #line 601 "Python/executor_cases.c.h" STACK_SHRINK(3); break; } @@ -606,7 +512,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *list = stack_pointer[-2]; PyObject *value = stack_pointer[-3]; - #line 670 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), STORE_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), STORE_SUBSCR); @@ -623,7 +528,6 @@ Py_DECREF(old_value); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 627 "Python/executor_cases.c.h" STACK_SHRINK(3); break; } @@ -632,13 +536,11 @@ PyObject *sub = stack_pointer[-1]; PyObject *dict = stack_pointer[-2]; PyObject *value = stack_pointer[-3]; - #line 689 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(dict), STORE_SUBSCR); STAT_INC(STORE_SUBSCR, hit); int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub, value); Py_DECREF(dict); if (err) goto pop_3_error; - #line 642 "Python/executor_cases.c.h" STACK_SHRINK(3); break; } @@ -646,15 +548,11 @@ case DELETE_SUBSCR: { PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; - #line 697 "Python/bytecodes.c" /* del container[sub] */ int err = PyObject_DelItem(container, sub); - #line 653 "Python/executor_cases.c.h" Py_DECREF(container); Py_DECREF(sub); - #line 700 "Python/bytecodes.c" if (err) goto pop_2_error; - #line 658 "Python/executor_cases.c.h" STACK_SHRINK(2); break; } @@ -662,14 +560,10 @@ case CALL_INTRINSIC_1: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 704 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_1); res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value); - #line 669 "Python/executor_cases.c.h" Py_DECREF(value); - #line 707 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 673 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -678,15 +572,11 @@ PyObject *value1 = stack_pointer[-1]; PyObject *value2 = stack_pointer[-2]; PyObject *res; - #line 711 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_2); res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1); - #line 685 "Python/executor_cases.c.h" Py_DECREF(value2); Py_DECREF(value1); - #line 714 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 690 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -695,7 +585,6 @@ case GET_AITER: { PyObject *obj = stack_pointer[-1]; PyObject *iter; - #line 819 "Python/bytecodes.c" unaryfunc getter = NULL; PyTypeObject *type = Py_TYPE(obj); @@ -708,16 +597,12 @@ "'async for' requires an object with " "__aiter__ method, got %.100s", type->tp_name); - #line 712 "Python/executor_cases.c.h" Py_DECREF(obj); - #line 832 "Python/bytecodes.c" if (true) goto pop_1_error; } iter = (*getter)(obj); - #line 719 "Python/executor_cases.c.h" Py_DECREF(obj); - #line 837 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; if (Py_TYPE(iter)->tp_as_async == NULL || @@ -730,7 +615,6 @@ Py_DECREF(iter); if (true) goto pop_1_error; } - #line 734 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -738,7 +622,6 @@ case GET_ANEXT: { PyObject *aiter = stack_pointer[-1]; PyObject *awaitable; - #line 852 "Python/bytecodes.c" unaryfunc getter = NULL; PyObject *next_iter = NULL; PyTypeObject *type = Py_TYPE(aiter); @@ -781,7 +664,6 @@ Py_DECREF(next_iter); } } - #line 785 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = awaitable; break; @@ -790,16 +672,13 @@ case GET_AWAITABLE: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 897 "Python/bytecodes.c" iter = _PyCoro_GetAwaitableIter(iterable); if (iter == NULL) { format_awaitable_error(tstate, Py_TYPE(iterable), oparg); } - #line 801 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 904 "Python/bytecodes.c" if (iter != NULL && PyCoro_CheckExact(iter)) { PyObject *yf = _PyGen_yf((PyGenObject*)iter); @@ -816,26 +695,21 @@ } if (iter == NULL) goto pop_1_error; - #line 820 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } case POP_EXCEPT: { PyObject *exc_value = stack_pointer[-1]; - #line 1034 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; Py_XSETREF(exc_info->exc_value, exc_value); - #line 830 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case LOAD_ASSERTION_ERROR: { PyObject *value; - #line 1085 "Python/bytecodes.c" value = Py_NewRef(PyExc_AssertionError); - #line 839 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -843,14 +717,12 @@ case LOAD_BUILD_CLASS: { PyObject *bc; - #line 1089 "Python/bytecodes.c" if (PyMapping_GetOptionalItem(BUILTINS(), &_Py_ID(__build_class__), &bc) < 0) goto error; if (bc == NULL) { _PyErr_SetString(tstate, PyExc_NameError, "__build_class__ not found"); if (true) goto error; } - #line 854 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = bc; break; @@ -858,33 +730,26 @@ case STORE_NAME: { PyObject *v = stack_pointer[-1]; - #line 1099 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err; if (ns == NULL) { _PyErr_Format(tstate, PyExc_SystemError, "no locals found when storing %R", name); - #line 869 "Python/executor_cases.c.h" Py_DECREF(v); - #line 1106 "Python/bytecodes.c" if (true) goto pop_1_error; } if (PyDict_CheckExact(ns)) err = PyDict_SetItem(ns, name, v); else err = PyObject_SetItem(ns, name, v); - #line 878 "Python/executor_cases.c.h" Py_DECREF(v); - #line 1113 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 882 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case DELETE_NAME: { - #line 1117 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err; @@ -901,14 +766,12 @@ name); goto error; } - #line 905 "Python/executor_cases.c.h" break; } case UNPACK_SEQUENCE: { static_assert(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE == 1, "incorrect cache size"); PyObject *seq = stack_pointer[-1]; - #line 1143 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -921,11 +784,8 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject **top = stack_pointer + oparg - 1; int res = unpack_iterable(tstate, seq, oparg, -1, top); - #line 925 "Python/executor_cases.c.h" Py_DECREF(seq); - #line 1156 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 929 "Python/executor_cases.c.h" STACK_SHRINK(1); STACK_GROW(oparg); break; @@ -934,14 +794,12 @@ case UNPACK_SEQUENCE_TWO_TUPLE: { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1160 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != 2, UNPACK_SEQUENCE); assert(oparg == 2); STAT_INC(UNPACK_SEQUENCE, hit); values[0] = Py_NewRef(PyTuple_GET_ITEM(seq, 1)); values[1] = Py_NewRef(PyTuple_GET_ITEM(seq, 0)); - #line 945 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -951,7 +809,6 @@ case UNPACK_SEQUENCE_TUPLE: { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1170 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -959,7 +816,6 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 963 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -969,7 +825,6 @@ case UNPACK_SEQUENCE_LIST: { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1181 "Python/bytecodes.c" DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyList_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -977,7 +832,6 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 981 "Python/executor_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -986,49 +840,36 @@ case UNPACK_EX: { PyObject *seq = stack_pointer[-1]; - #line 1192 "Python/bytecodes.c" int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); - #line 994 "Python/executor_cases.c.h" Py_DECREF(seq); - #line 1196 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 998 "Python/executor_cases.c.h" STACK_GROW((oparg & 0xFF) + (oparg >> 8)); break; } case DELETE_ATTR: { PyObject *owner = stack_pointer[-1]; - #line 1227 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_DelAttr(owner, name); - #line 1008 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1230 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1012 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case STORE_GLOBAL: { PyObject *v = stack_pointer[-1]; - #line 1234 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyDict_SetItem(GLOBALS(), name, v); - #line 1022 "Python/executor_cases.c.h" Py_DECREF(v); - #line 1237 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1026 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case DELETE_GLOBAL: { - #line 1241 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err; err = PyDict_DelItem(GLOBALS(), name); @@ -1040,13 +881,11 @@ } goto error; } - #line 1044 "Python/executor_cases.c.h" break; } case _LOAD_LOCALS: { PyObject *locals; - #line 1255 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1054,7 +893,6 @@ if (true) goto error; } Py_INCREF(locals); - #line 1058 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = locals; break; @@ -1063,7 +901,6 @@ case _LOAD_FROM_DICT_OR_GLOBALS: { PyObject *mod_or_class_dict = stack_pointer[-1]; PyObject *v; - #line 1267 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); if (PyMapping_GetOptionalItem(mod_or_class_dict, name, &v) < 0) { Py_DECREF(mod_or_class_dict); @@ -1090,7 +927,6 @@ } } } - #line 1094 "Python/executor_cases.c.h" stack_pointer[-1] = v; break; } @@ -1099,7 +935,6 @@ static_assert(INLINE_CACHE_ENTRIES_LOAD_GLOBAL == 4, "incorrect cache size"); PyObject *null = NULL; PyObject *v; - #line 1306 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1146,7 +981,6 @@ } } null = NULL; - #line 1150 "Python/executor_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = v; @@ -1160,37 +994,30 @@ case _GUARD_GLOBALS_VERSION: { uint16_t version = (uint16_t)operand; - #line 1358 "Python/bytecodes.c" PyDictObject *dict = (PyDictObject *)GLOBALS(); DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); assert(DK_IS_UNICODE(dict->ma_keys)); - #line 1169 "Python/executor_cases.c.h" break; } case _GUARD_BUILTINS_VERSION: { uint16_t version = (uint16_t)operand; - #line 1365 "Python/bytecodes.c" PyDictObject *dict = (PyDictObject *)BUILTINS(); DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); assert(DK_IS_UNICODE(dict->ma_keys)); - #line 1180 "Python/executor_cases.c.h" break; } case DELETE_FAST: { - #line 1404 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL); - #line 1189 "Python/executor_cases.c.h" break; } case DELETE_DEREF: { - #line 1421 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); // Can't use ERROR_IF here. @@ -1201,14 +1028,12 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 1205 "Python/executor_cases.c.h" break; } case LOAD_FROM_DICT_OR_DEREF: { PyObject *class_dict = stack_pointer[-1]; PyObject *value; - #line 1434 "Python/bytecodes.c" PyObject *name; assert(class_dict); assert(oparg >= 0 && oparg < _PyFrame_GetCode(frame)->co_nlocalsplus); @@ -1227,14 +1052,12 @@ } Py_INCREF(value); } - #line 1231 "Python/executor_cases.c.h" stack_pointer[-1] = value; break; } case LOAD_DEREF: { PyObject *value; - #line 1455 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { @@ -1242,7 +1065,6 @@ if (true) goto error; } Py_INCREF(value); - #line 1246 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; break; @@ -1250,18 +1072,15 @@ case STORE_DEREF: { PyObject *v = stack_pointer[-1]; - #line 1465 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 1259 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } case COPY_FREE_VARS: { - #line 1472 "Python/bytecodes.c" /* Copy closure variables to free variables */ PyCodeObject *co = _PyFrame_GetCode(frame); assert(PyFunction_Check(frame->f_funcobj)); @@ -1272,22 +1091,17 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 1276 "Python/executor_cases.c.h" break; } case BUILD_STRING: { PyObject **pieces = (stack_pointer - oparg); PyObject *str; - #line 1485 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 1285 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } - #line 1487 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1291 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -1297,10 +1111,8 @@ case BUILD_TUPLE: { PyObject **values = (stack_pointer - oparg); PyObject *tup; - #line 1491 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1304 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -1310,10 +1122,8 @@ case BUILD_LIST: { PyObject **values = (stack_pointer - oparg); PyObject *list; - #line 1496 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 1317 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -1323,7 +1133,6 @@ case LIST_EXTEND: { PyObject *iterable = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 1501 "Python/bytecodes.c" PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable); if (none_val == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) && @@ -1334,13 +1143,10 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 1338 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 1512 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val)); - #line 1344 "Python/executor_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); break; @@ -1349,13 +1155,9 @@ case SET_UPDATE: { PyObject *iterable = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 1519 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 1355 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 1521 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 1359 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -1363,7 +1165,6 @@ case BUILD_SET: { PyObject **values = (stack_pointer - oparg); PyObject *set; - #line 1525 "Python/bytecodes.c" set = PySet_New(NULL); if (set == NULL) goto error; @@ -1378,7 +1179,6 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 1382 "Python/executor_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -1388,7 +1188,6 @@ case BUILD_MAP: { PyObject **values = (stack_pointer - oparg*2); PyObject *map; - #line 1542 "Python/bytecodes.c" map = _PyDict_FromItems( values, 2, values+1, 2, @@ -1396,13 +1195,10 @@ if (map == NULL) goto error; - #line 1400 "Python/executor_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } - #line 1550 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 1406 "Python/executor_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -1410,7 +1206,6 @@ } case SETUP_ANNOTATIONS: { - #line 1554 "Python/bytecodes.c" int err; PyObject *ann_dict; if (LOCALS() == NULL) { @@ -1448,7 +1243,6 @@ Py_DECREF(ann_dict); } } - #line 1452 "Python/executor_cases.c.h" break; } @@ -1456,7 +1250,6 @@ PyObject *keys = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); PyObject *map; - #line 1594 "Python/bytecodes.c" if (!PyTuple_CheckExact(keys) || PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1466,14 +1259,11 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 1470 "Python/executor_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); - #line 1604 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 1477 "Python/executor_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; break; @@ -1481,7 +1271,6 @@ case DICT_UPDATE: { PyObject *update = stack_pointer[-1]; - #line 1608 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (PyDict_Update(dict, update) < 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { @@ -1489,12 +1278,9 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 1493 "Python/executor_cases.c.h" Py_DECREF(update); - #line 1616 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1498 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1502,17 +1288,13 @@ case DICT_MERGE: { PyObject *update = stack_pointer[-1]; - #line 1622 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 1511 "Python/executor_cases.c.h" Py_DECREF(update); - #line 1627 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 1516 "Python/executor_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); break; @@ -1521,13 +1303,11 @@ case MAP_ADD: { PyObject *value = stack_pointer[-1]; PyObject *key = stack_pointer[-2]; - #line 1633 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack assert(PyDict_CheckExact(dict)); /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 1531 "Python/executor_cases.c.h" STACK_SHRINK(2); break; } @@ -1538,20 +1318,16 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1716 "Python/bytecodes.c" assert(!(oparg & 1)); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); STAT_INC(LOAD_SUPER_ATTR, hit); PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL); - #line 1549 "Python/executor_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1723 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; - #line 1555 "Python/executor_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -1565,7 +1341,6 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2; PyObject *res; - #line 1727 "Python/bytecodes.c" assert(oparg & 1); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); @@ -1588,7 +1363,6 @@ res = res2; res2 = NULL; } - #line 1592 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; stack_pointer[-2] = res2; @@ -1600,7 +1374,6 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1768 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1634,9 +1407,7 @@ NULL | meth | arg1 | ... | argN */ - #line 1638 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1802 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -1645,12 +1416,9 @@ else { /* Classic, pushes one value. */ res = PyObject_GetAttr(owner, name); - #line 1649 "Python/executor_cases.c.h" Py_DECREF(owner); - #line 1811 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } - #line 1654 "Python/executor_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -1662,7 +1430,6 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2044 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1675,10 +1442,8 @@ #endif /* ENABLE_SPECIALIZATION */ assert((oparg >> 5) <= Py_GE); res = PyObject_RichCompare(left, right, oparg >> 5); - #line 1679 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2057 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; if (oparg & 16) { int res_bool = PyObject_IsTrue(res); @@ -1686,7 +1451,6 @@ if (res_bool < 0) goto pop_2_error; res = res_bool ? Py_True : Py_False; } - #line 1690 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1696,7 +1460,6 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2067 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -1708,7 +1471,6 @@ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1712 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1718,7 +1480,6 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2082 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -1734,7 +1495,6 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1738 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1744,7 +1504,6 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2101 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -1757,7 +1516,6 @@ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 1761 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -1767,14 +1525,10 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2116 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 1773 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2118 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1778 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1784,15 +1538,11 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2122 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 1790 "Python/executor_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2124 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; - #line 1796 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; break; @@ -1803,12 +1553,9 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 2129 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 1809 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2131 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -1816,10 +1563,8 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 1820 "Python/executor_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2139 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -1828,7 +1573,6 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 1832 "Python/executor_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; break; @@ -1838,21 +1582,15 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2150 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 1845 "Python/executor_cases.c.h" Py_DECREF(right); - #line 2153 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 1852 "Python/executor_cases.c.h" Py_DECREF(right); - #line 2158 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 1856 "Python/executor_cases.c.h" stack_pointer[-1] = b; break; } @@ -1860,17 +1598,13 @@ case IS_NONE: { PyObject *value = stack_pointer[-1]; PyObject *b; - #line 2237 "Python/bytecodes.c" if (Py_IsNone(value)) { b = Py_True; } else { b = Py_False; - #line 1870 "Python/executor_cases.c.h" Py_DECREF(value); - #line 2243 "Python/bytecodes.c" } - #line 1874 "Python/executor_cases.c.h" stack_pointer[-1] = b; break; } @@ -1878,13 +1612,11 @@ case GET_LEN: { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2260 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 1888 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; break; @@ -1895,16 +1627,13 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2268 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 1904 "Python/executor_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2273 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -1912,7 +1641,6 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 1916 "Python/executor_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; break; @@ -1921,10 +1649,8 @@ case MATCH_MAPPING: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2283 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 1928 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1933,10 +1659,8 @@ case MATCH_SEQUENCE: { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2288 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 1940 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -1946,11 +1670,9 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2293 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 1954 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; break; @@ -1959,14 +1681,10 @@ case GET_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2299 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 1966 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2302 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 1970 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -1974,7 +1692,6 @@ case GET_YIELD_FROM_ITER: { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2306 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -1997,11 +1714,8 @@ if (iter == NULL) { goto error; } - #line 2001 "Python/executor_cases.c.h" Py_DECREF(iterable); - #line 2329 "Python/bytecodes.c" } - #line 2005 "Python/executor_cases.c.h" stack_pointer[-1] = iter; break; } @@ -2011,7 +1725,6 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2561 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -2032,7 +1745,6 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 2036 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; break; @@ -2041,7 +1753,6 @@ case PUSH_EXC_INFO: { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2600 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -2051,7 +1762,6 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 2055 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -2060,7 +1770,6 @@ case EXIT_INIT_CHECK: { PyObject *should_be_none = stack_pointer[-1]; - #line 2999 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -2068,7 +1777,6 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 2072 "Python/executor_cases.c.h" STACK_SHRINK(1); break; } @@ -2076,7 +1784,6 @@ case MAKE_FUNCTION: { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3413 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -2088,7 +1795,6 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 2092 "Python/executor_cases.c.h" stack_pointer[-1] = func; break; } @@ -2096,7 +1802,6 @@ case SET_FUNCTION_ATTRIBUTE: { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3427 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -2121,7 +1826,6 @@ default: Py_UNREACHABLE(); } - #line 2125 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; break; @@ -2132,15 +1836,11 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3477 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 2138 "Python/executor_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3479 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 2144 "Python/executor_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -2150,14 +1850,12 @@ case CONVERT_VALUE: { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3483 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 2161 "Python/executor_cases.c.h" stack_pointer[-1] = result; break; } @@ -2165,7 +1863,6 @@ case FORMAT_SIMPLE: { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3492 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -2176,7 +1873,6 @@ else { res = value; } - #line 2180 "Python/executor_cases.c.h" stack_pointer[-1] = res; break; } @@ -2185,12 +1881,10 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3505 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 2194 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -2199,10 +1893,8 @@ case COPY: { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3512 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 2206 "Python/executor_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; break; @@ -2213,7 +1905,6 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3517 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2228,12 +1919,9 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 2232 "Python/executor_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3532 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 2237 "Python/executor_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; break; @@ -2242,9 +1930,7 @@ case SWAP: { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3537 "Python/bytecodes.c" assert(oparg >= 2); - #line 2248 "Python/executor_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; break; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index fcdf732315830..f7a18b43ff11f 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -8,7 +8,6 @@ } TARGET(RESUME) { - #line 136 "Python/bytecodes.c" assert(tstate->cframe == &cframe); assert(frame == cframe.current_frame); /* Possibly combine this with eval breaker */ @@ -20,12 +19,10 @@ else if (oparg < 2) { CHECK_EVAL_BREAKER(); } - #line 24 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_RESUME) { - #line 150 "Python/bytecodes.c" /* Possible performance enhancement: * We need to check the eval breaker anyway, can we * combine the instrument verison check and the eval breaker test? @@ -51,17 +48,14 @@ DISPATCH(); } } - #line 55 "Python/generated_cases.c.h" DISPATCH(); } TARGET(LOAD_FAST_CHECK) { PyObject *value; - #line 182 "Python/bytecodes.c" value = GETLOCAL(oparg); if (value == NULL) goto unbound_local_error; Py_INCREF(value); - #line 65 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -69,11 +63,9 @@ TARGET(LOAD_FAST) { PyObject *value; - #line 188 "Python/bytecodes.c" value = GETLOCAL(oparg); assert(value != NULL); Py_INCREF(value); - #line 77 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -81,11 +73,9 @@ TARGET(LOAD_FAST_AND_CLEAR) { PyObject *value; - #line 194 "Python/bytecodes.c" value = GETLOCAL(oparg); // do not use SETLOCAL here, it decrefs the old value GETLOCAL(oparg) = NULL; - #line 89 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -94,14 +84,12 @@ TARGET(LOAD_FAST_LOAD_FAST) { PyObject *value1; PyObject *value2; - #line 200 "Python/bytecodes.c" uint32_t oparg1 = oparg >> 4; uint32_t oparg2 = oparg & 15; value1 = GETLOCAL(oparg1); value2 = GETLOCAL(oparg2); Py_INCREF(value1); Py_INCREF(value2); - #line 105 "Python/generated_cases.c.h" STACK_GROW(2); stack_pointer[-1] = value2; stack_pointer[-2] = value1; @@ -110,10 +98,8 @@ TARGET(LOAD_CONST) { PyObject *value; - #line 209 "Python/bytecodes.c" value = GETITEM(FRAME_CO_CONSTS, oparg); Py_INCREF(value); - #line 117 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -121,9 +107,7 @@ TARGET(STORE_FAST) { PyObject *value = stack_pointer[-1]; - #line 214 "Python/bytecodes.c" SETLOCAL(oparg, value); - #line 127 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -131,13 +115,11 @@ TARGET(STORE_FAST_LOAD_FAST) { PyObject *value1 = stack_pointer[-1]; PyObject *value2; - #line 222 "Python/bytecodes.c" uint32_t oparg1 = oparg >> 4; uint32_t oparg2 = oparg & 15; SETLOCAL(oparg1, value1); value2 = GETLOCAL(oparg2); Py_INCREF(value2); - #line 141 "Python/generated_cases.c.h" stack_pointer[-1] = value2; DISPATCH(); } @@ -145,20 +127,16 @@ TARGET(STORE_FAST_STORE_FAST) { PyObject *value1 = stack_pointer[-1]; PyObject *value2 = stack_pointer[-2]; - #line 230 "Python/bytecodes.c" uint32_t oparg1 = oparg >> 4; uint32_t oparg2 = oparg & 15; SETLOCAL(oparg1, value1); SETLOCAL(oparg2, value2); - #line 154 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } TARGET(POP_TOP) { PyObject *value = stack_pointer[-1]; - #line 237 "Python/bytecodes.c" - #line 162 "Python/generated_cases.c.h" Py_DECREF(value); STACK_SHRINK(1); DISPATCH(); @@ -166,9 +144,7 @@ TARGET(PUSH_NULL) { PyObject *res; - #line 241 "Python/bytecodes.c" res = NULL; - #line 172 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -179,14 +155,10 @@ PyObject *_tmp_2 = stack_pointer[-2]; { PyObject *value = _tmp_1; - #line 237 "Python/bytecodes.c" - #line 184 "Python/generated_cases.c.h" Py_DECREF(value); } { PyObject *value = _tmp_2; - #line 237 "Python/bytecodes.c" - #line 190 "Python/generated_cases.c.h" Py_DECREF(value); } STACK_SHRINK(2); @@ -196,7 +168,6 @@ TARGET(INSTRUMENTED_END_FOR) { PyObject *value = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; - #line 247 "Python/bytecodes.c" /* Need to create a fake StopIteration error here, * to conform to PEP 380 */ if (PyGen_Check(receiver)) { @@ -206,7 +177,6 @@ } PyErr_SetRaisedException(NULL); } - #line 210 "Python/generated_cases.c.h" Py_DECREF(receiver); Py_DECREF(value); STACK_SHRINK(2); @@ -216,9 +186,7 @@ TARGET(END_SEND) { PyObject *value = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; - #line 260 "Python/bytecodes.c" Py_DECREF(receiver); - #line 222 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = value; DISPATCH(); @@ -227,7 +195,6 @@ TARGET(INSTRUMENTED_END_SEND) { PyObject *value = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; - #line 264 "Python/bytecodes.c" if (PyGen_Check(receiver) || PyCoro_CheckExact(receiver)) { PyErr_SetObject(PyExc_StopIteration, value); if (monitor_stop_iteration(tstate, frame, next_instr-1)) { @@ -236,7 +203,6 @@ PyErr_SetRaisedException(NULL); } Py_DECREF(receiver); - #line 240 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = value; DISPATCH(); @@ -245,13 +211,9 @@ TARGET(UNARY_NEGATIVE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 275 "Python/bytecodes.c" res = PyNumber_Negative(value); - #line 251 "Python/generated_cases.c.h" Py_DECREF(value); - #line 277 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 255 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -259,10 +221,8 @@ TARGET(UNARY_NOT) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 281 "Python/bytecodes.c" assert(PyBool_Check(value)); res = Py_IsFalse(value) ? Py_True : Py_False; - #line 266 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -272,7 +232,6 @@ static_assert(INLINE_CACHE_ENTRIES_TO_BOOL == 3, "incorrect cache size"); PyObject *value = stack_pointer[-1]; PyObject *res; - #line 296 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyToBoolCache *cache = (_PyToBoolCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -284,12 +243,9 @@ DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ int err = PyObject_IsTrue(value); - #line 288 "Python/generated_cases.c.h" Py_DECREF(value); - #line 308 "Python/bytecodes.c" if (err < 0) goto pop_1_error; res = err ? Py_True : Py_False; - #line 293 "Python/generated_cases.c.h" stack_pointer[-1] = res; next_instr += 3; DISPATCH(); @@ -297,10 +253,8 @@ TARGET(TO_BOOL_BOOL) { PyObject *value = stack_pointer[-1]; - #line 313 "Python/bytecodes.c" DEOPT_IF(!PyBool_Check(value), TO_BOOL); STAT_INC(TO_BOOL, hit); - #line 304 "Python/generated_cases.c.h" next_instr += 3; DISPATCH(); } @@ -308,7 +262,6 @@ TARGET(TO_BOOL_INT) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 318 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(value), TO_BOOL); STAT_INC(TO_BOOL, hit); if (_PyLong_IsZero((PyLongObject *)value)) { @@ -316,12 +269,9 @@ res = Py_False; } else { - #line 320 "Python/generated_cases.c.h" Py_DECREF(value); - #line 326 "Python/bytecodes.c" res = Py_True; } - #line 325 "Python/generated_cases.c.h" stack_pointer[-1] = res; next_instr += 3; DISPATCH(); @@ -330,11 +280,9 @@ TARGET(TO_BOOL_LIST) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 331 "Python/bytecodes.c" DEOPT_IF(!PyList_CheckExact(value), TO_BOOL); STAT_INC(TO_BOOL, hit); res = Py_SIZE(value) ? Py_True : Py_False; - #line 338 "Python/generated_cases.c.h" Py_DECREF(value); stack_pointer[-1] = res; next_instr += 3; @@ -344,12 +292,10 @@ TARGET(TO_BOOL_NONE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 338 "Python/bytecodes.c" // This one is a bit weird, because we expect *some* failures: DEOPT_IF(!Py_IsNone(value), TO_BOOL); STAT_INC(TO_BOOL, hit); res = Py_False; - #line 353 "Python/generated_cases.c.h" stack_pointer[-1] = res; next_instr += 3; DISPATCH(); @@ -358,7 +304,6 @@ TARGET(TO_BOOL_STR) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 345 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(value), TO_BOOL); STAT_INC(TO_BOOL, hit); if (value == &_Py_STR(empty)) { @@ -367,12 +312,9 @@ } else { assert(Py_SIZE(value)); - #line 371 "Python/generated_cases.c.h" Py_DECREF(value); - #line 354 "Python/bytecodes.c" res = Py_True; } - #line 376 "Python/generated_cases.c.h" stack_pointer[-1] = res; next_instr += 3; DISPATCH(); @@ -382,16 +324,12 @@ PyObject *value = stack_pointer[-1]; PyObject *res; uint32_t version = read_u32(&next_instr[1].cache); - #line 359 "Python/bytecodes.c" // This one is a bit weird, because we expect *some* failures: assert(version); DEOPT_IF(Py_TYPE(value)->tp_version_tag != version, TO_BOOL); STAT_INC(TO_BOOL, hit); - #line 391 "Python/generated_cases.c.h" Py_DECREF(value); - #line 364 "Python/bytecodes.c" res = Py_True; - #line 395 "Python/generated_cases.c.h" stack_pointer[-1] = res; next_instr += 3; DISPATCH(); @@ -400,13 +338,9 @@ TARGET(UNARY_INVERT) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 368 "Python/bytecodes.c" res = PyNumber_Invert(value); - #line 406 "Python/generated_cases.c.h" Py_DECREF(value); - #line 370 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 410 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -417,10 +351,8 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2; - #line 386 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); - #line 424 "Python/generated_cases.c.h" _tmp_2 = left; _tmp_1 = right; } @@ -428,13 +360,11 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res; - #line 391 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = _PyLong_Multiply((PyLongObject *)left, (PyLongObject *)right); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 438 "Python/generated_cases.c.h" _tmp_2 = res; } next_instr += 1; @@ -449,10 +379,8 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2; - #line 386 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); - #line 456 "Python/generated_cases.c.h" _tmp_2 = left; _tmp_1 = right; } @@ -460,13 +388,11 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res; - #line 399 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = _PyLong_Add((PyLongObject *)left, (PyLongObject *)right); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 470 "Python/generated_cases.c.h" _tmp_2 = res; } next_instr += 1; @@ -481,10 +407,8 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2; - #line 386 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), BINARY_OP); DEOPT_IF(!PyLong_CheckExact(right), BINARY_OP); - #line 488 "Python/generated_cases.c.h" _tmp_2 = left; _tmp_1 = right; } @@ -492,13 +416,11 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res; - #line 407 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = _PyLong_Subtract((PyLongObject *)left, (PyLongObject *)right); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); if (res == NULL) goto pop_2_error; - #line 502 "Python/generated_cases.c.h" _tmp_2 = res; } next_instr += 1; @@ -513,10 +435,8 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2; - #line 422 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); - #line 520 "Python/generated_cases.c.h" _tmp_2 = left; _tmp_1 = right; } @@ -524,13 +444,11 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res; - #line 427 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); double dres = ((PyFloatObject *)left)->ob_fval * ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 534 "Python/generated_cases.c.h" _tmp_2 = res; } next_instr += 1; @@ -545,10 +463,8 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2; - #line 422 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); - #line 552 "Python/generated_cases.c.h" _tmp_2 = left; _tmp_1 = right; } @@ -556,13 +472,11 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res; - #line 435 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); double dres = ((PyFloatObject *)left)->ob_fval + ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 566 "Python/generated_cases.c.h" _tmp_2 = res; } next_instr += 1; @@ -577,10 +491,8 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2; - #line 422 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), BINARY_OP); DEOPT_IF(!PyFloat_CheckExact(right), BINARY_OP); - #line 584 "Python/generated_cases.c.h" _tmp_2 = left; _tmp_1 = right; } @@ -588,13 +500,11 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res; - #line 443 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); double dres = ((PyFloatObject *)left)->ob_fval - ((PyFloatObject *)right)->ob_fval; DECREF_INPUTS_AND_REUSE_FLOAT(left, right, dres, res); - #line 598 "Python/generated_cases.c.h" _tmp_2 = res; } next_instr += 1; @@ -609,10 +519,8 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2; - #line 458 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(!PyUnicode_CheckExact(right), BINARY_OP); - #line 616 "Python/generated_cases.c.h" _tmp_2 = left; _tmp_1 = right; } @@ -620,13 +528,11 @@ PyObject *right = _tmp_1; PyObject *left = _tmp_2; PyObject *res; - #line 463 "Python/bytecodes.c" STAT_INC(BINARY_OP, hit); res = PyUnicode_Concat(left, right); _Py_DECREF_SPECIALIZED(left, _PyUnicode_ExactDealloc); _Py_DECREF_SPECIALIZED(right, _PyUnicode_ExactDealloc); if (res == NULL) goto pop_2_error; - #line 630 "Python/generated_cases.c.h" _tmp_2 = res; } next_instr += 1; @@ -641,17 +547,14 @@ { PyObject *right = _tmp_1; PyObject *left = _tmp_2; - #line 458 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), BINARY_OP); DEOPT_IF(!PyUnicode_CheckExact(right), BINARY_OP); - #line 648 "Python/generated_cases.c.h" _tmp_2 = left; _tmp_1 = right; } { PyObject *right = _tmp_1; PyObject *left = _tmp_2; - #line 480 "Python/bytecodes.c" _Py_CODEUNIT true_next = next_instr[INLINE_CACHE_ENTRIES_BINARY_OP]; assert(true_next.op.code == STORE_FAST); PyObject **target_local = &GETLOCAL(true_next.op.arg); @@ -675,7 +578,6 @@ if (*target_local == NULL) goto pop_2_error; // The STORE_FAST is already done. SKIP_OVER(INLINE_CACHE_ENTRIES_BINARY_OP + 1); - #line 679 "Python/generated_cases.c.h" } STACK_SHRINK(2); DISPATCH(); @@ -687,7 +589,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; PyObject *res; - #line 517 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinarySubscrCache *cache = (_PyBinarySubscrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -699,12 +600,9 @@ DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ res = PyObject_GetItem(container, sub); - #line 703 "Python/generated_cases.c.h" Py_DECREF(container); Py_DECREF(sub); - #line 529 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 708 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -716,7 +614,6 @@ PyObject *start = stack_pointer[-2]; PyObject *container = stack_pointer[-3]; PyObject *res; - #line 533 "Python/bytecodes.c" PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop); // Can't use ERROR_IF() here, because we haven't // DECREF'ed container yet, and we still own slice. @@ -729,7 +626,6 @@ } Py_DECREF(container); if (res == NULL) goto pop_3_error; - #line 733 "Python/generated_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = res; DISPATCH(); @@ -740,7 +636,6 @@ PyObject *start = stack_pointer[-2]; PyObject *container = stack_pointer[-3]; PyObject *v = stack_pointer[-4]; - #line 548 "Python/bytecodes.c" PyObject *slice = _PyBuildSlice_ConsumeRefs(start, stop); int err; if (slice == NULL) { @@ -753,7 +648,6 @@ Py_DECREF(v); Py_DECREF(container); if (err) goto pop_4_error; - #line 757 "Python/generated_cases.c.h" STACK_SHRINK(4); DISPATCH(); } @@ -762,7 +656,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *list = stack_pointer[-2]; PyObject *res; - #line 563 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), BINARY_SUBSCR); @@ -776,7 +669,6 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 780 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -787,7 +679,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *tuple = stack_pointer[-2]; PyObject *res; - #line 579 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), BINARY_SUBSCR); DEOPT_IF(!PyTuple_CheckExact(tuple), BINARY_SUBSCR); @@ -801,7 +692,6 @@ Py_INCREF(res); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(tuple); - #line 805 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -812,7 +702,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *dict = stack_pointer[-2]; PyObject *res; - #line 595 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(dict), BINARY_SUBSCR); STAT_INC(BINARY_SUBSCR, hit); res = PyDict_GetItemWithError(dict, sub); @@ -820,14 +709,11 @@ if (!_PyErr_Occurred(tstate)) { _PyErr_SetKeyError(sub); } - #line 824 "Python/generated_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); - #line 603 "Python/bytecodes.c" if (true) goto pop_2_error; } Py_INCREF(res); // Do this before DECREF'ing dict, sub - #line 831 "Python/generated_cases.c.h" Py_DECREF(dict); Py_DECREF(sub); STACK_SHRINK(1); @@ -839,7 +725,6 @@ TARGET(BINARY_SUBSCR_GETITEM) { PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; - #line 610 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, BINARY_SUBSCR); PyTypeObject *tp = Py_TYPE(container); DEOPT_IF(!PyType_HasFeature(tp, Py_TPFLAGS_HEAPTYPE), BINARY_SUBSCR); @@ -862,15 +747,12 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_BINARY_SUBSCR); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 866 "Python/generated_cases.c.h" } TARGET(LIST_APPEND) { PyObject *v = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 635 "Python/bytecodes.c" if (_PyList_AppendTakeRef((PyListObject *)list, v) < 0) goto pop_1_error; - #line 874 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -878,13 +760,9 @@ TARGET(SET_ADD) { PyObject *v = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 639 "Python/bytecodes.c" int err = PySet_Add(set, v); - #line 884 "Python/generated_cases.c.h" Py_DECREF(v); - #line 641 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 888 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -896,7 +774,6 @@ PyObject *container = stack_pointer[-2]; PyObject *v = stack_pointer[-3]; uint16_t counter = read_u16(&next_instr[0].cache); - #line 651 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { next_instr--; @@ -911,13 +788,10 @@ #endif /* ENABLE_SPECIALIZATION */ /* container[sub] = v */ int err = PyObject_SetItem(container, sub, v); - #line 915 "Python/generated_cases.c.h" Py_DECREF(v); Py_DECREF(container); Py_DECREF(sub); - #line 666 "Python/bytecodes.c" if (err) goto pop_3_error; - #line 921 "Python/generated_cases.c.h" STACK_SHRINK(3); next_instr += 1; DISPATCH(); @@ -927,7 +801,6 @@ PyObject *sub = stack_pointer[-1]; PyObject *list = stack_pointer[-2]; PyObject *value = stack_pointer[-3]; - #line 670 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(sub), STORE_SUBSCR); DEOPT_IF(!PyList_CheckExact(list), STORE_SUBSCR); @@ -944,7 +817,6 @@ Py_DECREF(old_value); _Py_DECREF_SPECIALIZED(sub, (destructor)PyObject_Free); Py_DECREF(list); - #line 948 "Python/generated_cases.c.h" STACK_SHRINK(3); next_instr += 1; DISPATCH(); @@ -954,13 +826,11 @@ PyObject *sub = stack_pointer[-1]; PyObject *dict = stack_pointer[-2]; PyObject *value = stack_pointer[-3]; - #line 689 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(dict), STORE_SUBSCR); STAT_INC(STORE_SUBSCR, hit); int err = _PyDict_SetItem_Take2((PyDictObject *)dict, sub, value); Py_DECREF(dict); if (err) goto pop_3_error; - #line 964 "Python/generated_cases.c.h" STACK_SHRINK(3); next_instr += 1; DISPATCH(); @@ -969,15 +839,11 @@ TARGET(DELETE_SUBSCR) { PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; - #line 697 "Python/bytecodes.c" /* del container[sub] */ int err = PyObject_DelItem(container, sub); - #line 976 "Python/generated_cases.c.h" Py_DECREF(container); Py_DECREF(sub); - #line 700 "Python/bytecodes.c" if (err) goto pop_2_error; - #line 981 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } @@ -985,14 +851,10 @@ TARGET(CALL_INTRINSIC_1) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 704 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_1); res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value); - #line 992 "Python/generated_cases.c.h" Py_DECREF(value); - #line 707 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; - #line 996 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -1001,15 +863,11 @@ PyObject *value1 = stack_pointer[-1]; PyObject *value2 = stack_pointer[-2]; PyObject *res; - #line 711 "Python/bytecodes.c" assert(oparg <= MAX_INTRINSIC_2); res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1); - #line 1008 "Python/generated_cases.c.h" Py_DECREF(value2); Py_DECREF(value1); - #line 714 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 1013 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -1017,7 +875,6 @@ TARGET(RAISE_VARARGS) { PyObject **args = (stack_pointer - oparg); - #line 718 "Python/bytecodes.c" PyObject *cause = NULL, *exc = NULL; switch (oparg) { case 2: @@ -1035,12 +892,10 @@ break; } if (true) { STACK_SHRINK(oparg); goto error; } - #line 1039 "Python/generated_cases.c.h" } TARGET(INTERPRETER_EXIT) { PyObject *retval = stack_pointer[-1]; - #line 738 "Python/bytecodes.c" assert(frame == &entry_frame); assert(_PyFrame_IsIncomplete(frame)); /* Restore previous cframe and return. */ @@ -1049,12 +904,10 @@ assert(!_PyErr_Occurred(tstate)); _Py_LeaveRecursiveCallTstate(tstate); return retval; - #line 1053 "Python/generated_cases.c.h" } TARGET(RETURN_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 749 "Python/bytecodes.c" STACK_SHRINK(1); assert(EMPTY()); _PyFrame_SetStackPointer(frame, stack_pointer); @@ -1067,12 +920,10 @@ frame->prev_instr += frame->return_offset; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1071 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_RETURN_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 764 "Python/bytecodes.c" int err = _Py_call_instrumentation_arg( tstate, PY_MONITORING_EVENT_PY_RETURN, frame, next_instr-1, retval); @@ -1089,11 +940,9 @@ frame->prev_instr += frame->return_offset; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1093 "Python/generated_cases.c.h" } TARGET(RETURN_CONST) { - #line 783 "Python/bytecodes.c" PyObject *retval = GETITEM(FRAME_CO_CONSTS, oparg); Py_INCREF(retval); assert(EMPTY()); @@ -1107,11 +956,9 @@ frame->prev_instr += frame->return_offset; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1111 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_RETURN_CONST) { - #line 799 "Python/bytecodes.c" PyObject *retval = GETITEM(FRAME_CO_CONSTS, oparg); int err = _Py_call_instrumentation_arg( tstate, PY_MONITORING_EVENT_PY_RETURN, @@ -1129,13 +976,11 @@ frame->prev_instr += frame->return_offset; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1133 "Python/generated_cases.c.h" } TARGET(GET_AITER) { PyObject *obj = stack_pointer[-1]; PyObject *iter; - #line 819 "Python/bytecodes.c" unaryfunc getter = NULL; PyTypeObject *type = Py_TYPE(obj); @@ -1148,16 +993,12 @@ "'async for' requires an object with " "__aiter__ method, got %.100s", type->tp_name); - #line 1152 "Python/generated_cases.c.h" Py_DECREF(obj); - #line 832 "Python/bytecodes.c" if (true) goto pop_1_error; } iter = (*getter)(obj); - #line 1159 "Python/generated_cases.c.h" Py_DECREF(obj); - #line 837 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; if (Py_TYPE(iter)->tp_as_async == NULL || @@ -1170,7 +1011,6 @@ Py_DECREF(iter); if (true) goto pop_1_error; } - #line 1174 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -1178,7 +1018,6 @@ TARGET(GET_ANEXT) { PyObject *aiter = stack_pointer[-1]; PyObject *awaitable; - #line 852 "Python/bytecodes.c" unaryfunc getter = NULL; PyObject *next_iter = NULL; PyTypeObject *type = Py_TYPE(aiter); @@ -1221,7 +1060,6 @@ Py_DECREF(next_iter); } } - #line 1225 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = awaitable; DISPATCH(); @@ -1230,16 +1068,13 @@ TARGET(GET_AWAITABLE) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 897 "Python/bytecodes.c" iter = _PyCoro_GetAwaitableIter(iterable); if (iter == NULL) { format_awaitable_error(tstate, Py_TYPE(iterable), oparg); } - #line 1241 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 904 "Python/bytecodes.c" if (iter != NULL && PyCoro_CheckExact(iter)) { PyObject *yf = _PyGen_yf((PyGenObject*)iter); @@ -1256,7 +1091,6 @@ } if (iter == NULL) goto pop_1_error; - #line 1260 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -1267,7 +1101,6 @@ PyObject *v = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; PyObject *retval; - #line 928 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PySendCache *cache = (_PySendCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1314,7 +1147,6 @@ } } Py_DECREF(v); - #line 1318 "Python/generated_cases.c.h" stack_pointer[-1] = retval; next_instr += 1; DISPATCH(); @@ -1323,7 +1155,6 @@ TARGET(SEND_GEN) { PyObject *v = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; - #line 977 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, SEND); PyGenObject *gen = (PyGenObject *)receiver; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type && @@ -1339,12 +1170,10 @@ tstate->exc_info = &gen->gi_exc_state; SKIP_OVER(INLINE_CACHE_ENTRIES_SEND); DISPATCH_INLINED(gen_frame); - #line 1343 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_YIELD_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 995 "Python/bytecodes.c" assert(frame != &entry_frame); assert(oparg >= 0); /* make the generator identify this as HAS_ARG */ PyGenObject *gen = _PyFrame_GetGenerator(frame); @@ -1362,12 +1191,10 @@ gen_frame->previous = NULL; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1366 "Python/generated_cases.c.h" } TARGET(YIELD_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 1015 "Python/bytecodes.c" // NOTE: It's important that YIELD_VALUE never raises an exception! // The compiler treats any exception raised here as a failed close() // or throw() call. @@ -1384,15 +1211,12 @@ gen_frame->previous = NULL; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1388 "Python/generated_cases.c.h" } TARGET(POP_EXCEPT) { PyObject *exc_value = stack_pointer[-1]; - #line 1034 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; Py_XSETREF(exc_info->exc_value, exc_value); - #line 1396 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -1400,7 +1224,6 @@ TARGET(RERAISE) { PyObject *exc = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); - #line 1039 "Python/bytecodes.c" assert(oparg >= 0 && oparg <= 2); if (oparg) { PyObject *lasti = values[0]; @@ -1418,26 +1241,21 @@ Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); goto exception_unwind; - #line 1422 "Python/generated_cases.c.h" } TARGET(END_ASYNC_FOR) { PyObject *exc = stack_pointer[-1]; PyObject *awaitable = stack_pointer[-2]; - #line 1059 "Python/bytecodes.c" assert(exc && PyExceptionInstance_Check(exc)); if (PyErr_GivenExceptionMatches(exc, PyExc_StopAsyncIteration)) { - #line 1431 "Python/generated_cases.c.h" Py_DECREF(awaitable); Py_DECREF(exc); - #line 1062 "Python/bytecodes.c" } else { Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); goto exception_unwind; } - #line 1441 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } @@ -1448,23 +1266,19 @@ PyObject *sub_iter = stack_pointer[-3]; PyObject *none; PyObject *value; - #line 1071 "Python/bytecodes.c" assert(throwflag); assert(exc_value && PyExceptionInstance_Check(exc_value)); if (PyErr_GivenExceptionMatches(exc_value, PyExc_StopIteration)) { value = Py_NewRef(((PyStopIterationObject *)exc_value)->value); - #line 1457 "Python/generated_cases.c.h" Py_DECREF(sub_iter); Py_DECREF(last_sent_val); Py_DECREF(exc_value); - #line 1076 "Python/bytecodes.c" none = Py_None; } else { _PyErr_SetRaisedException(tstate, Py_NewRef(exc_value)); goto exception_unwind; } - #line 1468 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = value; stack_pointer[-2] = none; @@ -1473,9 +1287,7 @@ TARGET(LOAD_ASSERTION_ERROR) { PyObject *value; - #line 1085 "Python/bytecodes.c" value = Py_NewRef(PyExc_AssertionError); - #line 1479 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -1483,14 +1295,12 @@ TARGET(LOAD_BUILD_CLASS) { PyObject *bc; - #line 1089 "Python/bytecodes.c" if (PyMapping_GetOptionalItem(BUILTINS(), &_Py_ID(__build_class__), &bc) < 0) goto error; if (bc == NULL) { _PyErr_SetString(tstate, PyExc_NameError, "__build_class__ not found"); if (true) goto error; } - #line 1494 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = bc; DISPATCH(); @@ -1498,33 +1308,26 @@ TARGET(STORE_NAME) { PyObject *v = stack_pointer[-1]; - #line 1099 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err; if (ns == NULL) { _PyErr_Format(tstate, PyExc_SystemError, "no locals found when storing %R", name); - #line 1509 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1106 "Python/bytecodes.c" if (true) goto pop_1_error; } if (PyDict_CheckExact(ns)) err = PyDict_SetItem(ns, name, v); else err = PyObject_SetItem(ns, name, v); - #line 1518 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1113 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1522 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(DELETE_NAME) { - #line 1117 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); PyObject *ns = LOCALS(); int err; @@ -1541,7 +1344,6 @@ name); goto error; } - #line 1545 "Python/generated_cases.c.h" DISPATCH(); } @@ -1549,7 +1351,6 @@ PREDICTED(UNPACK_SEQUENCE); static_assert(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE == 1, "incorrect cache size"); PyObject *seq = stack_pointer[-1]; - #line 1143 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1562,11 +1363,8 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject **top = stack_pointer + oparg - 1; int res = unpack_iterable(tstate, seq, oparg, -1, top); - #line 1566 "Python/generated_cases.c.h" Py_DECREF(seq); - #line 1156 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 1570 "Python/generated_cases.c.h" STACK_SHRINK(1); STACK_GROW(oparg); next_instr += 1; @@ -1576,14 +1374,12 @@ TARGET(UNPACK_SEQUENCE_TWO_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1160 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != 2, UNPACK_SEQUENCE); assert(oparg == 2); STAT_INC(UNPACK_SEQUENCE, hit); values[0] = Py_NewRef(PyTuple_GET_ITEM(seq, 1)); values[1] = Py_NewRef(PyTuple_GET_ITEM(seq, 0)); - #line 1587 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1594,7 +1390,6 @@ TARGET(UNPACK_SEQUENCE_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1170 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -1602,7 +1397,6 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 1606 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1613,7 +1407,6 @@ TARGET(UNPACK_SEQUENCE_LIST) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1181 "Python/bytecodes.c" DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyList_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -1621,7 +1414,6 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 1625 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1631,15 +1423,11 @@ TARGET(UNPACK_EX) { PyObject *seq = stack_pointer[-1]; - #line 1192 "Python/bytecodes.c" int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); - #line 1639 "Python/generated_cases.c.h" Py_DECREF(seq); - #line 1196 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 1643 "Python/generated_cases.c.h" STACK_GROW((oparg & 0xFF) + (oparg >> 8)); DISPATCH(); } @@ -1650,7 +1438,6 @@ PyObject *owner = stack_pointer[-1]; PyObject *v = stack_pointer[-2]; uint16_t counter = read_u16(&next_instr[0].cache); - #line 1207 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); @@ -1666,12 +1453,9 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_SetAttr(owner, name, v); - #line 1670 "Python/generated_cases.c.h" Py_DECREF(v); Py_DECREF(owner); - #line 1223 "Python/bytecodes.c" if (err) goto pop_2_error; - #line 1675 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -1679,34 +1463,25 @@ TARGET(DELETE_ATTR) { PyObject *owner = stack_pointer[-1]; - #line 1227 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_DelAttr(owner, name); - #line 1686 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1230 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1690 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(STORE_GLOBAL) { PyObject *v = stack_pointer[-1]; - #line 1234 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyDict_SetItem(GLOBALS(), name, v); - #line 1700 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1237 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1704 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(DELETE_GLOBAL) { - #line 1241 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err; err = PyDict_DelItem(GLOBALS(), name); @@ -1718,7 +1493,6 @@ } goto error; } - #line 1722 "Python/generated_cases.c.h" DISPATCH(); } @@ -1726,7 +1500,6 @@ PyObject *_tmp_1; { PyObject *locals; - #line 1255 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1734,7 +1507,6 @@ if (true) goto error; } Py_INCREF(locals); - #line 1738 "Python/generated_cases.c.h" _tmp_1 = locals; } STACK_GROW(1); @@ -1746,7 +1518,6 @@ PyObject *_tmp_1; { PyObject *locals; - #line 1255 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1754,13 +1525,11 @@ if (true) goto error; } Py_INCREF(locals); - #line 1758 "Python/generated_cases.c.h" _tmp_1 = locals; } { PyObject *mod_or_class_dict = _tmp_1; PyObject *v; - #line 1267 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); if (PyMapping_GetOptionalItem(mod_or_class_dict, name, &v) < 0) { Py_DECREF(mod_or_class_dict); @@ -1787,7 +1556,6 @@ } } } - #line 1791 "Python/generated_cases.c.h" _tmp_1 = v; } STACK_GROW(1); @@ -1800,7 +1568,6 @@ { PyObject *mod_or_class_dict = _tmp_1; PyObject *v; - #line 1267 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); if (PyMapping_GetOptionalItem(mod_or_class_dict, name, &v) < 0) { Py_DECREF(mod_or_class_dict); @@ -1827,7 +1594,6 @@ } } } - #line 1831 "Python/generated_cases.c.h" _tmp_1 = v; } stack_pointer[-1] = _tmp_1; @@ -1839,7 +1605,6 @@ static_assert(INLINE_CACHE_ENTRIES_LOAD_GLOBAL == 4, "incorrect cache size"); PyObject *null = NULL; PyObject *v; - #line 1306 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1886,7 +1651,6 @@ } } null = NULL; - #line 1890 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = v; @@ -1902,12 +1666,10 @@ } { uint16_t version = read_u16(&next_instr[1].cache); - #line 1358 "Python/bytecodes.c" PyDictObject *dict = (PyDictObject *)GLOBALS(); DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); assert(DK_IS_UNICODE(dict->ma_keys)); - #line 1911 "Python/generated_cases.c.h" } { } @@ -1915,7 +1677,6 @@ PyObject *null = NULL; PyObject *res; uint16_t index = read_u16(&next_instr[3].cache); - #line 1372 "Python/bytecodes.c" PyDictObject *dict = (PyDictObject *)GLOBALS(); PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(dict->ma_keys); res = entries[index].me_value; @@ -1923,7 +1684,6 @@ Py_INCREF(res); STAT_INC(LOAD_GLOBAL, hit); null = NULL; - #line 1927 "Python/generated_cases.c.h" if (oparg & 1) { _tmp_2 = null; } _tmp_1 = res; } @@ -1942,27 +1702,22 @@ } { uint16_t version = read_u16(&next_instr[1].cache); - #line 1358 "Python/bytecodes.c" PyDictObject *dict = (PyDictObject *)GLOBALS(); DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); assert(DK_IS_UNICODE(dict->ma_keys)); - #line 1951 "Python/generated_cases.c.h" } { uint16_t version = read_u16(&next_instr[2].cache); - #line 1365 "Python/bytecodes.c" PyDictObject *dict = (PyDictObject *)BUILTINS(); DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); assert(DK_IS_UNICODE(dict->ma_keys)); - #line 1960 "Python/generated_cases.c.h" } { PyObject *null = NULL; PyObject *res; uint16_t index = read_u16(&next_instr[3].cache); - #line 1382 "Python/bytecodes.c" PyDictObject *bdict = (PyDictObject *)BUILTINS(); PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(bdict->ma_keys); res = entries[index].me_value; @@ -1970,7 +1725,6 @@ Py_INCREF(res); STAT_INC(LOAD_GLOBAL, hit); null = NULL; - #line 1974 "Python/generated_cases.c.h" if (oparg & 1) { _tmp_2 = null; } _tmp_1 = res; } @@ -1983,16 +1737,13 @@ } TARGET(DELETE_FAST) { - #line 1404 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL); - #line 1991 "Python/generated_cases.c.h" DISPATCH(); } TARGET(MAKE_CELL) { - #line 1410 "Python/bytecodes.c" // "initial" is probably NULL but not if it's an arg (or set // via PyFrame_LocalsToFast() before MAKE_CELL has run). PyObject *initial = GETLOCAL(oparg); @@ -2001,12 +1752,10 @@ goto resume_with_error; } SETLOCAL(oparg, cell); - #line 2005 "Python/generated_cases.c.h" DISPATCH(); } TARGET(DELETE_DEREF) { - #line 1421 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); // Can't use ERROR_IF here. @@ -2017,14 +1766,12 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 2021 "Python/generated_cases.c.h" DISPATCH(); } TARGET(LOAD_FROM_DICT_OR_DEREF) { PyObject *class_dict = stack_pointer[-1]; PyObject *value; - #line 1434 "Python/bytecodes.c" PyObject *name; assert(class_dict); assert(oparg >= 0 && oparg < _PyFrame_GetCode(frame)->co_nlocalsplus); @@ -2043,14 +1790,12 @@ } Py_INCREF(value); } - #line 2047 "Python/generated_cases.c.h" stack_pointer[-1] = value; DISPATCH(); } TARGET(LOAD_DEREF) { PyObject *value; - #line 1455 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { @@ -2058,7 +1803,6 @@ if (true) goto error; } Py_INCREF(value); - #line 2062 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -2066,18 +1810,15 @@ TARGET(STORE_DEREF) { PyObject *v = stack_pointer[-1]; - #line 1465 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 2075 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(COPY_FREE_VARS) { - #line 1472 "Python/bytecodes.c" /* Copy closure variables to free variables */ PyCodeObject *co = _PyFrame_GetCode(frame); assert(PyFunction_Check(frame->f_funcobj)); @@ -2088,22 +1829,17 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 2092 "Python/generated_cases.c.h" DISPATCH(); } TARGET(BUILD_STRING) { PyObject **pieces = (stack_pointer - oparg); PyObject *str; - #line 1485 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 2101 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } - #line 1487 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2107 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -2113,10 +1849,8 @@ TARGET(BUILD_TUPLE) { PyObject **values = (stack_pointer - oparg); PyObject *tup; - #line 1491 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2120 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -2126,10 +1860,8 @@ TARGET(BUILD_LIST) { PyObject **values = (stack_pointer - oparg); PyObject *list; - #line 1496 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2133 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -2139,7 +1871,6 @@ TARGET(LIST_EXTEND) { PyObject *iterable = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 1501 "Python/bytecodes.c" PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable); if (none_val == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) && @@ -2150,13 +1881,10 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 2154 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1512 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val)); - #line 2160 "Python/generated_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); DISPATCH(); @@ -2165,13 +1893,9 @@ TARGET(SET_UPDATE) { PyObject *iterable = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 1519 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 2171 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1521 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 2175 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -2179,7 +1903,6 @@ TARGET(BUILD_SET) { PyObject **values = (stack_pointer - oparg); PyObject *set; - #line 1525 "Python/bytecodes.c" set = PySet_New(NULL); if (set == NULL) goto error; @@ -2194,7 +1917,6 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 2198 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -2204,7 +1926,6 @@ TARGET(BUILD_MAP) { PyObject **values = (stack_pointer - oparg*2); PyObject *map; - #line 1542 "Python/bytecodes.c" map = _PyDict_FromItems( values, 2, values+1, 2, @@ -2212,13 +1933,10 @@ if (map == NULL) goto error; - #line 2216 "Python/generated_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } - #line 1550 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 2222 "Python/generated_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -2226,7 +1944,6 @@ } TARGET(SETUP_ANNOTATIONS) { - #line 1554 "Python/bytecodes.c" int err; PyObject *ann_dict; if (LOCALS() == NULL) { @@ -2264,7 +1981,6 @@ Py_DECREF(ann_dict); } } - #line 2268 "Python/generated_cases.c.h" DISPATCH(); } @@ -2272,7 +1988,6 @@ PyObject *keys = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); PyObject *map; - #line 1594 "Python/bytecodes.c" if (!PyTuple_CheckExact(keys) || PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -2282,14 +1997,11 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 2286 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); - #line 1604 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 2293 "Python/generated_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; DISPATCH(); @@ -2297,7 +2009,6 @@ TARGET(DICT_UPDATE) { PyObject *update = stack_pointer[-1]; - #line 1608 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (PyDict_Update(dict, update) < 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { @@ -2305,12 +2016,9 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 2309 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1616 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 2314 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); DISPATCH(); @@ -2318,17 +2026,13 @@ TARGET(DICT_MERGE) { PyObject *update = stack_pointer[-1]; - #line 1622 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 2327 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1627 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 2332 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); DISPATCH(); @@ -2337,25 +2041,21 @@ TARGET(MAP_ADD) { PyObject *value = stack_pointer[-1]; PyObject *key = stack_pointer[-2]; - #line 1633 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack assert(PyDict_CheckExact(dict)); /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 2347 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } TARGET(INSTRUMENTED_LOAD_SUPER_ATTR) { - #line 1641 "Python/bytecodes.c" _PySuperAttrCache *cache = (_PySuperAttrCache *)next_instr; // cancel out the decrement that will happen in LOAD_SUPER_ATTR; we // don't want to specialize instrumented instructions INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(LOAD_SUPER_ATTR); - #line 2359 "Python/generated_cases.c.h" } TARGET(LOAD_SUPER_ATTR) { @@ -2366,7 +2066,6 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1655 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); int load_method = oparg & 1; #if ENABLE_SPECIALIZATION @@ -2408,16 +2107,13 @@ } } } - #line 2412 "Python/generated_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1697 "Python/bytecodes.c" if (super == NULL) goto pop_3_error; res = PyObject_GetAttr(super, name); Py_DECREF(super); if (res == NULL) goto pop_3_error; - #line 2421 "Python/generated_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2432,20 +2128,16 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1716 "Python/bytecodes.c" assert(!(oparg & 1)); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); STAT_INC(LOAD_SUPER_ATTR, hit); PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL); - #line 2443 "Python/generated_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1723 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; - #line 2449 "Python/generated_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2460,7 +2152,6 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2; PyObject *res; - #line 1727 "Python/bytecodes.c" assert(oparg & 1); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); @@ -2483,7 +2174,6 @@ res = res2; res2 = NULL; } - #line 2487 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; stack_pointer[-2] = res2; @@ -2497,7 +2187,6 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1768 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2531,9 +2220,7 @@ NULL | meth | arg1 | ... | argN */ - #line 2535 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1802 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -2542,12 +2229,9 @@ else { /* Classic, pushes one value. */ res = PyObject_GetAttr(owner, name); - #line 2546 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1811 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } - #line 2551 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -2561,7 +2245,6 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1820 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2574,7 +2257,6 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2578 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2589,7 +2271,6 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1836 "Python/bytecodes.c" DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; assert(dict != NULL); @@ -2602,7 +2283,6 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2606 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2617,7 +2297,6 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1852 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2644,7 +2323,6 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2648 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2659,7 +2337,6 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1882 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2669,7 +2346,6 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2673 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2684,7 +2360,6 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 1895 "Python/bytecodes.c" DEOPT_IF(!PyType_Check(cls), LOAD_ATTR); DEOPT_IF(((PyTypeObject *)cls)->tp_version_tag != type_version, @@ -2696,7 +2371,6 @@ res = descr; assert(res != NULL); Py_INCREF(res); - #line 2700 "Python/generated_cases.c.h" Py_DECREF(cls); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2710,7 +2384,6 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *fget = read_obj(&next_instr[5].cache); - #line 1910 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); @@ -2734,7 +2407,6 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_LOAD_ATTR); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 2738 "Python/generated_cases.c.h" } TARGET(LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN) { @@ -2742,7 +2414,6 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *getattribute = read_obj(&next_instr[5].cache); - #line 1936 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); DEOPT_IF(cls->tp_version_tag != type_version, LOAD_ATTR); @@ -2768,7 +2439,6 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_LOAD_ATTR); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 2772 "Python/generated_cases.c.h" } TARGET(STORE_ATTR_INSTANCE_VALUE) { @@ -2776,7 +2446,6 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1964 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2794,7 +2463,6 @@ Py_DECREF(old_value); } Py_DECREF(owner); - #line 2798 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2805,7 +2473,6 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t hint = read_u16(&next_instr[3].cache); - #line 1984 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2844,7 +2511,6 @@ /* PEP 509 */ dict->ma_version_tag = new_version; Py_DECREF(owner); - #line 2848 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2855,7 +2521,6 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 2025 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2865,7 +2530,6 @@ *(PyObject **)addr = value; Py_XDECREF(old_value); Py_DECREF(owner); - #line 2869 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2877,7 +2541,6 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2044 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2890,10 +2553,8 @@ #endif /* ENABLE_SPECIALIZATION */ assert((oparg >> 5) <= Py_GE); res = PyObject_RichCompare(left, right, oparg >> 5); - #line 2894 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2057 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; if (oparg & 16) { int res_bool = PyObject_IsTrue(res); @@ -2901,7 +2562,6 @@ if (res_bool < 0) goto pop_2_error; res = res_bool ? Py_True : Py_False; } - #line 2905 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2912,7 +2572,6 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2067 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2924,7 +2583,6 @@ _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 2928 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2935,7 +2593,6 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2082 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -2951,7 +2608,6 @@ _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 2955 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2962,7 +2618,6 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2101 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2975,7 +2630,6 @@ assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; // It's always a bool, so we don't care about oparg & 16. - #line 2979 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2986,14 +2640,10 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2116 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 2992 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2118 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 2997 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -3003,15 +2653,11 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2122 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 3009 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2124 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; - #line 3015 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -3022,12 +2668,9 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 2129 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 3028 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2131 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -3035,10 +2678,8 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 3039 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2139 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -3047,7 +2688,6 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 3051 "Python/generated_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; DISPATCH(); @@ -3057,21 +2697,15 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2150 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 3064 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2153 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 3071 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2158 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 3075 "Python/generated_cases.c.h" stack_pointer[-1] = b; DISPATCH(); } @@ -3080,15 +2714,11 @@ PyObject *fromlist = stack_pointer[-1]; PyObject *level = stack_pointer[-2]; PyObject *res; - #line 2162 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_name(tstate, frame, name, fromlist, level); - #line 3087 "Python/generated_cases.c.h" Py_DECREF(level); Py_DECREF(fromlist); - #line 2165 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 3092 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -3097,25 +2727,20 @@ TARGET(IMPORT_FROM) { PyObject *from = stack_pointer[-1]; PyObject *res; - #line 2169 "Python/bytecodes.c" PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); res = import_from(tstate, from, name); if (res == NULL) goto error; - #line 3105 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); } TARGET(JUMP_FORWARD) { - #line 2175 "Python/bytecodes.c" JUMPBY(oparg); - #line 3114 "Python/generated_cases.c.h" DISPATCH(); } TARGET(JUMP_BACKWARD) { - #line 2179 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); _Py_CODEUNIT *here = next_instr - 1; assert(oparg <= INSTR_OFFSET()); @@ -3134,12 +2759,10 @@ goto resume_frame; } #endif /* ENABLE_SPECIALIZATION */ - #line 3138 "Python/generated_cases.c.h" DISPATCH(); } TARGET(ENTER_EXECUTOR) { - #line 2210 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); PyCodeObject *code = _PyFrame_GetCode(frame); @@ -3154,25 +2777,20 @@ goto resume_with_error; } goto resume_frame; - #line 3158 "Python/generated_cases.c.h" } TARGET(POP_JUMP_IF_FALSE) { PyObject *cond = stack_pointer[-1]; - #line 2227 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsFalse(cond)); - #line 3166 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_TRUE) { PyObject *cond = stack_pointer[-1]; - #line 2232 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsTrue(cond)); - #line 3176 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -3182,25 +2800,19 @@ { PyObject *value = _tmp_1; PyObject *b; - #line 2237 "Python/bytecodes.c" if (Py_IsNone(value)) { b = Py_True; } else { b = Py_False; - #line 3192 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2243 "Python/bytecodes.c" } - #line 3196 "Python/generated_cases.c.h" _tmp_1 = b; } { PyObject *cond = _tmp_1; - #line 2232 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsTrue(cond)); - #line 3204 "Python/generated_cases.c.h" } STACK_SHRINK(1); DISPATCH(); @@ -3211,52 +2823,42 @@ { PyObject *value = _tmp_1; PyObject *b; - #line 2237 "Python/bytecodes.c" if (Py_IsNone(value)) { b = Py_True; } else { b = Py_False; - #line 3221 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2243 "Python/bytecodes.c" } - #line 3225 "Python/generated_cases.c.h" _tmp_1 = b; } { PyObject *cond = _tmp_1; - #line 2227 "Python/bytecodes.c" assert(PyBool_Check(cond)); JUMPBY(oparg * Py_IsFalse(cond)); - #line 3233 "Python/generated_cases.c.h" } STACK_SHRINK(1); DISPATCH(); } TARGET(JUMP_BACKWARD_NO_INTERRUPT) { - #line 2251 "Python/bytecodes.c" /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost * generator or coroutine, so we deliberately do not check it here. * (see bpo-30039). */ JUMPBY(-oparg); - #line 3247 "Python/generated_cases.c.h" DISPATCH(); } TARGET(GET_LEN) { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2260 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 3260 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; DISPATCH(); @@ -3267,16 +2869,13 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2268 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 3276 "Python/generated_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2273 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -3284,7 +2883,6 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 3288 "Python/generated_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; DISPATCH(); @@ -3293,10 +2891,8 @@ TARGET(MATCH_MAPPING) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2283 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 3300 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3305,10 +2901,8 @@ TARGET(MATCH_SEQUENCE) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2288 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 3312 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3318,11 +2912,9 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2293 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 3326 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; DISPATCH(); @@ -3331,14 +2923,10 @@ TARGET(GET_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2299 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 3338 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2302 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 3342 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3346,7 +2934,6 @@ TARGET(GET_YIELD_FROM_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2306 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -3369,11 +2956,8 @@ if (iter == NULL) { goto error; } - #line 3373 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2329 "Python/bytecodes.c" } - #line 3377 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3383,7 +2967,6 @@ static_assert(INLINE_CACHE_ENTRIES_FOR_ITER == 1, "incorrect cache size"); PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2347 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -3415,7 +2998,6 @@ DISPATCH(); } // Common case: no jump, leave it to the code generator - #line 3419 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3423,7 +3005,6 @@ } TARGET(INSTRUMENTED_FOR_ITER) { - #line 2381 "Python/bytecodes.c" _Py_CODEUNIT *here = next_instr-1; _Py_CODEUNIT *target; PyObject *iter = TOP(); @@ -3449,14 +3030,12 @@ target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER + oparg + 1; } INSTRUMENTED_JUMP(here, target, PY_MONITORING_EVENT_BRANCH); - #line 3453 "Python/generated_cases.c.h" DISPATCH(); } TARGET(FOR_ITER_LIST) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2409 "Python/bytecodes.c" DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit); @@ -3477,7 +3056,6 @@ DISPATCH(); end_for_iter_list: // Common case: no jump, leave it to the code generator - #line 3481 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3487,7 +3065,6 @@ TARGET(FOR_ITER_TUPLE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2432 "Python/bytecodes.c" _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3508,7 +3085,6 @@ DISPATCH(); end_for_iter_tuple: // Common case: no jump, leave it to the code generator - #line 3512 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3518,7 +3094,6 @@ TARGET(FOR_ITER_RANGE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2455 "Python/bytecodes.c" _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3537,7 +3112,6 @@ if (next == NULL) { goto error; } - #line 3541 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3546,7 +3120,6 @@ TARGET(FOR_ITER_GEN) { PyObject *iter = stack_pointer[-1]; - #line 2476 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER); @@ -3562,14 +3135,12 @@ assert(next_instr[oparg].op.code == END_FOR || next_instr[oparg].op.code == INSTRUMENTED_END_FOR); DISPATCH_INLINED(gen_frame); - #line 3566 "Python/generated_cases.c.h" } TARGET(BEFORE_ASYNC_WITH) { PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2494 "Python/bytecodes.c" PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { @@ -3592,16 +3163,13 @@ Py_DECREF(enter); goto error; } - #line 3596 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2517 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3605 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3612,7 +3180,6 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2526 "Python/bytecodes.c" /* pop the context manager, push its __exit__ and the * value returned from calling its __enter__ */ @@ -3638,16 +3205,13 @@ Py_DECREF(enter); goto error; } - #line 3642 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2552 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3651 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3659,7 +3223,6 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2561 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -3680,7 +3243,6 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 3684 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3689,7 +3251,6 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2600 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -3699,7 +3260,6 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 3703 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -3713,7 +3273,6 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2612 "Python/bytecodes.c" assert(oparg & 1); /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); @@ -3730,7 +3289,6 @@ res2 = Py_NewRef(descr); assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; - #line 3734 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3744,7 +3302,6 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2631 "Python/bytecodes.c" assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -3754,7 +3311,6 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - #line 3758 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3769,7 +3325,6 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2643 "Python/bytecodes.c" assert((oparg & 1) == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3782,11 +3337,8 @@ keys_version, LOAD_ATTR); STAT_INC(LOAD_ATTR, hit); assert(descr != NULL); - #line 3786 "Python/generated_cases.c.h" Py_DECREF(self); - #line 2656 "Python/bytecodes.c" res = Py_NewRef(descr); - #line 3790 "Python/generated_cases.c.h" STACK_GROW((0 ? 1 : 0)); stack_pointer[-1] = res; if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } @@ -3800,7 +3352,6 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2660 "Python/bytecodes.c" assert((oparg & 1) == 0); PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3808,11 +3359,8 @@ assert(self_cls->tp_dictoffset == 0); STAT_INC(LOAD_ATTR, hit); assert(descr != NULL); - #line 3812 "Python/generated_cases.c.h" Py_DECREF(self); - #line 2668 "Python/bytecodes.c" res = Py_NewRef(descr); - #line 3816 "Python/generated_cases.c.h" STACK_GROW((0 ? 1 : 0)); stack_pointer[-1] = res; if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } @@ -3826,7 +3374,6 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2672 "Python/bytecodes.c" assert(oparg & 1); PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); @@ -3840,7 +3387,6 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - #line 3844 "Python/generated_cases.c.h" STACK_GROW((1 ? 1 : 0)); stack_pointer[-1] = res; if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } @@ -3849,16 +3395,13 @@ } TARGET(KW_NAMES) { - #line 2688 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(FRAME_CO_CONSTS)); kwnames = GETITEM(FRAME_CO_CONSTS, oparg); - #line 3857 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_CALL) { - #line 2694 "Python/bytecodes.c" int is_meth = PEEK(oparg+2) != NULL; int total_args = oparg + is_meth; PyObject *function = PEEK(total_args + 1); @@ -3871,7 +3414,6 @@ _PyCallCache *cache = (_PyCallCache *)next_instr; INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(CALL); - #line 3875 "Python/generated_cases.c.h" } TARGET(CALL) { @@ -3881,7 +3423,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2740 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3963,7 +3504,6 @@ Py_DECREF(args[i]); } if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3967 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3975,7 +3515,6 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2828 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit); @@ -3985,7 +3524,6 @@ PEEK(oparg + 2) = Py_NewRef(meth); // method Py_DECREF(callable); GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS); - #line 3989 "Python/generated_cases.c.h" } TARGET(CALL_PY_EXACT_ARGS) { @@ -3994,7 +3532,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2840 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4020,7 +3557,6 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4024 "Python/generated_cases.c.h" } TARGET(CALL_PY_WITH_DEFAULTS) { @@ -4028,7 +3564,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2868 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -4064,7 +3599,6 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 4068 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_TYPE_1) { @@ -4072,7 +3606,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2906 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4082,7 +3615,6 @@ res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(obj); Py_DECREF(&PyType_Type); // I.e., callable - #line 4086 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4095,7 +3627,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2918 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4106,7 +3637,6 @@ Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); // I.e., callable if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4110 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4120,7 +3650,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2932 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -4131,7 +3660,6 @@ Py_DECREF(arg); Py_DECREF(&PyTuple_Type); // I.e., tuple if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4135 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4144,7 +3672,6 @@ PyObject **args = (stack_pointer - oparg); PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; - #line 2946 "Python/bytecodes.c" /* This instruction does the following: * 1. Creates the object (by calling ``object.__new__``) * 2. Pushes a shim frame to the frame stack (to cleanup after ``__init__``) @@ -4195,12 +3722,10 @@ * as it will be checked after start_frame */ tstate->py_recursion_remaining--; goto start_frame; - #line 4199 "Python/generated_cases.c.h" } TARGET(EXIT_INIT_CHECK) { PyObject *should_be_none = stack_pointer[-1]; - #line 2999 "Python/bytecodes.c" assert(STACK_LEVEL() == 2); if (should_be_none != Py_None) { PyErr_Format(PyExc_TypeError, @@ -4208,7 +3733,6 @@ Py_TYPE(should_be_none)->tp_name); goto error; } - #line 4212 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -4218,7 +3742,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3009 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4240,7 +3763,6 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4244 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4254,7 +3776,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3034 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4282,7 +3803,6 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4286 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4296,7 +3816,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3065 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4328,7 +3847,6 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 4332 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4342,7 +3860,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3100 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -4374,7 +3891,6 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4378 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4388,7 +3904,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3135 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -4413,7 +3928,6 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4417 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4426,7 +3940,6 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3162 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -4453,7 +3966,6 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4457 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4465,7 +3977,6 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 3192 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -4483,14 +3994,12 @@ SKIP_OVER(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 4487 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3212 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4521,7 +4030,6 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4525 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4534,7 +4042,6 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3246 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4563,7 +4070,6 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4567 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4576,7 +4082,6 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3278 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -4605,7 +4110,6 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4609 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4618,7 +4122,6 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3310 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4646,7 +4149,6 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4650 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4656,9 +4158,7 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { - #line 3341 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); - #line 4662 "Python/generated_cases.c.h" } TARGET(CALL_FUNCTION_EX) { @@ -4667,7 +4167,6 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 3345 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs)); @@ -4729,14 +4228,11 @@ } result = PyObject_Call(func, callargs, kwargs); } - #line 4733 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 3407 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4740 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4747,7 +4243,6 @@ TARGET(MAKE_FUNCTION) { PyObject *codeobj = stack_pointer[-1]; PyObject *func; - #line 3413 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4759,7 +4254,6 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4763 "Python/generated_cases.c.h" stack_pointer[-1] = func; DISPATCH(); } @@ -4767,7 +4261,6 @@ TARGET(SET_FUNCTION_ATTRIBUTE) { PyObject *func = stack_pointer[-1]; PyObject *attr = stack_pointer[-2]; - #line 3427 "Python/bytecodes.c" assert(PyFunction_Check(func)); PyFunctionObject *func_obj = (PyFunctionObject *)func; switch(oparg) { @@ -4792,14 +4285,12 @@ default: Py_UNREACHABLE(); } - #line 4796 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 3454 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4820,7 +4311,6 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4824 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4828,15 +4318,11 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3477 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4834 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3479 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4840 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4846,14 +4332,12 @@ TARGET(CONVERT_VALUE) { PyObject *value = stack_pointer[-1]; PyObject *result; - #line 3483 "Python/bytecodes.c" convertion_func_ptr conv_fn; assert(oparg >= FVC_STR && oparg <= FVC_ASCII); conv_fn = CONVERSION_FUNCTIONS[oparg]; result = conv_fn(value); Py_DECREF(value); if (result == NULL) goto pop_1_error; - #line 4857 "Python/generated_cases.c.h" stack_pointer[-1] = result; DISPATCH(); } @@ -4861,7 +4345,6 @@ TARGET(FORMAT_SIMPLE) { PyObject *value = stack_pointer[-1]; PyObject *res; - #line 3492 "Python/bytecodes.c" /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value)) { @@ -4872,7 +4355,6 @@ else { res = value; } - #line 4876 "Python/generated_cases.c.h" stack_pointer[-1] = res; DISPATCH(); } @@ -4881,12 +4363,10 @@ PyObject *fmt_spec = stack_pointer[-1]; PyObject *value = stack_pointer[-2]; PyObject *res; - #line 3505 "Python/bytecodes.c" res = PyObject_Format(value, fmt_spec); Py_DECREF(value); Py_DECREF(fmt_spec); if (res == NULL) goto pop_2_error; - #line 4890 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -4895,10 +4375,8 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3512 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4902 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4910,7 +4388,6 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3517 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -4925,12 +4402,9 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4929 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3532 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 4934 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -4940,16 +4414,13 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3537 "Python/bytecodes.c" assert(oparg >= 2); - #line 4946 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } TARGET(INSTRUMENTED_INSTRUCTION) { - #line 3541 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error; @@ -4961,48 +4432,38 @@ assert(next_opcode > 0 && next_opcode < 256); opcode = next_opcode; DISPATCH_GOTO(); - #line 4965 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_JUMP_FORWARD) { - #line 3555 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); - #line 4971 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) { - #line 3559 "Python/bytecodes.c" CHECK_EVAL_BREAKER(); INSTRUMENTED_JUMP(next_instr-1, next_instr+1-oparg, PY_MONITORING_EVENT_JUMP); - #line 4979 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { - #line 3564 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsTrue(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4990 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { - #line 3572 "Python/bytecodes.c" PyObject *cond = POP(); assert(PyBool_Check(cond)); _Py_CODEUNIT *here = next_instr - 1; int offset = Py_IsFalse(cond) * oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5001 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { - #line 3580 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -5014,12 +4475,10 @@ offset = 0; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5018 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { - #line 3594 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -5031,30 +4490,23 @@ offset = oparg; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 5035 "Python/generated_cases.c.h" DISPATCH(); } TARGET(EXTENDED_ARG) { - #line 3608 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 5046 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3616 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); - #line 5053 "Python/generated_cases.c.h" } TARGET(RESERVED) { - #line 3621 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 5060 "Python/generated_cases.c.h" } From webhook-mailer at python.org Wed Jul 12 13:24:03 2023 From: webhook-mailer at python.org (gvanrossum) Date: Wed, 12 Jul 2023 17:24:03 -0000 Subject: [Python-checkins] gh-106529: Split FOR_ITER_RANGE into uops (#106638) Message-ID: https://github.com/python/cpython/commit/dd1884dc5dc1a540c60e98ea1bc482a51d996564 commit: dd1884dc5dc1a540c60e98ea1bc482a51d996564 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-12T10:23:59-07:00 summary: gh-106529: Split FOR_ITER_RANGE into uops (#106638) For an example of what this does for Tier 1 and Tier 2, see https://github.com/python/cpython/issues/106529#issuecomment-1631649920 files: M Include/internal/pycore_opcode_metadata.h M Lib/test/test_capi/test_misc.py M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/optimizer.c diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index 317f42afea804..d2c1f9ad6e5fb 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -40,6 +40,9 @@ #define _GUARD_GLOBALS_VERSION 318 #define _GUARD_BUILTINS_VERSION 319 #define IS_NONE 320 +#define _ITER_CHECK_RANGE 321 +#define _ITER_EXHAUSTED_RANGE 322 +#define _ITER_NEXT_RANGE 323 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -1319,5 +1322,8 @@ const char * const _PyOpcode_uop_name[512] = { [318] = "_GUARD_GLOBALS_VERSION", [319] = "_GUARD_BUILTINS_VERSION", [320] = "IS_NONE", + [321] = "_ITER_CHECK_RANGE", + [322] = "_ITER_EXHAUSTED_RANGE", + [323] = "_ITER_NEXT_RANGE", }; #endif // NEED_OPCODE_METADATA diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 9c14a501875b6..abdf7ed897635 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2443,7 +2443,6 @@ def testfunc(x): i += 1 opt = _testinternalcapi.get_uop_optimizer() - with temporary_optimizer(opt): testfunc(1000) @@ -2580,13 +2579,33 @@ def testfunc(n): ex = get_first_executor(testfunc) self.assertIsNotNone(ex) - # for i, (opname, oparg) in enumerate(ex): - # print(f"{i:4d}: {opname:<20s} {oparg:4d}") uops = {opname for opname, _ in ex} # Since there is no JUMP_FORWARD instruction, # look for indirect evidence: the += operator self.assertIn("_BINARY_OP_ADD_INT", uops) + def test_for_iter_range(self): + def testfunc(n): + total = 0 + for i in range(n): + total += i + return total + # import dis; dis.dis(testfunc) + + opt = _testinternalcapi.get_uop_optimizer() + with temporary_optimizer(opt): + total = testfunc(10) + self.assertEqual(total, 45) + + ex = get_first_executor(testfunc) + self.assertIsNotNone(ex) + # for i, (opname, oparg) in enumerate(ex): + # print(f"{i:4d}: {opname:<20s} {oparg:3d}") + uops = {opname for opname, _ in ex} + self.assertIn("_ITER_EXHAUSTED_RANGE", uops) + # Verification that the jump goes past END_FOR + # is done by manual inspection of the output + if __name__ == "__main__": unittest.main() diff --git a/Python/bytecodes.c b/Python/bytecodes.c index f5ce2e72d2676..18862f87b65fa 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2451,9 +2451,14 @@ dummy_func( // Common case: no jump, leave it to the code generator } - inst(FOR_ITER_RANGE, (unused/1, iter -- iter, next)) { + op(_ITER_CHECK_RANGE, (iter -- iter)) { _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); + } + + op(_ITER_JUMP_RANGE, (iter -- iter)) { + _PyRangeIterObject *r = (_PyRangeIterObject *)iter; + assert(Py_TYPE(r) == &PyRangeIter_Type); STAT_INC(FOR_ITER, hit); if (r->len <= 0) { STACK_SHRINK(1); @@ -2463,15 +2468,29 @@ dummy_func( JUMPBY(oparg + 1); DISPATCH(); } + } + + // Only used by Tier 2 + op(_ITER_EXHAUSTED_RANGE, (iter -- iter, exhausted)) { + _PyRangeIterObject *r = (_PyRangeIterObject *)iter; + assert(Py_TYPE(r) == &PyRangeIter_Type); + exhausted = r->len <= 0 ? Py_True : Py_False; + } + + op(_ITER_NEXT_RANGE, (iter -- iter, next)) { + _PyRangeIterObject *r = (_PyRangeIterObject *)iter; + assert(Py_TYPE(r) == &PyRangeIter_Type); + assert(r->len > 0); long value = r->start; r->start = value + r->step; r->len--; next = PyLong_FromLong(value); - if (next == NULL) { - goto error; - } + ERROR_IF(next == NULL, error); } + macro(FOR_ITER_RANGE) = + unused/1 + _ITER_CHECK_RANGE + _ITER_JUMP_RANGE + _ITER_NEXT_RANGE; + inst(FOR_ITER_GEN, (unused/1, iter -- iter, unused)) { DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); PyGenObject *gen = (PyGenObject *)iter; diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 1df8feba352cf..2c2dbf429cec1 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1720,6 +1720,40 @@ break; } + case _ITER_CHECK_RANGE: { + PyObject *iter = stack_pointer[-1]; + _PyRangeIterObject *r = (_PyRangeIterObject *)iter; + DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); + break; + } + + case _ITER_EXHAUSTED_RANGE: { + PyObject *iter = stack_pointer[-1]; + PyObject *exhausted; + _PyRangeIterObject *r = (_PyRangeIterObject *)iter; + assert(Py_TYPE(r) == &PyRangeIter_Type); + exhausted = r->len <= 0 ? Py_True : Py_False; + STACK_GROW(1); + stack_pointer[-1] = exhausted; + break; + } + + case _ITER_NEXT_RANGE: { + PyObject *iter = stack_pointer[-1]; + PyObject *next; + _PyRangeIterObject *r = (_PyRangeIterObject *)iter; + assert(Py_TYPE(r) == &PyRangeIter_Type); + assert(r->len > 0); + long value = r->start; + r->start = value + r->step; + r->len--; + next = PyLong_FromLong(value); + if (next == NULL) goto error; + STACK_GROW(1); + stack_pointer[-1] = next; + break; + } + case WITH_EXCEPT_START: { PyObject *val = stack_pointer[-1]; PyObject *lasti = stack_pointer[-3]; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index f7a18b43ff11f..383432f51a89a 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3092,29 +3092,47 @@ } TARGET(FOR_ITER_RANGE) { - PyObject *iter = stack_pointer[-1]; - PyObject *next; - _PyRangeIterObject *r = (_PyRangeIterObject *)iter; - DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); - STAT_INC(FOR_ITER, hit); - if (r->len <= 0) { - STACK_SHRINK(1); - Py_DECREF(r); - SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); - // Jump over END_FOR instruction. - JUMPBY(oparg + 1); - DISPATCH(); + PyObject *_tmp_1; + PyObject *_tmp_2 = stack_pointer[-1]; + { + PyObject *iter = _tmp_2; + _PyRangeIterObject *r = (_PyRangeIterObject *)iter; + DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); + _tmp_2 = iter; } - long value = r->start; - r->start = value + r->step; - r->len--; - next = PyLong_FromLong(value); - if (next == NULL) { - goto error; + { + PyObject *iter = _tmp_2; + _PyRangeIterObject *r = (_PyRangeIterObject *)iter; + assert(Py_TYPE(r) == &PyRangeIter_Type); + STAT_INC(FOR_ITER, hit); + if (r->len <= 0) { + STACK_SHRINK(1); + Py_DECREF(r); + SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); + // Jump over END_FOR instruction. + JUMPBY(oparg + 1); + DISPATCH(); + } + _tmp_2 = iter; + } + { + PyObject *iter = _tmp_2; + PyObject *next; + _PyRangeIterObject *r = (_PyRangeIterObject *)iter; + assert(Py_TYPE(r) == &PyRangeIter_Type); + assert(r->len > 0); + long value = r->start; + r->start = value + r->step; + r->len--; + next = PyLong_FromLong(value); + if (next == NULL) goto error; + _tmp_2 = iter; + _tmp_1 = next; } - STACK_GROW(1); - stack_pointer[-1] = next; next_instr += 1; + STACK_GROW(1); + stack_pointer[-1] = _tmp_1; + stack_pointer[-2] = _tmp_2; DISPATCH(); } diff --git a/Python/optimizer.c b/Python/optimizer.c index c3fdee63a7ed4..abd2351f6b78b 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -479,6 +479,28 @@ translate_bytecode_to_trace( break; } + case FOR_ITER_RANGE: + { + // Assume jump unlikely (can a for-loop exit be likely?) + // Reserve 9 entries (4 here, 3 stub, plus SAVE_IP + EXIT_TRACE) + if (trace_length + 9 > max_length) { + DPRINTF(1, "Ran out of space for FOR_ITER_RANGE\n"); + goto done; + } + _Py_CODEUNIT *target_instr = // +1 at the end skips over END_FOR + instr + 1 + _PyOpcode_Caches[_PyOpcode_Deopt[opcode]] + oparg + 1; + max_length -= 3; // Really the start of the stubs + ADD_TO_TRACE(_ITER_CHECK_RANGE, 0); + ADD_TO_TRACE(_ITER_EXHAUSTED_RANGE, 0); + ADD_TO_TRACE(_POP_JUMP_IF_TRUE, max_length); + ADD_TO_TRACE(_ITER_NEXT_RANGE, 0); + + ADD_TO_STUB(max_length + 0, POP_TOP, 0); + ADD_TO_STUB(max_length + 1, SAVE_IP, INSTR_IP(target_instr, code)); + ADD_TO_STUB(max_length + 2, EXIT_TRACE, 0); + break; + } + default: { const struct opcode_macro_expansion *expansion = &_PyOpcode_macro_expansion[opcode]; @@ -574,8 +596,8 @@ translate_bytecode_to_trace( } } } - trace_length += buffer_size - max_length; } + trace_length += buffer_size - max_length; return trace_length; } else { From webhook-mailer at python.org Wed Jul 12 14:08:03 2023 From: webhook-mailer at python.org (gpshead) Date: Wed, 12 Jul 2023 18:08:03 -0000 Subject: [Python-checkins] gh-106236: Replace `assert` with `raise RuntimeError` in `threading.py` (#106237) Message-ID: https://github.com/python/cpython/commit/e4b88c1e4ac129b36f99a534387d64f7b8cda8ef commit: e4b88c1e4ac129b36f99a534387d64f7b8cda8ef branch: main author: Nikita Sobolev committer: gpshead date: 2023-07-12T11:07:59-07:00 summary: gh-106236: Replace `assert` with `raise RuntimeError` in `threading.py` (#106237) Replace `assert` with `raise ` in `threading.py` so that -OO does not alter _DummyThread behavior. files: A Misc/NEWS.d/next/Library/2023-06-29-15-10-44.gh-issue-106236.EAIX4l.rst M Lib/test/test_threading.py M Lib/threading.py diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index 9e4972ecb640d..4a91eef31c4b8 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -251,6 +251,14 @@ def f(mutex): #Issue 29376 self.assertTrue(threading._active[tid].is_alive()) self.assertRegex(repr(threading._active[tid]), '_DummyThread') + + # Issue gh-106236: + with self.assertRaises(RuntimeError): + threading._active[tid].join() + threading._active[tid]._started.clear() + with self.assertRaises(RuntimeError): + threading._active[tid].is_alive() + del threading._active[tid] # PyThreadState_SetAsyncExc() is a CPython-only gimmick, not (currently) diff --git a/Lib/threading.py b/Lib/threading.py index df273870fa427..e036cb891e196 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -1451,11 +1451,12 @@ def _stop(self): pass def is_alive(self): - assert not self._is_stopped and self._started.is_set() - return True + if not self._is_stopped and self._started.is_set(): + return True + raise RuntimeError("thread is not alive") def join(self, timeout=None): - assert False, "cannot join a dummy thread" + raise RuntimeError("cannot join a dummy thread") # Global API functions diff --git a/Misc/NEWS.d/next/Library/2023-06-29-15-10-44.gh-issue-106236.EAIX4l.rst b/Misc/NEWS.d/next/Library/2023-06-29-15-10-44.gh-issue-106236.EAIX4l.rst new file mode 100644 index 0000000000000..036bdb6ef59f6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-29-15-10-44.gh-issue-106236.EAIX4l.rst @@ -0,0 +1,2 @@ +Replace ``assert`` statements with ``raise RuntimeError`` in +:mod:`threading`, so that ``_DummyThread`` cannot be joined even with ``-OO``. From webhook-mailer at python.org Wed Jul 12 17:01:20 2023 From: webhook-mailer at python.org (ethanfurman) Date: Wed, 12 Jul 2023 21:01:20 -0000 Subject: [Python-checkins] gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106666) Message-ID: https://github.com/python/cpython/commit/357e9e9da3929cb9d55ea31896e66f488e44e8f2 commit: 357e9e9da3929cb9d55ea31896e66f488e44e8f2 branch: main author: Prince Roshan committer: ethanfurman date: 2023-07-12T14:01:17-07:00 summary: gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106666) files: A Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 202f0da028bdf..0c985b2c77856 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1218,6 +1218,12 @@ def __hash__(self): def __reduce_ex__(self, proto): return self.__class__, (self._value_, ) + def __deepcopy__(self,memo): + return self + + def __copy__(self): + return self + # enum.property is used to provide access to the `name` and # `value` attributes of enum members while keeping some measure of # protection from modification, while still allowing for an enumeration diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index adb1e0e52c548..a286411f7bcf5 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -804,9 +804,17 @@ def test_copy(self): TE = self.MainEnum copied = copy.copy(TE) self.assertEqual(copied, TE) + self.assertIs(copied, TE) deep = copy.deepcopy(TE) self.assertEqual(deep, TE) + self.assertIs(deep, TE) + def test_copy_member(self): + TE = self.MainEnum + copied = copy.copy(TE.first) + self.assertIs(copied, TE.first) + deep = copy.deepcopy(TE.first) + self.assertIs(deep, TE.first) class _FlagTests: diff --git a/Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst b/Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst new file mode 100644 index 0000000000000..d9c122f1d3c72 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst @@ -0,0 +1 @@ +Add __copy__ and __deepcopy__ in :mod:`enum` From webhook-mailer at python.org Wed Jul 12 17:33:51 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 12 Jul 2023 21:33:51 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: Annotate the `Destination` class (#106655) Message-ID: https://github.com/python/cpython/commit/a180e7a0df342e9f089998fc680be83ad2e49a79 commit: a180e7a0df342e9f089998fc680be83ad2e49a79 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-12T22:33:47+01:00 summary: gh-104050: Argument clinic: Annotate the `Destination` class (#106655) Co-authored-by: Nikita Sobolev files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index a0cf50b29c978..ce3039cdd8bff 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1954,27 +1954,32 @@ def dump(self): return "".join(texts) + at dc.dataclass(slots=True, repr=False) class Destination: - def __init__(self, name, type, clinic, *args): - self.name = name - self.type = type - self.clinic = clinic - self.buffers = BufferSeries() + name: str + type: str + clinic: Clinic + buffers: BufferSeries = dc.field(init=False, default_factory=BufferSeries) + filename: str = dc.field(init=False) # set in __post_init__ + args: dc.InitVar[tuple[str, ...]] = () + + def __post_init__(self, args: tuple[str, ...]) -> None: valid_types = ('buffer', 'file', 'suppress') - if type not in valid_types: + if self.type not in valid_types: fail( - f"Invalid destination type {type!r} for {name}, " + f"Invalid destination type {self.type!r} for {self.name}, " f"must be {', '.join(valid_types)}" ) - extra_arguments = 1 if type == "file" else 0 + extra_arguments = 1 if self.type == "file" else 0 if len(args) < extra_arguments: - fail(f"Not enough arguments for destination {name} new {type}") + fail(f"Not enough arguments for destination {self.name} new {self.type}") if len(args) > extra_arguments: - fail(f"Too many arguments for destination {name} new {type}") - if type =='file': + fail(f"Too many arguments for destination {self.name} new {self.type}") + if self.type =='file': d = {} - filename = clinic.filename + filename = self.clinic.filename + assert filename is not None d['path'] = filename dirname, basename = os.path.split(filename) if not dirname: @@ -1984,19 +1989,19 @@ def __init__(self, name, type, clinic, *args): d['basename_root'], d['basename_extension'] = os.path.splitext(filename) self.filename = args[0].format_map(d) - def __repr__(self): + def __repr__(self) -> str: if self.type == 'file': file_repr = " " + repr(self.filename) else: file_repr = '' return "".join(("")) - def clear(self): + def clear(self) -> None: if self.type != 'buffer': fail("Can't clear destination" + self.name + " , it's not of type buffer") self.buffers.clear() - def dump(self): + def dump(self) -> str: return self.buffers.dump() @@ -2164,11 +2169,11 @@ def add_destination( self, name: str, type: str, - *args + *args: str ) -> None: if name in self.destinations: fail("Destination already exists: " + repr(name)) - self.destinations[name] = Destination(name, type, self, *args) + self.destinations[name] = Destination(name, type, self, args) def get_destination(self, name: str) -> Destination: d = self.destinations.get(name) From webhook-mailer at python.org Wed Jul 12 17:58:54 2023 From: webhook-mailer at python.org (gpshead) Date: Wed, 12 Jul 2023 21:58:54 -0000 Subject: [Python-checkins] [3.12] gh-96747: Mention the PyPI `passlib` package in the `crypt` deprecation doc (GH-106660) (#106660) Message-ID: https://github.com/python/cpython/commit/ec7b05a0bebb2cc2ec42bf00cde2b66145880459 commit: ec7b05a0bebb2cc2ec42bf00cde2b66145880459 branch: 3.12 author: Yonatan Bitton committer: gpshead date: 2023-07-12T14:58:51-07:00 summary: [3.12] gh-96747: Mention the PyPI `passlib` package in the `crypt` deprecation doc (GH-106660) (#106660) * Added mention to passlib package as alternative to the deprecated crypt module. files: M Doc/library/crypt.rst diff --git a/Doc/library/crypt.rst b/Doc/library/crypt.rst index 740084b40c5ac..51f91463f5ff9 100644 --- a/Doc/library/crypt.rst +++ b/Doc/library/crypt.rst @@ -20,6 +20,7 @@ The :mod:`crypt` module is deprecated (see :pep:`PEP 594 <594#crypt>` for details and alternatives). The :mod:`hashlib` module is a potential replacement for certain use cases. + The `passlib `_ package can replace all use cases of this module. -------------- From webhook-mailer at python.org Wed Jul 12 17:59:57 2023 From: webhook-mailer at python.org (gpshead) Date: Wed, 12 Jul 2023 21:59:57 -0000 Subject: [Python-checkins] [3.11] [3.12] gh-96747: Mention the PyPI `passlib` package in the `crypt` deprecation doc (GH-106660) (GH-106660) (#106697) Message-ID: https://github.com/python/cpython/commit/eac0616df902d4a591c2585479fbedfbbda7cdd6 commit: eac0616df902d4a591c2585479fbedfbbda7cdd6 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gpshead date: 2023-07-12T14:59:54-07:00 summary: [3.11] [3.12] gh-96747: Mention the PyPI `passlib` package in the `crypt` deprecation doc (GH-106660) (GH-106660) (#106697) [3.12] gh-96747: Mention the PyPI `passlib` package in the `crypt` deprecation doc (GH-106660) (GH-106660) * Added mention to passlib package as alternative to the deprecated crypt module. (cherry picked from commit ec7b05a0bebb2cc2ec42bf00cde2b66145880459) Co-authored-by: Yonatan Bitton files: M Doc/library/crypt.rst diff --git a/Doc/library/crypt.rst b/Doc/library/crypt.rst index 740084b40c5ac..51f91463f5ff9 100644 --- a/Doc/library/crypt.rst +++ b/Doc/library/crypt.rst @@ -20,6 +20,7 @@ The :mod:`crypt` module is deprecated (see :pep:`PEP 594 <594#crypt>` for details and alternatives). The :mod:`hashlib` module is a potential replacement for certain use cases. + The `passlib `_ package can replace all use cases of this module. -------------- From webhook-mailer at python.org Wed Jul 12 18:47:57 2023 From: webhook-mailer at python.org (ethanfurman) Date: Wed, 12 Jul 2023 22:47:57 -0000 Subject: [Python-checkins] [3.12] gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106695) Message-ID: https://github.com/python/cpython/commit/77d9fdf23be4b705b65b08ed00bb5df7988ecf3c commit: 77d9fdf23be4b705b65b08ed00bb5df7988ecf3c branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ethanfurman date: 2023-07-12T15:47:53-07:00 summary: [3.12] gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106695) gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106666) (cherry picked from commit 357e9e9da3929cb9d55ea31896e66f488e44e8f2) Co-authored-by: Prince Roshan files: A Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 56ebfbd1a61f7..9114c5ccaaa95 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1242,6 +1242,12 @@ def __hash__(self): def __reduce_ex__(self, proto): return self.__class__, (self._value_, ) + def __deepcopy__(self,memo): + return self + + def __copy__(self): + return self + # enum.property is used to provide access to the `name` and # `value` attributes of enum members while keeping some measure of # protection from modification, while still allowing for an enumeration diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 592deb837914c..dbdc639d62ec1 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -804,9 +804,17 @@ def test_copy(self): TE = self.MainEnum copied = copy.copy(TE) self.assertEqual(copied, TE) + self.assertIs(copied, TE) deep = copy.deepcopy(TE) self.assertEqual(deep, TE) + self.assertIs(deep, TE) + def test_copy_member(self): + TE = self.MainEnum + copied = copy.copy(TE.first) + self.assertIs(copied, TE.first) + deep = copy.deepcopy(TE.first) + self.assertIs(deep, TE.first) class _FlagTests: diff --git a/Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst b/Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst new file mode 100644 index 0000000000000..d9c122f1d3c72 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst @@ -0,0 +1 @@ +Add __copy__ and __deepcopy__ in :mod:`enum` From webhook-mailer at python.org Wed Jul 12 18:48:20 2023 From: webhook-mailer at python.org (ethanfurman) Date: Wed, 12 Jul 2023 22:48:20 -0000 Subject: [Python-checkins] [3.11] gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106694) Message-ID: https://github.com/python/cpython/commit/a276ce4505abc72dc3cf3a3a906ad65ef1306203 commit: a276ce4505abc72dc3cf3a3a906ad65ef1306203 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ethanfurman date: 2023-07-12T15:48:16-07:00 summary: [3.11] gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106694) gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106666) (cherry picked from commit 357e9e9da3929cb9d55ea31896e66f488e44e8f2) Co-authored-by: Prince Roshan files: A Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 1184b5b13e6d7..1f447c878c1f8 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1227,6 +1227,12 @@ def __hash__(self): def __reduce_ex__(self, proto): return self.__class__, (self._value_, ) + def __deepcopy__(self,memo): + return self + + def __copy__(self): + return self + # enum.property is used to provide access to the `name` and # `value` attributes of enum members while keeping some measure of # protection from modification, while still allowing for an enumeration diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 442802823e7d0..cc66875bde02a 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -775,9 +775,17 @@ def test_copy(self): TE = self.MainEnum copied = copy.copy(TE) self.assertEqual(copied, TE) + self.assertIs(copied, TE) deep = copy.deepcopy(TE) self.assertEqual(deep, TE) + self.assertIs(deep, TE) + def test_copy_member(self): + TE = self.MainEnum + copied = copy.copy(TE.first) + self.assertIs(copied, TE.first) + deep = copy.deepcopy(TE.first) + self.assertIs(deep, TE.first) class _FlagTests: diff --git a/Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst b/Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst new file mode 100644 index 0000000000000..d9c122f1d3c72 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-12-04-58-45.gh-issue-106602.dGCcXe.rst @@ -0,0 +1 @@ +Add __copy__ and __deepcopy__ in :mod:`enum` From webhook-mailer at python.org Wed Jul 12 18:48:39 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 12 Jul 2023 22:48:39 -0000 Subject: [Python-checkins] gh-104683: Argument clinic: modernise `cpp.Monitor` (#106698) Message-ID: https://github.com/python/cpython/commit/8aa4beaad0d95917b1bb12d146bc15c1aa815e08 commit: 8aa4beaad0d95917b1bb12d146bc15c1aa815e08 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-12T22:48:36Z summary: gh-104683: Argument clinic: modernise `cpp.Monitor` (#106698) files: M Tools/clinic/cpp.py diff --git a/Tools/clinic/cpp.py b/Tools/clinic/cpp.py index c1a2eeef22dec..fbac81336b545 100644 --- a/Tools/clinic/cpp.py +++ b/Tools/clinic/cpp.py @@ -1,3 +1,4 @@ +import dataclasses as dc import re import sys from collections.abc import Callable @@ -15,6 +16,11 @@ def negate(condition: str) -> str: return condition[1:] return "!" + condition + +is_a_simple_defined = re.compile(r'^defined\s*\(\s*[A-Za-z0-9_]+\s*\)$').match + + + at dc.dataclass(repr=False) class Monitor: """ A simple C preprocessor that scans C source and computes, line by line, @@ -27,25 +33,20 @@ class Monitor: Anyway this implementation seems to work well enough for the CPython sources. """ + filename: str | None = None + _: dc.KW_ONLY + verbose: bool = False - is_a_simple_defined: Callable[[str], re.Match[str] | None] - is_a_simple_defined = re.compile(r'^defined\s*\(\s*[A-Za-z0-9_]+\s*\)$').match - - def __init__(self, filename: str | None = None, *, verbose: bool = False) -> None: + def __post_init__(self) -> None: self.stack: TokenStack = [] self.in_comment = False self.continuation: str | None = None self.line_number = 0 - self.filename = filename - self.verbose = verbose def __repr__(self) -> str: - return ''.join(( - '")) + return ( + f"" + ) def status(self) -> str: return str(self.line_number).rjust(4) + ": " + self.condition() @@ -152,7 +153,7 @@ def pop_stack() -> TokenAndCondition: if not condition: self.fail("Invalid format for #" + token + " line: no argument!") if token in {'if', 'elif'}: - if not self.is_a_simple_defined(condition): + if not is_a_simple_defined(condition): condition = "(" + condition + ")" if token == 'elif': previous_token, previous_condition = pop_stack() From webhook-mailer at python.org Wed Jul 12 18:49:33 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 12 Jul 2023 22:49:33 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Annotate nested function parser_body() in the CLanguage class (#106699) Message-ID: https://github.com/python/cpython/commit/2d43beec225a0495ffa0344f961b99717e5f1106 commit: 2d43beec225a0495ffa0344f961b99717e5f1106 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-12T22:49:30Z summary: gh-104050: Argument Clinic: Annotate nested function parser_body() in the CLanguage class (#106699) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index ce3039cdd8bff..8a75aba1e4de9 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -898,22 +898,24 @@ def output_templates(self, f): # parser_body_fields remembers the fields passed in to the # previous call to parser_body. this is used for an awful hack. parser_body_fields = () - def parser_body(prototype, *fields, declarations=''): + def parser_body( + prototype: str, + *fields: str, + declarations: str = '' + ) -> str: nonlocal parser_body_fields add, output = text_accumulator() add(prototype) parser_body_fields = fields - fields = list(fields) - fields.insert(0, normalize_snippet(""" + preamble = normalize_snippet(""" {{ {return_value_declaration} {parser_declarations} {declarations} {initializers} - """) + "\n") - # just imagine--your code is here in the middle - fields.append(normalize_snippet(""" + """) + "\n" + finale = normalize_snippet(""" {modifications} {return_value} = {c_basename}_impl({impl_arguments}); {return_conversion} @@ -923,8 +925,8 @@ def parser_body(prototype, *fields, declarations=''): {cleanup} return return_value; }} - """)) - for field in fields: + """) + for field in preamble, *fields, finale: add('\n') add(field) return linear_format(output(), parser_declarations=declarations) From webhook-mailer at python.org Wed Jul 12 22:50:49 2023 From: webhook-mailer at python.org (sweeneyde) Date: Thu, 13 Jul 2023 02:50:49 -0000 Subject: [Python-checkins] gh-105235: Prevent reading outside buffer during mmap.find() (#105252) Message-ID: https://github.com/python/cpython/commit/ab86426a3472ab68747815299d390b213793c3d1 commit: ab86426a3472ab68747815299d390b213793c3d1 branch: main author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2023-07-12T22:50:45-04:00 summary: gh-105235: Prevent reading outside buffer during mmap.find() (#105252) * Add a special case for s[-m:] == p in _PyBytes_Find * Add tests for _PyBytes_Find * Make sure that start <= end in mmap.find files: A Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst M Lib/test/test_mmap.py M Modules/_testinternalcapi.c M Modules/mmapmodule.c M Objects/bytesobject.c diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py index 517cbe0cb115a..bab868600895c 100644 --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -299,6 +299,27 @@ def test_find_end(self): self.assertEqual(m.find(b'one', 1, -2), -1) self.assertEqual(m.find(bytearray(b'one')), 0) + for i in range(-n-1, n+1): + for j in range(-n-1, n+1): + for p in [b"o", b"on", b"two", b"ones", b"s"]: + expected = data.find(p, i, j) + self.assertEqual(m.find(p, i, j), expected, (p, i, j)) + + def test_find_does_not_access_beyond_buffer(self): + try: + flags = mmap.MAP_PRIVATE | mmap.MAP_ANONYMOUS + PAGESIZE = mmap.PAGESIZE + PROT_NONE = 0 + PROT_READ = mmap.PROT_READ + except AttributeError as e: + raise unittest.SkipTest("mmap flags unavailable") from e + for i in range(0, 2049): + with mmap.mmap(-1, PAGESIZE * (i + 1), + flags=flags, prot=PROT_NONE) as guard: + with mmap.mmap(-1, PAGESIZE * (i + 2048), + flags=flags, prot=PROT_READ) as fm: + fm.find(b"fo", -2) + def test_rfind(self): # test the new 'end' parameter works as expected diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst new file mode 100644 index 0000000000000..c28d0101cd4ba --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst @@ -0,0 +1 @@ +Prevent out-of-bounds memory access during ``mmap.find()`` calls. diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 52e524a40672e..7745dd5abc22f 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -14,6 +14,7 @@ #include "interpreteridobject.h" // _PyInterpreterID_LookUp() #include "pycore_atomic_funcs.h" // _Py_atomic_int_get() #include "pycore_bitutils.h" // _Py_bswap32() +#include "pycore_bytesobject.h" // _PyBytes_Find() #include "pycore_compile.h" // _PyCompile_CodeGen, _PyCompile_OptimizeCfg, _PyCompile_Assemble #include "pycore_ceval.h" // _PyEval_AddPendingCall #include "pycore_fileutils.h" // _Py_normpath @@ -443,6 +444,118 @@ test_edit_cost(PyObject *self, PyObject *Py_UNUSED(args)) } +static int +check_bytes_find(const char *haystack0, const char *needle0, + int offset, Py_ssize_t expected) +{ + Py_ssize_t len_haystack = strlen(haystack0); + Py_ssize_t len_needle = strlen(needle0); + Py_ssize_t result_1 = _PyBytes_Find(haystack0, len_haystack, + needle0, len_needle, offset); + if (result_1 != expected) { + PyErr_Format(PyExc_AssertionError, + "Incorrect result_1: '%s' in '%s' (offset=%zd)", + needle0, haystack0, offset); + return -1; + } + // Allocate new buffer with no NULL terminator. + char *haystack = PyMem_Malloc(len_haystack); + if (haystack == NULL) { + PyErr_NoMemory(); + return -1; + } + char *needle = PyMem_Malloc(len_needle); + if (needle == NULL) { + PyMem_Free(haystack); + PyErr_NoMemory(); + return -1; + } + memcpy(haystack, haystack0, len_haystack); + memcpy(needle, needle0, len_needle); + Py_ssize_t result_2 = _PyBytes_Find(haystack, len_haystack, + needle, len_needle, offset); + PyMem_Free(haystack); + PyMem_Free(needle); + if (result_2 != expected) { + PyErr_Format(PyExc_AssertionError, + "Incorrect result_2: '%s' in '%s' (offset=%zd)", + needle0, haystack0, offset); + return -1; + } + return 0; +} + +static int +check_bytes_find_large(Py_ssize_t len_haystack, Py_ssize_t len_needle, + const char *needle) +{ + char *zeros = PyMem_RawCalloc(len_haystack, 1); + if (zeros == NULL) { + PyErr_NoMemory(); + return -1; + } + Py_ssize_t res = _PyBytes_Find(zeros, len_haystack, needle, len_needle, 0); + PyMem_RawFree(zeros); + if (res != -1) { + PyErr_Format(PyExc_AssertionError, + "check_bytes_find_large(%zd, %zd) found %zd", + len_haystack, len_needle, res); + return -1; + } + return 0; +} + +static PyObject * +test_bytes_find(PyObject *self, PyObject *Py_UNUSED(args)) +{ + #define CHECK(H, N, O, E) do { \ + if (check_bytes_find(H, N, O, E) < 0) { \ + return NULL; \ + } \ + } while (0) + + CHECK("", "", 0, 0); + CHECK("Python", "", 0, 0); + CHECK("Python", "", 3, 3); + CHECK("Python", "", 6, 6); + CHECK("Python", "yth", 0, 1); + CHECK("ython", "yth", 1, 1); + CHECK("thon", "yth", 2, -1); + CHECK("Python", "thon", 0, 2); + CHECK("ython", "thon", 1, 2); + CHECK("thon", "thon", 2, 2); + CHECK("hon", "thon", 3, -1); + CHECK("Pytho", "zz", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "ab", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "ba", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "bb", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab", "ab", 0, 30); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaba", "ba", 0, 30); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaabb", "bb", 0, 30); + #undef CHECK + + // Hunt for segfaults + // n, m chosen here so that (n - m) % (m + 1) == 0 + // This would make default_find in fastsearch.h access haystack[n]. + if (check_bytes_find_large(2048, 2, "ab") < 0) { + return NULL; + } + if (check_bytes_find_large(4096, 16, "0123456789abcdef") < 0) { + return NULL; + } + if (check_bytes_find_large(8192, 2, "ab") < 0) { + return NULL; + } + if (check_bytes_find_large(16384, 4, "abcd") < 0) { + return NULL; + } + if (check_bytes_find_large(32768, 2, "ab") < 0) { + return NULL; + } + Py_RETURN_NONE; +} + + static PyObject * normalize_path(PyObject *self, PyObject *filename) { @@ -1328,6 +1441,7 @@ static PyMethodDef module_functions[] = { {"reset_path_config", test_reset_path_config, METH_NOARGS}, {"test_atomic_funcs", test_atomic_funcs, METH_NOARGS}, {"test_edit_cost", test_edit_cost, METH_NOARGS}, + {"test_bytes_find", test_bytes_find, METH_NOARGS}, {"normalize_path", normalize_path, METH_O, NULL}, {"get_getpath_codeobject", get_getpath_codeobject, METH_NOARGS, NULL}, {"EncodeLocaleEx", encode_locale_ex, METH_VARARGS}, diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index fef27123a3282..c1cd5b0efaa3d 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -342,12 +342,17 @@ mmap_gfind(mmap_object *self, Py_ssize_t res; CHECK_VALID_OR_RELEASE(NULL, view); - if (reverse) { + if (end < start) { + res = -1; + } + else if (reverse) { + assert(0 <= start && start <= end && end <= self->size); res = _PyBytes_ReverseFind( self->data + start, end - start, view.buf, view.len, start); } else { + assert(0 <= start && start <= end && end <= self->size); res = _PyBytes_Find( self->data + start, end - start, view.buf, view.len, start); diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c index 477bc4d31e812..6b9231a9fa769 100644 --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1272,8 +1272,25 @@ _PyBytes_Find(const char *haystack, Py_ssize_t len_haystack, const char *needle, Py_ssize_t len_needle, Py_ssize_t offset) { - return stringlib_find(haystack, len_haystack, - needle, len_needle, offset); + assert(len_haystack >= 0); + assert(len_needle >= 0); + // Extra checks because stringlib_find accesses haystack[len_haystack]. + if (len_needle == 0) { + return offset; + } + if (len_needle > len_haystack) { + return -1; + } + assert(len_haystack >= 1); + Py_ssize_t res = stringlib_find(haystack, len_haystack - 1, + needle, len_needle, offset); + if (res == -1) { + Py_ssize_t last_align = len_haystack - len_needle; + if (memcmp(haystack + last_align, needle, len_needle) == 0) { + return offset + last_align; + } + } + return res; } Py_ssize_t From webhook-mailer at python.org Thu Jul 13 02:08:37 2023 From: webhook-mailer at python.org (vsajip) Date: Thu, 13 Jul 2023 06:08:37 -0000 Subject: [Python-checkins] =?utf-8?q?gh-89427=3A_Set_VIRTUAL=5FENV=5FPROM?= =?utf-8?q?PT_even_when_VIRTUAL=5FENV=5FDISABLE=5FPROMPT=E2=80=A6_=28GH-10?= =?utf-8?q?6643=29?= Message-ID: https://github.com/python/cpython/commit/af51bd7cda9c0cba149b882c1e501765595e5fc3 commit: af51bd7cda9c0cba149b882c1e501765595e5fc3 branch: main author: Jim Porter <826865+jimporter at users.noreply.github.com> committer: vsajip date: 2023-07-13T07:08:33+01:00 summary: gh-89427: Set VIRTUAL_ENV_PROMPT even when VIRTUAL_ENV_DISABLE_PROMPT? (GH-106643) files: A Misc/NEWS.d/next/Library/2023-07-11-12-34-04.gh-issue-89427.GOkCp9.rst M Lib/venv/scripts/common/Activate.ps1 M Lib/venv/scripts/common/activate M Lib/venv/scripts/posix/activate.csh M Lib/venv/scripts/posix/activate.fish diff --git a/Lib/venv/scripts/common/Activate.ps1 b/Lib/venv/scripts/common/Activate.ps1 index eeea3583fa130..d75b8fbcfc777 100644 --- a/Lib/venv/scripts/common/Activate.ps1 +++ b/Lib/venv/scripts/common/Activate.ps1 @@ -219,6 +219,8 @@ deactivate -nondestructive # that there is an activated venv. $env:VIRTUAL_ENV = $VenvDir +$env:VIRTUAL_ENV_PROMPT = $Prompt + if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) { Write-Verbose "Setting prompt to '$Prompt'" @@ -233,7 +235,6 @@ if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) { Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) " _OLD_VIRTUAL_PROMPT } - $env:VIRTUAL_ENV_PROMPT = $Prompt } # Clear PYTHONHOME diff --git a/Lib/venv/scripts/common/activate b/Lib/venv/scripts/common/activate index 408df5cb93b9e..458740a35b0d2 100644 --- a/Lib/venv/scripts/common/activate +++ b/Lib/venv/scripts/common/activate @@ -52,6 +52,9 @@ _OLD_VIRTUAL_PATH="$PATH" PATH="$VIRTUAL_ENV/__VENV_BIN_NAME__:$PATH" export PATH +VIRTUAL_ENV_PROMPT="__VENV_PROMPT__" +export VIRTUAL_ENV_PROMPT + # unset PYTHONHOME if set # this will fail if PYTHONHOME is set to the empty string (which is bad anyway) # could use `if (set -u; : $PYTHONHOME) ;` in bash @@ -64,8 +67,6 @@ if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then _OLD_VIRTUAL_PS1="${PS1:-}" PS1="__VENV_PROMPT__${PS1:-}" export PS1 - VIRTUAL_ENV_PROMPT="__VENV_PROMPT__" - export VIRTUAL_ENV_PROMPT fi # This should detect bash and zsh, which have a hash command that must diff --git a/Lib/venv/scripts/posix/activate.csh b/Lib/venv/scripts/posix/activate.csh index 5e8d66fa9e506..9caf138a919a8 100644 --- a/Lib/venv/scripts/posix/activate.csh +++ b/Lib/venv/scripts/posix/activate.csh @@ -13,13 +13,13 @@ setenv VIRTUAL_ENV "__VENV_DIR__" set _OLD_VIRTUAL_PATH="$PATH" setenv PATH "$VIRTUAL_ENV/__VENV_BIN_NAME__:$PATH" +setenv VIRTUAL_ENV_PROMPT "__VENV_PROMPT__" set _OLD_VIRTUAL_PROMPT="$prompt" if (! "$?VIRTUAL_ENV_DISABLE_PROMPT") then set prompt = "__VENV_PROMPT__$prompt" - setenv VIRTUAL_ENV_PROMPT "__VENV_PROMPT__" endif alias pydoc python -m pydoc diff --git a/Lib/venv/scripts/posix/activate.fish b/Lib/venv/scripts/posix/activate.fish index 91ad6442e0569..565df23d1e2a1 100644 --- a/Lib/venv/scripts/posix/activate.fish +++ b/Lib/venv/scripts/posix/activate.fish @@ -37,6 +37,7 @@ set -gx VIRTUAL_ENV "__VENV_DIR__" set -gx _OLD_VIRTUAL_PATH $PATH set -gx PATH "$VIRTUAL_ENV/__VENV_BIN_NAME__" $PATH +set -gx VIRTUAL_ENV_PROMPT "__VENV_PROMPT__" # Unset PYTHONHOME if set. if set -q PYTHONHOME @@ -65,5 +66,4 @@ if test -z "$VIRTUAL_ENV_DISABLE_PROMPT" end set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV" - set -gx VIRTUAL_ENV_PROMPT "__VENV_PROMPT__" end diff --git a/Misc/NEWS.d/next/Library/2023-07-11-12-34-04.gh-issue-89427.GOkCp9.rst b/Misc/NEWS.d/next/Library/2023-07-11-12-34-04.gh-issue-89427.GOkCp9.rst new file mode 100644 index 0000000000000..1605920cb8138 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-11-12-34-04.gh-issue-89427.GOkCp9.rst @@ -0,0 +1,2 @@ +Set the environment variable ``VIRTUAL_ENV_PROMPT`` at :mod:`venv` +activation, even when ``VIRTUAL_ENV_DISABLE_PROMPT`` is set. From webhook-mailer at python.org Thu Jul 13 02:13:00 2023 From: webhook-mailer at python.org (corona10) Date: Thu, 13 Jul 2023 06:13:00 -0000 Subject: [Python-checkins] gh-106628: email parsing speedup (gh-106629) Message-ID: https://github.com/python/cpython/commit/7e6ce48872fa3de98c986057764f35e1b2f4b936 commit: 7e6ce48872fa3de98c986057764f35e1b2f4b936 branch: main author: CF Bolz-Tereick committer: corona10 date: 2023-07-13T15:12:56+09:00 summary: gh-106628: email parsing speedup (gh-106629) files: A Misc/NEWS.d/next/Library/2023-07-11-16-36-22.gh-issue-106628.Kx8Zvc.rst M Lib/email/feedparser.py diff --git a/Lib/email/feedparser.py b/Lib/email/feedparser.py index 885097c7dda06..53d71f5022515 100644 --- a/Lib/email/feedparser.py +++ b/Lib/email/feedparser.py @@ -37,6 +37,8 @@ headerRE = re.compile(r'^(From |[\041-\071\073-\176]*:|[\t ])') EMPTYSTRING = '' NL = '\n' +boundaryendRE = re.compile( + r'(?P--)?(?P[ \t]*)(?P\r\n|\r|\n)?$') NeedMoreData = object() @@ -327,9 +329,10 @@ def _parsegen(self): # this onto the input stream until we've scanned past the # preamble. separator = '--' + boundary - boundaryre = re.compile( - '(?P' + re.escape(separator) + - r')(?P--)?(?P[ \t]*)(?P\r\n|\r|\n)?$') + def boundarymatch(line): + if not line.startswith(separator): + return None + return boundaryendRE.match(line, len(separator)) capturing_preamble = True preamble = [] linesep = False @@ -341,7 +344,7 @@ def _parsegen(self): continue if line == '': break - mo = boundaryre.match(line) + mo = boundarymatch(line) if mo: # If we're looking at the end boundary, we're done with # this multipart. If there was a newline at the end of @@ -373,13 +376,13 @@ def _parsegen(self): if line is NeedMoreData: yield NeedMoreData continue - mo = boundaryre.match(line) + mo = boundarymatch(line) if not mo: self._input.unreadline(line) break # Recurse to parse this subpart; the input stream points # at the subpart's first line. - self._input.push_eof_matcher(boundaryre.match) + self._input.push_eof_matcher(boundarymatch) for retval in self._parsegen(): if retval is NeedMoreData: yield NeedMoreData diff --git a/Misc/NEWS.d/next/Library/2023-07-11-16-36-22.gh-issue-106628.Kx8Zvc.rst b/Misc/NEWS.d/next/Library/2023-07-11-16-36-22.gh-issue-106628.Kx8Zvc.rst new file mode 100644 index 0000000000000..6fa276e901f64 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-11-16-36-22.gh-issue-106628.Kx8Zvc.rst @@ -0,0 +1,2 @@ +Speed up parsing of emails by about 20% by not compiling a new regular +expression for every single email. From webhook-mailer at python.org Thu Jul 13 03:51:16 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Thu, 13 Jul 2023 07:51:16 -0000 Subject: [Python-checkins] gh-106634: Corrected minor asyncio doc issues (#106671) Message-ID: https://github.com/python/cpython/commit/4b4a5b70aa8d47b1e2a0582b741c31b786da762a commit: 4b4a5b70aa8d47b1e2a0582b741c31b786da762a branch: main author: Chris Brett committer: kumaraditya303 date: 2023-07-13T13:21:13+05:30 summary: gh-106634: Corrected minor asyncio doc issues (#106671) files: M Lib/asyncio/base_events.py M Lib/asyncio/events.py M Misc/ACKS diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index f650e6b0488cf..b092c9343634e 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -727,7 +727,7 @@ def call_later(self, delay, callback, *args, context=None): always relative to the current time. Each callback will be called exactly once. If two callbacks - are scheduled for exactly the same time, it undefined which + are scheduled for exactly the same time, it is undefined which will be called first. Any positional arguments after the callback will be passed to diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py index ce44942186b27..0ccf85105e667 100644 --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -617,7 +617,7 @@ class AbstractEventLoopPolicy: def get_event_loop(self): """Get the event loop for the current context. - Returns an event loop object implementing the BaseEventLoop interface, + Returns an event loop object implementing the AbstractEventLoop interface, or raises an exception in case no event loop has been set for the current context and the current policy does not specify to create one. diff --git a/Misc/ACKS b/Misc/ACKS index ef0029a7e4119..645ad5b700baa 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -226,6 +226,7 @@ Erik Bray Brian Brazil Demian Brecht Dave Brennan +Christopher Richard James Brett Tom Bridgman Anthony Briggs Keith Briggs From webhook-mailer at python.org Thu Jul 13 09:30:39 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 13 Jul 2023 13:30:39 -0000 Subject: [Python-checkins] gh-106309: Deprecate typing.no_type_check_decorator (#106312) Message-ID: https://github.com/python/cpython/commit/32718f908cc92c474fd968912368b8a4500bd055 commit: 32718f908cc92c474fd968912368b8a4500bd055 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-13T14:30:35+01:00 summary: gh-106309: Deprecate typing.no_type_check_decorator (#106312) files: A Misc/NEWS.d/next/Library/2023-07-01-16-51-55.gh-issue-106309.hSlB17.rst M Doc/library/typing.rst M Doc/whatsnew/3.13.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 11af3ea3c9030..0cf875582f7f4 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -2849,6 +2849,9 @@ Functions and decorators This wraps the decorator with something that wraps the decorated function in :func:`no_type_check`. + .. deprecated-removed:: 3.13 3.15 + No type checker ever added support for ``@no_type_check_decorator``. It + is therefore deprecated, and will be removed in Python 3.15. .. decorator:: override @@ -3648,18 +3651,20 @@ Certain features in ``typing`` are deprecated and may be removed in a future version of Python. The following table summarizes major deprecations for your convenience. This is subject to change, and not all deprecations are listed. -+----------------------------------+---------------+-------------------+----------------+ -| Feature | Deprecated in | Projected removal | PEP/issue | -+==================================+===============+===================+================+ -| ``typing`` versions of standard | 3.9 | Undecided | :pep:`585` | -| collections | | | | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.ByteString`` | 3.9 | 3.14 | :gh:`91896` | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.Text`` | 3.11 | Undecided | :gh:`92332` | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.Hashable`` and | 3.12 | Undecided | :gh:`94309` | -| ``typing.Sized`` | | | | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.TypeAlias`` | 3.12 | Undecided | :pep:`695` | -+----------------------------------+---------------+-------------------+----------------+ ++-------------------------------------+---------------+-------------------+----------------+ +| Feature | Deprecated in | Projected removal | PEP/issue | ++=====================================+===============+===================+================+ +| ``typing`` versions of standard | 3.9 | Undecided | :pep:`585` | +| collections | | | | ++-------------------------------------+---------------+-------------------+----------------+ +| ``typing.ByteString`` | 3.9 | 3.14 | :gh:`91896` | ++-------------------------------------+---------------+-------------------+----------------+ +| ``typing.Text`` | 3.11 | Undecided | :gh:`92332` | ++-------------------------------------+---------------+-------------------+----------------+ +| ``typing.Hashable`` and | 3.12 | Undecided | :gh:`94309` | +| ``typing.Sized`` | | | | ++-------------------------------------+---------------+-------------------+----------------+ +| ``typing.TypeAlias`` | 3.12 | Undecided | :pep:`695` | ++-------------------------------------+---------------+-------------------+----------------+ +| ``typing.no_type_check_decorator`` | 3.13 | 3.15 | :gh:`106309` | ++-------------------------------------+---------------+-------------------+----------------+ diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index b7c436fc15161..06fcaf4608cdc 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -161,6 +161,10 @@ Deprecated ``NT = NamedTuple("NT", [])``. To create a TypedDict class with 0 fields, use ``class TD(TypedDict): pass`` or ``TD = TypedDict("TD", {})``. (Contributed by Alex Waygood in :gh:`105566` and :gh:`105570`.) +* :func:`typing.no_type_check_decorator` is deprecated, and scheduled for + removal in Python 3.15. After eight years in the :mod:`typing` module, it + has yet to be supported by any major type checkers. + (Contributed by Alex Waygood in :gh:`106309`.) * :mod:`array`'s ``'u'`` format code, deprecated in docs since Python 3.3, emits :exc:`DeprecationWarning` since 3.13 diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 1df21926d1f67..0450a87577ece 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -5794,10 +5794,14 @@ class F: get_type_hints(clazz) def test_meta_no_type_check(self): - - @no_type_check_decorator - def magic_decorator(func): - return func + depr_msg = ( + "'typing.no_type_check_decorator' is deprecated " + "and slated for removal in Python 3.15" + ) + with self.assertWarnsRegex(DeprecationWarning, depr_msg): + @no_type_check_decorator + def magic_decorator(func): + return func self.assertEqual(magic_decorator.__name__, 'magic_decorator') diff --git a/Lib/typing.py b/Lib/typing.py index 9187b74b0e2e1..387b4c5ad5284 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -2395,6 +2395,8 @@ def no_type_check_decorator(decorator): This wraps the decorator with something that wraps the decorated function in @no_type_check. """ + import warnings + warnings._deprecated("typing.no_type_check_decorator", remove=(3, 15)) @functools.wraps(decorator) def wrapped_decorator(*args, **kwds): func = decorator(*args, **kwds) diff --git a/Misc/NEWS.d/next/Library/2023-07-01-16-51-55.gh-issue-106309.hSlB17.rst b/Misc/NEWS.d/next/Library/2023-07-01-16-51-55.gh-issue-106309.hSlB17.rst new file mode 100644 index 0000000000000..5bd3880520871 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-01-16-51-55.gh-issue-106309.hSlB17.rst @@ -0,0 +1,2 @@ +Deprecate :func:`typing.no_type_check_decorator`. No major type checker ever +added support for this decorator. Patch by Alex Waygood. From webhook-mailer at python.org Thu Jul 13 11:36:22 2023 From: webhook-mailer at python.org (markshannon) Date: Thu, 13 Jul 2023 15:36:22 -0000 Subject: [Python-checkins] GH-104909: Split `LOAD_ATTR_INSTANCE_VALUE` into micro-ops (GH-106678) Message-ID: https://github.com/python/cpython/commit/487861c6aef2fbcd92ccabb05ea1b57d18299b29 commit: 487861c6aef2fbcd92ccabb05ea1b57d18299b29 branch: main author: Mark Shannon committer: markshannon date: 2023-07-13T16:36:19+01:00 summary: GH-104909: Split `LOAD_ATTR_INSTANCE_VALUE` into micro-ops (GH-106678) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-12-11-18-55.gh-issue-104909.DRUsuh.rst M Include/internal/pycore_opcode_metadata.h M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Tools/cases_generator/generate_cases.py diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index d2c1f9ad6e5fb..79bbe9a916f59 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -39,10 +39,12 @@ #define _SKIP_CACHE 317 #define _GUARD_GLOBALS_VERSION 318 #define _GUARD_BUILTINS_VERSION 319 -#define IS_NONE 320 -#define _ITER_CHECK_RANGE 321 -#define _ITER_EXHAUSTED_RANGE 322 -#define _ITER_NEXT_RANGE 323 +#define _GUARD_TYPE_VERSION 320 +#define _CHECK_MANAGED_OBJECT_HAS_VALUES 321 +#define IS_NONE 322 +#define _ITER_CHECK_RANGE 323 +#define _ITER_EXHAUSTED_RANGE 324 +#define _ITER_NEXT_RANGE 325 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -932,7 +934,7 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { } #endif -enum InstructionFormat { INSTR_FMT_IB, INSTR_FMT_IBC, INSTR_FMT_IBC00, INSTR_FMT_IBC000, INSTR_FMT_IBC00000000, INSTR_FMT_IX, INSTR_FMT_IXC, INSTR_FMT_IXC00, INSTR_FMT_IXC000 }; +enum InstructionFormat { INSTR_FMT_IB, INSTR_FMT_IBC, INSTR_FMT_IBC00, INSTR_FMT_IBC000, INSTR_FMT_IBC00000, INSTR_FMT_IBC00000000, INSTR_FMT_IX, INSTR_FMT_IXC, INSTR_FMT_IXC0, INSTR_FMT_IXC00, INSTR_FMT_IXC000 }; #define HAS_ARG_FLAG (1) #define HAS_CONST_FLAG (2) #define HAS_NAME_FLAG (4) @@ -1321,9 +1323,11 @@ const char * const _PyOpcode_uop_name[512] = { [317] = "_SKIP_CACHE", [318] = "_GUARD_GLOBALS_VERSION", [319] = "_GUARD_BUILTINS_VERSION", - [320] = "IS_NONE", - [321] = "_ITER_CHECK_RANGE", - [322] = "_ITER_EXHAUSTED_RANGE", - [323] = "_ITER_NEXT_RANGE", + [320] = "_GUARD_TYPE_VERSION", + [321] = "_CHECK_MANAGED_OBJECT_HAS_VALUES", + [322] = "IS_NONE", + [323] = "_ITER_CHECK_RANGE", + [324] = "_ITER_EXHAUSTED_RANGE", + [325] = "_ITER_NEXT_RANGE", }; #endif // NEED_OPCODE_METADATA diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-12-11-18-55.gh-issue-104909.DRUsuh.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-12-11-18-55.gh-issue-104909.DRUsuh.rst new file mode 100644 index 0000000000000..e0c1e67515a62 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-12-11-18-55.gh-issue-104909.DRUsuh.rst @@ -0,0 +1 @@ +Split :opcode:`LOAD_ATTR_INSTANCE_VALUE` into micro-ops. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 18862f87b65fa..176dbb584d098 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1816,14 +1816,21 @@ dummy_func( LOAD_ATTR, }; - inst(LOAD_ATTR_INSTANCE_VALUE, (unused/1, type_version/2, index/1, unused/5, owner -- res2 if (oparg & 1), res)) { + op(_GUARD_TYPE_VERSION, (type_version/2, owner -- owner)) { PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); - assert(tp->tp_dictoffset < 0); - assert(tp->tp_flags & Py_TPFLAGS_MANAGED_DICT); + } + + op(_CHECK_MANAGED_OBJECT_HAS_VALUES, (owner -- owner)) { + assert(Py_TYPE(owner)->tp_dictoffset < 0); + assert(Py_TYPE(owner)->tp_flags & Py_TPFLAGS_MANAGED_DICT); PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(owner); DEOPT_IF(!_PyDictOrValues_IsValues(dorv), LOAD_ATTR); + } + + op(_LOAD_ATTR_INSTANCE_VALUE, (index/1, unused/5, owner -- res2 if (oparg & 1), res)) { + PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(owner); res = _PyDictOrValues_GetValues(dorv)->values[index]; DEOPT_IF(res == NULL, LOAD_ATTR); STAT_INC(LOAD_ATTR, hit); @@ -1832,6 +1839,12 @@ dummy_func( DECREF_INPUTS(); } + macro(LOAD_ATTR_INSTANCE_VALUE) = + _SKIP_CACHE + // Skip over the counter + _GUARD_TYPE_VERSION + + _CHECK_MANAGED_OBJECT_HAS_VALUES + + _LOAD_ATTR_INSTANCE_VALUE; + inst(LOAD_ATTR_MODULE, (unused/1, type_version/2, index/1, unused/5, owner -- res2 if (oparg & 1), res)) { DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 2c2dbf429cec1..805ea06e07216 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1425,6 +1425,24 @@ break; } + case _GUARD_TYPE_VERSION: { + PyObject *owner = stack_pointer[-1]; + uint32_t type_version = (uint32_t)operand; + PyTypeObject *tp = Py_TYPE(owner); + assert(type_version != 0); + DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); + break; + } + + case _CHECK_MANAGED_OBJECT_HAS_VALUES: { + PyObject *owner = stack_pointer[-1]; + assert(Py_TYPE(owner)->tp_dictoffset < 0); + assert(Py_TYPE(owner)->tp_flags & Py_TPFLAGS_MANAGED_DICT); + PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(owner); + DEOPT_IF(!_PyDictOrValues_IsValues(dorv), LOAD_ATTR); + break; + } + case COMPARE_OP: { static_assert(INLINE_CACHE_ENTRIES_COMPARE_OP == 1, "incorrect cache size"); PyObject *right = stack_pointer[-1]; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 383432f51a89a..d43c7386bd6f6 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -2240,28 +2240,45 @@ } TARGET(LOAD_ATTR_INSTANCE_VALUE) { - PyObject *owner = stack_pointer[-1]; - PyObject *res2 = NULL; - PyObject *res; - uint32_t type_version = read_u32(&next_instr[1].cache); - uint16_t index = read_u16(&next_instr[3].cache); - PyTypeObject *tp = Py_TYPE(owner); - assert(type_version != 0); - DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); - assert(tp->tp_dictoffset < 0); - assert(tp->tp_flags & Py_TPFLAGS_MANAGED_DICT); - PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(owner); - DEOPT_IF(!_PyDictOrValues_IsValues(dorv), LOAD_ATTR); - res = _PyDictOrValues_GetValues(dorv)->values[index]; - DEOPT_IF(res == NULL, LOAD_ATTR); - STAT_INC(LOAD_ATTR, hit); - Py_INCREF(res); - res2 = NULL; - Py_DECREF(owner); - STACK_GROW(((oparg & 1) ? 1 : 0)); - stack_pointer[-1] = res; - if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } + PyObject *_tmp_1; + PyObject *_tmp_2 = stack_pointer[-1]; + { + } + { + PyObject *owner = _tmp_2; + uint32_t type_version = read_u32(&next_instr[1].cache); + PyTypeObject *tp = Py_TYPE(owner); + assert(type_version != 0); + DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); + _tmp_2 = owner; + } + { + PyObject *owner = _tmp_2; + assert(Py_TYPE(owner)->tp_dictoffset < 0); + assert(Py_TYPE(owner)->tp_flags & Py_TPFLAGS_MANAGED_DICT); + PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(owner); + DEOPT_IF(!_PyDictOrValues_IsValues(dorv), LOAD_ATTR); + _tmp_2 = owner; + } + { + PyObject *owner = _tmp_2; + PyObject *res2 = NULL; + PyObject *res; + uint16_t index = read_u16(&next_instr[3].cache); + PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(owner); + res = _PyDictOrValues_GetValues(dorv)->values[index]; + DEOPT_IF(res == NULL, LOAD_ATTR); + STAT_INC(LOAD_ATTR, hit); + Py_INCREF(res); + res2 = NULL; + Py_DECREF(owner); + if (oparg & 1) { _tmp_2 = res2; } + _tmp_1 = res; + } next_instr += 9; + STACK_GROW(((oparg & 1) ? 1 : 0)); + stack_pointer[-1] = _tmp_1; + if (oparg & 1) { stack_pointer[-2] = _tmp_2; } DISPATCH(); } diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index a20abcde85b7c..ba4ef1ee97693 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -839,19 +839,7 @@ def map_families(self) -> None: ) else: member_instr.family = family - elif member_macro := self.macro_instrs.get(member): - for part in member_macro.parts: - if isinstance(part, Component): - if part.instr.family not in (family, None): - self.error( - f"Component {part.instr.name} of macro {member} " - f"is a member of multiple families " - f"({part.instr.family.name}, {family.name}).", - family, - ) - else: - part.instr.family = family - else: + elif not self.macro_instrs.get(member): self.error( f"Unknown instruction {member!r} referenced in family {family.name!r}", family, From webhook-mailer at python.org Thu Jul 13 11:45:25 2023 From: webhook-mailer at python.org (terryjreedy) Date: Thu, 13 Jul 2023 15:45:25 -0000 Subject: [Python-checkins] gh-106690: Add a .coveragerc file to the CPython repository (#8150) Message-ID: https://github.com/python/cpython/commit/2f3ee02c22c4b42bf6075a75104c3cfbb4eb4c86 commit: 2f3ee02c22c4b42bf6075a75104c3cfbb4eb4c86 branch: main author: Ammar Askar committer: terryjreedy date: 2023-07-13T11:45:21-04:00 summary: gh-106690: Add a .coveragerc file to the CPython repository (#8150) The added file is the coverage default at some point in time + checking branches both ways + IDLE additions, labelled as such and somewhat designed to be unlikely to affect other files. Located in the CPython repository directory, it can be used where it is or copied elsewhere, depending on how one runs coverage. --------- Co-authored-by: Terry Jan Reedy Co-authored-by: Alex Waygood Co-authored-by: Erlend E. Aasland files: A .coveragerc A Misc/NEWS.d/next/Tests/2023-07-12-14-07-07.gh-issue-106690.NDz-oG.rst diff --git a/.coveragerc b/.coveragerc new file mode 100644 index 0000000000000..18bf2f40fe523 --- /dev/null +++ b/.coveragerc @@ -0,0 +1,19 @@ +[run] +branch = True + +[report] +# Regexes for lines to exclude from consideration +exclude_lines = + # Don't complain if non-runnable code isn't run: + if 0: + if __name__ == .__main__.: + + .*# pragma: no cover + .*# pragma: no branch + + # Additions for IDLE: + .*# htest # + if not (_htest or _utest): + if not .*_utest: + if .*_htest: + diff --git a/Misc/NEWS.d/next/Tests/2023-07-12-14-07-07.gh-issue-106690.NDz-oG.rst b/Misc/NEWS.d/next/Tests/2023-07-12-14-07-07.gh-issue-106690.NDz-oG.rst new file mode 100644 index 0000000000000..e7dc0ac222050 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-12-14-07-07.gh-issue-106690.NDz-oG.rst @@ -0,0 +1 @@ +Add .coveragerc to cpython repository for use with coverage package. From webhook-mailer at python.org Thu Jul 13 15:14:54 2023 From: webhook-mailer at python.org (gvanrossum) Date: Thu, 13 Jul 2023 19:14:54 -0000 Subject: [Python-checkins] gh-106701: Move the hand-written Tier 2 uops to bytecodes.c (#106702) Message-ID: https://github.com/python/cpython/commit/e6e0ea0113748db1e9fe675be6db9041cd5cce1f commit: e6e0ea0113748db1e9fe675be6db9041cd5cce1f branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-13T12:14:51-07:00 summary: gh-106701: Move the hand-written Tier 2 uops to bytecodes.c (#106702) This moves EXIT_TRACE, SAVE_IP, JUMP_TO_TOP, and _POP_JUMP_IF_{FALSE,TRUE} from ceval.c to bytecodes.c. They are no less special than before, but this way they are discoverable o the copy-and-patch tooling. files: M Include/internal/pycore_opcode_metadata.h M Python/bytecodes.c M Python/ceval.c M Python/executor_cases.c.h M Tools/cases_generator/generate_cases.py diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index 79bbe9a916f59..c88640777e3fb 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -21,30 +21,30 @@ #define EXIT_TRACE 300 #define SAVE_IP 301 -#define _POP_JUMP_IF_FALSE 302 -#define _POP_JUMP_IF_TRUE 303 -#define JUMP_TO_TOP 304 -#define _GUARD_BOTH_INT 305 -#define _BINARY_OP_MULTIPLY_INT 306 -#define _BINARY_OP_ADD_INT 307 -#define _BINARY_OP_SUBTRACT_INT 308 -#define _GUARD_BOTH_FLOAT 309 -#define _BINARY_OP_MULTIPLY_FLOAT 310 -#define _BINARY_OP_ADD_FLOAT 311 -#define _BINARY_OP_SUBTRACT_FLOAT 312 -#define _GUARD_BOTH_UNICODE 313 -#define _BINARY_OP_ADD_UNICODE 314 -#define _LOAD_LOCALS 315 -#define _LOAD_FROM_DICT_OR_GLOBALS 316 -#define _SKIP_CACHE 317 -#define _GUARD_GLOBALS_VERSION 318 -#define _GUARD_BUILTINS_VERSION 319 -#define _GUARD_TYPE_VERSION 320 -#define _CHECK_MANAGED_OBJECT_HAS_VALUES 321 -#define IS_NONE 322 -#define _ITER_CHECK_RANGE 323 -#define _ITER_EXHAUSTED_RANGE 324 -#define _ITER_NEXT_RANGE 325 +#define _GUARD_BOTH_INT 302 +#define _BINARY_OP_MULTIPLY_INT 303 +#define _BINARY_OP_ADD_INT 304 +#define _BINARY_OP_SUBTRACT_INT 305 +#define _GUARD_BOTH_FLOAT 306 +#define _BINARY_OP_MULTIPLY_FLOAT 307 +#define _BINARY_OP_ADD_FLOAT 308 +#define _BINARY_OP_SUBTRACT_FLOAT 309 +#define _GUARD_BOTH_UNICODE 310 +#define _BINARY_OP_ADD_UNICODE 311 +#define _LOAD_LOCALS 312 +#define _LOAD_FROM_DICT_OR_GLOBALS 313 +#define _SKIP_CACHE 314 +#define _GUARD_GLOBALS_VERSION 315 +#define _GUARD_BUILTINS_VERSION 316 +#define _GUARD_TYPE_VERSION 317 +#define _CHECK_MANAGED_OBJECT_HAS_VALUES 318 +#define IS_NONE 319 +#define _ITER_CHECK_RANGE 320 +#define _ITER_EXHAUSTED_RANGE 321 +#define _ITER_NEXT_RANGE 322 +#define _POP_JUMP_IF_FALSE 323 +#define _POP_JUMP_IF_TRUE 324 +#define JUMP_TO_TOP 325 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -1303,31 +1303,31 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [SWAP] = { .nuops = 1, .uops = { { SWAP, 0, 0 } } }, }; const char * const _PyOpcode_uop_name[512] = { - [300] = "EXIT_TRACE", - [301] = "SAVE_IP", - [302] = "_POP_JUMP_IF_FALSE", - [303] = "_POP_JUMP_IF_TRUE", - [304] = "JUMP_TO_TOP", - [305] = "_GUARD_BOTH_INT", - [306] = "_BINARY_OP_MULTIPLY_INT", - [307] = "_BINARY_OP_ADD_INT", - [308] = "_BINARY_OP_SUBTRACT_INT", - [309] = "_GUARD_BOTH_FLOAT", - [310] = "_BINARY_OP_MULTIPLY_FLOAT", - [311] = "_BINARY_OP_ADD_FLOAT", - [312] = "_BINARY_OP_SUBTRACT_FLOAT", - [313] = "_GUARD_BOTH_UNICODE", - [314] = "_BINARY_OP_ADD_UNICODE", - [315] = "_LOAD_LOCALS", - [316] = "_LOAD_FROM_DICT_OR_GLOBALS", - [317] = "_SKIP_CACHE", - [318] = "_GUARD_GLOBALS_VERSION", - [319] = "_GUARD_BUILTINS_VERSION", - [320] = "_GUARD_TYPE_VERSION", - [321] = "_CHECK_MANAGED_OBJECT_HAS_VALUES", - [322] = "IS_NONE", - [323] = "_ITER_CHECK_RANGE", - [324] = "_ITER_EXHAUSTED_RANGE", - [325] = "_ITER_NEXT_RANGE", + [EXIT_TRACE] = "EXIT_TRACE", + [SAVE_IP] = "SAVE_IP", + [_GUARD_BOTH_INT] = "_GUARD_BOTH_INT", + [_BINARY_OP_MULTIPLY_INT] = "_BINARY_OP_MULTIPLY_INT", + [_BINARY_OP_ADD_INT] = "_BINARY_OP_ADD_INT", + [_BINARY_OP_SUBTRACT_INT] = "_BINARY_OP_SUBTRACT_INT", + [_GUARD_BOTH_FLOAT] = "_GUARD_BOTH_FLOAT", + [_BINARY_OP_MULTIPLY_FLOAT] = "_BINARY_OP_MULTIPLY_FLOAT", + [_BINARY_OP_ADD_FLOAT] = "_BINARY_OP_ADD_FLOAT", + [_BINARY_OP_SUBTRACT_FLOAT] = "_BINARY_OP_SUBTRACT_FLOAT", + [_GUARD_BOTH_UNICODE] = "_GUARD_BOTH_UNICODE", + [_BINARY_OP_ADD_UNICODE] = "_BINARY_OP_ADD_UNICODE", + [_LOAD_LOCALS] = "_LOAD_LOCALS", + [_LOAD_FROM_DICT_OR_GLOBALS] = "_LOAD_FROM_DICT_OR_GLOBALS", + [_SKIP_CACHE] = "_SKIP_CACHE", + [_GUARD_GLOBALS_VERSION] = "_GUARD_GLOBALS_VERSION", + [_GUARD_BUILTINS_VERSION] = "_GUARD_BUILTINS_VERSION", + [_GUARD_TYPE_VERSION] = "_GUARD_TYPE_VERSION", + [_CHECK_MANAGED_OBJECT_HAS_VALUES] = "_CHECK_MANAGED_OBJECT_HAS_VALUES", + [IS_NONE] = "IS_NONE", + [_ITER_CHECK_RANGE] = "_ITER_CHECK_RANGE", + [_ITER_EXHAUSTED_RANGE] = "_ITER_EXHAUSTED_RANGE", + [_ITER_NEXT_RANGE] = "_ITER_NEXT_RANGE", + [_POP_JUMP_IF_FALSE] = "_POP_JUMP_IF_FALSE", + [_POP_JUMP_IF_TRUE] = "_POP_JUMP_IF_TRUE", + [JUMP_TO_TOP] = "JUMP_TO_TOP", }; #endif // NEED_OPCODE_METADATA diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 176dbb584d098..1fe9970e53cdf 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -3654,6 +3654,36 @@ dummy_func( Py_UNREACHABLE(); } + ///////// Tier-2 only opcodes ///////// + + op(_POP_JUMP_IF_FALSE, (flag -- )) { + if (Py_IsFalse(flag)) { + pc = oparg; + } + } + + op(_POP_JUMP_IF_TRUE, (flag -- )) { + if (Py_IsTrue(flag)) { + pc = oparg; + } + } + + op(JUMP_TO_TOP, (--)) { + pc = 0; + CHECK_EVAL_BREAKER(); + } + + op(SAVE_IP, (--)) { + frame->prev_instr = ip_offset + oparg; + } + + op(EXIT_TRACE, (--)) { + frame->prev_instr--; // Back up to just before destination + _PyFrame_SetStackPointer(frame, stack_pointer); + Py_DECREF(self); + return frame; + } + // END BYTECODES // diff --git a/Python/ceval.c b/Python/ceval.c index de44085d732cf..d6c72fa3ff386 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2764,46 +2764,6 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject #define ENABLE_SPECIALIZATION 0 #include "executor_cases.c.h" - // NOTE: These pop-jumps move the uop pc, not the bytecode ip - case _POP_JUMP_IF_FALSE: - { - if (Py_IsFalse(stack_pointer[-1])) { - pc = oparg; - } - stack_pointer--; - break; - } - - case _POP_JUMP_IF_TRUE: - { - if (Py_IsTrue(stack_pointer[-1])) { - pc = oparg; - } - stack_pointer--; - break; - } - - case JUMP_TO_TOP: - { - pc = 0; - CHECK_EVAL_BREAKER(); - break; - } - - case SAVE_IP: - { - frame->prev_instr = ip_offset + oparg; - break; - } - - case EXIT_TRACE: - { - frame->prev_instr--; // Back up to just before destination - _PyFrame_SetStackPointer(frame, stack_pointer); - Py_DECREF(self); - return frame; - } - default: { fprintf(stderr, "Unknown uop %d, operand %" PRIu64 "\n", opcode, operand); diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 805ea06e07216..ce54755d5d25f 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1987,3 +1987,39 @@ stack_pointer[-(2 + (oparg-2))] = top; break; } + + case _POP_JUMP_IF_FALSE: { + PyObject *flag = stack_pointer[-1]; + if (Py_IsFalse(flag)) { + pc = oparg; + } + STACK_SHRINK(1); + break; + } + + case _POP_JUMP_IF_TRUE: { + PyObject *flag = stack_pointer[-1]; + if (Py_IsTrue(flag)) { + pc = oparg; + } + STACK_SHRINK(1); + break; + } + + case JUMP_TO_TOP: { + pc = 0; + break; + } + + case SAVE_IP: { + frame->prev_instr = ip_offset + oparg; + break; + } + + case EXIT_TRACE: { + frame->prev_instr--; // Back up to just before destination + _PyFrame_SetStackPointer(frame, stack_pointer); + Py_DECREF(self); + return frame; + break; + } diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index ba4ef1ee97693..ce16271097b95 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -410,6 +410,8 @@ def __init__(self, inst: parser.InstDef): def is_viable_uop(self) -> bool: """Whether this instruction is viable as a uop.""" + if self.name == "EXIT_TRACE": + return True # This has 'return frame' but it's okay if self.always_exits: # print(f"Skipping {self.name} because it always exits") return False @@ -1278,7 +1280,7 @@ def write_metadata(self) -> None: typing.assert_never(thing) with self.out.block("const char * const _PyOpcode_uop_name[512] =", ";"): - self.write_uop_items(lambda name, counter: f"[{counter}] = \"{name}\",") + self.write_uop_items(lambda name, counter: f"[{name}] = \"{name}\",") self.out.emit("#endif // NEED_OPCODE_METADATA") @@ -1324,17 +1326,19 @@ def write_pseudo_instrs(self) -> None: def write_uop_items(self, make_text: typing.Callable[[str, int], str]) -> None: """Write '#define XXX NNN' for each uop""" counter = 300 # TODO: Avoid collision with pseudo instructions + seen = set() def add(name: str) -> None: + if name in seen: + return nonlocal counter self.out.emit(make_text(name, counter)) counter += 1 + seen.add(name) + # These two are first by convention add("EXIT_TRACE") add("SAVE_IP") - add("_POP_JUMP_IF_FALSE") - add("_POP_JUMP_IF_TRUE") - add("JUMP_TO_TOP") for instr in self.instrs.values(): if instr.kind == "op" and instr.is_viable_uop(): From webhook-mailer at python.org Thu Jul 13 15:18:57 2023 From: webhook-mailer at python.org (methane) Date: Thu, 13 Jul 2023 19:18:57 -0000 Subject: [Python-checkins] gh-106664: selectors: add get() method to _SelectorMapping (#106665) Message-ID: https://github.com/python/cpython/commit/8d2f3c36caf9ecdee1176314b18388aef6e7f2c2 commit: 8d2f3c36caf9ecdee1176314b18388aef6e7f2c2 branch: main author: J. Nick Koston committer: methane date: 2023-07-13T19:18:53Z summary: gh-106664: selectors: add get() method to _SelectorMapping (#106665) It can be used to avoid raising and catching KeyError twice via __getitem__. Co-authored-by: Inada Naoki files: A Misc/NEWS.d/next/Library/2023-07-12-03-04-45.gh-issue-106664.ZeUG78.rst M Lib/selectors.py M Lib/test/test_selectors.py diff --git a/Lib/selectors.py b/Lib/selectors.py index af6a4f94b5008..dfcc125dcd94e 100644 --- a/Lib/selectors.py +++ b/Lib/selectors.py @@ -66,12 +66,16 @@ def __init__(self, selector): def __len__(self): return len(self._selector._fd_to_key) + def get(self, fileobj, default=None): + fd = self._selector._fileobj_lookup(fileobj) + return self._selector._fd_to_key.get(fd, default) + def __getitem__(self, fileobj): - try: - fd = self._selector._fileobj_lookup(fileobj) - return self._selector._fd_to_key[fd] - except KeyError: - raise KeyError("{!r} is not registered".format(fileobj)) from None + fd = self._selector._fileobj_lookup(fileobj) + key = self._selector._fd_to_key.get(fd) + if key is None: + raise KeyError("{!r} is not registered".format(fileobj)) + return key def __iter__(self): return iter(self._selector._fd_to_key) diff --git a/Lib/test/test_selectors.py b/Lib/test/test_selectors.py index c2db88c203920..4545cbadb796f 100644 --- a/Lib/test/test_selectors.py +++ b/Lib/test/test_selectors.py @@ -223,6 +223,8 @@ def test_close(self): self.assertRaises(RuntimeError, s.get_key, wr) self.assertRaises(KeyError, mapping.__getitem__, rd) self.assertRaises(KeyError, mapping.__getitem__, wr) + self.assertEqual(mapping.get(rd), None) + self.assertEqual(mapping.get(wr), None) def test_get_key(self): s = self.SELECTOR() @@ -241,13 +243,17 @@ def test_get_map(self): self.addCleanup(s.close) rd, wr = self.make_socketpair() + sentinel = object() keys = s.get_map() self.assertFalse(keys) self.assertEqual(len(keys), 0) self.assertEqual(list(keys), []) + self.assertEqual(keys.get(rd), None) + self.assertEqual(keys.get(rd, sentinel), sentinel) key = s.register(rd, selectors.EVENT_READ, "data") self.assertIn(rd, keys) + self.assertEqual(key, keys.get(rd)) self.assertEqual(key, keys[rd]) self.assertEqual(len(keys), 1) self.assertEqual(list(keys), [rd.fileno()]) diff --git a/Misc/NEWS.d/next/Library/2023-07-12-03-04-45.gh-issue-106664.ZeUG78.rst b/Misc/NEWS.d/next/Library/2023-07-12-03-04-45.gh-issue-106664.ZeUG78.rst new file mode 100644 index 0000000000000..c278cad74bd04 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-12-03-04-45.gh-issue-106664.ZeUG78.rst @@ -0,0 +1 @@ +:mod:`selectors`: Add ``_SelectorMapping.get()`` method and optimize ``_SelectorMapping.__getitem__()``. From webhook-mailer at python.org Thu Jul 13 15:24:58 2023 From: webhook-mailer at python.org (barneygale) Date: Thu, 13 Jul 2023 19:24:58 -0000 Subject: [Python-checkins] docs: clarify Path.suffix (GH-106650) Message-ID: https://github.com/python/cpython/commit/f014f1567c081542aaf5cecce118edf35e09bcc5 commit: f014f1567c081542aaf5cecce118edf35e09bcc5 branch: main author: Ned Batchelder committer: barneygale date: 2023-07-13T20:24:54+01:00 summary: docs: clarify Path.suffix (GH-106650) files: M Doc/library/pathlib.rst M Doc/library/zipfile.rst diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst index 338575404ff0a..af81df217eea9 100644 --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -432,7 +432,7 @@ Pure paths provide the following methods and properties: .. attribute:: PurePath.suffix - The file extension of the final component, if any:: + The last dot-separated portion of the final component, if any:: >>> PurePosixPath('my/library/setup.py').suffix '.py' @@ -441,10 +441,11 @@ Pure paths provide the following methods and properties: >>> PurePosixPath('my/library').suffix '' + This is commonly called the file extension. .. attribute:: PurePath.suffixes - A list of the path's file extensions:: + A list of the path's suffixes, often called file extensions:: >>> PurePosixPath('my/library.tar.gar').suffixes ['.tar', '.gar'] diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst index 45f3d340bd82d..bd951e4872f11 100644 --- a/Doc/library/zipfile.rst +++ b/Doc/library/zipfile.rst @@ -577,7 +577,8 @@ Path objects are traversable using the ``/`` operator or ``joinpath``. .. data:: Path.suffix - The file extension of the final component. + The last dot-separated portion of the final component, if any. + This is commonly called the file extension. .. versionadded:: 3.11 Added :data:`Path.suffix` property. @@ -591,7 +592,7 @@ Path objects are traversable using the ``/`` operator or ``joinpath``. .. data:: Path.suffixes - A list of the path?s file extensions. + A list of the path?s suffixes, commonly called file extensions. .. versionadded:: 3.11 Added :data:`Path.suffixes` property. From webhook-mailer at python.org Thu Jul 13 18:18:36 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Thu, 13 Jul 2023 22:18:36 -0000 Subject: [Python-checkins] gh-106368: Increase Argument Clinic test coverage (#106728) Message-ID: https://github.com/python/cpython/commit/ec45c513d389510930a62631a21a1dbb3f3aabb7 commit: ec45c513d389510930a62631a21a1dbb3f3aabb7 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-13T22:18:32Z summary: gh-106368: Increase Argument Clinic test coverage (#106728) - improve output_parameter() coverage - improve coverage for Function.kind files: M Lib/test/clinic.test.c M Lib/test/test_clinic.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index da97c4bdd7e8e..2fd8760415dc7 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -3,6 +3,10 @@ output preset block [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=3c81ac2402d06a8b]*/ +/*[clinic input] +class Test "TestObj *" "TestType" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=fc7e50384d12b83f]*/ /*[clinic input] test_object_converter @@ -61,6 +65,58 @@ test_object_converter_impl(PyObject *module, PyObject *a, PyObject *b, /*[clinic end generated code: output=886f4f9b598726b6 input=005e6a8a711a869b]*/ +/*[clinic input] +cloned = test_object_converter +Check the clone feature. +[clinic start generated code]*/ + +PyDoc_STRVAR(cloned__doc__, +"cloned($module, a, b, c, d, /)\n" +"--\n" +"\n" +"Check the clone feature."); + +#define CLONED_METHODDEF \ + {"cloned", _PyCFunction_CAST(cloned), METH_FASTCALL, cloned__doc__}, + +static PyObject * +cloned_impl(PyObject *module, PyObject *a, PyObject *b, PyObject *c, + PyUnicode_Object *d); + +static PyObject * +cloned(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *a; + PyObject *b; + PyObject *c; + PyUnicode_Object *d; + + if (!_PyArg_CheckPositional("cloned", nargs, 4, 4)) { + goto exit; + } + a = args[0]; + if (!PyUnicode_FSConverter(args[1], &b)) { + goto exit; + } + if (!PyUnicode_Check(args[2])) { + _PyArg_BadArgument("cloned", "argument 3", "str", args[2]); + goto exit; + } + c = args[2]; + d = (PyUnicode_Object *)args[3]; + return_value = cloned_impl(module, a, b, c, d); + +exit: + return return_value; +} + +static PyObject * +cloned_impl(PyObject *module, PyObject *a, PyObject *b, PyObject *c, + PyUnicode_Object *d) +/*[clinic end generated code: output=026b483e27c38065 input=0543614019d6fcc7]*/ + + /*[clinic input] test_object_converter_one_arg @@ -4265,3 +4321,500 @@ static PyObject * mangle2_impl(PyObject *module, PyObject *args, PyObject *kwargs, PyObject *return_value) /*[clinic end generated code: output=2ebb62aaefe7590a input=391766fee51bad7a]*/ + + +/*[clinic input] +Test.cls_with_param + cls: defining_class + / + a: int +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_cls_with_param__doc__, +"cls_with_param($self, /, a)\n" +"--\n" +"\n"); + +#define TEST_CLS_WITH_PARAM_METHODDEF \ + {"cls_with_param", _PyCFunction_CAST(Test_cls_with_param), METH_METHOD|METH_FASTCALL|METH_KEYWORDS, Test_cls_with_param__doc__}, + +static PyObject * +Test_cls_with_param_impl(TestObj *self, PyTypeObject *cls, int a); + +static PyObject * +Test_cls_with_param(TestObj *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(a), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"a", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "cls_with_param", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int a; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + a = _PyLong_AsInt(args[0]); + if (a == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = Test_cls_with_param_impl(self, cls, a); + +exit: + return return_value; +} + +static PyObject * +Test_cls_with_param_impl(TestObj *self, PyTypeObject *cls, int a) +/*[clinic end generated code: output=00218e7f583e6c81 input=af158077bd237ef9]*/ + + +/*[clinic input] +Test.__init__ +Empty init method. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test___init____doc__, +"Test()\n" +"--\n" +"\n" +"Empty init method."); + +static int +Test___init___impl(TestObj *self); + +static int +Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + PyTypeObject *base_tp = TestType; + + if ((Py_IS_TYPE(self, base_tp) || + Py_TYPE(self)->tp_new == base_tp->tp_new) && + !_PyArg_NoPositional("Test", args)) { + goto exit; + } + if ((Py_IS_TYPE(self, base_tp) || + Py_TYPE(self)->tp_new == base_tp->tp_new) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + return_value = Test___init___impl((TestObj *)self); + +exit: + return return_value; +} + +static int +Test___init___impl(TestObj *self) +/*[clinic end generated code: output=f6a35c85bc5b408f input=4ea79fee54d0c3ff]*/ + + +/*[clinic input] + at classmethod +Test.__new__ +Empty new method. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test__doc__, +"Test()\n" +"--\n" +"\n" +"Empty new method."); + +static PyObject * +Test_impl(PyTypeObject *type); + +static PyObject * +Test(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyTypeObject *base_tp = TestType; + + if ((type == base_tp || type->tp_init == base_tp->tp_init) && + !_PyArg_NoPositional("Test", args)) { + goto exit; + } + if ((type == base_tp || type->tp_init == base_tp->tp_init) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + return_value = Test_impl(type); + +exit: + return return_value; +} + +static PyObject * +Test_impl(PyTypeObject *type) +/*[clinic end generated code: output=68a117adc057940f input=6fe98a19f097907f]*/ + + +/*[clinic input] +Test.cls_no_params + cls: defining_class + / +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_cls_no_params__doc__, +"cls_no_params($self, /)\n" +"--\n" +"\n"); + +#define TEST_CLS_NO_PARAMS_METHODDEF \ + {"cls_no_params", _PyCFunction_CAST(Test_cls_no_params), METH_METHOD|METH_FASTCALL|METH_KEYWORDS, Test_cls_no_params__doc__}, + +static PyObject * +Test_cls_no_params_impl(TestObj *self, PyTypeObject *cls); + +static PyObject * +Test_cls_no_params(TestObj *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + if (nargs) { + PyErr_SetString(PyExc_TypeError, "cls_no_params() takes no arguments"); + return NULL; + } + return Test_cls_no_params_impl(self, cls); +} + +static PyObject * +Test_cls_no_params_impl(TestObj *self, PyTypeObject *cls) +/*[clinic end generated code: output=cc8845f22cff3dcb input=e7e2e4e344e96a11]*/ + + +/*[clinic input] +Test.metho_not_default_return_converter -> int + a: object + / +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_metho_not_default_return_converter__doc__, +"metho_not_default_return_converter($self, a, /)\n" +"--\n" +"\n"); + +#define TEST_METHO_NOT_DEFAULT_RETURN_CONVERTER_METHODDEF \ + {"metho_not_default_return_converter", (PyCFunction)Test_metho_not_default_return_converter, METH_O, Test_metho_not_default_return_converter__doc__}, + +static int +Test_metho_not_default_return_converter_impl(TestObj *self, PyObject *a); + +static PyObject * +Test_metho_not_default_return_converter(TestObj *self, PyObject *a) +{ + PyObject *return_value = NULL; + int _return_value; + + _return_value = Test_metho_not_default_return_converter_impl(self, a); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyLong_FromLong((long)_return_value); + +exit: + return return_value; +} + +static int +Test_metho_not_default_return_converter_impl(TestObj *self, PyObject *a) +/*[clinic end generated code: output=3350de11bd538007 input=428657129b521177]*/ + + +/*[clinic input] +Test.an_metho_arg_named_arg + arg: int + Name should be mangled to 'arg_' in generated output. + / +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_an_metho_arg_named_arg__doc__, +"an_metho_arg_named_arg($self, arg, /)\n" +"--\n" +"\n" +"\n" +"\n" +" arg\n" +" Name should be mangled to \'arg_\' in generated output."); + +#define TEST_AN_METHO_ARG_NAMED_ARG_METHODDEF \ + {"an_metho_arg_named_arg", (PyCFunction)Test_an_metho_arg_named_arg, METH_O, Test_an_metho_arg_named_arg__doc__}, + +static PyObject * +Test_an_metho_arg_named_arg_impl(TestObj *self, int arg); + +static PyObject * +Test_an_metho_arg_named_arg(TestObj *self, PyObject *arg_) +{ + PyObject *return_value = NULL; + int arg; + + arg = _PyLong_AsInt(arg_); + if (arg == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = Test_an_metho_arg_named_arg_impl(self, arg); + +exit: + return return_value; +} + +static PyObject * +Test_an_metho_arg_named_arg_impl(TestObj *self, int arg) +/*[clinic end generated code: output=7d590626642194ae input=2a53a57cf5624f95]*/ + + +/*[clinic input] +Test.__init__ + *args: object + / +Varargs init method. For example, nargs is translated to PyTuple_GET_SIZE. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test___init____doc__, +"Test(*args)\n" +"--\n" +"\n" +"Varargs init method. For example, nargs is translated to PyTuple_GET_SIZE."); + +static int +Test___init___impl(TestObj *self, PyObject *args); + +static int +Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + PyTypeObject *base_tp = TestType; + PyObject *__clinic_args = NULL; + + if ((Py_IS_TYPE(self, base_tp) || + Py_TYPE(self)->tp_new == base_tp->tp_new) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + if (!_PyArg_CheckPositional("Test", PyTuple_GET_SIZE(args), 0, PY_SSIZE_T_MAX)) { + goto exit; + } + __clinic_args = PyTuple_GetSlice(0, -1); + return_value = Test___init___impl((TestObj *)self, __clinic_args); + +exit: + Py_XDECREF(__clinic_args); + return return_value; +} + +static int +Test___init___impl(TestObj *self, PyObject *args) +/*[clinic end generated code: output=0ed1009fe0dcf98d input=96c3ddc0cd38fc0c]*/ + + +/*[clinic input] + at classmethod +Test.__new__ + *args: object + / +Varargs new method. For example, nargs is translated to PyTuple_GET_SIZE. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test__doc__, +"Test(*args)\n" +"--\n" +"\n" +"Varargs new method. For example, nargs is translated to PyTuple_GET_SIZE."); + +static PyObject * +Test_impl(PyTypeObject *type, PyObject *args); + +static PyObject * +Test(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyTypeObject *base_tp = TestType; + PyObject *__clinic_args = NULL; + + if ((type == base_tp || type->tp_init == base_tp->tp_init) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + if (!_PyArg_CheckPositional("Test", PyTuple_GET_SIZE(args), 0, PY_SSIZE_T_MAX)) { + goto exit; + } + __clinic_args = PyTuple_GetSlice(0, -1); + return_value = Test_impl(type, __clinic_args); + +exit: + Py_XDECREF(__clinic_args); + return return_value; +} + +static PyObject * +Test_impl(PyTypeObject *type, PyObject *args) +/*[clinic end generated code: output=8b219f6633e2a2e9 input=26a672e2e9750120]*/ + + +/*[clinic input] +Test.__init__ + a: object +Init method with positional or keyword arguments. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test___init____doc__, +"Test(a)\n" +"--\n" +"\n" +"Init method with positional or keyword arguments."); + +static int +Test___init___impl(TestObj *self, PyObject *a); + +static int +Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(a), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"a", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "Test", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + PyObject * const *fastargs; + Py_ssize_t nargs = PyTuple_GET_SIZE(args); + PyObject *a; + + fastargs = _PyArg_UnpackKeywords(_PyTuple_CAST(args)->ob_item, nargs, kwargs, NULL, &_parser, 1, 1, 0, argsbuf); + if (!fastargs) { + goto exit; + } + a = fastargs[0]; + return_value = Test___init___impl((TestObj *)self, a); + +exit: + return return_value; +} + +static int +Test___init___impl(TestObj *self, PyObject *a) +/*[clinic end generated code: output=0b9ca79638ab3ecb input=a8f9222a6ab35c59]*/ + + +/*[clinic input] + at classmethod +Test.class_method +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_class_method__doc__, +"class_method($type, /)\n" +"--\n" +"\n"); + +#define TEST_CLASS_METHOD_METHODDEF \ + {"class_method", (PyCFunction)Test_class_method, METH_NOARGS|METH_CLASS, Test_class_method__doc__}, + +static PyObject * +Test_class_method_impl(PyTypeObject *type); + +static PyObject * +Test_class_method(PyTypeObject *type, PyObject *Py_UNUSED(ignored)) +{ + return Test_class_method_impl(type); +} + +static PyObject * +Test_class_method_impl(PyTypeObject *type) +/*[clinic end generated code: output=47fb7ecca1abcaaa input=43bc4a0494547b80]*/ + + +/*[clinic input] + at staticmethod +Test.static_method +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_static_method__doc__, +"static_method()\n" +"--\n" +"\n"); + +#define TEST_STATIC_METHOD_METHODDEF \ + {"static_method", (PyCFunction)Test_static_method, METH_NOARGS|METH_STATIC, Test_static_method__doc__}, + +static PyObject * +Test_static_method_impl(); + +static PyObject * +Test_static_method(void *null, PyObject *Py_UNUSED(ignored)) +{ + return Test_static_method_impl(); +} + +static PyObject * +Test_static_method_impl() +/*[clinic end generated code: output=82524a63025cf7ab input=dae892fac55ae72b]*/ + + +/*[clinic input] + at coexist +Test.meth_coexist +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_meth_coexist__doc__, +"meth_coexist($self, /)\n" +"--\n" +"\n"); + +#define TEST_METH_COEXIST_METHODDEF \ + {"meth_coexist", (PyCFunction)Test_meth_coexist, METH_NOARGS|METH_COEXIST, Test_meth_coexist__doc__}, + +static PyObject * +Test_meth_coexist_impl(TestObj *self); + +static PyObject * +Test_meth_coexist(TestObj *self, PyObject *Py_UNUSED(ignored)) +{ + return Test_meth_coexist_impl(self); +} + +static PyObject * +Test_meth_coexist_impl(TestObj *self) +/*[clinic end generated code: output=808a293d0cd27439 input=2a1d75b5e6fec6dd]*/ diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 685ba58642a5a..975840333e590 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1013,6 +1013,43 @@ def test_defining_class_param_cannot_be_optional(self): out = self.parse_function_should_fail(block) self.assertEqual(out, expected_error_msg) + def test_slot_methods_cannot_access_defining_class(self): + block = """ + module foo + class Foo "" "" + Foo.__init__ + cls: defining_class + a: object + """ + msg = "Slot methods cannot access their defining class." + with self.assertRaisesRegex(ValueError, msg): + self.parse_function(block) + + def test_new_must_be_a_class_method(self): + expected_error_msg = ( + "Error on line 0:\n" + "__new__ must be a class method!\n" + ) + out = self.parse_function_should_fail(""" + module foo + class Foo "" "" + Foo.__new__ + """) + self.assertEqual(out, expected_error_msg) + + def test_init_must_be_a_normal_method(self): + expected_error_msg = ( + "Error on line 0:\n" + "__init__ must be a normal method, not a class or static method!\n" + ) + out = self.parse_function_should_fail(""" + module foo + class Foo "" "" + @classmethod + Foo.__init__ + """) + self.assertEqual(out, expected_error_msg) + def test_unused_param(self): block = self.parse(""" module foo From webhook-mailer at python.org Thu Jul 13 18:52:24 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Thu, 13 Jul 2023 22:52:24 -0000 Subject: [Python-checkins] [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106728) (#106730) Message-ID: https://github.com/python/cpython/commit/6a1a6601582ea893a81fd613f6432c85fc14f460 commit: 6a1a6601582ea893a81fd613f6432c85fc14f460 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-13T22:52:21Z summary: [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106728) (#106730) - improve output_parameter() coverage - improve coverage for Function.kind (cherry picked from commit ec45c513d389510930a62631a21a1dbb3f3aabb7) Co-authored-by: Erlend E. Aasland files: M Lib/test/clinic.test.c M Lib/test/test_clinic.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index 564205274edd7..1544599c29fa0 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -3,6 +3,10 @@ output preset block [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=3c81ac2402d06a8b]*/ +/*[clinic input] +class Test "TestObj *" "TestType" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=fc7e50384d12b83f]*/ /*[clinic input] test_object_converter @@ -61,6 +65,58 @@ test_object_converter_impl(PyObject *module, PyObject *a, PyObject *b, /*[clinic end generated code: output=886f4f9b598726b6 input=005e6a8a711a869b]*/ +/*[clinic input] +cloned = test_object_converter +Check the clone feature. +[clinic start generated code]*/ + +PyDoc_STRVAR(cloned__doc__, +"cloned($module, a, b, c, d, /)\n" +"--\n" +"\n" +"Check the clone feature."); + +#define CLONED_METHODDEF \ + {"cloned", _PyCFunction_CAST(cloned), METH_FASTCALL, cloned__doc__}, + +static PyObject * +cloned_impl(PyObject *module, PyObject *a, PyObject *b, PyObject *c, + PyUnicode_Object *d); + +static PyObject * +cloned(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *a; + PyObject *b; + PyObject *c; + PyUnicode_Object *d; + + if (!_PyArg_CheckPositional("cloned", nargs, 4, 4)) { + goto exit; + } + a = args[0]; + if (!PyUnicode_FSConverter(args[1], &b)) { + goto exit; + } + if (!PyUnicode_Check(args[2])) { + _PyArg_BadArgument("cloned", "argument 3", "str", args[2]); + goto exit; + } + c = args[2]; + d = (PyUnicode_Object *)args[3]; + return_value = cloned_impl(module, a, b, c, d); + +exit: + return return_value; +} + +static PyObject * +cloned_impl(PyObject *module, PyObject *a, PyObject *b, PyObject *c, + PyUnicode_Object *d) +/*[clinic end generated code: output=026b483e27c38065 input=0543614019d6fcc7]*/ + + /*[clinic input] test_object_converter_one_arg @@ -4271,3 +4327,500 @@ static PyObject * mangle2_impl(PyObject *module, PyObject *args, PyObject *kwargs, PyObject *return_value) /*[clinic end generated code: output=2ebb62aaefe7590a input=391766fee51bad7a]*/ + + +/*[clinic input] +Test.cls_with_param + cls: defining_class + / + a: int +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_cls_with_param__doc__, +"cls_with_param($self, /, a)\n" +"--\n" +"\n"); + +#define TEST_CLS_WITH_PARAM_METHODDEF \ + {"cls_with_param", _PyCFunction_CAST(Test_cls_with_param), METH_METHOD|METH_FASTCALL|METH_KEYWORDS, Test_cls_with_param__doc__}, + +static PyObject * +Test_cls_with_param_impl(TestObj *self, PyTypeObject *cls, int a); + +static PyObject * +Test_cls_with_param(TestObj *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(a), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"a", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "cls_with_param", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int a; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + a = _PyLong_AsInt(args[0]); + if (a == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = Test_cls_with_param_impl(self, cls, a); + +exit: + return return_value; +} + +static PyObject * +Test_cls_with_param_impl(TestObj *self, PyTypeObject *cls, int a) +/*[clinic end generated code: output=00218e7f583e6c81 input=af158077bd237ef9]*/ + + +/*[clinic input] +Test.__init__ +Empty init method. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test___init____doc__, +"Test()\n" +"--\n" +"\n" +"Empty init method."); + +static int +Test___init___impl(TestObj *self); + +static int +Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + PyTypeObject *base_tp = TestType; + + if ((Py_IS_TYPE(self, base_tp) || + Py_TYPE(self)->tp_new == base_tp->tp_new) && + !_PyArg_NoPositional("Test", args)) { + goto exit; + } + if ((Py_IS_TYPE(self, base_tp) || + Py_TYPE(self)->tp_new == base_tp->tp_new) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + return_value = Test___init___impl((TestObj *)self); + +exit: + return return_value; +} + +static int +Test___init___impl(TestObj *self) +/*[clinic end generated code: output=f6a35c85bc5b408f input=4ea79fee54d0c3ff]*/ + + +/*[clinic input] + at classmethod +Test.__new__ +Empty new method. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test__doc__, +"Test()\n" +"--\n" +"\n" +"Empty new method."); + +static PyObject * +Test_impl(PyTypeObject *type); + +static PyObject * +Test(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyTypeObject *base_tp = TestType; + + if ((type == base_tp || type->tp_init == base_tp->tp_init) && + !_PyArg_NoPositional("Test", args)) { + goto exit; + } + if ((type == base_tp || type->tp_init == base_tp->tp_init) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + return_value = Test_impl(type); + +exit: + return return_value; +} + +static PyObject * +Test_impl(PyTypeObject *type) +/*[clinic end generated code: output=68a117adc057940f input=6fe98a19f097907f]*/ + + +/*[clinic input] +Test.cls_no_params + cls: defining_class + / +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_cls_no_params__doc__, +"cls_no_params($self, /)\n" +"--\n" +"\n"); + +#define TEST_CLS_NO_PARAMS_METHODDEF \ + {"cls_no_params", _PyCFunction_CAST(Test_cls_no_params), METH_METHOD|METH_FASTCALL|METH_KEYWORDS, Test_cls_no_params__doc__}, + +static PyObject * +Test_cls_no_params_impl(TestObj *self, PyTypeObject *cls); + +static PyObject * +Test_cls_no_params(TestObj *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + if (nargs) { + PyErr_SetString(PyExc_TypeError, "cls_no_params() takes no arguments"); + return NULL; + } + return Test_cls_no_params_impl(self, cls); +} + +static PyObject * +Test_cls_no_params_impl(TestObj *self, PyTypeObject *cls) +/*[clinic end generated code: output=cc8845f22cff3dcb input=e7e2e4e344e96a11]*/ + + +/*[clinic input] +Test.metho_not_default_return_converter -> int + a: object + / +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_metho_not_default_return_converter__doc__, +"metho_not_default_return_converter($self, a, /)\n" +"--\n" +"\n"); + +#define TEST_METHO_NOT_DEFAULT_RETURN_CONVERTER_METHODDEF \ + {"metho_not_default_return_converter", (PyCFunction)Test_metho_not_default_return_converter, METH_O, Test_metho_not_default_return_converter__doc__}, + +static int +Test_metho_not_default_return_converter_impl(TestObj *self, PyObject *a); + +static PyObject * +Test_metho_not_default_return_converter(TestObj *self, PyObject *a) +{ + PyObject *return_value = NULL; + int _return_value; + + _return_value = Test_metho_not_default_return_converter_impl(self, a); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyLong_FromLong((long)_return_value); + +exit: + return return_value; +} + +static int +Test_metho_not_default_return_converter_impl(TestObj *self, PyObject *a) +/*[clinic end generated code: output=3350de11bd538007 input=428657129b521177]*/ + + +/*[clinic input] +Test.an_metho_arg_named_arg + arg: int + Name should be mangled to 'arg_' in generated output. + / +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_an_metho_arg_named_arg__doc__, +"an_metho_arg_named_arg($self, arg, /)\n" +"--\n" +"\n" +"\n" +"\n" +" arg\n" +" Name should be mangled to \'arg_\' in generated output."); + +#define TEST_AN_METHO_ARG_NAMED_ARG_METHODDEF \ + {"an_metho_arg_named_arg", (PyCFunction)Test_an_metho_arg_named_arg, METH_O, Test_an_metho_arg_named_arg__doc__}, + +static PyObject * +Test_an_metho_arg_named_arg_impl(TestObj *self, int arg); + +static PyObject * +Test_an_metho_arg_named_arg(TestObj *self, PyObject *arg_) +{ + PyObject *return_value = NULL; + int arg; + + arg = _PyLong_AsInt(arg_); + if (arg == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = Test_an_metho_arg_named_arg_impl(self, arg); + +exit: + return return_value; +} + +static PyObject * +Test_an_metho_arg_named_arg_impl(TestObj *self, int arg) +/*[clinic end generated code: output=7d590626642194ae input=2a53a57cf5624f95]*/ + + +/*[clinic input] +Test.__init__ + *args: object + / +Varargs init method. For example, nargs is translated to PyTuple_GET_SIZE. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test___init____doc__, +"Test(*args)\n" +"--\n" +"\n" +"Varargs init method. For example, nargs is translated to PyTuple_GET_SIZE."); + +static int +Test___init___impl(TestObj *self, PyObject *args); + +static int +Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + PyTypeObject *base_tp = TestType; + PyObject *__clinic_args = NULL; + + if ((Py_IS_TYPE(self, base_tp) || + Py_TYPE(self)->tp_new == base_tp->tp_new) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + if (!_PyArg_CheckPositional("Test", PyTuple_GET_SIZE(args), 0, PY_SSIZE_T_MAX)) { + goto exit; + } + __clinic_args = PyTuple_GetSlice(0, -1); + return_value = Test___init___impl((TestObj *)self, __clinic_args); + +exit: + Py_XDECREF(__clinic_args); + return return_value; +} + +static int +Test___init___impl(TestObj *self, PyObject *args) +/*[clinic end generated code: output=0ed1009fe0dcf98d input=96c3ddc0cd38fc0c]*/ + + +/*[clinic input] + at classmethod +Test.__new__ + *args: object + / +Varargs new method. For example, nargs is translated to PyTuple_GET_SIZE. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test__doc__, +"Test(*args)\n" +"--\n" +"\n" +"Varargs new method. For example, nargs is translated to PyTuple_GET_SIZE."); + +static PyObject * +Test_impl(PyTypeObject *type, PyObject *args); + +static PyObject * +Test(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyTypeObject *base_tp = TestType; + PyObject *__clinic_args = NULL; + + if ((type == base_tp || type->tp_init == base_tp->tp_init) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + if (!_PyArg_CheckPositional("Test", PyTuple_GET_SIZE(args), 0, PY_SSIZE_T_MAX)) { + goto exit; + } + __clinic_args = PyTuple_GetSlice(0, -1); + return_value = Test_impl(type, __clinic_args); + +exit: + Py_XDECREF(__clinic_args); + return return_value; +} + +static PyObject * +Test_impl(PyTypeObject *type, PyObject *args) +/*[clinic end generated code: output=8b219f6633e2a2e9 input=26a672e2e9750120]*/ + + +/*[clinic input] +Test.__init__ + a: object +Init method with positional or keyword arguments. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test___init____doc__, +"Test(a)\n" +"--\n" +"\n" +"Init method with positional or keyword arguments."); + +static int +Test___init___impl(TestObj *self, PyObject *a); + +static int +Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(a), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"a", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "Test", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + PyObject * const *fastargs; + Py_ssize_t nargs = PyTuple_GET_SIZE(args); + PyObject *a; + + fastargs = _PyArg_UnpackKeywords(_PyTuple_CAST(args)->ob_item, nargs, kwargs, NULL, &_parser, 1, 1, 0, argsbuf); + if (!fastargs) { + goto exit; + } + a = fastargs[0]; + return_value = Test___init___impl((TestObj *)self, a); + +exit: + return return_value; +} + +static int +Test___init___impl(TestObj *self, PyObject *a) +/*[clinic end generated code: output=0b9ca79638ab3ecb input=a8f9222a6ab35c59]*/ + + +/*[clinic input] + at classmethod +Test.class_method +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_class_method__doc__, +"class_method($type, /)\n" +"--\n" +"\n"); + +#define TEST_CLASS_METHOD_METHODDEF \ + {"class_method", (PyCFunction)Test_class_method, METH_NOARGS|METH_CLASS, Test_class_method__doc__}, + +static PyObject * +Test_class_method_impl(PyTypeObject *type); + +static PyObject * +Test_class_method(PyTypeObject *type, PyObject *Py_UNUSED(ignored)) +{ + return Test_class_method_impl(type); +} + +static PyObject * +Test_class_method_impl(PyTypeObject *type) +/*[clinic end generated code: output=47fb7ecca1abcaaa input=43bc4a0494547b80]*/ + + +/*[clinic input] + at staticmethod +Test.static_method +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_static_method__doc__, +"static_method()\n" +"--\n" +"\n"); + +#define TEST_STATIC_METHOD_METHODDEF \ + {"static_method", (PyCFunction)Test_static_method, METH_NOARGS|METH_STATIC, Test_static_method__doc__}, + +static PyObject * +Test_static_method_impl(); + +static PyObject * +Test_static_method(void *null, PyObject *Py_UNUSED(ignored)) +{ + return Test_static_method_impl(); +} + +static PyObject * +Test_static_method_impl() +/*[clinic end generated code: output=82524a63025cf7ab input=dae892fac55ae72b]*/ + + +/*[clinic input] + at coexist +Test.meth_coexist +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_meth_coexist__doc__, +"meth_coexist($self, /)\n" +"--\n" +"\n"); + +#define TEST_METH_COEXIST_METHODDEF \ + {"meth_coexist", (PyCFunction)Test_meth_coexist, METH_NOARGS|METH_COEXIST, Test_meth_coexist__doc__}, + +static PyObject * +Test_meth_coexist_impl(TestObj *self); + +static PyObject * +Test_meth_coexist(TestObj *self, PyObject *Py_UNUSED(ignored)) +{ + return Test_meth_coexist_impl(self); +} + +static PyObject * +Test_meth_coexist_impl(TestObj *self) +/*[clinic end generated code: output=808a293d0cd27439 input=2a1d75b5e6fec6dd]*/ diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 685ba58642a5a..975840333e590 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1013,6 +1013,43 @@ def test_defining_class_param_cannot_be_optional(self): out = self.parse_function_should_fail(block) self.assertEqual(out, expected_error_msg) + def test_slot_methods_cannot_access_defining_class(self): + block = """ + module foo + class Foo "" "" + Foo.__init__ + cls: defining_class + a: object + """ + msg = "Slot methods cannot access their defining class." + with self.assertRaisesRegex(ValueError, msg): + self.parse_function(block) + + def test_new_must_be_a_class_method(self): + expected_error_msg = ( + "Error on line 0:\n" + "__new__ must be a class method!\n" + ) + out = self.parse_function_should_fail(""" + module foo + class Foo "" "" + Foo.__new__ + """) + self.assertEqual(out, expected_error_msg) + + def test_init_must_be_a_normal_method(self): + expected_error_msg = ( + "Error on line 0:\n" + "__init__ must be a normal method, not a class or static method!\n" + ) + out = self.parse_function_should_fail(""" + module foo + class Foo "" "" + @classmethod + Foo.__init__ + """) + self.assertEqual(out, expected_error_msg) + def test_unused_param(self): block = self.parse(""" module foo From webhook-mailer at python.org Thu Jul 13 18:54:08 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 13 Jul 2023 22:54:08 -0000 Subject: [Python-checkins] gh-104683: Argument clinic: use an enum to describe the different kinds of functions (#106721) Message-ID: https://github.com/python/cpython/commit/128a6c1d889639012c83c122b82c9cdf0b62d587 commit: 128a6c1d889639012c83c122b82c9cdf0b62d587 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-13T22:54:05Z summary: gh-104683: Argument clinic: use an enum to describe the different kinds of functions (#106721) Argument clinic: use an enum to describe the different kinds of functions files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 8a75aba1e4de9..2dce33690993d 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -807,7 +807,7 @@ def output_templates(self, f): default_return_converter = (not f.return_converter or f.return_converter.type == 'PyObject *') - new_or_init = f.kind in (METHOD_NEW, METHOD_INIT) + new_or_init = f.kind.new_or_init vararg = NO_VARARG pos_only = min_pos = max_pos = min_kw_only = pseudo_args = 0 @@ -1250,7 +1250,7 @@ def parser_body( if new_or_init: methoddef_define = '' - if f.kind == METHOD_NEW: + if f.kind is METHOD_NEW: parser_prototype = parser_prototype_keyword else: return_value_declaration = "int return_value = -1;" @@ -1475,7 +1475,7 @@ def render_function( last_group = 0 first_optional = len(selfless) positional = selfless and selfless[-1].is_positional_only() - new_or_init = f.kind in (METHOD_NEW, METHOD_INIT) + new_or_init = f.kind.new_or_init has_option_groups = False # offset i by -1 because first_optional needs to ignore self @@ -2441,9 +2441,28 @@ def __repr__(self) -> str: """.strip().split()) -INVALID, CALLABLE, STATIC_METHOD, CLASS_METHOD, METHOD_INIT, METHOD_NEW = """ -INVALID, CALLABLE, STATIC_METHOD, CLASS_METHOD, METHOD_INIT, METHOD_NEW -""".replace(",", "").strip().split() +class FunctionKind(enum.Enum): + INVALID = enum.auto() + CALLABLE = enum.auto() + STATIC_METHOD = enum.auto() + CLASS_METHOD = enum.auto() + METHOD_INIT = enum.auto() + METHOD_NEW = enum.auto() + + @functools.cached_property + def new_or_init(self) -> bool: + return self in {FunctionKind.METHOD_INIT, FunctionKind.METHOD_NEW} + + def __repr__(self) -> str: + return f"" + + +INVALID: Final = FunctionKind.INVALID +CALLABLE: Final = FunctionKind.CALLABLE +STATIC_METHOD: Final = FunctionKind.STATIC_METHOD +CLASS_METHOD: Final = FunctionKind.CLASS_METHOD +METHOD_INIT: Final = FunctionKind.METHOD_INIT +METHOD_NEW: Final = FunctionKind.METHOD_NEW ParamDict = dict[str, "Parameter"] ReturnConverterType = Callable[..., "CReturnConverter"] @@ -2471,7 +2490,7 @@ class Function: return_converter: CReturnConverter return_annotation: object = inspect.Signature.empty docstring: str = '' - kind: str = CALLABLE + kind: FunctionKind = CALLABLE coexist: bool = False # docstring_only means "don't generate a machine-readable # signature, just a normal docstring". it's True for @@ -2497,15 +2516,16 @@ def render_parameters(self) -> list[Parameter]: @property def methoddef_flags(self) -> str | None: - if self.kind in (METHOD_INIT, METHOD_NEW): + if self.kind.new_or_init: return None flags = [] - if self.kind == CLASS_METHOD: - flags.append('METH_CLASS') - elif self.kind == STATIC_METHOD: - flags.append('METH_STATIC') - else: - assert self.kind == CALLABLE, "unknown kind: " + repr(self.kind) + match self.kind: + case FunctionKind.CLASS_METHOD: + flags.append('METH_CLASS') + case FunctionKind.STATIC_METHOD: + flags.append('METH_STATIC') + case _ as kind: + assert kind is FunctionKind.CALLABLE, f"unknown kind: {kind!r}" if self.coexist: flags.append('METH_COEXIST') return '|'.join(flags) @@ -3888,7 +3908,7 @@ def correct_name_for_self( if f.cls: return "PyObject *", "self" return "PyObject *", "module" - if f.kind == STATIC_METHOD: + if f.kind is STATIC_METHOD: return "void *", "null" if f.kind in (CLASS_METHOD, METHOD_NEW): return "PyTypeObject *", "type" @@ -3921,9 +3941,8 @@ def pre_render(self): self.type = self.specified_type or self.type or default_type kind = self.function.kind - new_or_init = kind in (METHOD_NEW, METHOD_INIT) - if (kind == STATIC_METHOD) or new_or_init: + if kind is STATIC_METHOD or kind.new_or_init: self.show_in_signature = False # tp_new (METHOD_NEW) functions are of type newfunc: @@ -3973,7 +3992,7 @@ def render(self, parameter, data): parameter is a clinic.Parameter instance. data is a CRenderData instance. """ - if self.function.kind == STATIC_METHOD: + if self.function.kind is STATIC_METHOD: return self._render_self(parameter, data) @@ -3992,8 +4011,8 @@ def set_template_dict(self, template_dict): kind = self.function.kind cls = self.function.cls - if ((kind in (METHOD_NEW, METHOD_INIT)) and cls and cls.typedef): - if kind == METHOD_NEW: + if kind.new_or_init and cls and cls.typedef: + if kind is METHOD_NEW: type_check = ( '({0} == base_tp || {0}->tp_init == base_tp->tp_init)' ).format(self.name) @@ -4337,7 +4356,7 @@ class DSLParser: parameter_state: int seen_positional_with_default: bool indent: IndentStack - kind: str + kind: FunctionKind coexist: bool parameter_continuation: str preserve_output: bool @@ -4626,7 +4645,7 @@ def state_modulename_name(self, line: str | None) -> None: function_name = fields.pop() module, cls = self.clinic._module_and_class(fields) - if not (existing_function.kind == self.kind and existing_function.coexist == self.coexist): + if not (existing_function.kind is self.kind and existing_function.coexist == self.coexist): fail("'kind' of function and cloned function don't match! (@classmethod/@staticmethod/@coexist)") function = existing_function.copy( name=function_name, full_name=full_name, module=module, @@ -4679,11 +4698,11 @@ def state_modulename_name(self, line: str | None) -> None: fail(f"{fields[-1]} is a special method and cannot be converted to Argument Clinic! (Yet.)") if fields[-1] == '__new__': - if (self.kind != CLASS_METHOD) or (not cls): + if (self.kind is not CLASS_METHOD) or (not cls): fail("__new__ must be a class method!") self.kind = METHOD_NEW elif fields[-1] == '__init__': - if (self.kind != CALLABLE) or (not cls): + if (self.kind is not CALLABLE) or (not cls): fail("__init__ must be a normal method, not a class or static method!") self.kind = METHOD_INIT if not return_converter: @@ -5203,7 +5222,7 @@ def state_function_docstring(self, line): def format_docstring(self): f = self.function - new_or_init = f.kind in (METHOD_NEW, METHOD_INIT) + new_or_init = f.kind.new_or_init if new_or_init and not f.docstring: # don't render a docstring at all, no signature, nothing. return f.docstring From webhook-mailer at python.org Thu Jul 13 19:13:47 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Thu, 13 Jul 2023 23:13:47 -0000 Subject: [Python-checkins] [3.11] gh-106368: Increase Argument Clinic test coverage (#106728) (#106731) Message-ID: https://github.com/python/cpython/commit/d22224a401244de8eb424a54783d282a2533f7dd commit: d22224a401244de8eb424a54783d282a2533f7dd branch: 3.11 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-14T01:13:43+02:00 summary: [3.11] gh-106368: Increase Argument Clinic test coverage (#106728) (#106731) - improve output_parameter() coverage - improve coverage for Function.kind (cherry picked from commit ec45c513d389510930a62631a21a1dbb3f3aabb7) Co-authored-by: Erlend E. Aasland * Fix merge files: M Lib/test/clinic.test.c M Lib/test/test_clinic.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index 0abadbe7f3474..1eb5a3e7cce0d 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -3,6 +3,10 @@ output preset block [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=3c81ac2402d06a8b]*/ +/*[clinic input] +class Test "TestObj *" "TestType" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=fc7e50384d12b83f]*/ /*[clinic input] test_object_converter @@ -61,6 +65,58 @@ test_object_converter_impl(PyObject *module, PyObject *a, PyObject *b, /*[clinic end generated code: output=886f4f9b598726b6 input=005e6a8a711a869b]*/ +/*[clinic input] +cloned = test_object_converter +Check the clone feature. +[clinic start generated code]*/ + +PyDoc_STRVAR(cloned__doc__, +"cloned($module, a, b, c, d, /)\n" +"--\n" +"\n" +"Check the clone feature."); + +#define CLONED_METHODDEF \ + {"cloned", _PyCFunction_CAST(cloned), METH_FASTCALL, cloned__doc__}, + +static PyObject * +cloned_impl(PyObject *module, PyObject *a, PyObject *b, PyObject *c, + PyUnicode_Object *d); + +static PyObject * +cloned(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + PyObject *a; + PyObject *b; + PyObject *c; + PyUnicode_Object *d; + + if (!_PyArg_CheckPositional("cloned", nargs, 4, 4)) { + goto exit; + } + a = args[0]; + if (!PyUnicode_FSConverter(args[1], &b)) { + goto exit; + } + if (!PyUnicode_Check(args[2])) { + _PyArg_BadArgument("cloned", "argument 3", "str", args[2]); + goto exit; + } + c = args[2]; + d = (PyUnicode_Object *)args[3]; + return_value = cloned_impl(module, a, b, c, d); + +exit: + return return_value; +} + +static PyObject * +cloned_impl(PyObject *module, PyObject *a, PyObject *b, PyObject *c, + PyUnicode_Object *d) +/*[clinic end generated code: output=026b483e27c38065 input=0543614019d6fcc7]*/ + + /*[clinic input] test_object_converter_one_arg @@ -3556,3 +3612,453 @@ test_paramname_module(PyObject *module, PyObject *const *args, Py_ssize_t nargs, static PyObject * test_paramname_module_impl(PyObject *module, PyObject *mod) /*[clinic end generated code: output=23379a7ffa65c514 input=afefe259667f13ba]*/ + + +/*[clinic input] +Test.cls_with_param + cls: defining_class + / + a: int +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_cls_with_param__doc__, +"cls_with_param($self, /, a)\n" +"--\n" +"\n"); + +#define TEST_CLS_WITH_PARAM_METHODDEF \ + {"cls_with_param", _PyCFunction_CAST(Test_cls_with_param), METH_METHOD|METH_FASTCALL|METH_KEYWORDS, Test_cls_with_param__doc__}, + +static PyObject * +Test_cls_with_param_impl(TestObj *self, PyTypeObject *cls, int a); + +static PyObject * +Test_cls_with_param(TestObj *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"a", NULL}; + static _PyArg_Parser _parser = {NULL, _keywords, "cls_with_param", 0}; + PyObject *argsbuf[1]; + int a; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + a = _PyLong_AsInt(args[0]); + if (a == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = Test_cls_with_param_impl(self, cls, a); + +exit: + return return_value; +} + +static PyObject * +Test_cls_with_param_impl(TestObj *self, PyTypeObject *cls, int a) +/*[clinic end generated code: output=9c06a8cfc495b4d1 input=af158077bd237ef9]*/ + + +/*[clinic input] +Test.__init__ +Empty init method. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test___init____doc__, +"Test()\n" +"--\n" +"\n" +"Empty init method."); + +static int +Test___init___impl(TestObj *self); + +static int +Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + + if ((Py_IS_TYPE(self, TestType) || + Py_TYPE(self)->tp_new == TestType->tp_new) && + !_PyArg_NoPositional("Test", args)) { + goto exit; + } + if ((Py_IS_TYPE(self, TestType) || + Py_TYPE(self)->tp_new == TestType->tp_new) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + return_value = Test___init___impl((TestObj *)self); + +exit: + return return_value; +} + +static int +Test___init___impl(TestObj *self) +/*[clinic end generated code: output=f02b7d23eec3dc47 input=4ea79fee54d0c3ff]*/ + + +/*[clinic input] + at classmethod +Test.__new__ +Empty new method. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test__doc__, +"Test()\n" +"--\n" +"\n" +"Empty new method."); + +static PyObject * +Test_impl(PyTypeObject *type); + +static PyObject * +Test(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + + if ((type == TestType || + type->tp_init == TestType->tp_init) && + !_PyArg_NoPositional("Test", args)) { + goto exit; + } + if ((type == TestType || + type->tp_init == TestType->tp_init) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + return_value = Test_impl(type); + +exit: + return return_value; +} + +static PyObject * +Test_impl(PyTypeObject *type) +/*[clinic end generated code: output=3a8a564e799cf5ce input=6fe98a19f097907f]*/ + + +/*[clinic input] +Test.cls_no_params + cls: defining_class + / +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_cls_no_params__doc__, +"cls_no_params($self, /)\n" +"--\n" +"\n"); + +#define TEST_CLS_NO_PARAMS_METHODDEF \ + {"cls_no_params", _PyCFunction_CAST(Test_cls_no_params), METH_METHOD|METH_FASTCALL|METH_KEYWORDS, Test_cls_no_params__doc__}, + +static PyObject * +Test_cls_no_params_impl(TestObj *self, PyTypeObject *cls); + +static PyObject * +Test_cls_no_params(TestObj *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + if (nargs) { + PyErr_SetString(PyExc_TypeError, "cls_no_params() takes no arguments"); + return NULL; + } + return Test_cls_no_params_impl(self, cls); +} + +static PyObject * +Test_cls_no_params_impl(TestObj *self, PyTypeObject *cls) +/*[clinic end generated code: output=cc8845f22cff3dcb input=e7e2e4e344e96a11]*/ + + +/*[clinic input] +Test.metho_not_default_return_converter -> int + a: object + / +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_metho_not_default_return_converter__doc__, +"metho_not_default_return_converter($self, a, /)\n" +"--\n" +"\n"); + +#define TEST_METHO_NOT_DEFAULT_RETURN_CONVERTER_METHODDEF \ + {"metho_not_default_return_converter", (PyCFunction)Test_metho_not_default_return_converter, METH_O, Test_metho_not_default_return_converter__doc__}, + +static int +Test_metho_not_default_return_converter_impl(TestObj *self, PyObject *a); + +static PyObject * +Test_metho_not_default_return_converter(TestObj *self, PyObject *a) +{ + PyObject *return_value = NULL; + int _return_value; + + _return_value = Test_metho_not_default_return_converter_impl(self, a); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyLong_FromLong((long)_return_value); + +exit: + return return_value; +} + +static int +Test_metho_not_default_return_converter_impl(TestObj *self, PyObject *a) +/*[clinic end generated code: output=3350de11bd538007 input=428657129b521177]*/ + + +/*[clinic input] +Test.an_metho_arg_named_arg + arg: int + Name should be mangled to 'arg_' in generated output. + / +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_an_metho_arg_named_arg__doc__, +"an_metho_arg_named_arg($self, arg, /)\n" +"--\n" +"\n" +"\n" +"\n" +" arg\n" +" Name should be mangled to \'arg_\' in generated output."); + +#define TEST_AN_METHO_ARG_NAMED_ARG_METHODDEF \ + {"an_metho_arg_named_arg", (PyCFunction)Test_an_metho_arg_named_arg, METH_O, Test_an_metho_arg_named_arg__doc__}, + +static PyObject * +Test_an_metho_arg_named_arg_impl(TestObj *self, int arg); + +static PyObject * +Test_an_metho_arg_named_arg(TestObj *self, PyObject *arg_) +{ + PyObject *return_value = NULL; + int arg; + + arg = _PyLong_AsInt(arg_); + if (arg == -1 && PyErr_Occurred()) { + goto exit; + } + return_value = Test_an_metho_arg_named_arg_impl(self, arg); + +exit: + return return_value; +} + +static PyObject * +Test_an_metho_arg_named_arg_impl(TestObj *self, int arg) +/*[clinic end generated code: output=7d590626642194ae input=2a53a57cf5624f95]*/ + + +/*[clinic input] +Test.__init__ + *args: object + / +Varargs init method. For example, nargs is translated to PyTuple_GET_SIZE. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test___init____doc__, +"Test(*args)\n" +"--\n" +"\n" +"Varargs init method. For example, nargs is translated to PyTuple_GET_SIZE."); + +static int +Test___init___impl(TestObj *self, PyObject *args); + +static int +Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + PyObject *__clinic_args = NULL; + + if ((Py_IS_TYPE(self, TestType) || + Py_TYPE(self)->tp_new == TestType->tp_new) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + if (!_PyArg_CheckPositional("Test", PyTuple_GET_SIZE(args), 0, PY_SSIZE_T_MAX)) { + goto exit; + } + __clinic_args = PyTuple_GetSlice(0, -1); + return_value = Test___init___impl((TestObj *)self, __clinic_args); + +exit: + Py_XDECREF(__clinic_args); + return return_value; +} + +static int +Test___init___impl(TestObj *self, PyObject *args) +/*[clinic end generated code: output=126ad63fc2e5139e input=96c3ddc0cd38fc0c]*/ + + +/*[clinic input] + at classmethod +Test.__new__ + *args: object + / +Varargs new method. For example, nargs is translated to PyTuple_GET_SIZE. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test__doc__, +"Test(*args)\n" +"--\n" +"\n" +"Varargs new method. For example, nargs is translated to PyTuple_GET_SIZE."); + +static PyObject * +Test_impl(PyTypeObject *type, PyObject *args); + +static PyObject * +Test(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + PyObject *__clinic_args = NULL; + + if ((type == TestType || + type->tp_init == TestType->tp_init) && + !_PyArg_NoKeywords("Test", kwargs)) { + goto exit; + } + if (!_PyArg_CheckPositional("Test", PyTuple_GET_SIZE(args), 0, PY_SSIZE_T_MAX)) { + goto exit; + } + __clinic_args = PyTuple_GetSlice(0, -1); + return_value = Test_impl(type, __clinic_args); + +exit: + Py_XDECREF(__clinic_args); + return return_value; +} + +static PyObject * +Test_impl(PyTypeObject *type, PyObject *args) +/*[clinic end generated code: output=4f01d446cfe4aeb9 input=26a672e2e9750120]*/ + + +/*[clinic input] +Test.__init__ + a: object +Init method with positional or keyword arguments. +[clinic start generated code]*/ + +PyDoc_STRVAR(Test___init____doc__, +"Test(a)\n" +"--\n" +"\n" +"Init method with positional or keyword arguments."); + +static int +Test___init___impl(TestObj *self, PyObject *a); + +static int +Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + static const char * const _keywords[] = {"a", NULL}; + static _PyArg_Parser _parser = {NULL, _keywords, "Test", 0}; + PyObject *argsbuf[1]; + PyObject * const *fastargs; + Py_ssize_t nargs = PyTuple_GET_SIZE(args); + PyObject *a; + + fastargs = _PyArg_UnpackKeywords(_PyTuple_CAST(args)->ob_item, nargs, kwargs, NULL, &_parser, 1, 1, 0, argsbuf); + if (!fastargs) { + goto exit; + } + a = fastargs[0]; + return_value = Test___init___impl((TestObj *)self, a); + +exit: + return return_value; +} + +static int +Test___init___impl(TestObj *self, PyObject *a) +/*[clinic end generated code: output=5afcf1a525211a09 input=a8f9222a6ab35c59]*/ + + +/*[clinic input] + at classmethod +Test.class_method +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_class_method__doc__, +"class_method($type, /)\n" +"--\n" +"\n"); + +#define TEST_CLASS_METHOD_METHODDEF \ + {"class_method", (PyCFunction)Test_class_method, METH_NOARGS|METH_CLASS, Test_class_method__doc__}, + +static PyObject * +Test_class_method_impl(PyTypeObject *type); + +static PyObject * +Test_class_method(PyTypeObject *type, PyObject *Py_UNUSED(ignored)) +{ + return Test_class_method_impl(type); +} + +static PyObject * +Test_class_method_impl(PyTypeObject *type) +/*[clinic end generated code: output=47fb7ecca1abcaaa input=43bc4a0494547b80]*/ + + +/*[clinic input] + at staticmethod +Test.static_method +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_static_method__doc__, +"static_method()\n" +"--\n" +"\n"); + +#define TEST_STATIC_METHOD_METHODDEF \ + {"static_method", (PyCFunction)Test_static_method, METH_NOARGS|METH_STATIC, Test_static_method__doc__}, + +static PyObject * +Test_static_method_impl(); + +static PyObject * +Test_static_method(void *null, PyObject *Py_UNUSED(ignored)) +{ + return Test_static_method_impl(); +} + +static PyObject * +Test_static_method_impl() +/*[clinic end generated code: output=82524a63025cf7ab input=dae892fac55ae72b]*/ + + +/*[clinic input] + at coexist +Test.meth_coexist +[clinic start generated code]*/ + +PyDoc_STRVAR(Test_meth_coexist__doc__, +"meth_coexist($self, /)\n" +"--\n" +"\n"); + +#define TEST_METH_COEXIST_METHODDEF \ + {"meth_coexist", (PyCFunction)Test_meth_coexist, METH_NOARGS|METH_COEXIST, Test_meth_coexist__doc__}, + +static PyObject * +Test_meth_coexist_impl(TestObj *self); + +static PyObject * +Test_meth_coexist(TestObj *self, PyObject *Py_UNUSED(ignored)) +{ + return Test_meth_coexist_impl(self); +} + +static PyObject * +Test_meth_coexist_impl(TestObj *self) +/*[clinic end generated code: output=808a293d0cd27439 input=2a1d75b5e6fec6dd]*/ diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index f2e0c5e03f8b9..fa65a41ec30d0 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -997,6 +997,43 @@ def test_defining_class_param_cannot_be_optional(self): out = self.parse_function_should_fail(block) self.assertEqual(out, expected_error_msg) + def test_slot_methods_cannot_access_defining_class(self): + block = """ + module foo + class Foo "" "" + Foo.__init__ + cls: defining_class + a: object + """ + msg = "Slot methods cannot access their defining class." + with self.assertRaisesRegex(ValueError, msg): + self.parse_function(block) + + def test_new_must_be_a_class_method(self): + expected_error_msg = ( + "Error on line 0:\n" + "__new__ must be a class method!\n" + ) + out = self.parse_function_should_fail(""" + module foo + class Foo "" "" + Foo.__new__ + """) + self.assertEqual(out, expected_error_msg) + + def test_init_must_be_a_normal_method(self): + expected_error_msg = ( + "Error on line 0:\n" + "__init__ must be a normal method, not a class or static method!\n" + ) + out = self.parse_function_should_fail(""" + module foo + class Foo "" "" + @classmethod + Foo.__init__ + """) + self.assertEqual(out, expected_error_msg) + def parse(self, text): c = FakeClinic() parser = DSLParser(c) From webhook-mailer at python.org Thu Jul 13 20:27:39 2023 From: webhook-mailer at python.org (gvanrossum) Date: Fri, 14 Jul 2023 00:27:39 -0000 Subject: [Python-checkins] gh-106529: Split FOR_ITER_{LIST, TUPLE} into uops (#106696) Message-ID: https://github.com/python/cpython/commit/025995feadaeebeef5d808f2564f0fd65b704ea5 commit: 025995feadaeebeef5d808f2564f0fd65b704ea5 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-13T17:27:35-07:00 summary: gh-106529: Split FOR_ITER_{LIST,TUPLE} into uops (#106696) Also rename `_ITER_EXHAUSTED_XXX` to `_IS_ITER_EXHAUSTED_XXX` to make it clear this is a test. files: M Include/internal/pycore_opcode_metadata.h M Lib/test/test_capi/test_misc.py M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/optimizer.c diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index c88640777e3fb..e94732b64384b 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -39,12 +39,18 @@ #define _GUARD_TYPE_VERSION 317 #define _CHECK_MANAGED_OBJECT_HAS_VALUES 318 #define IS_NONE 319 -#define _ITER_CHECK_RANGE 320 -#define _ITER_EXHAUSTED_RANGE 321 -#define _ITER_NEXT_RANGE 322 -#define _POP_JUMP_IF_FALSE 323 -#define _POP_JUMP_IF_TRUE 324 -#define JUMP_TO_TOP 325 +#define _ITER_CHECK_LIST 320 +#define _IS_ITER_EXHAUSTED_LIST 321 +#define _ITER_NEXT_LIST 322 +#define _ITER_CHECK_TUPLE 323 +#define _IS_ITER_EXHAUSTED_TUPLE 324 +#define _ITER_NEXT_TUPLE 325 +#define _ITER_CHECK_RANGE 326 +#define _IS_ITER_EXHAUSTED_RANGE 327 +#define _ITER_NEXT_RANGE 328 +#define _POP_JUMP_IF_FALSE 329 +#define _POP_JUMP_IF_TRUE 330 +#define JUMP_TO_TOP 331 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -1323,8 +1329,14 @@ const char * const _PyOpcode_uop_name[512] = { [_GUARD_TYPE_VERSION] = "_GUARD_TYPE_VERSION", [_CHECK_MANAGED_OBJECT_HAS_VALUES] = "_CHECK_MANAGED_OBJECT_HAS_VALUES", [IS_NONE] = "IS_NONE", + [_ITER_CHECK_LIST] = "_ITER_CHECK_LIST", + [_IS_ITER_EXHAUSTED_LIST] = "_IS_ITER_EXHAUSTED_LIST", + [_ITER_NEXT_LIST] = "_ITER_NEXT_LIST", + [_ITER_CHECK_TUPLE] = "_ITER_CHECK_TUPLE", + [_IS_ITER_EXHAUSTED_TUPLE] = "_IS_ITER_EXHAUSTED_TUPLE", + [_ITER_NEXT_TUPLE] = "_ITER_NEXT_TUPLE", [_ITER_CHECK_RANGE] = "_ITER_CHECK_RANGE", - [_ITER_EXHAUSTED_RANGE] = "_ITER_EXHAUSTED_RANGE", + [_IS_ITER_EXHAUSTED_RANGE] = "_IS_ITER_EXHAUSTED_RANGE", [_ITER_NEXT_RANGE] = "_ITER_NEXT_RANGE", [_POP_JUMP_IF_FALSE] = "_POP_JUMP_IF_FALSE", [_POP_JUMP_IF_TRUE] = "_POP_JUMP_IF_TRUE", diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index abdf7ed897635..43c04463236a2 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2590,7 +2590,6 @@ def testfunc(n): for i in range(n): total += i return total - # import dis; dis.dis(testfunc) opt = _testinternalcapi.get_uop_optimizer() with temporary_optimizer(opt): @@ -2602,7 +2601,51 @@ def testfunc(n): # for i, (opname, oparg) in enumerate(ex): # print(f"{i:4d}: {opname:<20s} {oparg:3d}") uops = {opname for opname, _ in ex} - self.assertIn("_ITER_EXHAUSTED_RANGE", uops) + self.assertIn("_IS_ITER_EXHAUSTED_RANGE", uops) + # Verification that the jump goes past END_FOR + # is done by manual inspection of the output + + def test_for_iter_list(self): + def testfunc(a): + total = 0 + for i in a: + total += i + return total + + opt = _testinternalcapi.get_uop_optimizer() + with temporary_optimizer(opt): + a = list(range(10)) + total = testfunc(a) + self.assertEqual(total, 45) + + ex = get_first_executor(testfunc) + self.assertIsNotNone(ex) + # for i, (opname, oparg) in enumerate(ex): + # print(f"{i:4d}: {opname:<20s} {oparg:3d}") + uops = {opname for opname, _ in ex} + self.assertIn("_IS_ITER_EXHAUSTED_LIST", uops) + # Verification that the jump goes past END_FOR + # is done by manual inspection of the output + + def test_for_iter_tuple(self): + def testfunc(a): + total = 0 + for i in a: + total += i + return total + + opt = _testinternalcapi.get_uop_optimizer() + with temporary_optimizer(opt): + a = tuple(range(10)) + total = testfunc(a) + self.assertEqual(total, 45) + + ex = get_first_executor(testfunc) + self.assertIsNotNone(ex) + # for i, (opname, oparg) in enumerate(ex): + # print(f"{i:4d}: {opname:<20s} {oparg:3d}") + uops = {opname for opname, _ in ex} + self.assertIn("_IS_ITER_EXHAUSTED_TUPLE", uops) # Verification that the jump goes past END_FOR # is done by manual inspection of the output diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 1fe9970e53cdf..15b48ae9d8267 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -17,6 +17,7 @@ #include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_moduleobject.h" // PyModuleObject #include "pycore_opcode.h" // EXTRA_CASES +#include "pycore_opcode_metadata.h" // uop names #include "pycore_opcode_utils.h" // MAKE_FUNCTION_* #include "pycore_pyerrors.h" // _PyErr_GetRaisedException() #include "pycore_pystate.h" // _PyInterpreterState_GET() @@ -55,13 +56,14 @@ static PyObject *value, *value1, *value2, *left, *right, *res, *sum, *prod, *sub; static PyObject *container, *start, *stop, *v, *lhs, *rhs, *res2; static PyObject *list, *tuple, *dict, *owner, *set, *str, *tup, *map, *keys; -static PyObject *exit_func, *lasti, *val, *retval, *obj, *iter; +static PyObject *exit_func, *lasti, *val, *retval, *obj, *iter, *exhausted; static PyObject *aiter, *awaitable, *iterable, *w, *exc_value, *bc, *locals; static PyObject *orig, *excs, *update, *b, *fromlist, *level, *from; static PyObject **pieces, **values; static size_t jump; // Dummy variables for cache effects static uint16_t invert, counter, index, hint; +#define unused 0 // Used in a macro def, can't be static static uint32_t type_version; static PyObject * @@ -2418,52 +2420,108 @@ dummy_func( INSTRUMENTED_JUMP(here, target, PY_MONITORING_EVENT_BRANCH); } - inst(FOR_ITER_LIST, (unused/1, iter -- iter, next)) { + op(_ITER_CHECK_LIST, (iter -- iter)) { DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); + } + + op(_ITER_JUMP_LIST, (iter -- iter)) { _PyListIterObject *it = (_PyListIterObject *)iter; + assert(Py_TYPE(iter) == &PyListIter_Type); STAT_INC(FOR_ITER, hit); PyListObject *seq = it->it_seq; - if (seq) { - if (it->it_index < PyList_GET_SIZE(seq)) { - next = Py_NewRef(PyList_GET_ITEM(seq, it->it_index++)); - goto end_for_iter_list; // End of this instruction + if (seq == NULL || it->it_index >= PyList_GET_SIZE(seq)) { + if (seq != NULL) { + it->it_seq = NULL; + Py_DECREF(seq); } - it->it_seq = NULL; - Py_DECREF(seq); + Py_DECREF(iter); + STACK_SHRINK(1); + SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); + /* Jump forward oparg, then skip following END_FOR instruction */ + JUMPBY(oparg + 1); + DISPATCH(); } - Py_DECREF(iter); - STACK_SHRINK(1); - SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); - /* Jump forward oparg, then skip following END_FOR instruction */ - JUMPBY(oparg + 1); - DISPATCH(); - end_for_iter_list: - // Common case: no jump, leave it to the code generator } - inst(FOR_ITER_TUPLE, (unused/1, iter -- iter, next)) { + // Only used by Tier 2 + op(_IS_ITER_EXHAUSTED_LIST, (iter -- iter, exhausted)) { + _PyListIterObject *it = (_PyListIterObject *)iter; + assert(Py_TYPE(iter) == &PyListIter_Type); + PyListObject *seq = it->it_seq; + if (seq == NULL || it->it_index >= PyList_GET_SIZE(seq)) { + exhausted = Py_True; + } + else { + exhausted = Py_False; + } + } + + op(_ITER_NEXT_LIST, (iter -- iter, next)) { + _PyListIterObject *it = (_PyListIterObject *)iter; + assert(Py_TYPE(iter) == &PyListIter_Type); + PyListObject *seq = it->it_seq; + assert(seq); + assert(it->it_index < PyList_GET_SIZE(seq)); + next = Py_NewRef(PyList_GET_ITEM(seq, it->it_index++)); + } + + macro(FOR_ITER_LIST) = + unused/1 + // Skip over the counter + _ITER_CHECK_LIST + + _ITER_JUMP_LIST + + _ITER_NEXT_LIST; + + op(_ITER_CHECK_TUPLE, (iter -- iter)) { + DEOPT_IF(Py_TYPE(iter) != &PyTupleIter_Type, FOR_ITER); + } + + op(_ITER_JUMP_TUPLE, (iter -- iter)) { _PyTupleIterObject *it = (_PyTupleIterObject *)iter; - DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); + assert(Py_TYPE(iter) == &PyTupleIter_Type); STAT_INC(FOR_ITER, hit); PyTupleObject *seq = it->it_seq; - if (seq) { - if (it->it_index < PyTuple_GET_SIZE(seq)) { - next = Py_NewRef(PyTuple_GET_ITEM(seq, it->it_index++)); - goto end_for_iter_tuple; // End of this instruction + if (seq == NULL || it->it_index >= PyTuple_GET_SIZE(seq)) { + if (seq != NULL) { + it->it_seq = NULL; + Py_DECREF(seq); } - it->it_seq = NULL; - Py_DECREF(seq); + Py_DECREF(iter); + STACK_SHRINK(1); + SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); + /* Jump forward oparg, then skip following END_FOR instruction */ + JUMPBY(oparg + 1); + DISPATCH(); } - Py_DECREF(iter); - STACK_SHRINK(1); - SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); - /* Jump forward oparg, then skip following END_FOR instruction */ - JUMPBY(oparg + 1); - DISPATCH(); - end_for_iter_tuple: - // Common case: no jump, leave it to the code generator } + // Only used by Tier 2 + op(_IS_ITER_EXHAUSTED_TUPLE, (iter -- iter, exhausted)) { + _PyTupleIterObject *it = (_PyTupleIterObject *)iter; + assert(Py_TYPE(iter) == &PyTupleIter_Type); + PyTupleObject *seq = it->it_seq; + if (seq == NULL || it->it_index >= PyTuple_GET_SIZE(seq)) { + exhausted = Py_True; + } + else { + exhausted = Py_False; + } + } + + op(_ITER_NEXT_TUPLE, (iter -- iter, next)) { + _PyTupleIterObject *it = (_PyTupleIterObject *)iter; + assert(Py_TYPE(iter) == &PyTupleIter_Type); + PyTupleObject *seq = it->it_seq; + assert(seq); + assert(it->it_index < PyTuple_GET_SIZE(seq)); + next = Py_NewRef(PyTuple_GET_ITEM(seq, it->it_index++)); + } + + macro(FOR_ITER_TUPLE) = + unused/1 + // Skip over the counter + _ITER_CHECK_TUPLE + + _ITER_JUMP_TUPLE + + _ITER_NEXT_TUPLE; + op(_ITER_CHECK_RANGE, (iter -- iter)) { _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); @@ -2484,7 +2542,7 @@ dummy_func( } // Only used by Tier 2 - op(_ITER_EXHAUSTED_RANGE, (iter -- iter, exhausted)) { + op(_IS_ITER_EXHAUSTED_RANGE, (iter -- iter, exhausted)) { _PyRangeIterObject *r = (_PyRangeIterObject *)iter; assert(Py_TYPE(r) == &PyRangeIter_Type); exhausted = r->len <= 0 ? Py_True : Py_False; @@ -2502,7 +2560,10 @@ dummy_func( } macro(FOR_ITER_RANGE) = - unused/1 + _ITER_CHECK_RANGE + _ITER_JUMP_RANGE + _ITER_NEXT_RANGE; + unused/1 + // Skip over the counter + _ITER_CHECK_RANGE + + _ITER_JUMP_RANGE + + _ITER_NEXT_RANGE; inst(FOR_ITER_GEN, (unused/1, iter -- iter, unused)) { DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index ce54755d5d25f..626baece81460 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1738,6 +1738,80 @@ break; } + case _ITER_CHECK_LIST: { + PyObject *iter = stack_pointer[-1]; + DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); + break; + } + + case _IS_ITER_EXHAUSTED_LIST: { + PyObject *iter = stack_pointer[-1]; + PyObject *exhausted; + _PyListIterObject *it = (_PyListIterObject *)iter; + assert(Py_TYPE(iter) == &PyListIter_Type); + PyListObject *seq = it->it_seq; + if (seq == NULL || it->it_index >= PyList_GET_SIZE(seq)) { + exhausted = Py_True; + } + else { + exhausted = Py_False; + } + STACK_GROW(1); + stack_pointer[-1] = exhausted; + break; + } + + case _ITER_NEXT_LIST: { + PyObject *iter = stack_pointer[-1]; + PyObject *next; + _PyListIterObject *it = (_PyListIterObject *)iter; + assert(Py_TYPE(iter) == &PyListIter_Type); + PyListObject *seq = it->it_seq; + assert(seq); + assert(it->it_index < PyList_GET_SIZE(seq)); + next = Py_NewRef(PyList_GET_ITEM(seq, it->it_index++)); + STACK_GROW(1); + stack_pointer[-1] = next; + break; + } + + case _ITER_CHECK_TUPLE: { + PyObject *iter = stack_pointer[-1]; + DEOPT_IF(Py_TYPE(iter) != &PyTupleIter_Type, FOR_ITER); + break; + } + + case _IS_ITER_EXHAUSTED_TUPLE: { + PyObject *iter = stack_pointer[-1]; + PyObject *exhausted; + _PyTupleIterObject *it = (_PyTupleIterObject *)iter; + assert(Py_TYPE(iter) == &PyTupleIter_Type); + PyTupleObject *seq = it->it_seq; + if (seq == NULL || it->it_index >= PyTuple_GET_SIZE(seq)) { + exhausted = Py_True; + } + else { + exhausted = Py_False; + } + STACK_GROW(1); + stack_pointer[-1] = exhausted; + break; + } + + case _ITER_NEXT_TUPLE: { + PyObject *iter = stack_pointer[-1]; + PyObject *next; + _PyTupleIterObject *it = (_PyTupleIterObject *)iter; + assert(Py_TYPE(iter) == &PyTupleIter_Type); + PyTupleObject *seq = it->it_seq; + assert(seq); + assert(it->it_index < PyTuple_GET_SIZE(seq)); + next = Py_NewRef(PyTuple_GET_ITEM(seq, it->it_index++)); + STACK_GROW(1); + stack_pointer[-1] = next; + break; + } + case _ITER_CHECK_RANGE: { PyObject *iter = stack_pointer[-1]; _PyRangeIterObject *r = (_PyRangeIterObject *)iter; @@ -1745,7 +1819,7 @@ break; } - case _ITER_EXHAUSTED_RANGE: { + case _IS_ITER_EXHAUSTED_RANGE: { PyObject *iter = stack_pointer[-1]; PyObject *exhausted; _PyRangeIterObject *r = (_PyRangeIterObject *)iter; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index d43c7386bd6f6..68531dc074769 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3051,60 +3051,96 @@ } TARGET(FOR_ITER_LIST) { - PyObject *iter = stack_pointer[-1]; - PyObject *next; - DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); - _PyListIterObject *it = (_PyListIterObject *)iter; - STAT_INC(FOR_ITER, hit); - PyListObject *seq = it->it_seq; - if (seq) { - if (it->it_index < PyList_GET_SIZE(seq)) { - next = Py_NewRef(PyList_GET_ITEM(seq, it->it_index++)); - goto end_for_iter_list; // End of this instruction + PyObject *_tmp_1; + PyObject *_tmp_2 = stack_pointer[-1]; + { + PyObject *iter = _tmp_2; + DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); + _tmp_2 = iter; + } + { + PyObject *iter = _tmp_2; + _PyListIterObject *it = (_PyListIterObject *)iter; + assert(Py_TYPE(iter) == &PyListIter_Type); + STAT_INC(FOR_ITER, hit); + PyListObject *seq = it->it_seq; + if (seq == NULL || it->it_index >= PyList_GET_SIZE(seq)) { + if (seq != NULL) { + it->it_seq = NULL; + Py_DECREF(seq); + } + Py_DECREF(iter); + STACK_SHRINK(1); + SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); + /* Jump forward oparg, then skip following END_FOR instruction */ + JUMPBY(oparg + 1); + DISPATCH(); } - it->it_seq = NULL; - Py_DECREF(seq); + _tmp_2 = iter; + } + { + PyObject *iter = _tmp_2; + PyObject *next; + _PyListIterObject *it = (_PyListIterObject *)iter; + assert(Py_TYPE(iter) == &PyListIter_Type); + PyListObject *seq = it->it_seq; + assert(seq); + assert(it->it_index < PyList_GET_SIZE(seq)); + next = Py_NewRef(PyList_GET_ITEM(seq, it->it_index++)); + _tmp_2 = iter; + _tmp_1 = next; } - Py_DECREF(iter); - STACK_SHRINK(1); - SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); - /* Jump forward oparg, then skip following END_FOR instruction */ - JUMPBY(oparg + 1); - DISPATCH(); - end_for_iter_list: - // Common case: no jump, leave it to the code generator - STACK_GROW(1); - stack_pointer[-1] = next; next_instr += 1; + STACK_GROW(1); + stack_pointer[-1] = _tmp_1; + stack_pointer[-2] = _tmp_2; DISPATCH(); } TARGET(FOR_ITER_TUPLE) { - PyObject *iter = stack_pointer[-1]; - PyObject *next; - _PyTupleIterObject *it = (_PyTupleIterObject *)iter; - DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); - STAT_INC(FOR_ITER, hit); - PyTupleObject *seq = it->it_seq; - if (seq) { - if (it->it_index < PyTuple_GET_SIZE(seq)) { - next = Py_NewRef(PyTuple_GET_ITEM(seq, it->it_index++)); - goto end_for_iter_tuple; // End of this instruction + PyObject *_tmp_1; + PyObject *_tmp_2 = stack_pointer[-1]; + { + PyObject *iter = _tmp_2; + DEOPT_IF(Py_TYPE(iter) != &PyTupleIter_Type, FOR_ITER); + _tmp_2 = iter; + } + { + PyObject *iter = _tmp_2; + _PyTupleIterObject *it = (_PyTupleIterObject *)iter; + assert(Py_TYPE(iter) == &PyTupleIter_Type); + STAT_INC(FOR_ITER, hit); + PyTupleObject *seq = it->it_seq; + if (seq == NULL || it->it_index >= PyTuple_GET_SIZE(seq)) { + if (seq != NULL) { + it->it_seq = NULL; + Py_DECREF(seq); + } + Py_DECREF(iter); + STACK_SHRINK(1); + SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); + /* Jump forward oparg, then skip following END_FOR instruction */ + JUMPBY(oparg + 1); + DISPATCH(); } - it->it_seq = NULL; - Py_DECREF(seq); + _tmp_2 = iter; + } + { + PyObject *iter = _tmp_2; + PyObject *next; + _PyTupleIterObject *it = (_PyTupleIterObject *)iter; + assert(Py_TYPE(iter) == &PyTupleIter_Type); + PyTupleObject *seq = it->it_seq; + assert(seq); + assert(it->it_index < PyTuple_GET_SIZE(seq)); + next = Py_NewRef(PyTuple_GET_ITEM(seq, it->it_index++)); + _tmp_2 = iter; + _tmp_1 = next; } - Py_DECREF(iter); - STACK_SHRINK(1); - SKIP_OVER(INLINE_CACHE_ENTRIES_FOR_ITER); - /* Jump forward oparg, then skip following END_FOR instruction */ - JUMPBY(oparg + 1); - DISPATCH(); - end_for_iter_tuple: - // Common case: no jump, leave it to the code generator - STACK_GROW(1); - stack_pointer[-1] = next; next_instr += 1; + STACK_GROW(1); + stack_pointer[-1] = _tmp_1; + stack_pointer[-2] = _tmp_2; DISPATCH(); } diff --git a/Python/optimizer.c b/Python/optimizer.c index abd2351f6b78b..289b202f806ae 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -378,6 +378,7 @@ translate_bytecode_to_trace( _Py_CODEUNIT *initial_instr = instr; int trace_length = 0; int max_length = buffer_size; + int reserved = 0; #ifdef Py_DEBUG char *uop_debug = Py_GETENV("PYTHONUOPSDEBUG"); @@ -385,6 +386,9 @@ translate_bytecode_to_trace( if (uop_debug != NULL && *uop_debug >= '0') { lltrace = *uop_debug - '0'; // TODO: Parse an int and all that } +#endif + +#ifdef Py_DEBUG #define DPRINTF(level, ...) \ if (lltrace >= (level)) { fprintf(stderr, __VA_ARGS__); } #else @@ -397,6 +401,8 @@ translate_bytecode_to_trace( uop_name(OPCODE), \ (uint64_t)(OPERAND)); \ assert(trace_length < max_length); \ + assert(reserved > 0); \ + reserved--; \ trace[trace_length].opcode = (OPCODE); \ trace[trace_length].operand = (OPERAND); \ trace_length++; @@ -409,9 +415,23 @@ translate_bytecode_to_trace( (INDEX), \ uop_name(OPCODE), \ (uint64_t)(OPERAND)); \ + assert(reserved > 0); \ + reserved--; \ trace[(INDEX)].opcode = (OPCODE); \ trace[(INDEX)].operand = (OPERAND); +// Reserve space for n uops +#define RESERVE_RAW(n, opname) \ + if (trace_length + (n) > max_length) { \ + DPRINTF(2, "No room for %s (need %d, got %d)\n", \ + (opname), (n), max_length - trace_length); \ + goto done; \ + } \ + reserved = (n); // Keep ADD_TO_TRACE / ADD_TO_STUB honest + +// Reserve space for main+stub uops, plus 2 for SAVE_IP and EXIT_TRACE +#define RESERVE(main, stub) RESERVE_RAW((main) + (stub) + 2, uop_name(opcode)) + DPRINTF(4, "Optimizing %s (%s:%d) at byte offset %ld\n", PyUnicode_AsUTF8(code->co_qualname), @@ -420,16 +440,20 @@ translate_bytecode_to_trace( 2 * INSTR_IP(initial_instr, code)); for (;;) { + RESERVE_RAW(2, "epilogue"); // Always need space for SAVE_IP and EXIT_TRACE ADD_TO_TRACE(SAVE_IP, INSTR_IP(instr, code)); + int opcode = instr->op.code; int oparg = instr->op.arg; int extras = 0; + while (opcode == EXTENDED_ARG) { instr++; extras += 1; opcode = instr->op.code; oparg = (oparg << 8) | instr->op.arg; } + if (opcode == ENTER_EXECUTOR) { _PyExecutorObject *executor = (_PyExecutorObject *)code->co_executors->executors[oparg&255]; @@ -437,17 +461,14 @@ translate_bytecode_to_trace( DPRINTF(2, " * ENTER_EXECUTOR -> %s\n", _PyOpcode_OpName[opcode]); oparg = (oparg & 0xffffff00) | executor->vm_data.oparg; } + switch (opcode) { case POP_JUMP_IF_FALSE: case POP_JUMP_IF_TRUE: { // Assume jump unlikely (TODO: handle jump likely case) - // Reserve 5 entries (1 here, 2 stub, plus SAVE_IP + EXIT_TRACE) - if (trace_length + 5 > max_length) { - DPRINTF(1, "Ran out of space for POP_JUMP_IF_FALSE\n"); - goto done; - } + RESERVE(1, 2); _Py_CODEUNIT *target_instr = instr + 1 + _PyOpcode_Caches[_PyOpcode_Deopt[opcode]] + oparg; max_length -= 2; // Really the start of the stubs @@ -461,9 +482,8 @@ translate_bytecode_to_trace( case JUMP_BACKWARD: { - if (instr + 2 - oparg == initial_instr - && trace_length + 3 <= max_length) - { + if (instr + 2 - oparg == initial_instr) { + RESERVE(1, 0); ADD_TO_TRACE(JUMP_TO_TOP, 0); } else { @@ -474,26 +494,45 @@ translate_bytecode_to_trace( case JUMP_FORWARD: { + RESERVE(0, 0); // This will emit two SAVE_IP instructions; leave it to the optimizer instr += oparg; break; } + case FOR_ITER_LIST: + case FOR_ITER_TUPLE: case FOR_ITER_RANGE: { - // Assume jump unlikely (can a for-loop exit be likely?) - // Reserve 9 entries (4 here, 3 stub, plus SAVE_IP + EXIT_TRACE) - if (trace_length + 9 > max_length) { - DPRINTF(1, "Ran out of space for FOR_ITER_RANGE\n"); - goto done; + RESERVE(4, 3); + int check_op, exhausted_op, next_op; + switch (opcode) { + case FOR_ITER_LIST: + check_op = _ITER_CHECK_LIST; + exhausted_op = _IS_ITER_EXHAUSTED_LIST; + next_op = _ITER_NEXT_LIST; + break; + case FOR_ITER_TUPLE: + check_op = _ITER_CHECK_TUPLE; + exhausted_op = _IS_ITER_EXHAUSTED_TUPLE; + next_op = _ITER_NEXT_TUPLE; + break; + case FOR_ITER_RANGE: + check_op = _ITER_CHECK_RANGE; + exhausted_op = _IS_ITER_EXHAUSTED_RANGE; + next_op = _ITER_NEXT_RANGE; + break; + default: + Py_UNREACHABLE(); } + // Assume jump unlikely (can a for-loop exit be likely?) _Py_CODEUNIT *target_instr = // +1 at the end skips over END_FOR instr + 1 + _PyOpcode_Caches[_PyOpcode_Deopt[opcode]] + oparg + 1; max_length -= 3; // Really the start of the stubs - ADD_TO_TRACE(_ITER_CHECK_RANGE, 0); - ADD_TO_TRACE(_ITER_EXHAUSTED_RANGE, 0); + ADD_TO_TRACE(check_op, 0); + ADD_TO_TRACE(exhausted_op, 0); ADD_TO_TRACE(_POP_JUMP_IF_TRUE, max_length); - ADD_TO_TRACE(_ITER_NEXT_RANGE, 0); + ADD_TO_TRACE(next_op, 0); ADD_TO_STUB(max_length + 0, POP_TOP, 0); ADD_TO_STUB(max_length + 1, SAVE_IP, INSTR_IP(target_instr, code)); @@ -507,10 +546,7 @@ translate_bytecode_to_trace( if (expansion->nuops > 0) { // Reserve space for nuops (+ SAVE_IP + EXIT_TRACE) int nuops = expansion->nuops; - if (trace_length + nuops + 2 > max_length) { - DPRINTF(1, "Ran out of space for %s\n", uop_name(opcode)); - goto done; - } + RESERVE(nuops, 0); for (int i = 0; i < nuops; i++) { uint64_t operand; int offset = expansion->uops[i].offset; @@ -556,12 +592,14 @@ translate_bytecode_to_trace( } DPRINTF(2, "Unsupported opcode %s\n", uop_name(opcode)); goto done; // Break out of loop - } - } + } // End default + + } // End switch (opcode) + instr++; // Add cache size for opcode instr += _PyOpcode_Caches[_PyOpcode_Deopt[opcode]]; - } + } // End for (;;) done: // Skip short traces like SAVE_IP, LOAD_FAST, SAVE_IP, EXIT_TRACE @@ -610,6 +648,9 @@ translate_bytecode_to_trace( } return 0; +#undef RESERVE +#undef RESERVE_RAW +#undef INSTR_IP #undef ADD_TO_TRACE #undef DPRINTF } From webhook-mailer at python.org Fri Jul 14 02:55:52 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 14 Jul 2023 06:55:52 -0000 Subject: [Python-checkins] gh-105626: Change the default return value of `HTTPConnection.get_proxy_response_headers` (#105628) Message-ID: https://github.com/python/cpython/commit/490295d651d04ec3b3eff2a2cda7501191bad78a commit: 490295d651d04ec3b3eff2a2cda7501191bad78a branch: main author: Nikita Sobolev committer: gpshead date: 2023-07-13T23:55:49-07:00 summary: gh-105626: Change the default return value of `HTTPConnection.get_proxy_response_headers` (#105628) files: A Misc/NEWS.d/next/Library/2023-06-10-12-20-17.gh-issue-105626.XyZein.rst M Doc/library/http.client.rst M Lib/http/client.py M Lib/test/test_httplib.py diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst index 45291933d635b..b9ceab699cef6 100644 --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -390,7 +390,7 @@ HTTPConnection Objects Returns a dictionary with the headers of the response received from the proxy server to the CONNECT request. - If the CONNECT request was not sent, the method returns an empty dictionary. + If the CONNECT request was not sent, the method returns ``None``. .. versionadded:: 3.12 diff --git a/Lib/http/client.py b/Lib/http/client.py index 3d98e4eb54bb4..b35b1d6368aae 100644 --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -970,13 +970,12 @@ def get_proxy_response_headers(self): received from the proxy server to the CONNECT request sent to set the tunnel. - If the CONNECT request was not sent, the method returns - an empty dictionary. + If the CONNECT request was not sent, the method returns None. """ return ( _parse_header_lines(self._raw_proxy_headers) if self._raw_proxy_headers is not None - else {} + else None ) def connect(self): diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py index 8955d45fa93dd..fe8105ee2bb3f 100644 --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -2404,6 +2404,19 @@ def test_proxy_response_headers(self): headers = self.conn.get_proxy_response_headers() self.assertIn(expected_header, headers.items()) + def test_no_proxy_response_headers(self): + expected_header = ('X-Dummy', '1') + response_text = ( + 'HTTP/1.0 200 OK\r\n' + '{0}\r\n\r\n'.format(':'.join(expected_header)) + ) + + self.conn._create_connection = self._create_connection(response_text) + + self.conn.request('PUT', '/', '') + headers = self.conn.get_proxy_response_headers() + self.assertIsNone(headers) + def test_tunnel_leak(self): sock = None diff --git a/Misc/NEWS.d/next/Library/2023-06-10-12-20-17.gh-issue-105626.XyZein.rst b/Misc/NEWS.d/next/Library/2023-06-10-12-20-17.gh-issue-105626.XyZein.rst new file mode 100644 index 0000000000000..2a48361fa596c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-10-12-20-17.gh-issue-105626.XyZein.rst @@ -0,0 +1,3 @@ +Change the default return value of +:meth:`http.client.HTTPConnection.get_proxy_response_headers` to be ``None`` +and not ``{}``. From webhook-mailer at python.org Fri Jul 14 03:10:57 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 14 Jul 2023 07:10:57 -0000 Subject: [Python-checkins] gh-105293: Do not call SSL_CTX_set_session_id_context on client side SSL context (#105295) Message-ID: https://github.com/python/cpython/commit/21d98be42289369ccfbdcc38574cb9ab50ce1c02 commit: 21d98be42289369ccfbdcc38574cb9ab50ce1c02 branch: main author: Grant Ramsay committer: gpshead date: 2023-07-14T00:10:54-07:00 summary: gh-105293: Do not call SSL_CTX_set_session_id_context on client side SSL context (#105295) * gh-105293: Do not call SSL_CTX_set_session_id_context on client side SSL context Openssl states this is a "server side only" operation. Calling this on a client side socket can result in unexpected behavior * Add news entry on SSL "set session id context" changes files: A Misc/NEWS.d/next/Library/2023-07-14-14-53-58.gh-issue-105293.kimf_i.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Library/2023-07-14-14-53-58.gh-issue-105293.kimf_i.rst b/Misc/NEWS.d/next/Library/2023-07-14-14-53-58.gh-issue-105293.kimf_i.rst new file mode 100644 index 0000000000000..c263c8524aa96 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-14-14-53-58.gh-issue-105293.kimf_i.rst @@ -0,0 +1,2 @@ +Remove call to ``SSL_CTX_set_session_id_context`` during client side context +creation in the :mod:`ssl` module. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index df1496925f678..571de331e92cd 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -847,6 +847,15 @@ newPySSLSocket(PySSLContext *sslctx, PySocketSockObject *sock, _setSSLError(get_state_ctx(self), NULL, 0, __FILE__, __LINE__); return NULL; } + + if (socket_type == PY_SSL_SERVER) { +#define SID_CTX "Python" + /* Set the session id context (server-side only) */ + SSL_set_session_id_context(self->ssl, (const unsigned char *) SID_CTX, + sizeof(SID_CTX)); +#undef SID_CTX + } + /* bpo43522 and OpenSSL < 1.1.1l: copy hostflags manually */ #if !defined(LIBRESSL_VERSION_NUMBER) && OPENSSL_VERSION < 0x101010cf X509_VERIFY_PARAM *ssl_params = SSL_get0_param(self->ssl); @@ -3186,11 +3195,6 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) usage for no cost at all. */ SSL_CTX_set_mode(self->ctx, SSL_MODE_RELEASE_BUFFERS); -#define SID_CTX "Python" - SSL_CTX_set_session_id_context(self->ctx, (const unsigned char *) SID_CTX, - sizeof(SID_CTX)); -#undef SID_CTX - params = SSL_CTX_get0_param(self->ctx); /* Improve trust chain building when cross-signed intermediate certificates are present. See https://bugs.python.org/issue23476. */ From webhook-mailer at python.org Fri Jul 14 03:21:06 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 14 Jul 2023 07:21:06 -0000 Subject: [Python-checkins] [3.12] gh-105626: Change the default return value of `HTTPConnection.get_proxy_response_headers` (GH-105628) (#106738) Message-ID: https://github.com/python/cpython/commit/e68b2806717997db4fae338f906bd8d60ae9381f commit: e68b2806717997db4fae338f906bd8d60ae9381f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gpshead date: 2023-07-14T07:21:02Z summary: [3.12] gh-105626: Change the default return value of `HTTPConnection.get_proxy_response_headers` (GH-105628) (#106738) gh-105626: Change the default return value of `HTTPConnection.get_proxy_response_headers` (GH-105628) (cherry picked from commit 490295d651d04ec3b3eff2a2cda7501191bad78a) Co-authored-by: Nikita Sobolev files: A Misc/NEWS.d/next/Library/2023-06-10-12-20-17.gh-issue-105626.XyZein.rst M Doc/library/http.client.rst M Lib/http/client.py M Lib/test/test_httplib.py diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst index 45291933d635b..b9ceab699cef6 100644 --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -390,7 +390,7 @@ HTTPConnection Objects Returns a dictionary with the headers of the response received from the proxy server to the CONNECT request. - If the CONNECT request was not sent, the method returns an empty dictionary. + If the CONNECT request was not sent, the method returns ``None``. .. versionadded:: 3.12 diff --git a/Lib/http/client.py b/Lib/http/client.py index 3d98e4eb54bb4..b35b1d6368aae 100644 --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -970,13 +970,12 @@ def get_proxy_response_headers(self): received from the proxy server to the CONNECT request sent to set the tunnel. - If the CONNECT request was not sent, the method returns - an empty dictionary. + If the CONNECT request was not sent, the method returns None. """ return ( _parse_header_lines(self._raw_proxy_headers) if self._raw_proxy_headers is not None - else {} + else None ) def connect(self): diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py index 8955d45fa93dd..fe8105ee2bb3f 100644 --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -2404,6 +2404,19 @@ def test_proxy_response_headers(self): headers = self.conn.get_proxy_response_headers() self.assertIn(expected_header, headers.items()) + def test_no_proxy_response_headers(self): + expected_header = ('X-Dummy', '1') + response_text = ( + 'HTTP/1.0 200 OK\r\n' + '{0}\r\n\r\n'.format(':'.join(expected_header)) + ) + + self.conn._create_connection = self._create_connection(response_text) + + self.conn.request('PUT', '/', '') + headers = self.conn.get_proxy_response_headers() + self.assertIsNone(headers) + def test_tunnel_leak(self): sock = None diff --git a/Misc/NEWS.d/next/Library/2023-06-10-12-20-17.gh-issue-105626.XyZein.rst b/Misc/NEWS.d/next/Library/2023-06-10-12-20-17.gh-issue-105626.XyZein.rst new file mode 100644 index 0000000000000..2a48361fa596c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-10-12-20-17.gh-issue-105626.XyZein.rst @@ -0,0 +1,3 @@ +Change the default return value of +:meth:`http.client.HTTPConnection.get_proxy_response_headers` to be ``None`` +and not ``{}``. From webhook-mailer at python.org Fri Jul 14 03:38:07 2023 From: webhook-mailer at python.org (terryjreedy) Date: Fri, 14 Jul 2023 07:38:07 -0000 Subject: [Python-checkins] gh-106446: Fix failed doctest in stdtypes (#106447) Message-ID: https://github.com/python/cpython/commit/89867d2491c0c3ef77bc237899b2f0762f43c03c commit: 89867d2491c0c3ef77bc237899b2f0762f43c03c branch: main author: Charlie Zhao committer: terryjreedy date: 2023-07-14T03:38:03-04:00 summary: gh-106446: Fix failed doctest in stdtypes (#106447) --------- Co-authored-by: Terry Jan Reedy files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 8e049d9645c0b..fd51b1187576b 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -3953,7 +3953,7 @@ copying. >>> m = memoryview(bytearray(b'abc')) >>> mm = m.toreadonly() >>> mm.tolist() - [89, 98, 99] + [97, 98, 99] >>> mm[0] = 42 Traceback (most recent call last): File "", line 1, in @@ -4009,6 +4009,7 @@ copying. :mod:`struct` syntax. One of the formats must be a byte format ('B', 'b' or 'c'). The byte length of the result must be the same as the original length. + Note that all byte lengths may depend on the operating system. Cast 1D/long to 1D/unsigned bytes:: @@ -4039,8 +4040,8 @@ copying. >>> x = memoryview(b) >>> x[0] = b'a' Traceback (most recent call last): - File "", line 1, in - ValueError: memoryview: invalid value for format "B" + ... + TypeError: memoryview: invalid type for format 'B' >>> y = x.cast('c') >>> y[0] = b'a' >>> b @@ -4789,10 +4790,10 @@ An example of dictionary view usage:: >>> # set operations >>> keys & {'eggs', 'bacon', 'salad'} {'bacon'} - >>> keys ^ {'sausage', 'juice'} - {'juice', 'sausage', 'bacon', 'spam'} - >>> keys | ['juice', 'juice', 'juice'] - {'juice', 'sausage', 'bacon', 'spam', 'eggs'} + >>> keys ^ {'sausage', 'juice'} == {'juice', 'sausage', 'bacon', 'spam'} + True + >>> keys | ['juice', 'juice', 'juice'] == {'bacon', 'spam', 'juice'} + True >>> # get back a read-only proxy for the original dictionary >>> values.mapping @@ -4999,8 +5000,8 @@ exception to disallow mistakes like ``dict[str][str]``:: >>> dict[str][str] Traceback (most recent call last): - File "", line 1, in - TypeError: There are no type variables left in dict[str] + ... + TypeError: dict[str] is not a generic class However, such expressions are valid when :ref:`type variables ` are used. The index must have as many elements as there are type variable items @@ -5206,13 +5207,15 @@ enables cleaner type hinting syntax compared to :data:`typing.Union`. >>> isinstance("", int | str) True - However, union objects containing :ref:`parameterized generics - ` cannot be used:: + However, :ref:`parameterized generics ` in + union objects cannot be checked:: - >>> isinstance(1, int | list[int]) + >>> isinstance(1, int | list[int]) # short-circuit evaluation + True + >>> isinstance([1], int | list[int]) Traceback (most recent call last): - File "", line 1, in - TypeError: isinstance() argument 2 cannot contain a parameterized generic + ... + TypeError: isinstance() argument 2 cannot be a parameterized generic The user-exposed type for the union object can be accessed from :data:`types.UnionType` and used for :func:`isinstance` checks. An object cannot be @@ -5515,7 +5518,7 @@ types, where they are relevant. Some of these are not reported by the definition order. Example:: >>> int.__subclasses__() - [] + [, , , ] .. _int_max_str_digits: @@ -5551,7 +5554,7 @@ When an operation would exceed the limit, a :exc:`ValueError` is raised: >>> _ = int('2' * 5432) Traceback (most recent call last): ... - ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit. + ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit >>> i = int('2' * 4300) >>> len(str(i)) 4300 @@ -5559,7 +5562,7 @@ When an operation would exceed the limit, a :exc:`ValueError` is raised: >>> len(str(i_squared)) Traceback (most recent call last): ... - ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 8599 digits; use sys.set_int_max_str_digits() to increase the limit. + ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit >>> len(hex(i_squared)) 7144 >>> assert int(hex(i_squared), base=16) == i*i # Hexadecimal is unlimited. From webhook-mailer at python.org Fri Jul 14 04:01:39 2023 From: webhook-mailer at python.org (terryjreedy) Date: Fri, 14 Jul 2023 08:01:39 -0000 Subject: [Python-checkins] [3.12] gh-106446: Fix failed doctest in stdtypes (GH-106447) (#106741) Message-ID: https://github.com/python/cpython/commit/00eb4355e3c3ac891f0951d0ee46b0da3fdc2616 commit: 00eb4355e3c3ac891f0951d0ee46b0da3fdc2616 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-14T04:01:35-04:00 summary: [3.12] gh-106446: Fix failed doctest in stdtypes (GH-106447) (#106741) (cherry picked from commit 89867d2491c0c3ef77bc237899b2f0762f43c03c) Co-authored-by: Charlie Zhao Co-authored-by: Terry Jan Reedy files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 0ac727e0df38b..092f2719379e8 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -3950,7 +3950,7 @@ copying. >>> m = memoryview(bytearray(b'abc')) >>> mm = m.toreadonly() >>> mm.tolist() - [89, 98, 99] + [97, 98, 99] >>> mm[0] = 42 Traceback (most recent call last): File "", line 1, in @@ -4006,6 +4006,7 @@ copying. :mod:`struct` syntax. One of the formats must be a byte format ('B', 'b' or 'c'). The byte length of the result must be the same as the original length. + Note that all byte lengths may depend on the operating system. Cast 1D/long to 1D/unsigned bytes:: @@ -4036,8 +4037,8 @@ copying. >>> x = memoryview(b) >>> x[0] = b'a' Traceback (most recent call last): - File "", line 1, in - ValueError: memoryview: invalid value for format "B" + ... + TypeError: memoryview: invalid type for format 'B' >>> y = x.cast('c') >>> y[0] = b'a' >>> b @@ -4786,10 +4787,10 @@ An example of dictionary view usage:: >>> # set operations >>> keys & {'eggs', 'bacon', 'salad'} {'bacon'} - >>> keys ^ {'sausage', 'juice'} - {'juice', 'sausage', 'bacon', 'spam'} - >>> keys | ['juice', 'juice', 'juice'] - {'juice', 'sausage', 'bacon', 'spam', 'eggs'} + >>> keys ^ {'sausage', 'juice'} == {'juice', 'sausage', 'bacon', 'spam'} + True + >>> keys | ['juice', 'juice', 'juice'] == {'bacon', 'spam', 'juice'} + True >>> # get back a read-only proxy for the original dictionary >>> values.mapping @@ -4996,8 +4997,8 @@ exception to disallow mistakes like ``dict[str][str]``:: >>> dict[str][str] Traceback (most recent call last): - File "", line 1, in - TypeError: There are no type variables left in dict[str] + ... + TypeError: dict[str] is not a generic class However, such expressions are valid when :ref:`type variables ` are used. The index must have as many elements as there are type variable items @@ -5203,13 +5204,15 @@ enables cleaner type hinting syntax compared to :data:`typing.Union`. >>> isinstance("", int | str) True - However, union objects containing :ref:`parameterized generics - ` cannot be used:: + However, :ref:`parameterized generics ` in + union objects cannot be checked:: - >>> isinstance(1, int | list[int]) + >>> isinstance(1, int | list[int]) # short-circuit evaluation + True + >>> isinstance([1], int | list[int]) Traceback (most recent call last): - File "", line 1, in - TypeError: isinstance() argument 2 cannot contain a parameterized generic + ... + TypeError: isinstance() argument 2 cannot be a parameterized generic The user-exposed type for the union object can be accessed from :data:`types.UnionType` and used for :func:`isinstance` checks. An object cannot be @@ -5512,7 +5515,7 @@ types, where they are relevant. Some of these are not reported by the definition order. Example:: >>> int.__subclasses__() - [] + [, , , ] .. _int_max_str_digits: @@ -5548,7 +5551,7 @@ When an operation would exceed the limit, a :exc:`ValueError` is raised: >>> _ = int('2' * 5432) Traceback (most recent call last): ... - ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit. + ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit >>> i = int('2' * 4300) >>> len(str(i)) 4300 @@ -5556,7 +5559,7 @@ When an operation would exceed the limit, a :exc:`ValueError` is raised: >>> len(str(i_squared)) Traceback (most recent call last): ... - ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 8599 digits; use sys.set_int_max_str_digits() to increase the limit. + ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit >>> len(hex(i_squared)) 7144 >>> assert int(hex(i_squared), base=16) == i*i # Hexadecimal is unlimited. From webhook-mailer at python.org Fri Jul 14 04:02:29 2023 From: webhook-mailer at python.org (terryjreedy) Date: Fri, 14 Jul 2023 08:02:29 -0000 Subject: [Python-checkins] [3.11] gh-106446: Fix failed doctest in stdtypes (GH-106447) (#106742) Message-ID: https://github.com/python/cpython/commit/2186212191f2ef149d73073b4583d74a49d176e2 commit: 2186212191f2ef149d73073b4583d74a49d176e2 branch: 3.11 author: Terry Jan Reedy committer: terryjreedy date: 2023-07-14T04:02:25-04:00 summary: [3.11] gh-106446: Fix failed doctest in stdtypes (GH-106447) (#106742) (cherry picked from commit 89867d2491c0c3ef77bc237899b2f0762f43c03c) Co-authored-by: Charlie Zhao Co-authored-by: Terry Jan Reedy files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 31eedd0432555..f47ed686136ff 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -3903,7 +3903,7 @@ copying. >>> m = memoryview(bytearray(b'abc')) >>> mm = m.toreadonly() >>> mm.tolist() - [89, 98, 99] + [97, 98, 99] >>> mm[0] = 42 Traceback (most recent call last): File "", line 1, in @@ -3959,6 +3959,7 @@ copying. :mod:`struct` syntax. One of the formats must be a byte format ('B', 'b' or 'c'). The byte length of the result must be the same as the original length. + Note that all byte lengths may depend on the operating system. Cast 1D/long to 1D/unsigned bytes:: @@ -3989,8 +3990,8 @@ copying. >>> x = memoryview(b) >>> x[0] = b'a' Traceback (most recent call last): - File "", line 1, in - ValueError: memoryview: invalid value for format "B" + ... + TypeError: memoryview: invalid type for format 'B' >>> y = x.cast('c') >>> y[0] = b'a' >>> b @@ -4735,8 +4736,10 @@ An example of dictionary view usage:: >>> # set operations >>> keys & {'eggs', 'bacon', 'salad'} {'bacon'} - >>> keys ^ {'sausage', 'juice'} - {'juice', 'sausage', 'bacon', 'spam'} + >>> keys ^ {'sausage', 'juice'} == {'juice', 'sausage', 'bacon', 'spam'} + True + >>> keys | ['juice', 'juice', 'juice'] == {'bacon', 'spam', 'juice'} + True >>> # get back a read-only proxy for the original dictionary >>> values.mapping @@ -4943,8 +4946,8 @@ exception to disallow mistakes like ``dict[str][str]``:: >>> dict[str][str] Traceback (most recent call last): - File "", line 1, in - TypeError: There are no type variables left in dict[str] + ... + TypeError: dict[str] is not a generic class However, such expressions are valid when :ref:`type variables ` are used. The index must have as many elements as there are type variable items @@ -5150,13 +5153,15 @@ enables cleaner type hinting syntax compared to :data:`typing.Union`. >>> isinstance("", int | str) True - However, union objects containing :ref:`parameterized generics - ` cannot be used:: + However, :ref:`parameterized generics ` in + union objects cannot be checked:: - >>> isinstance(1, int | list[int]) + >>> isinstance(1, int | list[int]) # short-circuit evaluation + True + >>> isinstance([1], int | list[int]) Traceback (most recent call last): - File "", line 1, in - TypeError: isinstance() argument 2 cannot contain a parameterized generic + ... + TypeError: isinstance() argument 2 cannot be a parameterized generic The user-exposed type for the union object can be accessed from :data:`types.UnionType` and used for :func:`isinstance` checks. An object cannot be @@ -5472,7 +5477,7 @@ types, where they are relevant. Some of these are not reported by the definition order. Example:: >>> int.__subclasses__() - [] + [, , , ] .. _int_max_str_digits: @@ -5508,7 +5513,7 @@ When an operation would exceed the limit, a :exc:`ValueError` is raised: >>> _ = int('2' * 5432) Traceback (most recent call last): ... - ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit. + ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 5432 digits; use sys.set_int_max_str_digits() to increase the limit >>> i = int('2' * 4300) >>> len(str(i)) 4300 @@ -5516,7 +5521,7 @@ When an operation would exceed the limit, a :exc:`ValueError` is raised: >>> len(str(i_squared)) Traceback (most recent call last): ... - ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 8599 digits; use sys.set_int_max_str_digits() to increase the limit. + ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit >>> len(hex(i_squared)) 7144 >>> assert int(hex(i_squared), base=16) == i*i # Hexadecimal is unlimited. From webhook-mailer at python.org Fri Jul 14 05:40:35 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Fri, 14 Jul 2023 09:40:35 -0000 Subject: [Python-checkins] [3.12] gh-106634: Corrected minor asyncio doc issues (GH-106671) (#106712) Message-ID: https://github.com/python/cpython/commit/dfdded6bebc862edf1b286690b54538eccea8923 commit: dfdded6bebc862edf1b286690b54538eccea8923 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-14T15:10:32+05:30 summary: [3.12] gh-106634: Corrected minor asyncio doc issues (GH-106671) (#106712) gh-106634: Corrected minor asyncio doc issues (GH-106671) (cherry picked from commit 4b4a5b70aa8d47b1e2a0582b741c31b786da762a) Co-authored-by: Chris Brett files: M Lib/asyncio/base_events.py M Lib/asyncio/events.py M Misc/ACKS diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index 32d7e1c481ecc..259004650d5ab 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -727,7 +727,7 @@ def call_later(self, delay, callback, *args, context=None): always relative to the current time. Each callback will be called exactly once. If two callbacks - are scheduled for exactly the same time, it undefined which + are scheduled for exactly the same time, it is undefined which will be called first. Any positional arguments after the callback will be passed to diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py index ce44942186b27..0ccf85105e667 100644 --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -617,7 +617,7 @@ class AbstractEventLoopPolicy: def get_event_loop(self): """Get the event loop for the current context. - Returns an event loop object implementing the BaseEventLoop interface, + Returns an event loop object implementing the AbstractEventLoop interface, or raises an exception in case no event loop has been set for the current context and the current policy does not specify to create one. diff --git a/Misc/ACKS b/Misc/ACKS index ef0029a7e4119..645ad5b700baa 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -226,6 +226,7 @@ Erik Bray Brian Brazil Demian Brecht Dave Brennan +Christopher Richard James Brett Tom Bridgman Anthony Briggs Keith Briggs From webhook-mailer at python.org Fri Jul 14 08:21:01 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Fri, 14 Jul 2023 12:21:01 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Annotate `output_templates()` (#106732) Message-ID: https://github.com/python/cpython/commit/7c95345e4f93f4a2475418f17df5aae39dea861f commit: 7c95345e4f93f4a2475418f17df5aae39dea861f branch: main author: Erlend E. Aasland committer: AlexWaygood date: 2023-07-14T12:20:58Z summary: gh-104050: Argument Clinic: Annotate `output_templates()` (#106732) Co-authored-by: AlexWaygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 2dce33690993d..726ebc04f55bd 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -792,11 +792,14 @@ def docstring_for_c_string( add('"') return ''.join(text) - def output_templates(self, f): + def output_templates( + self, + f: Function + ) -> dict[str, str]: parameters = list(f.parameters.values()) assert parameters - assert isinstance(parameters[0].converter, self_converter) - del parameters[0] + first_param = parameters.pop(0) + assert isinstance(first_param.converter, self_converter) requires_defining_class = False if parameters and isinstance(parameters[0].converter, defining_class_converter): requires_defining_class = True @@ -809,7 +812,7 @@ def output_templates(self, f): new_or_init = f.kind.new_or_init - vararg = NO_VARARG + vararg: int | str = NO_VARARG pos_only = min_pos = max_pos = min_kw_only = pseudo_args = 0 for i, p in enumerate(parameters, 1): if p.is_keyword_only(): @@ -897,7 +900,7 @@ def output_templates(self, f): # parser_body_fields remembers the fields passed in to the # previous call to parser_body. this is used for an awful hack. - parser_body_fields = () + parser_body_fields: tuple[str, ...] = () def parser_body( prototype: str, *fields: str, @@ -932,6 +935,7 @@ def parser_body( return linear_format(output(), parser_declarations=declarations) if not parameters: + parser_code: list[str] | None if not requires_defining_class: # no parameters, METH_NOARGS flags = "METH_NOARGS" @@ -1165,7 +1169,7 @@ def parser_body( flags = 'METH_METHOD|' + flags parser_prototype = parser_prototype_def_class - add_label = None + add_label: str | None = None for i, p in enumerate(parameters): if isinstance(p.converter, defining_class_converter): raise ValueError("defining_class should be the first " @@ -1308,6 +1312,8 @@ def parser_body( cpp_if = "#if " + conditional cpp_endif = "#endif /* " + conditional + " */" + assert clinic is not None + assert f.full_name is not None if methoddef_define and f.full_name not in clinic.ifndef_symbols: clinic.ifndef_symbols.add(f.full_name) methoddef_ifndef = normalize_snippet(""" From webhook-mailer at python.org Fri Jul 14 13:38:32 2023 From: webhook-mailer at python.org (jaraco) Date: Fri, 14 Jul 2023 17:38:32 -0000 Subject: [Python-checkins] gh-106531: Remove importlib.resources._legacy (#106532) Message-ID: https://github.com/python/cpython/commit/243fdcb40ebeb177ce723911c1f7fad8a1fdf6cb commit: 243fdcb40ebeb177ce723911c1f7fad8a1fdf6cb branch: main author: Jason R. Coombs committer: jaraco date: 2023-07-14T13:38:28-04:00 summary: gh-106531: Remove importlib.resources._legacy (#106532) * gh-106531: Remove importlib.resources._legacy Syncs with importlib_resources 6.0. * Remove documentation for removed functionality. files: A Misc/NEWS.d/next/Library/2023-07-07-16-19-59.gh-issue-106531.eMfNm8.rst D Lib/importlib/resources/_legacy.py M Doc/library/importlib.resources.rst M Lib/importlib/resources/__init__.py diff --git a/Doc/library/importlib.resources.rst b/Doc/library/importlib.resources.rst index 755693840fecd..76faf73114477 100644 --- a/Doc/library/importlib.resources.rst +++ b/Doc/library/importlib.resources.rst @@ -94,159 +94,3 @@ for example, a package and its resources can be imported from a zip file using the file system is required. .. versionadded:: 3.9 - - -Deprecated functions -^^^^^^^^^^^^^^^^^^^^ - -An older, deprecated set of functions is still available, but is -scheduled for removal in a future version of Python. -The main drawback of these functions is that they do not support -directories: they assume all resources are located directly within a *package*. - -.. data:: Package - - Whenever a function accepts a ``Package`` argument, you can pass in - either a :class:`module object ` or a module name - as a string. You can only pass module objects whose - ``__spec__.submodule_search_locations`` is not ``None``. - - The ``Package`` type is defined as ``Union[str, ModuleType]``. - - .. deprecated:: 3.12 - - -.. data:: Resource - - For *resource* arguments of the functions below, you can pass in - the name of a resource as a string or - a :class:`path-like object `. - - The ``Resource`` type is defined as ``Union[str, os.PathLike]``. - - -.. function:: open_binary(package, resource) - - Open for binary reading the *resource* within *package*. - - *package* is either a name or a module object which conforms to the - ``Package`` requirements. *resource* is the name of the resource to open - within *package*; it may not contain path separators and it may not have - sub-resources (i.e. it cannot be a directory). This function returns a - ``typing.BinaryIO`` instance, a binary I/O stream open for reading. - - .. deprecated:: 3.11 - - Calls to this function can be replaced by:: - - files(package).joinpath(resource).open('rb') - - -.. function:: open_text(package, resource, encoding='utf-8', errors='strict') - - Open for text reading the *resource* within *package*. By default, the - resource is opened for reading as UTF-8. - - *package* is either a name or a module object which conforms to the - ``Package`` requirements. *resource* is the name of the resource to open - within *package*; it may not contain path separators and it may not have - sub-resources (i.e. it cannot be a directory). *encoding* and *errors* - have the same meaning as with built-in :func:`open`. - - This function returns a ``typing.TextIO`` instance, a text I/O stream open - for reading. - - .. deprecated:: 3.11 - - Calls to this function can be replaced by:: - - files(package).joinpath(resource).open('r', encoding=encoding) - - -.. function:: read_binary(package, resource) - - Read and return the contents of the *resource* within *package* as - ``bytes``. - - *package* is either a name or a module object which conforms to the - ``Package`` requirements. *resource* is the name of the resource to open - within *package*; it may not contain path separators and it may not have - sub-resources (i.e. it cannot be a directory). This function returns the - contents of the resource as :class:`bytes`. - - .. deprecated:: 3.11 - - Calls to this function can be replaced by:: - - files(package).joinpath(resource).read_bytes() - - -.. function:: read_text(package, resource, encoding='utf-8', errors='strict') - - Read and return the contents of *resource* within *package* as a ``str``. - By default, the contents are read as strict UTF-8. - - *package* is either a name or a module object which conforms to the - ``Package`` requirements. *resource* is the name of the resource to open - within *package*; it may not contain path separators and it may not have - sub-resources (i.e. it cannot be a directory). *encoding* and *errors* - have the same meaning as with built-in :func:`open`. This function - returns the contents of the resource as :class:`str`. - - .. deprecated:: 3.11 - - Calls to this function can be replaced by:: - - files(package).joinpath(resource).read_text(encoding=encoding) - - -.. function:: path(package, resource) - - Return the path to the *resource* as an actual file system path. This - function returns a context manager for use in a :keyword:`with` statement. - The context manager provides a :class:`pathlib.Path` object. - - Exiting the context manager cleans up any temporary file created when the - resource needs to be extracted from e.g. a zip file. - - *package* is either a name or a module object which conforms to the - ``Package`` requirements. *resource* is the name of the resource to open - within *package*; it may not contain path separators and it may not have - sub-resources (i.e. it cannot be a directory). - - .. deprecated:: 3.11 - - Calls to this function can be replaced using :func:`as_file`:: - - as_file(files(package).joinpath(resource)) - - -.. function:: is_resource(package, name) - - Return ``True`` if there is a resource named *name* in the package, - otherwise ``False``. - This function does not consider directories to be resources. - *package* is either a name or a module object which conforms to the - ``Package`` requirements. - - .. deprecated:: 3.11 - - Calls to this function can be replaced by:: - - files(package).joinpath(resource).is_file() - - -.. function:: contents(package) - - Return an iterable over the named items within the package. The iterable - returns :class:`str` resources (e.g. files) and non-resources - (e.g. directories). The iterable does not recurse into subdirectories. - - *package* is either a name or a module object which conforms to the - ``Package`` requirements. - - .. deprecated:: 3.11 - - Calls to this function can be replaced by:: - - (resource.name for resource in files(package).iterdir() if resource.is_file()) diff --git a/Lib/importlib/resources/__init__.py b/Lib/importlib/resources/__init__.py index 34e3a9950cc55..e6b60c18caa05 100644 --- a/Lib/importlib/resources/__init__.py +++ b/Lib/importlib/resources/__init__.py @@ -6,31 +6,12 @@ Package, ) -from ._legacy import ( - contents, - open_binary, - read_binary, - open_text, - read_text, - is_resource, - path, - Resource, -) - from .abc import ResourceReader __all__ = [ 'Package', - 'Resource', 'ResourceReader', 'as_file', - 'contents', 'files', - 'is_resource', - 'open_binary', - 'open_text', - 'path', - 'read_binary', - 'read_text', ] diff --git a/Lib/importlib/resources/_legacy.py b/Lib/importlib/resources/_legacy.py deleted file mode 100644 index b1ea8105dad6e..0000000000000 --- a/Lib/importlib/resources/_legacy.py +++ /dev/null @@ -1,120 +0,0 @@ -import functools -import os -import pathlib -import types -import warnings - -from typing import Union, Iterable, ContextManager, BinaryIO, TextIO, Any - -from . import _common - -Package = Union[types.ModuleType, str] -Resource = str - - -def deprecated(func): - @functools.wraps(func) - def wrapper(*args, **kwargs): - warnings.warn( - f"{func.__name__} is deprecated. Use files() instead. " - "Refer to https://importlib-resources.readthedocs.io" - "/en/latest/using.html#migrating-from-legacy for migration advice.", - DeprecationWarning, - stacklevel=2, - ) - return func(*args, **kwargs) - - return wrapper - - -def normalize_path(path: Any) -> str: - """Normalize a path by ensuring it is a string. - - If the resulting string contains path separators, an exception is raised. - """ - str_path = str(path) - parent, file_name = os.path.split(str_path) - if parent: - raise ValueError(f'{path!r} must be only a file name') - return file_name - - - at deprecated -def open_binary(package: Package, resource: Resource) -> BinaryIO: - """Return a file-like object opened for binary reading of the resource.""" - return (_common.files(package) / normalize_path(resource)).open('rb') - - - at deprecated -def read_binary(package: Package, resource: Resource) -> bytes: - """Return the binary contents of the resource.""" - return (_common.files(package) / normalize_path(resource)).read_bytes() - - - at deprecated -def open_text( - package: Package, - resource: Resource, - encoding: str = 'utf-8', - errors: str = 'strict', -) -> TextIO: - """Return a file-like object opened for text reading of the resource.""" - return (_common.files(package) / normalize_path(resource)).open( - 'r', encoding=encoding, errors=errors - ) - - - at deprecated -def read_text( - package: Package, - resource: Resource, - encoding: str = 'utf-8', - errors: str = 'strict', -) -> str: - """Return the decoded string of the resource. - - The decoding-related arguments have the same semantics as those of - bytes.decode(). - """ - with open_text(package, resource, encoding, errors) as fp: - return fp.read() - - - at deprecated -def contents(package: Package) -> Iterable[str]: - """Return an iterable of entries in `package`. - - Note that not all entries are resources. Specifically, directories are - not considered resources. Use `is_resource()` on each entry returned here - to check if it is a resource or not. - """ - return [path.name for path in _common.files(package).iterdir()] - - - at deprecated -def is_resource(package: Package, name: str) -> bool: - """True if `name` is a resource inside `package`. - - Directories are *not* resources. - """ - resource = normalize_path(name) - return any( - traversable.name == resource and traversable.is_file() - for traversable in _common.files(package).iterdir() - ) - - - at deprecated -def path( - package: Package, - resource: Resource, -) -> ContextManager[pathlib.Path]: - """A context manager providing a file path object to the resource. - - If the resource does not already exist on its own on the file system, - a temporary file will be created. If the file was created, the file - will be deleted upon exiting the context manager (no exception is - raised if the file was deleted prior to the context manager - exiting). - """ - return _common.as_file(_common.files(package) / normalize_path(resource)) diff --git a/Misc/NEWS.d/next/Library/2023-07-07-16-19-59.gh-issue-106531.eMfNm8.rst b/Misc/NEWS.d/next/Library/2023-07-07-16-19-59.gh-issue-106531.eMfNm8.rst new file mode 100644 index 0000000000000..a52107103c457 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-16-19-59.gh-issue-106531.eMfNm8.rst @@ -0,0 +1,3 @@ +Removed ``_legacy`` and the names it provided from ``importlib.resources``: +``Resource``, ``contents``, ``is_resource``, ``open_binary``, ``open_text``, +``path``, ``read_binary``, and ``read_text``. From webhook-mailer at python.org Fri Jul 14 13:41:56 2023 From: webhook-mailer at python.org (iritkatriel) Date: Fri, 14 Jul 2023 17:41:56 -0000 Subject: [Python-checkins] gh-105481: expose opcode metadata via the _opcode module (#106688) Message-ID: https://github.com/python/cpython/commit/6a70edf24ca217c5ed4a556d0df5748fc775c762 commit: 6a70edf24ca217c5ed4a556d0df5748fc775c762 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-14T18:41:52+01:00 summary: gh-105481: expose opcode metadata via the _opcode module (#106688) files: A Misc/NEWS.d/next/Library/2023-07-13-16-04-15.gh-issue-105481.pYSwMj.rst M Include/cpython/compile.h M Include/internal/pycore_opcode_metadata.h M Lib/test/test__opcode.py M Modules/_opcode.c M Modules/clinic/_opcode.c.h M Python/compile.c M Python/flowgraph.c M Tools/cases_generator/generate_cases.py diff --git a/Include/cpython/compile.h b/Include/cpython/compile.h index f5a62a8ec6dd0..cd7fd7bd37766 100644 --- a/Include/cpython/compile.h +++ b/Include/cpython/compile.h @@ -67,3 +67,9 @@ typedef struct { #define PY_INVALID_STACK_EFFECT INT_MAX PyAPI_FUNC(int) PyCompile_OpcodeStackEffect(int opcode, int oparg); PyAPI_FUNC(int) PyCompile_OpcodeStackEffectWithJump(int opcode, int oparg, int jump); + +PyAPI_FUNC(int) PyUnstable_OpcodeIsValid(int opcode); +PyAPI_FUNC(int) PyUnstable_OpcodeHasArg(int opcode); +PyAPI_FUNC(int) PyUnstable_OpcodeHasConst(int opcode); +PyAPI_FUNC(int) PyUnstable_OpcodeHasName(int opcode); +PyAPI_FUNC(int) PyUnstable_OpcodeHasJump(int opcode); diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index e94732b64384b..8373f56653b1c 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -3,6 +3,8 @@ // Python/bytecodes.c // Do not edit! +#include + #define IS_PSEUDO_INSTR(OP) ( \ ((OP) == LOAD_CLOSURE) || \ @@ -941,14 +943,20 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { #endif enum InstructionFormat { INSTR_FMT_IB, INSTR_FMT_IBC, INSTR_FMT_IBC00, INSTR_FMT_IBC000, INSTR_FMT_IBC00000, INSTR_FMT_IBC00000000, INSTR_FMT_IX, INSTR_FMT_IXC, INSTR_FMT_IXC0, INSTR_FMT_IXC00, INSTR_FMT_IXC000 }; + +#define IS_VALID_OPCODE(OP) \ + (((OP) >= 0) && ((OP) < OPCODE_METADATA_SIZE) && \ + (_PyOpcode_opcode_metadata[(OP)].valid_entry)) + #define HAS_ARG_FLAG (1) #define HAS_CONST_FLAG (2) #define HAS_NAME_FLAG (4) #define HAS_JUMP_FLAG (8) -#define OPCODE_HAS_ARG(OP) (_PyOpcode_opcode_metadata[(OP)].flags & (HAS_ARG_FLAG)) -#define OPCODE_HAS_CONST(OP) (_PyOpcode_opcode_metadata[(OP)].flags & (HAS_CONST_FLAG)) -#define OPCODE_HAS_NAME(OP) (_PyOpcode_opcode_metadata[(OP)].flags & (HAS_NAME_FLAG)) -#define OPCODE_HAS_JUMP(OP) (_PyOpcode_opcode_metadata[(OP)].flags & (HAS_JUMP_FLAG)) +#define OPCODE_HAS_ARG(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_ARG_FLAG)) +#define OPCODE_HAS_CONST(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_CONST_FLAG)) +#define OPCODE_HAS_NAME(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_NAME_FLAG)) +#define OPCODE_HAS_JUMP(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_JUMP_FLAG)) + struct opcode_metadata { bool valid_entry; enum InstructionFormat instr_format; @@ -971,12 +979,16 @@ struct opcode_macro_expansion { #define SAME_OPCODE_METADATA(OP1, OP2) \ (OPCODE_METADATA_FMT(OP1) == OPCODE_METADATA_FMT(OP2)) +#define OPCODE_METADATA_SIZE 512 +#define OPCODE_UOP_NAME_SIZE 512 +#define OPCODE_MACRO_EXPANSION_SIZE 256 + #ifndef NEED_OPCODE_METADATA -extern const struct opcode_metadata _PyOpcode_opcode_metadata[512]; -extern const struct opcode_macro_expansion _PyOpcode_macro_expansion[256]; -extern const char * const _PyOpcode_uop_name[512]; +extern const struct opcode_metadata _PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE]; +extern const struct opcode_macro_expansion _PyOpcode_macro_expansion[OPCODE_MACRO_EXPANSION_SIZE]; +extern const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE]; #else // if NEED_OPCODE_METADATA -const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { +const struct opcode_metadata _PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE] = { [NOP] = { true, INSTR_FMT_IX, 0 }, [RESUME] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, [INSTRUMENTED_RESUME] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, @@ -1194,7 +1206,7 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[512] = { [CACHE] = { true, INSTR_FMT_IX, 0 }, [RESERVED] = { true, INSTR_FMT_IX, 0 }, }; -const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { +const struct opcode_macro_expansion _PyOpcode_macro_expansion[OPCODE_MACRO_EXPANSION_SIZE] = { [NOP] = { .nuops = 1, .uops = { { NOP, 0, 0 } } }, [LOAD_FAST_CHECK] = { .nuops = 1, .uops = { { LOAD_FAST_CHECK, 0, 0 } } }, [LOAD_FAST] = { .nuops = 1, .uops = { { LOAD_FAST, 0, 0 } } }, @@ -1308,7 +1320,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] = { [BINARY_OP] = { .nuops = 1, .uops = { { BINARY_OP, 0, 0 } } }, [SWAP] = { .nuops = 1, .uops = { { SWAP, 0, 0 } } }, }; -const char * const _PyOpcode_uop_name[512] = { +const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE] = { [EXIT_TRACE] = "EXIT_TRACE", [SAVE_IP] = "SAVE_IP", [_GUARD_BOTH_INT] = "_GUARD_BOTH_INT", diff --git a/Lib/test/test__opcode.py b/Lib/test/test__opcode.py index 3e084928844d2..7d9553d9e383a 100644 --- a/Lib/test/test__opcode.py +++ b/Lib/test/test__opcode.py @@ -9,6 +9,70 @@ class OpcodeTests(unittest.TestCase): + def check_bool_function_result(self, func, ops, expected): + for op in ops: + if isinstance(op, str): + op = dis.opmap[op] + with self.subTest(opcode=op, func=func): + self.assertIsInstance(func(op), bool) + self.assertEqual(func(op), expected) + + def test_invalid_opcodes(self): + invalid = [-100, -1, 255, 512, 513, 1000] + self.check_bool_function_result(_opcode.is_valid, invalid, False) + self.check_bool_function_result(_opcode.has_arg, invalid, False) + self.check_bool_function_result(_opcode.has_const, invalid, False) + self.check_bool_function_result(_opcode.has_name, invalid, False) + self.check_bool_function_result(_opcode.has_jump, invalid, False) + + def test_is_valid(self): + names = [ + 'CACHE', + 'POP_TOP', + 'IMPORT_NAME', + 'JUMP', + 'INSTRUMENTED_RETURN_VALUE', + ] + opcodes = [dis.opmap[opname] for opname in names] + self.check_bool_function_result(_opcode.is_valid, opcodes, True) + + def test_has_arg(self): + has_arg = ['SWAP', 'LOAD_FAST', 'INSTRUMENTED_POP_JUMP_IF_TRUE', 'JUMP'] + no_arg = ['SETUP_WITH', 'POP_TOP', 'NOP', 'CACHE'] + self.check_bool_function_result(_opcode.has_arg, has_arg, True) + self.check_bool_function_result(_opcode.has_arg, no_arg, False) + + def test_has_const(self): + has_const = ['LOAD_CONST', 'RETURN_CONST', 'KW_NAMES'] + no_const = ['SETUP_WITH', 'POP_TOP', 'NOP', 'CACHE'] + self.check_bool_function_result(_opcode.has_const, has_const, True) + self.check_bool_function_result(_opcode.has_const, no_const, False) + + def test_has_name(self): + has_name = ['STORE_NAME', 'DELETE_ATTR', 'STORE_GLOBAL', 'IMPORT_FROM', + 'LOAD_FROM_DICT_OR_GLOBALS'] + no_name = ['SETUP_WITH', 'POP_TOP', 'NOP', 'CACHE'] + self.check_bool_function_result(_opcode.has_name, has_name, True) + self.check_bool_function_result(_opcode.has_name, no_name, False) + + def test_has_jump(self): + has_jump = ['FOR_ITER', 'JUMP_FORWARD', 'JUMP', 'POP_JUMP_IF_TRUE', 'SEND'] + no_jump = ['SETUP_WITH', 'POP_TOP', 'NOP', 'CACHE'] + self.check_bool_function_result(_opcode.has_jump, has_jump, True) + self.check_bool_function_result(_opcode.has_jump, no_jump, False) + + # the following test is part of the refactor, it will be removed soon + def test_against_legacy_bool_values(self): + # limiting to ops up to ENTER_EXECUTOR, because everything after that + # is not currently categorized correctly in opcode.py. + for op in range(0, opcode.opmap['ENTER_EXECUTOR']): + with self.subTest(op=op): + if opcode.opname[op] != f'<{op}>': + self.assertEqual(op in dis.hasarg, _opcode.has_arg(op)) + self.assertEqual(op in dis.hasconst, _opcode.has_const(op)) + self.assertEqual(op in dis.hasname, _opcode.has_name(op)) + self.assertEqual(op in dis.hasjrel, _opcode.has_jump(op)) + def test_stack_effect(self): self.assertEqual(stack_effect(dis.opmap['POP_TOP']), -1) self.assertEqual(stack_effect(dis.opmap['BUILD_SLICE'], 0), -1) diff --git a/Misc/NEWS.d/next/Library/2023-07-13-16-04-15.gh-issue-105481.pYSwMj.rst b/Misc/NEWS.d/next/Library/2023-07-13-16-04-15.gh-issue-105481.pYSwMj.rst new file mode 100644 index 0000000000000..bc2ba51d31aa9 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-13-16-04-15.gh-issue-105481.pYSwMj.rst @@ -0,0 +1 @@ +Expose opcode metadata through :mod:`_opcode`. diff --git a/Modules/_opcode.c b/Modules/_opcode.c index 3c0fce48770ac..b3b9873d21a5b 100644 --- a/Modules/_opcode.c +++ b/Modules/_opcode.c @@ -1,4 +1,5 @@ #include "Python.h" +#include "compile.h" #include "opcode.h" #include "internal/pycore_code.h" @@ -61,6 +62,91 @@ _opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg, /*[clinic input] +_opcode.is_valid -> bool + + opcode: int + +Return True if opcode is valid, False otherwise. +[clinic start generated code]*/ + +static int +_opcode_is_valid_impl(PyObject *module, int opcode) +/*[clinic end generated code: output=b0d918ea1d073f65 input=fe23e0aa194ddae0]*/ +{ + return PyUnstable_OpcodeIsValid(opcode); +} + +/*[clinic input] + +_opcode.has_arg -> bool + + opcode: int + +Return True if the opcode uses its oparg, False otherwise. +[clinic start generated code]*/ + +static int +_opcode_has_arg_impl(PyObject *module, int opcode) +/*[clinic end generated code: output=7a062d3b2dcc0815 input=93d878ba6361db5f]*/ +{ + return PyUnstable_OpcodeIsValid(opcode) && + PyUnstable_OpcodeHasArg(opcode); +} + +/*[clinic input] + +_opcode.has_const -> bool + + opcode: int + +Return True if the opcode accesses a constant, False otherwise. +[clinic start generated code]*/ + +static int +_opcode_has_const_impl(PyObject *module, int opcode) +/*[clinic end generated code: output=c646d5027c634120 input=a6999e4cf13f9410]*/ +{ + return PyUnstable_OpcodeIsValid(opcode) && + PyUnstable_OpcodeHasConst(opcode); +} + +/*[clinic input] + +_opcode.has_name -> bool + + opcode: int + +Return True if the opcode accesses an attribute by name, False otherwise. +[clinic start generated code]*/ + +static int +_opcode_has_name_impl(PyObject *module, int opcode) +/*[clinic end generated code: output=b49a83555c2fa517 input=448aa5e4bcc947ba]*/ +{ + return PyUnstable_OpcodeIsValid(opcode) && + PyUnstable_OpcodeHasName(opcode); +} + +/*[clinic input] + +_opcode.has_jump -> bool + + opcode: int + +Return True if the opcode has a jump target, False otherwise. +[clinic start generated code]*/ + +static int +_opcode_has_jump_impl(PyObject *module, int opcode) +/*[clinic end generated code: output=e9c583c669f1c46a input=35f711274357a0c3]*/ +{ + return PyUnstable_OpcodeIsValid(opcode) && + PyUnstable_OpcodeHasJump(opcode); + +} + +/*[clinic input] + _opcode.get_specialization_stats Return the specialization stats @@ -80,6 +166,11 @@ _opcode_get_specialization_stats_impl(PyObject *module) static PyMethodDef opcode_functions[] = { _OPCODE_STACK_EFFECT_METHODDEF + _OPCODE_IS_VALID_METHODDEF + _OPCODE_HAS_ARG_METHODDEF + _OPCODE_HAS_CONST_METHODDEF + _OPCODE_HAS_NAME_METHODDEF + _OPCODE_HAS_JUMP_METHODDEF _OPCODE_GET_SPECIALIZATION_STATS_METHODDEF {NULL, NULL, 0, NULL} }; diff --git a/Modules/clinic/_opcode.c.h b/Modules/clinic/_opcode.c.h index 3bd3ba0238743..3eb050e470c34 100644 --- a/Modules/clinic/_opcode.c.h +++ b/Modules/clinic/_opcode.c.h @@ -86,6 +86,321 @@ _opcode_stack_effect(PyObject *module, PyObject *const *args, Py_ssize_t nargs, return return_value; } +PyDoc_STRVAR(_opcode_is_valid__doc__, +"is_valid($module, /, opcode)\n" +"--\n" +"\n" +"Return True if opcode is valid, False otherwise."); + +#define _OPCODE_IS_VALID_METHODDEF \ + {"is_valid", _PyCFunction_CAST(_opcode_is_valid), METH_FASTCALL|METH_KEYWORDS, _opcode_is_valid__doc__}, + +static int +_opcode_is_valid_impl(PyObject *module, int opcode); + +static PyObject * +_opcode_is_valid(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(opcode), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"opcode", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "is_valid", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int opcode; + int _return_value; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + opcode = _PyLong_AsInt(args[0]); + if (opcode == -1 && PyErr_Occurred()) { + goto exit; + } + _return_value = _opcode_is_valid_impl(module, opcode); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} + +PyDoc_STRVAR(_opcode_has_arg__doc__, +"has_arg($module, /, opcode)\n" +"--\n" +"\n" +"Return True if the opcode uses its oparg, False otherwise."); + +#define _OPCODE_HAS_ARG_METHODDEF \ + {"has_arg", _PyCFunction_CAST(_opcode_has_arg), METH_FASTCALL|METH_KEYWORDS, _opcode_has_arg__doc__}, + +static int +_opcode_has_arg_impl(PyObject *module, int opcode); + +static PyObject * +_opcode_has_arg(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(opcode), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"opcode", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "has_arg", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int opcode; + int _return_value; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + opcode = _PyLong_AsInt(args[0]); + if (opcode == -1 && PyErr_Occurred()) { + goto exit; + } + _return_value = _opcode_has_arg_impl(module, opcode); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} + +PyDoc_STRVAR(_opcode_has_const__doc__, +"has_const($module, /, opcode)\n" +"--\n" +"\n" +"Return True if the opcode accesses a constant, False otherwise."); + +#define _OPCODE_HAS_CONST_METHODDEF \ + {"has_const", _PyCFunction_CAST(_opcode_has_const), METH_FASTCALL|METH_KEYWORDS, _opcode_has_const__doc__}, + +static int +_opcode_has_const_impl(PyObject *module, int opcode); + +static PyObject * +_opcode_has_const(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(opcode), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"opcode", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "has_const", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int opcode; + int _return_value; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + opcode = _PyLong_AsInt(args[0]); + if (opcode == -1 && PyErr_Occurred()) { + goto exit; + } + _return_value = _opcode_has_const_impl(module, opcode); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} + +PyDoc_STRVAR(_opcode_has_name__doc__, +"has_name($module, /, opcode)\n" +"--\n" +"\n" +"Return True if the opcode accesses an attribute by name, False otherwise."); + +#define _OPCODE_HAS_NAME_METHODDEF \ + {"has_name", _PyCFunction_CAST(_opcode_has_name), METH_FASTCALL|METH_KEYWORDS, _opcode_has_name__doc__}, + +static int +_opcode_has_name_impl(PyObject *module, int opcode); + +static PyObject * +_opcode_has_name(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(opcode), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"opcode", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "has_name", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int opcode; + int _return_value; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + opcode = _PyLong_AsInt(args[0]); + if (opcode == -1 && PyErr_Occurred()) { + goto exit; + } + _return_value = _opcode_has_name_impl(module, opcode); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} + +PyDoc_STRVAR(_opcode_has_jump__doc__, +"has_jump($module, /, opcode)\n" +"--\n" +"\n" +"Return True if the opcode has a jump target, False otherwise."); + +#define _OPCODE_HAS_JUMP_METHODDEF \ + {"has_jump", _PyCFunction_CAST(_opcode_has_jump), METH_FASTCALL|METH_KEYWORDS, _opcode_has_jump__doc__}, + +static int +_opcode_has_jump_impl(PyObject *module, int opcode); + +static PyObject * +_opcode_has_jump(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(opcode), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"opcode", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "has_jump", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int opcode; + int _return_value; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + opcode = _PyLong_AsInt(args[0]); + if (opcode == -1 && PyErr_Occurred()) { + goto exit; + } + _return_value = _opcode_has_jump_impl(module, opcode); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} + PyDoc_STRVAR(_opcode_get_specialization_stats__doc__, "get_specialization_stats($module, /)\n" "--\n" @@ -103,4 +418,4 @@ _opcode_get_specialization_stats(PyObject *module, PyObject *Py_UNUSED(ignored)) { return _opcode_get_specialization_stats_impl(module); } -/*[clinic end generated code: output=21e3d53a659c651a input=a9049054013a1b77]*/ +/*[clinic end generated code: output=ae2b2ef56d582180 input=a9049054013a1b77]*/ diff --git a/Python/compile.c b/Python/compile.c index d9e38cfdefaf2..9e86e06777ffa 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -36,7 +36,9 @@ #include "pycore_pystate.h" // _Py_GetConfig() #include "pycore_symtable.h" // PySTEntryObject, _PyFuture_FromAST() +#define NEED_OPCODE_METADATA #include "pycore_opcode_metadata.h" // _PyOpcode_opcode_metadata, _PyOpcode_num_popped/pushed +#undef NEED_OPCODE_METADATA #define COMP_GENEXP 0 #define COMP_LISTCOMP 1 @@ -864,6 +866,36 @@ PyCompile_OpcodeStackEffect(int opcode, int oparg) return stack_effect(opcode, oparg, -1); } +int +PyUnstable_OpcodeIsValid(int opcode) +{ + return IS_VALID_OPCODE(opcode); +} + +int +PyUnstable_OpcodeHasArg(int opcode) +{ + return OPCODE_HAS_ARG(opcode); +} + +int +PyUnstable_OpcodeHasConst(int opcode) +{ + return OPCODE_HAS_CONST(opcode); +} + +int +PyUnstable_OpcodeHasName(int opcode) +{ + return OPCODE_HAS_NAME(opcode); +} + +int +PyUnstable_OpcodeHasJump(int opcode) +{ + return OPCODE_HAS_JUMP(opcode); +} + static int codegen_addop_noarg(instr_sequence *seq, int opcode, location loc) { diff --git a/Python/flowgraph.c b/Python/flowgraph.c index 04f269c585383..e485ed103147a 100644 --- a/Python/flowgraph.c +++ b/Python/flowgraph.c @@ -7,9 +7,7 @@ #include "pycore_pymem.h" // _PyMem_IsPtrFreed() #include "pycore_opcode_utils.h" -#define NEED_OPCODE_METADATA -#include "pycore_opcode_metadata.h" // _PyOpcode_opcode_metadata, _PyOpcode_num_popped/pushed -#undef NEED_OPCODE_METADATA +#include "pycore_opcode_metadata.h" // OPCODE_HAS_ARG, etc #undef SUCCESS diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index ce16271097b95..6589289121863 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -308,7 +308,7 @@ def emit_macros(cls, out: Formatter): for name, value in flags.bitmask.items(): out.emit( f"#define OPCODE_{name[:-len('_FLAG')]}(OP) " - f"(_PyOpcode_opcode_metadata[(OP)].flags & ({name}))") + f"(_PyOpcode_opcode_metadata[OP].flags & ({name}))") @dataclasses.dataclass @@ -1192,6 +1192,8 @@ def write_metadata(self) -> None: self.write_provenance_header() + self.out.emit("\n#include ") + self.write_pseudo_instrs() self.out.emit("") @@ -1202,8 +1204,16 @@ def write_metadata(self) -> None: # Write type definitions self.out.emit(f"enum InstructionFormat {{ {', '.join(format_enums)} }};") + self.out.emit("") + self.out.emit( + "#define IS_VALID_OPCODE(OP) \\\n" + " (((OP) >= 0) && ((OP) < OPCODE_METADATA_SIZE) && \\\n" + " (_PyOpcode_opcode_metadata[(OP)].valid_entry))") + + self.out.emit("") InstructionFlags.emit_macros(self.out) + self.out.emit("") with self.out.block("struct opcode_metadata", ";"): self.out.emit("bool valid_entry;") self.out.emit("enum InstructionFormat instr_format;") @@ -1226,13 +1236,20 @@ def write_metadata(self) -> None: self.out.emit("") # Write metadata array declaration + self.out.emit("#define OPCODE_METADATA_SIZE 512") + self.out.emit("#define OPCODE_UOP_NAME_SIZE 512") + self.out.emit("#define OPCODE_MACRO_EXPANSION_SIZE 256") + self.out.emit("") self.out.emit("#ifndef NEED_OPCODE_METADATA") - self.out.emit("extern const struct opcode_metadata _PyOpcode_opcode_metadata[512];") - self.out.emit("extern const struct opcode_macro_expansion _PyOpcode_macro_expansion[256];") - self.out.emit("extern const char * const _PyOpcode_uop_name[512];") + self.out.emit("extern const struct opcode_metadata " + "_PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE];") + self.out.emit("extern const struct opcode_macro_expansion " + "_PyOpcode_macro_expansion[OPCODE_MACRO_EXPANSION_SIZE];") + self.out.emit("extern const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE];") self.out.emit("#else // if NEED_OPCODE_METADATA") - self.out.emit("const struct opcode_metadata _PyOpcode_opcode_metadata[512] = {") + self.out.emit("const struct opcode_metadata " + "_PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE] = {") # Write metadata for each instruction for thing in self.everything: @@ -1253,7 +1270,8 @@ def write_metadata(self) -> None: self.out.emit("};") with self.out.block( - "const struct opcode_macro_expansion _PyOpcode_macro_expansion[256] =", + "const struct opcode_macro_expansion " + "_PyOpcode_macro_expansion[OPCODE_MACRO_EXPANSION_SIZE] =", ";", ): # Write macro expansion for each non-pseudo instruction @@ -1279,7 +1297,7 @@ def write_metadata(self) -> None: case _: typing.assert_never(thing) - with self.out.block("const char * const _PyOpcode_uop_name[512] =", ";"): + with self.out.block("const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE] =", ";"): self.write_uop_items(lambda name, counter: f"[{name}] = \"{name}\",") self.out.emit("#endif // NEED_OPCODE_METADATA") From webhook-mailer at python.org Fri Jul 14 14:46:34 2023 From: webhook-mailer at python.org (methane) Date: Fri, 14 Jul 2023 18:46:34 -0000 Subject: [Python-checkins] gh-106554: replace `_BaseSelectorImpl._key_from_fd` with `dict.get` (#106555) Message-ID: https://github.com/python/cpython/commit/aeef8591e41b68341af308e56a744396c66879cc commit: aeef8591e41b68341af308e56a744396c66879cc branch: main author: J. Nick Koston committer: methane date: 2023-07-15T03:46:30+09:00 summary: gh-106554: replace `_BaseSelectorImpl._key_from_fd` with `dict.get` (#106555) files: A Misc/NEWS.d/next/Library/2023-07-09-01-59-24.gh-issue-106554.37c53J.rst M Lib/selectors.py diff --git a/Lib/selectors.py b/Lib/selectors.py index dfcc125dcd94e..6d82935445d4b 100644 --- a/Lib/selectors.py +++ b/Lib/selectors.py @@ -276,19 +276,6 @@ def close(self): def get_map(self): return self._map - def _key_from_fd(self, fd): - """Return the key associated to a given file descriptor. - - Parameters: - fd -- file descriptor - - Returns: - corresponding key, or None if not found - """ - try: - return self._fd_to_key[fd] - except KeyError: - return None class SelectSelector(_BaseSelectorImpl): @@ -336,7 +323,7 @@ def select(self, timeout=None): if fd in w: events |= EVENT_WRITE - key = self._key_from_fd(fd) + key = self._fd_to_key.get(fd) if key: ready.append((key, events & key.events)) return ready @@ -426,7 +413,7 @@ def select(self, timeout=None): if event & ~self._EVENT_WRITE: events |= EVENT_READ - key = self._key_from_fd(fd) + key = self._fd_to_key.get(fd) if key: ready.append((key, events & key.events)) return ready @@ -479,7 +466,7 @@ def select(self, timeout=None): if event & ~select.EPOLLOUT: events |= EVENT_READ - key = self._key_from_fd(fd) + key = self._fd_to_key.get(fd) if key: ready.append((key, events & key.events)) return ready @@ -574,7 +561,7 @@ def select(self, timeout=None): if flag == select.KQ_FILTER_WRITE: events |= EVENT_WRITE - key = self._key_from_fd(fd) + key = self._fd_to_key.get(fd) if key: ready.append((key, events & key.events)) return ready diff --git a/Misc/NEWS.d/next/Library/2023-07-09-01-59-24.gh-issue-106554.37c53J.rst b/Misc/NEWS.d/next/Library/2023-07-09-01-59-24.gh-issue-106554.37c53J.rst new file mode 100644 index 0000000000000..2136f3aa5a8eb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-09-01-59-24.gh-issue-106554.37c53J.rst @@ -0,0 +1 @@ +:mod:`selectors`: Reduce Selector overhead by using a ``dict.get()`` to lookup file descriptors. From webhook-mailer at python.org Fri Jul 14 14:49:06 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Fri, 14 Jul 2023 18:49:06 -0000 Subject: [Python-checkins] gh-106745: typing docs: Clarify that removal of PEP-585 aliases is not currently planned (#106748) Message-ID: https://github.com/python/cpython/commit/89ec0e952965b6a1be40e26c3ddc4131599e5ee9 commit: 89ec0e952965b6a1be40e26c3ddc4131599e5ee9 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-14T19:49:02+01:00 summary: gh-106745: typing docs: Clarify that removal of PEP-585 aliases is not currently planned (#106748) files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 0cf875582f7f4..0265a39ce646f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -3101,6 +3101,7 @@ Constant .. versionadded:: 3.5.2 .. _generic-concrete-collections: +.. _deprecated-aliases: Deprecated aliases ------------------ @@ -3109,16 +3110,21 @@ This module defines several deprecated aliases to pre-existing standard library classes. These were originally included in the typing module in order to support parameterizing these generic classes using ``[]``. However, the aliases became redundant in Python 3.9 when the -corresponding pre-existing classes were enhanced to support ``[]``. +corresponding pre-existing classes were enhanced to support ``[]`` (see +:pep:`585`). -The redundant types are deprecated as of Python 3.9 but no -deprecation warnings are issued by the interpreter. -It is expected that type checkers will flag the deprecated types -when the checked program targets Python 3.9 or newer. +The redundant types are deprecated as of Python 3.9. However, while the aliases +may be removed at some point, removal of these aliases is not currently +planned. As such, no deprecation warnings are currently issued by the +interpreter for these aliases. -The deprecated types will be removed from the :mod:`typing` module -no sooner than the first Python version released 5 years after the release of Python 3.9.0. -See details in :pep:`585`?*Type Hinting Generics In Standard Collections*. +If at some point it is decided to remove these deprecated aliases, a +deprecation warning will be issued by the interpreter for at least two releases +prior to removal. The aliases are guaranteed to remain in the typing module +without deprecation warnings until at least Python 3.14. + +Type checkers are encouraged to flag uses of the deprecated types if the +program they are checking targets a minimum Python version of 3.9 or newer. .. _corresponding-to-built-in-types: @@ -3651,20 +3657,34 @@ Certain features in ``typing`` are deprecated and may be removed in a future version of Python. The following table summarizes major deprecations for your convenience. This is subject to change, and not all deprecations are listed. -+-------------------------------------+---------------+-------------------+----------------+ -| Feature | Deprecated in | Projected removal | PEP/issue | -+=====================================+===============+===================+================+ -| ``typing`` versions of standard | 3.9 | Undecided | :pep:`585` | -| collections | | | | -+-------------------------------------+---------------+-------------------+----------------+ -| ``typing.ByteString`` | 3.9 | 3.14 | :gh:`91896` | -+-------------------------------------+---------------+-------------------+----------------+ -| ``typing.Text`` | 3.11 | Undecided | :gh:`92332` | -+-------------------------------------+---------------+-------------------+----------------+ -| ``typing.Hashable`` and | 3.12 | Undecided | :gh:`94309` | -| ``typing.Sized`` | | | | -+-------------------------------------+---------------+-------------------+----------------+ -| ``typing.TypeAlias`` | 3.12 | Undecided | :pep:`695` | -+-------------------------------------+---------------+-------------------+----------------+ -| ``typing.no_type_check_decorator`` | 3.13 | 3.15 | :gh:`106309` | -+-------------------------------------+---------------+-------------------+----------------+ +.. list-table:: + :header-rows: 1 + + * - Feature + - Deprecated in + - Projected removal + - PEP/issue + * - ``typing`` versions of standard collections + - 3.9 + - Undecided (see :ref:`deprecated-aliases` for more information) + - :pep:`585` + * - :class:`typing.ByteString` + - 3.9 + - 3.14 + - :gh:`91896` + * - :data:`typing.Text` + - 3.11 + - Undecided + - :gh:`92332` + * - :class:`typing.Hashable` and :class:`typing.Sized` + - 3.12 + - Undecided + - :gh:`94309` + * - :data:`typing.TypeAlias` + - 3.12 + - Undecided + - :pep:`695` + * - :func:`@typing.no_type_check_decorator ` + - 3.13 + - 3.15 + - :gh:`106309` From webhook-mailer at python.org Fri Jul 14 15:41:27 2023 From: webhook-mailer at python.org (iritkatriel) Date: Fri, 14 Jul 2023 19:41:27 -0000 Subject: [Python-checkins] gh-102799: replace internal sys.exc_info() call by sys.exception() (#106746) Message-ID: https://github.com/python/cpython/commit/fb32f35c0585b1dbb87b6f254818e1f485a50f65 commit: fb32f35c0585b1dbb87b6f254818e1f485a50f65 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-14T20:41:24+01:00 summary: gh-102799: replace internal sys.exc_info() call by sys.exception() (#106746) files: M Lib/logging/__init__.py diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py index 2a011b6d2ff15..46e86cb87ecfc 100644 --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -662,7 +662,7 @@ def formatException(self, ei): # See issues #9427, #1553375. Commented out for now. #if getattr(self, 'fullstack', False): # traceback.print_stack(tb.tb_frame.f_back, file=sio) - traceback.print_exception(ei[0], ei[1], tb, None, sio) + traceback.print_exception(ei[0], ei[1], tb, limit=None, file=sio) s = sio.getvalue() sio.close() if s[-1:] == "\n": @@ -1080,14 +1080,14 @@ def handleError(self, record): The record which was being processed is passed in to this method. """ if raiseExceptions and sys.stderr: # see issue 13807 - t, v, tb = sys.exc_info() + exc = sys.exception() try: sys.stderr.write('--- Logging error ---\n') - traceback.print_exception(t, v, tb, None, sys.stderr) + traceback.print_exception(exc, limit=None, file=sys.stderr) sys.stderr.write('Call stack:\n') # Walk the stack frame up until we're out of logging, # so as to print the calling context. - frame = tb.tb_frame + frame = exc.__traceback__.tb_frame while (frame and os.path.dirname(frame.f_code.co_filename) == __path__[0]): frame = frame.f_back @@ -1112,7 +1112,7 @@ def handleError(self, record): except OSError: #pragma: no cover pass # see issue 5971 finally: - del t, v, tb + del exc def __repr__(self): level = getLevelName(self.level) From webhook-mailer at python.org Fri Jul 14 16:40:49 2023 From: webhook-mailer at python.org (jaraco) Date: Fri, 14 Jul 2023 20:40:49 -0000 Subject: [Python-checkins] gh-106752: Move zipfile._path into its own package (#106753) Message-ID: https://github.com/python/cpython/commit/03185f0c150ebc52d41dd5ea6f369c7b5ba9fc16 commit: 03185f0c150ebc52d41dd5ea6f369c7b5ba9fc16 branch: main author: Jason R. Coombs committer: jaraco date: 2023-07-14T20:40:46Z summary: gh-106752: Move zipfile._path into its own package (#106753) * gh-106752: Move zipfile._path into its own package so it may have supplementary behavior. * Add blurb files: A Lib/test/test_zipfile/_path/__init__.py A Lib/test/test_zipfile/_path/_functools.py A Lib/test/test_zipfile/_path/_itertools.py A Lib/test/test_zipfile/_path/_support.py A Lib/test/test_zipfile/_path/_test_params.py A Lib/test/test_zipfile/_path/test_complexity.py A Lib/test/test_zipfile/_path/test_path.py A Lib/zipfile/_path/__init__.py A Misc/NEWS.d/next/Tests/2023-07-14-16-20-06.gh-issue-106752.gd1i6D.rst D Lib/test/test_zipfile/_functools.py D Lib/test/test_zipfile/_itertools.py D Lib/test/test_zipfile/_support.py D Lib/test/test_zipfile/_test_params.py D Lib/test/test_zipfile/test_complexity.py D Lib/test/test_zipfile/test_path.py D Lib/zipfile/_path.py M .github/CODEOWNERS M Makefile.pre.in diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 5b471c79f75ee..234a954cc7662 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -172,7 +172,7 @@ Doc/c-api/stable.rst @encukou **/*pathlib* @barneygale # zipfile.Path -**/*zipfile/*_path.py @jaraco +**/*zipfile/_path/* @jaraco # Argument Clinic /Tools/clinic/** @erlend-aasland @AlexWaygood diff --git a/Lib/test/test_zipfile/_path/__init__.py b/Lib/test/test_zipfile/_path/__init__.py new file mode 100644 index 0000000000000..e69de29bb2d1d diff --git a/Lib/test/test_zipfile/_functools.py b/Lib/test/test_zipfile/_path/_functools.py similarity index 100% rename from Lib/test/test_zipfile/_functools.py rename to Lib/test/test_zipfile/_path/_functools.py diff --git a/Lib/test/test_zipfile/_itertools.py b/Lib/test/test_zipfile/_path/_itertools.py similarity index 100% rename from Lib/test/test_zipfile/_itertools.py rename to Lib/test/test_zipfile/_path/_itertools.py diff --git a/Lib/test/test_zipfile/_support.py b/Lib/test/test_zipfile/_path/_support.py similarity index 100% rename from Lib/test/test_zipfile/_support.py rename to Lib/test/test_zipfile/_path/_support.py diff --git a/Lib/test/test_zipfile/_test_params.py b/Lib/test/test_zipfile/_path/_test_params.py similarity index 100% rename from Lib/test/test_zipfile/_test_params.py rename to Lib/test/test_zipfile/_path/_test_params.py diff --git a/Lib/test/test_zipfile/test_complexity.py b/Lib/test/test_zipfile/_path/test_complexity.py similarity index 100% rename from Lib/test/test_zipfile/test_complexity.py rename to Lib/test/test_zipfile/_path/test_complexity.py diff --git a/Lib/test/test_zipfile/test_path.py b/Lib/test/test_zipfile/_path/test_path.py similarity index 100% rename from Lib/test/test_zipfile/test_path.py rename to Lib/test/test_zipfile/_path/test_path.py diff --git a/Lib/zipfile/_path.py b/Lib/zipfile/_path/__init__.py similarity index 100% rename from Lib/zipfile/_path.py rename to Lib/zipfile/_path/__init__.py diff --git a/Makefile.pre.in b/Makefile.pre.in index 4b655513f656d..ddf524ac17d72 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -2125,7 +2125,7 @@ LIBSUBDIRS= asyncio \ wsgiref \ $(XMLLIBSUBDIRS) \ xmlrpc \ - zipfile \ + zipfile zipfile/_path \ zoneinfo \ __phello__ TESTSUBDIRS= idlelib/idle_test \ @@ -2229,6 +2229,7 @@ TESTSUBDIRS= idlelib/idle_test \ test/test_warnings \ test/test_warnings/data \ test/test_zipfile \ + test/test_zipfile/_path \ test/test_zoneinfo \ test/test_zoneinfo/data \ test/tkinterdata \ diff --git a/Misc/NEWS.d/next/Tests/2023-07-14-16-20-06.gh-issue-106752.gd1i6D.rst b/Misc/NEWS.d/next/Tests/2023-07-14-16-20-06.gh-issue-106752.gd1i6D.rst new file mode 100644 index 0000000000000..ba7257e361080 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-14-16-20-06.gh-issue-106752.gd1i6D.rst @@ -0,0 +1,2 @@ +Moved tests for ``zipfile.Path`` into ``Lib/test/test_zipfile/_path``. Made +``zipfile._path`` a package. From webhook-mailer at python.org Fri Jul 14 17:11:16 2023 From: webhook-mailer at python.org (jaraco) Date: Fri, 14 Jul 2023 21:11:16 -0000 Subject: [Python-checkins] [3.12] gh-106752: Move zipfile._path into its own package (GH-106753) (#106755) Message-ID: https://github.com/python/cpython/commit/30f62748e99ef2af3bfbac0e2d84dccf48c81512 commit: 30f62748e99ef2af3bfbac0e2d84dccf48c81512 branch: 3.12 author: Jason R. Coombs committer: jaraco date: 2023-07-14T21:11:13Z summary: [3.12] gh-106752: Move zipfile._path into its own package (GH-106753) (#106755) * gh-106752: Move zipfile._path into its own package so it may have supplementary behavior. * Add blurb. (cherry picked from commit 03185f0c150ebc52d41dd5ea6f369c7b5ba9fc16) files: A Lib/test/test_zipfile/_path/__init__.py A Lib/test/test_zipfile/_path/_functools.py A Lib/test/test_zipfile/_path/_itertools.py A Lib/test/test_zipfile/_path/_support.py A Lib/test/test_zipfile/_path/_test_params.py A Lib/test/test_zipfile/_path/test_complexity.py A Lib/test/test_zipfile/_path/test_path.py A Lib/zipfile/_path/__init__.py A Misc/NEWS.d/next/Tests/2023-07-14-16-20-06.gh-issue-106752.gd1i6D.rst D Lib/test/test_zipfile/_functools.py D Lib/test/test_zipfile/_itertools.py D Lib/test/test_zipfile/_support.py D Lib/test/test_zipfile/_test_params.py D Lib/test/test_zipfile/test_complexity.py D Lib/test/test_zipfile/test_path.py D Lib/zipfile/_path.py M .github/CODEOWNERS M Makefile.pre.in diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 32ba5355a5853..95fd51b743cd3 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -164,4 +164,4 @@ Lib/ast.py @isidentical **/*pathlib* @barneygale # zipfile.Path -**/*zipfile/*_path.py @jaraco +**/*zipfile/_path/* @jaraco diff --git a/Lib/test/test_zipfile/_path/__init__.py b/Lib/test/test_zipfile/_path/__init__.py new file mode 100644 index 0000000000000..e69de29bb2d1d diff --git a/Lib/test/test_zipfile/_functools.py b/Lib/test/test_zipfile/_path/_functools.py similarity index 100% rename from Lib/test/test_zipfile/_functools.py rename to Lib/test/test_zipfile/_path/_functools.py diff --git a/Lib/test/test_zipfile/_itertools.py b/Lib/test/test_zipfile/_path/_itertools.py similarity index 100% rename from Lib/test/test_zipfile/_itertools.py rename to Lib/test/test_zipfile/_path/_itertools.py diff --git a/Lib/test/test_zipfile/_support.py b/Lib/test/test_zipfile/_path/_support.py similarity index 100% rename from Lib/test/test_zipfile/_support.py rename to Lib/test/test_zipfile/_path/_support.py diff --git a/Lib/test/test_zipfile/_test_params.py b/Lib/test/test_zipfile/_path/_test_params.py similarity index 100% rename from Lib/test/test_zipfile/_test_params.py rename to Lib/test/test_zipfile/_path/_test_params.py diff --git a/Lib/test/test_zipfile/test_complexity.py b/Lib/test/test_zipfile/_path/test_complexity.py similarity index 100% rename from Lib/test/test_zipfile/test_complexity.py rename to Lib/test/test_zipfile/_path/test_complexity.py diff --git a/Lib/test/test_zipfile/test_path.py b/Lib/test/test_zipfile/_path/test_path.py similarity index 100% rename from Lib/test/test_zipfile/test_path.py rename to Lib/test/test_zipfile/_path/test_path.py diff --git a/Lib/zipfile/_path.py b/Lib/zipfile/_path/__init__.py similarity index 100% rename from Lib/zipfile/_path.py rename to Lib/zipfile/_path/__init__.py diff --git a/Makefile.pre.in b/Makefile.pre.in index 12788d11d1d14..8dacb570ccafa 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -2110,7 +2110,7 @@ LIBSUBDIRS= asyncio \ wsgiref \ $(XMLLIBSUBDIRS) \ xmlrpc \ - zipfile \ + zipfile zipfile/_path \ zoneinfo \ __phello__ TESTSUBDIRS= idlelib/idle_test \ @@ -2220,6 +2220,7 @@ TESTSUBDIRS= idlelib/idle_test \ test/test_warnings \ test/test_warnings/data \ test/test_zipfile \ + test/test_zipfile/_path \ test/test_zoneinfo \ test/test_zoneinfo/data \ test/tracedmodules \ diff --git a/Misc/NEWS.d/next/Tests/2023-07-14-16-20-06.gh-issue-106752.gd1i6D.rst b/Misc/NEWS.d/next/Tests/2023-07-14-16-20-06.gh-issue-106752.gd1i6D.rst new file mode 100644 index 0000000000000..ba7257e361080 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-14-16-20-06.gh-issue-106752.gd1i6D.rst @@ -0,0 +1,2 @@ +Moved tests for ``zipfile.Path`` into ``Lib/test/test_zipfile/_path``. Made +``zipfile._path`` a package. From webhook-mailer at python.org Fri Jul 14 20:22:09 2023 From: webhook-mailer at python.org (gvanrossum) Date: Sat, 15 Jul 2023 00:22:09 -0000 Subject: [Python-checkins] gh-106529: Fix subtle Tier 2 edge case with list iterator (#106756) Message-ID: https://github.com/python/cpython/commit/0db85eeba762e72f9f3c027e432cdebc627aac6c commit: 0db85eeba762e72f9f3c027e432cdebc627aac6c branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-14T17:22:06-07:00 summary: gh-106529: Fix subtle Tier 2 edge case with list iterator (#106756) The Tier 2 opcode _IS_ITER_EXHAUSTED_LIST (and _TUPLE) didn't set it->it_seq to NULL, causing a subtle bug that resulted in test_exhausted_iterator in list_tests.py to fail when running all tests with -Xuops. The bug was introduced in gh-106696. Added this as an explicit test. Also fixed the dependencies for ceval.o -- it depends on executor_cases.c.h. files: M Lib/test/test_capi/test_misc.py M Makefile.pre.in M Python/bytecodes.c M Python/executor_cases.c.h diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 43c04463236a2..6df918997b2b1 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2649,6 +2649,19 @@ def testfunc(a): # Verification that the jump goes past END_FOR # is done by manual inspection of the output + def test_list_edge_case(self): + def testfunc(it): + for x in it: + pass + + opt = _testinternalcapi.get_uop_optimizer() + with temporary_optimizer(opt): + a = [1, 2, 3] + it = iter(a) + testfunc(it) + a.append(4) + with self.assertRaises(StopIteration): + next(it) if __name__ == "__main__": unittest.main() diff --git a/Makefile.pre.in b/Makefile.pre.in index ddf524ac17d72..553b2aa480c18 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1564,6 +1564,7 @@ Python/ceval.o: \ $(srcdir)/Python/ceval_macros.h \ $(srcdir)/Python/condvar.h \ $(srcdir)/Python/generated_cases.c.h \ + $(srcdir)/Python/executor_cases.c.h \ $(srcdir)/Include/internal/pycore_opcode_metadata.h \ $(srcdir)/Python/opcode_targets.h diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 15b48ae9d8267..3432b02771346 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2448,7 +2448,12 @@ dummy_func( _PyListIterObject *it = (_PyListIterObject *)iter; assert(Py_TYPE(iter) == &PyListIter_Type); PyListObject *seq = it->it_seq; - if (seq == NULL || it->it_index >= PyList_GET_SIZE(seq)) { + if (seq == NULL) { + exhausted = Py_True; + } + else if (it->it_index >= PyList_GET_SIZE(seq)) { + Py_DECREF(seq); + it->it_seq = NULL; exhausted = Py_True; } else { @@ -2499,7 +2504,12 @@ dummy_func( _PyTupleIterObject *it = (_PyTupleIterObject *)iter; assert(Py_TYPE(iter) == &PyTupleIter_Type); PyTupleObject *seq = it->it_seq; - if (seq == NULL || it->it_index >= PyTuple_GET_SIZE(seq)) { + if (seq == NULL) { + exhausted = Py_True; + } + else if (it->it_index >= PyTuple_GET_SIZE(seq)) { + Py_DECREF(seq); + it->it_seq = NULL; exhausted = Py_True; } else { diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 626baece81460..ae21ffad94d80 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1750,7 +1750,12 @@ _PyListIterObject *it = (_PyListIterObject *)iter; assert(Py_TYPE(iter) == &PyListIter_Type); PyListObject *seq = it->it_seq; - if (seq == NULL || it->it_index >= PyList_GET_SIZE(seq)) { + if (seq == NULL) { + exhausted = Py_True; + } + else if (it->it_index >= PyList_GET_SIZE(seq)) { + Py_DECREF(seq); + it->it_seq = NULL; exhausted = Py_True; } else { @@ -1787,7 +1792,12 @@ _PyTupleIterObject *it = (_PyTupleIterObject *)iter; assert(Py_TYPE(iter) == &PyTupleIter_Type); PyTupleObject *seq = it->it_seq; - if (seq == NULL || it->it_index >= PyTuple_GET_SIZE(seq)) { + if (seq == NULL) { + exhausted = Py_True; + } + else if (it->it_index >= PyTuple_GET_SIZE(seq)) { + Py_DECREF(seq); + it->it_seq = NULL; exhausted = Py_True; } else { From webhook-mailer at python.org Fri Jul 14 22:15:17 2023 From: webhook-mailer at python.org (sweeneyde) Date: Sat, 15 Jul 2023 02:15:17 -0000 Subject: [Python-checkins] [3.12] gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252) (#106708) Message-ID: https://github.com/python/cpython/commit/4f3edd6b535b6a0b7352df134c0f445ab279bfc0 commit: 4f3edd6b535b6a0b7352df134c0f445ab279bfc0 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2023-07-14T22:15:14-04:00 summary: [3.12] gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252) (#106708) gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252) * Add a special case for s[-m:] == p in _PyBytes_Find * Add tests for _PyBytes_Find * Make sure that start <= end in mmap.find (cherry picked from commit ab86426a3472ab68747815299d390b213793c3d1) Co-authored-by: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> files: A Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst M Lib/test/test_mmap.py M Modules/_testinternalcapi.c M Modules/mmapmodule.c M Objects/bytesobject.c diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py index 517cbe0cb115a..bab868600895c 100644 --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -299,6 +299,27 @@ def test_find_end(self): self.assertEqual(m.find(b'one', 1, -2), -1) self.assertEqual(m.find(bytearray(b'one')), 0) + for i in range(-n-1, n+1): + for j in range(-n-1, n+1): + for p in [b"o", b"on", b"two", b"ones", b"s"]: + expected = data.find(p, i, j) + self.assertEqual(m.find(p, i, j), expected, (p, i, j)) + + def test_find_does_not_access_beyond_buffer(self): + try: + flags = mmap.MAP_PRIVATE | mmap.MAP_ANONYMOUS + PAGESIZE = mmap.PAGESIZE + PROT_NONE = 0 + PROT_READ = mmap.PROT_READ + except AttributeError as e: + raise unittest.SkipTest("mmap flags unavailable") from e + for i in range(0, 2049): + with mmap.mmap(-1, PAGESIZE * (i + 1), + flags=flags, prot=PROT_NONE) as guard: + with mmap.mmap(-1, PAGESIZE * (i + 2048), + flags=flags, prot=PROT_READ) as fm: + fm.find(b"fo", -2) + def test_rfind(self): # test the new 'end' parameter works as expected diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst new file mode 100644 index 0000000000000..c28d0101cd4ba --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst @@ -0,0 +1 @@ +Prevent out-of-bounds memory access during ``mmap.find()`` calls. diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 7e2def7c20b1b..98f99e4d66cf1 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -15,6 +15,7 @@ #include "frameobject.h" #include "pycore_atomic_funcs.h" // _Py_atomic_int_get() #include "pycore_bitutils.h" // _Py_bswap32() +#include "pycore_bytesobject.h" // _PyBytes_Find() #include "pycore_compile.h" // _PyCompile_CodeGen, _PyCompile_OptimizeCfg, _PyCompile_Assemble #include "pycore_ceval.h" // _PyEval_AddPendingCall #include "pycore_fileutils.h" // _Py_normpath @@ -441,6 +442,118 @@ test_edit_cost(PyObject *self, PyObject *Py_UNUSED(args)) } +static int +check_bytes_find(const char *haystack0, const char *needle0, + int offset, Py_ssize_t expected) +{ + Py_ssize_t len_haystack = strlen(haystack0); + Py_ssize_t len_needle = strlen(needle0); + Py_ssize_t result_1 = _PyBytes_Find(haystack0, len_haystack, + needle0, len_needle, offset); + if (result_1 != expected) { + PyErr_Format(PyExc_AssertionError, + "Incorrect result_1: '%s' in '%s' (offset=%zd)", + needle0, haystack0, offset); + return -1; + } + // Allocate new buffer with no NULL terminator. + char *haystack = PyMem_Malloc(len_haystack); + if (haystack == NULL) { + PyErr_NoMemory(); + return -1; + } + char *needle = PyMem_Malloc(len_needle); + if (needle == NULL) { + PyMem_Free(haystack); + PyErr_NoMemory(); + return -1; + } + memcpy(haystack, haystack0, len_haystack); + memcpy(needle, needle0, len_needle); + Py_ssize_t result_2 = _PyBytes_Find(haystack, len_haystack, + needle, len_needle, offset); + PyMem_Free(haystack); + PyMem_Free(needle); + if (result_2 != expected) { + PyErr_Format(PyExc_AssertionError, + "Incorrect result_2: '%s' in '%s' (offset=%zd)", + needle0, haystack0, offset); + return -1; + } + return 0; +} + +static int +check_bytes_find_large(Py_ssize_t len_haystack, Py_ssize_t len_needle, + const char *needle) +{ + char *zeros = PyMem_RawCalloc(len_haystack, 1); + if (zeros == NULL) { + PyErr_NoMemory(); + return -1; + } + Py_ssize_t res = _PyBytes_Find(zeros, len_haystack, needle, len_needle, 0); + PyMem_RawFree(zeros); + if (res != -1) { + PyErr_Format(PyExc_AssertionError, + "check_bytes_find_large(%zd, %zd) found %zd", + len_haystack, len_needle, res); + return -1; + } + return 0; +} + +static PyObject * +test_bytes_find(PyObject *self, PyObject *Py_UNUSED(args)) +{ + #define CHECK(H, N, O, E) do { \ + if (check_bytes_find(H, N, O, E) < 0) { \ + return NULL; \ + } \ + } while (0) + + CHECK("", "", 0, 0); + CHECK("Python", "", 0, 0); + CHECK("Python", "", 3, 3); + CHECK("Python", "", 6, 6); + CHECK("Python", "yth", 0, 1); + CHECK("ython", "yth", 1, 1); + CHECK("thon", "yth", 2, -1); + CHECK("Python", "thon", 0, 2); + CHECK("ython", "thon", 1, 2); + CHECK("thon", "thon", 2, 2); + CHECK("hon", "thon", 3, -1); + CHECK("Pytho", "zz", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "ab", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "ba", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "bb", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab", "ab", 0, 30); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaba", "ba", 0, 30); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaabb", "bb", 0, 30); + #undef CHECK + + // Hunt for segfaults + // n, m chosen here so that (n - m) % (m + 1) == 0 + // This would make default_find in fastsearch.h access haystack[n]. + if (check_bytes_find_large(2048, 2, "ab") < 0) { + return NULL; + } + if (check_bytes_find_large(4096, 16, "0123456789abcdef") < 0) { + return NULL; + } + if (check_bytes_find_large(8192, 2, "ab") < 0) { + return NULL; + } + if (check_bytes_find_large(16384, 4, "abcd") < 0) { + return NULL; + } + if (check_bytes_find_large(32768, 2, "ab") < 0) { + return NULL; + } + Py_RETURN_NONE; +} + + static PyObject * normalize_path(PyObject *self, PyObject *filename) { @@ -950,6 +1063,7 @@ static PyMethodDef module_functions[] = { {"reset_path_config", test_reset_path_config, METH_NOARGS}, {"test_atomic_funcs", test_atomic_funcs, METH_NOARGS}, {"test_edit_cost", test_edit_cost, METH_NOARGS}, + {"test_bytes_find", test_bytes_find, METH_NOARGS}, {"normalize_path", normalize_path, METH_O, NULL}, {"get_getpath_codeobject", get_getpath_codeobject, METH_NOARGS, NULL}, {"EncodeLocaleEx", encode_locale_ex, METH_VARARGS}, diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index 6bde9939eaa2c..1ffadccb91625 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -343,12 +343,17 @@ mmap_gfind(mmap_object *self, Py_ssize_t res; CHECK_VALID_OR_RELEASE(NULL, view); - if (reverse) { + if (end < start) { + res = -1; + } + else if (reverse) { + assert(0 <= start && start <= end && end <= self->size); res = _PyBytes_ReverseFind( self->data + start, end - start, view.buf, view.len, start); } else { + assert(0 <= start && start <= end && end <= self->size); res = _PyBytes_Find( self->data + start, end - start, view.buf, view.len, start); diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c index abbf3eeb16c35..f3a978c86c360 100644 --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1274,8 +1274,25 @@ _PyBytes_Find(const char *haystack, Py_ssize_t len_haystack, const char *needle, Py_ssize_t len_needle, Py_ssize_t offset) { - return stringlib_find(haystack, len_haystack, - needle, len_needle, offset); + assert(len_haystack >= 0); + assert(len_needle >= 0); + // Extra checks because stringlib_find accesses haystack[len_haystack]. + if (len_needle == 0) { + return offset; + } + if (len_needle > len_haystack) { + return -1; + } + assert(len_haystack >= 1); + Py_ssize_t res = stringlib_find(haystack, len_haystack - 1, + needle, len_needle, offset); + if (res == -1) { + Py_ssize_t last_align = len_haystack - len_needle; + if (memcmp(haystack + last_align, needle, len_needle) == 0) { + return offset + last_align; + } + } + return res; } Py_ssize_t From webhook-mailer at python.org Fri Jul 14 22:17:12 2023 From: webhook-mailer at python.org (sweeneyde) Date: Sat, 15 Jul 2023 02:17:12 -0000 Subject: [Python-checkins] =?utf-8?q?=5B3=2E11=5D_gh-105235=3A_Prevent_re?= =?utf-8?q?ading_outside_buffer_during_mmap=2Efind=28=29_=28=E2=80=A6_=28?= =?utf-8?q?=23106710=29?= Message-ID: https://github.com/python/cpython/commit/d488970ae6e4b0de1212e202602be686de49ab7a commit: d488970ae6e4b0de1212e202602be686de49ab7a branch: 3.11 author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2023-07-14T22:17:09-04:00 summary: [3.11] gh-105235: Prevent reading outside buffer during mmap.find() (? (#106710) [3.11] gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252) * Add a special case for s[-m:] == p in _PyBytes_Find * Add tests for _PyBytes_Find * Make sure that start <= end in mmap.find. (cherry picked from commit ab86426a3472ab68747815299d390b213793c3d1) files: A Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst M Lib/test/test_mmap.py M Modules/_testinternalcapi.c M Modules/mmapmodule.c M Objects/bytesobject.c diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py index 517cbe0cb115a..bab868600895c 100644 --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -299,6 +299,27 @@ def test_find_end(self): self.assertEqual(m.find(b'one', 1, -2), -1) self.assertEqual(m.find(bytearray(b'one')), 0) + for i in range(-n-1, n+1): + for j in range(-n-1, n+1): + for p in [b"o", b"on", b"two", b"ones", b"s"]: + expected = data.find(p, i, j) + self.assertEqual(m.find(p, i, j), expected, (p, i, j)) + + def test_find_does_not_access_beyond_buffer(self): + try: + flags = mmap.MAP_PRIVATE | mmap.MAP_ANONYMOUS + PAGESIZE = mmap.PAGESIZE + PROT_NONE = 0 + PROT_READ = mmap.PROT_READ + except AttributeError as e: + raise unittest.SkipTest("mmap flags unavailable") from e + for i in range(0, 2049): + with mmap.mmap(-1, PAGESIZE * (i + 1), + flags=flags, prot=PROT_NONE) as guard: + with mmap.mmap(-1, PAGESIZE * (i + 2048), + flags=flags, prot=PROT_READ) as fm: + fm.find(b"fo", -2) + def test_rfind(self): # test the new 'end' parameter works as expected diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst b/Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst new file mode 100644 index 0000000000000..c28d0101cd4ba --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-06-02-19-37-29.gh-issue-105235.fgFGTi.rst @@ -0,0 +1 @@ +Prevent out-of-bounds memory access during ``mmap.find()`` calls. diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 238de749fffc5..4a8c43658998f 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -14,6 +14,7 @@ #include "Python.h" #include "pycore_atomic_funcs.h" // _Py_atomic_int_get() #include "pycore_bitutils.h" // _Py_bswap32() +#include "pycore_bytesobject.h" // _PyBytes_Find() #include "pycore_fileutils.h" // _Py_normpath #include "pycore_frame.h" // _PyInterpreterFrame #include "pycore_gc.h" // PyGC_Head @@ -380,6 +381,118 @@ test_edit_cost(PyObject *self, PyObject *Py_UNUSED(args)) } +static int +check_bytes_find(const char *haystack0, const char *needle0, + int offset, Py_ssize_t expected) +{ + Py_ssize_t len_haystack = strlen(haystack0); + Py_ssize_t len_needle = strlen(needle0); + Py_ssize_t result_1 = _PyBytes_Find(haystack0, len_haystack, + needle0, len_needle, offset); + if (result_1 != expected) { + PyErr_Format(PyExc_AssertionError, + "Incorrect result_1: '%s' in '%s' (offset=%zd)", + needle0, haystack0, offset); + return -1; + } + // Allocate new buffer with no NULL terminator. + char *haystack = PyMem_Malloc(len_haystack); + if (haystack == NULL) { + PyErr_NoMemory(); + return -1; + } + char *needle = PyMem_Malloc(len_needle); + if (needle == NULL) { + PyMem_Free(haystack); + PyErr_NoMemory(); + return -1; + } + memcpy(haystack, haystack0, len_haystack); + memcpy(needle, needle0, len_needle); + Py_ssize_t result_2 = _PyBytes_Find(haystack, len_haystack, + needle, len_needle, offset); + PyMem_Free(haystack); + PyMem_Free(needle); + if (result_2 != expected) { + PyErr_Format(PyExc_AssertionError, + "Incorrect result_2: '%s' in '%s' (offset=%zd)", + needle0, haystack0, offset); + return -1; + } + return 0; +} + +static int +check_bytes_find_large(Py_ssize_t len_haystack, Py_ssize_t len_needle, + const char *needle) +{ + char *zeros = PyMem_RawCalloc(len_haystack, 1); + if (zeros == NULL) { + PyErr_NoMemory(); + return -1; + } + Py_ssize_t res = _PyBytes_Find(zeros, len_haystack, needle, len_needle, 0); + PyMem_RawFree(zeros); + if (res != -1) { + PyErr_Format(PyExc_AssertionError, + "check_bytes_find_large(%zd, %zd) found %zd", + len_haystack, len_needle, res); + return -1; + } + return 0; +} + +static PyObject * +test_bytes_find(PyObject *self, PyObject *Py_UNUSED(args)) +{ + #define CHECK(H, N, O, E) do { \ + if (check_bytes_find(H, N, O, E) < 0) { \ + return NULL; \ + } \ + } while (0) + + CHECK("", "", 0, 0); + CHECK("Python", "", 0, 0); + CHECK("Python", "", 3, 3); + CHECK("Python", "", 6, 6); + CHECK("Python", "yth", 0, 1); + CHECK("ython", "yth", 1, 1); + CHECK("thon", "yth", 2, -1); + CHECK("Python", "thon", 0, 2); + CHECK("ython", "thon", 1, 2); + CHECK("thon", "thon", 2, 2); + CHECK("hon", "thon", 3, -1); + CHECK("Pytho", "zz", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "ab", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "ba", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "bb", 0, -1); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab", "ab", 0, 30); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaba", "ba", 0, 30); + CHECK("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaabb", "bb", 0, 30); + #undef CHECK + + // Hunt for segfaults + // n, m chosen here so that (n - m) % (m + 1) == 0 + // This would make default_find in fastsearch.h access haystack[n]. + if (check_bytes_find_large(2048, 2, "ab") < 0) { + return NULL; + } + if (check_bytes_find_large(4096, 16, "0123456789abcdef") < 0) { + return NULL; + } + if (check_bytes_find_large(8192, 2, "ab") < 0) { + return NULL; + } + if (check_bytes_find_large(16384, 4, "abcd") < 0) { + return NULL; + } + if (check_bytes_find_large(32768, 2, "ab") < 0) { + return NULL; + } + Py_RETURN_NONE; +} + + static PyObject * normalize_path(PyObject *self, PyObject *filename) { @@ -537,6 +650,7 @@ static PyMethodDef TestMethods[] = { {"reset_path_config", test_reset_path_config, METH_NOARGS}, {"test_atomic_funcs", test_atomic_funcs, METH_NOARGS}, {"test_edit_cost", test_edit_cost, METH_NOARGS}, + {"test_bytes_find", test_bytes_find, METH_NOARGS}, {"normalize_path", normalize_path, METH_O, NULL}, {"get_getpath_codeobject", get_getpath_codeobject, METH_NOARGS, NULL}, {"EncodeLocaleEx", encode_locale_ex, METH_VARARGS}, diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index 4b0c1e27e0c53..8ff63f51188ad 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -351,12 +351,17 @@ mmap_gfind(mmap_object *self, Py_ssize_t res; CHECK_VALID_OR_RELEASE(NULL, view); - if (reverse) { + if (end < start) { + res = -1; + } + else if (reverse) { + assert(0 <= start && start <= end && end <= self->size); res = _PyBytes_ReverseFind( self->data + start, end - start, view.buf, view.len, start); } else { + assert(0 <= start && start <= end && end <= self->size); res = _PyBytes_Find( self->data + start, end - start, view.buf, view.len, start); diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c index 3b0ff9a19d977..279579f63418c 100644 --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1283,8 +1283,25 @@ _PyBytes_Find(const char *haystack, Py_ssize_t len_haystack, const char *needle, Py_ssize_t len_needle, Py_ssize_t offset) { - return stringlib_find(haystack, len_haystack, - needle, len_needle, offset); + assert(len_haystack >= 0); + assert(len_needle >= 0); + // Extra checks because stringlib_find accesses haystack[len_haystack]. + if (len_needle == 0) { + return offset; + } + if (len_needle > len_haystack) { + return -1; + } + assert(len_haystack >= 1); + Py_ssize_t res = stringlib_find(haystack, len_haystack - 1, + needle, len_needle, offset); + if (res == -1) { + Py_ssize_t last_align = len_haystack - len_needle; + if (memcmp(haystack + last_align, needle, len_needle) == 0) { + return offset + last_align; + } + } + return res; } Py_ssize_t From webhook-mailer at python.org Sat Jul 15 05:29:00 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 15 Jul 2023 09:29:00 -0000 Subject: [Python-checkins] gh-106368: Increase Argument Clinic BlockParser test coverage (#106759) Message-ID: https://github.com/python/cpython/commit/2d7d1aa4bcd5da0177458b22b1b856db76aa20d4 commit: 2d7d1aa4bcd5da0177458b22b1b856db76aa20d4 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-15T11:28:57+02:00 summary: gh-106368: Increase Argument Clinic BlockParser test coverage (#106759) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 975840333e590..b5744f7013d6a 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -18,6 +18,19 @@ from clinic import DSLParser +class _ParserBase(TestCase): + maxDiff = None + + def expect_parser_failure(self, parser, _input): + with support.captured_stdout() as stdout: + with self.assertRaises(SystemExit): + parser(_input) + return stdout.getvalue() + + def parse_function_should_fail(self, _input): + return self.expect_parser_failure(self.parse_function, _input) + + class FakeConverter: def __init__(self, name, args): self.name = name @@ -88,7 +101,15 @@ def directive(self, name, args): _module_and_class = clinic.Clinic._module_and_class -class ClinicWholeFileTest(TestCase): + +class ClinicWholeFileTest(_ParserBase): + def setUp(self): + self.clinic = clinic.Clinic(clinic.CLanguage(None), filename="test.c") + + def expect_failure(self, raw): + _input = dedent(raw).strip() + return self.expect_parser_failure(self.clinic.parse, _input) + def test_eol(self): # regression test: # clinic's block parser didn't recognize @@ -98,15 +119,86 @@ def test_eol(self): # so it would spit out an end line for you. # and since you really already had one, # the last line of the block got corrupted. - c = clinic.Clinic(clinic.CLanguage(None), filename="file") raw = "/*[clinic]\nfoo\n[clinic]*/" - cooked = c.parse(raw).splitlines() + cooked = self.clinic.parse(raw).splitlines() end_line = cooked[2].rstrip() # this test is redundant, it's just here explicitly to catch # the regression test so we don't forget what it looked like self.assertNotEqual(end_line, "[clinic]*/[clinic]*/") self.assertEqual(end_line, "[clinic]*/") + def test_mangled_marker_line(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/ + /*[clinic end generated code: foo]*/ + """ + msg = ( + 'Error in file "test.c" on line 3:\n' + "Mangled Argument Clinic marker line: '/*[clinic end generated code: foo]*/'\n" + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_checksum_mismatch(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/ + /*[clinic end generated code: output=0123456789abcdef input=fedcba9876543210]*/ + """ + msg = ( + 'Error in file "test.c" on line 3:\n' + 'Checksum mismatch!\n' + 'Expected: 0123456789abcdef\n' + 'Computed: da39a3ee5e6b4b0d\n' + ) + out = self.expect_failure(raw) + self.assertIn(msg, out) + + def test_garbage_after_stop_line(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/foobarfoobar! + """ + msg = ( + 'Error in file "test.c" on line 2:\n' + "Garbage after stop line: 'foobarfoobar!'\n" + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_whitespace_before_stop_line(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/ + """ + msg = ( + 'Error in file "test.c" on line 2:\n' + "Whitespace is not allowed before the stop line: ' [clinic start generated code]*/'\n" + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_parse_with_body_prefix(self): + clang = clinic.CLanguage(None) + clang.body_prefix = "//" + clang.start_line = "//[{dsl_name} start]" + clang.stop_line = "//[{dsl_name} stop]" + cl = clinic.Clinic(clang, filename="test.c") + raw = dedent(""" + //[clinic start] + //module test + //[clinic stop] + """).strip() + out = cl.parse(raw) + expected = dedent(""" + //[clinic start] + //module test + // + //[clinic stop] + /*[clinic end generated code: output=da39a3ee5e6b4b0d input=65fab8adff58cf08]*/ + """).lstrip() # Note, lstrip() because of the newline + self.assertEqual(out, expected) class ClinicGroupPermuterTest(TestCase): @@ -285,7 +377,7 @@ def test_clinic_1(self): """) -class ClinicParserTest(TestCase): +class ClinicParserTest(_ParserBase): def checkDocstring(self, fn, expected): self.assertTrue(hasattr(fn, "docstring")) self.assertEqual(fn.docstring.strip(), From webhook-mailer at python.org Sat Jul 15 05:48:04 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 15 Jul 2023 09:48:04 -0000 Subject: [Python-checkins] [3.11] gh-106368: Increase Argument Clinic BlockParser test coverage (GH-106759) (#106770) Message-ID: https://github.com/python/cpython/commit/b132b77fb11a6c9b8de5cf0f4afb8acc7f57a96c commit: b132b77fb11a6c9b8de5cf0f4afb8acc7f57a96c branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-15T09:48:01Z summary: [3.11] gh-106368: Increase Argument Clinic BlockParser test coverage (GH-106759) (#106770) (cherry picked from commit 2d7d1aa4bcd5da0177458b22b1b856db76aa20d4) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index fa65a41ec30d0..0e3fbb7627ab5 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -18,6 +18,19 @@ from clinic import DSLParser +class _ParserBase(TestCase): + maxDiff = None + + def expect_parser_failure(self, parser, _input): + with support.captured_stdout() as stdout: + with self.assertRaises(SystemExit): + parser(_input) + return stdout.getvalue() + + def parse_function_should_fail(self, _input): + return self.expect_parser_failure(self.parse_function, _input) + + class FakeConverter: def __init__(self, name, args): self.name = name @@ -88,7 +101,15 @@ def directive(self, name, args): _module_and_class = clinic.Clinic._module_and_class -class ClinicWholeFileTest(TestCase): + +class ClinicWholeFileTest(_ParserBase): + def setUp(self): + self.clinic = clinic.Clinic(clinic.CLanguage(None), filename="test.c") + + def expect_failure(self, raw): + _input = dedent(raw).strip() + return self.expect_parser_failure(self.clinic.parse, _input) + def test_eol(self): # regression test: # clinic's block parser didn't recognize @@ -98,15 +119,86 @@ def test_eol(self): # so it would spit out an end line for you. # and since you really already had one, # the last line of the block got corrupted. - c = clinic.Clinic(clinic.CLanguage(None), filename="file") raw = "/*[clinic]\nfoo\n[clinic]*/" - cooked = c.parse(raw).splitlines() + cooked = self.clinic.parse(raw).splitlines() end_line = cooked[2].rstrip() # this test is redundant, it's just here explicitly to catch # the regression test so we don't forget what it looked like self.assertNotEqual(end_line, "[clinic]*/[clinic]*/") self.assertEqual(end_line, "[clinic]*/") + def test_mangled_marker_line(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/ + /*[clinic end generated code: foo]*/ + """ + msg = ( + 'Error in file "test.c" on line 3:\n' + "Mangled Argument Clinic marker line: '/*[clinic end generated code: foo]*/'\n" + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_checksum_mismatch(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/ + /*[clinic end generated code: output=0123456789abcdef input=fedcba9876543210]*/ + """ + msg = ( + 'Error in file "test.c" on line 3:\n' + 'Checksum mismatch!\n' + 'Expected: 0123456789abcdef\n' + 'Computed: da39a3ee5e6b4b0d\n' + ) + out = self.expect_failure(raw) + self.assertIn(msg, out) + + def test_garbage_after_stop_line(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/foobarfoobar! + """ + msg = ( + 'Error in file "test.c" on line 2:\n' + "Garbage after stop line: 'foobarfoobar!'\n" + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_whitespace_before_stop_line(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/ + """ + msg = ( + 'Error in file "test.c" on line 2:\n' + "Whitespace is not allowed before the stop line: ' [clinic start generated code]*/'\n" + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_parse_with_body_prefix(self): + clang = clinic.CLanguage(None) + clang.body_prefix = "//" + clang.start_line = "//[{dsl_name} start]" + clang.stop_line = "//[{dsl_name} stop]" + cl = clinic.Clinic(clang, filename="test.c") + raw = dedent(""" + //[clinic start] + //module test + //[clinic stop] + """).strip() + out = cl.parse(raw) + expected = dedent(""" + //[clinic start] + //module test + // + //[clinic stop] + /*[clinic end generated code: output=da39a3ee5e6b4b0d input=65fab8adff58cf08]*/ + """).lstrip() # Note, lstrip() because of the newline + self.assertEqual(out, expected) class ClinicGroupPermuterTest(TestCase): @@ -285,7 +377,7 @@ def test_clinic_1(self): """) -class ClinicParserTest(TestCase): +class ClinicParserTest(_ParserBase): def checkDocstring(self, fn, expected): self.assertTrue(hasattr(fn, "docstring")) self.assertEqual(fn.docstring.strip(), From webhook-mailer at python.org Sat Jul 15 05:55:28 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 15 Jul 2023 09:55:28 -0000 Subject: [Python-checkins] [3.12] gh-106368: Increase Argument Clinic BlockParser test coverage (GH-106759) (#106769) Message-ID: https://github.com/python/cpython/commit/1fe841254e453679ffeb16aa48ef633e67b7c08d commit: 1fe841254e453679ffeb16aa48ef633e67b7c08d branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-15T09:55:25Z summary: [3.12] gh-106368: Increase Argument Clinic BlockParser test coverage (GH-106759) (#106769) (cherry picked from commit 2d7d1aa4bcd5da0177458b22b1b856db76aa20d4) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 975840333e590..b5744f7013d6a 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -18,6 +18,19 @@ from clinic import DSLParser +class _ParserBase(TestCase): + maxDiff = None + + def expect_parser_failure(self, parser, _input): + with support.captured_stdout() as stdout: + with self.assertRaises(SystemExit): + parser(_input) + return stdout.getvalue() + + def parse_function_should_fail(self, _input): + return self.expect_parser_failure(self.parse_function, _input) + + class FakeConverter: def __init__(self, name, args): self.name = name @@ -88,7 +101,15 @@ def directive(self, name, args): _module_and_class = clinic.Clinic._module_and_class -class ClinicWholeFileTest(TestCase): + +class ClinicWholeFileTest(_ParserBase): + def setUp(self): + self.clinic = clinic.Clinic(clinic.CLanguage(None), filename="test.c") + + def expect_failure(self, raw): + _input = dedent(raw).strip() + return self.expect_parser_failure(self.clinic.parse, _input) + def test_eol(self): # regression test: # clinic's block parser didn't recognize @@ -98,15 +119,86 @@ def test_eol(self): # so it would spit out an end line for you. # and since you really already had one, # the last line of the block got corrupted. - c = clinic.Clinic(clinic.CLanguage(None), filename="file") raw = "/*[clinic]\nfoo\n[clinic]*/" - cooked = c.parse(raw).splitlines() + cooked = self.clinic.parse(raw).splitlines() end_line = cooked[2].rstrip() # this test is redundant, it's just here explicitly to catch # the regression test so we don't forget what it looked like self.assertNotEqual(end_line, "[clinic]*/[clinic]*/") self.assertEqual(end_line, "[clinic]*/") + def test_mangled_marker_line(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/ + /*[clinic end generated code: foo]*/ + """ + msg = ( + 'Error in file "test.c" on line 3:\n' + "Mangled Argument Clinic marker line: '/*[clinic end generated code: foo]*/'\n" + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_checksum_mismatch(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/ + /*[clinic end generated code: output=0123456789abcdef input=fedcba9876543210]*/ + """ + msg = ( + 'Error in file "test.c" on line 3:\n' + 'Checksum mismatch!\n' + 'Expected: 0123456789abcdef\n' + 'Computed: da39a3ee5e6b4b0d\n' + ) + out = self.expect_failure(raw) + self.assertIn(msg, out) + + def test_garbage_after_stop_line(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/foobarfoobar! + """ + msg = ( + 'Error in file "test.c" on line 2:\n' + "Garbage after stop line: 'foobarfoobar!'\n" + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_whitespace_before_stop_line(self): + raw = """ + /*[clinic input] + [clinic start generated code]*/ + """ + msg = ( + 'Error in file "test.c" on line 2:\n' + "Whitespace is not allowed before the stop line: ' [clinic start generated code]*/'\n" + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_parse_with_body_prefix(self): + clang = clinic.CLanguage(None) + clang.body_prefix = "//" + clang.start_line = "//[{dsl_name} start]" + clang.stop_line = "//[{dsl_name} stop]" + cl = clinic.Clinic(clang, filename="test.c") + raw = dedent(""" + //[clinic start] + //module test + //[clinic stop] + """).strip() + out = cl.parse(raw) + expected = dedent(""" + //[clinic start] + //module test + // + //[clinic stop] + /*[clinic end generated code: output=da39a3ee5e6b4b0d input=65fab8adff58cf08]*/ + """).lstrip() # Note, lstrip() because of the newline + self.assertEqual(out, expected) class ClinicGroupPermuterTest(TestCase): @@ -285,7 +377,7 @@ def test_clinic_1(self): """) -class ClinicParserTest(TestCase): +class ClinicParserTest(_ParserBase): def checkDocstring(self, fn, expected): self.assertTrue(hasattr(fn, "docstring")) self.assertEqual(fn.docstring.strip(), From webhook-mailer at python.org Sat Jul 15 06:11:35 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 15 Jul 2023 10:11:35 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Annotate BlockParser (#106750) Message-ID: https://github.com/python/cpython/commit/bbf62979851283b601b2dac0073ab331ebeb3be9 commit: bbf62979851283b601b2dac0073ab331ebeb3be9 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-15T10:11:32Z summary: gh-104050: Argument Clinic: Annotate BlockParser (#106750) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 726ebc04f55bd..861a6507eae75 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1712,7 +1712,13 @@ class BlockParser: Iterator, yields Block objects. """ - def __init__(self, input, language, *, verify=True): + def __init__( + self, + input: str, + language: Language, + *, + verify: bool = True + ) -> None: """ "input" should be a str object with embedded \n characters. @@ -1730,15 +1736,15 @@ def __init__(self, input, language, *, verify=True): self.find_start_re = create_regex(before, after, whole_line=False) self.start_re = create_regex(before, after) self.verify = verify - self.last_checksum_re = None - self.last_dsl_name = None - self.dsl_name = None + self.last_checksum_re: re.Pattern[str] | None = None + self.last_dsl_name: str | None = None + self.dsl_name: str | None = None self.first_block = True - def __iter__(self): + def __iter__(self) -> BlockParser: return self - def __next__(self): + def __next__(self) -> Block: while True: if not self.input: raise StopIteration @@ -1755,18 +1761,18 @@ def __next__(self): return block - def is_start_line(self, line): + def is_start_line(self, line: str) -> str | None: match = self.start_re.match(line.lstrip()) return match.group(1) if match else None - def _line(self, lookahead=False): + def _line(self, lookahead: bool = False) -> str: self.line_number += 1 line = self.input.pop() if not lookahead: self.language.parse_line(line) return line - def parse_verbatim_block(self): + def parse_verbatim_block(self) -> Block: add, output = text_accumulator() self.block_start_line_number = self.line_number @@ -1780,13 +1786,13 @@ def parse_verbatim_block(self): return Block(output()) - def parse_clinic_block(self, dsl_name): + def parse_clinic_block(self, dsl_name: str) -> Block: input_add, input_output = text_accumulator() self.block_start_line_number = self.line_number + 1 stop_line = self.language.stop_line.format(dsl_name=dsl_name) body_prefix = self.language.body_prefix.format(dsl_name=dsl_name) - def is_stop_line(line): + def is_stop_line(line: str) -> bool: # make sure to recognize stop line even if it # doesn't end with EOL (it could be the very end of the file) if line.startswith(stop_line): @@ -1820,6 +1826,7 @@ def is_stop_line(line): checksum_re = create_regex(before, after, word=False) self.last_dsl_name = dsl_name self.last_checksum_re = checksum_re + assert checksum_re is not None # scan forward for checksum line output_add, output_output = text_accumulator() @@ -1834,6 +1841,7 @@ def is_stop_line(line): if self.is_start_line(line): break + output: str | None output = output_output() if arguments: d = {} From webhook-mailer at python.org Sat Jul 15 06:31:16 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 15 Jul 2023 10:31:16 -0000 Subject: [Python-checkins] [3.12] gh-106745: typing docs: Clarify that removal of PEP-585 aliases is not currently planned (#106748) (#106772) Message-ID: https://github.com/python/cpython/commit/e99b69c5aea1aed6e3ea5f550b07698291fc8aeb commit: e99b69c5aea1aed6e3ea5f550b07698291fc8aeb branch: 3.12 author: Alex Waygood committer: AlexWaygood date: 2023-07-15T11:31:12+01:00 summary: [3.12] gh-106745: typing docs: Clarify that removal of PEP-585 aliases is not currently planned (#106748) (#106772) files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 2a712a4be5890..e1acbd83a4173 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -3053,6 +3053,7 @@ Constant .. versionadded:: 3.5.2 .. _generic-concrete-collections: +.. _deprecated-aliases: Deprecated aliases ------------------ @@ -3061,16 +3062,21 @@ This module defines several deprecated aliases to pre-existing standard library classes. These were originally included in the typing module in order to support parameterizing these generic classes using ``[]``. However, the aliases became redundant in Python 3.9 when the -corresponding pre-existing classes were enhanced to support ``[]``. +corresponding pre-existing classes were enhanced to support ``[]`` (see +:pep:`585`). -The redundant types are deprecated as of Python 3.9 but no -deprecation warnings are issued by the interpreter. -It is expected that type checkers will flag the deprecated types -when the checked program targets Python 3.9 or newer. +The redundant types are deprecated as of Python 3.9. However, while the aliases +may be removed at some point, removal of these aliases is not currently +planned. As such, no deprecation warnings are currently issued by the +interpreter for these aliases. -The deprecated types will be removed from the :mod:`typing` module -no sooner than the first Python version released 5 years after the release of Python 3.9.0. -See details in :pep:`585`?*Type Hinting Generics In Standard Collections*. +If at some point it is decided to remove these deprecated aliases, a +deprecation warning will be issued by the interpreter for at least two releases +prior to removal. The aliases are guaranteed to remain in the typing module +without deprecation warnings until at least Python 3.14. + +Type checkers are encouraged to flag uses of the deprecated types if the +program they are checking targets a minimum Python version of 3.9 or newer. .. _corresponding-to-built-in-types: @@ -3611,21 +3617,34 @@ Certain features in ``typing`` are deprecated and may be removed in a future version of Python. The following table summarizes major deprecations for your convenience. This is subject to change, and not all deprecations are listed. -+----------------------------------+---------------+-------------------+----------------+ -| Feature | Deprecated in | Projected removal | PEP/issue | -+==================================+===============+===================+================+ -| ``typing.io`` and ``typing.re`` | 3.8 | 3.13 | :issue:`38291` | -| submodules | | | | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing`` versions of standard | 3.9 | Undecided | :pep:`585` | -| collections | | | | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.ByteString`` | 3.9 | 3.14 | :gh:`91896` | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.Text`` | 3.11 | Undecided | :gh:`92332` | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.Hashable`` and | 3.12 | Undecided | :gh:`94309` | -| ``typing.Sized`` | | | | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.TypeAlias`` | 3.12 | Undecided | :pep:`695` | -+----------------------------------+---------------+-------------------+----------------+ +.. list-table:: + :header-rows: 1 + + * - Feature + - Deprecated in + - Projected removal + - PEP/issue + * - ``typing.io`` and ``typing.re`` submodules + - 3.8 + - 3.13 + - :issue:`38291` + * - ``typing`` versions of standard collections + - 3.9 + - Undecided (see :ref:`deprecated-aliases` for more information) + - :pep:`585` + * - :class:`typing.ByteString` + - 3.9 + - 3.14 + - :gh:`91896` + * - :data:`typing.Text` + - 3.11 + - Undecided + - :gh:`92332` + * - :class:`typing.Hashable` and :class:`typing.Sized` + - 3.12 + - Undecided + - :gh:`94309` + * - :data:`typing.TypeAlias` + - 3.12 + - Undecided + - :pep:`695` From webhook-mailer at python.org Sat Jul 15 06:33:35 2023 From: webhook-mailer at python.org (methane) Date: Sat, 15 Jul 2023 10:33:35 -0000 Subject: [Python-checkins] gh-81283: compiler: remove indent from docstring (#106411) Message-ID: https://github.com/python/cpython/commit/2566b74b26bcce24199427acea392aed644f4b17 commit: 2566b74b26bcce24199427acea392aed644f4b17 branch: main author: Inada Naoki committer: methane date: 2023-07-15T19:33:32+09:00 summary: gh-81283: compiler: remove indent from docstring (#106411) Co-authored-by: ?ric files: A Misc/NEWS.d/next/Core and Builtins/2023-07-04-20-42-54.gh-issue-81283.hfh_MD.rst M Doc/whatsnew/3.13.rst M Include/internal/pycore_compile.h M Lib/inspect.py M Lib/test/test_doctest.py M Lib/test/test_inspect.py M Modules/_testinternalcapi.c M Modules/clinic/_testinternalcapi.c.h M Python/compile.c diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 06fcaf4608cdc..161d5fb1c59a3 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -79,6 +79,13 @@ Other Language Changes * Allow the *count* argument of :meth:`str.replace` to be a keyword. (Contributed by Hugo van Kemenade in :gh:`106487`.) +* Compiler now strip indents from docstrings. + This will reduce the size of :term:`bytecode cache ` (e.g. ``.pyc`` file). + For example, cache file size for ``sqlalchemy.orm.session`` in SQLAlchemy 2.0 + is reduced by about 5%. + This change will affect tools using docstrings, like :mod:`doctest`. + (Contributed by Inada Naoki in :gh:`81283`.) + New Modules =========== diff --git a/Include/internal/pycore_compile.h b/Include/internal/pycore_compile.h index e204d4d2457a1..beb37cced06db 100644 --- a/Include/internal/pycore_compile.h +++ b/Include/internal/pycore_compile.h @@ -91,6 +91,8 @@ int _PyCompile_ConstCacheMergeOne(PyObject *const_cache, PyObject **obj); /* Access compiler internals for unit testing */ +PyAPI_FUNC(PyObject*) _PyCompile_CleanDoc(PyObject *doc); + PyAPI_FUNC(PyObject*) _PyCompile_CodeGen( PyObject *ast, PyObject *filename, diff --git a/Lib/inspect.py b/Lib/inspect.py index a550202bb0d49..15f94a194856a 100644 --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -881,29 +881,28 @@ def cleandoc(doc): Any whitespace that can be uniformly removed from the second line onwards is removed.""" - try: - lines = doc.expandtabs().split('\n') - except UnicodeError: - return None - else: - # Find minimum indentation of any non-blank lines after first line. - margin = sys.maxsize - for line in lines[1:]: - content = len(line.lstrip()) - if content: - indent = len(line) - content - margin = min(margin, indent) - # Remove indentation. - if lines: - lines[0] = lines[0].lstrip() - if margin < sys.maxsize: - for i in range(1, len(lines)): lines[i] = lines[i][margin:] - # Remove any trailing or leading blank lines. - while lines and not lines[-1]: - lines.pop() - while lines and not lines[0]: - lines.pop(0) - return '\n'.join(lines) + lines = doc.expandtabs().split('\n') + + # Find minimum indentation of any non-blank lines after first line. + margin = sys.maxsize + for line in lines[1:]: + content = len(line.lstrip(' ')) + if content: + indent = len(line) - content + margin = min(margin, indent) + # Remove indentation. + if lines: + lines[0] = lines[0].lstrip(' ') + if margin < sys.maxsize: + for i in range(1, len(lines)): + lines[i] = lines[i][margin:] + # Remove any trailing or leading blank lines. + while lines and not lines[-1]: + lines.pop() + while lines and not lines[0]: + lines.pop(0) + return '\n'.join(lines) + def getfile(object): """Work out which source or compiled file an object was defined in.""" diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py index 542fcdb5cf6f6..bea52c6de7ec6 100644 --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest.py @@ -1287,14 +1287,14 @@ def optionflags(): r""" treated as equal: >>> def f(x): - ... '>>> print(1, 2, 3)\n 1 2\n 3' + ... '\n>>> print(1, 2, 3)\n 1 2\n 3' >>> # Without the flag: >>> test = doctest.DocTestFinder().find(f)[0] >>> doctest.DocTestRunner(verbose=False).run(test) ... # doctest: +ELLIPSIS ********************************************************************** - File ..., line 2, in f + File ..., line 3, in f Failed example: print(1, 2, 3) Expected: diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py index d89953ab60f02..64afeec351b35 100644 --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -596,9 +596,40 @@ def test_finddoc(self): self.assertEqual(finddoc(int.from_bytes), int.from_bytes.__doc__) self.assertEqual(finddoc(int.real), int.real.__doc__) + cleandoc_testdata = [ + # first line should have different margin + (' An\n indented\n docstring.', 'An\nindented\n docstring.'), + # trailing whitespace are not removed. + (' An \n \n indented \n docstring. ', + 'An \n \nindented \n docstring. '), + # NUL is not termination. + ('doc\0string\n\n second\0line\n third\0line\0', + 'doc\0string\n\nsecond\0line\nthird\0line\0'), + # first line is lstrip()-ped. other lines are kept when no margin.[w: + (' ', ''), + # compiler.cleandoc() doesn't strip leading/trailing newlines + # to keep maximum backward compatibility. + # inspect.cleandoc() removes them. + ('\n\n\n first paragraph\n\n second paragraph\n\n', + '\n\n\nfirst paragraph\n\n second paragraph\n\n'), + (' \n \n \n ', '\n \n \n '), + ] + def test_cleandoc(self): - self.assertEqual(inspect.cleandoc('An\n indented\n docstring.'), - 'An\nindented\ndocstring.') + func = inspect.cleandoc + for i, (input, expected) in enumerate(self.cleandoc_testdata): + # only inspect.cleandoc() strip \n + expected = expected.strip('\n') + with self.subTest(i=i): + self.assertEqual(func(input), expected) + + @cpython_only + def test_c_cleandoc(self): + import _testinternalcapi + func = _testinternalcapi.compiler_cleandoc + for i, (input, expected) in enumerate(self.cleandoc_testdata): + with self.subTest(i=i): + self.assertEqual(func(input), expected) def test_getcomments(self): self.assertEqual(inspect.getcomments(mod), '# line 1\n') diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-04-20-42-54.gh-issue-81283.hfh_MD.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-04-20-42-54.gh-issue-81283.hfh_MD.rst new file mode 100644 index 0000000000000..f673c665fe327 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-04-20-42-54.gh-issue-81283.hfh_MD.rst @@ -0,0 +1,3 @@ +Compiler now strips indents from docstrings. It reduces ``pyc`` file size 5% +when the module is heavily documented. This change affects to ``__doc__`` so +tools like doctest will be affected. diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 7745dd5abc22f..271ad6cfcaee3 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -15,7 +15,7 @@ #include "pycore_atomic_funcs.h" // _Py_atomic_int_get() #include "pycore_bitutils.h" // _Py_bswap32() #include "pycore_bytesobject.h" // _PyBytes_Find() -#include "pycore_compile.h" // _PyCompile_CodeGen, _PyCompile_OptimizeCfg, _PyCompile_Assemble +#include "pycore_compile.h" // _PyCompile_CodeGen, _PyCompile_OptimizeCfg, _PyCompile_Assemble, _PyCompile_CleanDoc #include "pycore_ceval.h" // _PyEval_AddPendingCall #include "pycore_fileutils.h" // _Py_normpath #include "pycore_frame.h" // _PyInterpreterFrame @@ -704,6 +704,23 @@ set_eval_frame_record(PyObject *self, PyObject *list) Py_RETURN_NONE; } +/*[clinic input] + +_testinternalcapi.compiler_cleandoc -> object + + doc: unicode + +C implementation of inspect.cleandoc(). +[clinic start generated code]*/ + +static PyObject * +_testinternalcapi_compiler_cleandoc_impl(PyObject *module, PyObject *doc) +/*[clinic end generated code: output=2dd203a80feff5bc input=2de03fab931d9cdc]*/ +{ + return _PyCompile_CleanDoc(doc); +} + + /*[clinic input] _testinternalcapi.compiler_codegen -> object @@ -1448,6 +1465,7 @@ static PyMethodDef module_functions[] = { {"DecodeLocaleEx", decode_locale_ex, METH_VARARGS}, {"set_eval_frame_default", set_eval_frame_default, METH_NOARGS, NULL}, {"set_eval_frame_record", set_eval_frame_record, METH_O, NULL}, + _TESTINTERNALCAPI_COMPILER_CLEANDOC_METHODDEF _TESTINTERNALCAPI_COMPILER_CODEGEN_METHODDEF _TESTINTERNALCAPI_OPTIMIZE_CFG_METHODDEF _TESTINTERNALCAPI_ASSEMBLE_CODE_OBJECT_METHODDEF diff --git a/Modules/clinic/_testinternalcapi.c.h b/Modules/clinic/_testinternalcapi.c.h index f512412587450..9419dcd751a0e 100644 --- a/Modules/clinic/_testinternalcapi.c.h +++ b/Modules/clinic/_testinternalcapi.c.h @@ -8,6 +8,65 @@ preserve #endif +PyDoc_STRVAR(_testinternalcapi_compiler_cleandoc__doc__, +"compiler_cleandoc($module, /, doc)\n" +"--\n" +"\n" +"C implementation of inspect.cleandoc()."); + +#define _TESTINTERNALCAPI_COMPILER_CLEANDOC_METHODDEF \ + {"compiler_cleandoc", _PyCFunction_CAST(_testinternalcapi_compiler_cleandoc), METH_FASTCALL|METH_KEYWORDS, _testinternalcapi_compiler_cleandoc__doc__}, + +static PyObject * +_testinternalcapi_compiler_cleandoc_impl(PyObject *module, PyObject *doc); + +static PyObject * +_testinternalcapi_compiler_cleandoc(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(doc), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"doc", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "compiler_cleandoc", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + PyObject *doc; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + if (!PyUnicode_Check(args[0])) { + _PyArg_BadArgument("compiler_cleandoc", "argument 'doc'", "str", args[0]); + goto exit; + } + doc = args[0]; + return_value = _testinternalcapi_compiler_cleandoc_impl(module, doc); + +exit: + return return_value; +} + PyDoc_STRVAR(_testinternalcapi_compiler_codegen__doc__, "compiler_codegen($module, /, ast, filename, optimize, compile_mode=0)\n" "--\n" @@ -206,4 +265,4 @@ _testinternalcapi_assemble_code_object(PyObject *module, PyObject *const *args, exit: return return_value; } -/*[clinic end generated code: output=2965f1578b986218 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=811d50772c8f285a input=a9049054013a1b77]*/ diff --git a/Python/compile.c b/Python/compile.c index 9e86e06777ffa..b80f7c01bcd90 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -1704,10 +1704,16 @@ compiler_body(struct compiler *c, location loc, asdl_stmt_seq *stmts) if (c->c_optimize < 2) { docstring = _PyAST_GetDocString(stmts); if (docstring) { + PyObject *cleandoc = _PyCompile_CleanDoc(docstring); + if (cleandoc == NULL) { + return ERROR; + } i = 1; st = (stmt_ty)asdl_seq_GET(stmts, 0); assert(st->kind == Expr_kind); - VISIT(c, expr, st->v.Expr.value); + location loc = LOC(st->v.Expr.value); + ADDOP_LOAD_CONST(c, loc, cleandoc); + Py_DECREF(cleandoc); RETURN_IF_ERROR(compiler_nameop(c, NO_LOCATION, &_Py_ID(__doc__), Store)); } } @@ -2252,11 +2258,19 @@ compiler_function_body(struct compiler *c, stmt_ty s, int is_async, Py_ssize_t f /* if not -OO mode, add docstring */ if (c->c_optimize < 2) { docstring = _PyAST_GetDocString(body); + if (docstring) { + docstring = _PyCompile_CleanDoc(docstring); + if (docstring == NULL) { + compiler_exit_scope(c); + return ERROR; + } + } } if (compiler_add_const(c->c_const_cache, c->u, docstring ? docstring : Py_None) < 0) { compiler_exit_scope(c); return ERROR; } + Py_XDECREF(docstring); c->u->u_metadata.u_argcount = asdl_seq_LEN(args->args); c->u->u_metadata.u_posonlyargcount = asdl_seq_LEN(args->posonlyargs); @@ -7967,6 +7981,89 @@ cfg_to_instructions(cfg_builder *g) return NULL; } +// C implementation of inspect.cleandoc() +// +// Difference from inspect.cleandoc(): +// - Do not remove leading and trailing blank lines to keep lineno. +PyObject * +_PyCompile_CleanDoc(PyObject *doc) +{ + doc = PyObject_CallMethod(doc, "expandtabs", NULL); + if (doc == NULL) { + return NULL; + } + + Py_ssize_t doc_size; + const char *doc_utf8 = PyUnicode_AsUTF8AndSize(doc, &doc_size); + if (doc_utf8 == NULL) { + Py_DECREF(doc); + return NULL; + } + const char *p = doc_utf8; + const char *pend = p + doc_size; + + // First pass: find minimum indentation of any non-blank lines + // after first line. + while (p < pend && *p++ != '\n') { + } + + Py_ssize_t margin = PY_SSIZE_T_MAX; + while (p < pend) { + const char *s = p; + while (*p == ' ') p++; + if (p < pend && *p != '\n') { + margin = Py_MIN(margin, p - s); + } + while (p < pend && *p++ != '\n') { + } + } + if (margin == PY_SSIZE_T_MAX) { + margin = 0; + } + + // Second pass: write cleandoc into buff. + + // copy first line without leading spaces. + p = doc_utf8; + while (*p == ' ') { + p++; + } + if (p == doc_utf8 && margin == 0 ) { + // doc is already clean. + return doc; + } + + char *buff = PyMem_Malloc(doc_size); + char *w = buff; + + while (p < pend) { + int ch = *w++ = *p++; + if (ch == '\n') { + break; + } + } + + // copy subsequent lines without margin. + while (p < pend) { + for (Py_ssize_t i = 0; i < margin; i++, p++) { + if (*p != ' ') { + assert(*p == '\n' || *p == '\0'); + break; + } + } + while (p < pend) { + int ch = *w++ = *p++; + if (ch == '\n') { + break; + } + } + } + + Py_DECREF(doc); + return PyUnicode_FromStringAndSize(buff, w - buff); +} + + PyObject * _PyCompile_CodeGen(PyObject *ast, PyObject *filename, PyCompilerFlags *pflags, int optimize, int compile_mode) From webhook-mailer at python.org Sat Jul 15 06:34:32 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 15 Jul 2023 10:34:32 -0000 Subject: [Python-checkins] [3.11] gh-106745: typing docs: Clarify that removal of PEP-585 aliases is not currently planned (#106748) (#106773) Message-ID: https://github.com/python/cpython/commit/7dead6a33ac786465c3d927d5c2025caf0c1a71b commit: 7dead6a33ac786465c3d927d5c2025caf0c1a71b branch: 3.11 author: Alex Waygood committer: AlexWaygood date: 2023-07-15T11:34:29+01:00 summary: [3.11] gh-106745: typing docs: Clarify that removal of PEP-585 aliases is not currently planned (#106748) (#106773) files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 25b7cfe703a13..d03f0af1bc6b6 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -2745,6 +2745,7 @@ Constant .. versionadded:: 3.5.2 .. _generic-concrete-collections: +.. _deprecated-typing-aliases: Deprecated aliases ------------------ @@ -2753,16 +2754,21 @@ This module defines several deprecated aliases to pre-existing standard library classes. These were originally included in the typing module in order to support parameterizing these generic classes using ``[]``. However, the aliases became redundant in Python 3.9 when the -corresponding pre-existing classes were enhanced to support ``[]``. +corresponding pre-existing classes were enhanced to support ``[]`` (see +:pep:`585`). -The redundant types are deprecated as of Python 3.9 but no -deprecation warnings are issued by the interpreter. -It is expected that type checkers will flag the deprecated types -when the checked program targets Python 3.9 or newer. +The redundant types are deprecated as of Python 3.9. However, while the aliases +may be removed at some point, removal of these aliases is not currently +planned. As such, no deprecation warnings are currently issued by the +interpreter for these aliases. -The deprecated types will be removed from the :mod:`typing` module -no sooner than the first Python version released 5 years after the release of Python 3.9.0. -See details in :pep:`585`?*Type Hinting Generics In Standard Collections*. +If at some point it is decided to remove these deprecated aliases, a +deprecation warning will be issued by the interpreter for at least two releases +prior to removal. The aliases are guaranteed to remain in the typing module +without deprecation warnings until at least Python 3.14. + +Type checkers are encouraged to flag uses of the deprecated types if the +program they are checking targets a minimum Python version of 3.9 or newer. .. _corresponding-to-built-in-types: @@ -3295,16 +3301,26 @@ Certain features in ``typing`` are deprecated and may be removed in a future version of Python. The following table summarizes major deprecations for your convenience. This is subject to change, and not all deprecations are listed. -+----------------------------------+---------------+-------------------+----------------+ -| Feature | Deprecated in | Projected removal | PEP/issue | -+==================================+===============+===================+================+ -| ``typing.io`` and ``typing.re`` | 3.8 | 3.13 | :issue:`38291` | -| submodules | | | | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing`` versions of standard | 3.9 | Undecided | :pep:`585` | -| collections | | | | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.ByteString`` | 3.9 | 3.14 | :gh:`91896` | -+----------------------------------+---------------+-------------------+----------------+ -| ``typing.Text`` | 3.11 | Undecided | :gh:`92332` | -+----------------------------------+---------------+-------------------+----------------+ +.. list-table:: + :header-rows: 1 + + * - Feature + - Deprecated in + - Projected removal + - PEP/issue + * - ``typing.io`` and ``typing.re`` submodules + - 3.8 + - 3.13 + - :issue:`38291` + * - ``typing`` versions of standard collections + - 3.9 + - Undecided (see :ref:`deprecated-typing-aliases` for more information) + - :pep:`585` + * - :class:`typing.ByteString` + - 3.9 + - 3.14 + - :gh:`91896` + * - :data:`typing.Text` + - 3.11 + - Undecided + - :gh:`92332` From webhook-mailer at python.org Sat Jul 15 09:21:20 2023 From: webhook-mailer at python.org (jaraco) Date: Sat, 15 Jul 2023 13:21:20 -0000 Subject: [Python-checkins] gh-106752: Sync with zipp 3.16.2 (#106757) Message-ID: https://github.com/python/cpython/commit/22980dc7c9dcec4b74fea815542601ef582c230e commit: 22980dc7c9dcec4b74fea815542601ef582c230e branch: main author: Jason R. Coombs committer: jaraco date: 2023-07-15T09:21:17-04:00 summary: gh-106752: Sync with zipp 3.16.2 (#106757) * gh-106752: Sync with zipp 3.16.2 * Add blurb files: A Lib/test/test_zipfile/_path/write-alpharep.py A Lib/zipfile/_path/glob.py A Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst M Lib/test/test_zipfile/_path/test_complexity.py M Lib/test/test_zipfile/_path/test_path.py M Lib/zipfile/_path/__init__.py diff --git a/Lib/test/test_zipfile/_path/test_complexity.py b/Lib/test/test_zipfile/_path/test_complexity.py index 3432dc39e56c4..7050937738af1 100644 --- a/Lib/test/test_zipfile/_path/test_complexity.py +++ b/Lib/test/test_zipfile/_path/test_complexity.py @@ -1,5 +1,9 @@ -import unittest +import io +import itertools +import math +import re import string +import unittest import zipfile from ._functools import compose @@ -9,9 +13,11 @@ big_o = import_or_skip('big_o') +pytest = import_or_skip('pytest') class TestComplexity(unittest.TestCase): + @pytest.mark.flaky def test_implied_dirs_performance(self): best, others = big_o.big_o( compose(consume, zipfile.CompleteDirs._implied_dirs), @@ -22,3 +28,76 @@ def test_implied_dirs_performance(self): min_n=1, ) assert best <= big_o.complexities.Linear + + def make_zip_path(self, depth=1, width=1) -> zipfile.Path: + """ + Construct a Path with width files at every level of depth. + """ + zf = zipfile.ZipFile(io.BytesIO(), mode='w') + pairs = itertools.product(self.make_deep_paths(depth), self.make_names(width)) + for path, name in pairs: + zf.writestr(f"{path}{name}.txt", b'') + zf.filename = "big un.zip" + return zipfile.Path(zf) + + @classmethod + def make_names(cls, width, letters=string.ascii_lowercase): + """ + >>> list(TestComplexity.make_names(2)) + ['a', 'b'] + >>> list(TestComplexity.make_names(30)) + ['aa', 'ab', ..., 'bd'] + """ + # determine how many products are needed to produce width + n_products = math.ceil(math.log(width, len(letters))) + inputs = (letters,) * n_products + combinations = itertools.product(*inputs) + names = map(''.join, combinations) + return itertools.islice(names, width) + + @classmethod + def make_deep_paths(cls, depth): + return map(cls.make_deep_path, range(depth)) + + @classmethod + def make_deep_path(cls, depth): + return ''.join(('d/',) * depth) + + def test_baseline_regex_complexity(self): + best, others = big_o.big_o( + lambda path: re.fullmatch(r'[^/]*\\.txt', path), + self.make_deep_path, + max_n=100, + min_n=1, + ) + assert best <= big_o.complexities.Constant + + @pytest.mark.flaky + def test_glob_depth(self): + best, others = big_o.big_o( + lambda path: consume(path.glob('*.txt')), + self.make_zip_path, + max_n=100, + min_n=1, + ) + assert best <= big_o.complexities.Quadratic + + @pytest.mark.flaky + def test_glob_width(self): + best, others = big_o.big_o( + lambda path: consume(path.glob('*.txt')), + lambda size: self.make_zip_path(width=size), + max_n=100, + min_n=1, + ) + assert best <= big_o.complexities.Linear + + @pytest.mark.flaky + def test_glob_width_and_depth(self): + best, others = big_o.big_o( + lambda path: consume(path.glob('*.txt')), + lambda size: self.make_zip_path(depth=size, width=size), + max_n=10, + min_n=1, + ) + assert best <= big_o.complexities.Linear diff --git a/Lib/test/test_zipfile/_path/test_path.py b/Lib/test/test_zipfile/_path/test_path.py index aff91e5399587..c66cb3cba69eb 100644 --- a/Lib/test/test_zipfile/_path/test_path.py +++ b/Lib/test/test_zipfile/_path/test_path.py @@ -41,9 +41,13 @@ def build_alpharep_fixture(): ? ??? d ? ? ??? e.txt ? ??? f.txt - ??? g - ??? h - ??? i.txt + ??? g + ? ??? h + ? ??? i.txt + ??? j + ??? k.bin + ??? l.baz + ??? m.bar This fixture has the following key characteristics: @@ -51,6 +55,7 @@ def build_alpharep_fixture(): - a file two levels deep (b/d/e) - multiple files in a directory (b/c, b/f) - a directory containing only a directory (g/h) + - a directory with files of different extensions (j/klm) "alpha" because it uses alphabet "rep" because it's a representative example @@ -62,6 +67,9 @@ def build_alpharep_fixture(): zf.writestr("b/d/e.txt", b"content of e") zf.writestr("b/f.txt", b"content of f") zf.writestr("g/h/i.txt", b"content of i") + zf.writestr("j/k.bin", b"content of k") + zf.writestr("j/l.baz", b"content of l") + zf.writestr("j/m.bar", b"content of m") zf.filename = "alpharep.zip" return zf @@ -92,7 +100,7 @@ def zipfile_ondisk(self, alpharep): def test_iterdir_and_types(self, alpharep): root = zipfile.Path(alpharep) assert root.is_dir() - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() assert a.is_file() assert b.is_dir() assert g.is_dir() @@ -112,7 +120,7 @@ def test_is_file_missing(self, alpharep): @pass_alpharep def test_iterdir_on_file(self, alpharep): root = zipfile.Path(alpharep) - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() with self.assertRaises(ValueError): a.iterdir() @@ -127,7 +135,7 @@ def test_subdir_is_dir(self, alpharep): @pass_alpharep def test_open(self, alpharep): root = zipfile.Path(alpharep) - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() with a.open(encoding="utf-8") as strm: data = strm.read() self.assertEqual(data, "content of a") @@ -229,7 +237,7 @@ def test_open_missing_directory(self): @pass_alpharep def test_read(self, alpharep): root = zipfile.Path(alpharep) - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() assert a.read_text(encoding="utf-8") == "content of a" # Also check positional encoding arg (gh-101144). assert a.read_text("utf-8") == "content of a" @@ -295,7 +303,7 @@ def test_mutability(self, alpharep): reflect that change. """ root = zipfile.Path(alpharep) - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() alpharep.writestr('foo.txt', 'foo') alpharep.writestr('bar/baz.txt', 'baz') assert any(child.name == 'foo.txt' for child in root.iterdir()) @@ -394,6 +402,13 @@ def test_suffixes(self, alpharep): e = root / '.hgrc' assert e.suffixes == [] + @pass_alpharep + def test_suffix_no_filename(self, alpharep): + alpharep.filename = None + root = zipfile.Path(alpharep) + assert root.joinpath('example').suffix == "" + assert root.joinpath('example').suffixes == [] + @pass_alpharep def test_stem(self, alpharep): """ @@ -411,6 +426,8 @@ def test_stem(self, alpharep): d = root / "d" assert d.stem == "d" + assert (root / ".gitignore").stem == ".gitignore" + @pass_alpharep def test_root_parent(self, alpharep): root = zipfile.Path(alpharep) @@ -442,12 +459,49 @@ def test_match_and_glob(self, alpharep): assert not root.match("*.txt") assert list(root.glob("b/c.*")) == [zipfile.Path(alpharep, "b/c.txt")] + assert list(root.glob("b/*.txt")) == [ + zipfile.Path(alpharep, "b/c.txt"), + zipfile.Path(alpharep, "b/f.txt"), + ] + @pass_alpharep + def test_glob_recursive(self, alpharep): + root = zipfile.Path(alpharep) files = root.glob("**/*.txt") assert all(each.match("*.txt") for each in files) assert list(root.glob("**/*.txt")) == list(root.rglob("*.txt")) + @pass_alpharep + def test_glob_subdirs(self, alpharep): + root = zipfile.Path(alpharep) + + assert list(root.glob("*/i.txt")) == [] + assert list(root.rglob("*/i.txt")) == [zipfile.Path(alpharep, "g/h/i.txt")] + + @pass_alpharep + def test_glob_does_not_overmatch_dot(self, alpharep): + root = zipfile.Path(alpharep) + + assert list(root.glob("*.xt")) == [] + + @pass_alpharep + def test_glob_single_char(self, alpharep): + root = zipfile.Path(alpharep) + + assert list(root.glob("a?txt")) == [zipfile.Path(alpharep, "a.txt")] + assert list(root.glob("a[.]txt")) == [zipfile.Path(alpharep, "a.txt")] + assert list(root.glob("a[?]txt")) == [] + + @pass_alpharep + def test_glob_chars(self, alpharep): + root = zipfile.Path(alpharep) + + assert list(root.glob("j/?.b[ai][nz]")) == [ + zipfile.Path(alpharep, "j/k.bin"), + zipfile.Path(alpharep, "j/l.baz"), + ] + def test_glob_empty(self): root = zipfile.Path(zipfile.ZipFile(io.BytesIO(), 'w')) with self.assertRaises(ValueError): diff --git a/Lib/test/test_zipfile/_path/write-alpharep.py b/Lib/test/test_zipfile/_path/write-alpharep.py new file mode 100644 index 0000000000000..48c09b537179f --- /dev/null +++ b/Lib/test/test_zipfile/_path/write-alpharep.py @@ -0,0 +1,4 @@ +from . import test_path + + +__name__ == '__main__' and test_path.build_alpharep_fixture().extractall('alpharep') diff --git a/Lib/zipfile/_path/__init__.py b/Lib/zipfile/_path/__init__.py index fd49a3ea91db5..78c413563bb2b 100644 --- a/Lib/zipfile/_path/__init__.py +++ b/Lib/zipfile/_path/__init__.py @@ -5,7 +5,8 @@ import contextlib import pathlib import re -import fnmatch + +from .glob import translate __all__ = ['Path'] @@ -296,21 +297,24 @@ def open(self, mode='r', *args, pwd=None, **kwargs): encoding, args, kwargs = _extract_text_encoding(*args, **kwargs) return io.TextIOWrapper(stream, encoding, *args, **kwargs) + def _base(self): + return pathlib.PurePosixPath(self.at or self.root.filename) + @property def name(self): - return pathlib.Path(self.at).name or self.filename.name + return self._base().name @property def suffix(self): - return pathlib.Path(self.at).suffix or self.filename.suffix + return self._base().suffix @property def suffixes(self): - return pathlib.Path(self.at).suffixes or self.filename.suffixes + return self._base().suffixes @property def stem(self): - return pathlib.Path(self.at).stem or self.filename.stem + return self._base().stem @property def filename(self): @@ -347,7 +351,7 @@ def iterdir(self): return filter(self._is_child, subs) def match(self, path_pattern): - return pathlib.Path(self.at).match(path_pattern) + return pathlib.PurePosixPath(self.at).match(path_pattern) def is_symlink(self): """ @@ -355,22 +359,13 @@ def is_symlink(self): """ return False - def _descendants(self): - for child in self.iterdir(): - yield child - if child.is_dir(): - yield from child._descendants() - def glob(self, pattern): if not pattern: raise ValueError(f"Unacceptable pattern: {pattern!r}") - matches = re.compile(fnmatch.translate(pattern)).fullmatch - return ( - child - for child in self._descendants() - if matches(str(child.relative_to(self))) - ) + prefix = re.escape(self.at) + matches = re.compile(prefix + translate(pattern)).fullmatch + return map(self._next, filter(matches, self.root.namelist())) def rglob(self, pattern): return self.glob(f'**/{pattern}') diff --git a/Lib/zipfile/_path/glob.py b/Lib/zipfile/_path/glob.py new file mode 100644 index 0000000000000..4a2e665e27078 --- /dev/null +++ b/Lib/zipfile/_path/glob.py @@ -0,0 +1,40 @@ +import re + + +def translate(pattern): + r""" + Given a glob pattern, produce a regex that matches it. + + >>> translate('*.txt') + '[^/]*\\.txt' + >>> translate('a?txt') + 'a.txt' + >>> translate('**/*') + '.*/[^/]*' + """ + return ''.join(map(replace, separate(pattern))) + + +def separate(pattern): + """ + Separate out character sets to avoid translating their contents. + + >>> [m.group(0) for m in separate('*.txt')] + ['*.txt'] + >>> [m.group(0) for m in separate('a[?]txt')] + ['a', '[?]', 'txt'] + """ + return re.finditer(r'([^\[]+)|(?P[\[].*?[\]])|([\[][^\]]*$)', pattern) + + +def replace(match): + """ + Perform the replacements for a match from :func:`separate`. + """ + + return match.group('set') or ( + re.escape(match.group(0)) + .replace('\\*\\*', r'.*') + .replace('\\*', r'[^/]*') + .replace('\\?', r'.') + ) diff --git a/Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst b/Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst new file mode 100644 index 0000000000000..bbc53d76decbc --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst @@ -0,0 +1,5 @@ +Fixed several bugs in zipfile.Path, including: in ``Path.match`, Windows +separators are no longer honored (and never were meant to be); Fixed +``name``/``suffix``/``suffixes``/``stem`` operations when no filename is +present and the Path is not at the root of the zipfile; Reworked glob for +performance and more correct matching behavior. From webhook-mailer at python.org Sat Jul 15 10:15:27 2023 From: webhook-mailer at python.org (jaraco) Date: Sat, 15 Jul 2023 14:15:27 -0000 Subject: [Python-checkins] [3.12] gh-106752: Sync with zipp 3.16.2 (GH-106757) (#106777) Message-ID: https://github.com/python/cpython/commit/060f58d877d738cc8659194f82864985f08a94ef commit: 060f58d877d738cc8659194f82864985f08a94ef branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: jaraco date: 2023-07-15T10:15:24-04:00 summary: [3.12] gh-106752: Sync with zipp 3.16.2 (GH-106757) (#106777) gh-106752: Sync with zipp 3.16.2 (GH-106757) * gh-106752: Sync with zipp 3.16.2 * Add blurb (cherry picked from commit 22980dc7c9dcec4b74fea815542601ef582c230e) Co-authored-by: Jason R. Coombs files: A Lib/test/test_zipfile/_path/write-alpharep.py A Lib/zipfile/_path/glob.py A Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst M Lib/test/test_zipfile/_path/test_complexity.py M Lib/test/test_zipfile/_path/test_path.py M Lib/zipfile/_path/__init__.py diff --git a/Lib/test/test_zipfile/_path/test_complexity.py b/Lib/test/test_zipfile/_path/test_complexity.py index 3432dc39e56c4..7050937738af1 100644 --- a/Lib/test/test_zipfile/_path/test_complexity.py +++ b/Lib/test/test_zipfile/_path/test_complexity.py @@ -1,5 +1,9 @@ -import unittest +import io +import itertools +import math +import re import string +import unittest import zipfile from ._functools import compose @@ -9,9 +13,11 @@ big_o = import_or_skip('big_o') +pytest = import_or_skip('pytest') class TestComplexity(unittest.TestCase): + @pytest.mark.flaky def test_implied_dirs_performance(self): best, others = big_o.big_o( compose(consume, zipfile.CompleteDirs._implied_dirs), @@ -22,3 +28,76 @@ def test_implied_dirs_performance(self): min_n=1, ) assert best <= big_o.complexities.Linear + + def make_zip_path(self, depth=1, width=1) -> zipfile.Path: + """ + Construct a Path with width files at every level of depth. + """ + zf = zipfile.ZipFile(io.BytesIO(), mode='w') + pairs = itertools.product(self.make_deep_paths(depth), self.make_names(width)) + for path, name in pairs: + zf.writestr(f"{path}{name}.txt", b'') + zf.filename = "big un.zip" + return zipfile.Path(zf) + + @classmethod + def make_names(cls, width, letters=string.ascii_lowercase): + """ + >>> list(TestComplexity.make_names(2)) + ['a', 'b'] + >>> list(TestComplexity.make_names(30)) + ['aa', 'ab', ..., 'bd'] + """ + # determine how many products are needed to produce width + n_products = math.ceil(math.log(width, len(letters))) + inputs = (letters,) * n_products + combinations = itertools.product(*inputs) + names = map(''.join, combinations) + return itertools.islice(names, width) + + @classmethod + def make_deep_paths(cls, depth): + return map(cls.make_deep_path, range(depth)) + + @classmethod + def make_deep_path(cls, depth): + return ''.join(('d/',) * depth) + + def test_baseline_regex_complexity(self): + best, others = big_o.big_o( + lambda path: re.fullmatch(r'[^/]*\\.txt', path), + self.make_deep_path, + max_n=100, + min_n=1, + ) + assert best <= big_o.complexities.Constant + + @pytest.mark.flaky + def test_glob_depth(self): + best, others = big_o.big_o( + lambda path: consume(path.glob('*.txt')), + self.make_zip_path, + max_n=100, + min_n=1, + ) + assert best <= big_o.complexities.Quadratic + + @pytest.mark.flaky + def test_glob_width(self): + best, others = big_o.big_o( + lambda path: consume(path.glob('*.txt')), + lambda size: self.make_zip_path(width=size), + max_n=100, + min_n=1, + ) + assert best <= big_o.complexities.Linear + + @pytest.mark.flaky + def test_glob_width_and_depth(self): + best, others = big_o.big_o( + lambda path: consume(path.glob('*.txt')), + lambda size: self.make_zip_path(depth=size, width=size), + max_n=10, + min_n=1, + ) + assert best <= big_o.complexities.Linear diff --git a/Lib/test/test_zipfile/_path/test_path.py b/Lib/test/test_zipfile/_path/test_path.py index aff91e5399587..c66cb3cba69eb 100644 --- a/Lib/test/test_zipfile/_path/test_path.py +++ b/Lib/test/test_zipfile/_path/test_path.py @@ -41,9 +41,13 @@ def build_alpharep_fixture(): ? ??? d ? ? ??? e.txt ? ??? f.txt - ??? g - ??? h - ??? i.txt + ??? g + ? ??? h + ? ??? i.txt + ??? j + ??? k.bin + ??? l.baz + ??? m.bar This fixture has the following key characteristics: @@ -51,6 +55,7 @@ def build_alpharep_fixture(): - a file two levels deep (b/d/e) - multiple files in a directory (b/c, b/f) - a directory containing only a directory (g/h) + - a directory with files of different extensions (j/klm) "alpha" because it uses alphabet "rep" because it's a representative example @@ -62,6 +67,9 @@ def build_alpharep_fixture(): zf.writestr("b/d/e.txt", b"content of e") zf.writestr("b/f.txt", b"content of f") zf.writestr("g/h/i.txt", b"content of i") + zf.writestr("j/k.bin", b"content of k") + zf.writestr("j/l.baz", b"content of l") + zf.writestr("j/m.bar", b"content of m") zf.filename = "alpharep.zip" return zf @@ -92,7 +100,7 @@ def zipfile_ondisk(self, alpharep): def test_iterdir_and_types(self, alpharep): root = zipfile.Path(alpharep) assert root.is_dir() - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() assert a.is_file() assert b.is_dir() assert g.is_dir() @@ -112,7 +120,7 @@ def test_is_file_missing(self, alpharep): @pass_alpharep def test_iterdir_on_file(self, alpharep): root = zipfile.Path(alpharep) - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() with self.assertRaises(ValueError): a.iterdir() @@ -127,7 +135,7 @@ def test_subdir_is_dir(self, alpharep): @pass_alpharep def test_open(self, alpharep): root = zipfile.Path(alpharep) - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() with a.open(encoding="utf-8") as strm: data = strm.read() self.assertEqual(data, "content of a") @@ -229,7 +237,7 @@ def test_open_missing_directory(self): @pass_alpharep def test_read(self, alpharep): root = zipfile.Path(alpharep) - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() assert a.read_text(encoding="utf-8") == "content of a" # Also check positional encoding arg (gh-101144). assert a.read_text("utf-8") == "content of a" @@ -295,7 +303,7 @@ def test_mutability(self, alpharep): reflect that change. """ root = zipfile.Path(alpharep) - a, b, g = root.iterdir() + a, b, g, j = root.iterdir() alpharep.writestr('foo.txt', 'foo') alpharep.writestr('bar/baz.txt', 'baz') assert any(child.name == 'foo.txt' for child in root.iterdir()) @@ -394,6 +402,13 @@ def test_suffixes(self, alpharep): e = root / '.hgrc' assert e.suffixes == [] + @pass_alpharep + def test_suffix_no_filename(self, alpharep): + alpharep.filename = None + root = zipfile.Path(alpharep) + assert root.joinpath('example').suffix == "" + assert root.joinpath('example').suffixes == [] + @pass_alpharep def test_stem(self, alpharep): """ @@ -411,6 +426,8 @@ def test_stem(self, alpharep): d = root / "d" assert d.stem == "d" + assert (root / ".gitignore").stem == ".gitignore" + @pass_alpharep def test_root_parent(self, alpharep): root = zipfile.Path(alpharep) @@ -442,12 +459,49 @@ def test_match_and_glob(self, alpharep): assert not root.match("*.txt") assert list(root.glob("b/c.*")) == [zipfile.Path(alpharep, "b/c.txt")] + assert list(root.glob("b/*.txt")) == [ + zipfile.Path(alpharep, "b/c.txt"), + zipfile.Path(alpharep, "b/f.txt"), + ] + @pass_alpharep + def test_glob_recursive(self, alpharep): + root = zipfile.Path(alpharep) files = root.glob("**/*.txt") assert all(each.match("*.txt") for each in files) assert list(root.glob("**/*.txt")) == list(root.rglob("*.txt")) + @pass_alpharep + def test_glob_subdirs(self, alpharep): + root = zipfile.Path(alpharep) + + assert list(root.glob("*/i.txt")) == [] + assert list(root.rglob("*/i.txt")) == [zipfile.Path(alpharep, "g/h/i.txt")] + + @pass_alpharep + def test_glob_does_not_overmatch_dot(self, alpharep): + root = zipfile.Path(alpharep) + + assert list(root.glob("*.xt")) == [] + + @pass_alpharep + def test_glob_single_char(self, alpharep): + root = zipfile.Path(alpharep) + + assert list(root.glob("a?txt")) == [zipfile.Path(alpharep, "a.txt")] + assert list(root.glob("a[.]txt")) == [zipfile.Path(alpharep, "a.txt")] + assert list(root.glob("a[?]txt")) == [] + + @pass_alpharep + def test_glob_chars(self, alpharep): + root = zipfile.Path(alpharep) + + assert list(root.glob("j/?.b[ai][nz]")) == [ + zipfile.Path(alpharep, "j/k.bin"), + zipfile.Path(alpharep, "j/l.baz"), + ] + def test_glob_empty(self): root = zipfile.Path(zipfile.ZipFile(io.BytesIO(), 'w')) with self.assertRaises(ValueError): diff --git a/Lib/test/test_zipfile/_path/write-alpharep.py b/Lib/test/test_zipfile/_path/write-alpharep.py new file mode 100644 index 0000000000000..48c09b537179f --- /dev/null +++ b/Lib/test/test_zipfile/_path/write-alpharep.py @@ -0,0 +1,4 @@ +from . import test_path + + +__name__ == '__main__' and test_path.build_alpharep_fixture().extractall('alpharep') diff --git a/Lib/zipfile/_path/__init__.py b/Lib/zipfile/_path/__init__.py index fd49a3ea91db5..78c413563bb2b 100644 --- a/Lib/zipfile/_path/__init__.py +++ b/Lib/zipfile/_path/__init__.py @@ -5,7 +5,8 @@ import contextlib import pathlib import re -import fnmatch + +from .glob import translate __all__ = ['Path'] @@ -296,21 +297,24 @@ def open(self, mode='r', *args, pwd=None, **kwargs): encoding, args, kwargs = _extract_text_encoding(*args, **kwargs) return io.TextIOWrapper(stream, encoding, *args, **kwargs) + def _base(self): + return pathlib.PurePosixPath(self.at or self.root.filename) + @property def name(self): - return pathlib.Path(self.at).name or self.filename.name + return self._base().name @property def suffix(self): - return pathlib.Path(self.at).suffix or self.filename.suffix + return self._base().suffix @property def suffixes(self): - return pathlib.Path(self.at).suffixes or self.filename.suffixes + return self._base().suffixes @property def stem(self): - return pathlib.Path(self.at).stem or self.filename.stem + return self._base().stem @property def filename(self): @@ -347,7 +351,7 @@ def iterdir(self): return filter(self._is_child, subs) def match(self, path_pattern): - return pathlib.Path(self.at).match(path_pattern) + return pathlib.PurePosixPath(self.at).match(path_pattern) def is_symlink(self): """ @@ -355,22 +359,13 @@ def is_symlink(self): """ return False - def _descendants(self): - for child in self.iterdir(): - yield child - if child.is_dir(): - yield from child._descendants() - def glob(self, pattern): if not pattern: raise ValueError(f"Unacceptable pattern: {pattern!r}") - matches = re.compile(fnmatch.translate(pattern)).fullmatch - return ( - child - for child in self._descendants() - if matches(str(child.relative_to(self))) - ) + prefix = re.escape(self.at) + matches = re.compile(prefix + translate(pattern)).fullmatch + return map(self._next, filter(matches, self.root.namelist())) def rglob(self, pattern): return self.glob(f'**/{pattern}') diff --git a/Lib/zipfile/_path/glob.py b/Lib/zipfile/_path/glob.py new file mode 100644 index 0000000000000..4a2e665e27078 --- /dev/null +++ b/Lib/zipfile/_path/glob.py @@ -0,0 +1,40 @@ +import re + + +def translate(pattern): + r""" + Given a glob pattern, produce a regex that matches it. + + >>> translate('*.txt') + '[^/]*\\.txt' + >>> translate('a?txt') + 'a.txt' + >>> translate('**/*') + '.*/[^/]*' + """ + return ''.join(map(replace, separate(pattern))) + + +def separate(pattern): + """ + Separate out character sets to avoid translating their contents. + + >>> [m.group(0) for m in separate('*.txt')] + ['*.txt'] + >>> [m.group(0) for m in separate('a[?]txt')] + ['a', '[?]', 'txt'] + """ + return re.finditer(r'([^\[]+)|(?P[\[].*?[\]])|([\[][^\]]*$)', pattern) + + +def replace(match): + """ + Perform the replacements for a match from :func:`separate`. + """ + + return match.group('set') or ( + re.escape(match.group(0)) + .replace('\\*\\*', r'.*') + .replace('\\*', r'[^/]*') + .replace('\\?', r'.') + ) diff --git a/Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst b/Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst new file mode 100644 index 0000000000000..bbc53d76decbc --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst @@ -0,0 +1,5 @@ +Fixed several bugs in zipfile.Path, including: in ``Path.match`, Windows +separators are no longer honored (and never were meant to be); Fixed +``name``/``suffix``/``suffixes``/``stem`` operations when no filename is +present and the Path is not at the root of the zipfile; Reworked glob for +performance and more correct matching behavior. From webhook-mailer at python.org Sat Jul 15 10:15:41 2023 From: webhook-mailer at python.org (jaraco) Date: Sat, 15 Jul 2023 14:15:41 -0000 Subject: [Python-checkins] [3.11] gh-106752: Sync with zipp 3.16.2 (GH-106757) (#106778) Message-ID: https://github.com/python/cpython/commit/465f5b09df5bb975ba6226712f2cf172b8421f0c commit: 465f5b09df5bb975ba6226712f2cf172b8421f0c branch: 3.11 author: Jason R. Coombs committer: jaraco date: 2023-07-15T10:15:38-04:00 summary: [3.11] gh-106752: Sync with zipp 3.16.2 (GH-106757) (#106778) * gh-106752: Sync with zipp 3.16.2 (#106757) * gh-106752: Sync with zipp 3.16.2 * Add blurb (cherry picked from commit 22980dc7c9dcec4b74fea815542601ef582c230e) * [3.11] gh-106752: Sync with zipp 3.16.2 (GH-106757) * gh-106752: Sync with zipp 3.16.2 * Add blurb. (cherry picked from commit 22980dc7c9dcec4b74fea815542601ef582c230e) Co-authored-by: Jason R. Coombs * Remove Python 3.12 concerns from changelog. files: A Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst M Lib/test/test_zipfile.py M Lib/zipfile.py diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index bb2c24c8c50c5..c8e0159765ec2 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -3443,6 +3443,13 @@ def test_suffixes(self, alpharep): e = root / '.hgrc' assert e.suffixes == [] + @pass_alpharep + def test_suffix_no_filename(self, alpharep): + alpharep.filename = None + root = zipfile.Path(alpharep) + assert root.joinpath('example').suffix == "" + assert root.joinpath('example').suffixes == [] + @pass_alpharep def test_stem(self, alpharep): """ @@ -3460,6 +3467,8 @@ def test_stem(self, alpharep): d = root / "d" assert d.stem == "d" + assert (root / ".gitignore").stem == ".gitignore" + @pass_alpharep def test_root_parent(self, alpharep): root = zipfile.Path(alpharep) diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 1dec6a8b97bc4..6189db5e3e420 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -2420,21 +2420,24 @@ def open(self, mode='r', *args, pwd=None, **kwargs): encoding, args, kwargs = _extract_text_encoding(*args, **kwargs) return io.TextIOWrapper(stream, encoding, *args, **kwargs) + def _base(self): + return pathlib.PurePosixPath(self.at or self.root.filename) + @property def name(self): - return pathlib.Path(self.at).name or self.filename.name + return self._base().name @property def suffix(self): - return pathlib.Path(self.at).suffix or self.filename.suffix + return self._base().suffix @property def suffixes(self): - return pathlib.Path(self.at).suffixes or self.filename.suffixes + return self._base().suffixes @property def stem(self): - return pathlib.Path(self.at).stem or self.filename.stem + return self._base().stem @property def filename(self): diff --git a/Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst b/Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst new file mode 100644 index 0000000000000..d36c97da49d1b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-14-16-54-13.gh-issue-106752.BT1Yxw.rst @@ -0,0 +1,3 @@ +Fixed several bug in zipfile.Path in +``name``/``suffix``/``suffixes``/``stem`` operations when no filename is +present and the Path is not at the root of the zipfile. From webhook-mailer at python.org Sat Jul 15 15:43:12 2023 From: webhook-mailer at python.org (rhettinger) Date: Sat, 15 Jul 2023 19:43:12 -0000 Subject: [Python-checkins] Add more examples to the recipe docs (GH-106782) Message-ID: https://github.com/python/cpython/commit/e2ec0bad67552e27174255db86dda90fc72e6694 commit: e2ec0bad67552e27174255db86dda90fc72e6694 branch: main author: Raymond Hettinger committer: rhettinger date: 2023-07-15T14:43:09-05:00 summary: Add more examples to the recipe docs (GH-106782) Demonstrate that factor() works for large composites and large primes. files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index a2d1798a2c6da..f88525456ff93 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -1045,6 +1045,8 @@ The following recipes have a more mathematical flavor: def factor(n): "Prime factors of n." # factor(99) --> 3 3 11 + # factor(1_000_000_000_000_007) --> 47 59 360620266859 + # factor(1_000_000_000_000_403) --> 1000000000000403 for prime in sieve(math.isqrt(n) + 1): while True: quotient, remainder = divmod(n, prime) From webhook-mailer at python.org Sat Jul 15 16:09:24 2023 From: webhook-mailer at python.org (rhettinger) Date: Sat, 15 Jul 2023 20:09:24 -0000 Subject: [Python-checkins] [3.12] Add more examples to the recipe docs (GH-106782) (GH-106783) Message-ID: https://github.com/python/cpython/commit/18d98bacb62e4beb06e56deb18e7605bff499168 commit: 18d98bacb62e4beb06e56deb18e7605bff499168 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: rhettinger date: 2023-07-15T15:09:20-05:00 summary: [3.12] Add more examples to the recipe docs (GH-106782) (GH-106783) files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index a2d1798a2c6da..f88525456ff93 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -1045,6 +1045,8 @@ The following recipes have a more mathematical flavor: def factor(n): "Prime factors of n." # factor(99) --> 3 3 11 + # factor(1_000_000_000_000_007) --> 47 59 360620266859 + # factor(1_000_000_000_000_403) --> 1000000000000403 for prime in sieve(math.isqrt(n) + 1): while True: quotient, remainder = divmod(n, prime) From webhook-mailer at python.org Sat Jul 15 16:23:13 2023 From: webhook-mailer at python.org (hauntsaninja) Date: Sat, 15 Jul 2023 20:23:13 -0000 Subject: [Python-checkins] faq/library: remove outdated section (#105996) Message-ID: https://github.com/python/cpython/commit/d46a42fd8e8915e03cc211ab9163058b6365ab0f commit: d46a42fd8e8915e03cc211ab9163058b6365ab0f branch: main author: Mathieu Dupuy committer: hauntsaninja <12621235+hauntsaninja at users.noreply.github.com> date: 2023-07-15T13:23:10-07:00 summary: faq/library: remove outdated section (#105996) files: M Doc/faq/library.rst diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst index 597caaa778e1c..22f7f846d261d 100644 --- a/Doc/faq/library.rst +++ b/Doc/faq/library.rst @@ -669,41 +669,6 @@ and client-side web systems. A summary of available frameworks is maintained by Paul Boddie at https://wiki.python.org/moin/WebProgramming\ . -Cameron Laird maintains a useful set of pages about Python web technologies at -https://web.archive.org/web/20210224183619/http://phaseit.net/claird/comp.lang.python/web_python. - - -How can I mimic CGI form submission (METHOD=POST)? --------------------------------------------------- - -I would like to retrieve web pages that are the result of POSTing a form. Is -there existing code that would let me do this easily? - -Yes. Here's a simple example that uses :mod:`urllib.request`:: - - #!/usr/local/bin/python - - import urllib.request - - # build the query string - qs = "First=Josephine&MI=Q&Last=Public" - - # connect and send the server a path - req = urllib.request.urlopen('http://www.some-server.out-there' - '/cgi-bin/some-cgi-script', data=qs) - with req: - msg, hdrs = req.read(), req.info() - -Note that in general for percent-encoded POST operations, query strings must be -quoted using :func:`urllib.parse.urlencode`. For example, to send -``name=Guy Steele, Jr.``:: - - >>> import urllib.parse - >>> urllib.parse.urlencode({'name': 'Guy Steele, Jr.'}) - 'name=Guy+Steele%2C+Jr.' - -.. seealso:: :ref:`urllib-howto` for extensive examples. - What module should I use to help with generating HTML? ------------------------------------------------------ From webhook-mailer at python.org Sat Jul 15 18:43:01 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 15 Jul 2023 22:43:01 -0000 Subject: [Python-checkins] Docs: Normalize Argument Clinic How-To section capitalization (#106788) Message-ID: https://github.com/python/cpython/commit/8c177294899b621fe04ae755abd41b4d319dd4b5 commit: 8c177294899b621fe04ae755abd41b4d319dd4b5 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-16T00:42:58+02:00 summary: Docs: Normalize Argument Clinic How-To section capitalization (#106788) files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 4620b4617e345..0f99cb64994ab 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -27,7 +27,8 @@ Argument Clinic How-To version of Argument Clinic that ships with the next version of CPython *could* be totally incompatible and break all your code. -The Goals Of Argument Clinic + +The goals of Argument Clinic ============================ Argument Clinic's primary goal @@ -78,7 +79,7 @@ and it should be able to do many interesting and smart things with all the information you give it. -Basic Concepts And Usage +Basic concepts and usage ======================== Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``. @@ -141,7 +142,7 @@ For the sake of clarity, here's the terminology we'll use with Argument Clinic: a block.) -Converting Your First Function +Converting your first function ============================== The best way to get a sense of how Argument Clinic works is to @@ -558,7 +559,8 @@ Let's dive in! Congratulations, you've ported your first function to work with Argument Clinic! -Advanced Topics + +Advanced topics =============== Now that you've had some experience working with Argument Clinic, it's time @@ -636,7 +638,8 @@ after the last argument). Currently the generated code will use :c:func:`PyArg_ParseTuple`, but this will change soon. -Optional Groups + +Optional groups --------------- Some legacy functions have a tricky approach to parsing their arguments: @@ -899,6 +902,7 @@ available. For each converter it'll show you all the parameters it accepts, along with the default value for each parameter. Just run ``Tools/clinic/clinic.py --converters`` to see the full list. + Py_buffer --------- @@ -908,7 +912,6 @@ you *must* not call :c:func:`PyBuffer_Release` on the provided buffer. Argument Clinic generates code that does it for you (in the parsing function). - Advanced converters ------------------- @@ -975,6 +978,7 @@ value called ``NULL`` for just this reason: from Python's perspective it behaves like a default value of ``None``, but the C variable is initialized with ``NULL``. + Expressions specified as default values --------------------------------------- @@ -1032,7 +1036,6 @@ you're not permitted to use: * Tuple/list/set/dict literals. - Using a return converter ------------------------ @@ -1146,6 +1149,7 @@ then modifying it. Cloning is an all-or nothing proposition. Also, the function you are cloning from must have been previously defined in the current file. + Calling Python code ------------------- @@ -1380,6 +1384,7 @@ handle initialization and cleanup. You can see more examples of custom converters in the CPython source tree; grep the C files for the string ``CConverter``. + Writing a custom return converter --------------------------------- @@ -1394,8 +1399,9 @@ write your own return converter, please read ``Tools/clinic/clinic.py``, specifically the implementation of ``CReturnConverter`` and all its subclasses. + METH_O and METH_NOARGS ----------------------------------------------- +---------------------- To convert a function using ``METH_O``, make sure the function's single argument is using the ``object`` converter, and mark the @@ -1415,8 +1421,9 @@ any arguments. You can still use a self converter, a return converter, and specify a ``type`` argument to the object converter for ``METH_O``. + tp_new and tp_init functions ----------------------------------------------- +---------------------------- You can convert ``tp_new`` and ``tp_init`` functions. Just name them ``__new__`` or ``__init__`` as appropriate. Notes: @@ -1437,6 +1444,7 @@ them ``__new__`` or ``__init__`` as appropriate. Notes: (If your function doesn't support keywords, the parsing function generated will throw an exception if it receives any.) + Changing and redirecting Clinic's output ---------------------------------------- @@ -1721,7 +1729,7 @@ the file was not modified by hand before it gets overwritten. The #ifdef trick ----------------------------------------------- +---------------- If you're converting a function that isn't available on all platforms, there's a trick you can use to make life a little easier. The existing @@ -1801,7 +1809,6 @@ Argument Clinic added to your file (it'll be at the very bottom), then move it above the ``PyMethodDef`` structure where that macro is used. - Using Argument Clinic in Python files ------------------------------------- From webhook-mailer at python.org Sat Jul 15 18:43:49 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 15 Jul 2023 22:43:49 -0000 Subject: [Python-checkins] [3.12] Docs: Normalize Argument Clinic How-To section capitalization (GH-106788) (#106791) Message-ID: https://github.com/python/cpython/commit/30c127fcec28898df424c390c283ed76652ef7fe commit: 30c127fcec28898df424c390c283ed76652ef7fe branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-15T22:43:46Z summary: [3.12] Docs: Normalize Argument Clinic How-To section capitalization (GH-106788) (#106791) (cherry picked from commit 8c177294899b621fe04ae755abd41b4d319dd4b5) Co-authored-by: Erlend E. Aasland files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 4620b4617e345..0f99cb64994ab 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -27,7 +27,8 @@ Argument Clinic How-To version of Argument Clinic that ships with the next version of CPython *could* be totally incompatible and break all your code. -The Goals Of Argument Clinic + +The goals of Argument Clinic ============================ Argument Clinic's primary goal @@ -78,7 +79,7 @@ and it should be able to do many interesting and smart things with all the information you give it. -Basic Concepts And Usage +Basic concepts and usage ======================== Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``. @@ -141,7 +142,7 @@ For the sake of clarity, here's the terminology we'll use with Argument Clinic: a block.) -Converting Your First Function +Converting your first function ============================== The best way to get a sense of how Argument Clinic works is to @@ -558,7 +559,8 @@ Let's dive in! Congratulations, you've ported your first function to work with Argument Clinic! -Advanced Topics + +Advanced topics =============== Now that you've had some experience working with Argument Clinic, it's time @@ -636,7 +638,8 @@ after the last argument). Currently the generated code will use :c:func:`PyArg_ParseTuple`, but this will change soon. -Optional Groups + +Optional groups --------------- Some legacy functions have a tricky approach to parsing their arguments: @@ -899,6 +902,7 @@ available. For each converter it'll show you all the parameters it accepts, along with the default value for each parameter. Just run ``Tools/clinic/clinic.py --converters`` to see the full list. + Py_buffer --------- @@ -908,7 +912,6 @@ you *must* not call :c:func:`PyBuffer_Release` on the provided buffer. Argument Clinic generates code that does it for you (in the parsing function). - Advanced converters ------------------- @@ -975,6 +978,7 @@ value called ``NULL`` for just this reason: from Python's perspective it behaves like a default value of ``None``, but the C variable is initialized with ``NULL``. + Expressions specified as default values --------------------------------------- @@ -1032,7 +1036,6 @@ you're not permitted to use: * Tuple/list/set/dict literals. - Using a return converter ------------------------ @@ -1146,6 +1149,7 @@ then modifying it. Cloning is an all-or nothing proposition. Also, the function you are cloning from must have been previously defined in the current file. + Calling Python code ------------------- @@ -1380,6 +1384,7 @@ handle initialization and cleanup. You can see more examples of custom converters in the CPython source tree; grep the C files for the string ``CConverter``. + Writing a custom return converter --------------------------------- @@ -1394,8 +1399,9 @@ write your own return converter, please read ``Tools/clinic/clinic.py``, specifically the implementation of ``CReturnConverter`` and all its subclasses. + METH_O and METH_NOARGS ----------------------------------------------- +---------------------- To convert a function using ``METH_O``, make sure the function's single argument is using the ``object`` converter, and mark the @@ -1415,8 +1421,9 @@ any arguments. You can still use a self converter, a return converter, and specify a ``type`` argument to the object converter for ``METH_O``. + tp_new and tp_init functions ----------------------------------------------- +---------------------------- You can convert ``tp_new`` and ``tp_init`` functions. Just name them ``__new__`` or ``__init__`` as appropriate. Notes: @@ -1437,6 +1444,7 @@ them ``__new__`` or ``__init__`` as appropriate. Notes: (If your function doesn't support keywords, the parsing function generated will throw an exception if it receives any.) + Changing and redirecting Clinic's output ---------------------------------------- @@ -1721,7 +1729,7 @@ the file was not modified by hand before it gets overwritten. The #ifdef trick ----------------------------------------------- +---------------- If you're converting a function that isn't available on all platforms, there's a trick you can use to make life a little easier. The existing @@ -1801,7 +1809,6 @@ Argument Clinic added to your file (it'll be at the very bottom), then move it above the ``PyMethodDef`` structure where that macro is used. - Using Argument Clinic in Python files ------------------------------------- From webhook-mailer at python.org Sat Jul 15 18:50:47 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 15 Jul 2023 22:50:47 -0000 Subject: [Python-checkins] [3.11] Docs: Normalize Argument Clinic How-To section capitalization (GH-106788) (#106792) Message-ID: https://github.com/python/cpython/commit/c73f9c045e3adc130bda6a3e2b986b639c0621d5 commit: c73f9c045e3adc130bda6a3e2b986b639c0621d5 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-16T00:50:43+02:00 summary: [3.11] Docs: Normalize Argument Clinic How-To section capitalization (GH-106788) (#106792) (cherry picked from commit 8c177294899b621fe04ae755abd41b4d319dd4b5) Co-authored-by: Erlend E. Aasland files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index c4bf6ef764777..70b3e21d81a61 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -27,7 +27,8 @@ Argument Clinic How-To version of Argument Clinic that ships with the next version of CPython *could* be totally incompatible and break all your code. -The Goals Of Argument Clinic + +The goals of Argument Clinic ============================ Argument Clinic's primary goal @@ -78,7 +79,7 @@ and it should be able to do many interesting and smart things with all the information you give it. -Basic Concepts And Usage +Basic concepts and usage ======================== Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``. @@ -141,7 +142,7 @@ For the sake of clarity, here's the terminology we'll use with Argument Clinic: a block.) -Converting Your First Function +Converting your first function ============================== The best way to get a sense of how Argument Clinic works is to @@ -550,7 +551,8 @@ Let's dive in! Congratulations, you've ported your first function to work with Argument Clinic! -Advanced Topics + +Advanced topics =============== Now that you've had some experience working with Argument Clinic, it's time @@ -628,7 +630,8 @@ after the last argument). Currently the generated code will use :c:func:`PyArg_ParseTuple`, but this will change soon. -Optional Groups + +Optional groups --------------- Some legacy functions have a tricky approach to parsing their arguments: @@ -888,6 +891,7 @@ available. For each converter it'll show you all the parameters it accepts, along with the default value for each parameter. Just run ``Tools/clinic/clinic.py --converters`` to see the full list. + Py_buffer --------- @@ -897,7 +901,6 @@ you *must* not call :c:func:`PyBuffer_Release` on the provided buffer. Argument Clinic generates code that does it for you (in the parsing function). - Advanced converters ------------------- @@ -964,6 +967,7 @@ value called ``NULL`` for just this reason: from Python's perspective it behaves like a default value of ``None``, but the C variable is initialized with ``NULL``. + Expressions specified as default values --------------------------------------- @@ -1021,7 +1025,6 @@ you're not permitted to use: * Tuple/list/set/dict literals. - Using a return converter ------------------------ @@ -1140,6 +1143,7 @@ then modifying it. Cloning is an all-or nothing proposition. Also, the function you are cloning from must have been previously defined in the current file. + Calling Python code ------------------- @@ -1374,6 +1378,7 @@ handle initialization and cleanup. You can see more examples of custom converters in the CPython source tree; grep the C files for the string ``CConverter``. + Writing a custom return converter --------------------------------- @@ -1388,8 +1393,9 @@ write your own return converter, please read ``Tools/clinic/clinic.py``, specifically the implementation of ``CReturnConverter`` and all its subclasses. + METH_O and METH_NOARGS ----------------------------------------------- +---------------------- To convert a function using ``METH_O``, make sure the function's single argument is using the ``object`` converter, and mark the @@ -1409,8 +1415,9 @@ any arguments. You can still use a self converter, a return converter, and specify a ``type`` argument to the object converter for ``METH_O``. + tp_new and tp_init functions ----------------------------------------------- +---------------------------- You can convert ``tp_new`` and ``tp_init`` functions. Just name them ``__new__`` or ``__init__`` as appropriate. Notes: @@ -1431,6 +1438,7 @@ them ``__new__`` or ``__init__`` as appropriate. Notes: (If your function doesn't support keywords, the parsing function generated will throw an exception if it receives any.) + Changing and redirecting Clinic's output ---------------------------------------- @@ -1715,7 +1723,7 @@ the file was not modified by hand before it gets overwritten. The #ifdef trick ----------------------------------------------- +---------------- If you're converting a function that isn't available on all platforms, there's a trick you can use to make life a little easier. The existing @@ -1795,7 +1803,6 @@ Argument Clinic added to your file (it'll be at the very bottom), then move it above the ``PyMethodDef`` structure where that macro is used. - Using Argument Clinic in Python files ------------------------------------- From webhook-mailer at python.org Sun Jul 16 03:03:28 2023 From: webhook-mailer at python.org (methane) Date: Sun, 16 Jul 2023 07:03:28 -0000 Subject: [Python-checkins] wasm: do not use inline comment in .editorconfig (GH-106610) Message-ID: https://github.com/python/cpython/commit/9aeb9d1e80a8b6ff3d613c6d7fc4efd49a9a8a79 commit: 9aeb9d1e80a8b6ff3d613c6d7fc4efd49a9a8a79 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: methane date: 2023-07-16T16:03:24+09:00 summary: wasm: do not use inline comment in .editorconfig (GH-106610) It is no longer valid since 0.15.0 https://github.com/editorconfig/specification/blob/v0.15/index.rstGH-no-inline-comments (cherry picked from commit 64c0890b697783db9b3f67e3bb4dcee1165a0b9b) Co-authored-by: Eisuke Kawashima files: M Tools/wasm/.editorconfig diff --git a/Tools/wasm/.editorconfig b/Tools/wasm/.editorconfig index da1aa6acaccc7..4de5fe5954d84 100644 --- a/Tools/wasm/.editorconfig +++ b/Tools/wasm/.editorconfig @@ -1,4 +1,5 @@ -root = false # This extends the root .editorconfig +# This extends the root .editorconfig +root = false [*.{html,js}] trim_trailing_whitespace = true From webhook-mailer at python.org Sun Jul 16 03:10:43 2023 From: webhook-mailer at python.org (methane) Date: Sun, 16 Jul 2023 07:10:43 -0000 Subject: [Python-checkins] wasm: do not use inline comment in .editorconfig (GH-106610) Message-ID: https://github.com/python/cpython/commit/f0df92a9ce433a700b7a464d1ca0615e9fe478b2 commit: f0df92a9ce433a700b7a464d1ca0615e9fe478b2 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: methane date: 2023-07-16T16:10:39+09:00 summary: wasm: do not use inline comment in .editorconfig (GH-106610) It is no longer valid since 0.15.0 https://github.com/editorconfig/specification/blob/v0.15/index.rstGH-no-inline-comments (cherry picked from commit 64c0890b697783db9b3f67e3bb4dcee1165a0b9b) Co-authored-by: Eisuke Kawashima files: M Tools/wasm/.editorconfig diff --git a/Tools/wasm/.editorconfig b/Tools/wasm/.editorconfig index da1aa6acaccc7..4de5fe5954d84 100644 --- a/Tools/wasm/.editorconfig +++ b/Tools/wasm/.editorconfig @@ -1,4 +1,5 @@ -root = false # This extends the root .editorconfig +# This extends the root .editorconfig +root = false [*.{html,js}] trim_trailing_whitespace = true From webhook-mailer at python.org Sun Jul 16 03:26:30 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 16 Jul 2023 07:26:30 -0000 Subject: [Python-checkins] Docs search: Replace jQuery with vanilla JavaScript (#106743) Message-ID: https://github.com/python/cpython/commit/c02ee4503151105dc892018ebc7f633e7f3f62f8 commit: c02ee4503151105dc892018ebc7f633e7f3f62f8 branch: main author: Hugo van Kemenade committer: hugovk date: 2023-07-16T10:26:26+03:00 summary: Docs search: Replace jQuery with vanilla JavaScript (#106743) * Replace jQuery with vanilla JavaScript * Switch 'var' to 'const' or 'let' files: M Doc/tools/templates/search.html diff --git a/Doc/tools/templates/search.html b/Doc/tools/templates/search.html index f2ac2ea0f0987..852974461380f 100644 --- a/Doc/tools/templates/search.html +++ b/Doc/tools/templates/search.html @@ -1,48 +1,62 @@ {% extends "!search.html" %} {% block extrahead %} {{ super() }} + -{% endblock %} \ No newline at end of file +{% endblock %} From webhook-mailer at python.org Sun Jul 16 03:30:02 2023 From: webhook-mailer at python.org (methane) Date: Sun, 16 Jul 2023 07:30:02 -0000 Subject: [Python-checkins] Doc: devmode: add -Xdev option to example (#106253) Message-ID: https://github.com/python/cpython/commit/83bd568d2b57337a91ef046c1f52f9ebb03a7803 commit: 83bd568d2b57337a91ef046c1f52f9ebb03a7803 branch: main author: Simone Rubino committer: methane date: 2023-07-16T16:29:58+09:00 summary: Doc: devmode: add -Xdev option to example (#106253) files: M Doc/library/devmode.rst diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst index 977735990ffe9..b2bad48a07e27 100644 --- a/Doc/library/devmode.rst +++ b/Doc/library/devmode.rst @@ -198,7 +198,7 @@ descriptor" error when finalizing the file object: .. code-block:: shell-session - $ python script.py + $ python -X dev script.py import os script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> main() From webhook-mailer at python.org Sun Jul 16 03:32:30 2023 From: webhook-mailer at python.org (methane) Date: Sun, 16 Jul 2023 07:32:30 -0000 Subject: [Python-checkins] Doc: devmode: add -Xdev option to example (GH-106253) Message-ID: https://github.com/python/cpython/commit/e4658bf44ea5da5a2b8934a336c396d6815a4586 commit: e4658bf44ea5da5a2b8934a336c396d6815a4586 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: methane date: 2023-07-16T16:32:26+09:00 summary: Doc: devmode: add -Xdev option to example (GH-106253) Doc: devmode: add -Xdev option to example (GH-106253) (cherry picked from commit 83bd568d2b57337a91ef046c1f52f9ebb03a7803) Co-authored-by: Simone Rubino files: M Doc/library/devmode.rst diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst index 977735990ffe9..b2bad48a07e27 100644 --- a/Doc/library/devmode.rst +++ b/Doc/library/devmode.rst @@ -198,7 +198,7 @@ descriptor" error when finalizing the file object: .. code-block:: shell-session - $ python script.py + $ python -X dev script.py import os script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> main() From webhook-mailer at python.org Sun Jul 16 04:38:12 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 16 Jul 2023 08:38:12 -0000 Subject: [Python-checkins] [3.12] Docs search: Replace jQuery with vanilla JavaScript (GH-106743) (#106802) Message-ID: https://github.com/python/cpython/commit/30c8915a199690a74b0be4a83e377cf4c4ff9da4 commit: 30c8915a199690a74b0be4a83e377cf4c4ff9da4 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-16T11:38:08+03:00 summary: [3.12] Docs search: Replace jQuery with vanilla JavaScript (GH-106743) (#106802) Docs search: Replace jQuery with vanilla JavaScript (GH-106743) * Replace jQuery with vanilla JavaScript * Switch 'var' to 'const' or 'let' (cherry picked from commit c02ee4503151105dc892018ebc7f633e7f3f62f8) Co-authored-by: Hugo van Kemenade files: M Doc/tools/templates/search.html diff --git a/Doc/tools/templates/search.html b/Doc/tools/templates/search.html index f2ac2ea0f0987..852974461380f 100644 --- a/Doc/tools/templates/search.html +++ b/Doc/tools/templates/search.html @@ -1,48 +1,62 @@ {% extends "!search.html" %} {% block extrahead %} {{ super() }} + -{% endblock %} \ No newline at end of file +{% endblock %} From webhook-mailer at python.org Sun Jul 16 04:38:49 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 16 Jul 2023 08:38:49 -0000 Subject: [Python-checkins] [3.11] Docs search: Replace jQuery with vanilla JavaScript (GH-106743) (#106803) Message-ID: https://github.com/python/cpython/commit/501178ac9d13e95b5c70f28a01046b681acbf1f0 commit: 501178ac9d13e95b5c70f28a01046b681acbf1f0 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-16T11:38:46+03:00 summary: [3.11] Docs search: Replace jQuery with vanilla JavaScript (GH-106743) (#106803) Docs search: Replace jQuery with vanilla JavaScript (GH-106743) * Replace jQuery with vanilla JavaScript * Switch 'var' to 'const' or 'let' (cherry picked from commit c02ee4503151105dc892018ebc7f633e7f3f62f8) Co-authored-by: Hugo van Kemenade files: M Doc/tools/templates/search.html diff --git a/Doc/tools/templates/search.html b/Doc/tools/templates/search.html index f2ac2ea0f0987..852974461380f 100644 --- a/Doc/tools/templates/search.html +++ b/Doc/tools/templates/search.html @@ -1,48 +1,62 @@ {% extends "!search.html" %} {% block extrahead %} {{ super() }} + -{% endblock %} \ No newline at end of file +{% endblock %} From webhook-mailer at python.org Sun Jul 16 07:43:59 2023 From: webhook-mailer at python.org (methane) Date: Sun, 16 Jul 2023 11:43:59 -0000 Subject: [Python-checkins] Doc: devmode: add -Xdev option to example (GH-106253) Message-ID: https://github.com/python/cpython/commit/77104f551158cd399071e0d92b165d02569be6d7 commit: 77104f551158cd399071e0d92b165d02569be6d7 branch: 3.11 author: Inada Naoki committer: methane date: 2023-07-16T11:43:56Z summary: Doc: devmode: add -Xdev option to example (GH-106253) (cherry picked from commit 83bd568d2b57337a91ef046c1f52f9ebb03a7803) Co-authored-by: Simone Rubino files: M Doc/library/devmode.rst diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst index 44e7d4f541d81..fd82b3a5b8631 100644 --- a/Doc/library/devmode.rst +++ b/Doc/library/devmode.rst @@ -198,7 +198,7 @@ descriptor" error when finalizing the file object: .. code-block:: shell-session - $ python3 script.py + $ python3 -X dev script.py import os script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> main() From webhook-mailer at python.org Sun Jul 16 08:23:57 2023 From: webhook-mailer at python.org (methane) Date: Sun, 16 Jul 2023 12:23:57 -0000 Subject: [Python-checkins] Doc: fix section levels of devmode doc (#106801) Message-ID: https://github.com/python/cpython/commit/e58960160fcb4fce63177fcd9ef605f887377767 commit: e58960160fcb4fce63177fcd9ef605f887377767 branch: main author: Inada Naoki committer: methane date: 2023-07-16T21:23:54+09:00 summary: Doc: fix section levels of devmode doc (#106801) files: M Doc/library/devmode.rst diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst index b2bad48a07e27..80ac13b116c1d 100644 --- a/Doc/library/devmode.rst +++ b/Doc/library/devmode.rst @@ -16,7 +16,7 @@ setting the :envvar:`PYTHONDEVMODE` environment variable to ``1``. See also :ref:`Python debug build `. Effects of the Python Development Mode -====================================== +-------------------------------------- Enabling the Python Development Mode is similar to the following command, but with additional effects described below:: @@ -107,7 +107,7 @@ value can be read from :data:`sys.flags.dev_mode `. ResourceWarning Example -======================= +----------------------- Example of a script counting the number of lines of the text file specified in the command line:: @@ -171,7 +171,7 @@ application more deterministic and more reliable. Bad file descriptor error example -================================= +--------------------------------- Script displaying the first line of itself:: From webhook-mailer at python.org Sun Jul 16 08:25:38 2023 From: webhook-mailer at python.org (methane) Date: Sun, 16 Jul 2023 12:25:38 -0000 Subject: [Python-checkins] Doc: fix section levels of devmode doc (GH-106801) Message-ID: https://github.com/python/cpython/commit/0afd5305056c0be37ccaf7b367f1ea70ad9b02e3 commit: 0afd5305056c0be37ccaf7b367f1ea70ad9b02e3 branch: 3.12 author: Inada Naoki committer: methane date: 2023-07-16T21:25:35+09:00 summary: Doc: fix section levels of devmode doc (GH-106801) (cherry picked from commit e58960160fcb4fce63177fcd9ef605f887377767) files: M Doc/library/devmode.rst diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst index b2bad48a07e27..80ac13b116c1d 100644 --- a/Doc/library/devmode.rst +++ b/Doc/library/devmode.rst @@ -16,7 +16,7 @@ setting the :envvar:`PYTHONDEVMODE` environment variable to ``1``. See also :ref:`Python debug build `. Effects of the Python Development Mode -====================================== +-------------------------------------- Enabling the Python Development Mode is similar to the following command, but with additional effects described below:: @@ -107,7 +107,7 @@ value can be read from :data:`sys.flags.dev_mode `. ResourceWarning Example -======================= +----------------------- Example of a script counting the number of lines of the text file specified in the command line:: @@ -171,7 +171,7 @@ application more deterministic and more reliable. Bad file descriptor error example -================================= +--------------------------------- Script displaying the first line of itself:: From webhook-mailer at python.org Sun Jul 16 08:27:55 2023 From: webhook-mailer at python.org (methane) Date: Sun, 16 Jul 2023 12:27:55 -0000 Subject: [Python-checkins] Doc: fix section levels of devmode doc (GH-106801) Message-ID: https://github.com/python/cpython/commit/9532f0e6dd96aaba7aa50135ecab69923a674a15 commit: 9532f0e6dd96aaba7aa50135ecab69923a674a15 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: methane date: 2023-07-16T21:27:51+09:00 summary: Doc: fix section levels of devmode doc (GH-106801) Doc: fix section levels of devmode doc (GH-106801) (cherry picked from commit e58960160fcb4fce63177fcd9ef605f887377767) Co-authored-by: Inada Naoki files: M Doc/library/devmode.rst diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst index fd82b3a5b8631..90138dd2a75f9 100644 --- a/Doc/library/devmode.rst +++ b/Doc/library/devmode.rst @@ -16,7 +16,7 @@ setting the :envvar:`PYTHONDEVMODE` environment variable to ``1``. See also :ref:`Python debug build `. Effects of the Python Development Mode -====================================== +-------------------------------------- Enabling the Python Development Mode is similar to the following command, but with additional effects described below:: @@ -107,7 +107,7 @@ value can be read from :data:`sys.flags.dev_mode `. ResourceWarning Example -======================= +----------------------- Example of a script counting the number of lines of the text file specified in the command line:: @@ -171,7 +171,7 @@ application more deterministic and more reliable. Bad file descriptor error example -================================= +--------------------------------- Script displaying the first line of itself:: From webhook-mailer at python.org Sun Jul 16 11:16:37 2023 From: webhook-mailer at python.org (gvanrossum) Date: Sun, 16 Jul 2023 15:16:37 -0000 Subject: [Python-checkins] gh-106706: Streamline family syntax in cases generator DSL (#106716) Message-ID: https://github.com/python/cpython/commit/cc25ca16ee406db936dfbd2337cbd14b12ccc4b7 commit: cc25ca16ee406db936dfbd2337cbd14b12ccc4b7 branch: main author: Kevin Diem committer: gvanrossum date: 2023-07-16T08:16:34-07:00 summary: gh-106706: Streamline family syntax in cases generator DSL (#106716) >From `family(opname, STRUCTSIZE) = OPNAME + SPEC1 + ... + SPECn;` to `family(OPNAME, STRUCTSIZE) = SPEC1 + ... + SPECn;` files: A Misc/NEWS.d/next/Tools-Demos/2023-07-13-12-08-35.gh-issue-106706.29zp8E.rst M Python/bytecodes.c M Tools/cases_generator/generate_cases.py M Tools/cases_generator/interpreter_definition.md M Tools/cases_generator/test_generator.py diff --git a/Misc/NEWS.d/next/Tools-Demos/2023-07-13-12-08-35.gh-issue-106706.29zp8E.rst b/Misc/NEWS.d/next/Tools-Demos/2023-07-13-12-08-35.gh-issue-106706.29zp8E.rst new file mode 100644 index 0000000000000..bbd8e8eddda60 --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2023-07-13-12-08-35.gh-issue-106706.29zp8E.rst @@ -0,0 +1,3 @@ +Change bytecode syntax for families +to remove redundant name matching +pseudo syntax. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 3432b02771346..3c3992c068b06 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -284,8 +284,7 @@ dummy_func( res = Py_IsFalse(value) ? Py_True : Py_False; } - family(to_bool, INLINE_CACHE_ENTRIES_TO_BOOL) = { - TO_BOOL, + family(TO_BOOL, INLINE_CACHE_ENTRIES_TO_BOOL) = { TO_BOOL_ALWAYS_TRUE, TO_BOOL_BOOL, TO_BOOL_INT, @@ -372,8 +371,7 @@ dummy_func( ERROR_IF(res == NULL, error); } - family(binary_op, INLINE_CACHE_ENTRIES_BINARY_OP) = { - BINARY_OP, + family(BINARY_OP, INLINE_CACHE_ENTRIES_BINARY_OP) = { BINARY_OP_MULTIPLY_INT, BINARY_OP_ADD_INT, BINARY_OP_SUBTRACT_INT, @@ -507,8 +505,7 @@ dummy_func( macro(BINARY_OP_INPLACE_ADD_UNICODE) = _GUARD_BOTH_UNICODE + _BINARY_OP_INPLACE_ADD_UNICODE; - family(binary_subscr, INLINE_CACHE_ENTRIES_BINARY_SUBSCR) = { - BINARY_SUBSCR, + family(BINARY_SUBSCR, INLINE_CACHE_ENTRIES_BINARY_SUBSCR) = { BINARY_SUBSCR_DICT, BINARY_SUBSCR_GETITEM, BINARY_SUBSCR_LIST_INT, @@ -643,8 +640,7 @@ dummy_func( ERROR_IF(err, error); } - family(store_subscr, INLINE_CACHE_ENTRIES_STORE_SUBSCR) = { - STORE_SUBSCR, + family(STORE_SUBSCR, INLINE_CACHE_ENTRIES_STORE_SUBSCR) = { STORE_SUBSCR_DICT, STORE_SUBSCR_LIST_INT, }; @@ -921,8 +917,7 @@ dummy_func( ERROR_IF(iter == NULL, error); } - family(send, INLINE_CACHE_ENTRIES_SEND) = { - SEND, + family(SEND, INLINE_CACHE_ENTRIES_SEND) = { SEND_GEN, }; @@ -1134,8 +1129,7 @@ dummy_func( } } - family(unpack_sequence, INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE) = { - UNPACK_SEQUENCE, + family(UNPACK_SEQUENCE, INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE) = { UNPACK_SEQUENCE_TWO_TUPLE, UNPACK_SEQUENCE_TUPLE, UNPACK_SEQUENCE_LIST, @@ -1198,8 +1192,7 @@ dummy_func( ERROR_IF(res == 0, error); } - family(store_attr, INLINE_CACHE_ENTRIES_STORE_ATTR) = { - STORE_ATTR, + family(STORE_ATTR, INLINE_CACHE_ENTRIES_STORE_ATTR) = { STORE_ATTR_INSTANCE_VALUE, STORE_ATTR_SLOT, STORE_ATTR_WITH_HINT, @@ -1298,8 +1291,7 @@ dummy_func( macro(LOAD_FROM_DICT_OR_GLOBALS) = _LOAD_FROM_DICT_OR_GLOBALS; - family(load_global, INLINE_CACHE_ENTRIES_LOAD_GLOBAL) = { - LOAD_GLOBAL, + family(LOAD_GLOBAL, INLINE_CACHE_ENTRIES_LOAD_GLOBAL) = { LOAD_GLOBAL_MODULE, LOAD_GLOBAL_BUILTIN, }; @@ -1647,8 +1639,7 @@ dummy_func( GO_TO_INSTRUCTION(LOAD_SUPER_ATTR); } - family(load_super_attr, INLINE_CACHE_ENTRIES_LOAD_SUPER_ATTR) = { - LOAD_SUPER_ATTR, + family(LOAD_SUPER_ATTR, INLINE_CACHE_ENTRIES_LOAD_SUPER_ATTR) = { LOAD_SUPER_ATTR_ATTR, LOAD_SUPER_ATTR_METHOD, }; @@ -1750,8 +1741,7 @@ dummy_func( } } - family(load_attr, INLINE_CACHE_ENTRIES_LOAD_ATTR) = { - LOAD_ATTR, + family(LOAD_ATTR, INLINE_CACHE_ENTRIES_LOAD_ATTR) = { LOAD_ATTR_INSTANCE_VALUE, LOAD_ATTR_MODULE, LOAD_ATTR_WITH_HINT, @@ -2048,8 +2038,7 @@ dummy_func( Py_DECREF(owner); } - family(compare_op, INLINE_CACHE_ENTRIES_COMPARE_OP) = { - COMPARE_OP, + family(COMPARE_OP, INLINE_CACHE_ENTRIES_COMPARE_OP) = { COMPARE_OP_FLOAT, COMPARE_OP_INT, COMPARE_OP_STR, @@ -2350,8 +2339,7 @@ dummy_func( // This is optimized by skipping that instruction and combining // its effect (popping 'iter' instead of pushing 'next'.) - family(for_iter, INLINE_CACHE_ENTRIES_FOR_ITER) = { - FOR_ITER, + family(FOR_ITER, INLINE_CACHE_ENTRIES_FOR_ITER) = { FOR_ITER_LIST, FOR_ITER_TUPLE, FOR_ITER_RANGE, @@ -2810,8 +2798,7 @@ dummy_func( // Cache layout: counter/1, func_version/2 // Neither CALL_INTRINSIC_1/2 nor CALL_FUNCTION_EX are members! - family(call, INLINE_CACHE_ENTRIES_CALL) = { - CALL, + family(CALL, INLINE_CACHE_ENTRIES_CALL) = { CALL_BOUND_METHOD_EXACT_ARGS, CALL_PY_EXACT_ARGS, CALL_PY_WITH_DEFAULTS, diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 6589289121863..3edd8ee51ba64 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -438,7 +438,7 @@ def write(self, out: Formatter, tier: Tiers = TIER_ONE) -> None: """Write one instruction, sans prologue and epilogue.""" # Write a static assertion that a family's cache size is correct if family := self.family: - if self.name == family.members[0]: + if self.name == family.name: if cache_size := family.size: out.emit( f"static_assert({cache_size} == " @@ -831,7 +831,7 @@ def find_predictions(self) -> None: def map_families(self) -> None: """Link instruction names back to their family, if they have one.""" for family in self.families.values(): - for member in family.members: + for member in [family.name] + family.members: if member_instr := self.instrs.get(member): if member_instr.family not in (family, None): self.error( @@ -855,8 +855,11 @@ def check_families(self) -> None: - All members must have the same cache, input and output effects """ for family in self.families.values(): - if len(family.members) < 2: - self.error(f"Family {family.name!r} has insufficient members", family) + if family.name not in self.macro_instrs and family.name not in self.instrs: + self.error( + f"Family {family.name!r} has unknown instruction {family.name!r}", + family, + ) members = [ member for member in family.members @@ -867,10 +870,8 @@ def check_families(self) -> None: self.error( f"Family {family.name!r} has unknown members: {unknown}", family ) - if len(members) < 2: - continue - expected_effects = self.effect_counts(members[0]) - for member in members[1:]: + expected_effects = self.effect_counts(family.name) + for member in members: member_effects = self.effect_counts(member) if member_effects != expected_effects: self.error( @@ -1311,11 +1312,10 @@ def write_metadata(self) -> None: self.out.emit("") self.out.emit("_specializations = {") for name, family in self.families.items(): - assert len(family.members) > 1 with self.out.indent(): - self.out.emit(f"\"{family.members[0]}\": [") + self.out.emit(f"\"{family.name}\": [") with self.out.indent(): - for m in family.members[1:]: + for m in family.members: self.out.emit(f"\"{m}\",") self.out.emit(f"],") self.out.emit("}") @@ -1551,9 +1551,8 @@ def write_macro(self, mac: MacroInstruction) -> None: self.out.emit(f"next_instr += {cache_adjust};") if ( - last_instr - and (family := last_instr.family) - and mac.name == family.members[0] + (family := self.families.get(mac.name)) + and mac.name == family.name and (cache_size := family.size) ): self.out.emit( diff --git a/Tools/cases_generator/interpreter_definition.md b/Tools/cases_generator/interpreter_definition.md index c03870ef59eb4..f141848631d04 100644 --- a/Tools/cases_generator/interpreter_definition.md +++ b/Tools/cases_generator/interpreter_definition.md @@ -347,7 +347,7 @@ For explanations see "Generating the interpreter" below.) ### Defining an instruction family -A _family_ represents a specializable instruction and its specializations. +A _family_ maps a specializable instruction to its specializations. Example: These opcodes all share the same instruction format): ```C diff --git a/Tools/cases_generator/test_generator.py b/Tools/cases_generator/test_generator.py index e374ac41e6a94..e44273429b740 100644 --- a/Tools/cases_generator/test_generator.py +++ b/Tools/cases_generator/test_generator.py @@ -287,7 +287,7 @@ def test_macro_instruction(): inst(OP3, (unused/5, arg2, left, right -- res)) { res = op3(arg2, left, right); } - family(op, INLINE_CACHE_ENTRIES_OP) = { OP, OP3 }; + family(OP, INLINE_CACHE_ENTRIES_OP) = { OP3 }; """ output = """ TARGET(OP1) { From webhook-mailer at python.org Sun Jul 16 11:30:42 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sun, 16 Jul 2023 15:30:42 -0000 Subject: [Python-checkins] gh-105726: Add `__slots__` to `AbstractContextManager` and `AbstractAsyncContextManager` (#106771) Message-ID: https://github.com/python/cpython/commit/55408f86d78259f18c56c5e1ea51e0f8dcdbeb67 commit: 55408f86d78259f18c56c5e1ea51e0f8dcdbeb67 branch: main author: Grigoriev Semyon <33061489+grigoriev-semyon at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-16T15:30:39Z summary: gh-105726: Add `__slots__` to `AbstractContextManager` and `AbstractAsyncContextManager` (#106771) Co-authored-by: Kumar Aditya files: A Misc/NEWS.d/next/Library/2023-07-15-12-52-50.gh-issue-105726.NGthO8.rst M Lib/contextlib.py M Lib/test/test_contextlib.py M Lib/test/test_contextlib_async.py diff --git a/Lib/contextlib.py b/Lib/contextlib.py index b5acbcb9e6d77..95947aceccc30 100644 --- a/Lib/contextlib.py +++ b/Lib/contextlib.py @@ -20,6 +20,8 @@ class AbstractContextManager(abc.ABC): __class_getitem__ = classmethod(GenericAlias) + __slots__ = () + def __enter__(self): """Return `self` upon entering the runtime context.""" return self @@ -42,6 +44,8 @@ class AbstractAsyncContextManager(abc.ABC): __class_getitem__ = classmethod(GenericAlias) + __slots__ = () + async def __aenter__(self): """Return `self` upon entering the runtime context.""" return self diff --git a/Lib/test/test_contextlib.py b/Lib/test/test_contextlib.py index 0f8351ab8108a..ecc5a43dad43d 100644 --- a/Lib/test/test_contextlib.py +++ b/Lib/test/test_contextlib.py @@ -24,6 +24,16 @@ def __exit__(self, *args): manager = DefaultEnter() self.assertIs(manager.__enter__(), manager) + def test_slots(self): + class DefaultContextManager(AbstractContextManager): + __slots__ = () + + def __exit__(self, *args): + super().__exit__(*args) + + with self.assertRaises(AttributeError): + DefaultContextManager().var = 42 + def test_exit_is_abstract(self): class MissingExit(AbstractContextManager): pass diff --git a/Lib/test/test_contextlib_async.py b/Lib/test/test_contextlib_async.py index 3d43ed0fcab16..bb72ae74e5845 100644 --- a/Lib/test/test_contextlib_async.py +++ b/Lib/test/test_contextlib_async.py @@ -37,6 +37,18 @@ async def __aexit__(self, *args): async with manager as context: self.assertIs(manager, context) + @_async_test + async def test_slots(self): + class DefaultAsyncContextManager(AbstractAsyncContextManager): + __slots__ = () + + async def __aexit__(self, *args): + await super().__aexit__(*args) + + with self.assertRaises(AttributeError): + manager = DefaultAsyncContextManager() + manager.var = 42 + @_async_test async def test_async_gen_propagates_generator_exit(self): # A regression test for https://bugs.python.org/issue33786. diff --git a/Misc/NEWS.d/next/Library/2023-07-15-12-52-50.gh-issue-105726.NGthO8.rst b/Misc/NEWS.d/next/Library/2023-07-15-12-52-50.gh-issue-105726.NGthO8.rst new file mode 100644 index 0000000000000..434f93240eccd --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-15-12-52-50.gh-issue-105726.NGthO8.rst @@ -0,0 +1,3 @@ +Added ``__slots__`` to :class:`contextlib.AbstractContextManager` and :class:`contextlib.AbstractAsyncContextManager` +so that child classes can use ``__slots__``. + From webhook-mailer at python.org Sun Jul 16 13:14:11 2023 From: webhook-mailer at python.org (JulienPalard) Date: Sun, 16 Jul 2023 17:14:11 -0000 Subject: [Python-checkins] Fix the french used in the email documentation (GH-106279) Message-ID: https://github.com/python/cpython/commit/4dc593477a2e8a5c22e3e2346aaae05ca46b12cb commit: 4dc593477a2e8a5c22e3e2346aaae05ca46b12cb branch: main author: Jean-Baptiste Poupon committer: JulienPalard date: 2023-07-16T19:14:08+02:00 summary: Fix the french used in the email documentation (GH-106279) * Fix the french used in the email documentation The french used in one of the example was either machine translated a while ago or written by someone who does not speak french. Fixed it by using grammatically correct french. files: M Doc/includes/email-alternative.py M Doc/library/email.examples.rst diff --git a/Doc/includes/email-alternative.py b/Doc/includes/email-alternative.py index df7ca6f3faa33..26b302b495c7a 100644 --- a/Doc/includes/email-alternative.py +++ b/Doc/includes/email-alternative.py @@ -8,14 +8,14 @@ # Create the base text message. msg = EmailMessage() -msg['Subject'] = "Ayons asperges pour le d?jeuner" +msg['Subject'] = "Pourquoi pas des asperges pour ce midi ?" msg['From'] = Address("Pep? Le Pew", "pepe", "example.com") msg['To'] = (Address("Penelope Pussycat", "penelope", "example.com"), Address("Fabrette Pussycat", "fabrette", "example.com")) msg.set_content("""\ Salut! -Cela ressemble ? un excellent recipie[1] d?jeuner. +Cette recette [1] sera s?rement un tr?s bon repas. [1] http://www.yummly.com/recipe/Roasted-Asparagus-Epicurious-203718 @@ -31,10 +31,10 @@

Salut!

-

Cela ressemble ? un excellent +

Cette - recipie - d?jeuner. + recette + sera s?rement un tr?s bon repas.

diff --git a/Doc/library/email.examples.rst b/Doc/library/email.examples.rst index fc964622809d0..492a8354d8bf8 100644 --- a/Doc/library/email.examples.rst +++ b/Doc/library/email.examples.rst @@ -55,11 +55,11 @@ Up to the prompt, the output from the above is: To: Penelope Pussycat , Fabrette Pussycat From: Pep? Le Pew - Subject: Ayons asperges pour le d?jeuner + Subject: Pourquoi pas des asperges pour ce midi ? Salut! - Cela ressemble ? un excellent recipie[1] d?jeuner. + Cette recette [1] sera s?rement un tr?s bon repas. .. rubric:: Footnotes From webhook-mailer at python.org Sun Jul 16 19:05:28 2023 From: webhook-mailer at python.org (gvanrossum) Date: Sun, 16 Jul 2023 23:05:28 -0000 Subject: [Python-checkins] gh-105540: Convert `pytest` tests of `cases_generator` to regular tests (#106713) Message-ID: https://github.com/python/cpython/commit/c41320701b28904064c89a0a29775efed6b6d053 commit: c41320701b28904064c89a0a29775efed6b6d053 branch: main author: Nikita Sobolev committer: gvanrossum date: 2023-07-16T16:05:24-07:00 summary: gh-105540: Convert `pytest` tests of `cases_generator` to regular tests (#106713) files: A Lib/test/test_generated_cases.py D Tools/cases_generator/test_generator.py M Tools/cases_generator/generate_cases.py diff --git a/Tools/cases_generator/test_generator.py b/Lib/test/test_generated_cases.py similarity index 58% rename from Tools/cases_generator/test_generator.py rename to Lib/test/test_generated_cases.py index e44273429b740..ba0e5e8b0f695 100644 --- a/Tools/cases_generator/test_generator.py +++ b/Lib/test/test_generated_cases.py @@ -1,97 +1,148 @@ -# Sorry for using pytest, these tests are mostly just for me. -# Use pytest -vv for best results. - import tempfile +import unittest +import os + +from test import test_tools + +test_tools.skip_if_missing('cases_generator') +with test_tools.imports_under_tool('cases_generator'): + import generate_cases + from parser import StackEffect + + +class TestEffects(unittest.TestCase): + def test_effect_sizes(self): + input_effects = [ + x := StackEffect("x", "", "", ""), + y := StackEffect("y", "", "", "oparg"), + z := StackEffect("z", "", "", "oparg*2"), + ] + output_effects = [ + StackEffect("a", "", "", ""), + StackEffect("b", "", "", "oparg*4"), + StackEffect("c", "", "", ""), + ] + other_effects = [ + StackEffect("p", "", "", "oparg<<1"), + StackEffect("q", "", "", ""), + StackEffect("r", "", "", ""), + ] + self.assertEqual(generate_cases.effect_size(x), (1, "")) + self.assertEqual(generate_cases.effect_size(y), (0, "oparg")) + self.assertEqual(generate_cases.effect_size(z), (0, "oparg*2")) + + self.assertEqual( + generate_cases.list_effect_size(input_effects), + (1, "oparg + oparg*2"), + ) + self.assertEqual( + generate_cases.list_effect_size(output_effects), + (2, "oparg*4"), + ) + self.assertEqual( + generate_cases.list_effect_size(other_effects), + (2, "(oparg<<1)"), + ) + + self.assertEqual( + generate_cases.string_effect_size( + generate_cases.list_effect_size(input_effects), + ), "1 + oparg + oparg*2", + ) + self.assertEqual( + generate_cases.string_effect_size( + generate_cases.list_effect_size(output_effects), + ), + "2 + oparg*4", + ) + self.assertEqual( + generate_cases.string_effect_size( + generate_cases.list_effect_size(other_effects), + ), + "2 + (oparg<<1)", + ) + + +class TestGeneratedCases(unittest.TestCase): + def setUp(self) -> None: + super().setUp() -import generate_cases -from parser import StackEffect - - -def test_effect_sizes(): - input_effects = [ - x := StackEffect("x", "", "", ""), - y := StackEffect("y", "", "", "oparg"), - z := StackEffect("z", "", "", "oparg*2"), - ] - output_effects = [ - StackEffect("a", "", "", ""), - StackEffect("b", "", "", "oparg*4"), - StackEffect("c", "", "", ""), - ] - other_effects = [ - StackEffect("p", "", "", "oparg<<1"), - StackEffect("q", "", "", ""), - StackEffect("r", "", "", ""), - ] - assert generate_cases.effect_size(x) == (1, "") - assert generate_cases.effect_size(y) == (0, "oparg") - assert generate_cases.effect_size(z) == (0, "oparg*2") - - assert generate_cases.list_effect_size(input_effects) == (1, "oparg + oparg*2") - assert generate_cases.list_effect_size(output_effects) == (2, "oparg*4") - assert generate_cases.list_effect_size(other_effects) == (2, "(oparg<<1)") - - assert generate_cases.string_effect_size(generate_cases.list_effect_size(input_effects)) == "1 + oparg + oparg*2" - assert generate_cases.string_effect_size(generate_cases.list_effect_size(output_effects)) == "2 + oparg*4" - assert generate_cases.string_effect_size(generate_cases.list_effect_size(other_effects)) == "2 + (oparg<<1)" - - -def run_cases_test(input: str, expected: str): - temp_input = tempfile.NamedTemporaryFile("w+") - temp_input.write(generate_cases.BEGIN_MARKER) - temp_input.write(input) - temp_input.write(generate_cases.END_MARKER) - temp_input.flush() - temp_output = tempfile.NamedTemporaryFile("w+") - temp_metadata = tempfile.NamedTemporaryFile("w+") - temp_pymetadata = tempfile.NamedTemporaryFile("w+") - temp_executor = tempfile.NamedTemporaryFile("w+") - a = generate_cases.Analyzer( - [temp_input.name], - temp_output.name, - temp_metadata.name, - temp_pymetadata.name, - temp_executor.name, - ) - a.parse() - a.analyze() - if a.errors: - raise RuntimeError(f"Found {a.errors} errors") - a.write_instructions() - temp_output.seek(0) - lines = temp_output.readlines() - while lines and lines[0].startswith("// "): - lines.pop(0) - actual = "".join(lines) - # if actual.rstrip() != expected.rstrip(): - # print("Actual:") - # print(actual) - # print("Expected:") - # print(expected) - # print("End") - assert actual.rstrip() == expected.rstrip() - -def test_inst_no_args(): - input = """ + self.temp_dir = tempfile.gettempdir() + self.temp_input_filename = os.path.join(self.temp_dir, "input.txt") + self.temp_output_filename = os.path.join(self.temp_dir, "output.txt") + self.temp_metadata_filename = os.path.join(self.temp_dir, "metadata.txt") + self.temp_pymetadata_filename = os.path.join(self.temp_dir, "pymetadata.txt") + self.temp_executor_filename = os.path.join(self.temp_dir, "executor.txt") + + def tearDown(self) -> None: + for filename in [ + self.temp_input_filename, + self.temp_output_filename, + self.temp_metadata_filename, + self.temp_pymetadata_filename, + self.temp_executor_filename, + ]: + try: + os.remove(filename) + except: + pass + super().tearDown() + + def run_cases_test(self, input: str, expected: str): + with open(self.temp_input_filename, "w+") as temp_input: + temp_input.write(generate_cases.BEGIN_MARKER) + temp_input.write(input) + temp_input.write(generate_cases.END_MARKER) + temp_input.flush() + + a = generate_cases.Analyzer( + [self.temp_input_filename], + self.temp_output_filename, + self.temp_metadata_filename, + self.temp_pymetadata_filename, + self.temp_executor_filename, + ) + a.parse() + a.analyze() + if a.errors: + raise RuntimeError(f"Found {a.errors} errors") + a.write_instructions() + + with open(self.temp_output_filename) as temp_output: + lines = temp_output.readlines() + while lines and lines[0].startswith("// "): + lines.pop(0) + actual = "".join(lines) + # if actual.rstrip() != expected.rstrip(): + # print("Actual:") + # print(actual) + # print("Expected:") + # print(expected) + # print("End") + + self.assertEqual(actual.rstrip(), expected.rstrip()) + + def test_inst_no_args(self): + input = """ inst(OP, (--)) { spam(); } """ - output = """ + output = """ TARGET(OP) { spam(); DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_inst_one_pop(): - input = """ + def test_inst_one_pop(self): + input = """ inst(OP, (value --)) { spam(); } """ - output = """ + output = """ TARGET(OP) { PyObject *value = stack_pointer[-1]; spam(); @@ -99,15 +150,15 @@ def test_inst_one_pop(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_inst_one_push(): - input = """ + def test_inst_one_push(self): + input = """ inst(OP, (-- res)) { spam(); } """ - output = """ + output = """ TARGET(OP) { PyObject *res; spam(); @@ -116,15 +167,15 @@ def test_inst_one_push(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_inst_one_push_one_pop(): - input = """ + def test_inst_one_push_one_pop(self): + input = """ inst(OP, (value -- res)) { spam(); } """ - output = """ + output = """ TARGET(OP) { PyObject *value = stack_pointer[-1]; PyObject *res; @@ -133,15 +184,15 @@ def test_inst_one_push_one_pop(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_binary_op(): - input = """ + def test_binary_op(self): + input = """ inst(OP, (left, right -- res)) { spam(); } """ - output = """ + output = """ TARGET(OP) { PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; @@ -152,15 +203,15 @@ def test_binary_op(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_overlap(): - input = """ + def test_overlap(self): + input = """ inst(OP, (left, right -- left, result)) { spam(); } """ - output = """ + output = """ TARGET(OP) { PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; @@ -170,10 +221,10 @@ def test_overlap(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_predictions_and_eval_breaker(): - input = """ + def test_predictions_and_eval_breaker(self): + input = """ inst(OP1, (--)) { } inst(OP3, (arg -- res)) { @@ -181,7 +232,7 @@ def test_predictions_and_eval_breaker(): CHECK_EVAL_BREAKER(); } """ - output = """ + output = """ TARGET(OP1) { PREDICTED(OP1); DISPATCH(); @@ -196,43 +247,43 @@ def test_predictions_and_eval_breaker(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_error_if_plain(): - input = """ + def test_error_if_plain(self): + input = """ inst(OP, (--)) { ERROR_IF(cond, label); } """ - output = """ + output = """ TARGET(OP) { if (cond) goto label; DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_error_if_plain_with_comment(): - input = """ + def test_error_if_plain_with_comment(self): + input = """ inst(OP, (--)) { ERROR_IF(cond, label); // Comment is ok } """ - output = """ + output = """ TARGET(OP) { if (cond) goto label; DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_error_if_pop(): - input = """ + def test_error_if_pop(self): + input = """ inst(OP, (left, right -- res)) { ERROR_IF(cond, label); } """ - output = """ + output = """ TARGET(OP) { PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; @@ -243,14 +294,14 @@ def test_error_if_pop(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_cache_effect(): - input = """ + def test_cache_effect(self): + input = """ inst(OP, (counter/1, extra/2, value --)) { } """ - output = """ + output = """ TARGET(OP) { PyObject *value = stack_pointer[-1]; uint16_t counter = read_u16(&next_instr[0].cache); @@ -260,23 +311,23 @@ def test_cache_effect(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_suppress_dispatch(): - input = """ + def test_suppress_dispatch(self): + input = """ inst(OP, (--)) { goto somewhere; } """ - output = """ + output = """ TARGET(OP) { goto somewhere; } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_macro_instruction(): - input = """ + def test_macro_instruction(self): + input = """ inst(OP1, (counter/1, left, right -- left, right)) { op1(left, right); } @@ -289,7 +340,7 @@ def test_macro_instruction(): } family(OP, INLINE_CACHE_ENTRIES_OP) = { OP3 }; """ - output = """ + output = """ TARGET(OP1) { PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; @@ -339,15 +390,15 @@ def test_macro_instruction(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_array_input(): - input = """ + def test_array_input(self): + input = """ inst(OP, (below, values[oparg*2], above --)) { spam(); } """ - output = """ + output = """ TARGET(OP) { PyObject *above = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg*2)); @@ -358,15 +409,15 @@ def test_array_input(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_array_output(): - input = """ + def test_array_output(self): + input = """ inst(OP, (unused, unused -- below, values[oparg*3], above)) { spam(values, oparg); } """ - output = """ + output = """ TARGET(OP) { PyObject *below; PyObject **values = stack_pointer - (2) + 1; @@ -378,15 +429,15 @@ def test_array_output(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_array_input_output(): - input = """ + def test_array_input_output(self): + input = """ inst(OP, (values[oparg] -- values[oparg], above)) { spam(values, oparg); } """ - output = """ + output = """ TARGET(OP) { PyObject **values = (stack_pointer - oparg); PyObject *above; @@ -396,15 +447,15 @@ def test_array_input_output(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_array_error_if(): - input = """ + def test_array_error_if(self): + input = """ inst(OP, (extra, values[oparg] --)) { ERROR_IF(oparg == 0, somewhere); } """ - output = """ + output = """ TARGET(OP) { PyObject **values = (stack_pointer - oparg); PyObject *extra = stack_pointer[-(1 + oparg)]; @@ -414,15 +465,15 @@ def test_array_error_if(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_cond_effect(): - input = """ + def test_cond_effect(self): + input = """ inst(OP, (aa, input if ((oparg & 1) == 1), cc -- xx, output if (oparg & 2), zz)) { output = spam(oparg, input); } """ - output = """ + output = """ TARGET(OP) { PyObject *cc = stack_pointer[-1]; PyObject *input = ((oparg & 1) == 1) ? stack_pointer[-(1 + (((oparg & 1) == 1) ? 1 : 0))] : NULL; @@ -439,10 +490,10 @@ def test_cond_effect(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) -def test_macro_cond_effect(): - input = """ + def test_macro_cond_effect(self): + input = """ op(A, (left, middle, right --)) { # Body of A } @@ -451,7 +502,7 @@ def test_macro_cond_effect(): } macro(M) = A + B; """ - output = """ + output = """ TARGET(M) { PyObject *_tmp_1 = stack_pointer[-1]; PyObject *_tmp_2 = stack_pointer[-2]; @@ -479,4 +530,8 @@ def test_macro_cond_effect(): DISPATCH(); } """ - run_cases_test(input, output) + self.run_cases_test(input, output) + + +if __name__ == "__main__": + unittest.main() diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 3edd8ee51ba64..df5de6e299aaa 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -156,16 +156,9 @@ def __init__( self.emit_line_directives = emit_line_directives self.comment = comment self.lineno = 1 - filename = os.path.relpath(self.stream.name, ROOT) - # Make filename more user-friendly and less platform-specific - filename = filename.replace("\\", "/") - if filename.startswith("./"): - filename = filename[2:] - if filename.endswith(".new"): - filename = filename[:-4] - self.filename = filename + self.filename = prettify_filename(self.stream.name) self.nominal_lineno = 1 - self.nominal_filename = filename + self.nominal_filename = self.filename def write_raw(self, s: str) -> None: self.stream.write(s) @@ -737,12 +730,8 @@ def parse_file(self, filename: str, instrs_idx: dict[str, int]) -> None: with open(filename) as file: src = file.read() - filename = os.path.relpath(filename, ROOT) - # Make filename more user-friendly and less platform-specific - filename = filename.replace("\\", "/") - if filename.startswith("./"): - filename = filename[2:] - psr = parser.Parser(src, filename=filename) + + psr = parser.Parser(src, filename=prettify_filename(filename)) # Skip until begin marker while tkn := psr.next(raw=True): @@ -1149,7 +1138,7 @@ def write_function( def from_source_files(self) -> str: paths = f"\n{self.out.comment} ".join( - os.path.relpath(filename, ROOT).replace(os.path.sep, posixpath.sep) + prettify_filename(filename) for filename in self.input_filenames ) return f"{self.out.comment} from:\n{self.out.comment} {paths}\n" @@ -1597,6 +1586,17 @@ def wrap_macro(self, mac: MacroInstruction): self.out.emit(f"DISPATCH();") +def prettify_filename(filename: str) -> str: + # Make filename more user-friendly and less platform-specific, + # it is only used for error reporting at this point. + filename = filename.replace("\\", "/") + if filename.startswith("./"): + filename = filename[2:] + if filename.endswith(".new"): + filename = filename[:-4] + return filename + + def extract_block_text(block: parser.Block) -> tuple[list[str], bool, int]: # Get lines of text with proper dedent blocklines = block.text.splitlines(True) From webhook-mailer at python.org Sun Jul 16 20:04:13 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 17 Jul 2023 00:04:13 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Annotate Clinic.parse() (#106760) Message-ID: https://github.com/python/cpython/commit/383dcbebcda576e3a3fd18c9246364f67bb65df5 commit: 383dcbebcda576e3a3fd18c9246364f67bb65df5 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-17T00:04:10Z summary: gh-104050: Argument Clinic: Annotate Clinic.parse() (#106760) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 861a6507eae75..9b7069e9b8fcb 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -42,6 +42,7 @@ Literal, NamedTuple, NoReturn, + Protocol, TypeGuard, overload, ) @@ -2055,7 +2056,12 @@ def write_file(filename: str, new_contents: str) -> None: ClassDict = dict[str, "Class"] DestinationDict = dict[str, Destination] ModuleDict = dict[str, "Module"] -ParserDict = dict[str, "DSLParser"] + + +class Parser(Protocol): + def __init__(self, clinic: Clinic) -> None: ... + def parse(self, block: Block) -> None: ... + clinic = None class Clinic: @@ -2113,7 +2119,7 @@ def __init__( ) -> None: # maps strings to Parser objects. # (instantiated from the "parsers" global.) - self.parsers: ParserDict = {} + self.parsers: dict[str, Parser] = {} self.language: CLanguage = language if printer: fail("Custom printers are broken right now") @@ -2205,7 +2211,7 @@ def get_destination_buffer( d = self.get_destination(name) return d.buffers[item] - def parse(self, input): + def parse(self, input: str) -> str: printer = self.printer self.block_parser = BlockParser(input, self.language, verify=self.verify) for block in self.block_parser: @@ -5521,7 +5527,10 @@ def state_terminal(self, line): # "clinic", handles the Clinic DSL # "python", handles running Python code # -parsers = {'clinic' : DSLParser, 'python': PythonParser} +parsers: dict[str, Callable[[Clinic], Parser]] = { + 'clinic': DSLParser, + 'python': PythonParser, +} clinic = None From webhook-mailer at python.org Sun Jul 16 20:09:14 2023 From: webhook-mailer at python.org (corona10) Date: Mon, 17 Jul 2023 00:09:14 -0000 Subject: [Python-checkins] gh-106797: Remove warning logs from Python/generated_cases.c.h (gh-106798) Message-ID: https://github.com/python/cpython/commit/48956cc60ea05bc50b6cd73e53dd9a7d4b1dac9f commit: 48956cc60ea05bc50b6cd73e53dd9a7d4b1dac9f branch: main author: Dong-hee Na committer: corona10 date: 2023-07-17T09:09:11+09:00 summary: gh-106797: Remove warning logs from Python/generated_cases.c.h (gh-106798) files: M Include/internal/pycore_opcode_metadata.h M Python/generated_cases.c.h M Tools/cases_generator/generate_cases.py diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index 8373f56653b1c..3b2eab23e092f 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -839,15 +839,15 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { case PUSH_EXC_INFO: return 2; case LOAD_ATTR_METHOD_WITH_VALUES: - return (1 ? 1 : 0) + 1; + return 1 + 1; case LOAD_ATTR_METHOD_NO_DICT: - return (1 ? 1 : 0) + 1; + return 1 + 1; case LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES: - return (0 ? 1 : 0) + 1; + return 0 + 1; case LOAD_ATTR_NONDESCRIPTOR_NO_DICT: - return (0 ? 1 : 0) + 1; + return 0 + 1; case LOAD_ATTR_METHOD_LAZY_DICT: - return (1 ? 1 : 0) + 1; + return 1 + 1; case KW_NAMES: return 0; case INSTRUMENTED_CALL: diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 68531dc074769..392914c0521e9 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3360,9 +3360,9 @@ res2 = Py_NewRef(descr); assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; - STACK_GROW((1 ? 1 : 0)); + STACK_GROW(1); stack_pointer[-1] = res; - if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } + stack_pointer[-(1 + 1)] = res2; next_instr += 9; DISPATCH(); } @@ -3382,16 +3382,15 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - STACK_GROW((1 ? 1 : 0)); + STACK_GROW(1); stack_pointer[-1] = res; - if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } + stack_pointer[-(1 + 1)] = res2; next_instr += 9; DISPATCH(); } TARGET(LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES) { PyObject *self = stack_pointer[-1]; - PyObject *res2 = NULL; PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); @@ -3410,16 +3409,14 @@ assert(descr != NULL); Py_DECREF(self); res = Py_NewRef(descr); - STACK_GROW((0 ? 1 : 0)); + STACK_GROW(0); stack_pointer[-1] = res; - if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } next_instr += 9; DISPATCH(); } TARGET(LOAD_ATTR_NONDESCRIPTOR_NO_DICT) { PyObject *self = stack_pointer[-1]; - PyObject *res2 = NULL; PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); @@ -3432,9 +3429,8 @@ assert(descr != NULL); Py_DECREF(self); res = Py_NewRef(descr); - STACK_GROW((0 ? 1 : 0)); + STACK_GROW(0); stack_pointer[-1] = res; - if (0) { stack_pointer[-(1 + (0 ? 1 : 0))] = res2; } next_instr += 9; DISPATCH(); } @@ -3458,9 +3454,9 @@ assert(_PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR)); res2 = Py_NewRef(descr); res = self; - STACK_GROW((1 ? 1 : 0)); + STACK_GROW(1); stack_pointer[-1] = res; - if (1) { stack_pointer[-(1 + (1 ? 1 : 0))] = res2; } + stack_pointer[-(1 + 1)] = res2; next_instr += 9; DISPATCH(); } diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index df5de6e299aaa..a0a8b8cbe4bab 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -98,6 +98,8 @@ def effect_size(effect: StackEffect) -> tuple[int, str]: assert not effect.cond, "Array effects cannot have a condition" return 0, effect.size elif effect.cond: + if effect.cond in ("0", "1"): + return 0, effect.cond return 0, f"{maybe_parenthesize(effect.cond)} ? 1 : 0" else: return 1, "" @@ -217,7 +219,7 @@ def stack_adjust( self.emit(f"STACK_GROW({osym});") def declare(self, dst: StackEffect, src: StackEffect | None): - if dst.name == UNUSED: + if dst.name == UNUSED or dst.cond == "0": return typ = f"{dst.type}" if dst.type else "PyObject *" if src: @@ -241,7 +243,10 @@ def assign(self, dst: StackEffect, src: StackEffect): self.emit(f"Py_XSETREF({dst.name}, {cast}{src.name});") else: stmt = f"{dst.name} = {cast}{src.name};" - if src.cond: + if src.cond and src.cond != "1": + if src.cond == "0": + # It will not be executed + return stmt = f"if ({src.cond}) {{ {stmt} }}" self.emit(stmt) @@ -1067,7 +1072,10 @@ def effect_str(effects: list[StackEffect]) -> str: for effect in comp.instr.output_effects: assert not effect.size, effect if effect.cond: - pushed_symbolic.append(maybe_parenthesize(f"{maybe_parenthesize(effect.cond)} ? 1 : 0")) + if effect.cond in ("0", "1"): + pushed_symbolic.append(effect.cond) + else: + pushed_symbolic.append(maybe_parenthesize(f"{maybe_parenthesize(effect.cond)} ? 1 : 0")) sp += 1 high = max(sp, high) if high != max(0, sp): From webhook-mailer at python.org Sun Jul 16 22:37:10 2023 From: webhook-mailer at python.org (rhettinger) Date: Mon, 17 Jul 2023 02:37:10 -0000 Subject: [Python-checkins] Add more recipe tests. Make the factor recipe a bit faster and clearer. (GH-106817) Message-ID: https://github.com/python/cpython/commit/babb22da5a25c18a2d203bf72ba35e7861ca60ee commit: babb22da5a25c18a2d203bf72ba35e7861ca60ee branch: main author: Raymond Hettinger committer: rhettinger date: 2023-07-16T21:37:07-05:00 summary: Add more recipe tests. Make the factor recipe a bit faster and clearer. (GH-106817) files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index f88525456ff93..730736bbb59ed 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -1049,11 +1049,10 @@ The following recipes have a more mathematical flavor: # factor(1_000_000_000_000_403) --> 1000000000000403 for prime in sieve(math.isqrt(n) + 1): while True: - quotient, remainder = divmod(n, prime) - if remainder: + if n % prime: break yield prime - n = quotient + n //= prime if n == 1: return if n > 1: @@ -1354,6 +1353,12 @@ The following recipes have a more mathematical flavor: >>> set(sieve(10_000)).isdisjoint(carmichael) True + >>> list(factor(99)) # Code example 1 + [3, 3, 11] + >>> list(factor(1_000_000_000_000_007)) # Code example 2 + [47, 59, 360620266859] + >>> list(factor(1_000_000_000_000_403)) # Code example 3 + [1000000000000403] >>> list(factor(0)) [] >>> list(factor(1)) From webhook-mailer at python.org Sun Jul 16 22:48:01 2023 From: webhook-mailer at python.org (rhettinger) Date: Mon, 17 Jul 2023 02:48:01 -0000 Subject: [Python-checkins] [3.12] Add more recipe tests. Make the factor recipe a bit faster and clearer. (GH-106817) (GH-106818) Message-ID: https://github.com/python/cpython/commit/9c00dc02cf87d6edecc47586a9aac1fde1517868 commit: 9c00dc02cf87d6edecc47586a9aac1fde1517868 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: rhettinger date: 2023-07-16T21:47:58-05:00 summary: [3.12] Add more recipe tests. Make the factor recipe a bit faster and clearer. (GH-106817) (GH-106818) files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index f88525456ff93..730736bbb59ed 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -1049,11 +1049,10 @@ The following recipes have a more mathematical flavor: # factor(1_000_000_000_000_403) --> 1000000000000403 for prime in sieve(math.isqrt(n) + 1): while True: - quotient, remainder = divmod(n, prime) - if remainder: + if n % prime: break yield prime - n = quotient + n //= prime if n == 1: return if n > 1: @@ -1354,6 +1353,12 @@ The following recipes have a more mathematical flavor: >>> set(sieve(10_000)).isdisjoint(carmichael) True + >>> list(factor(99)) # Code example 1 + [3, 3, 11] + >>> list(factor(1_000_000_000_000_007)) # Code example 2 + [47, 59, 360620266859] + >>> list(factor(1_000_000_000_000_403)) # Code example 3 + [1000000000000403] >>> list(factor(0)) [] >>> list(factor(1)) From webhook-mailer at python.org Sun Jul 16 23:36:06 2023 From: webhook-mailer at python.org (terryjreedy) Date: Mon, 17 Jul 2023 03:36:06 -0000 Subject: [Python-checkins] gh-106780: Add __match_args__ to tutorial example (#106784) Message-ID: https://github.com/python/cpython/commit/7aa89e505d893cd5e6f33b84d66e5fa769089931 commit: 7aa89e505d893cd5e6f33b84d66e5fa769089931 branch: main author: Terry Jan Reedy committer: terryjreedy date: 2023-07-16T23:36:03-04:00 summary: gh-106780: Add __match_args__ to tutorial example (#106784) Add Point definition with this attribute before example that needs it. files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index 4336bf50df40a..e140f51f1dda7 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -343,7 +343,13 @@ Dotted names (like ``foo.bar``), attribute names (the ``x=`` and ``y=`` above) o (recognized by the "(...)" next to them like ``Point`` above) are never assigned to. Patterns can be arbitrarily nested. For example, if we have a short -list of points, we could match it like this:: +list of Points, with ``__match_args__`` added, we could match it like this:: + + class Point: + __match_args__ = ('x', 'y') + def __init__(self, x, y): + self.x = x + self.y = y match points: case []: From webhook-mailer at python.org Mon Jul 17 04:15:39 2023 From: webhook-mailer at python.org (terryjreedy) Date: Mon, 17 Jul 2023 08:15:39 -0000 Subject: [Python-checkins] [3.12] gh-106780: Add __match_args__ to tutorial example (GH-106784) (#106819) Message-ID: https://github.com/python/cpython/commit/11b3d38310e98d1fc079938c0ec1b3992a0c7c03 commit: 11b3d38310e98d1fc079938c0ec1b3992a0c7c03 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-17T04:15:35-04:00 summary: [3.12] gh-106780: Add __match_args__ to tutorial example (GH-106784) (#106819) Add Point definition with this attribute before example that needs it. (cherry picked from commit 7aa89e505d893cd5e6f33b84d66e5fa769089931) Co-authored-by: Terry Jan Reedy files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index 4336bf50df40a..e140f51f1dda7 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -343,7 +343,13 @@ Dotted names (like ``foo.bar``), attribute names (the ``x=`` and ``y=`` above) o (recognized by the "(...)" next to them like ``Point`` above) are never assigned to. Patterns can be arbitrarily nested. For example, if we have a short -list of points, we could match it like this:: +list of Points, with ``__match_args__`` added, we could match it like this:: + + class Point: + __match_args__ = ('x', 'y') + def __init__(self, x, y): + self.x = x + self.y = y match points: case []: From webhook-mailer at python.org Mon Jul 17 04:15:56 2023 From: webhook-mailer at python.org (terryjreedy) Date: Mon, 17 Jul 2023 08:15:56 -0000 Subject: [Python-checkins] [3.11] gh-106780: Add __match_args__ to tutorial example (GH-106784) (#106820) Message-ID: https://github.com/python/cpython/commit/6193f783d02190a66f287b3a36c187a9323dcc2b commit: 6193f783d02190a66f287b3a36c187a9323dcc2b branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2023-07-17T04:15:52-04:00 summary: [3.11] gh-106780: Add __match_args__ to tutorial example (GH-106784) (#106820) Add Point definition with this attribute before example that needs it. (cherry picked from commit 7aa89e505d893cd5e6f33b84d66e5fa769089931) Co-authored-by: Terry Jan Reedy files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index 4336bf50df40a..e140f51f1dda7 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -343,7 +343,13 @@ Dotted names (like ``foo.bar``), attribute names (the ``x=`` and ``y=`` above) o (recognized by the "(...)" next to them like ``Point`` above) are never assigned to. Patterns can be arbitrarily nested. For example, if we have a short -list of points, we could match it like this:: +list of Points, with ``__match_args__`` added, we could match it like this:: + + class Point: + __match_args__ = ('x', 'y') + def __init__(self, x, y): + self.x = x + self.y = y match points: case []: From webhook-mailer at python.org Mon Jul 17 05:28:37 2023 From: webhook-mailer at python.org (iritkatriel) Date: Mon, 17 Jul 2023 09:28:37 -0000 Subject: [Python-checkins] gh-106789: avoid importing pprint from sysconfig (#106790) Message-ID: https://github.com/python/cpython/commit/5ecedbd26692b9fbdd7aad81b991869bf650f929 commit: 5ecedbd26692b9fbdd7aad81b991869bf650f929 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-17T10:28:33+01:00 summary: gh-106789: avoid importing pprint from sysconfig (#106790) files: A Misc/NEWS.d/next/Library/2023-07-16-10-40-34.gh-issue-106789.NvyE3C.rst M Lib/opcode.py M Lib/sysconfig.py diff --git a/Lib/opcode.py b/Lib/opcode.py index bc885051c6454..1b36300785aae 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -6,26 +6,14 @@ __all__ = ["cmp_op", "hasarg", "hasconst", "hasname", "hasjrel", "hasjabs", "haslocal", "hascompare", "hasfree", "hasexc", "opname", "opmap", - "HAVE_ARGUMENT", "EXTENDED_ARG"] - -# It's a chicken-and-egg I'm afraid: -# We're imported before _opcode's made. -# With exception unheeded -# (stack_effect is not needed) -# Both our chickens and eggs are allayed. -# --Larry Hastings, 2013/11/23 - -try: - from _opcode import stack_effect - __all__.append('stack_effect') -except ImportError: - pass - -# _opcode_metadata may not be ready during early stages of the build -try: + "stack_effect", "HAVE_ARGUMENT", "EXTENDED_ARG"] + +from _opcode import stack_effect + +import sys +# The build uses older versions of Python which do not have _opcode_metadata +if sys.version_info[:2] >= (3, 13): from _opcode_metadata import _specializations, _specialized_instructions -except ModuleNotFoundError: - pass cmp_op = ('<', '<=', '==', '!=', '>', '>=') diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py index 122d441bd19f5..a8b5c5f7dfba5 100644 --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -465,10 +465,14 @@ def _get_sysconfigdata_name(): f'_sysconfigdata_{sys.abiflags}_{sys.platform}_{multiarch}', ) +def _print_config_dict(d, stream): + print ("{", file=stream) + for k, v in sorted(d.items()): + print(f" {k!r}: {v!r},", file=stream) + print ("}", file=stream) def _generate_posix_vars(): """Generate the Python module containing build-time variables.""" - import pprint vars = {} # load the installed Makefile: makefile = get_makefile_filename() @@ -523,7 +527,7 @@ def _generate_posix_vars(): f.write('# system configuration generated and used by' ' the sysconfig module\n') f.write('build_time_vars = ') - pprint.pprint(vars, stream=f) + _print_config_dict(vars, stream=f) # Create file used for sys.path fixup -- see Modules/getpath.c with open('pybuilddir.txt', 'w', encoding='utf8') as f: diff --git a/Misc/NEWS.d/next/Library/2023-07-16-10-40-34.gh-issue-106789.NvyE3C.rst b/Misc/NEWS.d/next/Library/2023-07-16-10-40-34.gh-issue-106789.NvyE3C.rst new file mode 100644 index 0000000000000..532f8059740da --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-16-10-40-34.gh-issue-106789.NvyE3C.rst @@ -0,0 +1 @@ +Remove import of :mod:``pprint`` from :mod:``sysconfig``. From webhook-mailer at python.org Mon Jul 17 07:47:12 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 17 Jul 2023 11:47:12 -0000 Subject: [Python-checkins] gh-104050: Improve Argument Clinic type annotation coverage (#106810) Message-ID: https://github.com/python/cpython/commit/036bb7365607ab7e5cf901f1ac4256f9ae1be82c commit: 036bb7365607ab7e5cf901f1ac4256f9ae1be82c branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-17T13:47:08+02:00 summary: gh-104050: Improve Argument Clinic type annotation coverage (#106810) Add various missing annotations in the following classes: - BlockPrinter - CConverter - CLanguage - FormatCounterFormatter - Language - _TextAccumulator Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 9b7069e9b8fcb..311f0a1a56a03 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -109,7 +109,7 @@ class _TextAccumulator(NamedTuple): def _text_accumulator() -> _TextAccumulator: text: list[str] = [] - def output(): + def output() -> str: s = ''.join(text) text.clear() return s @@ -433,10 +433,10 @@ class FormatCounterFormatter(string.Formatter): the counts dict would now look like {'a': 2, 'b': 1, 'c': 1} """ - def __init__(self): - self.counts = collections.Counter() + def __init__(self) -> None: + self.counts = collections.Counter[str]() - def get_value(self, key, args, kwargs): + def get_value(self, key: str, args, kwargs) -> str: # type: ignore[override] self.counts[key] += 1 return '' @@ -447,18 +447,25 @@ class Language(metaclass=abc.ABCMeta): stop_line = "" checksum_line = "" - def __init__(self, filename): + def __init__(self, filename: str) -> None: pass @abc.abstractmethod - def render(self, clinic, signatures): + def render( + self, + clinic: Clinic | None, + signatures: Iterable[Module | Class | Function] + ) -> str: pass - def parse_line(self, line): + def parse_line(self, line: str) -> None: pass - def validate(self): - def assert_only_one(attr, *additional_fields): + def validate(self) -> None: + def assert_only_one( + attr: str, + *additional_fields: str + ) -> None: """ Ensures that the string found at getattr(self, attr) contains exactly one formatter replacement string for @@ -485,10 +492,10 @@ def assert_only_one(attr, *additional_fields): """ fields = ['dsl_name'] fields.extend(additional_fields) - line = getattr(self, attr) + line: str = getattr(self, attr) fcf = FormatCounterFormatter() fcf.format(line) - def local_fail(should_be_there_but_isnt): + def local_fail(should_be_there_but_isnt: bool) -> None: if should_be_there_but_isnt: fail("{} {} must contain {{{}}} exactly once!".format( self.__class__.__name__, attr, name)) @@ -749,10 +756,10 @@ class CLanguage(Language): stop_line = "[{dsl_name} start generated code]*/" checksum_line = "/*[{dsl_name} end generated code: {arguments}]*/" - def __init__(self, filename): + def __init__(self, filename: str) -> None: super().__init__(filename) self.cpp = cpp.Monitor(filename) - self.cpp.fail = fail + self.cpp.fail = fail # type: ignore[method-assign] def parse_line(self, line: str) -> None: self.cpp.writeline(line) @@ -935,6 +942,7 @@ def parser_body( add(field) return linear_format(output(), parser_declarations=declarations) + parsearg: str | None if not parameters: parser_code: list[str] | None if not requires_defining_class: @@ -1880,7 +1888,12 @@ class BlockPrinter: language: Language f: io.StringIO = dc.field(default_factory=io.StringIO) - def print_block(self, block, *, core_includes=False): + def print_block( + self, + block: Block, + *, + core_includes: bool = False + ) -> None: input = block.input output = block.output dsl_name = block.dsl_name @@ -1931,7 +1944,7 @@ def print_block(self, block, *, core_includes=False): write(self.language.checksum_line.format(dsl_name=dsl_name, arguments=arguments)) write("\n") - def write(self, text): + def write(self, text: str) -> None: self.f.write(text) @@ -2755,7 +2768,7 @@ class CConverter(metaclass=CConverterAutoRegister): # If not None, should be a string representing a pointer to a # PyTypeObject (e.g. "&PyUnicode_Type"). # Only used by the 'O!' format unit (and the "object" converter). - subclass_of = None + subclass_of: str | None = None # Do we want an adjacent '_length' variable for this variable? # Only used by format units ending with '#'. @@ -2948,7 +2961,7 @@ def simple_declaration(self, by_reference=False, *, in_parser=False): prototype.append(name) return "".join(prototype) - def declaration(self, *, in_parser=False): + def declaration(self, *, in_parser=False) -> str: """ The C statement to declare this variable. """ @@ -3006,7 +3019,7 @@ def pre_render(self): """ pass - def parse_arg(self, argname, displayname): + def parse_arg(self, argname: str, displayname: str): if self.format_unit == 'O&': return """ if (!{converter}({argname}, &{paramname})) {{{{ From webhook-mailer at python.org Mon Jul 17 11:55:34 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 17 Jul 2023 15:55:34 -0000 Subject: [Python-checkins] gh-106687: _ssl: use uint64_t for SSL options (#106700) Message-ID: https://github.com/python/cpython/commit/ad95c7253a70e559e7d3f25d53f4772f28bb8b44 commit: ad95c7253a70e559e7d3f25d53f4772f28bb8b44 branch: main author: Victor Stinner committer: vstinner date: 2023-07-17T17:55:30+02:00 summary: gh-106687: _ssl: use uint64_t for SSL options (#106700) SSL_CTX_get_options() uses uint64_t for options: https://www.openssl.org/docs/man3.1/man3/SSL_CTX_get_options.html Fix this compiler warning on Windows with MSC: conversion from 'uint64_t' to 'long', possible loss of data files: M Lib/test/test_ssl.py M Modules/_ssl.c diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index d46ce5e60e214..6117ca3fdba1b 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -339,6 +339,15 @@ def test_constants(self): ssl.OP_NO_TLSv1_2 self.assertEqual(ssl.PROTOCOL_TLS, ssl.PROTOCOL_SSLv23) + def test_options(self): + # gh-106687: SSL options values are unsigned integer (uint64_t) + for name in dir(ssl): + if not name.startswith('OP_'): + continue + with self.subTest(option=name): + value = getattr(ssl, name) + self.assertGreaterEqual(value, 0, f"ssl.{name}") + def test_ssl_types(self): ssl_types = [ _ssl._SSLContext, @@ -951,6 +960,7 @@ def test_get_ciphers(self): ) def test_options(self): + # Test default SSLContext options ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) # OP_ALL | OP_NO_SSLv2 | OP_NO_SSLv3 is the default value default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3) @@ -959,16 +969,30 @@ def test_options(self): OP_SINGLE_DH_USE | OP_SINGLE_ECDH_USE | OP_ENABLE_MIDDLEBOX_COMPAT) self.assertEqual(default, ctx.options) + + # disallow TLSv1 with warnings_helper.check_warnings(): ctx.options |= ssl.OP_NO_TLSv1 self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options) + + # allow TLSv1 with warnings_helper.check_warnings(): ctx.options = (ctx.options & ~ssl.OP_NO_TLSv1) self.assertEqual(default, ctx.options) + + # clear all options ctx.options = 0 # Ubuntu has OP_NO_SSLv3 forced on by default self.assertEqual(0, ctx.options & ~ssl.OP_NO_SSLv3) + # invalid options + with self.assertRaises(OverflowError): + ctx.options = -1 + with self.assertRaises(OverflowError): + ctx.options = 2 ** 100 + with self.assertRaises(TypeError): + ctx.options = "abc" + def test_verify_mode_protocol(self): with warnings_helper.check_warnings(): ctx = ssl.SSLContext(ssl.PROTOCOL_TLS) diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 571de331e92cd..0cf4d3e9dc8c9 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -3025,7 +3025,7 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) /*[clinic end generated code: output=2cf0d7a0741b6bd1 input=8d58a805b95fc534]*/ { PySSLContext *self; - long options; + uint64_t options; const SSL_METHOD *method = NULL; SSL_CTX *ctx = NULL; X509_VERIFY_PARAM *params; @@ -3618,20 +3618,32 @@ PyDoc_STRVAR(PySSLContext_security_level_doc, "The current security level"); static PyObject * get_options(PySSLContext *self, void *c) { - return PyLong_FromLong(SSL_CTX_get_options(self->ctx)); + uint64_t options = SSL_CTX_get_options(self->ctx); + Py_BUILD_ASSERT(sizeof(unsigned long long) >= sizeof(options)); + return PyLong_FromUnsignedLongLong(options); } static int set_options(PySSLContext *self, PyObject *arg, void *c) { - long new_opts, opts, set, clear; - long opt_no = ( + PyObject *new_opts_obj; + unsigned long long new_opts_arg; + uint64_t new_opts, opts, clear, set; + uint64_t opt_no = ( SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_TLSv1 | SSL_OP_NO_TLSv1_1 | SSL_OP_NO_TLSv1_2 | SSL_OP_NO_TLSv1_3 ); - if (!PyArg_Parse(arg, "l", &new_opts)) + if (!PyArg_Parse(arg, "O!", &PyLong_Type, &new_opts_obj)) { return -1; + } + new_opts_arg = PyLong_AsUnsignedLongLong(new_opts_obj); + if (new_opts_arg == (unsigned long long)-1 && PyErr_Occurred()) { + return -1; + } + Py_BUILD_ASSERT(sizeof(new_opts) >= sizeof(new_opts_arg)); + new_opts = (uint64_t)new_opts_arg; + opts = SSL_CTX_get_options(self->ctx); clear = opts & ~new_opts; set = ~opts & new_opts; @@ -3645,8 +3657,9 @@ set_options(PySSLContext *self, PyObject *arg, void *c) if (clear) { SSL_CTX_clear_options(self->ctx, clear); } - if (set) + if (set) { SSL_CTX_set_options(self->ctx, set); + } return 0; } @@ -5754,10 +5767,24 @@ sslmodule_init_socketapi(PyObject *module) return 0; } + static int -sslmodule_init_constants(PyObject *m) +sslmodule_add_option(PyObject *m, const char *name, uint64_t value) { + Py_BUILD_ASSERT(sizeof(unsigned long long) >= sizeof(value)); + PyObject *obj = PyLong_FromUnsignedLongLong(value); + if (obj == NULL) { + return -1; + } + int res = PyModule_AddObjectRef(m, name, obj); + Py_DECREF(obj); + return res; +} + +static int +sslmodule_init_constants(PyObject *m) +{ PyModule_AddStringConstant(m, "_DEFAULT_CIPHERS", PY_SSL_DEFAULT_CIPHER_STRING); @@ -5877,46 +5904,47 @@ sslmodule_init_constants(PyObject *m) PyModule_AddIntConstant(m, "PROTOCOL_TLSv1_2", PY_SSL_VERSION_TLS1_2); +#define ADD_OPTION(NAME, VALUE) if (sslmodule_add_option(m, NAME, (VALUE)) < 0) return -1 + /* protocol options */ - PyModule_AddIntConstant(m, "OP_ALL", - SSL_OP_ALL & ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS); - PyModule_AddIntConstant(m, "OP_NO_SSLv2", SSL_OP_NO_SSLv2); - PyModule_AddIntConstant(m, "OP_NO_SSLv3", SSL_OP_NO_SSLv3); - PyModule_AddIntConstant(m, "OP_NO_TLSv1", SSL_OP_NO_TLSv1); - PyModule_AddIntConstant(m, "OP_NO_TLSv1_1", SSL_OP_NO_TLSv1_1); - PyModule_AddIntConstant(m, "OP_NO_TLSv1_2", SSL_OP_NO_TLSv1_2); + ADD_OPTION("OP_ALL", SSL_OP_ALL & ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS); + ADD_OPTION("OP_NO_SSLv2", SSL_OP_NO_SSLv2); + ADD_OPTION("OP_NO_SSLv3", SSL_OP_NO_SSLv3); + ADD_OPTION("OP_NO_TLSv1", SSL_OP_NO_TLSv1); + ADD_OPTION("OP_NO_TLSv1_1", SSL_OP_NO_TLSv1_1); + ADD_OPTION("OP_NO_TLSv1_2", SSL_OP_NO_TLSv1_2); #ifdef SSL_OP_NO_TLSv1_3 - PyModule_AddIntConstant(m, "OP_NO_TLSv1_3", SSL_OP_NO_TLSv1_3); + ADD_OPTION("OP_NO_TLSv1_3", SSL_OP_NO_TLSv1_3); #else - PyModule_AddIntConstant(m, "OP_NO_TLSv1_3", 0); + ADD_OPTION("OP_NO_TLSv1_3", 0); #endif - PyModule_AddIntConstant(m, "OP_CIPHER_SERVER_PREFERENCE", + ADD_OPTION("OP_CIPHER_SERVER_PREFERENCE", SSL_OP_CIPHER_SERVER_PREFERENCE); - PyModule_AddIntConstant(m, "OP_SINGLE_DH_USE", SSL_OP_SINGLE_DH_USE); - PyModule_AddIntConstant(m, "OP_NO_TICKET", SSL_OP_NO_TICKET); - PyModule_AddIntConstant(m, "OP_LEGACY_SERVER_CONNECT", + ADD_OPTION("OP_SINGLE_DH_USE", SSL_OP_SINGLE_DH_USE); + ADD_OPTION("OP_NO_TICKET", SSL_OP_NO_TICKET); + ADD_OPTION("OP_LEGACY_SERVER_CONNECT", SSL_OP_LEGACY_SERVER_CONNECT); #ifdef SSL_OP_SINGLE_ECDH_USE - PyModule_AddIntConstant(m, "OP_SINGLE_ECDH_USE", SSL_OP_SINGLE_ECDH_USE); + ADD_OPTION("OP_SINGLE_ECDH_USE", SSL_OP_SINGLE_ECDH_USE); #endif #ifdef SSL_OP_NO_COMPRESSION - PyModule_AddIntConstant(m, "OP_NO_COMPRESSION", + ADD_OPTION("OP_NO_COMPRESSION", SSL_OP_NO_COMPRESSION); #endif #ifdef SSL_OP_ENABLE_MIDDLEBOX_COMPAT - PyModule_AddIntConstant(m, "OP_ENABLE_MIDDLEBOX_COMPAT", + ADD_OPTION("OP_ENABLE_MIDDLEBOX_COMPAT", SSL_OP_ENABLE_MIDDLEBOX_COMPAT); #endif #ifdef SSL_OP_NO_RENEGOTIATION - PyModule_AddIntConstant(m, "OP_NO_RENEGOTIATION", + ADD_OPTION("OP_NO_RENEGOTIATION", SSL_OP_NO_RENEGOTIATION); #endif #ifdef SSL_OP_IGNORE_UNEXPECTED_EOF - PyModule_AddIntConstant(m, "OP_IGNORE_UNEXPECTED_EOF", + ADD_OPTION("OP_IGNORE_UNEXPECTED_EOF", SSL_OP_IGNORE_UNEXPECTED_EOF); #endif #ifdef SSL_OP_ENABLE_KTLS - PyModule_AddIntConstant(m, "OP_ENABLE_KTLS", SSL_OP_ENABLE_KTLS); + ADD_OPTION("OP_ENABLE_KTLS", SSL_OP_ENABLE_KTLS); #endif #ifdef X509_CHECK_FLAG_ALWAYS_CHECK_SUBJECT From webhook-mailer at python.org Mon Jul 17 12:32:22 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 17 Jul 2023 16:32:22 -0000 Subject: [Python-checkins] [3.12] gh-106687: _ssl: use uint64_t for SSL options (GH-106700) (#106827) Message-ID: https://github.com/python/cpython/commit/497bfd5047d2088718d0b2d8f14fc8022abec502 commit: 497bfd5047d2088718d0b2d8f14fc8022abec502 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-17T16:32:17Z summary: [3.12] gh-106687: _ssl: use uint64_t for SSL options (GH-106700) (#106827) gh-106687: _ssl: use uint64_t for SSL options (GH-106700) SSL_CTX_get_options() uses uint64_t for options: https://www.openssl.org/docs/man3.1/man3/SSL_CTX_get_options.html Fix this compiler warning on Windows with MSC: conversion from 'uint64_t' to 'long', possible loss of data (cherry picked from commit ad95c7253a70e559e7d3f25d53f4772f28bb8b44) Co-authored-by: Victor Stinner files: M Lib/test/test_ssl.py M Modules/_ssl.c diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index d46ce5e60e214..6117ca3fdba1b 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -339,6 +339,15 @@ def test_constants(self): ssl.OP_NO_TLSv1_2 self.assertEqual(ssl.PROTOCOL_TLS, ssl.PROTOCOL_SSLv23) + def test_options(self): + # gh-106687: SSL options values are unsigned integer (uint64_t) + for name in dir(ssl): + if not name.startswith('OP_'): + continue + with self.subTest(option=name): + value = getattr(ssl, name) + self.assertGreaterEqual(value, 0, f"ssl.{name}") + def test_ssl_types(self): ssl_types = [ _ssl._SSLContext, @@ -951,6 +960,7 @@ def test_get_ciphers(self): ) def test_options(self): + # Test default SSLContext options ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) # OP_ALL | OP_NO_SSLv2 | OP_NO_SSLv3 is the default value default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3) @@ -959,16 +969,30 @@ def test_options(self): OP_SINGLE_DH_USE | OP_SINGLE_ECDH_USE | OP_ENABLE_MIDDLEBOX_COMPAT) self.assertEqual(default, ctx.options) + + # disallow TLSv1 with warnings_helper.check_warnings(): ctx.options |= ssl.OP_NO_TLSv1 self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options) + + # allow TLSv1 with warnings_helper.check_warnings(): ctx.options = (ctx.options & ~ssl.OP_NO_TLSv1) self.assertEqual(default, ctx.options) + + # clear all options ctx.options = 0 # Ubuntu has OP_NO_SSLv3 forced on by default self.assertEqual(0, ctx.options & ~ssl.OP_NO_SSLv3) + # invalid options + with self.assertRaises(OverflowError): + ctx.options = -1 + with self.assertRaises(OverflowError): + ctx.options = 2 ** 100 + with self.assertRaises(TypeError): + ctx.options = "abc" + def test_verify_mode_protocol(self): with warnings_helper.check_warnings(): ctx = ssl.SSLContext(ssl.PROTOCOL_TLS) diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 7a13821f9d7b5..5f90584a99c08 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2996,7 +2996,7 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) /*[clinic end generated code: output=2cf0d7a0741b6bd1 input=8d58a805b95fc534]*/ { PySSLContext *self; - long options; + uint64_t options; const SSL_METHOD *method = NULL; SSL_CTX *ctx = NULL; X509_VERIFY_PARAM *params; @@ -3594,20 +3594,32 @@ PyDoc_STRVAR(PySSLContext_security_level_doc, "The current security level"); static PyObject * get_options(PySSLContext *self, void *c) { - return PyLong_FromLong(SSL_CTX_get_options(self->ctx)); + uint64_t options = SSL_CTX_get_options(self->ctx); + Py_BUILD_ASSERT(sizeof(unsigned long long) >= sizeof(options)); + return PyLong_FromUnsignedLongLong(options); } static int set_options(PySSLContext *self, PyObject *arg, void *c) { - long new_opts, opts, set, clear; - long opt_no = ( + PyObject *new_opts_obj; + unsigned long long new_opts_arg; + uint64_t new_opts, opts, clear, set; + uint64_t opt_no = ( SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_TLSv1 | SSL_OP_NO_TLSv1_1 | SSL_OP_NO_TLSv1_2 | SSL_OP_NO_TLSv1_3 ); - if (!PyArg_Parse(arg, "l", &new_opts)) + if (!PyArg_Parse(arg, "O!", &PyLong_Type, &new_opts_obj)) { return -1; + } + new_opts_arg = PyLong_AsUnsignedLongLong(new_opts_obj); + if (new_opts_arg == (unsigned long long)-1 && PyErr_Occurred()) { + return -1; + } + Py_BUILD_ASSERT(sizeof(new_opts) >= sizeof(new_opts_arg)); + new_opts = (uint64_t)new_opts_arg; + opts = SSL_CTX_get_options(self->ctx); clear = opts & ~new_opts; set = ~opts & new_opts; @@ -3621,8 +3633,9 @@ set_options(PySSLContext *self, PyObject *arg, void *c) if (clear) { SSL_CTX_clear_options(self->ctx, clear); } - if (set) + if (set) { SSL_CTX_set_options(self->ctx, set); + } return 0; } @@ -5731,10 +5744,24 @@ sslmodule_init_socketapi(PyObject *module) return 0; } + static int -sslmodule_init_constants(PyObject *m) +sslmodule_add_option(PyObject *m, const char *name, uint64_t value) { + Py_BUILD_ASSERT(sizeof(unsigned long long) >= sizeof(value)); + PyObject *obj = PyLong_FromUnsignedLongLong(value); + if (obj == NULL) { + return -1; + } + int res = PyModule_AddObjectRef(m, name, obj); + Py_DECREF(obj); + return res; +} + +static int +sslmodule_init_constants(PyObject *m) +{ PyModule_AddStringConstant(m, "_DEFAULT_CIPHERS", PY_SSL_DEFAULT_CIPHER_STRING); @@ -5854,46 +5881,47 @@ sslmodule_init_constants(PyObject *m) PyModule_AddIntConstant(m, "PROTOCOL_TLSv1_2", PY_SSL_VERSION_TLS1_2); +#define ADD_OPTION(NAME, VALUE) if (sslmodule_add_option(m, NAME, (VALUE)) < 0) return -1 + /* protocol options */ - PyModule_AddIntConstant(m, "OP_ALL", - SSL_OP_ALL & ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS); - PyModule_AddIntConstant(m, "OP_NO_SSLv2", SSL_OP_NO_SSLv2); - PyModule_AddIntConstant(m, "OP_NO_SSLv3", SSL_OP_NO_SSLv3); - PyModule_AddIntConstant(m, "OP_NO_TLSv1", SSL_OP_NO_TLSv1); - PyModule_AddIntConstant(m, "OP_NO_TLSv1_1", SSL_OP_NO_TLSv1_1); - PyModule_AddIntConstant(m, "OP_NO_TLSv1_2", SSL_OP_NO_TLSv1_2); + ADD_OPTION("OP_ALL", SSL_OP_ALL & ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS); + ADD_OPTION("OP_NO_SSLv2", SSL_OP_NO_SSLv2); + ADD_OPTION("OP_NO_SSLv3", SSL_OP_NO_SSLv3); + ADD_OPTION("OP_NO_TLSv1", SSL_OP_NO_TLSv1); + ADD_OPTION("OP_NO_TLSv1_1", SSL_OP_NO_TLSv1_1); + ADD_OPTION("OP_NO_TLSv1_2", SSL_OP_NO_TLSv1_2); #ifdef SSL_OP_NO_TLSv1_3 - PyModule_AddIntConstant(m, "OP_NO_TLSv1_3", SSL_OP_NO_TLSv1_3); + ADD_OPTION("OP_NO_TLSv1_3", SSL_OP_NO_TLSv1_3); #else - PyModule_AddIntConstant(m, "OP_NO_TLSv1_3", 0); + ADD_OPTION("OP_NO_TLSv1_3", 0); #endif - PyModule_AddIntConstant(m, "OP_CIPHER_SERVER_PREFERENCE", + ADD_OPTION("OP_CIPHER_SERVER_PREFERENCE", SSL_OP_CIPHER_SERVER_PREFERENCE); - PyModule_AddIntConstant(m, "OP_SINGLE_DH_USE", SSL_OP_SINGLE_DH_USE); - PyModule_AddIntConstant(m, "OP_NO_TICKET", SSL_OP_NO_TICKET); - PyModule_AddIntConstant(m, "OP_LEGACY_SERVER_CONNECT", + ADD_OPTION("OP_SINGLE_DH_USE", SSL_OP_SINGLE_DH_USE); + ADD_OPTION("OP_NO_TICKET", SSL_OP_NO_TICKET); + ADD_OPTION("OP_LEGACY_SERVER_CONNECT", SSL_OP_LEGACY_SERVER_CONNECT); #ifdef SSL_OP_SINGLE_ECDH_USE - PyModule_AddIntConstant(m, "OP_SINGLE_ECDH_USE", SSL_OP_SINGLE_ECDH_USE); + ADD_OPTION("OP_SINGLE_ECDH_USE", SSL_OP_SINGLE_ECDH_USE); #endif #ifdef SSL_OP_NO_COMPRESSION - PyModule_AddIntConstant(m, "OP_NO_COMPRESSION", + ADD_OPTION("OP_NO_COMPRESSION", SSL_OP_NO_COMPRESSION); #endif #ifdef SSL_OP_ENABLE_MIDDLEBOX_COMPAT - PyModule_AddIntConstant(m, "OP_ENABLE_MIDDLEBOX_COMPAT", + ADD_OPTION("OP_ENABLE_MIDDLEBOX_COMPAT", SSL_OP_ENABLE_MIDDLEBOX_COMPAT); #endif #ifdef SSL_OP_NO_RENEGOTIATION - PyModule_AddIntConstant(m, "OP_NO_RENEGOTIATION", + ADD_OPTION("OP_NO_RENEGOTIATION", SSL_OP_NO_RENEGOTIATION); #endif #ifdef SSL_OP_IGNORE_UNEXPECTED_EOF - PyModule_AddIntConstant(m, "OP_IGNORE_UNEXPECTED_EOF", + ADD_OPTION("OP_IGNORE_UNEXPECTED_EOF", SSL_OP_IGNORE_UNEXPECTED_EOF); #endif #ifdef SSL_OP_ENABLE_KTLS - PyModule_AddIntConstant(m, "OP_ENABLE_KTLS", SSL_OP_ENABLE_KTLS); + ADD_OPTION("OP_ENABLE_KTLS", SSL_OP_ENABLE_KTLS); #endif #ifdef X509_CHECK_FLAG_ALWAYS_CHECK_SUBJECT From webhook-mailer at python.org Mon Jul 17 13:06:09 2023 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 17 Jul 2023 17:06:09 -0000 Subject: [Python-checkins] gh-106529: Generate uops for POP_JUMP_IF_[NOT_]NONE (#106796) Message-ID: https://github.com/python/cpython/commit/b2b261ab2a2d4ff000c6248dbc52247c78cfa5ab commit: b2b261ab2a2d4ff000c6248dbc52247c78cfa5ab branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-17T10:06:05-07:00 summary: gh-106529: Generate uops for POP_JUMP_IF_[NOT_]NONE (#106796) These aren't automatically translated because (ironically) they are macros deferring to POP_JUMP_IF_{TRUE,FALSE}, which are not viable uops (being manually translated). The hack is that we emit IS_NONE and then set opcode and jump to the POP_JUMP_IF_{TRUE,FALSE} translation code. files: M Lib/test/test_capi/test_misc.py M Python/optimizer.c diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 6df918997b2b1..c0dcff825758a 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2532,6 +2532,36 @@ def testfunc(n): uops = {opname for opname, _ in ex} self.assertIn("_POP_JUMP_IF_FALSE", uops) + def test_pop_jump_if_none(self): + def testfunc(a): + for x in a: + if x is None: + x = 0 + + opt = _testinternalcapi.get_uop_optimizer() + with temporary_optimizer(opt): + testfunc([1, 2, 3]) + + ex = get_first_executor(testfunc) + self.assertIsNotNone(ex) + uops = {opname for opname, _ in ex} + self.assertIn("_POP_JUMP_IF_TRUE", uops) + + def test_pop_jump_if_not_none(self): + def testfunc(a): + for x in a: + if x is not None: + x = 0 + + opt = _testinternalcapi.get_uop_optimizer() + with temporary_optimizer(opt): + testfunc([1, 2, 3]) + + ex = get_first_executor(testfunc) + self.assertIsNotNone(ex) + uops = {opname for opname, _ in ex} + self.assertIn("_POP_JUMP_IF_FALSE", uops) + def test_pop_jump_if_true(self): def testfunc(n): i = 0 diff --git a/Python/optimizer.c b/Python/optimizer.c index 289b202f806ae..693ba375971ae 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -464,9 +464,26 @@ translate_bytecode_to_trace( switch (opcode) { + case POP_JUMP_IF_NONE: + { + RESERVE(2, 2); + ADD_TO_TRACE(IS_NONE, 0); + opcode = POP_JUMP_IF_TRUE; + goto pop_jump_if_bool; + } + + case POP_JUMP_IF_NOT_NONE: + { + RESERVE(2, 2); + ADD_TO_TRACE(IS_NONE, 0); + opcode = POP_JUMP_IF_FALSE; + goto pop_jump_if_bool; + } + case POP_JUMP_IF_FALSE: case POP_JUMP_IF_TRUE: { +pop_jump_if_bool: // Assume jump unlikely (TODO: handle jump likely case) RESERVE(1, 2); _Py_CODEUNIT *target_instr = From webhook-mailer at python.org Mon Jul 17 13:07:55 2023 From: webhook-mailer at python.org (zware) Date: Mon, 17 Jul 2023 17:07:55 -0000 Subject: [Python-checkins] [3.11] gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649) (GH-106761) Message-ID: https://github.com/python/cpython/commit/263d8aa017bd6d858aa9488154bc160fa7249a9e commit: 263d8aa017bd6d858aa9488154bc160fa7249a9e branch: 3.11 author: Zachary Ware committer: zware date: 2023-07-17T12:07:52-05:00 summary: [3.11] gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649) (GH-106761) Co-authored-by: Steve Dower files: A Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst M PCbuild/get_externals.bat M PCbuild/openssl.props M PCbuild/python.props M PCbuild/readme.txt M PCbuild/regen.targets diff --git a/Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst b/Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst new file mode 100644 index 0000000000000..11f411be0f17c --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2023-07-11-20-48-17.gh-issue-99079.CIMftz.rst @@ -0,0 +1 @@ +Update Windows build to use OpenSSL 3.0.9 diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat index 4fc37efd4e333..d41955ac482c7 100644 --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -53,7 +53,7 @@ echo.Fetching external libraries... set libraries= set libraries=%libraries% bzip2-1.0.8 if NOT "%IncludeLibffiSrc%"=="false" set libraries=%libraries% libffi-3.4.4 -if NOT "%IncludeSSLSrc%"=="false" set libraries=%libraries% openssl-1.1.1u +if NOT "%IncludeSSLSrc%"=="false" set libraries=%libraries% openssl-3.0.9 set libraries=%libraries% sqlite-3.42.0.0 if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tcl-core-8.6.12.1 if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tk-8.6.12.1 @@ -77,7 +77,7 @@ echo.Fetching external binaries... set binaries= if NOT "%IncludeLibffi%"=="false" set binaries=%binaries% libffi-3.4.4 -if NOT "%IncludeSSL%"=="false" set binaries=%binaries% openssl-bin-1.1.1u +if NOT "%IncludeSSL%"=="false" set binaries=%binaries% openssl-bin-3.0.9 if NOT "%IncludeTkinter%"=="false" set binaries=%binaries% tcltk-8.6.12.1 if NOT "%IncludeSSLSrc%"=="false" set binaries=%binaries% nasm-2.11.06 diff --git a/PCbuild/openssl.props b/PCbuild/openssl.props index 6081d3c8c6964..a4186f01da21c 100644 --- a/PCbuild/openssl.props +++ b/PCbuild/openssl.props @@ -10,10 +10,10 @@ - <_DLLSuffix>-1_1 + <_DLLSuffix>-3 <_DLLSuffix Condition="$(Platform) == 'ARM'">$(_DLLSuffix)-arm <_DLLSuffix Condition="$(Platform) == 'ARM64'">$(_DLLSuffix)-arm64 - $(_DLLSuffix) + $(_DLLSuffix) <_SSLDLL Include="$(opensslOutDir)\libcrypto$(_DLLSuffix).dll" /> diff --git a/PCbuild/python.props b/PCbuild/python.props index 68052ef668aa6..d3586235c8265 100644 --- a/PCbuild/python.props +++ b/PCbuild/python.props @@ -74,8 +74,8 @@ $(ExternalsDir)libffi-3.4.4\ $(libffiDir)$(ArchName)\ $(libffiOutDir)include - $(ExternalsDir)openssl-1.1.1u\ - $(ExternalsDir)openssl-bin-1.1.1u\$(ArchName)\ + $(ExternalsDir)openssl-3.0.9\ + $(ExternalsDir)openssl-bin-3.0.9\$(ArchName)\ $(opensslOutDir)include $(ExternalsDir)\nasm-2.11.06\ $(ExternalsDir)\zlib-1.2.13\ diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt index d48f7e18eb307..e9a0917227467 100644 --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -168,7 +168,7 @@ _lzma Homepage: https://tukaani.org/xz/ _ssl - Python wrapper for version 1.1.1u of the OpenSSL secure sockets + Python wrapper for version 3.0 of the OpenSSL secure sockets library, which is downloaded from our binaries repository at https://github.com/python/cpython-bin-deps. diff --git a/PCbuild/regen.targets b/PCbuild/regen.targets index 24b5ced1de0e0..d7cb6dbe72ace 100644 --- a/PCbuild/regen.targets +++ b/PCbuild/regen.targets @@ -97,8 +97,9 @@ <_LicenseSources Include="$(PySourcePath)LICENSE; $(PySourcePath)PC\crtlicense.txt; $(bz2Dir)LICENSE; - $(opensslOutDir)LICENSE; $(libffiDir)LICENSE;" /> + <_LicenseSources Include="$(opensslOutDir)LICENSE.txt" Condition="Exists('$(opensslOutDir)LICENSE.txt')" /> + <_LicenseSources Include="$(opensslOutDir)LICENSE" Condition="!Exists('$(opensslOutDir)LICENSE.txt')" /> <_LicenseSources Include="$(tcltkDir)tcllicense.terms; $(tcltkDir)tklicense.terms; $(tcltkDir)tixlicense.terms" Condition="$(IncludeTkinter)" /> From webhook-mailer at python.org Mon Jul 17 14:03:01 2023 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 17 Jul 2023 18:03:01 -0000 Subject: [Python-checkins] gh-106581: Add 10 new opcodes by allowing `assert(kwnames == NULL)` (#106707) Message-ID: https://github.com/python/cpython/commit/2b94a05a0e45e4aae030a28b716a038ef529f8ef commit: 2b94a05a0e45e4aae030a28b716a038ef529f8ef branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-17T11:02:58-07:00 summary: gh-106581: Add 10 new opcodes by allowing `assert(kwnames == NULL)` (#106707) By turning `assert(kwnames == NULL)` into a macro that is not in the "forbidden" list, many instructions that formerly were skipped because they contained such an assert (but no other mention of `kwnames`) are now supported in Tier 2. This covers 10 instructions in total (all specializations of `CALL` that invoke some C code): - `CALL_NO_KW_TYPE_1` - `CALL_NO_KW_STR_1` - `CALL_NO_KW_TUPLE_1` - `CALL_NO_KW_BUILTIN_O` - `CALL_NO_KW_BUILTIN_FAST` - `CALL_NO_KW_LEN` - `CALL_NO_KW_ISINSTANCE` - `CALL_NO_KW_METHOD_DESCRIPTOR_O` - `CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS` - `CALL_NO_KW_METHOD_DESCRIPTOR_FAST` files: M Include/internal/pycore_opcode_metadata.h M Python/bytecodes.c M Python/ceval.c M Python/ceval_macros.h M Python/executor_cases.c.h M Python/generated_cases.c.h M Tools/cases_generator/generate_cases.py diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index 3b2eab23e092f..028736e115b3f 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -1309,7 +1309,17 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[OPCODE_MACRO_EXPAN [GET_YIELD_FROM_ITER] = { .nuops = 1, .uops = { { GET_YIELD_FROM_ITER, 0, 0 } } }, [WITH_EXCEPT_START] = { .nuops = 1, .uops = { { WITH_EXCEPT_START, 0, 0 } } }, [PUSH_EXC_INFO] = { .nuops = 1, .uops = { { PUSH_EXC_INFO, 0, 0 } } }, + [CALL_NO_KW_TYPE_1] = { .nuops = 1, .uops = { { CALL_NO_KW_TYPE_1, 0, 0 } } }, + [CALL_NO_KW_STR_1] = { .nuops = 1, .uops = { { CALL_NO_KW_STR_1, 0, 0 } } }, + [CALL_NO_KW_TUPLE_1] = { .nuops = 1, .uops = { { CALL_NO_KW_TUPLE_1, 0, 0 } } }, [EXIT_INIT_CHECK] = { .nuops = 1, .uops = { { EXIT_INIT_CHECK, 0, 0 } } }, + [CALL_NO_KW_BUILTIN_O] = { .nuops = 1, .uops = { { CALL_NO_KW_BUILTIN_O, 0, 0 } } }, + [CALL_NO_KW_BUILTIN_FAST] = { .nuops = 1, .uops = { { CALL_NO_KW_BUILTIN_FAST, 0, 0 } } }, + [CALL_NO_KW_LEN] = { .nuops = 1, .uops = { { CALL_NO_KW_LEN, 0, 0 } } }, + [CALL_NO_KW_ISINSTANCE] = { .nuops = 1, .uops = { { CALL_NO_KW_ISINSTANCE, 0, 0 } } }, + [CALL_NO_KW_METHOD_DESCRIPTOR_O] = { .nuops = 1, .uops = { { CALL_NO_KW_METHOD_DESCRIPTOR_O, 0, 0 } } }, + [CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS] = { .nuops = 1, .uops = { { CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS, 0, 0 } } }, + [CALL_NO_KW_METHOD_DESCRIPTOR_FAST] = { .nuops = 1, .uops = { { CALL_NO_KW_METHOD_DESCRIPTOR_FAST, 0, 0 } } }, [MAKE_FUNCTION] = { .nuops = 1, .uops = { { MAKE_FUNCTION, 0, 0 } } }, [SET_FUNCTION_ATTRIBUTE] = { .nuops = 1, .uops = { { SET_FUNCTION_ATTRIBUTE, 0, 0 } } }, [BUILD_SLICE] = { .nuops = 1, .uops = { { BUILD_SLICE, 0, 0 } } }, diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 3c3992c068b06..652372cb23dc5 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2776,7 +2776,7 @@ dummy_func( } inst(KW_NAMES, (--)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg < PyTuple_GET_SIZE(FRAME_CO_CONSTS)); kwnames = GETITEM(FRAME_CO_CONSTS, oparg); } @@ -2927,7 +2927,7 @@ dummy_func( } inst(CALL_PY_EXACT_ARGS, (unused/1, func_version/2, method, callable, args[oparg] -- unused)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; int argcount = oparg; @@ -2955,7 +2955,7 @@ dummy_func( } inst(CALL_PY_WITH_DEFAULTS, (unused/1, func_version/2, method, callable, args[oparg] -- unused)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; int argcount = oparg; @@ -2993,7 +2993,7 @@ dummy_func( } inst(CALL_NO_KW_TYPE_1, (unused/1, unused/2, null, callable, args[oparg] -- res)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); PyObject *obj = args[0]; @@ -3005,7 +3005,7 @@ dummy_func( } inst(CALL_NO_KW_STR_1, (unused/1, unused/2, null, callable, args[oparg] -- res)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); DEOPT_IF(callable != (PyObject *)&PyUnicode_Type, CALL); @@ -3019,7 +3019,7 @@ dummy_func( } inst(CALL_NO_KW_TUPLE_1, (unused/1, unused/2, null, callable, args[oparg] -- res)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); DEOPT_IF(callable != (PyObject *)&PyTuple_Type, CALL); @@ -3038,7 +3038,7 @@ dummy_func( * 2. Pushes a shim frame to the frame stack (to cleanup after ``__init__``) * 3. Pushes the frame for ``__init__`` to the frame stack * */ - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); _PyCallCache *cache = (_PyCallCache *)next_instr; DEOPT_IF(null != NULL, CALL); DEOPT_IF(!PyType_Check(callable), CALL); @@ -3122,7 +3122,7 @@ dummy_func( inst(CALL_NO_KW_BUILTIN_O, (unused/1, unused/2, method, callable, args[oparg] -- res)) { /* Builtin METH_O functions */ - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3153,7 +3153,7 @@ dummy_func( inst(CALL_NO_KW_BUILTIN_FAST, (unused/1, unused/2, method, callable, args[oparg] -- res)) { /* Builtin METH_FASTCALL functions, without keywords */ - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3222,7 +3222,7 @@ dummy_func( } inst(CALL_NO_KW_LEN, (unused/1, unused/2, method, callable, args[oparg] -- res)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); /* len(o) */ int is_meth = method != NULL; int total_args = oparg; @@ -3249,7 +3249,7 @@ dummy_func( } inst(CALL_NO_KW_ISINSTANCE, (unused/1, unused/2, method, callable, args[oparg] -- res)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); /* isinstance(o, o2) */ int is_meth = method != NULL; int total_args = oparg; @@ -3279,7 +3279,7 @@ dummy_func( // This is secretly a super-instruction inst(CALL_NO_KW_LIST_APPEND, (unused/1, unused/2, method, self, args[oparg] -- unused)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); assert(method != NULL); PyInterpreterState *interp = _PyInterpreterState_GET(); @@ -3299,7 +3299,7 @@ dummy_func( } inst(CALL_NO_KW_METHOD_DESCRIPTOR_O, (unused/1, unused/2, method, unused, args[oparg] -- res)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3365,7 +3365,7 @@ dummy_func( } inst(CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS, (unused/1, unused/2, method, unused, args[oparg] -- res)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; int total_args = oparg; @@ -3397,7 +3397,7 @@ dummy_func( } inst(CALL_NO_KW_METHOD_DESCRIPTOR_FAST, (unused/1, unused/2, method, unused, args[oparg] -- res)) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); int is_meth = method != NULL; int total_args = oparg; if (is_meth) { diff --git a/Python/ceval.c b/Python/ceval.c index d6c72fa3ff386..f13ba9883d981 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2706,6 +2706,9 @@ void Py_LeaveRecursiveCall(void) ///////////////////// Experimental UOp Interpreter ///////////////////// +#undef ASSERT_KWNAMES_IS_NULL +#define ASSERT_KWNAMES_IS_NULL() (void)0 + #undef DEOPT_IF #define DEOPT_IF(COND, INSTNAME) \ if ((COND)) { \ @@ -2746,6 +2749,7 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject int opcode; uint64_t operand; int oparg; + for (;;) { opcode = self->trace[pc].opcode; operand = self->trace[pc].operand; diff --git a/Python/ceval_macros.h b/Python/ceval_macros.h index 72800aaaaa2ac..874bd45becf0c 100644 --- a/Python/ceval_macros.h +++ b/Python/ceval_macros.h @@ -349,3 +349,5 @@ static const convertion_func_ptr CONVERSION_FUNCTIONS[4] = { [FVC_REPR] = PyObject_Repr, [FVC_ASCII] = PyObject_ASCII }; + +#define ASSERT_KWNAMES_IS_NULL() assert(kwnames == NULL) diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index ae21ffad94d80..d85e23b5abb8e 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1904,6 +1904,68 @@ break; } + case CALL_NO_KW_TYPE_1: { + PyObject **args = (stack_pointer - oparg); + PyObject *callable = stack_pointer[-(1 + oparg)]; + PyObject *null = stack_pointer[-(2 + oparg)]; + PyObject *res; + ASSERT_KWNAMES_IS_NULL(); + assert(oparg == 1); + DEOPT_IF(null != NULL, CALL); + PyObject *obj = args[0]; + DEOPT_IF(callable != (PyObject *)&PyType_Type, CALL); + STAT_INC(CALL, hit); + res = Py_NewRef(Py_TYPE(obj)); + Py_DECREF(obj); + Py_DECREF(&PyType_Type); // I.e., callable + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + + case CALL_NO_KW_STR_1: { + PyObject **args = (stack_pointer - oparg); + PyObject *callable = stack_pointer[-(1 + oparg)]; + PyObject *null = stack_pointer[-(2 + oparg)]; + PyObject *res; + ASSERT_KWNAMES_IS_NULL(); + assert(oparg == 1); + DEOPT_IF(null != NULL, CALL); + DEOPT_IF(callable != (PyObject *)&PyUnicode_Type, CALL); + STAT_INC(CALL, hit); + PyObject *arg = args[0]; + res = PyObject_Str(arg); + Py_DECREF(arg); + Py_DECREF(&PyUnicode_Type); // I.e., callable + if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + + case CALL_NO_KW_TUPLE_1: { + PyObject **args = (stack_pointer - oparg); + PyObject *callable = stack_pointer[-(1 + oparg)]; + PyObject *null = stack_pointer[-(2 + oparg)]; + PyObject *res; + ASSERT_KWNAMES_IS_NULL(); + assert(oparg == 1); + DEOPT_IF(null != NULL, CALL); + DEOPT_IF(callable != (PyObject *)&PyTuple_Type, CALL); + STAT_INC(CALL, hit); + PyObject *arg = args[0]; + res = PySequence_Tuple(arg); + Py_DECREF(arg); + Py_DECREF(&PyTuple_Type); // I.e., tuple + if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + case EXIT_INIT_CHECK: { PyObject *should_be_none = stack_pointer[-1]; assert(STACK_LEVEL() == 2); @@ -1917,6 +1979,273 @@ break; } + case CALL_NO_KW_BUILTIN_O: { + PyObject **args = (stack_pointer - oparg); + PyObject *callable = stack_pointer[-(1 + oparg)]; + PyObject *method = stack_pointer[-(2 + oparg)]; + PyObject *res; + /* Builtin METH_O functions */ + ASSERT_KWNAMES_IS_NULL(); + int is_meth = method != NULL; + int total_args = oparg; + if (is_meth) { + callable = method; + args--; + total_args++; + } + DEOPT_IF(total_args != 1, CALL); + DEOPT_IF(!PyCFunction_CheckExact(callable), CALL); + DEOPT_IF(PyCFunction_GET_FLAGS(callable) != METH_O, CALL); + STAT_INC(CALL, hit); + PyCFunction cfunc = PyCFunction_GET_FUNCTION(callable); + // This is slower but CPython promises to check all non-vectorcall + // function calls. + if (_Py_EnterRecursiveCallTstate(tstate, " while calling a Python object")) { + goto error; + } + PyObject *arg = args[0]; + res = _PyCFunction_TrampolineCall(cfunc, PyCFunction_GET_SELF(callable), arg); + _Py_LeaveRecursiveCallTstate(tstate); + assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); + + Py_DECREF(arg); + Py_DECREF(callable); + if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + + case CALL_NO_KW_BUILTIN_FAST: { + PyObject **args = (stack_pointer - oparg); + PyObject *callable = stack_pointer[-(1 + oparg)]; + PyObject *method = stack_pointer[-(2 + oparg)]; + PyObject *res; + /* Builtin METH_FASTCALL functions, without keywords */ + ASSERT_KWNAMES_IS_NULL(); + int is_meth = method != NULL; + int total_args = oparg; + if (is_meth) { + callable = method; + args--; + total_args++; + } + DEOPT_IF(!PyCFunction_CheckExact(callable), CALL); + DEOPT_IF(PyCFunction_GET_FLAGS(callable) != METH_FASTCALL, CALL); + STAT_INC(CALL, hit); + PyCFunction cfunc = PyCFunction_GET_FUNCTION(callable); + /* res = func(self, args, nargs) */ + res = ((_PyCFunctionFast)(void(*)(void))cfunc)( + PyCFunction_GET_SELF(callable), + args, + total_args); + assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); + + /* Free the arguments. */ + for (int i = 0; i < total_args; i++) { + Py_DECREF(args[i]); + } + Py_DECREF(callable); + if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } + /* Not deopting because this doesn't mean our optimization was + wrong. `res` can be NULL for valid reasons. Eg. getattr(x, + 'invalid'). In those cases an exception is set, so we must + handle it. + */ + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + + case CALL_NO_KW_LEN: { + PyObject **args = (stack_pointer - oparg); + PyObject *callable = stack_pointer[-(1 + oparg)]; + PyObject *method = stack_pointer[-(2 + oparg)]; + PyObject *res; + ASSERT_KWNAMES_IS_NULL(); + /* len(o) */ + int is_meth = method != NULL; + int total_args = oparg; + if (is_meth) { + callable = method; + args--; + total_args++; + } + DEOPT_IF(total_args != 1, CALL); + PyInterpreterState *interp = _PyInterpreterState_GET(); + DEOPT_IF(callable != interp->callable_cache.len, CALL); + STAT_INC(CALL, hit); + PyObject *arg = args[0]; + Py_ssize_t len_i = PyObject_Length(arg); + if (len_i < 0) { + goto error; + } + res = PyLong_FromSsize_t(len_i); + assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); + + Py_DECREF(callable); + Py_DECREF(arg); + if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + + case CALL_NO_KW_ISINSTANCE: { + PyObject **args = (stack_pointer - oparg); + PyObject *callable = stack_pointer[-(1 + oparg)]; + PyObject *method = stack_pointer[-(2 + oparg)]; + PyObject *res; + ASSERT_KWNAMES_IS_NULL(); + /* isinstance(o, o2) */ + int is_meth = method != NULL; + int total_args = oparg; + if (is_meth) { + callable = method; + args--; + total_args++; + } + DEOPT_IF(total_args != 2, CALL); + PyInterpreterState *interp = _PyInterpreterState_GET(); + DEOPT_IF(callable != interp->callable_cache.isinstance, CALL); + STAT_INC(CALL, hit); + PyObject *cls = args[1]; + PyObject *inst = args[0]; + int retval = PyObject_IsInstance(inst, cls); + if (retval < 0) { + goto error; + } + res = PyBool_FromLong(retval); + assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); + + Py_DECREF(inst); + Py_DECREF(cls); + Py_DECREF(callable); + if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + + case CALL_NO_KW_METHOD_DESCRIPTOR_O: { + PyObject **args = (stack_pointer - oparg); + PyObject *method = stack_pointer[-(2 + oparg)]; + PyObject *res; + ASSERT_KWNAMES_IS_NULL(); + int is_meth = method != NULL; + int total_args = oparg; + if (is_meth) { + args--; + total_args++; + } + PyMethodDescrObject *callable = + (PyMethodDescrObject *)PEEK(total_args + 1); + DEOPT_IF(total_args != 2, CALL); + DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), CALL); + PyMethodDef *meth = callable->d_method; + DEOPT_IF(meth->ml_flags != METH_O, CALL); + PyObject *arg = args[1]; + PyObject *self = args[0]; + DEOPT_IF(!Py_IS_TYPE(self, callable->d_common.d_type), CALL); + STAT_INC(CALL, hit); + PyCFunction cfunc = meth->ml_meth; + // This is slower but CPython promises to check all non-vectorcall + // function calls. + if (_Py_EnterRecursiveCallTstate(tstate, " while calling a Python object")) { + goto error; + } + res = _PyCFunction_TrampolineCall(cfunc, self, arg); + _Py_LeaveRecursiveCallTstate(tstate); + assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); + Py_DECREF(self); + Py_DECREF(arg); + Py_DECREF(callable); + if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + + case CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS: { + PyObject **args = (stack_pointer - oparg); + PyObject *method = stack_pointer[-(2 + oparg)]; + PyObject *res; + ASSERT_KWNAMES_IS_NULL(); + assert(oparg == 0 || oparg == 1); + int is_meth = method != NULL; + int total_args = oparg; + if (is_meth) { + args--; + total_args++; + } + DEOPT_IF(total_args != 1, CALL); + PyMethodDescrObject *callable = (PyMethodDescrObject *)SECOND(); + DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), CALL); + PyMethodDef *meth = callable->d_method; + PyObject *self = args[0]; + DEOPT_IF(!Py_IS_TYPE(self, callable->d_common.d_type), CALL); + DEOPT_IF(meth->ml_flags != METH_NOARGS, CALL); + STAT_INC(CALL, hit); + PyCFunction cfunc = meth->ml_meth; + // This is slower but CPython promises to check all non-vectorcall + // function calls. + if (_Py_EnterRecursiveCallTstate(tstate, " while calling a Python object")) { + goto error; + } + res = _PyCFunction_TrampolineCall(cfunc, self, NULL); + _Py_LeaveRecursiveCallTstate(tstate); + assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); + Py_DECREF(self); + Py_DECREF(callable); + if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + + case CALL_NO_KW_METHOD_DESCRIPTOR_FAST: { + PyObject **args = (stack_pointer - oparg); + PyObject *method = stack_pointer[-(2 + oparg)]; + PyObject *res; + ASSERT_KWNAMES_IS_NULL(); + int is_meth = method != NULL; + int total_args = oparg; + if (is_meth) { + args--; + total_args++; + } + PyMethodDescrObject *callable = + (PyMethodDescrObject *)PEEK(total_args + 1); + /* Builtin METH_FASTCALL methods, without keywords */ + DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), CALL); + PyMethodDef *meth = callable->d_method; + DEOPT_IF(meth->ml_flags != METH_FASTCALL, CALL); + PyObject *self = args[0]; + DEOPT_IF(!Py_IS_TYPE(self, callable->d_common.d_type), CALL); + STAT_INC(CALL, hit); + _PyCFunctionFast cfunc = + (_PyCFunctionFast)(void(*)(void))meth->ml_meth; + int nargs = total_args - 1; + res = cfunc(self, args + 1, nargs); + assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); + /* Clear the stack of the arguments. */ + for (int i = 0; i < total_args; i++) { + Py_DECREF(args[i]); + } + Py_DECREF(callable); + if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } + STACK_SHRINK(oparg); + STACK_SHRINK(1); + stack_pointer[-1] = res; + break; + } + case MAKE_FUNCTION: { PyObject *codeobj = stack_pointer[-1]; PyObject *func; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 392914c0521e9..1fd76715dc3e4 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3462,7 +3462,7 @@ } TARGET(KW_NAMES) { - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg < PyTuple_GET_SIZE(FRAME_CO_CONSTS)); kwnames = GETITEM(FRAME_CO_CONSTS, oparg); DISPATCH(); @@ -3599,7 +3599,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; int argcount = oparg; @@ -3631,7 +3631,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; int argcount = oparg; @@ -3673,7 +3673,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); PyObject *obj = args[0]; @@ -3694,7 +3694,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); DEOPT_IF(callable != (PyObject *)&PyUnicode_Type, CALL); @@ -3717,7 +3717,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); DEOPT_IF(callable != (PyObject *)&PyTuple_Type, CALL); @@ -3744,7 +3744,7 @@ * 2. Pushes a shim frame to the frame stack (to cleanup after ``__init__``) * 3. Pushes the frame for ``__init__`` to the frame stack * */ - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); _PyCallCache *cache = (_PyCallCache *)next_instr; DEOPT_IF(null != NULL, CALL); DEOPT_IF(!PyType_Check(callable), CALL); @@ -3844,7 +3844,7 @@ PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; /* Builtin METH_O functions */ - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3884,7 +3884,7 @@ PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; /* Builtin METH_FASTCALL functions, without keywords */ - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3971,7 +3971,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); /* len(o) */ int is_meth = method != NULL; int total_args = oparg; @@ -4007,7 +4007,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); /* isinstance(o, o2) */ int is_meth = method != NULL; int total_args = oparg; @@ -4044,7 +4044,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); assert(method != NULL); PyInterpreterState *interp = _PyInterpreterState_GET(); @@ -4067,7 +4067,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4149,7 +4149,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; int total_args = oparg; @@ -4189,7 +4189,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - assert(kwnames == NULL); + ASSERT_KWNAMES_IS_NULL(); int is_meth = method != NULL; int total_args = oparg; if (is_meth) { diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index a0a8b8cbe4bab..112f29a83e4c1 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -408,27 +408,31 @@ def __init__(self, inst: parser.InstDef): def is_viable_uop(self) -> bool: """Whether this instruction is viable as a uop.""" + dprint: typing.Callable[..., None] = lambda *args, **kwargs: None + # if self.name.startswith("CALL"): + # dprint = print + if self.name == "EXIT_TRACE": return True # This has 'return frame' but it's okay if self.always_exits: - # print(f"Skipping {self.name} because it always exits") + dprint(f"Skipping {self.name} because it always exits") return False if self.instr_flags.HAS_ARG_FLAG: # If the instruction uses oparg, it cannot use any caches if self.active_caches: - # print(f"Skipping {self.name} because it uses oparg and caches") + dprint(f"Skipping {self.name} because it uses oparg and caches") return False else: # If it doesn't use oparg, it can have one cache entry if len(self.active_caches) > 1: - # print(f"Skipping {self.name} because it has >1 cache entries") + dprint(f"Skipping {self.name} because it has >1 cache entries") return False res = True for forbidden in FORBIDDEN_NAMES_IN_UOPS: # NOTE: To disallow unspecialized uops, use # if variable_used(self.inst, forbidden): if variable_used_unspecialized(self.inst, forbidden): - # print(f"Skipping {self.name} because it uses {forbidden}") + dprint(f"Skipping {self.name} because it uses {forbidden}") res = False return res @@ -1499,6 +1503,8 @@ def write_executor_instructions(self) -> None: with self.out.block(f"case {thing.name}:"): instr.write(self.out, tier=TIER_TWO) self.out.emit("break;") + # elif instr.kind != "op": + # print(f"NOTE: {thing.name} is not a viable uop") case parser.Macro(): pass case parser.Pseudo(): From webhook-mailer at python.org Mon Jul 17 14:57:44 2023 From: webhook-mailer at python.org (cjw296) Date: Mon, 17 Jul 2023 18:57:44 -0000 Subject: [Python-checkins] gh-61215: threadingmock: Improve test suite to avoid race conditions (#106822) Message-ID: https://github.com/python/cpython/commit/7e96370a946a2ca0f2f25af4ce5b3b59f020721b commit: 7e96370a946a2ca0f2f25af4ce5b3b59f020721b branch: main author: Mario Corchero committer: cjw296 date: 2023-07-17T18:57:40Z summary: gh-61215: threadingmock: Improve test suite to avoid race conditions (#106822) threadingmock: Improve test suite to avoid race conditions Simplify tests and split them into multiple tests to prevent assertions from triggering race conditions. Additionally, we rely on calling the mocks without delay to validate the functionality of matching calls. files: M Lib/test/test_unittest/testmock/testthreadingmock.py diff --git a/Lib/test/test_unittest/testmock/testthreadingmock.py b/Lib/test/test_unittest/testmock/testthreadingmock.py index b6e12bcb3cda9..94e71921d9bc0 100644 --- a/Lib/test/test_unittest/testmock/testthreadingmock.py +++ b/Lib/test/test_unittest/testmock/testthreadingmock.py @@ -8,6 +8,8 @@ threading_helper.requires_working_threading(module=True) +VERY_SHORT_TIMEOUT = 0.1 + class Something: def method_1(self): @@ -93,167 +95,86 @@ def test_no_name_clash(self): waitable_mock.wait_until_called() waitable_mock.wait_until_any_call_with("works") - def test_wait_success(self): + def test_patch(self): waitable_mock = self._make_mock(spec=Something) - with patch(f"{__name__}.Something", waitable_mock): - something = Something() - self.run_async(something.method_1, delay=0.01) - something.method_1.wait_until_called() - something.method_1.wait_until_any_call_with() - something.method_1.assert_called() - - def test_wait_success_with_instance_timeout(self): - waitable_mock = self._make_mock(timeout=1) - - with patch(f"{__name__}.Something", waitable_mock): - something = Something() - self.run_async(something.method_1, delay=0.01) - something.method_1.wait_until_called() - something.method_1.wait_until_any_call_with() - something.method_1.assert_called() - - def test_wait_failed_with_instance_timeout(self): - waitable_mock = self._make_mock(timeout=0.01) - - with patch(f"{__name__}.Something", waitable_mock): - something = Something() - self.run_async(something.method_1, delay=0.5) - self.assertRaises(AssertionError, waitable_mock.method_1.wait_until_called) - self.assertRaises( - AssertionError, waitable_mock.method_1.wait_until_any_call_with - ) - - def test_wait_success_with_timeout_override(self): - waitable_mock = self._make_mock(timeout=0.01) - - with patch(f"{__name__}.Something", waitable_mock): - something = Something() - self.run_async(something.method_1, delay=0.05) - something.method_1.wait_until_called(timeout=1) - - def test_wait_failed_with_timeout_override(self): - waitable_mock = self._make_mock(timeout=1) - - with patch(f"{__name__}.Something", waitable_mock): - something = Something() - self.run_async(something.method_1, delay=0.5) - with self.assertRaises(AssertionError): - something.method_1.wait_until_called(timeout=0.05) - - def test_wait_success_called_before(self): - waitable_mock = self._make_mock() - with patch(f"{__name__}.Something", waitable_mock): something = Something() something.method_1() something.method_1.wait_until_called() - something.method_1.wait_until_any_call_with() - something.method_1.assert_called() - - def test_wait_magic_method(self): - waitable_mock = self._make_mock() - with patch(f"{__name__}.Something", waitable_mock): - something = Something() - self.run_async(something.method_1.__str__, delay=0.01) - something.method_1.__str__.wait_until_called() - something.method_1.__str__.assert_called() - - def test_wait_until_any_call_with_positional(self): + def test_wait_already_called_success(self): waitable_mock = self._make_mock(spec=Something) + waitable_mock.method_1() + waitable_mock.method_1.wait_until_called() + waitable_mock.method_1.wait_until_any_call_with() + waitable_mock.method_1.assert_called() - with patch(f"{__name__}.Something", waitable_mock): - something = Something() - self.run_async(something.method_1, 1, delay=0.2) - self.assertNotIn(call(1), something.method_1.mock_calls) - self.run_async(something.method_1, 2, delay=0.5) - self.run_async(something.method_1, 3, delay=0.6) - - something.method_1.wait_until_any_call_with(1) - something.method_1.assert_called_with(1) - self.assertNotIn(call(2), something.method_1.mock_calls) - self.assertNotIn(call(3), something.method_1.mock_calls) - - something.method_1.wait_until_any_call_with(3) - self.assertIn(call(2), something.method_1.mock_calls) - something.method_1.wait_until_any_call_with(2) - - def test_wait_until_any_call_with_keywords(self): + def test_wait_until_called_success(self): waitable_mock = self._make_mock(spec=Something) + self.run_async(waitable_mock.method_1, delay=VERY_SHORT_TIMEOUT) + waitable_mock.method_1.wait_until_called() - with patch(f"{__name__}.Something", waitable_mock): - something = Something() - self.run_async(something.method_1, a=1, delay=0.2) - self.assertNotIn(call(a=1), something.method_1.mock_calls) - self.run_async(something.method_1, b=2, delay=0.5) - self.run_async(something.method_1, c=3, delay=0.6) - - something.method_1.wait_until_any_call_with(a=1) - something.method_1.assert_called_with(a=1) - self.assertNotIn(call(b=2), something.method_1.mock_calls) - self.assertNotIn(call(c=3), something.method_1.mock_calls) - - something.method_1.wait_until_any_call_with(c=3) - self.assertIn(call(b=2), something.method_1.mock_calls) - something.method_1.wait_until_any_call_with(b=2) - - def test_wait_until_any_call_with_no_argument_fails_when_called_with_arg(self): - waitable_mock = self._make_mock(timeout=0.01) - - with patch(f"{__name__}.Something", waitable_mock): - something = Something() - something.method_1(1) - - something.method_1.assert_called_with(1) - with self.assertRaises(AssertionError): - something.method_1.wait_until_any_call_with() + def test_wait_until_called_method_timeout(self): + waitable_mock = self._make_mock(spec=Something) + with self.assertRaises(AssertionError): + waitable_mock.method_1.wait_until_called(timeout=VERY_SHORT_TIMEOUT) - something.method_1() - something.method_1.wait_until_any_call_with() + def test_wait_until_called_instance_timeout(self): + waitable_mock = self._make_mock(spec=Something, timeout=VERY_SHORT_TIMEOUT) + with self.assertRaises(AssertionError): + waitable_mock.method_1.wait_until_called() - def test_wait_until_any_call_with_global_default(self): + def test_wait_until_called_global_timeout(self): with patch.object(ThreadingMock, "DEFAULT_TIMEOUT"): - ThreadingMock.DEFAULT_TIMEOUT = 0.01 - m = self._make_mock() + ThreadingMock.DEFAULT_TIMEOUT = VERY_SHORT_TIMEOUT + waitable_mock = self._make_mock(spec=Something) with self.assertRaises(AssertionError): - m.wait_until_any_call_with() - with self.assertRaises(AssertionError): - m.wait_until_called() + waitable_mock.method_1.wait_until_called() - m() - m.wait_until_any_call_with() - assert ThreadingMock.DEFAULT_TIMEOUT != 0.01 + def test_wait_until_any_call_with_success(self): + waitable_mock = self._make_mock() + self.run_async(waitable_mock, delay=VERY_SHORT_TIMEOUT) + waitable_mock.wait_until_any_call_with() - def test_wait_until_any_call_with_change_global_and_override(self): - with patch.object(ThreadingMock, "DEFAULT_TIMEOUT"): - ThreadingMock.DEFAULT_TIMEOUT = 0.01 + def test_wait_until_any_call_with_instance_timeout(self): + waitable_mock = self._make_mock(timeout=VERY_SHORT_TIMEOUT) + with self.assertRaises(AssertionError): + waitable_mock.wait_until_any_call_with() - m1 = self._make_mock() - self.run_async(m1, delay=0.1) + def test_wait_until_any_call_global_timeout(self): + with patch.object(ThreadingMock, "DEFAULT_TIMEOUT"): + ThreadingMock.DEFAULT_TIMEOUT = VERY_SHORT_TIMEOUT + waitable_mock = self._make_mock() with self.assertRaises(AssertionError): - m1.wait_until_called() + waitable_mock.wait_until_any_call_with() - m2 = self._make_mock(timeout=1) - self.run_async(m2, delay=0.1) - m2.wait_until_called() - - m3 = self._make_mock() - self.run_async(m3, delay=0.1) - m3.wait_until_called(timeout=1) - - m4 = self._make_mock() - self.run_async(m4, delay=0.1) - m4.wait_until_called(timeout=None) + def test_wait_until_any_call_positional(self): + waitable_mock = self._make_mock(timeout=VERY_SHORT_TIMEOUT) + waitable_mock.method_1(1, 2, 3) + waitable_mock.method_1.wait_until_any_call_with(1, 2, 3) + with self.assertRaises(AssertionError): + waitable_mock.method_1.wait_until_any_call_with(2, 3, 1) + with self.assertRaises(AssertionError): + waitable_mock.method_1.wait_until_any_call_with() - m5 = self._make_mock(timeout=None) - self.run_async(m5, delay=0.1) - m5.wait_until_called() + def test_wait_until_any_call_kw(self): + waitable_mock = self._make_mock(timeout=VERY_SHORT_TIMEOUT) + waitable_mock.method_1(a=1, b=2) + waitable_mock.method_1.wait_until_any_call_with(a=1, b=2) + with self.assertRaises(AssertionError): + waitable_mock.method_1.wait_until_any_call_with(a=2, b=1) + with self.assertRaises(AssertionError): + waitable_mock.method_1.wait_until_any_call_with() - assert ThreadingMock.DEFAULT_TIMEOUT != 0.01 + def test_magic_methods_success(self): + waitable_mock = self._make_mock() + str(waitable_mock) + waitable_mock.__str__.wait_until_called() + waitable_mock.__str__.assert_called() def test_reset_mock_resets_wait(self): - m = self._make_mock(timeout=0.01) + m = self._make_mock(timeout=VERY_SHORT_TIMEOUT) with self.assertRaises(AssertionError): m.wait_until_called() From webhook-mailer at python.org Mon Jul 17 15:12:37 2023 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 17 Jul 2023 19:12:37 -0000 Subject: [Python-checkins] gh-106603: Make uop struct a triple (opcode, oparg, operand) (#106794) Message-ID: https://github.com/python/cpython/commit/8e9a1a032233f06ce0f1acdf5f983d614c8745a5 commit: 8e9a1a032233f06ce0f1acdf5f983d614c8745a5 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-17T12:12:33-07:00 summary: gh-106603: Make uop struct a triple (opcode, oparg, operand) (#106794) files: M Include/internal/pycore_opcode_metadata.h M Include/internal/pycore_uops.h M Lib/test/test_capi/test_misc.py M Python/bytecodes.c M Python/ceval.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/optimizer.c M Tools/cases_generator/generate_cases.py diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index 028736e115b3f..c3a0dbb478a7c 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -38,21 +38,24 @@ #define _SKIP_CACHE 314 #define _GUARD_GLOBALS_VERSION 315 #define _GUARD_BUILTINS_VERSION 316 -#define _GUARD_TYPE_VERSION 317 -#define _CHECK_MANAGED_OBJECT_HAS_VALUES 318 -#define IS_NONE 319 -#define _ITER_CHECK_LIST 320 -#define _IS_ITER_EXHAUSTED_LIST 321 -#define _ITER_NEXT_LIST 322 -#define _ITER_CHECK_TUPLE 323 -#define _IS_ITER_EXHAUSTED_TUPLE 324 -#define _ITER_NEXT_TUPLE 325 -#define _ITER_CHECK_RANGE 326 -#define _IS_ITER_EXHAUSTED_RANGE 327 -#define _ITER_NEXT_RANGE 328 -#define _POP_JUMP_IF_FALSE 329 -#define _POP_JUMP_IF_TRUE 330 -#define JUMP_TO_TOP 331 +#define _LOAD_GLOBAL_MODULE 317 +#define _LOAD_GLOBAL_BUILTINS 318 +#define _GUARD_TYPE_VERSION 319 +#define _CHECK_MANAGED_OBJECT_HAS_VALUES 320 +#define _LOAD_ATTR_INSTANCE_VALUE 321 +#define IS_NONE 322 +#define _ITER_CHECK_LIST 323 +#define _IS_ITER_EXHAUSTED_LIST 324 +#define _ITER_NEXT_LIST 325 +#define _ITER_CHECK_TUPLE 326 +#define _IS_ITER_EXHAUSTED_TUPLE 327 +#define _ITER_NEXT_TUPLE 328 +#define _ITER_CHECK_RANGE 329 +#define _IS_ITER_EXHAUSTED_RANGE 330 +#define _ITER_NEXT_RANGE 331 +#define _POP_JUMP_IF_FALSE 332 +#define _POP_JUMP_IF_TRUE 333 +#define JUMP_TO_TOP 334 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -1245,7 +1248,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[OPCODE_MACRO_EXPAN [BINARY_SUBSCR_DICT] = { .nuops = 1, .uops = { { BINARY_SUBSCR_DICT, 0, 0 } } }, [LIST_APPEND] = { .nuops = 1, .uops = { { LIST_APPEND, 0, 0 } } }, [SET_ADD] = { .nuops = 1, .uops = { { SET_ADD, 0, 0 } } }, - [STORE_SUBSCR] = { .nuops = 1, .uops = { { STORE_SUBSCR, 1, 0 } } }, + [STORE_SUBSCR] = { .nuops = 1, .uops = { { STORE_SUBSCR, 0, 0 } } }, [STORE_SUBSCR_LIST_INT] = { .nuops = 1, .uops = { { STORE_SUBSCR_LIST_INT, 0, 0 } } }, [STORE_SUBSCR_DICT] = { .nuops = 1, .uops = { { STORE_SUBSCR_DICT, 0, 0 } } }, [DELETE_SUBSCR] = { .nuops = 1, .uops = { { DELETE_SUBSCR, 0, 0 } } }, @@ -1264,6 +1267,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[OPCODE_MACRO_EXPAN [UNPACK_SEQUENCE_TUPLE] = { .nuops = 1, .uops = { { UNPACK_SEQUENCE_TUPLE, 0, 0 } } }, [UNPACK_SEQUENCE_LIST] = { .nuops = 1, .uops = { { UNPACK_SEQUENCE_LIST, 0, 0 } } }, [UNPACK_EX] = { .nuops = 1, .uops = { { UNPACK_EX, 0, 0 } } }, + [STORE_ATTR] = { .nuops = 1, .uops = { { STORE_ATTR, 0, 0 } } }, [DELETE_ATTR] = { .nuops = 1, .uops = { { DELETE_ATTR, 0, 0 } } }, [STORE_GLOBAL] = { .nuops = 1, .uops = { { STORE_GLOBAL, 0, 0 } } }, [DELETE_GLOBAL] = { .nuops = 1, .uops = { { DELETE_GLOBAL, 0, 0 } } }, @@ -1271,6 +1275,8 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[OPCODE_MACRO_EXPAN [LOAD_NAME] = { .nuops = 2, .uops = { { _LOAD_LOCALS, 0, 0 }, { _LOAD_FROM_DICT_OR_GLOBALS, 0, 0 } } }, [LOAD_FROM_DICT_OR_GLOBALS] = { .nuops = 1, .uops = { { _LOAD_FROM_DICT_OR_GLOBALS, 0, 0 } } }, [LOAD_GLOBAL] = { .nuops = 1, .uops = { { LOAD_GLOBAL, 0, 0 } } }, + [LOAD_GLOBAL_MODULE] = { .nuops = 4, .uops = { { _SKIP_CACHE, 0, 0 }, { _GUARD_GLOBALS_VERSION, 1, 1 }, { _SKIP_CACHE, 0, 0 }, { _LOAD_GLOBAL_MODULE, 1, 3 } } }, + [LOAD_GLOBAL_BUILTIN] = { .nuops = 4, .uops = { { _SKIP_CACHE, 0, 0 }, { _GUARD_GLOBALS_VERSION, 1, 1 }, { _GUARD_BUILTINS_VERSION, 1, 2 }, { _LOAD_GLOBAL_BUILTINS, 1, 3 } } }, [DELETE_FAST] = { .nuops = 1, .uops = { { DELETE_FAST, 0, 0 } } }, [DELETE_DEREF] = { .nuops = 1, .uops = { { DELETE_DEREF, 0, 0 } } }, [LOAD_FROM_DICT_OR_DEREF] = { .nuops = 1, .uops = { { LOAD_FROM_DICT_OR_DEREF, 0, 0 } } }, @@ -1292,6 +1298,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[OPCODE_MACRO_EXPAN [LOAD_SUPER_ATTR_ATTR] = { .nuops = 1, .uops = { { LOAD_SUPER_ATTR_ATTR, 0, 0 } } }, [LOAD_SUPER_ATTR_METHOD] = { .nuops = 1, .uops = { { LOAD_SUPER_ATTR_METHOD, 0, 0 } } }, [LOAD_ATTR] = { .nuops = 1, .uops = { { LOAD_ATTR, 0, 0 } } }, + [LOAD_ATTR_INSTANCE_VALUE] = { .nuops = 4, .uops = { { _SKIP_CACHE, 0, 0 }, { _GUARD_TYPE_VERSION, 2, 1 }, { _CHECK_MANAGED_OBJECT_HAS_VALUES, 0, 0 }, { _LOAD_ATTR_INSTANCE_VALUE, 1, 3 } } }, [COMPARE_OP] = { .nuops = 1, .uops = { { COMPARE_OP, 0, 0 } } }, [COMPARE_OP_FLOAT] = { .nuops = 1, .uops = { { COMPARE_OP_FLOAT, 0, 0 } } }, [COMPARE_OP_INT] = { .nuops = 1, .uops = { { COMPARE_OP_INT, 0, 0 } } }, @@ -1348,8 +1355,11 @@ const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE] = { [_SKIP_CACHE] = "_SKIP_CACHE", [_GUARD_GLOBALS_VERSION] = "_GUARD_GLOBALS_VERSION", [_GUARD_BUILTINS_VERSION] = "_GUARD_BUILTINS_VERSION", + [_LOAD_GLOBAL_MODULE] = "_LOAD_GLOBAL_MODULE", + [_LOAD_GLOBAL_BUILTINS] = "_LOAD_GLOBAL_BUILTINS", [_GUARD_TYPE_VERSION] = "_GUARD_TYPE_VERSION", [_CHECK_MANAGED_OBJECT_HAS_VALUES] = "_CHECK_MANAGED_OBJECT_HAS_VALUES", + [_LOAD_ATTR_INSTANCE_VALUE] = "_LOAD_ATTR_INSTANCE_VALUE", [IS_NONE] = "IS_NONE", [_ITER_CHECK_LIST] = "_ITER_CHECK_LIST", [_IS_ITER_EXHAUSTED_LIST] = "_IS_ITER_EXHAUSTED_LIST", diff --git a/Include/internal/pycore_uops.h b/Include/internal/pycore_uops.h index 5ed275fb85767..edb141cc79f75 100644 --- a/Include/internal/pycore_uops.h +++ b/Include/internal/pycore_uops.h @@ -11,8 +11,9 @@ extern "C" { #define _Py_UOP_MAX_TRACE_LENGTH 32 typedef struct { - int opcode; - uint64_t operand; // Sometimes oparg, sometimes a cache entry + uint32_t opcode; + uint32_t oparg; + uint64_t operand; // A cache entry } _PyUOpInstruction; typedef struct { diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index c0dcff825758a..4e519fa73c50c 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2448,7 +2448,7 @@ def testfunc(x): ex = get_first_executor(testfunc) self.assertIsNotNone(ex) - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("SAVE_IP", uops) self.assertIn("LOAD_FAST", uops) @@ -2493,7 +2493,7 @@ def many_vars(): ex = get_first_executor(many_vars) self.assertIsNotNone(ex) - self.assertIn(("LOAD_FAST", 259), list(ex)) + self.assertIn(("LOAD_FAST", 259, 0), list(ex)) def test_unspecialized_unpack(self): # An example of an unspecialized opcode @@ -2514,7 +2514,7 @@ def testfunc(x): ex = get_first_executor(testfunc) self.assertIsNotNone(ex) - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("UNPACK_SEQUENCE", uops) def test_pop_jump_if_false(self): @@ -2529,7 +2529,7 @@ def testfunc(n): ex = get_first_executor(testfunc) self.assertIsNotNone(ex) - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("_POP_JUMP_IF_FALSE", uops) def test_pop_jump_if_none(self): @@ -2544,7 +2544,7 @@ def testfunc(a): ex = get_first_executor(testfunc) self.assertIsNotNone(ex) - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("_POP_JUMP_IF_TRUE", uops) def test_pop_jump_if_not_none(self): @@ -2559,7 +2559,7 @@ def testfunc(a): ex = get_first_executor(testfunc) self.assertIsNotNone(ex) - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("_POP_JUMP_IF_FALSE", uops) def test_pop_jump_if_true(self): @@ -2574,7 +2574,7 @@ def testfunc(n): ex = get_first_executor(testfunc) self.assertIsNotNone(ex) - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("_POP_JUMP_IF_TRUE", uops) def test_jump_backward(self): @@ -2589,7 +2589,7 @@ def testfunc(n): ex = get_first_executor(testfunc) self.assertIsNotNone(ex) - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("JUMP_TO_TOP", uops) def test_jump_forward(self): @@ -2609,7 +2609,7 @@ def testfunc(n): ex = get_first_executor(testfunc) self.assertIsNotNone(ex) - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} # Since there is no JUMP_FORWARD instruction, # look for indirect evidence: the += operator self.assertIn("_BINARY_OP_ADD_INT", uops) @@ -2630,7 +2630,7 @@ def testfunc(n): self.assertIsNotNone(ex) # for i, (opname, oparg) in enumerate(ex): # print(f"{i:4d}: {opname:<20s} {oparg:3d}") - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("_IS_ITER_EXHAUSTED_RANGE", uops) # Verification that the jump goes past END_FOR # is done by manual inspection of the output @@ -2652,7 +2652,7 @@ def testfunc(a): self.assertIsNotNone(ex) # for i, (opname, oparg) in enumerate(ex): # print(f"{i:4d}: {opname:<20s} {oparg:3d}") - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("_IS_ITER_EXHAUSTED_LIST", uops) # Verification that the jump goes past END_FOR # is done by manual inspection of the output @@ -2674,7 +2674,7 @@ def testfunc(a): self.assertIsNotNone(ex) # for i, (opname, oparg) in enumerate(ex): # print(f"{i:4d}: {opname:<20s} {oparg:3d}") - uops = {opname for opname, _ in ex} + uops = {opname for opname, _, _ in ex} self.assertIn("_IS_ITER_EXHAUSTED_TUPLE", uops) # Verification that the jump goes past END_FOR # is done by manual inspection of the output diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 652372cb23dc5..19fb138ee64cb 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -645,18 +645,16 @@ dummy_func( STORE_SUBSCR_LIST_INT, }; - inst(STORE_SUBSCR, (counter/1, v, container, sub -- )) { + inst(STORE_SUBSCR, (unused/1, v, container, sub -- )) { #if ENABLE_SPECIALIZATION - if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { + _PyStoreSubscrCache *cache = (_PyStoreSubscrCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { next_instr--; _Py_Specialize_StoreSubscr(container, sub, next_instr); DISPATCH_SAME_OPARG(); } STAT_INC(STORE_SUBSCR, deferred); - _PyStoreSubscrCache *cache = (_PyStoreSubscrCache *)next_instr; DECREMENT_ADAPTIVE_COUNTER(cache->counter); - #else - (void)counter; // Unused. #endif /* ENABLE_SPECIALIZATION */ /* container[sub] = v */ int err = PyObject_SetItem(container, sub, v); @@ -1198,19 +1196,17 @@ dummy_func( STORE_ATTR_WITH_HINT, }; - inst(STORE_ATTR, (counter/1, unused/3, v, owner --)) { + inst(STORE_ATTR, (unused/1, unused/3, v, owner --)) { #if ENABLE_SPECIALIZATION - if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { + _PyAttrCache *cache = (_PyAttrCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); next_instr--; _Py_Specialize_StoreAttr(owner, next_instr, name); DISPATCH_SAME_OPARG(); } STAT_INC(STORE_ATTR, deferred); - _PyAttrCache *cache = (_PyAttrCache *)next_instr; DECREMENT_ADAPTIVE_COUNTER(cache->counter); - #else - (void)counter; // Unused. #endif /* ENABLE_SPECIALIZATION */ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_SetAttr(owner, name, v); diff --git a/Python/ceval.c b/Python/ceval.c index f13ba9883d981..b56ddfb4bd286 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2747,17 +2747,18 @@ _PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject _Py_CODEUNIT *ip_offset = (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive; int pc = 0; int opcode; - uint64_t operand; int oparg; + uint64_t operand; for (;;) { opcode = self->trace[pc].opcode; + oparg = self->trace[pc].oparg; operand = self->trace[pc].operand; - oparg = (int)operand; DPRINTF(3, - "%4d: uop %s, operand %" PRIu64 ", stack_level %d\n", + "%4d: uop %s, oparg %d, operand %" PRIu64 ", stack_level %d\n", pc, opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode], + oparg, operand, (int)(stack_pointer - _PyFrame_Stackbase(frame))); pc++; diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index d85e23b5abb8e..f492c1fa9d8e3 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -485,18 +485,15 @@ PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; PyObject *v = stack_pointer[-3]; - uint16_t counter = (uint16_t)operand; #if ENABLE_SPECIALIZATION - if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { + _PyStoreSubscrCache *cache = (_PyStoreSubscrCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { next_instr--; _Py_Specialize_StoreSubscr(container, sub, next_instr); DISPATCH_SAME_OPARG(); } STAT_INC(STORE_SUBSCR, deferred); - _PyStoreSubscrCache *cache = (_PyStoreSubscrCache *)next_instr; DECREMENT_ADAPTIVE_COUNTER(cache->counter); - #else - (void)counter; // Unused. #endif /* ENABLE_SPECIALIZATION */ /* container[sub] = v */ int err = PyObject_SetItem(container, sub, v); @@ -849,6 +846,30 @@ break; } + case STORE_ATTR: { + static_assert(INLINE_CACHE_ENTRIES_STORE_ATTR == 4, "incorrect cache size"); + PyObject *owner = stack_pointer[-1]; + PyObject *v = stack_pointer[-2]; + #if ENABLE_SPECIALIZATION + _PyAttrCache *cache = (_PyAttrCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { + PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); + next_instr--; + _Py_Specialize_StoreAttr(owner, next_instr, name); + DISPATCH_SAME_OPARG(); + } + STAT_INC(STORE_ATTR, deferred); + DECREMENT_ADAPTIVE_COUNTER(cache->counter); + #endif /* ENABLE_SPECIALIZATION */ + PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); + int err = PyObject_SetAttr(owner, name, v); + Py_DECREF(v); + Py_DECREF(owner); + if (err) goto pop_2_error; + STACK_SHRINK(2); + break; + } + case DELETE_ATTR: { PyObject *owner = stack_pointer[-1]; PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); @@ -1010,6 +1031,42 @@ break; } + case _LOAD_GLOBAL_MODULE: { + PyObject *null = NULL; + PyObject *res; + uint16_t index = (uint16_t)operand; + PyDictObject *dict = (PyDictObject *)GLOBALS(); + PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(dict->ma_keys); + res = entries[index].me_value; + DEOPT_IF(res == NULL, LOAD_GLOBAL); + Py_INCREF(res); + STAT_INC(LOAD_GLOBAL, hit); + null = NULL; + STACK_GROW(1); + STACK_GROW(((oparg & 1) ? 1 : 0)); + stack_pointer[-1] = res; + if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = null; } + break; + } + + case _LOAD_GLOBAL_BUILTINS: { + PyObject *null = NULL; + PyObject *res; + uint16_t index = (uint16_t)operand; + PyDictObject *bdict = (PyDictObject *)BUILTINS(); + PyDictUnicodeEntry *entries = DK_UNICODE_ENTRIES(bdict->ma_keys); + res = entries[index].me_value; + DEOPT_IF(res == NULL, LOAD_GLOBAL); + Py_INCREF(res); + STAT_INC(LOAD_GLOBAL, hit); + null = NULL; + STACK_GROW(1); + STACK_GROW(((oparg & 1) ? 1 : 0)); + stack_pointer[-1] = res; + if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = null; } + break; + } + case DELETE_FAST: { PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; @@ -1443,6 +1500,24 @@ break; } + case _LOAD_ATTR_INSTANCE_VALUE: { + PyObject *owner = stack_pointer[-1]; + PyObject *res2 = NULL; + PyObject *res; + uint16_t index = (uint16_t)operand; + PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(owner); + res = _PyDictOrValues_GetValues(dorv)->values[index]; + DEOPT_IF(res == NULL, LOAD_ATTR); + STAT_INC(LOAD_ATTR, hit); + Py_INCREF(res); + res2 = NULL; + Py_DECREF(owner); + STACK_GROW(((oparg & 1) ? 1 : 0)); + stack_pointer[-1] = res; + if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } + break; + } + case COMPARE_OP: { static_assert(INLINE_CACHE_ENTRIES_COMPARE_OP == 1, "incorrect cache size"); PyObject *right = stack_pointer[-1]; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 1fd76715dc3e4..0148078d18bdc 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -773,18 +773,15 @@ PyObject *sub = stack_pointer[-1]; PyObject *container = stack_pointer[-2]; PyObject *v = stack_pointer[-3]; - uint16_t counter = read_u16(&next_instr[0].cache); #if ENABLE_SPECIALIZATION - if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { + _PyStoreSubscrCache *cache = (_PyStoreSubscrCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { next_instr--; _Py_Specialize_StoreSubscr(container, sub, next_instr); DISPATCH_SAME_OPARG(); } STAT_INC(STORE_SUBSCR, deferred); - _PyStoreSubscrCache *cache = (_PyStoreSubscrCache *)next_instr; DECREMENT_ADAPTIVE_COUNTER(cache->counter); - #else - (void)counter; // Unused. #endif /* ENABLE_SPECIALIZATION */ /* container[sub] = v */ int err = PyObject_SetItem(container, sub, v); @@ -1437,19 +1434,16 @@ static_assert(INLINE_CACHE_ENTRIES_STORE_ATTR == 4, "incorrect cache size"); PyObject *owner = stack_pointer[-1]; PyObject *v = stack_pointer[-2]; - uint16_t counter = read_u16(&next_instr[0].cache); #if ENABLE_SPECIALIZATION - if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { + _PyAttrCache *cache = (_PyAttrCache *)next_instr; + if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); next_instr--; _Py_Specialize_StoreAttr(owner, next_instr, name); DISPATCH_SAME_OPARG(); } STAT_INC(STORE_ATTR, deferred); - _PyAttrCache *cache = (_PyAttrCache *)next_instr; DECREMENT_ADAPTIVE_COUNTER(cache->counter); - #else - (void)counter; // Unused. #endif /* ENABLE_SPECIALIZATION */ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg); int err = PyObject_SetAttr(owner, name, v); diff --git a/Python/optimizer.c b/Python/optimizer.c index 693ba375971ae..3d385a1506cba 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -344,13 +344,19 @@ uop_item(_PyUOpExecutorObject *self, Py_ssize_t index) if (oname == NULL) { return NULL; } + PyObject *oparg = PyLong_FromUnsignedLong(self->trace[index].oparg); + if (oparg == NULL) { + Py_DECREF(oname); + return NULL; + } PyObject *operand = PyLong_FromUnsignedLongLong(self->trace[index].operand); if (operand == NULL) { + Py_DECREF(oparg); Py_DECREF(oname); return NULL; } - PyObject *args[2] = { oname, operand }; - return _PyTuple_FromArraySteal(args, 2); + PyObject *args[3] = { oname, oparg, operand }; + return _PyTuple_FromArraySteal(args, 3); } PySequenceMethods uop_as_sequence = { @@ -395,29 +401,33 @@ translate_bytecode_to_trace( #define DPRINTF(level, ...) #endif -#define ADD_TO_TRACE(OPCODE, OPERAND) \ +#define ADD_TO_TRACE(OPCODE, OPARG, OPERAND) \ DPRINTF(2, \ - " ADD_TO_TRACE(%s, %" PRIu64 ")\n", \ + " ADD_TO_TRACE(%s, %d, %" PRIu64 ")\n", \ uop_name(OPCODE), \ + (OPARG), \ (uint64_t)(OPERAND)); \ assert(trace_length < max_length); \ assert(reserved > 0); \ reserved--; \ trace[trace_length].opcode = (OPCODE); \ + trace[trace_length].oparg = (OPARG); \ trace[trace_length].operand = (OPERAND); \ trace_length++; #define INSTR_IP(INSTR, CODE) \ - ((long)((INSTR) - ((_Py_CODEUNIT *)(CODE)->co_code_adaptive))) + ((uint32_t)((INSTR) - ((_Py_CODEUNIT *)(CODE)->co_code_adaptive))) -#define ADD_TO_STUB(INDEX, OPCODE, OPERAND) \ - DPRINTF(2, " ADD_TO_STUB(%d, %s, %" PRIu64 ")\n", \ +#define ADD_TO_STUB(INDEX, OPCODE, OPARG, OPERAND) \ + DPRINTF(2, " ADD_TO_STUB(%d, %s, %d, %" PRIu64 ")\n", \ (INDEX), \ uop_name(OPCODE), \ + (OPARG), \ (uint64_t)(OPERAND)); \ assert(reserved > 0); \ reserved--; \ trace[(INDEX)].opcode = (OPCODE); \ + trace[(INDEX)].oparg = (OPARG); \ trace[(INDEX)].operand = (OPERAND); // Reserve space for n uops @@ -433,7 +443,7 @@ translate_bytecode_to_trace( #define RESERVE(main, stub) RESERVE_RAW((main) + (stub) + 2, uop_name(opcode)) DPRINTF(4, - "Optimizing %s (%s:%d) at byte offset %ld\n", + "Optimizing %s (%s:%d) at byte offset %d\n", PyUnicode_AsUTF8(code->co_qualname), PyUnicode_AsUTF8(code->co_filename), code->co_firstlineno, @@ -441,11 +451,11 @@ translate_bytecode_to_trace( for (;;) { RESERVE_RAW(2, "epilogue"); // Always need space for SAVE_IP and EXIT_TRACE - ADD_TO_TRACE(SAVE_IP, INSTR_IP(instr, code)); + ADD_TO_TRACE(SAVE_IP, INSTR_IP(instr, code), 0); - int opcode = instr->op.code; - int oparg = instr->op.arg; - int extras = 0; + uint32_t opcode = instr->op.code; + uint32_t oparg = instr->op.arg; + uint32_t extras = 0; while (opcode == EXTENDED_ARG) { instr++; @@ -467,7 +477,7 @@ translate_bytecode_to_trace( case POP_JUMP_IF_NONE: { RESERVE(2, 2); - ADD_TO_TRACE(IS_NONE, 0); + ADD_TO_TRACE(IS_NONE, 0, 0); opcode = POP_JUMP_IF_TRUE; goto pop_jump_if_bool; } @@ -475,7 +485,7 @@ translate_bytecode_to_trace( case POP_JUMP_IF_NOT_NONE: { RESERVE(2, 2); - ADD_TO_TRACE(IS_NONE, 0); + ADD_TO_TRACE(IS_NONE, 0, 0); opcode = POP_JUMP_IF_FALSE; goto pop_jump_if_bool; } @@ -489,11 +499,11 @@ translate_bytecode_to_trace( _Py_CODEUNIT *target_instr = instr + 1 + _PyOpcode_Caches[_PyOpcode_Deopt[opcode]] + oparg; max_length -= 2; // Really the start of the stubs - int uopcode = opcode == POP_JUMP_IF_TRUE ? + uint32_t uopcode = opcode == POP_JUMP_IF_TRUE ? _POP_JUMP_IF_TRUE : _POP_JUMP_IF_FALSE; - ADD_TO_TRACE(uopcode, max_length); - ADD_TO_STUB(max_length, SAVE_IP, INSTR_IP(target_instr, code)); - ADD_TO_STUB(max_length + 1, EXIT_TRACE, 0); + ADD_TO_TRACE(uopcode, max_length, 0); + ADD_TO_STUB(max_length, SAVE_IP, INSTR_IP(target_instr, code), 0); + ADD_TO_STUB(max_length + 1, EXIT_TRACE, 0, 0); break; } @@ -501,7 +511,7 @@ translate_bytecode_to_trace( { if (instr + 2 - oparg == initial_instr) { RESERVE(1, 0); - ADD_TO_TRACE(JUMP_TO_TOP, 0); + ADD_TO_TRACE(JUMP_TO_TOP, 0, 0); } else { DPRINTF(2, "JUMP_BACKWARD not to top ends trace\n"); @@ -546,14 +556,14 @@ translate_bytecode_to_trace( _Py_CODEUNIT *target_instr = // +1 at the end skips over END_FOR instr + 1 + _PyOpcode_Caches[_PyOpcode_Deopt[opcode]] + oparg + 1; max_length -= 3; // Really the start of the stubs - ADD_TO_TRACE(check_op, 0); - ADD_TO_TRACE(exhausted_op, 0); - ADD_TO_TRACE(_POP_JUMP_IF_TRUE, max_length); - ADD_TO_TRACE(next_op, 0); - - ADD_TO_STUB(max_length + 0, POP_TOP, 0); - ADD_TO_STUB(max_length + 1, SAVE_IP, INSTR_IP(target_instr, code)); - ADD_TO_STUB(max_length + 2, EXIT_TRACE, 0); + ADD_TO_TRACE(check_op, 0, 0); + ADD_TO_TRACE(exhausted_op, 0, 0); + ADD_TO_TRACE(_POP_JUMP_IF_TRUE, max_length, 0); + ADD_TO_TRACE(next_op, 0, 0); + + ADD_TO_STUB(max_length + 0, POP_TOP, 0, 0); + ADD_TO_STUB(max_length + 1, SAVE_IP, INSTR_IP(target_instr, code), 0); + ADD_TO_STUB(max_length + 2, EXIT_TRACE, 0, 0); break; } @@ -564,19 +574,20 @@ translate_bytecode_to_trace( // Reserve space for nuops (+ SAVE_IP + EXIT_TRACE) int nuops = expansion->nuops; RESERVE(nuops, 0); + uint32_t orig_oparg = oparg; // For OPARG_TOP/BOTTOM for (int i = 0; i < nuops; i++) { - uint64_t operand; + oparg = orig_oparg; + uint64_t operand = 0; int offset = expansion->uops[i].offset; switch (expansion->uops[i].size) { case OPARG_FULL: - operand = oparg; if (extras && OPCODE_HAS_JUMP(opcode)) { if (opcode == JUMP_BACKWARD_NO_INTERRUPT) { - operand -= extras; + oparg -= extras; } else { assert(opcode != JUMP_BACKWARD); - operand += extras; + oparg += extras; } } break; @@ -590,10 +601,10 @@ translate_bytecode_to_trace( operand = read_u64(&instr[offset].cache); break; case OPARG_TOP: // First half of super-instr - operand = oparg >> 4; + oparg = orig_oparg >> 4; break; case OPARG_BOTTOM: // Second half of super-instr - operand = oparg & 0xF; + oparg = orig_oparg & 0xF; break; default: fprintf(stderr, @@ -603,7 +614,7 @@ translate_bytecode_to_trace( expansion->uops[i].offset); Py_FatalError("garbled expansion"); } - ADD_TO_TRACE(expansion->uops[i].uop, operand); + ADD_TO_TRACE(expansion->uops[i].uop, oparg, operand); } break; } @@ -621,9 +632,9 @@ translate_bytecode_to_trace( done: // Skip short traces like SAVE_IP, LOAD_FAST, SAVE_IP, EXIT_TRACE if (trace_length > 3) { - ADD_TO_TRACE(EXIT_TRACE, 0); + ADD_TO_TRACE(EXIT_TRACE, 0, 0); DPRINTF(1, - "Created a trace for %s (%s:%d) at byte offset %ld -- length %d\n", + "Created a trace for %s (%s:%d) at byte offset %d -- length %d\n", PyUnicode_AsUTF8(code->co_qualname), PyUnicode_AsUTF8(code->co_filename), code->co_firstlineno, @@ -644,10 +655,10 @@ translate_bytecode_to_trace( if (trace[i].opcode == _POP_JUMP_IF_FALSE || trace[i].opcode == _POP_JUMP_IF_TRUE) { - uint64_t target = trace[i].operand; - if (target >= (uint64_t)max_length) { + int target = trace[i].oparg; + if (target >= max_length) { target += trace_length - max_length; - trace[i].operand = target; + trace[i].oparg = target; } } } @@ -657,7 +668,7 @@ translate_bytecode_to_trace( } else { DPRINTF(4, - "No trace for %s (%s:%d) at byte offset %ld\n", + "No trace for %s (%s:%d) at byte offset %d\n", PyUnicode_AsUTF8(code->co_qualname), PyUnicode_AsUTF8(code->co_filename), code->co_firstlineno, diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 112f29a83e4c1..037bee107cb13 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -417,16 +417,9 @@ def is_viable_uop(self) -> bool: if self.always_exits: dprint(f"Skipping {self.name} because it always exits") return False - if self.instr_flags.HAS_ARG_FLAG: - # If the instruction uses oparg, it cannot use any caches - if self.active_caches: - dprint(f"Skipping {self.name} because it uses oparg and caches") - return False - else: - # If it doesn't use oparg, it can have one cache entry - if len(self.active_caches) > 1: - dprint(f"Skipping {self.name} because it has >1 cache entries") - return False + if len(self.active_caches) > 1: + # print(f"Skipping {self.name} because it has >1 cache entries") + return False res = True for forbidden in FORBIDDEN_NAMES_IN_UOPS: # NOTE: To disallow unspecialized uops, use @@ -1374,7 +1367,7 @@ def write_macro_expansions(self, name: str, parts: MacroParts) -> None: if not part.instr.is_viable_uop(): print(f"NOTE: Part {part.instr.name} of {name} is not a viable uop") return - if part.instr.instr_flags.HAS_ARG_FLAG or not part.active_caches: + if not part.active_caches: size, offset = OPARG_SIZES["OPARG_FULL"], 0 else: # If this assert triggers, is_viable_uops() lied From webhook-mailer at python.org Mon Jul 17 15:55:44 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 17 Jul 2023 19:55:44 -0000 Subject: [Python-checkins] gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (#106832) Message-ID: https://github.com/python/cpython/commit/ebf2c56b33553a448da8f60fcd89a622f071b5f4 commit: ebf2c56b33553a448da8f60fcd89a622f071b5f4 branch: main author: Nikita Sobolev committer: vstinner date: 2023-07-17T19:55:40Z summary: gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (#106832) files: A Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst b/Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst new file mode 100644 index 0000000000000..d3b9862684539 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst @@ -0,0 +1,2 @@ +Fix potential missing ``NULL`` check of ``d2i_SSL_SESSION`` result in +``_ssl.c``. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 0cf4d3e9dc8c9..8612b3dd53924 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2808,7 +2808,7 @@ _ssl_session_dup(SSL_SESSION *session) { /* get length */ slen = i2d_SSL_SESSION(session, NULL); if (slen == 0 || slen > 0xFF00) { - PyErr_SetString(PyExc_ValueError, "i2d() failed."); + PyErr_SetString(PyExc_ValueError, "i2d() failed"); goto error; } if ((senc = PyMem_Malloc(slen)) == NULL) { @@ -2817,12 +2817,13 @@ _ssl_session_dup(SSL_SESSION *session) { } p = senc; if (!i2d_SSL_SESSION(session, &p)) { - PyErr_SetString(PyExc_ValueError, "i2d() failed."); + PyErr_SetString(PyExc_ValueError, "i2d() failed"); goto error; } const_p = senc; newsession = d2i_SSL_SESSION(NULL, &const_p, slen); - if (session == NULL) { + if (newsession == NULL) { + PyErr_SetString(PyExc_ValueError, "d2i() failed"); goto error; } PyMem_Free(senc); From webhook-mailer at python.org Mon Jul 17 16:31:14 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 17 Jul 2023 20:31:14 -0000 Subject: [Python-checkins] [3.11] gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (GH-106832) (#106836) Message-ID: https://github.com/python/cpython/commit/a782d51913222e340b75b2dd50e646be80dd035b commit: a782d51913222e340b75b2dd50e646be80dd035b branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-17T20:31:10Z summary: [3.11] gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (GH-106832) (#106836) gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (GH-106832) (cherry picked from commit ebf2c56b33553a448da8f60fcd89a622f071b5f4) Co-authored-by: Nikita Sobolev files: A Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst b/Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst new file mode 100644 index 0000000000000..d3b9862684539 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst @@ -0,0 +1,2 @@ +Fix potential missing ``NULL`` check of ``d2i_SSL_SESSION`` result in +``_ssl.c``. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index c1a8eaaa2fa0a..e3bb38e769c9b 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2800,7 +2800,7 @@ _ssl_session_dup(SSL_SESSION *session) { /* get length */ slen = i2d_SSL_SESSION(session, NULL); if (slen == 0 || slen > 0xFF00) { - PyErr_SetString(PyExc_ValueError, "i2d() failed."); + PyErr_SetString(PyExc_ValueError, "i2d() failed"); goto error; } if ((senc = PyMem_Malloc(slen)) == NULL) { @@ -2809,12 +2809,13 @@ _ssl_session_dup(SSL_SESSION *session) { } p = senc; if (!i2d_SSL_SESSION(session, &p)) { - PyErr_SetString(PyExc_ValueError, "i2d() failed."); + PyErr_SetString(PyExc_ValueError, "i2d() failed"); goto error; } const_p = senc; newsession = d2i_SSL_SESSION(NULL, &const_p, slen); - if (session == NULL) { + if (newsession == NULL) { + PyErr_SetString(PyExc_ValueError, "d2i() failed"); goto error; } PyMem_Free(senc); From webhook-mailer at python.org Mon Jul 17 16:40:18 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 17 Jul 2023 20:40:18 -0000 Subject: [Python-checkins] [3.12] gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (GH-106832) (#106835) Message-ID: https://github.com/python/cpython/commit/2eef81e05ece14796e8e922ecac8e572a6e6d5b0 commit: 2eef81e05ece14796e8e922ecac8e572a6e6d5b0 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-17T20:40:15Z summary: [3.12] gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (GH-106832) (#106835) gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (GH-106832) (cherry picked from commit ebf2c56b33553a448da8f60fcd89a622f071b5f4) Co-authored-by: Nikita Sobolev files: A Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst b/Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst new file mode 100644 index 0000000000000..d3b9862684539 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-17-21-45-15.gh-issue-106831.RqVq9X.rst @@ -0,0 +1,2 @@ +Fix potential missing ``NULL`` check of ``d2i_SSL_SESSION`` result in +``_ssl.c``. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 5f90584a99c08..a3fb12e483750 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2779,7 +2779,7 @@ _ssl_session_dup(SSL_SESSION *session) { /* get length */ slen = i2d_SSL_SESSION(session, NULL); if (slen == 0 || slen > 0xFF00) { - PyErr_SetString(PyExc_ValueError, "i2d() failed."); + PyErr_SetString(PyExc_ValueError, "i2d() failed"); goto error; } if ((senc = PyMem_Malloc(slen)) == NULL) { @@ -2788,12 +2788,13 @@ _ssl_session_dup(SSL_SESSION *session) { } p = senc; if (!i2d_SSL_SESSION(session, &p)) { - PyErr_SetString(PyExc_ValueError, "i2d() failed."); + PyErr_SetString(PyExc_ValueError, "i2d() failed"); goto error; } const_p = senc; newsession = d2i_SSL_SESSION(NULL, &const_p, slen); - if (session == NULL) { + if (newsession == NULL) { + PyErr_SetString(PyExc_ValueError, "d2i() failed"); goto error; } PyMem_Free(senc); From webhook-mailer at python.org Mon Jul 17 16:55:13 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 17 Jul 2023 20:55:13 -0000 Subject: [Python-checkins] gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (#106833) Message-ID: https://github.com/python/cpython/commit/22379c60ab8f8b49e75da9bd032a8722af50b409 commit: 22379c60ab8f8b49e75da9bd032a8722af50b409 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-17T20:55:10Z summary: gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (#106833) files: M Lib/test/clinic.test.c M Lib/test/test_clinic.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index 2fd8760415dc7..da99e58c77f02 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -3726,6 +3726,47 @@ test_preprocessor_guarded_else_impl(PyObject *module) /*[clinic end generated code: output=13af7670aac51b12 input=6657ab31d74c29fc]*/ #endif +#ifndef CONDITION_C +/*[clinic input] +test_preprocessor_guarded_ifndef_condition_c +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_ifndef_condition_c_impl(PyObject *module) +/*[clinic end generated code: output=ed422e8c895bb0a5 input=e9b50491cea2b668]*/ +#else +/*[clinic input] +test_preprocessor_guarded_ifndef_not_condition_c +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_ifndef_not_condition_c_impl(PyObject *module) +/*[clinic end generated code: output=de6f4c6a67f8c536 input=da74e30e01c6f2c5]*/ +#endif + +#if \ +CONDITION_D +/*[clinic input] +test_preprocessor_guarded_if_with_continuation +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_if_with_continuation_impl(PyObject *module) +/*[clinic end generated code: output=3d0712ca9e2d15b9 input=4a956fd91be30284]*/ +#endif + +#if CONDITION_E ||?CONDITION_F +#warning "different type of CPP directive" +/*[clinic input] +test_preprocessor_guarded_if_e_or_f +Makes sure cpp.Monitor handles other directives than preprocessor conditionals. +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_if_e_or_f_impl(PyObject *module) +/*[clinic end generated code: output=e49d24ff64ad88bc input=57b9c37f938bc4f1]*/ +#endif + /*[clinic input] dump buffer output pop @@ -3785,6 +3826,79 @@ test_preprocessor_guarded_else(PyObject *module, PyObject *Py_UNUSED(ignored)) #endif /* !defined(CONDITION_A) && !(CONDITION_B) */ +#if !defined(CONDITION_C) + +PyDoc_STRVAR(test_preprocessor_guarded_ifndef_condition_c__doc__, +"test_preprocessor_guarded_ifndef_condition_c($module, /)\n" +"--\n" +"\n"); + +#define TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF \ + {"test_preprocessor_guarded_ifndef_condition_c", (PyCFunction)test_preprocessor_guarded_ifndef_condition_c, METH_NOARGS, test_preprocessor_guarded_ifndef_condition_c__doc__}, + +static PyObject * +test_preprocessor_guarded_ifndef_condition_c(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_ifndef_condition_c_impl(module); +} + +#endif /* !defined(CONDITION_C) */ + +#if defined(CONDITION_C) + +PyDoc_STRVAR(test_preprocessor_guarded_ifndef_not_condition_c__doc__, +"test_preprocessor_guarded_ifndef_not_condition_c($module, /)\n" +"--\n" +"\n"); + +#define TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF \ + {"test_preprocessor_guarded_ifndef_not_condition_c", (PyCFunction)test_preprocessor_guarded_ifndef_not_condition_c, METH_NOARGS, test_preprocessor_guarded_ifndef_not_condition_c__doc__}, + +static PyObject * +test_preprocessor_guarded_ifndef_not_condition_c(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_ifndef_not_condition_c_impl(module); +} + +#endif /* defined(CONDITION_C) */ + +#if (CONDITION_D) + +PyDoc_STRVAR(test_preprocessor_guarded_if_with_continuation__doc__, +"test_preprocessor_guarded_if_with_continuation($module, /)\n" +"--\n" +"\n"); + +#define TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF \ + {"test_preprocessor_guarded_if_with_continuation", (PyCFunction)test_preprocessor_guarded_if_with_continuation, METH_NOARGS, test_preprocessor_guarded_if_with_continuation__doc__}, + +static PyObject * +test_preprocessor_guarded_if_with_continuation(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_if_with_continuation_impl(module); +} + +#endif /* (CONDITION_D) */ + +#if (CONDITION_E || CONDITION_F) + +PyDoc_STRVAR(test_preprocessor_guarded_if_e_or_f__doc__, +"test_preprocessor_guarded_if_e_or_f($module, /)\n" +"--\n" +"\n" +"Makes sure cpp.Monitor handles other directives than preprocessor conditionals."); + +#define TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF \ + {"test_preprocessor_guarded_if_e_or_f", (PyCFunction)test_preprocessor_guarded_if_e_or_f, METH_NOARGS, test_preprocessor_guarded_if_e_or_f__doc__}, + +static PyObject * +test_preprocessor_guarded_if_e_or_f(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_if_e_or_f_impl(module); +} + +#endif /* (CONDITION_E || CONDITION_F) */ + #ifndef TEST_PREPROCESSOR_GUARDED_CONDITION_A_METHODDEF #define TEST_PREPROCESSOR_GUARDED_CONDITION_A_METHODDEF #endif /* !defined(TEST_PREPROCESSOR_GUARDED_CONDITION_A_METHODDEF) */ @@ -3796,7 +3910,23 @@ test_preprocessor_guarded_else(PyObject *module, PyObject *Py_UNUSED(ignored)) #ifndef TEST_PREPROCESSOR_GUARDED_ELSE_METHODDEF #define TEST_PREPROCESSOR_GUARDED_ELSE_METHODDEF #endif /* !defined(TEST_PREPROCESSOR_GUARDED_ELSE_METHODDEF) */ -/*[clinic end generated code: output=3804bb18d454038c input=3fc80c9989d2f2e1]*/ + +#ifndef TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF) */ + +#ifndef TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF) */ + +#ifndef TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF) */ + +#ifndef TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF) */ +/*[clinic end generated code: output=fcfae7cac7a99e62 input=3fc80c9989d2f2e1]*/ /*[clinic input] test_vararg_and_posonly diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index b5744f7013d6a..e925ecca1b9c5 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -200,6 +200,55 @@ def test_parse_with_body_prefix(self): """).lstrip() # Note, lstrip() because of the newline self.assertEqual(out, expected) + def test_cpp_monitor_fail_nested_block_comment(self): + raw = """ + /* start + /* nested + */ + */ + """ + msg = ( + 'Error in file "test.c" on line 2:\n' + 'Nested block comment!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_cpp_monitor_fail_invalid_format_noarg(self): + raw = """ + #if + a() + #endif + """ + msg = ( + 'Error in file "test.c" on line 1:\n' + 'Invalid format for #if line: no argument!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_cpp_monitor_fail_invalid_format_toomanyargs(self): + raw = """ + #ifdef A B + a() + #endif + """ + msg = ( + 'Error in file "test.c" on line 1:\n' + 'Invalid format for #ifdef line: should be exactly one argument!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_cpp_monitor_fail_no_matching_if(self): + raw = '#else' + msg = ( + 'Error in file "test.c" on line 1:\n' + '#else without matching #if / #ifdef / #ifndef!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + class ClinicGroupPermuterTest(TestCase): def _test(self, l, m, r, output): From webhook-mailer at python.org Mon Jul 17 17:21:02 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 17 Jul 2023 21:21:02 -0000 Subject: [Python-checkins] [3.12] gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (GH-106833) (#106838) Message-ID: https://github.com/python/cpython/commit/941ac1e19df176b68537f3d952a70f0e52659bf6 commit: 941ac1e19df176b68537f3d952a70f0e52659bf6 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-17T21:20:59Z summary: [3.12] gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (GH-106833) (#106838) (cherry picked from commit 22379c60ab8f8b49e75da9bd032a8722af50b409) Co-authored-by: Erlend E. Aasland files: M Lib/test/clinic.test.c M Lib/test/test_clinic.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index 1544599c29fa0..5748bbc71df5a 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -3732,6 +3732,47 @@ test_preprocessor_guarded_else_impl(PyObject *module) /*[clinic end generated code: output=13af7670aac51b12 input=6657ab31d74c29fc]*/ #endif +#ifndef CONDITION_C +/*[clinic input] +test_preprocessor_guarded_ifndef_condition_c +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_ifndef_condition_c_impl(PyObject *module) +/*[clinic end generated code: output=ed422e8c895bb0a5 input=e9b50491cea2b668]*/ +#else +/*[clinic input] +test_preprocessor_guarded_ifndef_not_condition_c +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_ifndef_not_condition_c_impl(PyObject *module) +/*[clinic end generated code: output=de6f4c6a67f8c536 input=da74e30e01c6f2c5]*/ +#endif + +#if \ +CONDITION_D +/*[clinic input] +test_preprocessor_guarded_if_with_continuation +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_if_with_continuation_impl(PyObject *module) +/*[clinic end generated code: output=3d0712ca9e2d15b9 input=4a956fd91be30284]*/ +#endif + +#if CONDITION_E ||?CONDITION_F +#warning "different type of CPP directive" +/*[clinic input] +test_preprocessor_guarded_if_e_or_f +Makes sure cpp.Monitor handles other directives than preprocessor conditionals. +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_if_e_or_f_impl(PyObject *module) +/*[clinic end generated code: output=e49d24ff64ad88bc input=57b9c37f938bc4f1]*/ +#endif + /*[clinic input] dump buffer output pop @@ -3791,6 +3832,79 @@ test_preprocessor_guarded_else(PyObject *module, PyObject *Py_UNUSED(ignored)) #endif /* !defined(CONDITION_A) && !(CONDITION_B) */ +#if !defined(CONDITION_C) + +PyDoc_STRVAR(test_preprocessor_guarded_ifndef_condition_c__doc__, +"test_preprocessor_guarded_ifndef_condition_c($module, /)\n" +"--\n" +"\n"); + +#define TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF \ + {"test_preprocessor_guarded_ifndef_condition_c", (PyCFunction)test_preprocessor_guarded_ifndef_condition_c, METH_NOARGS, test_preprocessor_guarded_ifndef_condition_c__doc__}, + +static PyObject * +test_preprocessor_guarded_ifndef_condition_c(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_ifndef_condition_c_impl(module); +} + +#endif /* !defined(CONDITION_C) */ + +#if defined(CONDITION_C) + +PyDoc_STRVAR(test_preprocessor_guarded_ifndef_not_condition_c__doc__, +"test_preprocessor_guarded_ifndef_not_condition_c($module, /)\n" +"--\n" +"\n"); + +#define TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF \ + {"test_preprocessor_guarded_ifndef_not_condition_c", (PyCFunction)test_preprocessor_guarded_ifndef_not_condition_c, METH_NOARGS, test_preprocessor_guarded_ifndef_not_condition_c__doc__}, + +static PyObject * +test_preprocessor_guarded_ifndef_not_condition_c(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_ifndef_not_condition_c_impl(module); +} + +#endif /* defined(CONDITION_C) */ + +#if (CONDITION_D) + +PyDoc_STRVAR(test_preprocessor_guarded_if_with_continuation__doc__, +"test_preprocessor_guarded_if_with_continuation($module, /)\n" +"--\n" +"\n"); + +#define TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF \ + {"test_preprocessor_guarded_if_with_continuation", (PyCFunction)test_preprocessor_guarded_if_with_continuation, METH_NOARGS, test_preprocessor_guarded_if_with_continuation__doc__}, + +static PyObject * +test_preprocessor_guarded_if_with_continuation(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_if_with_continuation_impl(module); +} + +#endif /* (CONDITION_D) */ + +#if (CONDITION_E || CONDITION_F) + +PyDoc_STRVAR(test_preprocessor_guarded_if_e_or_f__doc__, +"test_preprocessor_guarded_if_e_or_f($module, /)\n" +"--\n" +"\n" +"Makes sure cpp.Monitor handles other directives than preprocessor conditionals."); + +#define TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF \ + {"test_preprocessor_guarded_if_e_or_f", (PyCFunction)test_preprocessor_guarded_if_e_or_f, METH_NOARGS, test_preprocessor_guarded_if_e_or_f__doc__}, + +static PyObject * +test_preprocessor_guarded_if_e_or_f(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_if_e_or_f_impl(module); +} + +#endif /* (CONDITION_E || CONDITION_F) */ + #ifndef TEST_PREPROCESSOR_GUARDED_CONDITION_A_METHODDEF #define TEST_PREPROCESSOR_GUARDED_CONDITION_A_METHODDEF #endif /* !defined(TEST_PREPROCESSOR_GUARDED_CONDITION_A_METHODDEF) */ @@ -3802,7 +3916,23 @@ test_preprocessor_guarded_else(PyObject *module, PyObject *Py_UNUSED(ignored)) #ifndef TEST_PREPROCESSOR_GUARDED_ELSE_METHODDEF #define TEST_PREPROCESSOR_GUARDED_ELSE_METHODDEF #endif /* !defined(TEST_PREPROCESSOR_GUARDED_ELSE_METHODDEF) */ -/*[clinic end generated code: output=3804bb18d454038c input=3fc80c9989d2f2e1]*/ + +#ifndef TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF) */ + +#ifndef TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF) */ + +#ifndef TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF) */ + +#ifndef TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF) */ +/*[clinic end generated code: output=fcfae7cac7a99e62 input=3fc80c9989d2f2e1]*/ /*[clinic input] test_vararg_and_posonly diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index b5744f7013d6a..e925ecca1b9c5 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -200,6 +200,55 @@ def test_parse_with_body_prefix(self): """).lstrip() # Note, lstrip() because of the newline self.assertEqual(out, expected) + def test_cpp_monitor_fail_nested_block_comment(self): + raw = """ + /* start + /* nested + */ + */ + """ + msg = ( + 'Error in file "test.c" on line 2:\n' + 'Nested block comment!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_cpp_monitor_fail_invalid_format_noarg(self): + raw = """ + #if + a() + #endif + """ + msg = ( + 'Error in file "test.c" on line 1:\n' + 'Invalid format for #if line: no argument!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_cpp_monitor_fail_invalid_format_toomanyargs(self): + raw = """ + #ifdef A B + a() + #endif + """ + msg = ( + 'Error in file "test.c" on line 1:\n' + 'Invalid format for #ifdef line: should be exactly one argument!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_cpp_monitor_fail_no_matching_if(self): + raw = '#else' + msg = ( + 'Error in file "test.c" on line 1:\n' + '#else without matching #if / #ifdef / #ifndef!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + class ClinicGroupPermuterTest(TestCase): def _test(self, l, m, r, output): From webhook-mailer at python.org Mon Jul 17 17:22:46 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 17 Jul 2023 21:22:46 -0000 Subject: [Python-checkins] [3.11] gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (GH-106833) (#106839) Message-ID: https://github.com/python/cpython/commit/a7acc5cb5c5327abeffe107d20b5d609fe0bdfad commit: a7acc5cb5c5327abeffe107d20b5d609fe0bdfad branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-17T21:22:43Z summary: [3.11] gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (GH-106833) (#106839) (cherry picked from commit 22379c60ab8f8b49e75da9bd032a8722af50b409) Co-authored-by: Erlend E. Aasland files: M Lib/test/clinic.test.c M Lib/test/test_clinic.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index 1eb5a3e7cce0d..e4d02f83bc636 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -3278,6 +3278,47 @@ test_preprocessor_guarded_else_impl(PyObject *module) /*[clinic end generated code: output=13af7670aac51b12 input=6657ab31d74c29fc]*/ #endif +#ifndef CONDITION_C +/*[clinic input] +test_preprocessor_guarded_ifndef_condition_c +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_ifndef_condition_c_impl(PyObject *module) +/*[clinic end generated code: output=ed422e8c895bb0a5 input=e9b50491cea2b668]*/ +#else +/*[clinic input] +test_preprocessor_guarded_ifndef_not_condition_c +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_ifndef_not_condition_c_impl(PyObject *module) +/*[clinic end generated code: output=de6f4c6a67f8c536 input=da74e30e01c6f2c5]*/ +#endif + +#if \ +CONDITION_D +/*[clinic input] +test_preprocessor_guarded_if_with_continuation +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_if_with_continuation_impl(PyObject *module) +/*[clinic end generated code: output=3d0712ca9e2d15b9 input=4a956fd91be30284]*/ +#endif + +#if CONDITION_E ||?CONDITION_F +#warning "different type of CPP directive" +/*[clinic input] +test_preprocessor_guarded_if_e_or_f +Makes sure cpp.Monitor handles other directives than preprocessor conditionals. +[clinic start generated code]*/ + +static PyObject * +test_preprocessor_guarded_if_e_or_f_impl(PyObject *module) +/*[clinic end generated code: output=e49d24ff64ad88bc input=57b9c37f938bc4f1]*/ +#endif + /*[clinic input] dump buffer output pop @@ -3337,6 +3378,79 @@ test_preprocessor_guarded_else(PyObject *module, PyObject *Py_UNUSED(ignored)) #endif /* !defined(CONDITION_A) && !(CONDITION_B) */ +#if !defined(CONDITION_C) + +PyDoc_STRVAR(test_preprocessor_guarded_ifndef_condition_c__doc__, +"test_preprocessor_guarded_ifndef_condition_c($module, /)\n" +"--\n" +"\n"); + +#define TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF \ + {"test_preprocessor_guarded_ifndef_condition_c", (PyCFunction)test_preprocessor_guarded_ifndef_condition_c, METH_NOARGS, test_preprocessor_guarded_ifndef_condition_c__doc__}, + +static PyObject * +test_preprocessor_guarded_ifndef_condition_c(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_ifndef_condition_c_impl(module); +} + +#endif /* !defined(CONDITION_C) */ + +#if defined(CONDITION_C) + +PyDoc_STRVAR(test_preprocessor_guarded_ifndef_not_condition_c__doc__, +"test_preprocessor_guarded_ifndef_not_condition_c($module, /)\n" +"--\n" +"\n"); + +#define TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF \ + {"test_preprocessor_guarded_ifndef_not_condition_c", (PyCFunction)test_preprocessor_guarded_ifndef_not_condition_c, METH_NOARGS, test_preprocessor_guarded_ifndef_not_condition_c__doc__}, + +static PyObject * +test_preprocessor_guarded_ifndef_not_condition_c(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_ifndef_not_condition_c_impl(module); +} + +#endif /* defined(CONDITION_C) */ + +#if (CONDITION_D) + +PyDoc_STRVAR(test_preprocessor_guarded_if_with_continuation__doc__, +"test_preprocessor_guarded_if_with_continuation($module, /)\n" +"--\n" +"\n"); + +#define TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF \ + {"test_preprocessor_guarded_if_with_continuation", (PyCFunction)test_preprocessor_guarded_if_with_continuation, METH_NOARGS, test_preprocessor_guarded_if_with_continuation__doc__}, + +static PyObject * +test_preprocessor_guarded_if_with_continuation(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_if_with_continuation_impl(module); +} + +#endif /* (CONDITION_D) */ + +#if (CONDITION_E || CONDITION_F) + +PyDoc_STRVAR(test_preprocessor_guarded_if_e_or_f__doc__, +"test_preprocessor_guarded_if_e_or_f($module, /)\n" +"--\n" +"\n" +"Makes sure cpp.Monitor handles other directives than preprocessor conditionals."); + +#define TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF \ + {"test_preprocessor_guarded_if_e_or_f", (PyCFunction)test_preprocessor_guarded_if_e_or_f, METH_NOARGS, test_preprocessor_guarded_if_e_or_f__doc__}, + +static PyObject * +test_preprocessor_guarded_if_e_or_f(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return test_preprocessor_guarded_if_e_or_f_impl(module); +} + +#endif /* (CONDITION_E || CONDITION_F) */ + #ifndef TEST_PREPROCESSOR_GUARDED_CONDITION_A_METHODDEF #define TEST_PREPROCESSOR_GUARDED_CONDITION_A_METHODDEF #endif /* !defined(TEST_PREPROCESSOR_GUARDED_CONDITION_A_METHODDEF) */ @@ -3348,7 +3462,23 @@ test_preprocessor_guarded_else(PyObject *module, PyObject *Py_UNUSED(ignored)) #ifndef TEST_PREPROCESSOR_GUARDED_ELSE_METHODDEF #define TEST_PREPROCESSOR_GUARDED_ELSE_METHODDEF #endif /* !defined(TEST_PREPROCESSOR_GUARDED_ELSE_METHODDEF) */ -/*[clinic end generated code: output=3804bb18d454038c input=3fc80c9989d2f2e1]*/ + +#ifndef TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IFNDEF_CONDITION_C_METHODDEF) */ + +#ifndef TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IFNDEF_NOT_CONDITION_C_METHODDEF) */ + +#ifndef TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IF_WITH_CONTINUATION_METHODDEF) */ + +#ifndef TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF + #define TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF +#endif /* !defined(TEST_PREPROCESSOR_GUARDED_IF_E_OR_F_METHODDEF) */ +/*[clinic end generated code: output=fcfae7cac7a99e62 input=3fc80c9989d2f2e1]*/ /*[clinic input] test_vararg_and_posonly diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 0e3fbb7627ab5..cef121d9a1ead 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -200,6 +200,55 @@ def test_parse_with_body_prefix(self): """).lstrip() # Note, lstrip() because of the newline self.assertEqual(out, expected) + def test_cpp_monitor_fail_nested_block_comment(self): + raw = """ + /* start + /* nested + */ + */ + """ + msg = ( + 'Error in file "test.c" on line 2:\n' + 'Nested block comment!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_cpp_monitor_fail_invalid_format_noarg(self): + raw = """ + #if + a() + #endif + """ + msg = ( + 'Error in file "test.c" on line 1:\n' + 'Invalid format for #if line: no argument!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_cpp_monitor_fail_invalid_format_toomanyargs(self): + raw = """ + #ifdef A B + a() + #endif + """ + msg = ( + 'Error in file "test.c" on line 1:\n' + 'Invalid format for #ifdef line: should be exactly one argument!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + + def test_cpp_monitor_fail_no_matching_if(self): + raw = '#else' + msg = ( + 'Error in file "test.c" on line 1:\n' + '#else without matching #if / #ifdef / #ifndef!\n' + ) + out = self.expect_failure(raw) + self.assertEqual(out, msg) + class ClinicGroupPermuterTest(TestCase): def _test(self, l, m, r, output): From webhook-mailer at python.org Mon Jul 17 18:10:07 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 17 Jul 2023 22:10:07 -0000 Subject: [Python-checkins] Add Erlend as CODEOWNER for Argument Clinic docs (#106840) Message-ID: https://github.com/python/cpython/commit/1654916c4864a741507617020453147acf20c9f3 commit: 1654916c4864a741507617020453147acf20c9f3 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-17T22:10:03Z summary: Add Erlend as CODEOWNER for Argument Clinic docs (#106840) files: M .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 234a954cc7662..882ba9e9c9ebe 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -177,3 +177,4 @@ Doc/c-api/stable.rst @encukou # Argument Clinic /Tools/clinic/** @erlend-aasland @AlexWaygood /Lib/test/test_clinic.py @erlend-aasland @AlexWaygood +Doc/howto/clinic.rst @erlend-aasland From webhook-mailer at python.org Mon Jul 17 18:31:21 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 17 Jul 2023 22:31:21 -0000 Subject: [Python-checkins] [3.12] gh-101538: Add experimental wasi-threads build (GH-101537) (#106834) Message-ID: https://github.com/python/cpython/commit/e903c16a6c3fe0adb594f93acc56e719590c4790 commit: e903c16a6c3fe0adb594f93acc56e719590c4790 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-18T00:31:17+02:00 summary: [3.12] gh-101538: Add experimental wasi-threads build (GH-101537) (#106834) (cherry picked from commit d8f87cdf94a6533c5cf2d25e09e6fa3eb06720b9) Co-authored-by: YAMAMOTO Takashi Co-authored-by: Brett Cannon Co-authored-by: Erlend E. Aasland files: A Misc/NEWS.d/next/Build/2023-02-03-21-36-42.gh-issue-101538.sF5F6S.rst M Python/thread_pthread.h M Tools/wasm/wasm_build.py M configure M configure.ac diff --git a/Misc/NEWS.d/next/Build/2023-02-03-21-36-42.gh-issue-101538.sF5F6S.rst b/Misc/NEWS.d/next/Build/2023-02-03-21-36-42.gh-issue-101538.sF5F6S.rst new file mode 100644 index 0000000000000..4b83c303b3d2c --- /dev/null +++ b/Misc/NEWS.d/next/Build/2023-02-03-21-36-42.gh-issue-101538.sF5F6S.rst @@ -0,0 +1 @@ +Add experimental wasi-threads support. Patch by Takashi Yamamoto. diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h index 76d6f3bcdf9c4..f96c57da64636 100644 --- a/Python/thread_pthread.h +++ b/Python/thread_pthread.h @@ -356,7 +356,15 @@ PyThread_exit_thread(void) { if (!initialized) exit(0); +#if defined(__wasi__) + /* + * wasi-threads doesn't have pthread_exit right now + * cf. https://github.com/WebAssembly/wasi-threads/issues/7 + */ + abort(); +#else pthread_exit(0); +#endif } #ifdef USE_SEMAPHORES diff --git a/Tools/wasm/wasm_build.py b/Tools/wasm/wasm_build.py index 241a5d4eed5ae..c9947057a9075 100755 --- a/Tools/wasm/wasm_build.py +++ b/Tools/wasm/wasm_build.py @@ -480,7 +480,6 @@ def configure_cmd(self) -> List[str]: cmd.append(f"--{opt}-wasm-dynamic-linking") if self.pthreads is not None: - assert self.host.is_emscripten opt = "enable" if self.pthreads else "disable" cmd.append(f"--{opt}-wasm-pthreads") @@ -745,6 +744,13 @@ def build_emports(self, force: bool = False): support_level=SupportLevel.supported, host=Host.wasm32_wasi, ), + # wasm32-wasi-threads + BuildProfile( + "wasi-threads", + support_level=SupportLevel.experimental, + host=Host.wasm32_wasi, + pthreads=True, + ), # no SDK available yet # BuildProfile( # "wasm64-wasi", diff --git a/configure b/configure index 248b14e254f6d..91bf1d2c02370 100755 --- a/configure +++ b/configure @@ -6899,7 +6899,11 @@ cat > conftest.c <>confdefs.h LIBS="$LIBS -lwasi-emulated-signal -lwasi-emulated-getpid -lwasi-emulated-process-clocks" echo "#define _WASI_EMULATED_SIGNAL 1" >> confdefs.h + if test "x$enable_wasm_pthreads" = xyes +then : + + # Note: update CFLAGS because ac_compile/ac_link needs this too. + # without this, configure fails to find pthread_create, sem_init, + # etc because they are only available in the sysroot for + # wasm32-wasi-threads. + as_fn_append CFLAGS " -target wasm32-wasi-threads -pthread" + as_fn_append CFLAGS_NODIST " -target wasm32-wasi-threads -pthread" + as_fn_append LDFLAGS_NODIST " -target wasm32-wasi-threads -pthread" + as_fn_append LDFLAGS_NODIST " -Wl,--import-memory" + as_fn_append LDFLAGS_NODIST " -Wl,--max-memory=10485760" + +fi + as_fn_append LDFLAGS_NODIST " -z stack-size=524288 -Wl,--stack-first -Wl,--initial-memory=10485760" ;; #( diff --git a/configure.ac b/configure.ac index 96c96e4b0d735..0770f68a23f61 100644 --- a/configure.ac +++ b/configure.ac @@ -1079,7 +1079,11 @@ cat > conftest.c <> confdefs.h + AS_VAR_IF([enable_wasm_pthreads], [yes], [ + # Note: update CFLAGS because ac_compile/ac_link needs this too. + # without this, configure fails to find pthread_create, sem_init, + # etc because they are only available in the sysroot for + # wasm32-wasi-threads. + AS_VAR_APPEND([CFLAGS], [" -target wasm32-wasi-threads -pthread"]) + AS_VAR_APPEND([CFLAGS_NODIST], [" -target wasm32-wasi-threads -pthread"]) + AS_VAR_APPEND([LDFLAGS_NODIST], [" -target wasm32-wasi-threads -pthread"]) + AS_VAR_APPEND([LDFLAGS_NODIST], [" -Wl,--import-memory"]) + AS_VAR_APPEND([LDFLAGS_NODIST], [" -Wl,--max-memory=10485760"]) + ]) + dnl increase initial memory and stack size, move stack first dnl https://github.com/WebAssembly/wasi-libc/issues/233 AS_VAR_APPEND([LDFLAGS_NODIST], [" -z stack-size=524288 -Wl,--stack-first -Wl,--initial-memory=10485760"]) From webhook-mailer at python.org Mon Jul 17 18:37:14 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 17 Jul 2023 22:37:14 -0000 Subject: [Python-checkins] gh-104683: Argument Clinic: Modernise parse_special_symbol() (#106837) Message-ID: https://github.com/python/cpython/commit/00e52acebd2beb2663202bfc4be0ce79ba77361e commit: 00e52acebd2beb2663202bfc4be0ce79ba77361e branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-17T22:37:11Z summary: gh-104683: Argument Clinic: Modernise parse_special_symbol() (#106837) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 311f0a1a56a03..bff8935df13bc 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4864,11 +4864,21 @@ def state_parameter(self, line: str | None) -> None: self.parameter_continuation = line[:-1] return - line = line.lstrip() - - if line in ('*', '/', '[', ']'): - self.parse_special_symbol(line) - return + func = self.function + match line.lstrip(): + case '*': + self.parse_star(func) + case '[': + self.parse_opening_square_bracket(func) + case ']': + self.parse_closing_square_bracket(func) + case '/': + self.parse_slash(func) + case param: + self.parse_parameter(param) + + def parse_parameter(self, line: str) -> None: + assert self.function is not None match self.parameter_state: case ParamState.START | ParamState.REQUIRED: @@ -5146,57 +5156,71 @@ def parse_converter( "Annotations must be either a name, a function call, or a string." ) - def parse_special_symbol(self, symbol): - if symbol == '*': - if self.keyword_only: - fail("Function " + self.function.name + " uses '*' more than once.") - self.keyword_only = True - elif symbol == '[': - match self.parameter_state: - case ParamState.START | ParamState.LEFT_SQUARE_BEFORE: - self.parameter_state = ParamState.LEFT_SQUARE_BEFORE - case ParamState.REQUIRED | ParamState.GROUP_AFTER: - self.parameter_state = ParamState.GROUP_AFTER - case st: - fail(f"Function {self.function.name} has an unsupported group configuration. (Unexpected state {st}.b)") - self.group += 1 - self.function.docstring_only = True - elif symbol == ']': - if not self.group: - fail("Function " + self.function.name + " has a ] without a matching [.") - if not any(p.group == self.group for p in self.function.parameters.values()): - fail("Function " + self.function.name + " has an empty group.\nAll groups must contain at least one parameter.") - self.group -= 1 - match self.parameter_state: - case ParamState.LEFT_SQUARE_BEFORE | ParamState.GROUP_BEFORE: - self.parameter_state = ParamState.GROUP_BEFORE - case ParamState.GROUP_AFTER | ParamState.RIGHT_SQUARE_AFTER: - self.parameter_state = ParamState.RIGHT_SQUARE_AFTER - case st: - fail(f"Function {self.function.name} has an unsupported group configuration. (Unexpected state {st}.c)") - elif symbol == '/': - if self.positional_only: - fail("Function " + self.function.name + " uses '/' more than once.") - self.positional_only = True - # REQUIRED and OPTIONAL are allowed here, that allows positional-only without option groups - # to work (and have default values!) - allowed = { - ParamState.REQUIRED, - ParamState.OPTIONAL, - ParamState.RIGHT_SQUARE_AFTER, - ParamState.GROUP_BEFORE, - } - if (self.parameter_state not in allowed) or self.group: - fail(f"Function {self.function.name} has an unsupported group configuration. (Unexpected state {self.parameter_state}.d)") - if self.keyword_only: - fail("Function " + self.function.name + " mixes keyword-only and positional-only parameters, which is unsupported.") - # fixup preceding parameters - for p in self.function.parameters.values(): - if p.is_vararg(): - continue - if (p.kind != inspect.Parameter.POSITIONAL_OR_KEYWORD and not isinstance(p.converter, self_converter)): - fail("Function " + self.function.name + " mixes keyword-only and positional-only parameters, which is unsupported.") - p.kind = inspect.Parameter.POSITIONAL_ONLY + def parse_star(self, function: Function) -> None: + """Parse keyword-only parameter marker '*'.""" + if self.keyword_only: + fail(f"Function {function.name} uses '*' more than once.") + self.keyword_only = True + + def parse_opening_square_bracket(self, function: Function) -> None: + """Parse opening parameter group symbol '['.""" + match self.parameter_state: + case ParamState.START | ParamState.LEFT_SQUARE_BEFORE: + self.parameter_state = ParamState.LEFT_SQUARE_BEFORE + case ParamState.REQUIRED | ParamState.GROUP_AFTER: + self.parameter_state = ParamState.GROUP_AFTER + case st: + fail(f"Function {function.name} has an unsupported group configuration. " + f"(Unexpected state {st}.b)") + self.group += 1 + function.docstring_only = True + + def parse_closing_square_bracket(self, function: Function) -> None: + """Parse closing parameter group symbol ']'.""" + if not self.group: + fail(f"Function {function.name} has a ] without a matching [.") + if not any(p.group == self.group for p in function.parameters.values()): + fail(f"Function {function.name} has an empty group.\n" + "All groups must contain at least one parameter.") + self.group -= 1 + match self.parameter_state: + case ParamState.LEFT_SQUARE_BEFORE | ParamState.GROUP_BEFORE: + self.parameter_state = ParamState.GROUP_BEFORE + case ParamState.GROUP_AFTER | ParamState.RIGHT_SQUARE_AFTER: + self.parameter_state = ParamState.RIGHT_SQUARE_AFTER + case st: + fail(f"Function {function.name} has an unsupported group configuration. " + f"(Unexpected state {st}.c)") + + def parse_slash(self, function: Function) -> None: + """Parse positional-only parameter marker '/'.""" + if self.positional_only: + fail(f"Function {function.name} uses '/' more than once.") + self.positional_only = True + # REQUIRED and OPTIONAL are allowed here, that allows positional-only + # without option groups to work (and have default values!) + allowed = { + ParamState.REQUIRED, + ParamState.OPTIONAL, + ParamState.RIGHT_SQUARE_AFTER, + ParamState.GROUP_BEFORE, + } + if (self.parameter_state not in allowed) or self.group: + fail(f"Function {function.name} has an unsupported group configuration. " + f"(Unexpected state {self.parameter_state}.d)") + if self.keyword_only: + fail(f"Function {function.name} mixes keyword-only and " + "positional-only parameters, which is unsupported.") + # fixup preceding parameters + for p in function.parameters.values(): + if p.is_vararg(): + continue + if (p.kind is not inspect.Parameter.POSITIONAL_OR_KEYWORD and + not isinstance(p.converter, self_converter) + ): + fail(f"Function {function.name} mixes keyword-only and " + "positional-only parameters, which is unsupported.") + p.kind = inspect.Parameter.POSITIONAL_ONLY def state_parameter_docstring_start(self, line: str | None) -> None: self.parameter_docstring_indent = len(self.indent.margin) From webhook-mailer at python.org Mon Jul 17 21:30:44 2023 From: webhook-mailer at python.org (gvanrossum) Date: Tue, 18 Jul 2023 01:30:44 -0000 Subject: [Python-checkins] Small fixes to code generator (#106845) Message-ID: https://github.com/python/cpython/commit/1e36ca63f9f5e0399efe13a80499cef290314c2a commit: 1e36ca63f9f5e0399efe13a80499cef290314c2a branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-18T01:30:41Z summary: Small fixes to code generator (#106845) These repair nits I found in PR gh-106798 (issue gh-106797) and in PR gh-106716 (issue gh-106706). files: M Include/internal/pycore_opcode_metadata.h M Python/generated_cases.c.h M Tools/cases_generator/generate_cases.py diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index c3a0dbb478a7c..a5844b3135d39 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -842,15 +842,15 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { case PUSH_EXC_INFO: return 2; case LOAD_ATTR_METHOD_WITH_VALUES: - return 1 + 1; + return 2; case LOAD_ATTR_METHOD_NO_DICT: - return 1 + 1; + return 2; case LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES: - return 0 + 1; + return 1; case LOAD_ATTR_NONDESCRIPTOR_NO_DICT: - return 0 + 1; + return 1; case LOAD_ATTR_METHOD_LAZY_DICT: - return 1 + 1; + return 2; case KW_NAMES: return 0; case INSTRUMENTED_CALL: diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 0148078d18bdc..0a8e4da46b8be 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3356,7 +3356,7 @@ res = self; STACK_GROW(1); stack_pointer[-1] = res; - stack_pointer[-(1 + 1)] = res2; + stack_pointer[-2] = res2; next_instr += 9; DISPATCH(); } @@ -3378,7 +3378,7 @@ res = self; STACK_GROW(1); stack_pointer[-1] = res; - stack_pointer[-(1 + 1)] = res2; + stack_pointer[-2] = res2; next_instr += 9; DISPATCH(); } @@ -3403,7 +3403,6 @@ assert(descr != NULL); Py_DECREF(self); res = Py_NewRef(descr); - STACK_GROW(0); stack_pointer[-1] = res; next_instr += 9; DISPATCH(); @@ -3423,7 +3422,6 @@ assert(descr != NULL); Py_DECREF(self); res = Py_NewRef(descr); - STACK_GROW(0); stack_pointer[-1] = res; next_instr += 9; DISPATCH(); @@ -3450,7 +3448,7 @@ res = self; STACK_GROW(1); stack_pointer[-1] = res; - stack_pointer[-(1 + 1)] = res2; + stack_pointer[-2] = res2; next_instr += 9; DISPATCH(); } diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 037bee107cb13..2713fc6774e84 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -99,7 +99,7 @@ def effect_size(effect: StackEffect) -> tuple[int, str]: return 0, effect.size elif effect.cond: if effect.cond in ("0", "1"): - return 0, effect.cond + return int(effect.cond), "" return 0, f"{maybe_parenthesize(effect.cond)} ? 1 : 0" else: return 1, "" @@ -841,9 +841,9 @@ def map_families(self) -> None: def check_families(self) -> None: """Check each family: - - Must have at least 2 members - - All members must be known instructions - - All members must have the same cache, input and output effects + - Must have at least 2 members (including head) + - Head and all members must be known instructions + - Head and all members must have the same cache, input and output effects """ for family in self.families.values(): if family.name not in self.macro_instrs and family.name not in self.instrs: @@ -868,7 +868,7 @@ def check_families(self) -> None: self.error( f"Family {family.name!r} has inconsistent " f"(cache, input, output) effects:\n" - f" {family.members[0]} = {expected_effects}; " + f" {family.name} = {expected_effects}; " f"{member} = {member_effects}", family, ) From webhook-mailer at python.org Mon Jul 17 23:44:20 2023 From: webhook-mailer at python.org (methane) Date: Tue, 18 Jul 2023 03:44:20 -0000 Subject: [Python-checkins] gh-106843: fix memleak in _PyCompile_CleanDoc (#106846) Message-ID: https://github.com/python/cpython/commit/ece3b9d12a2f47da8b144f185dfba9b2b725fc82 commit: ece3b9d12a2f47da8b144f185dfba9b2b725fc82 branch: main author: Inada Naoki committer: methane date: 2023-07-18T03:44:16Z summary: gh-106843: fix memleak in _PyCompile_CleanDoc (#106846) files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index b80f7c01bcd90..2a735382c0cfd 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -2267,6 +2267,7 @@ compiler_function_body(struct compiler *c, stmt_ty s, int is_async, Py_ssize_t f } } if (compiler_add_const(c->c_const_cache, c->u, docstring ? docstring : Py_None) < 0) { + Py_XDECREF(docstring); compiler_exit_scope(c); return ERROR; } @@ -8060,7 +8061,9 @@ _PyCompile_CleanDoc(PyObject *doc) } Py_DECREF(doc); - return PyUnicode_FromStringAndSize(buff, w - buff); + PyObject *res = PyUnicode_FromStringAndSize(buff, w - buff); + PyMem_Free(buff); + return res; } From webhook-mailer at python.org Tue Jul 18 01:57:02 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 05:57:02 -0000 Subject: [Python-checkins] gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720) Message-ID: https://github.com/python/cpython/commit/e1c295e3da9ff5a3eb6b009a1f821d80e564ac87 commit: e1c295e3da9ff5a3eb6b009a1f821d80e564ac87 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-18T08:56:58+03:00 summary: gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720) No longer suppress arbitrary errors. Simplify the code. files: A Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst M Objects/moduleobject.c M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst new file mode 100644 index 0000000000000..dc4bef193a322 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst @@ -0,0 +1,2 @@ +No longer suppress arbitrary errors in the ``__annotations__`` getter and +setter in the type and module types. diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index 4071b5a3f1a62..ba20534c3bdd8 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -937,26 +937,20 @@ static PyObject * module_get_annotations(PyModuleObject *m, void *Py_UNUSED(ignored)) { PyObject *dict = PyObject_GetAttr((PyObject *)m, &_Py_ID(__dict__)); - - if ((dict == NULL) || !PyDict_Check(dict)) { + if (dict == NULL) { + return NULL; + } + if (!PyDict_Check(dict)) { PyErr_Format(PyExc_TypeError, ".__dict__ is not a dictionary"); - Py_XDECREF(dict); + Py_DECREF(dict); return NULL; } - PyObject *annotations; - /* there's no _PyDict_GetItemId without WithError, so let's LBYL. */ - if (PyDict_Contains(dict, &_Py_ID(__annotations__))) { - annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); - /* - ** _PyDict_GetItemIdWithError could still fail, - ** for instance with a well-timed Ctrl-C or a MemoryError. - ** so let's be totally safe. - */ - if (annotations) { - Py_INCREF(annotations); - } - } else { + PyObject *annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); + if (annotations) { + Py_INCREF(annotations); + } + else if (!PyErr_Occurred()) { annotations = PyDict_New(); if (annotations) { int result = PyDict_SetItem( @@ -975,8 +969,10 @@ module_set_annotations(PyModuleObject *m, PyObject *value, void *Py_UNUSED(ignor { int ret = -1; PyObject *dict = PyObject_GetAttr((PyObject *)m, &_Py_ID(__dict__)); - - if ((dict == NULL) || !PyDict_Check(dict)) { + if (dict == NULL) { + return -1; + } + if (!PyDict_Check(dict)) { PyErr_Format(PyExc_TypeError, ".__dict__ is not a dictionary"); goto exit; } @@ -984,19 +980,17 @@ module_set_annotations(PyModuleObject *m, PyObject *value, void *Py_UNUSED(ignor if (value != NULL) { /* set */ ret = PyDict_SetItem(dict, &_Py_ID(__annotations__), value); - goto exit; } - - /* delete */ - if (!PyDict_Contains(dict, &_Py_ID(__annotations__))) { - PyErr_Format(PyExc_AttributeError, "__annotations__"); - goto exit; + else { + /* delete */ + ret = PyDict_DelItem(dict, &_Py_ID(__annotations__)); + if (ret < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) { + PyErr_SetString(PyExc_AttributeError, "__annotations__"); + } } - ret = PyDict_DelItem(dict, &_Py_ID(__annotations__)); - exit: - Py_XDECREF(dict); + Py_DECREF(dict); return ret; } diff --git a/Objects/typeobject.c b/Objects/typeobject.c index b1f9f1280fd04..7e5282cabd1bf 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1451,24 +1451,17 @@ type_get_annotations(PyTypeObject *type, void *context) } PyObject *annotations; - /* there's no _PyDict_GetItemId without WithError, so let's LBYL. */ PyObject *dict = lookup_tp_dict(type); - if (PyDict_Contains(dict, &_Py_ID(__annotations__))) { - annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); - /* - ** PyDict_GetItemWithError could still fail, - ** for instance with a well-timed Ctrl-C or a MemoryError. - ** so let's be totally safe. - */ - if (annotations) { - if (Py_TYPE(annotations)->tp_descr_get) { - annotations = Py_TYPE(annotations)->tp_descr_get( - annotations, NULL, (PyObject *)type); - } else { - Py_INCREF(annotations); - } + annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); + if (annotations) { + if (Py_TYPE(annotations)->tp_descr_get) { + annotations = Py_TYPE(annotations)->tp_descr_get( + annotations, NULL, (PyObject *)type); + } else { + Py_INCREF(annotations); } - } else { + } + else if (!PyErr_Occurred()) { annotations = PyDict_New(); if (annotations) { int result = PyDict_SetItem( @@ -1500,11 +1493,10 @@ type_set_annotations(PyTypeObject *type, PyObject *value, void *context) result = PyDict_SetItem(dict, &_Py_ID(__annotations__), value); } else { /* delete */ - if (!PyDict_Contains(dict, &_Py_ID(__annotations__))) { - PyErr_Format(PyExc_AttributeError, "__annotations__"); - return -1; - } result = PyDict_DelItem(dict, &_Py_ID(__annotations__)); + if (result < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) { + PyErr_SetString(PyExc_AttributeError, "__annotations__"); + } } if (result == 0) { From webhook-mailer at python.org Tue Jul 18 02:00:25 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 06:00:25 -0000 Subject: [Python-checkins] gh-86493: Fix possible leaks in modules initialization: _curses_panel, _decimal, posix, xxsubtype (GH-106767) Message-ID: https://github.com/python/cpython/commit/745492355b94d109e47827e5865846f25ae42d26 commit: 745492355b94d109e47827e5865846f25ae42d26 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-18T09:00:22+03:00 summary: gh-86493: Fix possible leaks in modules initialization: _curses_panel, _decimal, posix, xxsubtype (GH-106767) files: M Modules/_curses_panel.c M Modules/_decimal/_decimal.c M Modules/posixmodule.c M Modules/xxsubtype.c diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c index a3124ff80551e..292b57c083d0c 100644 --- a/Modules/_curses_panel.c +++ b/Modules/_curses_panel.c @@ -662,8 +662,7 @@ _curses_panel_exec(PyObject *mod) state->PyCursesError = PyErr_NewException( "_curses_panel.error", NULL, NULL); - if (PyModule_AddObject(mod, "error", Py_NewRef(state->PyCursesError)) < 0) { - Py_DECREF(state->PyCursesError); + if (PyModule_AddObjectRef(mod, "error", state->PyCursesError) < 0) { return -1; } diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index e3dc304066b45..f9dc6e875fa5f 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -5957,7 +5957,7 @@ PyInit__decimal(void) Py_DECREF(base); /* add to module */ - CHECK_INT(PyModule_AddObject(m, cm->name, Py_NewRef(cm->ex))); + CHECK_INT(PyModule_AddObjectRef(m, cm->name, cm->ex)); /* add to signal tuple */ PyTuple_SET_ITEM(state->SignalTuple, i, Py_NewRef(cm->ex)); @@ -5986,39 +5986,39 @@ PyInit__decimal(void) ASSIGN_PTR(cm->ex, PyErr_NewException(cm->fqname, base, NULL)); Py_DECREF(base); - CHECK_INT(PyModule_AddObject(m, cm->name, Py_NewRef(cm->ex))); + CHECK_INT(PyModule_AddObjectRef(m, cm->name, cm->ex)); } /* Init default context template first */ ASSIGN_PTR(state->default_context_template, PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL)); - CHECK_INT(PyModule_AddObject(m, "DefaultContext", - Py_NewRef(state->default_context_template))); + CHECK_INT(PyModule_AddObjectRef(m, "DefaultContext", + state->default_context_template)); #ifndef WITH_DECIMAL_CONTEXTVAR ASSIGN_PTR(state->tls_context_key, PyUnicode_FromString("___DECIMAL_CTX__")); - CHECK_INT(PyModule_AddObject(m, "HAVE_CONTEXTVAR", Py_NewRef(Py_False))); + CHECK_INT(PyModule_AddObjectRef(m, "HAVE_CONTEXTVAR", Py_False)); #else ASSIGN_PTR(state->current_context_var, PyContextVar_New("decimal_context", NULL)); - CHECK_INT(PyModule_AddObject(m, "HAVE_CONTEXTVAR", Py_NewRef(Py_True))); + CHECK_INT(PyModule_AddObjectRef(m, "HAVE_CONTEXTVAR", Py_True)); #endif - CHECK_INT(PyModule_AddObject(m, "HAVE_THREADS", Py_NewRef(Py_True))); + CHECK_INT(PyModule_AddObjectRef(m, "HAVE_THREADS", Py_True)); /* Init basic context template */ ASSIGN_PTR(state->basic_context_template, PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL)); init_basic_context(state->basic_context_template); - CHECK_INT(PyModule_AddObject(m, "BasicContext", - Py_NewRef(state->basic_context_template))); + CHECK_INT(PyModule_AddObjectRef(m, "BasicContext", + state->basic_context_template)); /* Init extended context template */ ASSIGN_PTR(state->extended_context_template, PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL)); init_extended_context(state->extended_context_template); - CHECK_INT(PyModule_AddObject(m, "ExtendedContext", - Py_NewRef(state->extended_context_template))); + CHECK_INT(PyModule_AddObjectRef(m, "ExtendedContext", + state->extended_context_template)); /* Init mpd_ssize_t constants */ @@ -6037,7 +6037,7 @@ PyInit__decimal(void) /* Init string constants */ for (i = 0; i < _PY_DEC_ROUND_GUARD; i++) { ASSIGN_PTR(state->round_map[i], PyUnicode_InternFromString(mpd_round_string[i])); - CHECK_INT(PyModule_AddObject(m, mpd_round_string[i], Py_NewRef(state->round_map[i]))); + CHECK_INT(PyModule_AddObjectRef(m, mpd_round_string[i], state->round_map[i])); } /* Add specification version number */ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index aef802c232c6c..fd5e491f4611e 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -16793,57 +16793,49 @@ posixmodule_exec(PyObject *m) if (setup_confname_tables(m)) return -1; - PyModule_AddObject(m, "error", Py_NewRef(PyExc_OSError)); + if (PyModule_AddObjectRef(m, "error", PyExc_OSError) < 0) { + return -1; + } #if defined(HAVE_WAITID) && !defined(__APPLE__) waitid_result_desc.name = MODNAME ".waitid_result"; - PyObject *WaitidResultType = (PyObject *)PyStructSequence_NewType(&waitid_result_desc); - if (WaitidResultType == NULL) { + state->WaitidResultType = (PyObject *)PyStructSequence_NewType(&waitid_result_desc); + if (PyModule_AddObjectRef(m, "waitid_result", state->WaitidResultType) < 0) { return -1; } - PyModule_AddObject(m, "waitid_result", Py_NewRef(WaitidResultType)); - state->WaitidResultType = WaitidResultType; #endif stat_result_desc.name = "os.stat_result"; /* see issue #19209 */ stat_result_desc.fields[7].name = PyStructSequence_UnnamedField; stat_result_desc.fields[8].name = PyStructSequence_UnnamedField; stat_result_desc.fields[9].name = PyStructSequence_UnnamedField; - PyObject *StatResultType = (PyObject *)PyStructSequence_NewType(&stat_result_desc); - if (StatResultType == NULL) { + state->StatResultType = (PyObject *)PyStructSequence_NewType(&stat_result_desc); + if (PyModule_AddObjectRef(m, "stat_result", state->StatResultType) < 0) { return -1; } - PyModule_AddObject(m, "stat_result", Py_NewRef(StatResultType)); - state->StatResultType = StatResultType; - state->statresult_new_orig = ((PyTypeObject *)StatResultType)->tp_new; - ((PyTypeObject *)StatResultType)->tp_new = statresult_new; + state->statresult_new_orig = ((PyTypeObject *)state->StatResultType)->tp_new; + ((PyTypeObject *)state->StatResultType)->tp_new = statresult_new; statvfs_result_desc.name = "os.statvfs_result"; /* see issue #19209 */ - PyObject *StatVFSResultType = (PyObject *)PyStructSequence_NewType(&statvfs_result_desc); - if (StatVFSResultType == NULL) { + state->StatVFSResultType = (PyObject *)PyStructSequence_NewType(&statvfs_result_desc); + if (PyModule_AddObjectRef(m, "statvfs_result", state->StatVFSResultType) < 0) { return -1; } - PyModule_AddObject(m, "statvfs_result", Py_NewRef(StatVFSResultType)); - state->StatVFSResultType = StatVFSResultType; #if defined(HAVE_SCHED_SETPARAM) || defined(HAVE_SCHED_SETSCHEDULER) || defined(POSIX_SPAWN_SETSCHEDULER) || defined(POSIX_SPAWN_SETSCHEDPARAM) sched_param_desc.name = MODNAME ".sched_param"; - PyObject *SchedParamType = (PyObject *)PyStructSequence_NewType(&sched_param_desc); - if (SchedParamType == NULL) { + state->SchedParamType = (PyObject *)PyStructSequence_NewType(&sched_param_desc); + if (PyModule_AddObjectRef(m, "sched_param", state->SchedParamType) < 0) { return -1; } - PyModule_AddObject(m, "sched_param", Py_NewRef(SchedParamType)); - state->SchedParamType = SchedParamType; - ((PyTypeObject *)SchedParamType)->tp_new = os_sched_param; + ((PyTypeObject *)state->SchedParamType)->tp_new = os_sched_param; #endif /* initialize TerminalSize_info */ - PyObject *TerminalSizeType = (PyObject *)PyStructSequence_NewType(&TerminalSize_desc); - if (TerminalSizeType == NULL) { + state->TerminalSizeType = (PyObject *)PyStructSequence_NewType(&TerminalSize_desc); + if (PyModule_AddObjectRef(m, "terminal_size", state->TerminalSizeType) < 0) { return -1; } - PyModule_AddObject(m, "terminal_size", Py_NewRef(TerminalSizeType)); - state->TerminalSizeType = TerminalSizeType; /* initialize scandir types */ PyObject *ScandirIteratorType = PyType_FromModuleAndSpec(m, &ScandirIteratorType_spec, NULL); @@ -16852,28 +16844,21 @@ posixmodule_exec(PyObject *m) } state->ScandirIteratorType = ScandirIteratorType; - PyObject *DirEntryType = PyType_FromModuleAndSpec(m, &DirEntryType_spec, NULL); - if (DirEntryType == NULL) { + state->DirEntryType = PyType_FromModuleAndSpec(m, &DirEntryType_spec, NULL); + if (PyModule_AddObjectRef(m, "DirEntry", state->DirEntryType) < 0) { return -1; } - PyModule_AddObject(m, "DirEntry", Py_NewRef(DirEntryType)); - state->DirEntryType = DirEntryType; times_result_desc.name = MODNAME ".times_result"; - PyObject *TimesResultType = (PyObject *)PyStructSequence_NewType(×_result_desc); - if (TimesResultType == NULL) { + state->TimesResultType = (PyObject *)PyStructSequence_NewType(×_result_desc); + if (PyModule_AddObjectRef(m, "times_result", state->TimesResultType) < 0) { return -1; } - PyModule_AddObject(m, "times_result", Py_NewRef(TimesResultType)); - state->TimesResultType = TimesResultType; - PyTypeObject *UnameResultType = PyStructSequence_NewType(&uname_result_desc); - if (UnameResultType == NULL) { + state->UnameResultType = (PyObject *)PyStructSequence_NewType(&uname_result_desc); + if (PyModule_AddObjectRef(m, "uname_result", state->UnameResultType) < 0) { return -1; } - ; - PyModule_AddObject(m, "uname_result", Py_NewRef(UnameResultType)); - state->UnameResultType = (PyObject *)UnameResultType; if ((state->billion = PyLong_FromLong(1000000000)) == NULL) return -1; diff --git a/Modules/xxsubtype.c b/Modules/xxsubtype.c index 744ba7bf5d28b..9e4a3d66ef41b 100644 --- a/Modules/xxsubtype.c +++ b/Modules/xxsubtype.c @@ -274,12 +274,10 @@ xxsubtype_exec(PyObject* m) if (PyType_Ready(&spamdict_type) < 0) return -1; - if (PyModule_AddObject(m, "spamlist", - Py_NewRef(&spamlist_type)) < 0) + if (PyModule_AddObjectRef(m, "spamlist", (PyObject *)&spamlist_type) < 0) return -1; - if (PyModule_AddObject(m, "spamdict", - Py_NewRef(&spamdict_type)) < 0) + if (PyModule_AddObjectRef(m, "spamdict", (PyObject *)&spamdict_type) < 0) return -1; return 0; } From webhook-mailer at python.org Tue Jul 18 02:42:09 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 06:42:09 -0000 Subject: [Python-checkins] bpo-42327: C API: Add PyModule_Add() function (GH-23443) Message-ID: https://github.com/python/cpython/commit/83ac1284909433f3f77c0a4f459996b1ba3f1a4d commit: 83ac1284909433f3f77c0a4f459996b1ba3f1a4d branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-18T09:42:05+03:00 summary: bpo-42327: C API: Add PyModule_Add() function (GH-23443) It is a fixed implementation of PyModule_AddObject() which consistently steals reference both on success and on failure. files: A Misc/NEWS.d/next/C API/2020-11-11-22-36-29.bpo-42327.ODSZBM.rst M Doc/c-api/module.rst M Doc/data/stable_abi.dat M Doc/whatsnew/3.13.rst M Include/modsupport.h M Lib/test/test_stable_abi_ctypes.py M Misc/stable_abi.toml M PC/python3dll.c M Python/modsupport.c diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index 230b471d473be..d35b302fce6aa 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -486,12 +486,29 @@ state: .. versionadded:: 3.10 +.. c:function:: int PyModule_Add(PyObject *module, const char *name, PyObject *value) + + Similar to :c:func:`PyModule_AddObjectRef`, but "steals" a reference + to *value*. + It can be called with a result of function that returns a new reference + without bothering to check its result or even saving it to a variable. + + Example usage:: + + if (PyModule_Add(module, "spam", PyBytes_FromString(value)) < 0) { + goto error; + } + + .. versionadded:: 3.13 + + .. c:function:: int PyModule_AddObject(PyObject *module, const char *name, PyObject *value) Similar to :c:func:`PyModule_AddObjectRef`, but steals a reference to *value* on success (if it returns ``0``). - The new :c:func:`PyModule_AddObjectRef` function is recommended, since it is + The new :c:func:`PyModule_Add` or :c:func:`PyModule_AddObjectRef` + functions are recommended, since it is easy to introduce reference leaks by misusing the :c:func:`PyModule_AddObject` function. @@ -501,44 +518,24 @@ state: only decrements the reference count of *value* **on success**. This means that its return value must be checked, and calling code must - :c:func:`Py_DECREF` *value* manually on error. + :c:func:`Py_XDECREF` *value* manually on error. Example usage:: - static int - add_spam(PyObject *module, int value) - { - PyObject *obj = PyLong_FromLong(value); - if (obj == NULL) { - return -1; - } - if (PyModule_AddObject(module, "spam", obj) < 0) { - Py_DECREF(obj); - return -1; - } - // PyModule_AddObject() stole a reference to obj: - // Py_DECREF(obj) is not needed here - return 0; - } - - The example can also be written without checking explicitly if *obj* is - ``NULL``:: + PyObject *obj = PyBytes_FromString(value); + if (PyModule_AddObject(module, "spam", obj) < 0) { + // If 'obj' is not NULL and PyModule_AddObject() failed, + // 'obj' strong reference must be deleted with Py_XDECREF(). + // If 'obj' is NULL, Py_XDECREF() does nothing. + Py_XDECREF(obj); + goto error; + } + // PyModule_AddObject() stole a reference to obj: + // Py_XDECREF(obj) is not needed here. - static int - add_spam(PyObject *module, int value) - { - PyObject *obj = PyLong_FromLong(value); - if (PyModule_AddObject(module, "spam", obj) < 0) { - Py_XDECREF(obj); - return -1; - } - // PyModule_AddObject() stole a reference to obj: - // Py_DECREF(obj) is not needed here - return 0; - } + .. deprecated:: 3.13 - Note that ``Py_XDECREF()`` should be used instead of ``Py_DECREF()`` in - this case, since *obj* can be ``NULL``. + :c:func:`PyModule_AddObject` is :term:`soft deprecated`. .. c:function:: int PyModule_AddIntConstant(PyObject *module, const char *name, long value) diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index e3dd3dab27a03..aa1edf5463705 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -399,6 +399,7 @@ type,PyModuleDef,3.2,,full-abi type,PyModuleDef_Base,3.2,,full-abi function,PyModuleDef_Init,3.5,, var,PyModuleDef_Type,3.5,, +function,PyModule_Add,3.13,, function,PyModule_AddFunctions,3.7,, function,PyModule_AddIntConstant,3.2,, function,PyModule_AddObject,3.2,, diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 161d5fb1c59a3..0181e16f7d9b9 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -774,6 +774,11 @@ New Features If the assertion fails, make sure that the size is set before. (Contributed by Victor Stinner in :gh:`106168`.) +* Add :c:func:`PyModule_Add` function: similar to + :c:func:`PyModule_AddObjectRef` and :c:func:`PyModule_AddObject` but + always steals a reference to the value. + (Contributed by Serhiy Storchaka in :gh:`86493`.) + Porting to Python 3.13 ---------------------- diff --git a/Include/modsupport.h b/Include/modsupport.h index 7d4cfe853aaa7..51061c5bc8090 100644 --- a/Include/modsupport.h +++ b/Include/modsupport.h @@ -23,12 +23,18 @@ PyAPI_FUNC(PyObject *) Py_BuildValue(const char *, ...); PyAPI_FUNC(PyObject *) Py_VaBuildValue(const char *, va_list); // Add an attribute with name 'name' and value 'obj' to the module 'mod. -// On success, return 0 on success. +// On success, return 0. // On error, raise an exception and return -1. PyAPI_FUNC(int) PyModule_AddObjectRef(PyObject *mod, const char *name, PyObject *value); -// Similar to PyModule_AddObjectRef() but steal a reference to 'obj' -// (Py_DECREF(obj)) on success (if it returns 0). +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030d0000 +// Similar to PyModule_AddObjectRef() but steal a reference to 'value'. +PyAPI_FUNC(int) PyModule_Add(PyObject *mod, const char *name, PyObject *value); +#endif /* Py_LIMITED_API */ + +// Similar to PyModule_AddObjectRef() and PyModule_Add() but steal +// a reference to 'value' on success and only on success. +// Errorprone. Should not be used in new code. PyAPI_FUNC(int) PyModule_AddObject(PyObject *mod, const char *, PyObject *value); PyAPI_FUNC(int) PyModule_AddIntConstant(PyObject *, const char *, long); diff --git a/Lib/test/test_stable_abi_ctypes.py b/Lib/test/test_stable_abi_ctypes.py index e92e986b29337..4e74bb374c93b 100644 --- a/Lib/test/test_stable_abi_ctypes.py +++ b/Lib/test/test_stable_abi_ctypes.py @@ -425,6 +425,7 @@ def test_windows_feature_macros(self): "PyMethodDescr_Type", "PyModuleDef_Init", "PyModuleDef_Type", + "PyModule_Add", "PyModule_AddFunctions", "PyModule_AddIntConstant", "PyModule_AddObject", diff --git a/Misc/NEWS.d/next/C API/2020-11-11-22-36-29.bpo-42327.ODSZBM.rst b/Misc/NEWS.d/next/C API/2020-11-11-22-36-29.bpo-42327.ODSZBM.rst new file mode 100644 index 0000000000000..3d935aceb57a7 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2020-11-11-22-36-29.bpo-42327.ODSZBM.rst @@ -0,0 +1 @@ +Add :func:`PyModule_Add` function: similar to :c:func:`PyModule_AddObjectRef` and :c:func:`PyModule_AddObject`, but always steals a reference to the value. diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml index 8ea8fde68b833..dd2c9910b83cc 100644 --- a/Misc/stable_abi.toml +++ b/Misc/stable_abi.toml @@ -2444,3 +2444,5 @@ added = '3.13' [function.PyMapping_GetOptionalItemString] added = '3.13' +[function.PyModule_Add] + added = '3.13' diff --git a/PC/python3dll.c b/PC/python3dll.c index 8f2df6950cfc0..0b54c5a707231 100755 --- a/PC/python3dll.c +++ b/PC/python3dll.c @@ -374,6 +374,7 @@ EXPORT_FUNC(PyMemoryView_FromBuffer) EXPORT_FUNC(PyMemoryView_FromMemory) EXPORT_FUNC(PyMemoryView_FromObject) EXPORT_FUNC(PyMemoryView_GetContiguous) +EXPORT_FUNC(PyModule_Add) EXPORT_FUNC(PyModule_AddFunctions) EXPORT_FUNC(PyModule_AddIntConstant) EXPORT_FUNC(PyModule_AddObject) diff --git a/Python/modsupport.c b/Python/modsupport.c index 3db95f1f07284..18b3322ae81d1 100644 --- a/Python/modsupport.c +++ b/Python/modsupport.c @@ -606,13 +606,16 @@ PyModule_AddObjectRef(PyObject *mod, const char *name, PyObject *value) PyModule_GetName(mod)); return -1; } - - if (PyDict_SetItemString(dict, name, value)) { - return -1; - } - return 0; + return PyDict_SetItemString(dict, name, value); } +int +PyModule_Add(PyObject *mod, const char *name, PyObject *value) +{ + int res = PyModule_AddObjectRef(mod, name, value); + Py_XDECREF(value); + return res; +} int PyModule_AddObject(PyObject *mod, const char *name, PyObject *value) @@ -627,25 +630,13 @@ PyModule_AddObject(PyObject *mod, const char *name, PyObject *value) int PyModule_AddIntConstant(PyObject *m, const char *name, long value) { - PyObject *obj = PyLong_FromLong(value); - if (!obj) { - return -1; - } - int res = PyModule_AddObjectRef(m, name, obj); - Py_DECREF(obj); - return res; + return PyModule_Add(m, name, PyLong_FromLong(value)); } int PyModule_AddStringConstant(PyObject *m, const char *name, const char *value) { - PyObject *obj = PyUnicode_FromString(value); - if (!obj) { - return -1; - } - int res = PyModule_AddObjectRef(m, name, obj); - Py_DECREF(obj); - return res; + return PyModule_Add(m, name, PyUnicode_FromString(value)); } int From webhook-mailer at python.org Tue Jul 18 03:01:25 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 07:01:25 -0000 Subject: [Python-checkins] [3.12] gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720) (GH-106848) Message-ID: https://github.com/python/cpython/commit/d671c6567aa5b5c4478aadcd623f5c3b8261b5b1 commit: d671c6567aa5b5c4478aadcd623f5c3b8261b5b1 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-18T10:01:22+03:00 summary: [3.12] gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720) (GH-106848) gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720) No longer suppress arbitrary errors. Simplify the code. (cherry picked from commit e1c295e3da9ff5a3eb6b009a1f821d80e564ac87) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst M Objects/moduleobject.c M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst new file mode 100644 index 0000000000000..dc4bef193a322 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst @@ -0,0 +1,2 @@ +No longer suppress arbitrary errors in the ``__annotations__`` getter and +setter in the type and module types. diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index 985be58d02c78..4daf1a929e054 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -935,26 +935,20 @@ static PyObject * module_get_annotations(PyModuleObject *m, void *Py_UNUSED(ignored)) { PyObject *dict = PyObject_GetAttr((PyObject *)m, &_Py_ID(__dict__)); - - if ((dict == NULL) || !PyDict_Check(dict)) { + if (dict == NULL) { + return NULL; + } + if (!PyDict_Check(dict)) { PyErr_Format(PyExc_TypeError, ".__dict__ is not a dictionary"); - Py_XDECREF(dict); + Py_DECREF(dict); return NULL; } - PyObject *annotations; - /* there's no _PyDict_GetItemId without WithError, so let's LBYL. */ - if (PyDict_Contains(dict, &_Py_ID(__annotations__))) { - annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); - /* - ** _PyDict_GetItemIdWithError could still fail, - ** for instance with a well-timed Ctrl-C or a MemoryError. - ** so let's be totally safe. - */ - if (annotations) { - Py_INCREF(annotations); - } - } else { + PyObject *annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); + if (annotations) { + Py_INCREF(annotations); + } + else if (!PyErr_Occurred()) { annotations = PyDict_New(); if (annotations) { int result = PyDict_SetItem( @@ -973,8 +967,10 @@ module_set_annotations(PyModuleObject *m, PyObject *value, void *Py_UNUSED(ignor { int ret = -1; PyObject *dict = PyObject_GetAttr((PyObject *)m, &_Py_ID(__dict__)); - - if ((dict == NULL) || !PyDict_Check(dict)) { + if (dict == NULL) { + return -1; + } + if (!PyDict_Check(dict)) { PyErr_Format(PyExc_TypeError, ".__dict__ is not a dictionary"); goto exit; } @@ -982,19 +978,17 @@ module_set_annotations(PyModuleObject *m, PyObject *value, void *Py_UNUSED(ignor if (value != NULL) { /* set */ ret = PyDict_SetItem(dict, &_Py_ID(__annotations__), value); - goto exit; } - - /* delete */ - if (!PyDict_Contains(dict, &_Py_ID(__annotations__))) { - PyErr_Format(PyExc_AttributeError, "__annotations__"); - goto exit; + else { + /* delete */ + ret = PyDict_DelItem(dict, &_Py_ID(__annotations__)); + if (ret < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) { + PyErr_SetString(PyExc_AttributeError, "__annotations__"); + } } - ret = PyDict_DelItem(dict, &_Py_ID(__annotations__)); - exit: - Py_XDECREF(dict); + Py_DECREF(dict); return ret; } diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 6662379515f95..40e187d4d8d22 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1449,24 +1449,17 @@ type_get_annotations(PyTypeObject *type, void *context) } PyObject *annotations; - /* there's no _PyDict_GetItemId without WithError, so let's LBYL. */ PyObject *dict = lookup_tp_dict(type); - if (PyDict_Contains(dict, &_Py_ID(__annotations__))) { - annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); - /* - ** PyDict_GetItemWithError could still fail, - ** for instance with a well-timed Ctrl-C or a MemoryError. - ** so let's be totally safe. - */ - if (annotations) { - if (Py_TYPE(annotations)->tp_descr_get) { - annotations = Py_TYPE(annotations)->tp_descr_get( - annotations, NULL, (PyObject *)type); - } else { - Py_INCREF(annotations); - } + annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); + if (annotations) { + if (Py_TYPE(annotations)->tp_descr_get) { + annotations = Py_TYPE(annotations)->tp_descr_get( + annotations, NULL, (PyObject *)type); + } else { + Py_INCREF(annotations); } - } else { + } + else if (!PyErr_Occurred()) { annotations = PyDict_New(); if (annotations) { int result = PyDict_SetItem( @@ -1498,11 +1491,10 @@ type_set_annotations(PyTypeObject *type, PyObject *value, void *context) result = PyDict_SetItem(dict, &_Py_ID(__annotations__), value); } else { /* delete */ - if (!PyDict_Contains(dict, &_Py_ID(__annotations__))) { - PyErr_Format(PyExc_AttributeError, "__annotations__"); - return -1; - } result = PyDict_DelItem(dict, &_Py_ID(__annotations__)); + if (result < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) { + PyErr_SetString(PyExc_AttributeError, "__annotations__"); + } } if (result == 0) { From webhook-mailer at python.org Tue Jul 18 03:04:02 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 07:04:02 -0000 Subject: [Python-checkins] [3.12] gh-86493: Fix possible leaks in modules initialization: _curses_panel, _decimal, posix, xxsubtype (GH-106767) (#106849) Message-ID: https://github.com/python/cpython/commit/970cb8eabaaf5a8311f1aba4ca4968ef7385fce8 commit: 970cb8eabaaf5a8311f1aba4ca4968ef7385fce8 branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-18T10:03:59+03:00 summary: [3.12] gh-86493: Fix possible leaks in modules initialization: _curses_panel, _decimal, posix, xxsubtype (GH-106767) (#106849) (cherry picked from commit 745492355b94d109e47827e5865846f25ae42d26) files: M Modules/_curses_panel.c M Modules/_decimal/_decimal.c M Modules/posixmodule.c M Modules/xxsubtype.c diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c index a3124ff80551e..292b57c083d0c 100644 --- a/Modules/_curses_panel.c +++ b/Modules/_curses_panel.c @@ -662,8 +662,7 @@ _curses_panel_exec(PyObject *mod) state->PyCursesError = PyErr_NewException( "_curses_panel.error", NULL, NULL); - if (PyModule_AddObject(mod, "error", Py_NewRef(state->PyCursesError)) < 0) { - Py_DECREF(state->PyCursesError); + if (PyModule_AddObjectRef(mod, "error", state->PyCursesError) < 0) { return -1; } diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 0005081f7c5e8..70e18edb46453 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -5871,16 +5871,15 @@ PyInit__decimal(void) ASSIGN_PTR(m, PyModule_Create(&_decimal_module)); /* Add types to the module */ - CHECK_INT(PyModule_AddObject(m, "Decimal", Py_NewRef(&PyDec_Type))); - CHECK_INT(PyModule_AddObject(m, "Context", - Py_NewRef(&PyDecContext_Type))); - CHECK_INT(PyModule_AddObject(m, "DecimalTuple", Py_NewRef(DecimalTuple))); + CHECK_INT(PyModule_AddObjectRef(m, "Decimal", (PyObject *)&PyDec_Type)); + CHECK_INT(PyModule_AddObjectRef(m, "Context", (PyObject *)&PyDecContext_Type)); + CHECK_INT(PyModule_AddObjectRef(m, "DecimalTuple", (PyObject *)DecimalTuple)); /* Create top level exception */ ASSIGN_PTR(DecimalException, PyErr_NewException( "decimal.DecimalException", PyExc_ArithmeticError, NULL)); - CHECK_INT(PyModule_AddObject(m, "DecimalException", Py_NewRef(DecimalException))); + CHECK_INT(PyModule_AddObjectRef(m, "DecimalException", DecimalException)); /* Create signal tuple */ ASSIGN_PTR(SignalTuple, PyTuple_New(SIGNAL_MAP_LEN)); @@ -5920,7 +5919,7 @@ PyInit__decimal(void) Py_DECREF(base); /* add to module */ - CHECK_INT(PyModule_AddObject(m, cm->name, Py_NewRef(cm->ex))); + CHECK_INT(PyModule_AddObjectRef(m, cm->name, cm->ex)); /* add to signal tuple */ PyTuple_SET_ITEM(SignalTuple, i, Py_NewRef(cm->ex)); @@ -5949,38 +5948,38 @@ PyInit__decimal(void) ASSIGN_PTR(cm->ex, PyErr_NewException(cm->fqname, base, NULL)); Py_DECREF(base); - CHECK_INT(PyModule_AddObject(m, cm->name, Py_NewRef(cm->ex))); + CHECK_INT(PyModule_AddObjectRef(m, cm->name, cm->ex)); } /* Init default context template first */ ASSIGN_PTR(default_context_template, PyObject_CallObject((PyObject *)&PyDecContext_Type, NULL)); - CHECK_INT(PyModule_AddObject(m, "DefaultContext", - Py_NewRef(default_context_template))); + CHECK_INT(PyModule_AddObjectRef(m, "DefaultContext", + default_context_template)); #ifndef WITH_DECIMAL_CONTEXTVAR ASSIGN_PTR(tls_context_key, PyUnicode_FromString("___DECIMAL_CTX__")); - CHECK_INT(PyModule_AddObject(m, "HAVE_CONTEXTVAR", Py_NewRef(Py_False))); + CHECK_INT(PyModule_AddObjectRef(m, "HAVE_CONTEXTVAR", Py_False)); #else ASSIGN_PTR(current_context_var, PyContextVar_New("decimal_context", NULL)); - CHECK_INT(PyModule_AddObject(m, "HAVE_CONTEXTVAR", Py_NewRef(Py_True))); + CHECK_INT(PyModule_AddObjectRef(m, "HAVE_CONTEXTVAR", Py_True)); #endif - CHECK_INT(PyModule_AddObject(m, "HAVE_THREADS", Py_NewRef(Py_True))); + CHECK_INT(PyModule_AddObjectRef(m, "HAVE_THREADS", Py_True)); /* Init basic context template */ ASSIGN_PTR(basic_context_template, PyObject_CallObject((PyObject *)&PyDecContext_Type, NULL)); init_basic_context(basic_context_template); - CHECK_INT(PyModule_AddObject(m, "BasicContext", - Py_NewRef(basic_context_template))); + CHECK_INT(PyModule_AddObjectRef(m, "BasicContext", + basic_context_template)); /* Init extended context template */ ASSIGN_PTR(extended_context_template, PyObject_CallObject((PyObject *)&PyDecContext_Type, NULL)); init_extended_context(extended_context_template); - CHECK_INT(PyModule_AddObject(m, "ExtendedContext", - Py_NewRef(extended_context_template))); + CHECK_INT(PyModule_AddObjectRef(m, "ExtendedContext", + extended_context_template)); /* Init mpd_ssize_t constants */ @@ -5999,7 +5998,7 @@ PyInit__decimal(void) /* Init string constants */ for (i = 0; i < _PY_DEC_ROUND_GUARD; i++) { ASSIGN_PTR(round_map[i], PyUnicode_InternFromString(mpd_round_string[i])); - CHECK_INT(PyModule_AddObject(m, mpd_round_string[i], Py_NewRef(round_map[i]))); + CHECK_INT(PyModule_AddObjectRef(m, mpd_round_string[i], round_map[i])); } /* Add specification version number */ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 81c9990220893..0f82f1afa07aa 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -16790,57 +16790,49 @@ posixmodule_exec(PyObject *m) if (setup_confname_tables(m)) return -1; - PyModule_AddObject(m, "error", Py_NewRef(PyExc_OSError)); + if (PyModule_AddObjectRef(m, "error", PyExc_OSError) < 0) { + return -1; + } #if defined(HAVE_WAITID) && !defined(__APPLE__) waitid_result_desc.name = MODNAME ".waitid_result"; - PyObject *WaitidResultType = (PyObject *)PyStructSequence_NewType(&waitid_result_desc); - if (WaitidResultType == NULL) { + state->WaitidResultType = (PyObject *)PyStructSequence_NewType(&waitid_result_desc); + if (PyModule_AddObjectRef(m, "waitid_result", state->WaitidResultType) < 0) { return -1; } - PyModule_AddObject(m, "waitid_result", Py_NewRef(WaitidResultType)); - state->WaitidResultType = WaitidResultType; #endif stat_result_desc.name = "os.stat_result"; /* see issue #19209 */ stat_result_desc.fields[7].name = PyStructSequence_UnnamedField; stat_result_desc.fields[8].name = PyStructSequence_UnnamedField; stat_result_desc.fields[9].name = PyStructSequence_UnnamedField; - PyObject *StatResultType = (PyObject *)PyStructSequence_NewType(&stat_result_desc); - if (StatResultType == NULL) { + state->StatResultType = (PyObject *)PyStructSequence_NewType(&stat_result_desc); + if (PyModule_AddObjectRef(m, "stat_result", state->StatResultType) < 0) { return -1; } - PyModule_AddObject(m, "stat_result", Py_NewRef(StatResultType)); - state->StatResultType = StatResultType; - state->statresult_new_orig = ((PyTypeObject *)StatResultType)->tp_new; - ((PyTypeObject *)StatResultType)->tp_new = statresult_new; + state->statresult_new_orig = ((PyTypeObject *)state->StatResultType)->tp_new; + ((PyTypeObject *)state->StatResultType)->tp_new = statresult_new; statvfs_result_desc.name = "os.statvfs_result"; /* see issue #19209 */ - PyObject *StatVFSResultType = (PyObject *)PyStructSequence_NewType(&statvfs_result_desc); - if (StatVFSResultType == NULL) { + state->StatVFSResultType = (PyObject *)PyStructSequence_NewType(&statvfs_result_desc); + if (PyModule_AddObjectRef(m, "statvfs_result", state->StatVFSResultType) < 0) { return -1; } - PyModule_AddObject(m, "statvfs_result", Py_NewRef(StatVFSResultType)); - state->StatVFSResultType = StatVFSResultType; #if defined(HAVE_SCHED_SETPARAM) || defined(HAVE_SCHED_SETSCHEDULER) || defined(POSIX_SPAWN_SETSCHEDULER) || defined(POSIX_SPAWN_SETSCHEDPARAM) sched_param_desc.name = MODNAME ".sched_param"; - PyObject *SchedParamType = (PyObject *)PyStructSequence_NewType(&sched_param_desc); - if (SchedParamType == NULL) { + state->SchedParamType = (PyObject *)PyStructSequence_NewType(&sched_param_desc); + if (PyModule_AddObjectRef(m, "sched_param", state->SchedParamType) < 0) { return -1; } - PyModule_AddObject(m, "sched_param", Py_NewRef(SchedParamType)); - state->SchedParamType = SchedParamType; - ((PyTypeObject *)SchedParamType)->tp_new = os_sched_param; + ((PyTypeObject *)state->SchedParamType)->tp_new = os_sched_param; #endif /* initialize TerminalSize_info */ - PyObject *TerminalSizeType = (PyObject *)PyStructSequence_NewType(&TerminalSize_desc); - if (TerminalSizeType == NULL) { + state->TerminalSizeType = (PyObject *)PyStructSequence_NewType(&TerminalSize_desc); + if (PyModule_AddObjectRef(m, "terminal_size", state->TerminalSizeType) < 0) { return -1; } - PyModule_AddObject(m, "terminal_size", Py_NewRef(TerminalSizeType)); - state->TerminalSizeType = TerminalSizeType; /* initialize scandir types */ PyObject *ScandirIteratorType = PyType_FromModuleAndSpec(m, &ScandirIteratorType_spec, NULL); @@ -16849,28 +16841,21 @@ posixmodule_exec(PyObject *m) } state->ScandirIteratorType = ScandirIteratorType; - PyObject *DirEntryType = PyType_FromModuleAndSpec(m, &DirEntryType_spec, NULL); - if (DirEntryType == NULL) { + state->DirEntryType = PyType_FromModuleAndSpec(m, &DirEntryType_spec, NULL); + if (PyModule_AddObjectRef(m, "DirEntry", state->DirEntryType) < 0) { return -1; } - PyModule_AddObject(m, "DirEntry", Py_NewRef(DirEntryType)); - state->DirEntryType = DirEntryType; times_result_desc.name = MODNAME ".times_result"; - PyObject *TimesResultType = (PyObject *)PyStructSequence_NewType(×_result_desc); - if (TimesResultType == NULL) { + state->TimesResultType = (PyObject *)PyStructSequence_NewType(×_result_desc); + if (PyModule_AddObjectRef(m, "times_result", state->TimesResultType) < 0) { return -1; } - PyModule_AddObject(m, "times_result", Py_NewRef(TimesResultType)); - state->TimesResultType = TimesResultType; - PyTypeObject *UnameResultType = PyStructSequence_NewType(&uname_result_desc); - if (UnameResultType == NULL) { + state->UnameResultType = (PyObject *)PyStructSequence_NewType(&uname_result_desc); + if (PyModule_AddObjectRef(m, "uname_result", state->UnameResultType) < 0) { return -1; } - ; - PyModule_AddObject(m, "uname_result", Py_NewRef(UnameResultType)); - state->UnameResultType = (PyObject *)UnameResultType; if ((state->billion = PyLong_FromLong(1000000000)) == NULL) return -1; diff --git a/Modules/xxsubtype.c b/Modules/xxsubtype.c index 744ba7bf5d28b..9e4a3d66ef41b 100644 --- a/Modules/xxsubtype.c +++ b/Modules/xxsubtype.c @@ -274,12 +274,10 @@ xxsubtype_exec(PyObject* m) if (PyType_Ready(&spamdict_type) < 0) return -1; - if (PyModule_AddObject(m, "spamlist", - Py_NewRef(&spamlist_type)) < 0) + if (PyModule_AddObjectRef(m, "spamlist", (PyObject *)&spamlist_type) < 0) return -1; - if (PyModule_AddObject(m, "spamdict", - Py_NewRef(&spamdict_type)) < 0) + if (PyModule_AddObjectRef(m, "spamdict", (PyObject *)&spamdict_type) < 0) return -1; return 0; } From webhook-mailer at python.org Tue Jul 18 03:19:49 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 07:19:49 -0000 Subject: [Python-checkins] [3.11] gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720) (GH-106850) Message-ID: https://github.com/python/cpython/commit/fb0487405314d3df4ce35f45850a6f7b7f041df1 commit: fb0487405314d3df4ce35f45850a6f7b7f041df1 branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-18T10:19:45+03:00 summary: [3.11] gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720) (GH-106850) No longer suppress arbitrary errors. Simplify the code. (cherry picked from commit e1c295e3da9ff5a3eb6b009a1f821d80e564ac87) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst M Objects/moduleobject.c M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst new file mode 100644 index 0000000000000..dc4bef193a322 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-15-59-07.gh-issue-106719.jmVrsv.rst @@ -0,0 +1,2 @@ +No longer suppress arbitrary errors in the ``__annotations__`` getter and +setter in the type and module types. diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index dba20a0b36463..d18f2060f2f86 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -881,26 +881,20 @@ static PyObject * module_get_annotations(PyModuleObject *m, void *Py_UNUSED(ignored)) { PyObject *dict = PyObject_GetAttr((PyObject *)m, &_Py_ID(__dict__)); - - if ((dict == NULL) || !PyDict_Check(dict)) { + if (dict == NULL) { + return NULL; + } + if (!PyDict_Check(dict)) { PyErr_Format(PyExc_TypeError, ".__dict__ is not a dictionary"); - Py_XDECREF(dict); + Py_DECREF(dict); return NULL; } - PyObject *annotations; - /* there's no _PyDict_GetItemId without WithError, so let's LBYL. */ - if (PyDict_Contains(dict, &_Py_ID(__annotations__))) { - annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); - /* - ** _PyDict_GetItemIdWithError could still fail, - ** for instance with a well-timed Ctrl-C or a MemoryError. - ** so let's be totally safe. - */ - if (annotations) { - Py_INCREF(annotations); - } - } else { + PyObject *annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); + if (annotations) { + Py_INCREF(annotations); + } + else if (!PyErr_Occurred()) { annotations = PyDict_New(); if (annotations) { int result = PyDict_SetItem( @@ -919,8 +913,10 @@ module_set_annotations(PyModuleObject *m, PyObject *value, void *Py_UNUSED(ignor { int ret = -1; PyObject *dict = PyObject_GetAttr((PyObject *)m, &_Py_ID(__dict__)); - - if ((dict == NULL) || !PyDict_Check(dict)) { + if (dict == NULL) { + return -1; + } + if (!PyDict_Check(dict)) { PyErr_Format(PyExc_TypeError, ".__dict__ is not a dictionary"); goto exit; } @@ -928,19 +924,17 @@ module_set_annotations(PyModuleObject *m, PyObject *value, void *Py_UNUSED(ignor if (value != NULL) { /* set */ ret = PyDict_SetItem(dict, &_Py_ID(__annotations__), value); - goto exit; } - - /* delete */ - if (!PyDict_Contains(dict, &_Py_ID(__annotations__))) { - PyErr_Format(PyExc_AttributeError, "__annotations__"); - goto exit; + else { + /* delete */ + ret = PyDict_DelItem(dict, &_Py_ID(__annotations__)); + if (ret < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) { + PyErr_SetString(PyExc_AttributeError, "__annotations__"); + } } - ret = PyDict_DelItem(dict, &_Py_ID(__annotations__)); - exit: - Py_XDECREF(dict); + Py_DECREF(dict); return ret; } diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 782a7e9722883..4bdf1881581fc 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -912,24 +912,16 @@ type_get_annotations(PyTypeObject *type, void *context) } PyObject *annotations; - /* there's no _PyDict_GetItemId without WithError, so let's LBYL. */ - if (PyDict_Contains(type->tp_dict, &_Py_ID(__annotations__))) { - annotations = PyDict_GetItemWithError( - type->tp_dict, &_Py_ID(__annotations__)); - /* - ** PyDict_GetItemWithError could still fail, - ** for instance with a well-timed Ctrl-C or a MemoryError. - ** so let's be totally safe. - */ - if (annotations) { - if (Py_TYPE(annotations)->tp_descr_get) { - annotations = Py_TYPE(annotations)->tp_descr_get( - annotations, NULL, (PyObject *)type); - } else { - Py_INCREF(annotations); - } + annotations = PyDict_GetItemWithError(type->tp_dict, &_Py_ID(__annotations__)); + if (annotations) { + if (Py_TYPE(annotations)->tp_descr_get) { + annotations = Py_TYPE(annotations)->tp_descr_get( + annotations, NULL, (PyObject *)type); + } else { + Py_INCREF(annotations); } - } else { + } + else if (!PyErr_Occurred()) { annotations = PyDict_New(); if (annotations) { int result = PyDict_SetItem( @@ -960,11 +952,10 @@ type_set_annotations(PyTypeObject *type, PyObject *value, void *context) result = PyDict_SetItem(type->tp_dict, &_Py_ID(__annotations__), value); } else { /* delete */ - if (!PyDict_Contains(type->tp_dict, &_Py_ID(__annotations__))) { - PyErr_Format(PyExc_AttributeError, "__annotations__"); - return -1; - } result = PyDict_DelItem(type->tp_dict, &_Py_ID(__annotations__)); + if (result < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) { + PyErr_SetString(PyExc_AttributeError, "__annotations__"); + } } if (result == 0) { From webhook-mailer at python.org Tue Jul 18 03:50:51 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 07:50:51 -0000 Subject: [Python-checkins] gh-86493: Fix possible leaks in some modules initialization (GH-106768) Message-ID: https://github.com/python/cpython/commit/3e65baee72131b49f4ce8ca2da568a6f2001ce93 commit: 3e65baee72131b49f4ce8ca2da568a6f2001ce93 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-18T10:50:47+03:00 summary: gh-86493: Fix possible leaks in some modules initialization (GH-106768) Fix _ssl, _stat, _testinternalcapi, _threadmodule, cmath, math, posix, time. files: M Modules/_ssl.c M Modules/_stat.c M Modules/_testinternalcapi.c M Modules/_threadmodule.c M Modules/cmathmodule.c M Modules/mathmodule.c M Modules/posixmodule.c M Modules/timemodule.c diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 8612b3dd53924..7c8f4225d178a 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -6118,22 +6118,22 @@ sslmodule_init_versioninfo(PyObject *m) */ libver = OpenSSL_version_num(); r = PyLong_FromUnsignedLong(libver); - if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION_NUMBER", r)) + if (PyModule_Add(m, "OPENSSL_VERSION_NUMBER", r) < 0) return -1; parse_openssl_version(libver, &major, &minor, &fix, &patch, &status); r = Py_BuildValue("IIIII", major, minor, fix, patch, status); - if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION_INFO", r)) + if (PyModule_Add(m, "OPENSSL_VERSION_INFO", r) < 0) return -1; r = PyUnicode_FromString(OpenSSL_version(OPENSSL_VERSION)); - if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION", r)) + if (PyModule_Add(m, "OPENSSL_VERSION", r) < 0) return -1; libver = OPENSSL_VERSION_NUMBER; parse_openssl_version(libver, &major, &minor, &fix, &patch, &status); r = Py_BuildValue("IIIII", major, minor, fix, patch, status); - if (r == NULL || PyModule_AddObject(m, "_OPENSSL_API_VERSION", r)) + if (PyModule_Add(m, "_OPENSSL_API_VERSION", r) < 0) return -1; return 0; diff --git a/Modules/_stat.c b/Modules/_stat.c index 9747d848cbacb..6cea26175dee5 100644 --- a/Modules/_stat.c +++ b/Modules/_stat.c @@ -591,17 +591,17 @@ stat_exec(PyObject *module) ADD_INT_MACRO(module, FILE_ATTRIBUTE_TEMPORARY); ADD_INT_MACRO(module, FILE_ATTRIBUTE_VIRTUAL); - if (PyModule_AddObject(module, "IO_REPARSE_TAG_SYMLINK", - PyLong_FromUnsignedLong(IO_REPARSE_TAG_SYMLINK)) < 0) { - return -1; + if (PyModule_Add(module, "IO_REPARSE_TAG_SYMLINK", + PyLong_FromUnsignedLong(IO_REPARSE_TAG_SYMLINK)) < 0) { + return -1; } - if (PyModule_AddObject(module, "IO_REPARSE_TAG_MOUNT_POINT", - PyLong_FromUnsignedLong(IO_REPARSE_TAG_MOUNT_POINT)) < 0) { - return -1; + if (PyModule_Add(module, "IO_REPARSE_TAG_MOUNT_POINT", + PyLong_FromUnsignedLong(IO_REPARSE_TAG_MOUNT_POINT)) < 0) { + return -1; } - if (PyModule_AddObject(module, "IO_REPARSE_TAG_APPEXECLINK", - PyLong_FromUnsignedLong(IO_REPARSE_TAG_APPEXECLINK)) < 0) { - return -1; + if (PyModule_Add(module, "IO_REPARSE_TAG_APPEXECLINK", + PyLong_FromUnsignedLong(IO_REPARSE_TAG_APPEXECLINK)) < 0) { + return -1; } #endif diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 271ad6cfcaee3..74c932fa921cd 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -1511,13 +1511,13 @@ static PyMethodDef module_functions[] = { static int module_exec(PyObject *module) { - if (PyModule_AddObject(module, "SIZEOF_PYGC_HEAD", - PyLong_FromSsize_t(sizeof(PyGC_Head))) < 0) { + if (PyModule_Add(module, "SIZEOF_PYGC_HEAD", + PyLong_FromSsize_t(sizeof(PyGC_Head))) < 0) { return 1; } - if (PyModule_AddObject(module, "SIZEOF_TIME_T", - PyLong_FromSsize_t(sizeof(time_t))) < 0) { + if (PyModule_Add(module, "SIZEOF_TIME_T", + PyLong_FromSsize_t(sizeof(time_t))) < 0) { return 1; } diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index 82393309b6703..d8a797f34dbc4 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -1671,8 +1671,8 @@ thread_module_exec(PyObject *module) // Round towards minus infinity timeout_max = floor(timeout_max); - if (PyModule_AddObject(module, "TIMEOUT_MAX", - PyFloat_FromDouble(timeout_max)) < 0) { + if (PyModule_Add(module, "TIMEOUT_MAX", + PyFloat_FromDouble(timeout_max)) < 0) { return -1; } diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c index db8b321e72e8c..57bc55632be48 100644 --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -1217,30 +1217,29 @@ static PyMethodDef cmath_methods[] = { static int cmath_exec(PyObject *mod) { - if (PyModule_AddObject(mod, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { + if (PyModule_Add(mod, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { return -1; } - if (PyModule_AddObject(mod, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { + if (PyModule_Add(mod, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { return -1; } // 2pi - if (PyModule_AddObject(mod, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { + if (PyModule_Add(mod, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { return -1; } - if (PyModule_AddObject(mod, "inf", PyFloat_FromDouble(Py_INFINITY)) < 0) { + if (PyModule_Add(mod, "inf", PyFloat_FromDouble(Py_INFINITY)) < 0) { return -1; } Py_complex infj = {0.0, Py_INFINITY}; - if (PyModule_AddObject(mod, "infj", - PyComplex_FromCComplex(infj)) < 0) { + if (PyModule_Add(mod, "infj", PyComplex_FromCComplex(infj)) < 0) { return -1; } - if (PyModule_AddObject(mod, "nan", PyFloat_FromDouble(fabs(Py_NAN))) < 0) { + if (PyModule_Add(mod, "nan", PyFloat_FromDouble(fabs(Py_NAN))) < 0) { return -1; } Py_complex nanj = {0.0, fabs(Py_NAN)}; - if (PyModule_AddObject(mod, "nanj", PyComplex_FromCComplex(nanj)) < 0) { + if (PyModule_Add(mod, "nanj", PyComplex_FromCComplex(nanj)) < 0) { return -1; } diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index f5679fe3a6f36..0d238c086ac78 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -4037,20 +4037,20 @@ math_exec(PyObject *module) if (state->str___trunc__ == NULL) { return -1; } - if (PyModule_AddObject(module, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { + if (PyModule_Add(module, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { return -1; } - if (PyModule_AddObject(module, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { + if (PyModule_Add(module, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { return -1; } // 2pi - if (PyModule_AddObject(module, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { + if (PyModule_Add(module, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { return -1; } - if (PyModule_AddObject(module, "inf", PyFloat_FromDouble(Py_INFINITY)) < 0) { + if (PyModule_Add(module, "inf", PyFloat_FromDouble(Py_INFINITY)) < 0) { return -1; } - if (PyModule_AddObject(module, "nan", PyFloat_FromDouble(fabs(Py_NAN))) < 0) { + if (PyModule_Add(module, "nan", PyFloat_FromDouble(fabs(Py_NAN))) < 0) { return -1; } return 0; diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index fd5e491f4611e..23bf978d0cdbf 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -13466,7 +13466,7 @@ setup_confname_table(struct constdef *table, size_t tablesize, } Py_DECREF(o); } - return PyModule_AddObject(module, tablename, d); + return PyModule_Add(module, tablename, d); } /* Return -1 on failure, 0 on success. */ @@ -16781,11 +16781,9 @@ posixmodule_exec(PyObject *m) #endif /* Initialize environ dictionary */ - PyObject *v = convertenviron(); - Py_XINCREF(v); - if (v == NULL || PyModule_AddObject(m, "environ", v) != 0) + if (PyModule_Add(m, "environ", convertenviron()) != 0) { return -1; - Py_DECREF(v); + } if (all_ins(m)) return -1; @@ -16900,9 +16898,7 @@ posixmodule_exec(PyObject *m) Py_DECREF(unicode); } - PyModule_AddObject(m, "_have_functions", list); - - return 0; + return PyModule_Add(m, "_have_functions", list); } diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 3607855dbd8f2..912710219bd01 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1790,11 +1790,9 @@ init_timezone(PyObject *m) return -1; } #endif // MS_WINDOWS - PyObject *tzname_obj = Py_BuildValue("(NN)", otz0, otz1); - if (tzname_obj == NULL) { + if (PyModule_Add(m, "tzname", Py_BuildValue("(NN)", otz0, otz1)) < 0) { return -1; } - PyModule_AddObject(m, "tzname", tzname_obj); #else // !HAVE_DECL_TZNAME static const time_t YEAR = (365 * 24 + 6) * 3600; time_t t; @@ -1837,10 +1835,9 @@ init_timezone(PyObject *m) PyModule_AddIntConstant(m, "daylight", janzone != julyzone); tzname_obj = Py_BuildValue("(zz)", janname, julyname); } - if (tzname_obj == NULL) { + if (PyModule_Add(m, "tzname", tzname_obj) < 0) { return -1; } - PyModule_AddObject(m, "tzname", tzname_obj); #endif // !HAVE_DECL_TZNAME if (PyErr_Occurred()) { From webhook-mailer at python.org Tue Jul 18 04:50:20 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 18 Jul 2023 08:50:20 -0000 Subject: [Python-checkins] Docs: Normalise Argument Clinic advanced topics headings (#106842) Message-ID: https://github.com/python/cpython/commit/4cb0b9c0a9f6a4154238c98013d2679229b1f794 commit: 4cb0b9c0a9f6a4154238c98013d2679229b1f794 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-18T10:50:17+02:00 summary: Docs: Normalise Argument Clinic advanced topics headings (#106842) Co-authored-by: Ezio Melotti files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 0f99cb64994ab..12d7a77d43209 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -560,15 +560,12 @@ Let's dive in! Congratulations, you've ported your first function to work with Argument Clinic! -Advanced topics -=============== +How-to guides +============= -Now that you've had some experience working with Argument Clinic, it's time -for some advanced topics. - -Symbolic default values ------------------------ +How to use symbolic default values +---------------------------------- The default value you provide for a parameter can't be any arbitrary expression. Currently the following are explicitly supported: @@ -583,8 +580,8 @@ expression. Currently the following are explicitly supported: to allow full expressions like ``CONSTANT - 1``.) -Renaming the C functions and variables generated by Argument Clinic -------------------------------------------------------------------- +How to to rename C functions and variables generated by Argument Clinic +----------------------------------------------------------------------- Argument Clinic automatically names the functions it generates for you. Occasionally this may cause a problem, if the generated name collides with @@ -626,8 +623,8 @@ array) would be ``file``, but the C variable would be named ``file_obj``. You can use this to rename the ``self`` parameter too! -Converting functions using PyArg_UnpackTuple --------------------------------------------- +How to convert functions using ``PyArg_UnpackTuple`` +---------------------------------------------------- To convert a function parsing its arguments with :c:func:`PyArg_UnpackTuple`, simply write out all the arguments, specifying each as an ``object``. You @@ -639,8 +636,8 @@ Currently the generated code will use :c:func:`PyArg_ParseTuple`, but this will change soon. -Optional groups ---------------- +How to use optional groups +-------------------------- Some legacy functions have a tricky approach to parsing their arguments: they count the number of positional arguments, then use a ``switch`` statement @@ -732,8 +729,8 @@ Notes: use optional groups for new code. -Using real Argument Clinic converters, instead of "legacy converters" ---------------------------------------------------------------------- +How to use real Argument Clinic converters, instead of "legacy converters" +-------------------------------------------------------------------------- To save time, and to minimize how much you need to learn to achieve your first port to Argument Clinic, the walkthrough above tells @@ -903,8 +900,8 @@ it accepts, along with the default value for each parameter. Just run ``Tools/clinic/clinic.py --converters`` to see the full list. -Py_buffer ---------- +How to use the ``Py_buffer`` converter +-------------------------------------- When using the ``Py_buffer`` converter (or the ``'s*'``, ``'w*'``, ``'*y'``, or ``'z*'`` legacy converters), @@ -912,8 +909,8 @@ you *must* not call :c:func:`PyBuffer_Release` on the provided buffer. Argument Clinic generates code that does it for you (in the parsing function). -Advanced converters -------------------- +How to use advanced converters +------------------------------ Remember those format units you skipped for your first time because they were advanced? Here's how to handle those too. @@ -944,8 +941,8 @@ hard-coded encoding strings for parameters whose format units start with ``e``. .. _default_values: -Parameter default values ------------------------- +How to assign default values to parameter +----------------------------------------- Default values for parameters can be any of a number of values. At their simplest, they can be string, int, or float literals: @@ -968,8 +965,8 @@ There's also special support for a default value of ``NULL``, and for simple expressions, documented in the following sections. -The ``NULL`` default value --------------------------- +How to use the ``NULL`` default value +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For string and object parameters, you can set them to ``None`` to indicate that there's no default. However, that means the C variable will be @@ -979,8 +976,8 @@ behaves like a default value of ``None``, but the C variable is initialized with ``NULL``. -Expressions specified as default values ---------------------------------------- +How to use expressions as default values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The default value for a parameter can be more than just a literal value. It can be an entire expression, using math operators and looking up attributes @@ -1036,8 +1033,8 @@ you're not permitted to use: * Tuple/list/set/dict literals. -Using a return converter ------------------------- +How to use return converters +---------------------------- By default, the impl function Argument Clinic generates for you returns :c:type:`PyObject * `. @@ -1106,8 +1103,8 @@ their parameters (if any), just run ``Tools/clinic/clinic.py --converters`` for the full list. -Cloning existing functions --------------------------- +How to clone existing functions +------------------------------- If you have a number of functions that look similar, you may be able to use Clinic's "clone" feature. When you clone an existing function, @@ -1150,8 +1147,8 @@ Also, the function you are cloning from must have been previously defined in the current file. -Calling Python code -------------------- +How to call Python code +----------------------- The rest of the advanced topics require you to write Python code which lives inside your C file and modifies Argument Clinic's @@ -1178,8 +1175,8 @@ variable to the C code:: /*[python checksum:...]*/ -Using a "self converter" ------------------------- +How to use the "self converter" +------------------------------- Argument Clinic automatically adds a "self" parameter for you using a default converter. It automatically sets the ``type`` @@ -1230,8 +1227,8 @@ type for ``self``, it's best to create your own converter, subclassing [clinic start generated code]*/ -Using a "defining class" converter ----------------------------------- +How to use the "defining class" converter +----------------------------------------- Argument Clinic facilitates gaining access to the defining class of a method. This is useful for :ref:`heap type ` methods that need to fetch @@ -1293,8 +1290,8 @@ state. Example from the ``setattro`` slot method in See also :pep:`573`. -Writing a custom converter --------------------------- +How to write a custom converter +------------------------------- As we hinted at in the previous section... you can write your own converters! A converter is simply a Python class that inherits from ``CConverter``. @@ -1385,8 +1382,8 @@ You can see more examples of custom converters in the CPython source tree; grep the C files for the string ``CConverter``. -Writing a custom return converter ---------------------------------- +How to write a custom return converter +-------------------------------------- Writing a custom return converter is much like writing a custom converter. Except it's somewhat simpler, because return @@ -1400,8 +1397,8 @@ specifically the implementation of ``CReturnConverter`` and all its subclasses. -METH_O and METH_NOARGS ----------------------- +How to convert ``METH_O`` and ``METH_NOARGS`` functions +------------------------------------------------------- To convert a function using ``METH_O``, make sure the function's single argument is using the ``object`` converter, and mark the @@ -1422,8 +1419,8 @@ You can still use a self converter, a return converter, and specify a ``type`` argument to the object converter for ``METH_O``. -tp_new and tp_init functions ----------------------------- +How to convert ``tp_new`` and ``tp_init`` functions +--------------------------------------------------- You can convert ``tp_new`` and ``tp_init`` functions. Just name them ``__new__`` or ``__init__`` as appropriate. Notes: @@ -1445,8 +1442,8 @@ them ``__new__`` or ``__init__`` as appropriate. Notes: generated will throw an exception if it receives any.) -Changing and redirecting Clinic's output ----------------------------------------- +How to change and redirect Clinic's output +------------------------------------------ It can be inconvenient to have Clinic's output interspersed with your conventional hand-edited C code. Luckily, Clinic is configurable: @@ -1728,8 +1725,8 @@ it in a Clinic block lets Clinic use its existing checksum functionality to ensu the file was not modified by hand before it gets overwritten. -The #ifdef trick ----------------- +How to use the ``#ifdef`` trick +------------------------------- If you're converting a function that isn't available on all platforms, there's a trick you can use to make life a little easier. The existing @@ -1809,8 +1806,8 @@ Argument Clinic added to your file (it'll be at the very bottom), then move it above the ``PyMethodDef`` structure where that macro is used. -Using Argument Clinic in Python files -------------------------------------- +How to use Argument Clinic in Python files +------------------------------------------ It's actually possible to use Argument Clinic to preprocess Python files. There's no point to using Argument Clinic blocks, of course, as the output From webhook-mailer at python.org Tue Jul 18 05:53:21 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 09:53:21 -0000 Subject: [Python-checkins] [3.11] gh-86493: Fix possible leaks in modules initialization: _curses_panel, _decimal, posix, xxsubtype (GH-106767) (GH-106849) (GH-106851) Message-ID: https://github.com/python/cpython/commit/3841af49e5ba1bcbcccb4c5a4b386cbf13874c90 commit: 3841af49e5ba1bcbcccb4c5a4b386cbf13874c90 branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-18T12:53:17+03:00 summary: [3.11] gh-86493: Fix possible leaks in modules initialization: _curses_panel, _decimal, posix, xxsubtype (GH-106767) (GH-106849) (GH-106851) (cherry picked from commit 745492355b94d109e47827e5865846f25ae42d26) (cherry picked from commit 970cb8eabaaf5a8311f1aba4ca4968ef7385fce8) files: M Modules/_curses_panel.c M Modules/_decimal/_decimal.c M Modules/posixmodule.c M Modules/xxsubtype.c diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c index 0b328f9665a0a..c4be9aeaaee3b 100644 --- a/Modules/_curses_panel.c +++ b/Modules/_curses_panel.c @@ -670,9 +670,7 @@ _curses_panel_exec(PyObject *mod) state->PyCursesError = PyErr_NewException( "_curses_panel.error", NULL, NULL); - Py_INCREF(state->PyCursesError); - if (PyModule_AddObject(mod, "error", state->PyCursesError) < 0) { - Py_DECREF(state->PyCursesError); + if (PyModule_AddObjectRef(mod, "error", state->PyCursesError) < 0) { return -1; } diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 0e7d379e006cb..5cacdca3f44ba 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -5922,21 +5922,15 @@ PyInit__decimal(void) /* Add types to the module */ - Py_INCREF(&PyDec_Type); - CHECK_INT(PyModule_AddObject(m, "Decimal", (PyObject *)&PyDec_Type)); - Py_INCREF(&PyDecContext_Type); - CHECK_INT(PyModule_AddObject(m, "Context", - (PyObject *)&PyDecContext_Type)); - Py_INCREF(DecimalTuple); - CHECK_INT(PyModule_AddObject(m, "DecimalTuple", (PyObject *)DecimalTuple)); - + CHECK_INT(PyModule_AddObjectRef(m, "Decimal", (PyObject *)&PyDec_Type)); + CHECK_INT(PyModule_AddObjectRef(m, "Context", (PyObject *)&PyDecContext_Type)); + CHECK_INT(PyModule_AddObjectRef(m, "DecimalTuple", (PyObject *)DecimalTuple)); /* Create top level exception */ ASSIGN_PTR(DecimalException, PyErr_NewException( "decimal.DecimalException", PyExc_ArithmeticError, NULL)); - Py_INCREF(DecimalException); - CHECK_INT(PyModule_AddObject(m, "DecimalException", DecimalException)); + CHECK_INT(PyModule_AddObjectRef(m, "DecimalException", DecimalException)); /* Create signal tuple */ ASSIGN_PTR(SignalTuple, PyTuple_New(SIGNAL_MAP_LEN)); @@ -5976,8 +5970,7 @@ PyInit__decimal(void) Py_DECREF(base); /* add to module */ - Py_INCREF(cm->ex); - CHECK_INT(PyModule_AddObject(m, cm->name, cm->ex)); + CHECK_INT(PyModule_AddObjectRef(m, cm->name, cm->ex)); /* add to signal tuple */ Py_INCREF(cm->ex); @@ -6007,45 +6000,39 @@ PyInit__decimal(void) ASSIGN_PTR(cm->ex, PyErr_NewException(cm->fqname, base, NULL)); Py_DECREF(base); - Py_INCREF(cm->ex); - CHECK_INT(PyModule_AddObject(m, cm->name, cm->ex)); + CHECK_INT(PyModule_AddObjectRef(m, cm->name, cm->ex)); } /* Init default context template first */ ASSIGN_PTR(default_context_template, PyObject_CallObject((PyObject *)&PyDecContext_Type, NULL)); - Py_INCREF(default_context_template); - CHECK_INT(PyModule_AddObject(m, "DefaultContext", - default_context_template)); + CHECK_INT(PyModule_AddObjectRef(m, "DefaultContext", + default_context_template)); #ifndef WITH_DECIMAL_CONTEXTVAR ASSIGN_PTR(tls_context_key, PyUnicode_FromString("___DECIMAL_CTX__")); - Py_INCREF(Py_False); - CHECK_INT(PyModule_AddObject(m, "HAVE_CONTEXTVAR", Py_False)); + CHECK_INT(PyModule_AddObjectRef(m, "HAVE_CONTEXTVAR", Py_False)); #else ASSIGN_PTR(current_context_var, PyContextVar_New("decimal_context", NULL)); - Py_INCREF(Py_True); - CHECK_INT(PyModule_AddObject(m, "HAVE_CONTEXTVAR", Py_True)); + CHECK_INT(PyModule_AddObjectRef(m, "HAVE_CONTEXTVAR", Py_True)); #endif - Py_INCREF(Py_True); - CHECK_INT(PyModule_AddObject(m, "HAVE_THREADS", Py_True)); + CHECK_INT(PyModule_AddObjectRef(m, "HAVE_THREADS", Py_True)); /* Init basic context template */ ASSIGN_PTR(basic_context_template, PyObject_CallObject((PyObject *)&PyDecContext_Type, NULL)); init_basic_context(basic_context_template); - Py_INCREF(basic_context_template); - CHECK_INT(PyModule_AddObject(m, "BasicContext", - basic_context_template)); + CHECK_INT(PyModule_AddObjectRef(m, "BasicContext", + basic_context_template)); /* Init extended context template */ ASSIGN_PTR(extended_context_template, PyObject_CallObject((PyObject *)&PyDecContext_Type, NULL)); init_extended_context(extended_context_template); Py_INCREF(extended_context_template); - CHECK_INT(PyModule_AddObject(m, "ExtendedContext", - extended_context_template)); + CHECK_INT(PyModule_AddObjectRef(m, "ExtendedContext", + extended_context_template)); /* Init mpd_ssize_t constants */ @@ -6064,8 +6051,7 @@ PyInit__decimal(void) /* Init string constants */ for (i = 0; i < _PY_DEC_ROUND_GUARD; i++) { ASSIGN_PTR(round_map[i], PyUnicode_InternFromString(mpd_round_string[i])); - Py_INCREF(round_map[i]); - CHECK_INT(PyModule_AddObject(m, mpd_round_string[i], round_map[i])); + CHECK_INT(PyModule_AddObjectRef(m, mpd_round_string[i], round_map[i])); } /* Add specification version number */ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 7c367b9d2e981..91da07908f115 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -15849,42 +15849,34 @@ posixmodule_exec(PyObject *m) if (setup_confname_tables(m)) return -1; - Py_INCREF(PyExc_OSError); - PyModule_AddObject(m, "error", PyExc_OSError); + if (PyModule_AddObjectRef(m, "error", PyExc_OSError) < 0) { + return -1; + } #if defined(HAVE_WAITID) && !defined(__APPLE__) waitid_result_desc.name = MODNAME ".waitid_result"; - PyObject *WaitidResultType = (PyObject *)PyStructSequence_NewType(&waitid_result_desc); - if (WaitidResultType == NULL) { + state->WaitidResultType = (PyObject *)PyStructSequence_NewType(&waitid_result_desc); + if (PyModule_AddObjectRef(m, "waitid_result", state->WaitidResultType) < 0) { return -1; } - Py_INCREF(WaitidResultType); - PyModule_AddObject(m, "waitid_result", WaitidResultType); - state->WaitidResultType = WaitidResultType; #endif stat_result_desc.name = "os.stat_result"; /* see issue #19209 */ stat_result_desc.fields[7].name = PyStructSequence_UnnamedField; stat_result_desc.fields[8].name = PyStructSequence_UnnamedField; stat_result_desc.fields[9].name = PyStructSequence_UnnamedField; - PyObject *StatResultType = (PyObject *)PyStructSequence_NewType(&stat_result_desc); - if (StatResultType == NULL) { + state->StatResultType = (PyObject *)PyStructSequence_NewType(&stat_result_desc); + if (PyModule_AddObjectRef(m, "stat_result", state->StatResultType) < 0) { return -1; } - Py_INCREF(StatResultType); - PyModule_AddObject(m, "stat_result", StatResultType); - state->StatResultType = StatResultType; - structseq_new = ((PyTypeObject *)StatResultType)->tp_new; - ((PyTypeObject *)StatResultType)->tp_new = statresult_new; + structseq_new = ((PyTypeObject *)state->StatResultType)->tp_new; + ((PyTypeObject *)state->StatResultType)->tp_new = statresult_new; statvfs_result_desc.name = "os.statvfs_result"; /* see issue #19209 */ - PyObject *StatVFSResultType = (PyObject *)PyStructSequence_NewType(&statvfs_result_desc); - if (StatVFSResultType == NULL) { + state->StatVFSResultType = (PyObject *)PyStructSequence_NewType(&statvfs_result_desc); + if (PyModule_AddObjectRef(m, "statvfs_result", state->StatVFSResultType) < 0) { return -1; } - Py_INCREF(StatVFSResultType); - PyModule_AddObject(m, "statvfs_result", StatVFSResultType); - state->StatVFSResultType = StatVFSResultType; #ifdef NEED_TICKS_PER_SECOND # if defined(HAVE_SYSCONF) && defined(_SC_CLK_TCK) ticks_per_second = sysconf(_SC_CLK_TCK); @@ -15897,24 +15889,18 @@ posixmodule_exec(PyObject *m) #if defined(HAVE_SCHED_SETPARAM) || defined(HAVE_SCHED_SETSCHEDULER) || defined(POSIX_SPAWN_SETSCHEDULER) || defined(POSIX_SPAWN_SETSCHEDPARAM) sched_param_desc.name = MODNAME ".sched_param"; - PyObject *SchedParamType = (PyObject *)PyStructSequence_NewType(&sched_param_desc); - if (SchedParamType == NULL) { + state->SchedParamType = (PyObject *)PyStructSequence_NewType(&sched_param_desc); + if (PyModule_AddObjectRef(m, "sched_param", state->SchedParamType) < 0) { return -1; } - Py_INCREF(SchedParamType); - PyModule_AddObject(m, "sched_param", SchedParamType); - state->SchedParamType = SchedParamType; - ((PyTypeObject *)SchedParamType)->tp_new = os_sched_param; + ((PyTypeObject *)state->SchedParamType)->tp_new = os_sched_param; #endif /* initialize TerminalSize_info */ - PyObject *TerminalSizeType = (PyObject *)PyStructSequence_NewType(&TerminalSize_desc); - if (TerminalSizeType == NULL) { + state->TerminalSizeType = (PyObject *)PyStructSequence_NewType(&TerminalSize_desc); + if (PyModule_AddObjectRef(m, "terminal_size", state->TerminalSizeType) < 0) { return -1; } - Py_INCREF(TerminalSizeType); - PyModule_AddObject(m, "terminal_size", TerminalSizeType); - state->TerminalSizeType = TerminalSizeType; /* initialize scandir types */ PyObject *ScandirIteratorType = PyType_FromModuleAndSpec(m, &ScandirIteratorType_spec, NULL); @@ -15923,30 +15909,21 @@ posixmodule_exec(PyObject *m) } state->ScandirIteratorType = ScandirIteratorType; - PyObject *DirEntryType = PyType_FromModuleAndSpec(m, &DirEntryType_spec, NULL); - if (DirEntryType == NULL) { + state->DirEntryType = PyType_FromModuleAndSpec(m, &DirEntryType_spec, NULL); + if (PyModule_AddObjectRef(m, "DirEntry", state->DirEntryType) < 0) { return -1; } - Py_INCREF(DirEntryType); - PyModule_AddObject(m, "DirEntry", DirEntryType); - state->DirEntryType = DirEntryType; times_result_desc.name = MODNAME ".times_result"; - PyObject *TimesResultType = (PyObject *)PyStructSequence_NewType(×_result_desc); - if (TimesResultType == NULL) { + state->TimesResultType = (PyObject *)PyStructSequence_NewType(×_result_desc); + if (PyModule_AddObjectRef(m, "times_result", state->TimesResultType) < 0) { return -1; } - Py_INCREF(TimesResultType); - PyModule_AddObject(m, "times_result", TimesResultType); - state->TimesResultType = TimesResultType; - PyTypeObject *UnameResultType = PyStructSequence_NewType(&uname_result_desc); - if (UnameResultType == NULL) { + state->UnameResultType = (PyObject *)PyStructSequence_NewType(&uname_result_desc); + if (PyModule_AddObjectRef(m, "uname_result", state->UnameResultType) < 0) { return -1; } - Py_INCREF(UnameResultType); - PyModule_AddObject(m, "uname_result", (PyObject *)UnameResultType); - state->UnameResultType = (PyObject *)UnameResultType; if ((state->billion = PyLong_FromLong(1000000000)) == NULL) return -1; diff --git a/Modules/xxsubtype.c b/Modules/xxsubtype.c index 12306f2fc5247..7f3a13f32033a 100644 --- a/Modules/xxsubtype.c +++ b/Modules/xxsubtype.c @@ -279,14 +279,12 @@ xxsubtype_exec(PyObject* m) if (PyType_Ready(&spamdict_type) < 0) return -1; - Py_INCREF(&spamlist_type); - if (PyModule_AddObject(m, "spamlist", - (PyObject *) &spamlist_type) < 0) + if (PyModule_AddObjectRef(m, "spamlist", + (PyObject *) &spamlist_type) < 0) return -1; - Py_INCREF(&spamdict_type); - if (PyModule_AddObject(m, "spamdict", - (PyObject *) &spamdict_type) < 0) + if (PyModule_AddObjectRef(m, "spamdict", + (PyObject *) &spamdict_type) < 0) return -1; return 0; } From webhook-mailer at python.org Tue Jul 18 06:16:35 2023 From: webhook-mailer at python.org (methane) Date: Tue, 18 Jul 2023 10:16:35 -0000 Subject: [Python-checkins] gh-106751: selectors: optimize EpollSelector.select() (#106754) Message-ID: https://github.com/python/cpython/commit/aecf6aca515a203a823a87c711f15cbb82097c8b commit: aecf6aca515a203a823a87c711f15cbb82097c8b branch: main author: J. Nick Koston committer: methane date: 2023-07-18T10:16:32Z summary: gh-106751: selectors: optimize EpollSelector.select() (#106754) Co-authored-by: Pieter Eendebak files: A Misc/NEWS.d/next/Library/2023-07-14-20-31-09.gh-issue-106751.52F6yQ.rst M Lib/selectors.py diff --git a/Lib/selectors.py b/Lib/selectors.py index 6d82935445d4b..a42d156340641 100644 --- a/Lib/selectors.py +++ b/Lib/selectors.py @@ -430,6 +430,9 @@ class PollSelector(_PollLikeSelector): if hasattr(select, 'epoll'): + _NOT_EPOLLIN = ~select.EPOLLIN + _NOT_EPOLLOUT = ~select.EPOLLOUT + class EpollSelector(_PollLikeSelector): """Epoll-based selector.""" _selector_cls = select.epoll @@ -452,22 +455,20 @@ def select(self, timeout=None): # epoll_wait() expects `maxevents` to be greater than zero; # we want to make sure that `select()` can be called when no # FD is registered. - max_ev = max(len(self._fd_to_key), 1) + max_ev = len(self._fd_to_key) or 1 ready = [] try: fd_event_list = self._selector.poll(timeout, max_ev) except InterruptedError: return ready - for fd, event in fd_event_list: - events = 0 - if event & ~select.EPOLLIN: - events |= EVENT_WRITE - if event & ~select.EPOLLOUT: - events |= EVENT_READ - key = self._fd_to_key.get(fd) + fd_to_key = self._fd_to_key + for fd, event in fd_event_list: + key = fd_to_key.get(fd) if key: + events = ((event & _NOT_EPOLLIN and EVENT_WRITE) + | (event & _NOT_EPOLLOUT and EVENT_READ)) ready.append((key, events & key.events)) return ready diff --git a/Misc/NEWS.d/next/Library/2023-07-14-20-31-09.gh-issue-106751.52F6yQ.rst b/Misc/NEWS.d/next/Library/2023-07-14-20-31-09.gh-issue-106751.52F6yQ.rst new file mode 100644 index 0000000000000..486b1f9bbd0a9 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-14-20-31-09.gh-issue-106751.52F6yQ.rst @@ -0,0 +1 @@ +:mod:`selectors`: Optimize ``EpollSelector.select()`` code by moving some code outside of the loop. From webhook-mailer at python.org Tue Jul 18 06:24:48 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 18 Jul 2023 10:24:48 -0000 Subject: [Python-checkins] [3.12] Docs: Normalise Argument Clinic advanced topics headings (GH-106842) (#106853) Message-ID: https://github.com/python/cpython/commit/b79f3b36c318be8b27d1737a819e33145193801c commit: b79f3b36c318be8b27d1737a819e33145193801c branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-18T12:24:44+02:00 summary: [3.12] Docs: Normalise Argument Clinic advanced topics headings (GH-106842) (#106853) (cherry picked from commit 4cb0b9c0a9f6a4154238c98013d2679229b1f794) Co-authored-by: Erlend E. Aasland Co-authored-by: Ezio Melotti files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 0f99cb64994ab..12d7a77d43209 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -560,15 +560,12 @@ Let's dive in! Congratulations, you've ported your first function to work with Argument Clinic! -Advanced topics -=============== +How-to guides +============= -Now that you've had some experience working with Argument Clinic, it's time -for some advanced topics. - -Symbolic default values ------------------------ +How to use symbolic default values +---------------------------------- The default value you provide for a parameter can't be any arbitrary expression. Currently the following are explicitly supported: @@ -583,8 +580,8 @@ expression. Currently the following are explicitly supported: to allow full expressions like ``CONSTANT - 1``.) -Renaming the C functions and variables generated by Argument Clinic -------------------------------------------------------------------- +How to to rename C functions and variables generated by Argument Clinic +----------------------------------------------------------------------- Argument Clinic automatically names the functions it generates for you. Occasionally this may cause a problem, if the generated name collides with @@ -626,8 +623,8 @@ array) would be ``file``, but the C variable would be named ``file_obj``. You can use this to rename the ``self`` parameter too! -Converting functions using PyArg_UnpackTuple --------------------------------------------- +How to convert functions using ``PyArg_UnpackTuple`` +---------------------------------------------------- To convert a function parsing its arguments with :c:func:`PyArg_UnpackTuple`, simply write out all the arguments, specifying each as an ``object``. You @@ -639,8 +636,8 @@ Currently the generated code will use :c:func:`PyArg_ParseTuple`, but this will change soon. -Optional groups ---------------- +How to use optional groups +-------------------------- Some legacy functions have a tricky approach to parsing their arguments: they count the number of positional arguments, then use a ``switch`` statement @@ -732,8 +729,8 @@ Notes: use optional groups for new code. -Using real Argument Clinic converters, instead of "legacy converters" ---------------------------------------------------------------------- +How to use real Argument Clinic converters, instead of "legacy converters" +-------------------------------------------------------------------------- To save time, and to minimize how much you need to learn to achieve your first port to Argument Clinic, the walkthrough above tells @@ -903,8 +900,8 @@ it accepts, along with the default value for each parameter. Just run ``Tools/clinic/clinic.py --converters`` to see the full list. -Py_buffer ---------- +How to use the ``Py_buffer`` converter +-------------------------------------- When using the ``Py_buffer`` converter (or the ``'s*'``, ``'w*'``, ``'*y'``, or ``'z*'`` legacy converters), @@ -912,8 +909,8 @@ you *must* not call :c:func:`PyBuffer_Release` on the provided buffer. Argument Clinic generates code that does it for you (in the parsing function). -Advanced converters -------------------- +How to use advanced converters +------------------------------ Remember those format units you skipped for your first time because they were advanced? Here's how to handle those too. @@ -944,8 +941,8 @@ hard-coded encoding strings for parameters whose format units start with ``e``. .. _default_values: -Parameter default values ------------------------- +How to assign default values to parameter +----------------------------------------- Default values for parameters can be any of a number of values. At their simplest, they can be string, int, or float literals: @@ -968,8 +965,8 @@ There's also special support for a default value of ``NULL``, and for simple expressions, documented in the following sections. -The ``NULL`` default value --------------------------- +How to use the ``NULL`` default value +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For string and object parameters, you can set them to ``None`` to indicate that there's no default. However, that means the C variable will be @@ -979,8 +976,8 @@ behaves like a default value of ``None``, but the C variable is initialized with ``NULL``. -Expressions specified as default values ---------------------------------------- +How to use expressions as default values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The default value for a parameter can be more than just a literal value. It can be an entire expression, using math operators and looking up attributes @@ -1036,8 +1033,8 @@ you're not permitted to use: * Tuple/list/set/dict literals. -Using a return converter ------------------------- +How to use return converters +---------------------------- By default, the impl function Argument Clinic generates for you returns :c:type:`PyObject * `. @@ -1106,8 +1103,8 @@ their parameters (if any), just run ``Tools/clinic/clinic.py --converters`` for the full list. -Cloning existing functions --------------------------- +How to clone existing functions +------------------------------- If you have a number of functions that look similar, you may be able to use Clinic's "clone" feature. When you clone an existing function, @@ -1150,8 +1147,8 @@ Also, the function you are cloning from must have been previously defined in the current file. -Calling Python code -------------------- +How to call Python code +----------------------- The rest of the advanced topics require you to write Python code which lives inside your C file and modifies Argument Clinic's @@ -1178,8 +1175,8 @@ variable to the C code:: /*[python checksum:...]*/ -Using a "self converter" ------------------------- +How to use the "self converter" +------------------------------- Argument Clinic automatically adds a "self" parameter for you using a default converter. It automatically sets the ``type`` @@ -1230,8 +1227,8 @@ type for ``self``, it's best to create your own converter, subclassing [clinic start generated code]*/ -Using a "defining class" converter ----------------------------------- +How to use the "defining class" converter +----------------------------------------- Argument Clinic facilitates gaining access to the defining class of a method. This is useful for :ref:`heap type ` methods that need to fetch @@ -1293,8 +1290,8 @@ state. Example from the ``setattro`` slot method in See also :pep:`573`. -Writing a custom converter --------------------------- +How to write a custom converter +------------------------------- As we hinted at in the previous section... you can write your own converters! A converter is simply a Python class that inherits from ``CConverter``. @@ -1385,8 +1382,8 @@ You can see more examples of custom converters in the CPython source tree; grep the C files for the string ``CConverter``. -Writing a custom return converter ---------------------------------- +How to write a custom return converter +-------------------------------------- Writing a custom return converter is much like writing a custom converter. Except it's somewhat simpler, because return @@ -1400,8 +1397,8 @@ specifically the implementation of ``CReturnConverter`` and all its subclasses. -METH_O and METH_NOARGS ----------------------- +How to convert ``METH_O`` and ``METH_NOARGS`` functions +------------------------------------------------------- To convert a function using ``METH_O``, make sure the function's single argument is using the ``object`` converter, and mark the @@ -1422,8 +1419,8 @@ You can still use a self converter, a return converter, and specify a ``type`` argument to the object converter for ``METH_O``. -tp_new and tp_init functions ----------------------------- +How to convert ``tp_new`` and ``tp_init`` functions +--------------------------------------------------- You can convert ``tp_new`` and ``tp_init`` functions. Just name them ``__new__`` or ``__init__`` as appropriate. Notes: @@ -1445,8 +1442,8 @@ them ``__new__`` or ``__init__`` as appropriate. Notes: generated will throw an exception if it receives any.) -Changing and redirecting Clinic's output ----------------------------------------- +How to change and redirect Clinic's output +------------------------------------------ It can be inconvenient to have Clinic's output interspersed with your conventional hand-edited C code. Luckily, Clinic is configurable: @@ -1728,8 +1725,8 @@ it in a Clinic block lets Clinic use its existing checksum functionality to ensu the file was not modified by hand before it gets overwritten. -The #ifdef trick ----------------- +How to use the ``#ifdef`` trick +------------------------------- If you're converting a function that isn't available on all platforms, there's a trick you can use to make life a little easier. The existing @@ -1809,8 +1806,8 @@ Argument Clinic added to your file (it'll be at the very bottom), then move it above the ``PyMethodDef`` structure where that macro is used. -Using Argument Clinic in Python files -------------------------------------- +How to use Argument Clinic in Python files +------------------------------------------ It's actually possible to use Argument Clinic to preprocess Python files. There's no point to using Argument Clinic blocks, of course, as the output From webhook-mailer at python.org Tue Jul 18 06:25:28 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 18 Jul 2023 10:25:28 -0000 Subject: [Python-checkins] [3.11] Docs: Normalise Argument Clinic advanced topics headings (GH-106842) (#106852) Message-ID: https://github.com/python/cpython/commit/38553442c88c331d91562b77f023345e0b33b5f9 commit: 38553442c88c331d91562b77f023345e0b33b5f9 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-18T12:25:25+02:00 summary: [3.11] Docs: Normalise Argument Clinic advanced topics headings (GH-106842) (#106852) (cherry picked from commit 4cb0b9c0a9f6a4154238c98013d2679229b1f794) Co-authored-by: Erlend E. Aasland Co-authored-by: Ezio Melotti files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 70b3e21d81a61..287a4acebb133 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -552,15 +552,12 @@ Let's dive in! Congratulations, you've ported your first function to work with Argument Clinic! -Advanced topics -=============== +How-to guides +============= -Now that you've had some experience working with Argument Clinic, it's time -for some advanced topics. - -Symbolic default values ------------------------ +How to use symbolic default values +---------------------------------- The default value you provide for a parameter can't be any arbitrary expression. Currently the following are explicitly supported: @@ -575,8 +572,8 @@ expression. Currently the following are explicitly supported: to allow full expressions like ``CONSTANT - 1``.) -Renaming the C functions and variables generated by Argument Clinic -------------------------------------------------------------------- +How to to rename C functions and variables generated by Argument Clinic +----------------------------------------------------------------------- Argument Clinic automatically names the functions it generates for you. Occasionally this may cause a problem, if the generated name collides with @@ -618,8 +615,8 @@ array) would be ``file``, but the C variable would be named ``file_obj``. You can use this to rename the ``self`` parameter too! -Converting functions using PyArg_UnpackTuple --------------------------------------------- +How to convert functions using ``PyArg_UnpackTuple`` +---------------------------------------------------- To convert a function parsing its arguments with :c:func:`PyArg_UnpackTuple`, simply write out all the arguments, specifying each as an ``object``. You @@ -631,8 +628,8 @@ Currently the generated code will use :c:func:`PyArg_ParseTuple`, but this will change soon. -Optional groups ---------------- +How to use optional groups +-------------------------- Some legacy functions have a tricky approach to parsing their arguments: they count the number of positional arguments, then use a ``switch`` statement @@ -724,8 +721,8 @@ Notes: use optional groups for new code. -Using real Argument Clinic converters, instead of "legacy converters" ---------------------------------------------------------------------- +How to use real Argument Clinic converters, instead of "legacy converters" +-------------------------------------------------------------------------- To save time, and to minimize how much you need to learn to achieve your first port to Argument Clinic, the walkthrough above tells @@ -892,8 +889,8 @@ it accepts, along with the default value for each parameter. Just run ``Tools/clinic/clinic.py --converters`` to see the full list. -Py_buffer ---------- +How to use the ``Py_buffer`` converter +-------------------------------------- When using the ``Py_buffer`` converter (or the ``'s*'``, ``'w*'``, ``'*y'``, or ``'z*'`` legacy converters), @@ -901,8 +898,8 @@ you *must* not call :c:func:`PyBuffer_Release` on the provided buffer. Argument Clinic generates code that does it for you (in the parsing function). -Advanced converters -------------------- +How to use advanced converters +------------------------------ Remember those format units you skipped for your first time because they were advanced? Here's how to handle those too. @@ -933,8 +930,8 @@ hard-coded encoding strings for parameters whose format units start with ``e``. .. _default_values: -Parameter default values ------------------------- +How to assign default values to parameter +----------------------------------------- Default values for parameters can be any of a number of values. At their simplest, they can be string, int, or float literals: @@ -957,8 +954,8 @@ There's also special support for a default value of ``NULL``, and for simple expressions, documented in the following sections. -The ``NULL`` default value --------------------------- +How to use the ``NULL`` default value +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For string and object parameters, you can set them to ``None`` to indicate that there's no default. However, that means the C variable will be @@ -968,8 +965,8 @@ behaves like a default value of ``None``, but the C variable is initialized with ``NULL``. -Expressions specified as default values ---------------------------------------- +How to use expressions as default values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The default value for a parameter can be more than just a literal value. It can be an entire expression, using math operators and looking up attributes @@ -1025,8 +1022,8 @@ you're not permitted to use: * Tuple/list/set/dict literals. -Using a return converter ------------------------- +How to use return converters +---------------------------- By default, the impl function Argument Clinic generates for you returns :c:type:`PyObject * `. @@ -1100,8 +1097,8 @@ their parameters (if any), just run ``Tools/clinic/clinic.py --converters`` for the full list. -Cloning existing functions --------------------------- +How to clone existing functions +------------------------------- If you have a number of functions that look similar, you may be able to use Clinic's "clone" feature. When you clone an existing function, @@ -1144,8 +1141,8 @@ Also, the function you are cloning from must have been previously defined in the current file. -Calling Python code -------------------- +How to call Python code +----------------------- The rest of the advanced topics require you to write Python code which lives inside your C file and modifies Argument Clinic's @@ -1172,8 +1169,8 @@ variable to the C code:: /*[python checksum:...]*/ -Using a "self converter" ------------------------- +How to use the "self converter" +------------------------------- Argument Clinic automatically adds a "self" parameter for you using a default converter. It automatically sets the ``type`` @@ -1224,8 +1221,8 @@ type for ``self``, it's best to create your own converter, subclassing [clinic start generated code]*/ -Using a "defining class" converter ----------------------------------- +How to use the "defining class" converter +----------------------------------------- Argument Clinic facilitates gaining access to the defining class of a method. This is useful for :ref:`heap type ` methods that need to fetch @@ -1287,8 +1284,8 @@ state. Example from the ``setattro`` slot method in See also :pep:`573`. -Writing a custom converter --------------------------- +How to write a custom converter +------------------------------- As we hinted at in the previous section... you can write your own converters! A converter is simply a Python class that inherits from ``CConverter``. @@ -1379,8 +1376,8 @@ You can see more examples of custom converters in the CPython source tree; grep the C files for the string ``CConverter``. -Writing a custom return converter ---------------------------------- +How to write a custom return converter +-------------------------------------- Writing a custom return converter is much like writing a custom converter. Except it's somewhat simpler, because return @@ -1394,8 +1391,8 @@ specifically the implementation of ``CReturnConverter`` and all its subclasses. -METH_O and METH_NOARGS ----------------------- +How to convert ``METH_O`` and ``METH_NOARGS`` functions +------------------------------------------------------- To convert a function using ``METH_O``, make sure the function's single argument is using the ``object`` converter, and mark the @@ -1416,8 +1413,8 @@ You can still use a self converter, a return converter, and specify a ``type`` argument to the object converter for ``METH_O``. -tp_new and tp_init functions ----------------------------- +How to convert ``tp_new`` and ``tp_init`` functions +--------------------------------------------------- You can convert ``tp_new`` and ``tp_init`` functions. Just name them ``__new__`` or ``__init__`` as appropriate. Notes: @@ -1439,8 +1436,8 @@ them ``__new__`` or ``__init__`` as appropriate. Notes: generated will throw an exception if it receives any.) -Changing and redirecting Clinic's output ----------------------------------------- +How to change and redirect Clinic's output +------------------------------------------ It can be inconvenient to have Clinic's output interspersed with your conventional hand-edited C code. Luckily, Clinic is configurable: @@ -1722,8 +1719,8 @@ it in a Clinic block lets Clinic use its existing checksum functionality to ensu the file was not modified by hand before it gets overwritten. -The #ifdef trick ----------------- +How to use the ``#ifdef`` trick +------------------------------- If you're converting a function that isn't available on all platforms, there's a trick you can use to make life a little easier. The existing @@ -1803,8 +1800,8 @@ Argument Clinic added to your file (it'll be at the very bottom), then move it above the ``PyMethodDef`` structure where that macro is used. -Using Argument Clinic in Python files -------------------------------------- +How to use Argument Clinic in Python files +------------------------------------------ It's actually possible to use Argument Clinic to preprocess Python files. There's no point to using Argument Clinic blocks, of course, as the output From webhook-mailer at python.org Tue Jul 18 08:14:13 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 12:14:13 -0000 Subject: [Python-checkins] [3.12] gh-86493: Fix possible leaks in some modules initialization (GH-106768) (GH-106855) Message-ID: https://github.com/python/cpython/commit/a423ddbdeada8a2fd8657453b9e9f58ba0dd921d commit: a423ddbdeada8a2fd8657453b9e9f58ba0dd921d branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-18T15:14:10+03:00 summary: [3.12] gh-86493: Fix possible leaks in some modules initialization (GH-106768) (GH-106855) Fix _ssl, _stat, _testinternalcapi, _threadmodule, cmath, math, posix, time. (cherry picked from commit 3e65baee72131b49f4ce8ca2da568a6f2001ce93) files: M Include/cpython/modsupport.h M Modules/_ssl.c M Modules/_stat.c M Modules/_testinternalcapi.c M Modules/_threadmodule.c M Modules/cmathmodule.c M Modules/mathmodule.c M Modules/posixmodule.c M Modules/timemodule.c M Python/modsupport.c diff --git a/Include/cpython/modsupport.h b/Include/cpython/modsupport.h index 88f34fe7513bf..2259291aff67d 100644 --- a/Include/cpython/modsupport.h +++ b/Include/cpython/modsupport.h @@ -106,3 +106,4 @@ PyAPI_FUNC(PyObject * const *) _PyArg_UnpackKeywordsWithVararg( (minpos), (maxpos), (minkw), (buf))) PyAPI_FUNC(PyObject *) _PyModule_CreateInitialized(PyModuleDef*, int apiver); +PyAPI_FUNC(int) _PyModule_Add(PyObject *, const char *, PyObject *); diff --git a/Modules/_ssl.c b/Modules/_ssl.c index a3fb12e483750..619b4f4e94d06 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -6095,22 +6095,22 @@ sslmodule_init_versioninfo(PyObject *m) */ libver = OpenSSL_version_num(); r = PyLong_FromUnsignedLong(libver); - if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION_NUMBER", r)) + if (_PyModule_Add(m, "OPENSSL_VERSION_NUMBER", r) < 0) return -1; parse_openssl_version(libver, &major, &minor, &fix, &patch, &status); r = Py_BuildValue("IIIII", major, minor, fix, patch, status); - if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION_INFO", r)) + if (_PyModule_Add(m, "OPENSSL_VERSION_INFO", r) < 0) return -1; r = PyUnicode_FromString(OpenSSL_version(OPENSSL_VERSION)); - if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION", r)) + if (_PyModule_Add(m, "OPENSSL_VERSION", r) < 0) return -1; libver = OPENSSL_VERSION_NUMBER; parse_openssl_version(libver, &major, &minor, &fix, &patch, &status); r = Py_BuildValue("IIIII", major, minor, fix, patch, status); - if (r == NULL || PyModule_AddObject(m, "_OPENSSL_API_VERSION", r)) + if (_PyModule_Add(m, "_OPENSSL_API_VERSION", r) < 0) return -1; return 0; diff --git a/Modules/_stat.c b/Modules/_stat.c index 4218799103b59..4ec2bd251831f 100644 --- a/Modules/_stat.c +++ b/Modules/_stat.c @@ -592,17 +592,17 @@ stat_exec(PyObject *module) ADD_INT_MACRO(module, FILE_ATTRIBUTE_TEMPORARY); ADD_INT_MACRO(module, FILE_ATTRIBUTE_VIRTUAL); - if (PyModule_AddObject(module, "IO_REPARSE_TAG_SYMLINK", - PyLong_FromUnsignedLong(IO_REPARSE_TAG_SYMLINK)) < 0) { - return -1; + if (_PyModule_Add(module, "IO_REPARSE_TAG_SYMLINK", + PyLong_FromUnsignedLong(IO_REPARSE_TAG_SYMLINK)) < 0) { + return -1; } - if (PyModule_AddObject(module, "IO_REPARSE_TAG_MOUNT_POINT", - PyLong_FromUnsignedLong(IO_REPARSE_TAG_MOUNT_POINT)) < 0) { - return -1; + if (_PyModule_Add(module, "IO_REPARSE_TAG_MOUNT_POINT", + PyLong_FromUnsignedLong(IO_REPARSE_TAG_MOUNT_POINT)) < 0) { + return -1; } - if (PyModule_AddObject(module, "IO_REPARSE_TAG_APPEXECLINK", - PyLong_FromUnsignedLong(IO_REPARSE_TAG_APPEXECLINK)) < 0) { - return -1; + if (_PyModule_Add(module, "IO_REPARSE_TAG_APPEXECLINK", + PyLong_FromUnsignedLong(IO_REPARSE_TAG_APPEXECLINK)) < 0) { + return -1; } #endif diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 98f99e4d66cf1..6fc31f61c144d 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -1092,8 +1092,8 @@ static PyMethodDef module_functions[] = { static int module_exec(PyObject *module) { - if (PyModule_AddObject(module, "SIZEOF_PYGC_HEAD", - PyLong_FromSsize_t(sizeof(PyGC_Head))) < 0) { + if (_PyModule_Add(module, "SIZEOF_PYGC_HEAD", + PyLong_FromSsize_t(sizeof(PyGC_Head))) < 0) { return 1; } diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index c553d039462af..04f4400a9315b 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -1671,8 +1671,8 @@ thread_module_exec(PyObject *module) // Round towards minus infinity timeout_max = floor(timeout_max); - if (PyModule_AddObject(module, "TIMEOUT_MAX", - PyFloat_FromDouble(timeout_max)) < 0) { + if (_PyModule_Add(module, "TIMEOUT_MAX", + PyFloat_FromDouble(timeout_max)) < 0) { return -1; } diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c index 1a31bdc824bb0..25491e655849d 100644 --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -1216,30 +1216,29 @@ static PyMethodDef cmath_methods[] = { static int cmath_exec(PyObject *mod) { - if (PyModule_AddObject(mod, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { + if (_PyModule_Add(mod, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { return -1; } - if (PyModule_AddObject(mod, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { + if (_PyModule_Add(mod, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { return -1; } // 2pi - if (PyModule_AddObject(mod, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { + if (_PyModule_Add(mod, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { return -1; } - if (PyModule_AddObject(mod, "inf", PyFloat_FromDouble(Py_INFINITY)) < 0) { + if (_PyModule_Add(mod, "inf", PyFloat_FromDouble(Py_INFINITY)) < 0) { return -1; } Py_complex infj = {0.0, Py_INFINITY}; - if (PyModule_AddObject(mod, "infj", - PyComplex_FromCComplex(infj)) < 0) { + if (_PyModule_Add(mod, "infj", PyComplex_FromCComplex(infj)) < 0) { return -1; } - if (PyModule_AddObject(mod, "nan", PyFloat_FromDouble(fabs(Py_NAN))) < 0) { + if (_PyModule_Add(mod, "nan", PyFloat_FromDouble(fabs(Py_NAN))) < 0) { return -1; } Py_complex nanj = {0.0, fabs(Py_NAN)}; - if (PyModule_AddObject(mod, "nanj", PyComplex_FromCComplex(nanj)) < 0) { + if (_PyModule_Add(mod, "nanj", PyComplex_FromCComplex(nanj)) < 0) { return -1; } diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index f5679fe3a6f36..7b1104ba5ac40 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -4037,20 +4037,20 @@ math_exec(PyObject *module) if (state->str___trunc__ == NULL) { return -1; } - if (PyModule_AddObject(module, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { + if (_PyModule_Add(module, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { return -1; } - if (PyModule_AddObject(module, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { + if (_PyModule_Add(module, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { return -1; } // 2pi - if (PyModule_AddObject(module, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { + if (_PyModule_Add(module, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { return -1; } - if (PyModule_AddObject(module, "inf", PyFloat_FromDouble(Py_INFINITY)) < 0) { + if (_PyModule_Add(module, "inf", PyFloat_FromDouble(Py_INFINITY)) < 0) { return -1; } - if (PyModule_AddObject(module, "nan", PyFloat_FromDouble(fabs(Py_NAN))) < 0) { + if (_PyModule_Add(module, "nan", PyFloat_FromDouble(fabs(Py_NAN))) < 0) { return -1; } return 0; diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 0f82f1afa07aa..fde1e4f461bc9 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -13463,7 +13463,7 @@ setup_confname_table(struct constdef *table, size_t tablesize, } Py_DECREF(o); } - return PyModule_AddObject(module, tablename, d); + return _PyModule_Add(module, tablename, d); } /* Return -1 on failure, 0 on success. */ @@ -16778,11 +16778,9 @@ posixmodule_exec(PyObject *m) #endif /* Initialize environ dictionary */ - PyObject *v = convertenviron(); - Py_XINCREF(v); - if (v == NULL || PyModule_AddObject(m, "environ", v) != 0) + if (_PyModule_Add(m, "environ", convertenviron()) != 0) { return -1; - Py_DECREF(v); + } if (all_ins(m)) return -1; @@ -16897,9 +16895,7 @@ posixmodule_exec(PyObject *m) Py_DECREF(unicode); } - PyModule_AddObject(m, "_have_functions", list); - - return 0; + return _PyModule_Add(m, "_have_functions", list); } diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 3607855dbd8f2..f5b0f39e14abc 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1790,11 +1790,9 @@ init_timezone(PyObject *m) return -1; } #endif // MS_WINDOWS - PyObject *tzname_obj = Py_BuildValue("(NN)", otz0, otz1); - if (tzname_obj == NULL) { + if (_PyModule_Add(m, "tzname", Py_BuildValue("(NN)", otz0, otz1)) < 0) { return -1; } - PyModule_AddObject(m, "tzname", tzname_obj); #else // !HAVE_DECL_TZNAME static const time_t YEAR = (365 * 24 + 6) * 3600; time_t t; @@ -1837,10 +1835,9 @@ init_timezone(PyObject *m) PyModule_AddIntConstant(m, "daylight", janzone != julyzone); tzname_obj = Py_BuildValue("(zz)", janname, julyname); } - if (tzname_obj == NULL) { + if (_PyModule_Add(m, "tzname", tzname_obj) < 0) { return -1; } - PyModule_AddObject(m, "tzname", tzname_obj); #endif // !HAVE_DECL_TZNAME if (PyErr_Occurred()) { diff --git a/Python/modsupport.c b/Python/modsupport.c index be229c987b8a7..df4ae35a52576 100644 --- a/Python/modsupport.c +++ b/Python/modsupport.c @@ -649,13 +649,16 @@ PyModule_AddObjectRef(PyObject *mod, const char *name, PyObject *value) PyModule_GetName(mod)); return -1; } - - if (PyDict_SetItemString(dict, name, value)) { - return -1; - } - return 0; + return PyDict_SetItemString(dict, name, value); } +int +_PyModule_Add(PyObject *mod, const char *name, PyObject *value) +{ + int res = PyModule_AddObjectRef(mod, name, value); + Py_XDECREF(value); + return res; +} int PyModule_AddObject(PyObject *mod, const char *name, PyObject *value) @@ -670,25 +673,13 @@ PyModule_AddObject(PyObject *mod, const char *name, PyObject *value) int PyModule_AddIntConstant(PyObject *m, const char *name, long value) { - PyObject *obj = PyLong_FromLong(value); - if (!obj) { - return -1; - } - int res = PyModule_AddObjectRef(m, name, obj); - Py_DECREF(obj); - return res; + return _PyModule_Add(m, name, PyLong_FromLong(value)); } int PyModule_AddStringConstant(PyObject *m, const char *name, const char *value) { - PyObject *obj = PyUnicode_FromString(value); - if (!obj) { - return -1; - } - int res = PyModule_AddObjectRef(m, name, obj); - Py_DECREF(obj); - return res; + return _PyModule_Add(m, name, PyUnicode_FromString(value)); } int From webhook-mailer at python.org Tue Jul 18 11:13:55 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 18 Jul 2023 15:13:55 -0000 Subject: [Python-checkins] gh-106535: Document soft deprecations in What's New In Python 3.13 (#106859) Message-ID: https://github.com/python/cpython/commit/3535ef1eec2563bbd7bff7c830465441fbbf759e commit: 3535ef1eec2563bbd7bff7c830465441fbbf759e branch: main author: Victor Stinner committer: vstinner date: 2023-07-18T17:13:51+02:00 summary: gh-106535: Document soft deprecations in What's New In Python 3.13 (#106859) files: M Doc/whatsnew/3.13.rst diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 0181e16f7d9b9..479d08b24b112 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -184,6 +184,13 @@ Deprecated Replace ``ctypes.ARRAY(item_type, size)`` with ``item_type * size``. (Contributed by Victor Stinner in :gh:`105733`.) +* The :mod:`getopt` and :mod:`optparse` modules are now + :term:`soft deprecated`: the :mod:`argparse` should be used for new projects. + Previously, the :mod:`optparse` module was already deprecated, its removal + was not scheduled, and no warnings was emitted: so there is no change in + practice. + (Contributed by Victor Stinner in :gh:`106535`.) + Pending Removal in Python 3.14 ------------------------------ @@ -946,6 +953,11 @@ Removed :c:func:`PyInterpreterState_Get()` on Python 3.8 and older. (Contributed by Victor Stinner in :gh:`106320`.) +* The :c:func:`PyModule_AddObject` function is now :term:`soft deprecated`: + :c:func:`PyModule_Add` or :c:func:`PyModule_AddObjectRef` functions should + be used instead. + (Contributed by Serhiy Storchaka in :gh:`86493`.) + Pending Removal in Python 3.14 ------------------------------ From webhook-mailer at python.org Tue Jul 18 14:42:49 2023 From: webhook-mailer at python.org (iritkatriel) Date: Tue, 18 Jul 2023 18:42:49 -0000 Subject: [Python-checkins] gh-105481: Generate the opcode lists in dis from data extracted from bytecodes.c (#106758) Message-ID: https://github.com/python/cpython/commit/40f3f11a773b854c6d94746aa3b1881c8ac71b0f commit: 40f3f11a773b854c6d94746aa3b1881c8ac71b0f branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-18T19:42:44+01:00 summary: gh-105481: Generate the opcode lists in dis from data extracted from bytecodes.c (#106758) files: A Misc/NEWS.d/next/Library/2023-07-17-16-46-00.gh-issue-105481.fek_Nn.rst M .gitattributes M Doc/library/dis.rst M Include/cpython/compile.h M Include/internal/pycore_opcode_metadata.h M Lib/opcode.py M Lib/test/test__opcode.py M Modules/_opcode.c M Modules/clinic/_opcode.c.h M Python/bytecodes.c M Python/ceval_macros.h M Python/compile.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Tools/build/generate_opcode_h.py M Tools/cases_generator/generate_cases.py diff --git a/.gitattributes b/.gitattributes index 2616da74b48c0..5d5558da711b1 100644 --- a/.gitattributes +++ b/.gitattributes @@ -87,7 +87,6 @@ Programs/test_frozenmain.h generated Python/Python-ast.c generated Python/executor_cases.c.h generated Python/generated_cases.c.h generated -Include/internal/pycore_opcode_metadata.h generated Python/opcode_targets.h generated Python/stdlib_module_names.h generated Tools/peg_generator/pegen/grammar_parser.py generated diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 099b6410f165e..6beaad3825aba 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -1803,15 +1803,12 @@ instructions: Sequence of bytecodes that access an attribute by name. -.. data:: hasjrel - - Sequence of bytecodes that have a relative jump target. +.. data:: hasjump + Sequence of bytecodes that have a jump target. All jumps + are relative. -.. data:: hasjabs - - Sequence of bytecodes that have an absolute jump target. - + .. versionadded:: 3.13 .. data:: haslocal @@ -1827,3 +1824,20 @@ instructions: Sequence of bytecodes that set an exception handler. .. versionadded:: 3.12 + + +.. data:: hasjrel + + Sequence of bytecodes that have a relative jump target. + + .. deprecated:: 3.13 + All jumps are now relative. Use :data:`hasjump`. + + +.. data:: hasjabs + + Sequence of bytecodes that have an absolute jump target. + + .. deprecated:: 3.13 + All jumps are now relative. This list is empty. + diff --git a/Include/cpython/compile.h b/Include/cpython/compile.h index cd7fd7bd37766..fd52697840203 100644 --- a/Include/cpython/compile.h +++ b/Include/cpython/compile.h @@ -73,3 +73,7 @@ PyAPI_FUNC(int) PyUnstable_OpcodeHasArg(int opcode); PyAPI_FUNC(int) PyUnstable_OpcodeHasConst(int opcode); PyAPI_FUNC(int) PyUnstable_OpcodeHasName(int opcode); PyAPI_FUNC(int) PyUnstable_OpcodeHasJump(int opcode); +PyAPI_FUNC(int) PyUnstable_OpcodeHasFree(int opcode); +PyAPI_FUNC(int) PyUnstable_OpcodeHasLocal(int opcode); +PyAPI_FUNC(int) PyUnstable_OpcodeHasExc(int opcode); + diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index a5844b3135d39..d525913f8a7ab 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -955,10 +955,14 @@ enum InstructionFormat { INSTR_FMT_IB, INSTR_FMT_IBC, INSTR_FMT_IBC00, INSTR_FMT #define HAS_CONST_FLAG (2) #define HAS_NAME_FLAG (4) #define HAS_JUMP_FLAG (8) +#define HAS_FREE_FLAG (16) +#define HAS_LOCAL_FLAG (32) #define OPCODE_HAS_ARG(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_ARG_FLAG)) #define OPCODE_HAS_CONST(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_CONST_FLAG)) #define OPCODE_HAS_NAME(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_NAME_FLAG)) #define OPCODE_HAS_JUMP(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_JUMP_FLAG)) +#define OPCODE_HAS_FREE(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_FREE_FLAG)) +#define OPCODE_HAS_LOCAL(OP) (_PyOpcode_opcode_metadata[OP].flags & (HAS_LOCAL_FLAG)) struct opcode_metadata { bool valid_entry; @@ -995,16 +999,16 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE] = { [NOP] = { true, INSTR_FMT_IX, 0 }, [RESUME] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, [INSTRUMENTED_RESUME] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [LOAD_CLOSURE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [LOAD_FAST_CHECK] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [LOAD_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [LOAD_FAST_AND_CLEAR] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [LOAD_FAST_LOAD_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, + [LOAD_CLOSURE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, + [LOAD_FAST_CHECK] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, + [LOAD_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, + [LOAD_FAST_AND_CLEAR] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, + [LOAD_FAST_LOAD_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, [LOAD_CONST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_CONST_FLAG }, - [STORE_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [STORE_FAST_MAYBE_NULL] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [STORE_FAST_LOAD_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [STORE_FAST_STORE_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, + [STORE_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, + [STORE_FAST_MAYBE_NULL] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, + [STORE_FAST_LOAD_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, + [STORE_FAST_STORE_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, [POP_TOP] = { true, INSTR_FMT_IX, 0 }, [PUSH_NULL] = { true, INSTR_FMT_IX, 0 }, [END_FOR] = { true, INSTR_FMT_IB, 0 }, @@ -1028,7 +1032,7 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE] = { [BINARY_OP_ADD_FLOAT] = { true, INSTR_FMT_IBC, 0 }, [BINARY_OP_SUBTRACT_FLOAT] = { true, INSTR_FMT_IBC, 0 }, [BINARY_OP_ADD_UNICODE] = { true, INSTR_FMT_IBC, 0 }, - [BINARY_OP_INPLACE_ADD_UNICODE] = { true, INSTR_FMT_IB, 0 }, + [BINARY_OP_INPLACE_ADD_UNICODE] = { true, INSTR_FMT_IB, HAS_LOCAL_FLAG }, [BINARY_SUBSCR] = { true, INSTR_FMT_IXC, 0 }, [BINARY_SLICE] = { true, INSTR_FMT_IX, 0 }, [STORE_SLICE] = { true, INSTR_FMT_IX, 0 }, @@ -1080,12 +1084,12 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE] = { [LOAD_GLOBAL] = { true, INSTR_FMT_IBC000, HAS_ARG_FLAG | HAS_NAME_FLAG }, [LOAD_GLOBAL_MODULE] = { true, INSTR_FMT_IBC000, HAS_ARG_FLAG }, [LOAD_GLOBAL_BUILTIN] = { true, INSTR_FMT_IBC000, HAS_ARG_FLAG }, - [DELETE_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [MAKE_CELL] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [DELETE_DEREF] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [LOAD_FROM_DICT_OR_DEREF] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [LOAD_DEREF] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, - [STORE_DEREF] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, + [DELETE_FAST] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_LOCAL_FLAG }, + [MAKE_CELL] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_FREE_FLAG }, + [DELETE_DEREF] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_FREE_FLAG }, + [LOAD_FROM_DICT_OR_DEREF] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_FREE_FLAG }, + [LOAD_DEREF] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_FREE_FLAG }, + [STORE_DEREF] = { true, INSTR_FMT_IB, HAS_ARG_FLAG | HAS_FREE_FLAG }, [COPY_FREE_VARS] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, [BUILD_STRING] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, [BUILD_TUPLE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG }, diff --git a/Lib/opcode.py b/Lib/opcode.py index 1b36300785aae..08dfd2674dca7 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -4,10 +4,12 @@ operate on bytecodes (e.g. peephole optimizers). """ -__all__ = ["cmp_op", "hasarg", "hasconst", "hasname", "hasjrel", "hasjabs", - "haslocal", "hascompare", "hasfree", "hasexc", "opname", "opmap", - "stack_effect", "HAVE_ARGUMENT", "EXTENDED_ARG"] +# Note that __all__ is further extended below +__all__ = ["cmp_op", "opname", "opmap", "stack_effect", "hascompare", + "HAVE_ARGUMENT", "EXTENDED_ARG"] + +import _opcode from _opcode import stack_effect import sys @@ -17,55 +19,24 @@ cmp_op = ('<', '<=', '==', '!=', '>', '>=') -hasarg = [] -hasconst = [] -hasname = [] -hasjrel = [] -hasjabs = [] -haslocal = [] -hascompare = [] -hasfree = [] -hasexc = [] - ENABLE_SPECIALIZATION = True def is_pseudo(op): return op >= MIN_PSEUDO_OPCODE and op <= MAX_PSEUDO_OPCODE -oplists = [hasarg, hasconst, hasname, hasjrel, hasjabs, - haslocal, hascompare, hasfree, hasexc] - opmap = {} -## pseudo opcodes (used in the compiler) mapped to the values -##?they can become in the actual code. +# pseudo opcodes (used in the compiler) mapped to the values +#?they can become in the actual code. _pseudo_ops = {} def def_op(name, op): opmap[name] = op -def name_op(name, op): - def_op(name, op) - hasname.append(op) - -def jrel_op(name, op): - def_op(name, op) - hasjrel.append(op) - -def jabs_op(name, op): - def_op(name, op) - hasjabs.append(op) - def pseudo_op(name, op, real_ops): def_op(name, op) _pseudo_ops[name] = real_ops - # add the pseudo opcode to the lists its targets are in - for oplist in oplists: - res = [opmap[rop] in oplist for rop in real_ops] - if any(res): - assert all(res) - oplist.append(op) # Instruction opcodes for compiled code @@ -137,74 +108,61 @@ def pseudo_op(name, op, real_ops): HAVE_ARGUMENT = 90 # real opcodes from here have an argument: -name_op('STORE_NAME', 90) # Index in name list -name_op('DELETE_NAME', 91) # "" +def_op('STORE_NAME', 90) # Index in name list +def_op('DELETE_NAME', 91) # "" def_op('UNPACK_SEQUENCE', 92) # Number of tuple items -jrel_op('FOR_ITER', 93) +def_op('FOR_ITER', 93) def_op('UNPACK_EX', 94) -name_op('STORE_ATTR', 95) # Index in name list -name_op('DELETE_ATTR', 96) # "" -name_op('STORE_GLOBAL', 97) # "" -name_op('DELETE_GLOBAL', 98) # "" +def_op('STORE_ATTR', 95) # Index in name list +def_op('DELETE_ATTR', 96) # "" +def_op('STORE_GLOBAL', 97) # "" +def_op('DELETE_GLOBAL', 98) # "" def_op('SWAP', 99) def_op('LOAD_CONST', 100) # Index in const list -hasconst.append(100) -name_op('LOAD_NAME', 101) # Index in name list +def_op('LOAD_NAME', 101) # Index in name list def_op('BUILD_TUPLE', 102) # Number of tuple items def_op('BUILD_LIST', 103) # Number of list items def_op('BUILD_SET', 104) # Number of set items def_op('BUILD_MAP', 105) # Number of dict entries -name_op('LOAD_ATTR', 106) # Index in name list +def_op('LOAD_ATTR', 106) # Index in name list def_op('COMPARE_OP', 107) # Comparison operator -hascompare.append(107) -name_op('IMPORT_NAME', 108) # Index in name list -name_op('IMPORT_FROM', 109) # Index in name list -jrel_op('JUMP_FORWARD', 110) # Number of words to skip - -jrel_op('POP_JUMP_IF_FALSE', 114) -jrel_op('POP_JUMP_IF_TRUE', 115) -name_op('LOAD_GLOBAL', 116) # Index in name list +def_op('IMPORT_NAME', 108) # Index in name list +def_op('IMPORT_FROM', 109) # Index in name list +def_op('JUMP_FORWARD', 110) # Number of words to skip + +def_op('POP_JUMP_IF_FALSE', 114) +def_op('POP_JUMP_IF_TRUE', 115) +def_op('LOAD_GLOBAL', 116) # Index in name list def_op('IS_OP', 117) def_op('CONTAINS_OP', 118) def_op('RERAISE', 119) def_op('COPY', 120) def_op('RETURN_CONST', 121) -hasconst.append(121) def_op('BINARY_OP', 122) -jrel_op('SEND', 123) # Number of words to skip +def_op('SEND', 123) # Number of words to skip def_op('LOAD_FAST', 124) # Local variable number, no null check -haslocal.append(124) def_op('STORE_FAST', 125) # Local variable number -haslocal.append(125) def_op('DELETE_FAST', 126) # Local variable number -haslocal.append(126) def_op('LOAD_FAST_CHECK', 127) # Local variable number -haslocal.append(127) -jrel_op('POP_JUMP_IF_NOT_NONE', 128) -jrel_op('POP_JUMP_IF_NONE', 129) +def_op('POP_JUMP_IF_NOT_NONE', 128) +def_op('POP_JUMP_IF_NONE', 129) def_op('RAISE_VARARGS', 130) # Number of raise arguments (1, 2, or 3) def_op('GET_AWAITABLE', 131) def_op('BUILD_SLICE', 133) # Number of items -jrel_op('JUMP_BACKWARD_NO_INTERRUPT', 134) # Number of words to skip (backwards) +def_op('JUMP_BACKWARD_NO_INTERRUPT', 134) # Number of words to skip (backwards) def_op('MAKE_CELL', 135) -hasfree.append(135) def_op('LOAD_DEREF', 137) -hasfree.append(137) def_op('STORE_DEREF', 138) -hasfree.append(138) def_op('DELETE_DEREF', 139) -hasfree.append(139) -jrel_op('JUMP_BACKWARD', 140) # Number of words to skip (backwards) -name_op('LOAD_SUPER_ATTR', 141) +def_op('JUMP_BACKWARD', 140) # Number of words to skip (backwards) +def_op('LOAD_SUPER_ATTR', 141) def_op('CALL_FUNCTION_EX', 142) # Flags def_op('LOAD_FAST_AND_CLEAR', 143) # Local variable number -haslocal.append(143) def_op('EXTENDED_ARG', 144) -EXTENDED_ARG = 144 +EXTENDED_ARG = opmap['EXTENDED_ARG'] def_op('LIST_APPEND', 145) def_op('SET_ADD', 146) def_op('MAP_ADD', 147) -hasfree.append(148) def_op('COPY_FREE_VARS', 149) def_op('YIELD_VALUE', 150) def_op('RESUME', 151) # This must be kept in sync with deepfreeze.py @@ -224,12 +182,10 @@ def pseudo_op(name, op, real_ops): def_op('STORE_FAST_STORE_FAST', 170) def_op('CALL', 171) def_op('KW_NAMES', 172) -hasconst.append(172) def_op('CALL_INTRINSIC_1', 173) def_op('CALL_INTRINSIC_2', 174) -name_op('LOAD_FROM_DICT_OR_GLOBALS', 175) +def_op('LOAD_FROM_DICT_OR_GLOBALS', 175) def_op('LOAD_FROM_DICT_OR_DEREF', 176) -hasfree.append(176) def_op('SET_FUNCTION_ATTRIBUTE', 177) # Attribute # Optimizer hook @@ -258,16 +214,12 @@ def pseudo_op(name, op, real_ops): def_op('INSTRUMENTED_LINE', 254) # 255 is reserved -hasarg.extend([op for op in opmap.values() if op >= HAVE_ARGUMENT]) MIN_PSEUDO_OPCODE = 256 pseudo_op('SETUP_FINALLY', 256, ['NOP']) -hasexc.append(256) pseudo_op('SETUP_CLEANUP', 257, ['NOP']) -hasexc.append(257) pseudo_op('SETUP_WITH', 258, ['NOP']) -hasexc.append(258) pseudo_op('POP_BLOCK', 259, ['NOP']) pseudo_op('JUMP', 260, ['JUMP_FORWARD', 'JUMP_BACKWARD']) @@ -283,12 +235,29 @@ def pseudo_op(name, op, real_ops): MAX_PSEUDO_OPCODE = MIN_PSEUDO_OPCODE + len(_pseudo_ops) - 1 -del def_op, name_op, jrel_op, jabs_op, pseudo_op +del def_op, pseudo_op opname = ['<%r>' % (op,) for op in range(MAX_PSEUDO_OPCODE + 1)] for op, i in opmap.items(): opname[i] = op +# The build uses older versions of Python which do not have _opcode.has_* functions +if sys.version_info[:2] >= (3, 13): + # These lists are documented as part of the dis module's API + hasarg = [op for op in opmap.values() if _opcode.has_arg(op)] + hasconst = [op for op in opmap.values() if _opcode.has_const(op)] + hasname = [op for op in opmap.values() if _opcode.has_name(op)] + hasjump = [op for op in opmap.values() if _opcode.has_jump(op)] + hasjrel = hasjump # for backward compatibility + hasjabs = [] + hasfree = [op for op in opmap.values() if _opcode.has_free(op)] + haslocal = [op for op in opmap.values() if _opcode.has_local(op)] + hasexc = [op for op in opmap.values() if _opcode.has_exc(op)] + + __all__.extend(["hasarg", "hasconst", "hasname", "hasjump", "hasjrel", + "hasjabs", "hasfree", "haslocal", "hasexc"]) + +hascompare = [opmap["COMPARE_OP"]] _nb_ops = [ ("NB_ADD", "+"), diff --git a/Lib/test/test__opcode.py b/Lib/test/test__opcode.py index 7d9553d9e383a..b3a9bcbe16045 100644 --- a/Lib/test/test__opcode.py +++ b/Lib/test/test__opcode.py @@ -7,16 +7,7 @@ from _opcode import stack_effect -class OpcodeTests(unittest.TestCase): - - def check_bool_function_result(self, func, ops, expected): - for op in ops: - if isinstance(op, str): - op = dis.opmap[op] - with self.subTest(opcode=op, func=func): - self.assertIsInstance(func(op), bool) - self.assertEqual(func(op), expected) - +class OpListTests(unittest.TestCase): def test_invalid_opcodes(self): invalid = [-100, -1, 255, 512, 513, 1000] self.check_bool_function_result(_opcode.is_valid, invalid, False) @@ -24,6 +15,9 @@ def test_invalid_opcodes(self): self.check_bool_function_result(_opcode.has_const, invalid, False) self.check_bool_function_result(_opcode.has_name, invalid, False) self.check_bool_function_result(_opcode.has_jump, invalid, False) + self.check_bool_function_result(_opcode.has_free, invalid, False) + self.check_bool_function_result(_opcode.has_local, invalid, False) + self.check_bool_function_result(_opcode.has_exc, invalid, False) def test_is_valid(self): names = [ @@ -36,43 +30,24 @@ def test_is_valid(self): opcodes = [dis.opmap[opname] for opname in names] self.check_bool_function_result(_opcode.is_valid, opcodes, True) - def test_has_arg(self): - has_arg = ['SWAP', 'LOAD_FAST', 'INSTRUMENTED_POP_JUMP_IF_TRUE', 'JUMP'] - no_arg = ['SETUP_WITH', 'POP_TOP', 'NOP', 'CACHE'] - self.check_bool_function_result(_opcode.has_arg, has_arg, True) - self.check_bool_function_result(_opcode.has_arg, no_arg, False) - - def test_has_const(self): - has_const = ['LOAD_CONST', 'RETURN_CONST', 'KW_NAMES'] - no_const = ['SETUP_WITH', 'POP_TOP', 'NOP', 'CACHE'] - self.check_bool_function_result(_opcode.has_const, has_const, True) - self.check_bool_function_result(_opcode.has_const, no_const, False) - - def test_has_name(self): - has_name = ['STORE_NAME', 'DELETE_ATTR', 'STORE_GLOBAL', 'IMPORT_FROM', - 'LOAD_FROM_DICT_OR_GLOBALS'] - no_name = ['SETUP_WITH', 'POP_TOP', 'NOP', 'CACHE'] - self.check_bool_function_result(_opcode.has_name, has_name, True) - self.check_bool_function_result(_opcode.has_name, no_name, False) + def test_oplists(self): + def check_function(self, func, expected): + for op in [-10, 520]: + with self.subTest(opcode=op, func=func): + res = func(op) + self.assertIsInstance(res, bool) + self.assertEqual(res, op in expected) - def test_has_jump(self): - has_jump = ['FOR_ITER', 'JUMP_FORWARD', 'JUMP', 'POP_JUMP_IF_TRUE', 'SEND'] - no_jump = ['SETUP_WITH', 'POP_TOP', 'NOP', 'CACHE'] - self.check_bool_function_result(_opcode.has_jump, has_jump, True) - self.check_bool_function_result(_opcode.has_jump, no_jump, False) + check_function(self, _opcode.has_arg, dis.hasarg) + check_function(self, _opcode.has_const, dis.hasconst) + check_function(self, _opcode.has_name, dis.hasname) + check_function(self, _opcode.has_jump, dis.hasjump) + check_function(self, _opcode.has_free, dis.hasfree) + check_function(self, _opcode.has_local, dis.haslocal) + check_function(self, _opcode.has_exc, dis.hasexc) - # the following test is part of the refactor, it will be removed soon - def test_against_legacy_bool_values(self): - # limiting to ops up to ENTER_EXECUTOR, because everything after that - # is not currently categorized correctly in opcode.py. - for op in range(0, opcode.opmap['ENTER_EXECUTOR']): - with self.subTest(op=op): - if opcode.opname[op] != f'<{op}>': - self.assertEqual(op in dis.hasarg, _opcode.has_arg(op)) - self.assertEqual(op in dis.hasconst, _opcode.has_const(op)) - self.assertEqual(op in dis.hasname, _opcode.has_name(op)) - self.assertEqual(op in dis.hasjrel, _opcode.has_jump(op)) +class OpListTests(unittest.TestCase): def test_stack_effect(self): self.assertEqual(stack_effect(dis.opmap['POP_TOP']), -1) self.assertEqual(stack_effect(dis.opmap['BUILD_SLICE'], 0), -1) diff --git a/Misc/NEWS.d/next/Library/2023-07-17-16-46-00.gh-issue-105481.fek_Nn.rst b/Misc/NEWS.d/next/Library/2023-07-17-16-46-00.gh-issue-105481.fek_Nn.rst new file mode 100644 index 0000000000000..d82eb987c83e9 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-17-16-46-00.gh-issue-105481.fek_Nn.rst @@ -0,0 +1 @@ +The various opcode lists in the :mod:`dis` module are now generated from bytecodes.c instead of explicitly constructed in opcode.py. diff --git a/Modules/_opcode.c b/Modules/_opcode.c index b3b9873d21a5b..daabdce165577 100644 --- a/Modules/_opcode.c +++ b/Modules/_opcode.c @@ -147,6 +147,63 @@ _opcode_has_jump_impl(PyObject *module, int opcode) /*[clinic input] +_opcode.has_free -> bool + + opcode: int + +Return True if the opcode accesses a free variable, False otherwise. + +Note that 'free' in this context refers to names in the current scope +that are referenced by inner scopes or names in outer scopes that are +referenced from this scope. It does not include references to global +or builtin scopes. +[clinic start generated code]*/ + +static int +_opcode_has_free_impl(PyObject *module, int opcode) +/*[clinic end generated code: output=d81ae4d79af0ee26 input=117dcd5c19c1139b]*/ +{ + return PyUnstable_OpcodeIsValid(opcode) && + PyUnstable_OpcodeHasFree(opcode); + +} + +/*[clinic input] + +_opcode.has_local -> bool + + opcode: int + +Return True if the opcode accesses a local variable, False otherwise. +[clinic start generated code]*/ + +static int +_opcode_has_local_impl(PyObject *module, int opcode) +/*[clinic end generated code: output=da5a8616b7a5097b input=9a798ee24aaef49d]*/ +{ + return PyUnstable_OpcodeIsValid(opcode) && + PyUnstable_OpcodeHasLocal(opcode); +} + +/*[clinic input] + +_opcode.has_exc -> bool + + opcode: int + +Return True if the opcode sets an exception handler, False otherwise. +[clinic start generated code]*/ + +static int +_opcode_has_exc_impl(PyObject *module, int opcode) +/*[clinic end generated code: output=41b68dff0ec82a52 input=db0e4bdb9bf13fa5]*/ +{ + return PyUnstable_OpcodeIsValid(opcode) && + PyUnstable_OpcodeHasExc(opcode); +} + +/*[clinic input] + _opcode.get_specialization_stats Return the specialization stats @@ -171,6 +228,9 @@ opcode_functions[] = { _OPCODE_HAS_CONST_METHODDEF _OPCODE_HAS_NAME_METHODDEF _OPCODE_HAS_JUMP_METHODDEF + _OPCODE_HAS_FREE_METHODDEF + _OPCODE_HAS_LOCAL_METHODDEF + _OPCODE_HAS_EXC_METHODDEF _OPCODE_GET_SPECIALIZATION_STATS_METHODDEF {NULL, NULL, 0, NULL} }; diff --git a/Modules/clinic/_opcode.c.h b/Modules/clinic/_opcode.c.h index 3eb050e470c34..e6381fa73a550 100644 --- a/Modules/clinic/_opcode.c.h +++ b/Modules/clinic/_opcode.c.h @@ -401,6 +401,200 @@ _opcode_has_jump(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyOb return return_value; } +PyDoc_STRVAR(_opcode_has_free__doc__, +"has_free($module, /, opcode)\n" +"--\n" +"\n" +"Return True if the opcode accesses a free variable, False otherwise.\n" +"\n" +"Note that \'free\' in this context refers to names in the current scope\n" +"that are referenced by inner scopes or names in outer scopes that are\n" +"referenced from this scope. It does not include references to global\n" +"or builtin scopes."); + +#define _OPCODE_HAS_FREE_METHODDEF \ + {"has_free", _PyCFunction_CAST(_opcode_has_free), METH_FASTCALL|METH_KEYWORDS, _opcode_has_free__doc__}, + +static int +_opcode_has_free_impl(PyObject *module, int opcode); + +static PyObject * +_opcode_has_free(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(opcode), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"opcode", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "has_free", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int opcode; + int _return_value; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + opcode = _PyLong_AsInt(args[0]); + if (opcode == -1 && PyErr_Occurred()) { + goto exit; + } + _return_value = _opcode_has_free_impl(module, opcode); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} + +PyDoc_STRVAR(_opcode_has_local__doc__, +"has_local($module, /, opcode)\n" +"--\n" +"\n" +"Return True if the opcode accesses a local variable, False otherwise."); + +#define _OPCODE_HAS_LOCAL_METHODDEF \ + {"has_local", _PyCFunction_CAST(_opcode_has_local), METH_FASTCALL|METH_KEYWORDS, _opcode_has_local__doc__}, + +static int +_opcode_has_local_impl(PyObject *module, int opcode); + +static PyObject * +_opcode_has_local(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(opcode), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"opcode", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "has_local", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int opcode; + int _return_value; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + opcode = _PyLong_AsInt(args[0]); + if (opcode == -1 && PyErr_Occurred()) { + goto exit; + } + _return_value = _opcode_has_local_impl(module, opcode); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} + +PyDoc_STRVAR(_opcode_has_exc__doc__, +"has_exc($module, /, opcode)\n" +"--\n" +"\n" +"Return True if the opcode sets an exception handler, False otherwise."); + +#define _OPCODE_HAS_EXC_METHODDEF \ + {"has_exc", _PyCFunction_CAST(_opcode_has_exc), METH_FASTCALL|METH_KEYWORDS, _opcode_has_exc__doc__}, + +static int +_opcode_has_exc_impl(PyObject *module, int opcode); + +static PyObject * +_opcode_has_exc(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE) + + #define NUM_KEYWORDS 1 + static struct { + PyGC_Head _this_is_not_used; + PyObject_VAR_HEAD + PyObject *ob_item[NUM_KEYWORDS]; + } _kwtuple = { + .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) + .ob_item = { &_Py_ID(opcode), }, + }; + #undef NUM_KEYWORDS + #define KWTUPLE (&_kwtuple.ob_base.ob_base) + + #else // !Py_BUILD_CORE + # define KWTUPLE NULL + #endif // !Py_BUILD_CORE + + static const char * const _keywords[] = {"opcode", NULL}; + static _PyArg_Parser _parser = { + .keywords = _keywords, + .fname = "has_exc", + .kwtuple = KWTUPLE, + }; + #undef KWTUPLE + PyObject *argsbuf[1]; + int opcode; + int _return_value; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + opcode = _PyLong_AsInt(args[0]); + if (opcode == -1 && PyErr_Occurred()) { + goto exit; + } + _return_value = _opcode_has_exc_impl(module, opcode); + if ((_return_value == -1) && PyErr_Occurred()) { + goto exit; + } + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} + PyDoc_STRVAR(_opcode_get_specialization_stats__doc__, "get_specialization_stats($module, /)\n" "--\n" @@ -418,4 +612,4 @@ _opcode_get_specialization_stats(PyObject *module, PyObject *Py_UNUSED(ignored)) { return _opcode_get_specialization_stats_impl(module); } -/*[clinic end generated code: output=ae2b2ef56d582180 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=e507bf14fb2796f8 input=a9049054013a1b77]*/ diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 19fb138ee64cb..ea136a3fca2e0 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -3604,7 +3604,7 @@ dummy_func( _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { next_instr--; - _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg, &GETLOCAL(0)); + _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg, LOCALS_ARRAY); DISPATCH_SAME_OPARG(); } STAT_INC(BINARY_OP, deferred); diff --git a/Python/ceval_macros.h b/Python/ceval_macros.h index 874bd45becf0c..c2c323317d10f 100644 --- a/Python/ceval_macros.h +++ b/Python/ceval_macros.h @@ -234,6 +234,7 @@ GETITEM(PyObject *v, Py_ssize_t i) { /* Local variable macros */ +#define LOCALS_ARRAY (frame->localsplus) #define GETLOCAL(i) (frame->localsplus[i]) /* The SETLOCAL() macro must not DECREF the local variable in-place and diff --git a/Python/compile.c b/Python/compile.c index 2a735382c0cfd..d5405b4656182 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -896,6 +896,24 @@ PyUnstable_OpcodeHasJump(int opcode) return OPCODE_HAS_JUMP(opcode); } +int +PyUnstable_OpcodeHasFree(int opcode) +{ + return OPCODE_HAS_FREE(opcode); +} + +int +PyUnstable_OpcodeHasLocal(int opcode) +{ + return OPCODE_HAS_LOCAL(opcode); +} + +int +PyUnstable_OpcodeHasExc(int opcode) +{ + return IS_BLOCK_PUSH_OPCODE(opcode); +} + static int codegen_addop_noarg(instr_sequence *seq, int opcode, location loc) { diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index f492c1fa9d8e3..e1f8b9f208c76 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -2449,7 +2449,7 @@ _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { next_instr--; - _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg, &GETLOCAL(0)); + _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg, LOCALS_ARRAY); DISPATCH_SAME_OPARG(); } STAT_INC(BINARY_OP, deferred); diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 0a8e4da46b8be..b2b0aa6ece481 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -4451,7 +4451,7 @@ _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { next_instr--; - _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg, &GETLOCAL(0)); + _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg, LOCALS_ARRAY); DISPATCH_SAME_OPARG(); } STAT_INC(BINARY_OP, deferred); diff --git a/Tools/build/generate_opcode_h.py b/Tools/build/generate_opcode_h.py index 2e841e6097aa2..5b0560e6b21a9 100644 --- a/Tools/build/generate_opcode_h.py +++ b/Tools/build/generate_opcode_h.py @@ -84,13 +84,7 @@ def main(opcode_py, opcode = get_python_module_dict(opcode_py) opmap = opcode['opmap'] opname = opcode['opname'] - hasarg = opcode['hasarg'] - hasconst = opcode['hasconst'] - hasjrel = opcode['hasjrel'] - hasjabs = opcode['hasjabs'] is_pseudo = opcode['is_pseudo'] - _pseudo_ops = opcode['_pseudo_ops'] - ENABLE_SPECIALIZATION = opcode["ENABLE_SPECIALIZATION"] MIN_PSEUDO_OPCODE = opcode["MIN_PSEUDO_OPCODE"] diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 2713fc6774e84..33eff548a1880 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -261,6 +261,8 @@ class InstructionFlags: HAS_CONST_FLAG: bool HAS_NAME_FLAG: bool HAS_JUMP_FLAG: bool + HAS_FREE_FLAG: bool + HAS_LOCAL_FLAG: bool def __post_init__(self): self.bitmask = { @@ -269,16 +271,25 @@ def __post_init__(self): @staticmethod def fromInstruction(instr: "AnyInstruction"): + + has_free = (variable_used(instr, "PyCell_New") or + variable_used(instr, "PyCell_GET") or + variable_used(instr, "PyCell_SET")) + return InstructionFlags( HAS_ARG_FLAG=variable_used(instr, "oparg"), HAS_CONST_FLAG=variable_used(instr, "FRAME_CO_CONSTS"), HAS_NAME_FLAG=variable_used(instr, "FRAME_CO_NAMES"), HAS_JUMP_FLAG=variable_used(instr, "JUMPBY"), + HAS_FREE_FLAG=has_free, + HAS_LOCAL_FLAG=(variable_used(instr, "GETLOCAL") or + variable_used(instr, "SETLOCAL")) and + not has_free, ) @staticmethod def newEmpty(): - return InstructionFlags(False, False, False, False) + return InstructionFlags(False, False, False, False, False, False) def add(self, other: "InstructionFlags") -> None: for name, value in dataclasses.asdict(other).items(): From webhook-mailer at python.org Tue Jul 18 16:59:57 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 18 Jul 2023 20:59:57 -0000 Subject: [Python-checkins] gh-86493: Use PyModule_Add() instead of PyModule_AddObjectRef() (GH-106860) Message-ID: https://github.com/python/cpython/commit/a293fa5915c21b21f5cb8ed9649fbdb37b4c1421 commit: a293fa5915c21b21f5cb8ed9649fbdb37b4c1421 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-18T23:59:53+03:00 summary: gh-86493: Use PyModule_Add() instead of PyModule_AddObjectRef() (GH-106860) files: M Modules/_ctypes/_ctypes.c M Modules/_hashopenssl.c M Modules/_json.c M Modules/_sre/sre.c M Modules/_ssl.c M Modules/_testcapi/mem.c M Modules/_testcapi/watchers.c M Modules/_testsinglephase.c M Modules/pyexpat.c M Modules/socketmodule.c M Modules/unicodedata.c M PC/msvcrtmodule.c M PC/winreg.c M Python/import.c diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index 7624c15ac522d..200fd36748c40 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -5740,15 +5740,9 @@ _ctypes_add_objects(PyObject *mod) { #define MOD_ADD(name, expr) \ do { \ - PyObject *obj = (expr); \ - if (obj == NULL) { \ + if (PyModule_Add(mod, name, (expr)) < 0) { \ return -1; \ } \ - if (PyModule_AddObjectRef(mod, name, obj) < 0) { \ - Py_DECREF(obj); \ - return -1; \ - } \ - Py_DECREF(obj); \ } while (0) MOD_ADD("_pointer_type_cache", Py_NewRef(_ctypes_ptrtype_cache)); diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c index 011cb765ed82e..246eea7409882 100644 --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -2189,7 +2189,6 @@ hashlib_init_constructors(PyObject *module) */ PyModuleDef *mdef; PyMethodDef *fdef; - PyObject *proxy; PyObject *func, *name_obj; _hashlibstate *state = get_hashlib_state(module); @@ -2224,17 +2223,8 @@ hashlib_init_constructors(PyObject *module) } } - proxy = PyDictProxy_New(state->constructs); - if (proxy == NULL) { - return -1; - } - - int rc = PyModule_AddObjectRef(module, "_constructors", proxy); - Py_DECREF(proxy); - if (rc < 0) { - return -1; - } - return 0; + return PyModule_Add(module, "_constructors", + PyDictProxy_New(state->constructs)); } static int diff --git a/Modules/_json.c b/Modules/_json.c index 360fb453cd111..2d0e30d70932b 100644 --- a/Modules/_json.c +++ b/Modules/_json.c @@ -1756,22 +1756,12 @@ static int _json_exec(PyObject *module) { PyObject *PyScannerType = PyType_FromSpec(&PyScannerType_spec); - if (PyScannerType == NULL) { - return -1; - } - int rc = PyModule_AddObjectRef(module, "make_scanner", PyScannerType); - Py_DECREF(PyScannerType); - if (rc < 0) { + if (PyModule_Add(module, "make_scanner", PyScannerType) < 0) { return -1; } PyObject *PyEncoderType = PyType_FromSpec(&PyEncoderType_spec); - if (PyEncoderType == NULL) { - return -1; - } - rc = PyModule_AddObjectRef(module, "make_encoder", PyEncoderType); - Py_DECREF(PyEncoderType); - if (rc < 0) { + if (PyModule_Add(module, "make_encoder", PyEncoderType) < 0) { return -1; } diff --git a/Modules/_sre/sre.c b/Modules/_sre/sre.c index f34a353432dec..ddbdc9f478aab 100644 --- a/Modules/_sre/sre.c +++ b/Modules/_sre/sre.c @@ -3195,12 +3195,7 @@ do { \ #define ADD_ULONG_CONSTANT(module, name, value) \ do { \ - PyObject *o = PyLong_FromUnsignedLong(value); \ - if (!o) \ - goto error; \ - int res = PyModule_AddObjectRef(module, name, o); \ - Py_DECREF(o); \ - if (res < 0) { \ + if (PyModule_Add(module, name, PyLong_FromUnsignedLong(value)) < 0) { \ goto error; \ } \ } while (0) diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 7c8f4225d178a..ed720b4295f8e 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -5773,13 +5773,7 @@ static int sslmodule_add_option(PyObject *m, const char *name, uint64_t value) { Py_BUILD_ASSERT(sizeof(unsigned long long) >= sizeof(value)); - PyObject *obj = PyLong_FromUnsignedLongLong(value); - if (obj == NULL) { - return -1; - } - int res = PyModule_AddObjectRef(m, name, obj); - Py_DECREF(obj); - return res; + return PyModule_Add(m, name, PyLong_FromUnsignedLongLong(value)); } diff --git a/Modules/_testcapi/mem.c b/Modules/_testcapi/mem.c index 16bda66af554a..979b3a4b2b1af 100644 --- a/Modules/_testcapi/mem.c +++ b/Modules/_testcapi/mem.c @@ -692,13 +692,11 @@ _PyTestCapi_Init_Mem(PyObject *mod) PyObject *v; #ifdef WITH_PYMALLOC - v = Py_NewRef(Py_True); + v = Py_True; #else - v = Py_NewRef(Py_False); + v = Py_False; #endif - int rc = PyModule_AddObjectRef(mod, "WITH_PYMALLOC", v); - Py_DECREF(v); - if (rc < 0) { + if (PyModule_AddObjectRef(mod, "WITH_PYMALLOC", v) < 0) { return -1; } diff --git a/Modules/_testcapi/watchers.c b/Modules/_testcapi/watchers.c index 7167943fffab3..4cf567b331498 100644 --- a/Modules/_testcapi/watchers.c +++ b/Modules/_testcapi/watchers.c @@ -516,13 +516,7 @@ static PyFunction_WatchCallback func_watcher_callbacks[NUM_TEST_FUNC_WATCHERS] = static int add_func_event(PyObject *module, const char *name, PyFunction_WatchEvent event) { - PyObject *value = PyLong_FromLong(event); - if (value == NULL) { - return -1; - } - int ok = PyModule_AddObjectRef(module, name, value); - Py_DECREF(value); - return ok; + return PyModule_Add(module, name, PyLong_FromLong(event)); } static PyObject * diff --git a/Modules/_testsinglephase.c b/Modules/_testsinglephase.c index 922b41005e841..c42a15a0eff49 100644 --- a/Modules/_testsinglephase.c +++ b/Modules/_testsinglephase.c @@ -137,13 +137,7 @@ init_module(PyObject *module, module_state *state) } double d = _PyTime_AsSecondsDouble(state->initialized); - PyObject *initialized = PyFloat_FromDouble(d); - if (initialized == NULL) { - return -1; - } - int rc = PyModule_AddObjectRef(module, "_module_initialized", initialized); - Py_DECREF(initialized); - if (rc < 0) { + if (PyModule_Add(module, "_module_initialized", PyFloat_FromDouble(d)) < 0) { return -1; } diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index 5721ed4412be5..bd8a98a46579a 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1812,16 +1812,13 @@ add_errors_module(PyObject *mod) goto error; } - int rc = PyModule_AddObjectRef(errors_module, "codes", codes_dict); - Py_CLEAR(codes_dict); - if (rc < 0) { - goto error; + if (PyModule_Add(errors_module, "codes", codes_dict) < 0) { + Py_DECREF(rev_codes_dict); + return -1; } - rc = PyModule_AddObjectRef(errors_module, "messages", rev_codes_dict); - Py_CLEAR(rev_codes_dict); - if (rc < 0) { - goto error; + if (PyModule_Add(errors_module, "messages", rev_codes_dict) < 0) { + return -1; } return 0; diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 1d3f34b857a1d..39bbc91171237 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -7425,9 +7425,7 @@ socket_exec(PyObject *m) sock_free_api(capi); goto error; } - int rc = PyModule_AddObjectRef(m, PySocket_CAPI_NAME, capsule); - Py_DECREF(capsule); - if (rc < 0) { + if (PyModule_Add(m, PySocket_CAPI_NAME, capsule) < 0) { goto error; } @@ -8818,13 +8816,7 @@ socket_exec(PyObject *m) }; int i; for (i = 0; i < Py_ARRAY_LENGTH(codes); ++i) { - PyObject *tmp = PyLong_FromUnsignedLong(codes[i]); - if (tmp == NULL) { - goto error; - } - int rc = PyModule_AddObjectRef(m, names[i], tmp); - Py_DECREF(tmp); - if (rc < 0) { + if (PyModule_Add(m, names[i], PyLong_FromUnsignedLong(codes[i])) < 0) { goto error; } } diff --git a/Modules/unicodedata.c b/Modules/unicodedata.c index b6e50528a23c8..966123f4624c0 100644 --- a/Modules/unicodedata.c +++ b/Modules/unicodedata.c @@ -1496,13 +1496,7 @@ unicodedata_exec(PyObject *module) } /* Export C API */ - PyObject *capsule = unicodedata_create_capi(); - if (capsule == NULL) { - return -1; - } - int rc = PyModule_AddObjectRef(module, "_ucnhash_CAPI", capsule); - Py_DECREF(capsule); - if (rc < 0) { + if (PyModule_Add(module, "_ucnhash_CAPI", unicodedata_create_capi()) < 0) { return -1; } return 0; diff --git a/PC/msvcrtmodule.c b/PC/msvcrtmodule.c index 53ef26b732f61..afc810adcf749 100644 --- a/PC/msvcrtmodule.c +++ b/PC/msvcrtmodule.c @@ -565,15 +565,9 @@ static struct PyMethodDef msvcrt_functions[] = { }; static int -insertptr(PyObject *mod, char *name, void *value) +insertptr(PyObject *mod, const char *name, void *value) { - PyObject *v = PyLong_FromVoidPtr(value); - if (v == NULL) { - return -1; - } - int rc = PyModule_AddObjectRef(mod, name, v); - Py_DECREF(v); - return rc; + return PyModule_Add(mod, name, PyLong_FromVoidPtr(value)); } #define INSERTINT(MOD, NAME, VAL) do { \ @@ -646,12 +640,7 @@ exec_module(PyObject* m) _VC_CRT_MINOR_VERSION, _VC_CRT_BUILD_VERSION, _VC_CRT_RBUILD_VERSION); - if (version == NULL) { - return -1; - } - int st = PyModule_AddObjectRef(m, "CRT_ASSEMBLY_VERSION", version); - Py_DECREF(version); - if (st < 0) { + if (PyModule_Add(m, "CRT_ASSEMBLY_VERSION", version) < 0) { return -1; } #endif diff --git a/PC/winreg.c b/PC/winreg.c index aa2055c7e619d..5252f78a9bdf7 100644 --- a/PC/winreg.c +++ b/PC/winreg.c @@ -2079,15 +2079,9 @@ static struct PyMethodDef winreg_methods[] = { } while (0) static int -inskey(PyObject *mod, char *name, HKEY key) +inskey(PyObject *mod, const char *name, HKEY key) { - PyObject *v = PyLong_FromVoidPtr(key); - if (v == NULL) { - return -1; - } - int rc = PyModule_AddObjectRef(mod, name, v); - Py_DECREF(v); - return rc; + return PyModule_Add(mod, name, PyLong_FromVoidPtr(key)); } #define ADD_KEY(VAL) do { \ diff --git a/Python/import.c b/Python/import.c index 3e52a4e4eb145..cf993cbd62a2e 100644 --- a/Python/import.c +++ b/Python/import.c @@ -3844,14 +3844,9 @@ imp_module_exec(PyObject *module) { const wchar_t *mode = _Py_GetConfig()->check_hash_pycs_mode; PyObject *pyc_mode = PyUnicode_FromWideChar(mode, -1); - if (pyc_mode == NULL) { + if (PyModule_Add(module, "check_hash_based_pycs", pyc_mode) < 0) { return -1; } - if (PyModule_AddObjectRef(module, "check_hash_based_pycs", pyc_mode) < 0) { - Py_DECREF(pyc_mode); - return -1; - } - Py_DECREF(pyc_mode); return 0; } From webhook-mailer at python.org Tue Jul 18 18:46:54 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 18 Jul 2023 22:46:54 -0000 Subject: [Python-checkins] Docs: Argument Clinic: Group guides about default values (#106872) Message-ID: https://github.com/python/cpython/commit/505eede38d141d43e40e246319b157e3c77211d3 commit: 505eede38d141d43e40e246319b157e3c77211d3 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-19T00:46:50+02:00 summary: Docs: Argument Clinic: Group guides about default values (#106872) Previous ToC layout (excerpt): - How to use symbolic default values ... - How to assign default values to parameter - How to use the ``NULL`` default value - How to use expressions as default values New layout: - How to assign default values to parameter - The ``NULL`` default value - Symbolic default values - Expressions as default values files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 12d7a77d43209..efeb22c618b51 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -564,22 +564,6 @@ How-to guides ============= -How to use symbolic default values ----------------------------------- - -The default value you provide for a parameter can't be any arbitrary -expression. Currently the following are explicitly supported: - -* Numeric constants (integer and float) -* String constants -* ``True``, ``False``, and ``None`` -* Simple symbolic constants like ``sys.maxsize``, which must - start with the name of the module - -(In the future, this may need to get even more elaborate, -to allow full expressions like ``CONSTANT - 1``.) - - How to to rename C functions and variables generated by Argument Clinic ----------------------------------------------------------------------- @@ -965,8 +949,8 @@ There's also special support for a default value of ``NULL``, and for simple expressions, documented in the following sections. -How to use the ``NULL`` default value -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The ``NULL`` default value +^^^^^^^^^^^^^^^^^^^^^^^^^^ For string and object parameters, you can set them to ``None`` to indicate that there's no default. However, that means the C variable will be @@ -976,8 +960,24 @@ behaves like a default value of ``None``, but the C variable is initialized with ``NULL``. -How to use expressions as default values -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Symbolic default values +^^^^^^^^^^^^^^^^^^^^^^^ + +The default value you provide for a parameter can't be any arbitrary +expression. Currently the following are explicitly supported: + +* Numeric constants (integer and float) +* String constants +* ``True``, ``False``, and ``None`` +* Simple symbolic constants like ``sys.maxsize``, which must + start with the name of the module + +(In the future, this may need to get even more elaborate, +to allow full expressions like ``CONSTANT - 1``.) + + +Expressions as default values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The default value for a parameter can be more than just a literal value. It can be an entire expression, using math operators and looking up attributes From webhook-mailer at python.org Tue Jul 18 19:16:08 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 18 Jul 2023 23:16:08 -0000 Subject: [Python-checkins] [3.11] Docs: Argument Clinic: Group guides about default values (GH-106872) (#106873) Message-ID: https://github.com/python/cpython/commit/0c47ed7bbf156e9fc01e276ef053206f3e6a3e62 commit: 0c47ed7bbf156e9fc01e276ef053206f3e6a3e62 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-19T01:16:04+02:00 summary: [3.11] Docs: Argument Clinic: Group guides about default values (GH-106872) (#106873) Docs: Argument Clinic: Group guides about default values (GH-106872) Previous ToC layout (excerpt): - How to use symbolic default values ... - How to assign default values to parameter - How to use the ``NULL`` default value - How to use expressions as default values New layout: - How to assign default values to parameter - The ``NULL`` default value - Symbolic default values - Expressions as default values (cherry picked from commit 505eede38d141d43e40e246319b157e3c77211d3) Co-authored-by: Erlend E. Aasland files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 287a4acebb133..c2e5a68fb9eb2 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -556,22 +556,6 @@ How-to guides ============= -How to use symbolic default values ----------------------------------- - -The default value you provide for a parameter can't be any arbitrary -expression. Currently the following are explicitly supported: - -* Numeric constants (integer and float) -* String constants -* ``True``, ``False``, and ``None`` -* Simple symbolic constants like ``sys.maxsize``, which must - start with the name of the module - -(In the future, this may need to get even more elaborate, -to allow full expressions like ``CONSTANT - 1``.) - - How to to rename C functions and variables generated by Argument Clinic ----------------------------------------------------------------------- @@ -954,8 +938,8 @@ There's also special support for a default value of ``NULL``, and for simple expressions, documented in the following sections. -How to use the ``NULL`` default value -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The ``NULL`` default value +^^^^^^^^^^^^^^^^^^^^^^^^^^ For string and object parameters, you can set them to ``None`` to indicate that there's no default. However, that means the C variable will be @@ -965,8 +949,24 @@ behaves like a default value of ``None``, but the C variable is initialized with ``NULL``. -How to use expressions as default values -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Symbolic default values +^^^^^^^^^^^^^^^^^^^^^^^ + +The default value you provide for a parameter can't be any arbitrary +expression. Currently the following are explicitly supported: + +* Numeric constants (integer and float) +* String constants +* ``True``, ``False``, and ``None`` +* Simple symbolic constants like ``sys.maxsize``, which must + start with the name of the module + +(In the future, this may need to get even more elaborate, +to allow full expressions like ``CONSTANT - 1``.) + + +Expressions as default values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The default value for a parameter can be more than just a literal value. It can be an entire expression, using math operators and looking up attributes From webhook-mailer at python.org Tue Jul 18 19:16:22 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Tue, 18 Jul 2023 23:16:22 -0000 Subject: [Python-checkins] [3.12] Docs: Argument Clinic: Group guides about default values (GH-106872) (#106874) Message-ID: https://github.com/python/cpython/commit/0c106a91e8e54bbf114cb54540dddb2a39a0e2be commit: 0c106a91e8e54bbf114cb54540dddb2a39a0e2be branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-19T01:16:19+02:00 summary: [3.12] Docs: Argument Clinic: Group guides about default values (GH-106872) (#106874) Docs: Argument Clinic: Group guides about default values (GH-106872) Previous ToC layout (excerpt): - How to use symbolic default values ... - How to assign default values to parameter - How to use the ``NULL`` default value - How to use expressions as default values New layout: - How to assign default values to parameter - The ``NULL`` default value - Symbolic default values - Expressions as default values (cherry picked from commit 505eede38d141d43e40e246319b157e3c77211d3) Co-authored-by: Erlend E. Aasland files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 12d7a77d43209..efeb22c618b51 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -564,22 +564,6 @@ How-to guides ============= -How to use symbolic default values ----------------------------------- - -The default value you provide for a parameter can't be any arbitrary -expression. Currently the following are explicitly supported: - -* Numeric constants (integer and float) -* String constants -* ``True``, ``False``, and ``None`` -* Simple symbolic constants like ``sys.maxsize``, which must - start with the name of the module - -(In the future, this may need to get even more elaborate, -to allow full expressions like ``CONSTANT - 1``.) - - How to to rename C functions and variables generated by Argument Clinic ----------------------------------------------------------------------- @@ -965,8 +949,8 @@ There's also special support for a default value of ``NULL``, and for simple expressions, documented in the following sections. -How to use the ``NULL`` default value -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The ``NULL`` default value +^^^^^^^^^^^^^^^^^^^^^^^^^^ For string and object parameters, you can set them to ``None`` to indicate that there's no default. However, that means the C variable will be @@ -976,8 +960,24 @@ behaves like a default value of ``None``, but the C variable is initialized with ``NULL``. -How to use expressions as default values -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Symbolic default values +^^^^^^^^^^^^^^^^^^^^^^^ + +The default value you provide for a parameter can't be any arbitrary +expression. Currently the following are explicitly supported: + +* Numeric constants (integer and float) +* String constants +* ``True``, ``False``, and ``None`` +* Simple symbolic constants like ``sys.maxsize``, which must + start with the name of the module + +(In the future, this may need to get even more elaborate, +to allow full expressions like ``CONSTANT - 1``.) + + +Expressions as default values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The default value for a parameter can be more than just a literal value. It can be an entire expression, using math operators and looking up attributes From webhook-mailer at python.org Tue Jul 18 19:20:34 2023 From: webhook-mailer at python.org (carljm) Date: Tue, 18 Jul 2023 23:20:34 -0000 Subject: [Python-checkins] gh-106727: Make `inspect.getsource` smarter for class for same name definitions (#106815) Message-ID: https://github.com/python/cpython/commit/663854d73b35feeb004ae0970e45b53ca27774a1 commit: 663854d73b35feeb004ae0970e45b53ca27774a1 branch: main author: Tian Gao committer: carljm date: 2023-07-18T23:20:31Z summary: gh-106727: Make `inspect.getsource` smarter for class for same name definitions (#106815) files: A Misc/NEWS.d/next/Library/2023-07-16-23-59-33.gh-issue-106727.bk3uCu.rst M Lib/inspect.py M Lib/test/inspect_fodder2.py M Lib/test/test_inspect.py diff --git a/Lib/inspect.py b/Lib/inspect.py index 15f94a194856a..675714dc8b3f7 100644 --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -1034,9 +1034,13 @@ class ClassFoundException(Exception): class _ClassFinder(ast.NodeVisitor): - def __init__(self, qualname): + def __init__(self, cls, tree, lines, qualname): self.stack = [] + self.cls = cls + self.tree = tree + self.lines = lines self.qualname = qualname + self.lineno_found = [] def visit_FunctionDef(self, node): self.stack.append(node.name) @@ -1057,11 +1061,48 @@ def visit_ClassDef(self, node): line_number = node.lineno # decrement by one since lines starts with indexing by zero - line_number -= 1 - raise ClassFoundException(line_number) + self.lineno_found.append((line_number - 1, node.end_lineno)) self.generic_visit(node) self.stack.pop() + def get_lineno(self): + self.visit(self.tree) + lineno_found_number = len(self.lineno_found) + if lineno_found_number == 0: + raise OSError('could not find class definition') + elif lineno_found_number == 1: + return self.lineno_found[0][0] + else: + # We have multiple candidates for the class definition. + # Now we have to guess. + + # First, let's see if there are any method definitions + for member in self.cls.__dict__.values(): + if isinstance(member, types.FunctionType): + for lineno, end_lineno in self.lineno_found: + if lineno <= member.__code__.co_firstlineno <= end_lineno: + return lineno + + class_strings = [(''.join(self.lines[lineno: end_lineno]), lineno) + for lineno, end_lineno in self.lineno_found] + + # Maybe the class has a docstring and it's unique? + if self.cls.__doc__: + ret = None + for candidate, lineno in class_strings: + if self.cls.__doc__.strip() in candidate: + if ret is None: + ret = lineno + else: + break + else: + if ret is not None: + return ret + + # We are out of ideas, just return the last one found, which is + # slightly better than previous ones + return self.lineno_found[-1][0] + def findsource(object): """Return the entire source file and starting line number for an object. @@ -1098,14 +1139,8 @@ def findsource(object): qualname = object.__qualname__ source = ''.join(lines) tree = ast.parse(source) - class_finder = _ClassFinder(qualname) - try: - class_finder.visit(tree) - except ClassFoundException as e: - line_number = e.args[0] - return lines, line_number - else: - raise OSError('could not find class definition') + class_finder = _ClassFinder(object, tree, lines, qualname) + return lines, class_finder.get_lineno() if ismethod(object): object = object.__func__ diff --git a/Lib/test/inspect_fodder2.py b/Lib/test/inspect_fodder2.py index 0346461369460..8639cf2e72cd7 100644 --- a/Lib/test/inspect_fodder2.py +++ b/Lib/test/inspect_fodder2.py @@ -290,3 +290,23 @@ def complex_decorated(foo=0, bar=lambda: 0): nested_lambda = ( lambda right: [].map( lambda length: ())) + +# line 294 +if True: + class cls296: + def f(): + pass +else: + class cls296: + def g(): + pass + +# line 304 +if False: + class cls310: + def f(): + pass +else: + class cls310: + def g(): + pass diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py index 64afeec351b35..33a593f3591d6 100644 --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -949,7 +949,6 @@ def test_class_decorator(self): self.assertSourceEqual(mod2.cls196.cls200, 198, 201) def test_class_inside_conditional(self): - self.assertSourceEqual(mod2.cls238, 238, 240) self.assertSourceEqual(mod2.cls238.cls239, 239, 240) def test_multiple_children_classes(self): @@ -975,6 +974,10 @@ def test_nested_class_definition_inside_async_function(self): self.assertSourceEqual(mod2.cls226, 231, 235) self.assertSourceEqual(asyncio.run(mod2.cls226().func232()), 233, 234) + def test_class_definition_same_name_diff_methods(self): + self.assertSourceEqual(mod2.cls296, 296, 298) + self.assertSourceEqual(mod2.cls310, 310, 312) + class TestNoEOL(GetSourceBase): def setUp(self): self.tempdir = TESTFN + '_dir' diff --git a/Misc/NEWS.d/next/Library/2023-07-16-23-59-33.gh-issue-106727.bk3uCu.rst b/Misc/NEWS.d/next/Library/2023-07-16-23-59-33.gh-issue-106727.bk3uCu.rst new file mode 100644 index 0000000000000..e4ea0ce1890d2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-16-23-59-33.gh-issue-106727.bk3uCu.rst @@ -0,0 +1 @@ +Make :func:`inspect.getsource` smarter for class for same name definitions From webhook-mailer at python.org Tue Jul 18 21:18:27 2023 From: webhook-mailer at python.org (corona10) Date: Wed, 19 Jul 2023 01:18:27 -0000 Subject: [Python-checkins] gh-106751: Optimize KqueueSelector.select() for many iteration case (gh-106864) Message-ID: https://github.com/python/cpython/commit/7513e2e7e48f6c004ed9bce55f2dcc6b388e02cd commit: 7513e2e7e48f6c004ed9bce55f2dcc6b388e02cd branch: main author: Dong-hee Na committer: corona10 date: 2023-07-19T10:18:23+09:00 summary: gh-106751: Optimize KqueueSelector.select() for many iteration case (gh-106864) files: A Misc/NEWS.d/next/Library/2023-07-18-23-05-12.gh-issue-106751.tVvzN_.rst M Lib/selectors.py diff --git a/Lib/selectors.py b/Lib/selectors.py index a42d156340641..d13405963f219 100644 --- a/Lib/selectors.py +++ b/Lib/selectors.py @@ -547,23 +547,21 @@ def select(self, timeout=None): # If max_ev is 0, kqueue will ignore the timeout. For consistent # behavior with the other selector classes, we prevent that here # (using max). See https://bugs.python.org/issue29255 - max_ev = max(len(self._fd_to_key), 1) + max_ev = len(self._fd_to_key) or 1 ready = [] try: kev_list = self._selector.control(None, max_ev, timeout) except InterruptedError: return ready + + fd_to_key_get = self._fd_to_key.get for kev in kev_list: fd = kev.ident flag = kev.filter - events = 0 - if flag == select.KQ_FILTER_READ: - events |= EVENT_READ - if flag == select.KQ_FILTER_WRITE: - events |= EVENT_WRITE - - key = self._fd_to_key.get(fd) + key = fd_to_key_get(fd) if key: + events = ((flag == select.KQ_FILTER_READ and EVENT_READ) + | (flag == select.KQ_FILTER_WRITE and EVENT_WRITE)) ready.append((key, events & key.events)) return ready diff --git a/Misc/NEWS.d/next/Library/2023-07-18-23-05-12.gh-issue-106751.tVvzN_.rst b/Misc/NEWS.d/next/Library/2023-07-18-23-05-12.gh-issue-106751.tVvzN_.rst new file mode 100644 index 0000000000000..1cb8424b6221e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-18-23-05-12.gh-issue-106751.tVvzN_.rst @@ -0,0 +1,2 @@ +Optimize :meth:`KqueueSelector.select` for many iteration case. Patch By +Dong-hee Na. From webhook-mailer at python.org Wed Jul 19 02:12:42 2023 From: webhook-mailer at python.org (corona10) Date: Wed, 19 Jul 2023 06:12:42 -0000 Subject: [Python-checkins] gh-106751: Optimize SelectSelector.select() for many iteration case (gh-106879) Message-ID: https://github.com/python/cpython/commit/e6f96cf9c62e38514e8f5465a1c43f85d861adb2 commit: e6f96cf9c62e38514e8f5465a1c43f85d861adb2 branch: main author: Dong-hee Na committer: corona10 date: 2023-07-19T15:12:38+09:00 summary: gh-106751: Optimize SelectSelector.select() for many iteration case (gh-106879) files: A Misc/NEWS.d/next/Library/2023-07-19-10-45-24.gh-issue-106751.3HJ1of.rst M Lib/selectors.py diff --git a/Lib/selectors.py b/Lib/selectors.py index d13405963f219..13497a2409723 100644 --- a/Lib/selectors.py +++ b/Lib/selectors.py @@ -314,17 +314,15 @@ def select(self, timeout=None): r, w, _ = self._select(self._readers, self._writers, [], timeout) except InterruptedError: return ready - r = set(r) - w = set(w) - for fd in r | w: - events = 0 - if fd in r: - events |= EVENT_READ - if fd in w: - events |= EVENT_WRITE - - key = self._fd_to_key.get(fd) + r = frozenset(r) + w = frozenset(w) + rw = r | w + fd_to_key_get = self._fd_to_key.get + for fd in rw: + key = fd_to_key_get(fd) if key: + events = ((fd in r and EVENT_READ) + | (fd in w and EVENT_WRITE)) ready.append((key, events & key.events)) return ready diff --git a/Misc/NEWS.d/next/Library/2023-07-19-10-45-24.gh-issue-106751.3HJ1of.rst b/Misc/NEWS.d/next/Library/2023-07-19-10-45-24.gh-issue-106751.3HJ1of.rst new file mode 100644 index 0000000000000..2696b560371d1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-19-10-45-24.gh-issue-106751.3HJ1of.rst @@ -0,0 +1,2 @@ +Optimize :meth:`SelectSelector.select` for many iteration case. Patch By +Dong-hee Na. From webhook-mailer at python.org Wed Jul 19 02:40:42 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 19 Jul 2023 06:40:42 -0000 Subject: [Python-checkins] [3.11] gh-86493: Fix possible leaks in some modules initialization (GH-106768) (GH-106855) (GH-106863) Message-ID: https://github.com/python/cpython/commit/fced79f91e186dd6a608221ad7cee2eeb443c40d commit: fced79f91e186dd6a608221ad7cee2eeb443c40d branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-19T09:40:38+03:00 summary: [3.11] gh-86493: Fix possible leaks in some modules initialization (GH-106768) (GH-106855) (GH-106863) [3.11] [3.12] gh-86493: Fix possible leaks in some modules initialization (GH-106768) (GH-106855) Fix _ssl, _stat, _testinternalcapi, _threadmodule, cmath, math, posix, time. (cherry picked from commit 3e65baee72131b49f4ce8ca2da568a6f2001ce93). (cherry picked from commit a423ddbdeada8a2fd8657453b9e9f58ba0dd921d) files: M Include/cpython/modsupport.h M Modules/_ssl.c M Modules/_stat.c M Modules/_testinternalcapi.c M Modules/_threadmodule.c M Modules/cmathmodule.c M Modules/mathmodule.c M Modules/posixmodule.c M Modules/timemodule.c M Python/modsupport.c diff --git a/Include/cpython/modsupport.h b/Include/cpython/modsupport.h index 769eb52bf6e3a..205e174243987 100644 --- a/Include/cpython/modsupport.h +++ b/Include/cpython/modsupport.h @@ -103,5 +103,6 @@ PyAPI_FUNC(PyObject * const *) _PyArg_UnpackKeywordsWithVararg( (minpos), (maxpos), (minkw), (buf))) PyAPI_FUNC(PyObject *) _PyModule_CreateInitialized(PyModuleDef*, int apiver); +PyAPI_FUNC(int) _PyModule_Add(PyObject *, const char *, PyObject *); PyAPI_DATA(const char *) _Py_PackageContext; diff --git a/Modules/_ssl.c b/Modules/_ssl.c index e3bb38e769c9b..0925722392446 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -6130,22 +6130,22 @@ sslmodule_init_versioninfo(PyObject *m) */ libver = OpenSSL_version_num(); r = PyLong_FromUnsignedLong(libver); - if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION_NUMBER", r)) + if (_PyModule_Add(m, "OPENSSL_VERSION_NUMBER", r) < 0) return -1; parse_openssl_version(libver, &major, &minor, &fix, &patch, &status); r = Py_BuildValue("IIIII", major, minor, fix, patch, status); - if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION_INFO", r)) + if (_PyModule_Add(m, "OPENSSL_VERSION_INFO", r) < 0) return -1; r = PyUnicode_FromString(OpenSSL_version(OPENSSL_VERSION)); - if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION", r)) + if (_PyModule_Add(m, "OPENSSL_VERSION", r) < 0) return -1; libver = OPENSSL_VERSION_NUMBER; parse_openssl_version(libver, &major, &minor, &fix, &patch, &status); r = Py_BuildValue("IIIII", major, minor, fix, patch, status); - if (r == NULL || PyModule_AddObject(m, "_OPENSSL_API_VERSION", r)) + if (_PyModule_Add(m, "_OPENSSL_API_VERSION", r) < 0) return -1; return 0; diff --git a/Modules/_stat.c b/Modules/_stat.c index 546e6a5f94ca1..6e0d6902fe75f 100644 --- a/Modules/_stat.c +++ b/Modules/_stat.c @@ -592,17 +592,17 @@ stat_exec(PyObject *module) ADD_INT_MACRO(module, FILE_ATTRIBUTE_TEMPORARY); ADD_INT_MACRO(module, FILE_ATTRIBUTE_VIRTUAL); - if (PyModule_AddObject(module, "IO_REPARSE_TAG_SYMLINK", - PyLong_FromUnsignedLong(IO_REPARSE_TAG_SYMLINK)) < 0) { - return -1; + if (_PyModule_Add(module, "IO_REPARSE_TAG_SYMLINK", + PyLong_FromUnsignedLong(IO_REPARSE_TAG_SYMLINK)) < 0) { + return -1; } - if (PyModule_AddObject(module, "IO_REPARSE_TAG_MOUNT_POINT", - PyLong_FromUnsignedLong(IO_REPARSE_TAG_MOUNT_POINT)) < 0) { - return -1; + if (_PyModule_Add(module, "IO_REPARSE_TAG_MOUNT_POINT", + PyLong_FromUnsignedLong(IO_REPARSE_TAG_MOUNT_POINT)) < 0) { + return -1; } - if (PyModule_AddObject(module, "IO_REPARSE_TAG_APPEXECLINK", - PyLong_FromUnsignedLong(IO_REPARSE_TAG_APPEXECLINK)) < 0) { - return -1; + if (_PyModule_Add(module, "IO_REPARSE_TAG_APPEXECLINK", + PyLong_FromUnsignedLong(IO_REPARSE_TAG_APPEXECLINK)) < 0) { + return -1; } #endif diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 4a8c43658998f..7e60da5906801 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -682,7 +682,7 @@ PyInit__testinternalcapi(void) return NULL; } - if (PyModule_AddObject(module, "SIZEOF_PYGC_HEAD", + if (_PyModule_Add(module, "SIZEOF_PYGC_HEAD", PyLong_FromSsize_t(sizeof(PyGC_Head))) < 0) { goto error; } diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index 879c94a109690..199e3b89d9921 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -1648,8 +1648,8 @@ thread_module_exec(PyObject *module) // Round towards minus infinity timeout_max = floor(timeout_max); - if (PyModule_AddObject(module, "TIMEOUT_MAX", - PyFloat_FromDouble(timeout_max)) < 0) { + if (_PyModule_Add(module, "TIMEOUT_MAX", + PyFloat_FromDouble(timeout_max)) < 0) { return -1; } diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c index 53e34061d5377..7fffb31cff0f2 100644 --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -1264,30 +1264,28 @@ static PyMethodDef cmath_methods[] = { static int cmath_exec(PyObject *mod) { - if (PyModule_AddObject(mod, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { + if (_PyModule_Add(mod, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { return -1; } - if (PyModule_AddObject(mod, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { + if (_PyModule_Add(mod, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { return -1; } // 2pi - if (PyModule_AddObject(mod, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { + if (_PyModule_Add(mod, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { return -1; } - if (PyModule_AddObject(mod, "inf", PyFloat_FromDouble(m_inf())) < 0) { + if (_PyModule_Add(mod, "inf", PyFloat_FromDouble(m_inf())) < 0) { return -1; } - if (PyModule_AddObject(mod, "infj", - PyComplex_FromCComplex(c_infj())) < 0) { + if (_PyModule_Add(mod, "infj", PyComplex_FromCComplex(c_infj())) < 0) { return -1; } #if _PY_SHORT_FLOAT_REPR == 1 - if (PyModule_AddObject(mod, "nan", PyFloat_FromDouble(m_nan())) < 0) { + if (_PyModule_Add(mod, "nan", PyFloat_FromDouble(m_nan())) < 0) { return -1; } - if (PyModule_AddObject(mod, "nanj", - PyComplex_FromCComplex(c_nanj())) < 0) { + if (_PyModule_Add(mod, "nanj", PyComplex_FromCComplex(c_nanj())) < 0) { return -1; } #endif diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 0a907a0c04ba1..5f5b71c4c0010 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -3825,21 +3825,21 @@ math_ulp_impl(PyObject *module, double x) static int math_exec(PyObject *module) { - if (PyModule_AddObject(module, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { + if (_PyModule_Add(module, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { return -1; } - if (PyModule_AddObject(module, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { + if (_PyModule_Add(module, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { return -1; } // 2pi - if (PyModule_AddObject(module, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { + if (_PyModule_Add(module, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { return -1; } - if (PyModule_AddObject(module, "inf", PyFloat_FromDouble(m_inf())) < 0) { + if (_PyModule_Add(module, "inf", PyFloat_FromDouble(m_inf())) < 0) { return -1; } #if _PY_SHORT_FLOAT_REPR == 1 - if (PyModule_AddObject(module, "nan", PyFloat_FromDouble(m_nan())) < 0) { + if (_PyModule_Add(module, "nan", PyFloat_FromDouble(m_nan())) < 0) { return -1; } #endif diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 91da07908f115..c000a32032287 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -12563,7 +12563,7 @@ setup_confname_table(struct constdef *table, size_t tablesize, } Py_DECREF(o); } - return PyModule_AddObject(module, tablename, d); + return _PyModule_Add(module, tablename, d); } /* Return -1 on failure, 0 on success. */ @@ -15837,11 +15837,9 @@ posixmodule_exec(PyObject *m) #endif /* Initialize environ dictionary */ - PyObject *v = convertenviron(); - Py_XINCREF(v); - if (v == NULL || PyModule_AddObject(m, "environ", v) != 0) + if (_PyModule_Add(m, "environ", convertenviron()) != 0) { return -1; - Py_DECREF(v); + } if (all_ins(m)) return -1; @@ -15965,9 +15963,7 @@ posixmodule_exec(PyObject *m) Py_DECREF(unicode); } - PyModule_AddObject(m, "_have_functions", list); - - return 0; + return _PyModule_Add(m, "_have_functions", list); } diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 18f9ddb909c02..b8e0e481cbb5e 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1775,11 +1775,9 @@ init_timezone(PyObject *m) return -1; } #endif // MS_WINDOWS - PyObject *tzname_obj = Py_BuildValue("(NN)", otz0, otz1); - if (tzname_obj == NULL) { + if (_PyModule_Add(m, "tzname", Py_BuildValue("(NN)", otz0, otz1)) < 0) { return -1; } - PyModule_AddObject(m, "tzname", tzname_obj); #else // !HAVE_DECL_TZNAME static const time_t YEAR = (365 * 24 + 6) * 3600; time_t t; @@ -1822,10 +1820,9 @@ init_timezone(PyObject *m) PyModule_AddIntConstant(m, "daylight", janzone != julyzone); tzname_obj = Py_BuildValue("(zz)", janname, julyname); } - if (tzname_obj == NULL) { + if (_PyModule_Add(m, "tzname", tzname_obj) < 0) { return -1; } - PyModule_AddObject(m, "tzname", tzname_obj); #endif // !HAVE_DECL_TZNAME if (PyErr_Occurred()) { diff --git a/Python/modsupport.c b/Python/modsupport.c index 8655daa1fc5e0..89ffae8c0f16d 100644 --- a/Python/modsupport.c +++ b/Python/modsupport.c @@ -658,13 +658,16 @@ PyModule_AddObjectRef(PyObject *mod, const char *name, PyObject *value) PyModule_GetName(mod)); return -1; } - - if (PyDict_SetItemString(dict, name, value)) { - return -1; - } - return 0; + return PyDict_SetItemString(dict, name, value); } +int +_PyModule_Add(PyObject *mod, const char *name, PyObject *value) +{ + int res = PyModule_AddObjectRef(mod, name, value); + Py_XDECREF(value); + return res; +} int PyModule_AddObject(PyObject *mod, const char *name, PyObject *value) @@ -679,25 +682,13 @@ PyModule_AddObject(PyObject *mod, const char *name, PyObject *value) int PyModule_AddIntConstant(PyObject *m, const char *name, long value) { - PyObject *obj = PyLong_FromLong(value); - if (!obj) { - return -1; - } - int res = PyModule_AddObjectRef(m, name, obj); - Py_DECREF(obj); - return res; + return _PyModule_Add(m, name, PyLong_FromLong(value)); } int PyModule_AddStringConstant(PyObject *m, const char *name, const char *value) { - PyObject *obj = PyUnicode_FromString(value); - if (!obj) { - return -1; - } - int res = PyModule_AddObjectRef(m, name, obj); - Py_DECREF(obj); - return res; + return _PyModule_Add(m, name, PyUnicode_FromString(value)); } int From webhook-mailer at python.org Wed Jul 19 07:03:54 2023 From: webhook-mailer at python.org (iritkatriel) Date: Wed, 19 Jul 2023 11:03:54 -0000 Subject: [Python-checkins] gh-104090: Fix unittest collectedDurations resources leak (#106795) Message-ID: https://github.com/python/cpython/commit/70b961ed93f67e34d0624e178f6029c886afaeee commit: 70b961ed93f67e34d0624e178f6029c886afaeee branch: main author: Yonatan Bitton committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-19T12:03:47+01:00 summary: gh-104090: Fix unittest collectedDurations resources leak (#106795) files: A Misc/NEWS.d/next/Tests/2023-07-16-02-57-08.gh-issue-104090.cKtK7g.rst M Lib/unittest/result.py diff --git a/Lib/unittest/result.py b/Lib/unittest/result.py index 7757dba9670b4..3ace0a5b7bf2e 100644 --- a/Lib/unittest/result.py +++ b/Lib/unittest/result.py @@ -166,7 +166,8 @@ def addDuration(self, test, elapsed): """ # support for a TextTestRunner using an old TestResult class if hasattr(self, "collectedDurations"): - self.collectedDurations.append((test, elapsed)) + # Pass test repr and not the test object itself to avoid resources leak + self.collectedDurations.append((str(test), elapsed)) def wasSuccessful(self): """Tells whether or not this result was a success.""" diff --git a/Misc/NEWS.d/next/Tests/2023-07-16-02-57-08.gh-issue-104090.cKtK7g.rst b/Misc/NEWS.d/next/Tests/2023-07-16-02-57-08.gh-issue-104090.cKtK7g.rst new file mode 100644 index 0000000000000..5cc6c5bbe1544 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-16-02-57-08.gh-issue-104090.cKtK7g.rst @@ -0,0 +1 @@ +Avoid creating a reference to the test object in :meth:`~unittest.TestResult.collectedDurations`. From webhook-mailer at python.org Wed Jul 19 07:07:43 2023 From: webhook-mailer at python.org (vstinner) Date: Wed, 19 Jul 2023 11:07:43 -0000 Subject: [Python-checkins] Export _PyEval_SetProfile() as a function, not data (#106887) Message-ID: https://github.com/python/cpython/commit/a1a3193990cd6658c1fe859b88a2bc03971a16df commit: a1a3193990cd6658c1fe859b88a2bc03971a16df branch: main author: Victor Stinner committer: vstinner date: 2023-07-19T11:07:40Z summary: Export _PyEval_SetProfile() as a function, not data (#106887) files: M Include/cpython/ceval.h diff --git a/Include/cpython/ceval.h b/Include/cpython/ceval.h index a9616bd6a4f51..5255d715142b9 100644 --- a/Include/cpython/ceval.h +++ b/Include/cpython/ceval.h @@ -4,7 +4,7 @@ PyAPI_FUNC(void) PyEval_SetProfile(Py_tracefunc, PyObject *); PyAPI_FUNC(void) PyEval_SetProfileAllThreads(Py_tracefunc, PyObject *); -PyAPI_DATA(int) _PyEval_SetProfile(PyThreadState *tstate, Py_tracefunc func, PyObject *arg); +PyAPI_FUNC(int) _PyEval_SetProfile(PyThreadState *tstate, Py_tracefunc func, PyObject *arg); PyAPI_FUNC(void) PyEval_SetTrace(Py_tracefunc, PyObject *); PyAPI_FUNC(void) PyEval_SetTraceAllThreads(Py_tracefunc, PyObject *); PyAPI_FUNC(int) _PyEval_SetTrace(PyThreadState *tstate, Py_tracefunc func, PyObject *arg); From webhook-mailer at python.org Wed Jul 19 07:34:58 2023 From: webhook-mailer at python.org (iritkatriel) Date: Wed, 19 Jul 2023 11:34:58 -0000 Subject: [Python-checkins] [3.12] gh-104090: Fix unittest collectedDurations resources leak (GH-106795) (#106888) Message-ID: https://github.com/python/cpython/commit/b1c50b80a6197ba1e41539ec0984a9925956cc1d commit: b1c50b80a6197ba1e41539ec0984a9925956cc1d branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-19T11:34:54Z summary: [3.12] gh-104090: Fix unittest collectedDurations resources leak (GH-106795) (#106888) gh-104090: Fix unittest collectedDurations resources leak (GH-106795) (cherry picked from commit 70b961ed93f67e34d0624e178f6029c886afaeee) Co-authored-by: Yonatan Bitton files: A Misc/NEWS.d/next/Tests/2023-07-16-02-57-08.gh-issue-104090.cKtK7g.rst M Lib/unittest/result.py diff --git a/Lib/unittest/result.py b/Lib/unittest/result.py index 7757dba9670b4..3ace0a5b7bf2e 100644 --- a/Lib/unittest/result.py +++ b/Lib/unittest/result.py @@ -166,7 +166,8 @@ def addDuration(self, test, elapsed): """ # support for a TextTestRunner using an old TestResult class if hasattr(self, "collectedDurations"): - self.collectedDurations.append((test, elapsed)) + # Pass test repr and not the test object itself to avoid resources leak + self.collectedDurations.append((str(test), elapsed)) def wasSuccessful(self): """Tells whether or not this result was a success.""" diff --git a/Misc/NEWS.d/next/Tests/2023-07-16-02-57-08.gh-issue-104090.cKtK7g.rst b/Misc/NEWS.d/next/Tests/2023-07-16-02-57-08.gh-issue-104090.cKtK7g.rst new file mode 100644 index 0000000000000..5cc6c5bbe1544 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-16-02-57-08.gh-issue-104090.cKtK7g.rst @@ -0,0 +1 @@ +Avoid creating a reference to the test object in :meth:`~unittest.TestResult.collectedDurations`. From webhook-mailer at python.org Wed Jul 19 14:00:00 2023 From: webhook-mailer at python.org (barneygale) Date: Wed, 19 Jul 2023 18:00:00 -0000 Subject: [Python-checkins] GH-100502: Add `pathlib.PurePath.pathmod` attribute (GH-106533) Message-ID: https://github.com/python/cpython/commit/c6c5665ee0c0a5ddc96da255c9a62daa332c32b3 commit: c6c5665ee0c0a5ddc96da255c9a62daa332c32b3 branch: main author: Barney Gale committer: barneygale date: 2023-07-19T18:59:55+01:00 summary: GH-100502: Add `pathlib.PurePath.pathmod` attribute (GH-106533) This instance attribute stores the implementation of `os.path` used for low-level path operations: either `posixpath` or `ntpath`. files: A Misc/NEWS.d/next/Library/2023-07-07-21-15-17.gh-issue-100502.Iici1B.rst M Doc/library/pathlib.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst index af81df217eea9..01dabe286969b 100644 --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -303,6 +303,13 @@ Methods and properties Pure paths provide the following methods and properties: +.. attribute:: PurePath.pathmod + + The implementation of the :mod:`os.path` module used for low-level path + operations: either ``posixpath`` or ``ntpath``. + + .. versionadded:: 3.13 + .. attribute:: PurePath.drive A string representing the drive letter or name, if any:: diff --git a/Lib/pathlib.py b/Lib/pathlib.py index f3813e0410990..8ff4d4ea19168 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -56,8 +56,8 @@ def _ignore_error(exception): @functools.cache -def _is_case_sensitive(flavour): - return flavour.normcase('Aa') == 'Aa' +def _is_case_sensitive(pathmod): + return pathmod.normcase('Aa') == 'Aa' # # Globbing helpers @@ -293,7 +293,7 @@ class PurePath: # path. It's set when `__hash__()` is called for the first time. '_hash', ) - _flavour = os.path + pathmod = os.path def __new__(cls, *args, **kwargs): """Construct a PurePath from one or several strings and or existing @@ -314,7 +314,7 @@ def __init__(self, *args): paths = [] for arg in args: if isinstance(arg, PurePath): - if arg._flavour is ntpath and self._flavour is posixpath: + if arg.pathmod is ntpath and self.pathmod is posixpath: # GH-103631: Convert separators for backwards compatibility. paths.extend(path.replace('\\', '/') for path in arg._raw_paths) else: @@ -343,11 +343,11 @@ def with_segments(self, *pathsegments): def _parse_path(cls, path): if not path: return '', '', [] - sep = cls._flavour.sep - altsep = cls._flavour.altsep + sep = cls.pathmod.sep + altsep = cls.pathmod.altsep if altsep: path = path.replace(altsep, sep) - drv, root, rel = cls._flavour.splitroot(path) + drv, root, rel = cls.pathmod.splitroot(path) if not root and drv.startswith(sep) and not drv.endswith(sep): drv_parts = drv.split(sep) if len(drv_parts) == 4 and drv_parts[2] not in '?.': @@ -366,7 +366,7 @@ def _load_parts(self): elif len(paths) == 1: path = paths[0] else: - path = self._flavour.join(*paths) + path = self.pathmod.join(*paths) drv, root, tail = self._parse_path(path) self._drv = drv self._root = root @@ -384,10 +384,10 @@ def _from_parsed_parts(self, drv, root, tail): @classmethod def _format_parsed_parts(cls, drv, root, tail): if drv or root: - return drv + root + cls._flavour.sep.join(tail) - elif tail and cls._flavour.splitdrive(tail[0])[0]: + return drv + root + cls.pathmod.sep.join(tail) + elif tail and cls.pathmod.splitdrive(tail[0])[0]: tail = ['.'] + tail - return cls._flavour.sep.join(tail) + return cls.pathmod.sep.join(tail) def __str__(self): """Return the string representation of the path, suitable for @@ -405,8 +405,7 @@ def __fspath__(self): def as_posix(self): """Return the string representation of the path with forward (/) slashes.""" - f = self._flavour - return str(self).replace(f.sep, '/') + return str(self).replace(self.pathmod.sep, '/') def __bytes__(self): """Return the bytes representation of the path. This is only @@ -442,7 +441,7 @@ def _str_normcase(self): try: return self._str_normcase_cached except AttributeError: - if _is_case_sensitive(self._flavour): + if _is_case_sensitive(self.pathmod): self._str_normcase_cached = str(self) else: self._str_normcase_cached = str(self).lower() @@ -454,7 +453,7 @@ def _parts_normcase(self): try: return self._parts_normcase_cached except AttributeError: - self._parts_normcase_cached = self._str_normcase.split(self._flavour.sep) + self._parts_normcase_cached = self._str_normcase.split(self.pathmod.sep) return self._parts_normcase_cached @property @@ -467,14 +466,14 @@ def _lines(self): if path_str == '.': self._lines_cached = '' else: - trans = _SWAP_SEP_AND_NEWLINE[self._flavour.sep] + trans = _SWAP_SEP_AND_NEWLINE[self.pathmod.sep] self._lines_cached = path_str.translate(trans) return self._lines_cached def __eq__(self, other): if not isinstance(other, PurePath): return NotImplemented - return self._str_normcase == other._str_normcase and self._flavour is other._flavour + return self._str_normcase == other._str_normcase and self.pathmod is other.pathmod def __hash__(self): try: @@ -484,22 +483,22 @@ def __hash__(self): return self._hash def __lt__(self, other): - if not isinstance(other, PurePath) or self._flavour is not other._flavour: + if not isinstance(other, PurePath) or self.pathmod is not other.pathmod: return NotImplemented return self._parts_normcase < other._parts_normcase def __le__(self, other): - if not isinstance(other, PurePath) or self._flavour is not other._flavour: + if not isinstance(other, PurePath) or self.pathmod is not other.pathmod: return NotImplemented return self._parts_normcase <= other._parts_normcase def __gt__(self, other): - if not isinstance(other, PurePath) or self._flavour is not other._flavour: + if not isinstance(other, PurePath) or self.pathmod is not other.pathmod: return NotImplemented return self._parts_normcase > other._parts_normcase def __ge__(self, other): - if not isinstance(other, PurePath) or self._flavour is not other._flavour: + if not isinstance(other, PurePath) or self.pathmod is not other.pathmod: return NotImplemented return self._parts_normcase >= other._parts_normcase @@ -584,9 +583,9 @@ def with_name(self, name): """Return a new path with the file name changed.""" if not self.name: raise ValueError("%r has an empty name" % (self,)) - f = self._flavour - drv, root, tail = f.splitroot(name) - if drv or root or not tail or f.sep in tail or (f.altsep and f.altsep in tail): + m = self.pathmod + drv, root, tail = m.splitroot(name) + if drv or root or not tail or m.sep in tail or (m.altsep and m.altsep in tail): raise ValueError("Invalid name %r" % (name)) return self._from_parsed_parts(self.drive, self.root, self._tail[:-1] + [name]) @@ -600,8 +599,8 @@ def with_suffix(self, suffix): has no suffix, add given suffix. If the given suffix is an empty string, remove the suffix from the path. """ - f = self._flavour - if f.sep in suffix or f.altsep and f.altsep in suffix: + m = self.pathmod + if m.sep in suffix or m.altsep and m.altsep in suffix: raise ValueError("Invalid suffix %r" % (suffix,)) if suffix and not suffix.startswith('.') or suffix == '.': raise ValueError("Invalid suffix %r" % (suffix)) @@ -702,22 +701,22 @@ def parents(self): def is_absolute(self): """True if the path is absolute (has both a root and, if applicable, a drive).""" - if self._flavour is ntpath: + if self.pathmod is ntpath: # ntpath.isabs() is defective - see GH-44626. return bool(self.drive and self.root) - elif self._flavour is posixpath: + elif self.pathmod is posixpath: # Optimization: work with raw paths on POSIX. for path in self._raw_paths: if path.startswith('/'): return True return False else: - return self._flavour.isabs(str(self)) + return self.pathmod.isabs(str(self)) def is_reserved(self): """Return True if the path contains one of the special names reserved by the system, if any.""" - if self._flavour is posixpath or not self._tail: + if self.pathmod is posixpath or not self._tail: return False # NOTE: the rules for reserved names seem somewhat complicated @@ -737,7 +736,7 @@ def match(self, path_pattern, *, case_sensitive=None): if not isinstance(path_pattern, PurePath): path_pattern = self.with_segments(path_pattern) if case_sensitive is None: - case_sensitive = _is_case_sensitive(self._flavour) + case_sensitive = _is_case_sensitive(self.pathmod) pattern = _compile_pattern_lines(path_pattern._lines, case_sensitive) if path_pattern.drive or path_pattern.root: return pattern.match(self._lines) is not None @@ -758,7 +757,7 @@ class PurePosixPath(PurePath): On a POSIX system, instantiating a PurePath should return this object. However, you can also instantiate it directly on any system. """ - _flavour = posixpath + pathmod = posixpath __slots__ = () @@ -768,7 +767,7 @@ class PureWindowsPath(PurePath): On a Windows system, instantiating a PurePath should return this object. However, you can also instantiate it directly on any system. """ - _flavour = ntpath + pathmod = ntpath __slots__ = () @@ -858,7 +857,7 @@ def is_mount(self): """ Check if this path is a mount point """ - return self._flavour.ismount(self) + return os.path.ismount(self) def is_symlink(self): """ @@ -879,7 +878,7 @@ def is_junction(self): """ Whether this path is a junction. """ - return self._flavour.isjunction(self) + return os.path.isjunction(self) def is_block_device(self): """ @@ -954,7 +953,8 @@ def samefile(self, other_path): other_st = other_path.stat() except AttributeError: other_st = self.with_segments(other_path).stat() - return self._flavour.samestat(st, other_st) + return (st.st_ino == other_st.st_ino and + st.st_dev == other_st.st_dev) def open(self, mode='r', buffering=-1, encoding=None, errors=None, newline=None): @@ -1017,7 +1017,7 @@ def _scandir(self): return os.scandir(self) def _make_child_relpath(self, name): - sep = self._flavour.sep + sep = self.pathmod.sep lines_name = name.replace('\n', sep) lines_str = self._lines path_str = str(self) @@ -1062,7 +1062,7 @@ def _glob(self, pattern, case_sensitive, follow_symlinks): raise ValueError("Unacceptable pattern: {!r}".format(pattern)) pattern_parts = list(path_pattern._tail) - if pattern[-1] in (self._flavour.sep, self._flavour.altsep): + if pattern[-1] in (self.pathmod.sep, self.pathmod.altsep): # GH-65238: pathlib doesn't preserve trailing slash. Add it back. pattern_parts.append('') if pattern_parts[-1] == '**': @@ -1071,7 +1071,7 @@ def _glob(self, pattern, case_sensitive, follow_symlinks): if case_sensitive is None: # TODO: evaluate case-sensitivity of each directory in _select_children(). - case_sensitive = _is_case_sensitive(self._flavour) + case_sensitive = _is_case_sensitive(self.pathmod) # If symlinks are handled consistently, and the pattern does not # contain '..' components, then we can use a 'walk-and-match' strategy @@ -1204,7 +1204,7 @@ def absolute(self): return self elif self.drive: # There is a CWD on each drive-letter drive. - cwd = self._flavour.abspath(self.drive) + cwd = os.path.abspath(self.drive) else: cwd = os.getcwd() # Fast path for "empty" paths, e.g. Path("."), Path("") or Path(). @@ -1230,7 +1230,7 @@ def check_eloop(e): raise RuntimeError("Symlink loop from %r" % e.filename) try: - s = self._flavour.realpath(self, strict=strict) + s = os.path.realpath(self, strict=strict) except OSError as e: check_eloop(e) raise @@ -1394,7 +1394,7 @@ def expanduser(self): """ if (not (self.drive or self.root) and self._tail and self._tail[0][:1] == '~'): - homedir = self._flavour.expanduser(self._tail[0]) + homedir = os.path.expanduser(self._tail[0]) if homedir[:1] == "~": raise RuntimeError("Could not determine home directory.") drv, root, tail = self._parse_path(homedir) diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index eb2b0cfb26e85..78948e3b72032 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -66,9 +66,9 @@ class PurePathTest(unittest.TestCase): def setUp(self): p = self.cls('a') - self.flavour = p._flavour - self.sep = self.flavour.sep - self.altsep = self.flavour.altsep + self.pathmod = p.pathmod + self.sep = self.pathmod.sep + self.altsep = self.pathmod.altsep def test_constructor_common(self): P = self.cls @@ -93,17 +93,17 @@ def test_concrete_class(self): p = self.cls('a') self.assertIs(type(p), expected) - def test_different_flavours_unequal(self): + def test_different_pathmods_unequal(self): p = self.cls('a') - if p._flavour is posixpath: + if p.pathmod is posixpath: q = pathlib.PureWindowsPath('a') else: q = pathlib.PurePosixPath('a') self.assertNotEqual(p, q) - def test_different_flavours_unordered(self): + def test_different_pathmods_unordered(self): p = self.cls('a') - if p._flavour is posixpath: + if p.pathmod is posixpath: q = pathlib.PureWindowsPath('a') else: q = pathlib.PurePosixPath('a') @@ -188,16 +188,16 @@ def _get_drive_root_parts(self, parts): return path.drive, path.root, path.parts def _check_drive_root_parts(self, arg, *expected): - sep = self.flavour.sep + sep = self.pathmod.sep actual = self._get_drive_root_parts([x.replace('/', sep) for x in arg]) self.assertEqual(actual, expected) - if altsep := self.flavour.altsep: + if altsep := self.pathmod.altsep: actual = self._get_drive_root_parts([x.replace('/', altsep) for x in arg]) self.assertEqual(actual, expected) def test_drive_root_parts_common(self): check = self._check_drive_root_parts - sep = self.flavour.sep + sep = self.pathmod.sep # Unanchored parts. check((), '', '', ()) check(('a',), '', '', ('a',)) @@ -657,7 +657,7 @@ def test_with_suffix_common(self): self.assertRaises(ValueError, P('a/b').with_suffix, './.d') self.assertRaises(ValueError, P('a/b').with_suffix, '.d/.') self.assertRaises(ValueError, P('a/b').with_suffix, - (self.flavour.sep, 'd')) + (self.pathmod.sep, 'd')) def test_relative_to_common(self): P = self.cls @@ -2392,8 +2392,8 @@ def test_concrete_class(self): p = self.cls('a') self.assertIs(type(p), expected) - def test_unsupported_flavour(self): - if self.cls._flavour is os.path: + def test_unsupported_pathmod(self): + if self.cls.pathmod is os.path: self.skipTest("path flavour is supported") else: self.assertRaises(pathlib.UnsupportedOperation, self.cls) @@ -2848,9 +2848,9 @@ def test_symlink_to_unsupported(self): def test_is_junction(self): P = self.cls(BASE) - with mock.patch.object(P._flavour, 'isjunction'): - self.assertEqual(P.is_junction(), P._flavour.isjunction.return_value) - P._flavour.isjunction.assert_called_once_with(P) + with mock.patch.object(P.pathmod, 'isjunction'): + self.assertEqual(P.is_junction(), P.pathmod.isjunction.return_value) + P.pathmod.isjunction.assert_called_once_with(P) @unittest.skipUnless(hasattr(os, "mkfifo"), "os.mkfifo() required") @unittest.skipIf(sys.platform == "vxworks", diff --git a/Misc/NEWS.d/next/Library/2023-07-07-21-15-17.gh-issue-100502.Iici1B.rst b/Misc/NEWS.d/next/Library/2023-07-07-21-15-17.gh-issue-100502.Iici1B.rst new file mode 100644 index 0000000000000..eea9564118df9 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-21-15-17.gh-issue-100502.Iici1B.rst @@ -0,0 +1,3 @@ +Add :attr:`pathlib.PurePath.pathmod` class attribute that stores the +implementation of :mod:`os.path` used for low-level path operations: either +``posixpath`` or ``ntpath``. From webhook-mailer at python.org Wed Jul 19 15:56:30 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 19 Jul 2023 19:56:30 -0000 Subject: [Python-checkins] gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (#106901) Message-ID: https://github.com/python/cpython/commit/1e1f4e91a905bab3103250a3ceadac0693b926d9 commit: 1e1f4e91a905bab3103250a3ceadac0693b926d9 branch: main author: Jack Nelson committer: AlexWaygood date: 2023-07-19T20:56:26+01:00 summary: gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (#106901) And later versions of 3.10, 3.9 files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 38f2e2f510c17..1ef8a5ab832e4 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1593,6 +1593,9 @@ Do not instantiate the :class:`Server` class directly. .. versionchanged:: 3.7 Server object is an asynchronous context manager since Python 3.7. + .. versionchanged:: 3.11 + This class was exposed publicly as ``asyncio.Server`` in Python 3.9.11, 3.10.3 and 3.11. + .. method:: close() Stop serving: close listening sockets and set the :attr:`sockets` From webhook-mailer at python.org Wed Jul 19 15:56:30 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 19 Jul 2023 19:56:30 -0000 Subject: [Python-checkins] gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (#106901) Message-ID: https://github.com/python/cpython/commit/1e1f4e91a905bab3103250a3ceadac0693b926d9 commit: 1e1f4e91a905bab3103250a3ceadac0693b926d9 branch: main author: Jack Nelson committer: AlexWaygood date: 2023-07-19T20:56:26+01:00 summary: gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (#106901) And later versions of 3.10, 3.9 files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 38f2e2f510c17..1ef8a5ab832e4 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1593,6 +1593,9 @@ Do not instantiate the :class:`Server` class directly. .. versionchanged:: 3.7 Server object is an asynchronous context manager since Python 3.7. + .. versionchanged:: 3.11 + This class was exposed publicly as ``asyncio.Server`` in Python 3.9.11, 3.10.3 and 3.11. + .. method:: close() Stop serving: close listening sockets and set the :attr:`sockets` From webhook-mailer at python.org Wed Jul 19 16:04:00 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 19 Jul 2023 20:04:00 -0000 Subject: [Python-checkins] [3.12] gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (GH-106901) (#106902) Message-ID: https://github.com/python/cpython/commit/c16cf9b6e59a4b9f374644edf463ec187c268c3d commit: c16cf9b6e59a4b9f374644edf463ec187c268c3d branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-19T21:03:56+01:00 summary: [3.12] gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (GH-106901) (#106902) gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (GH-106901) And later versions of 3.10, 3.9 (cherry picked from commit 1e1f4e91a905bab3103250a3ceadac0693b926d9) Co-authored-by: Jack Nelson files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 38f2e2f510c17..1ef8a5ab832e4 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1593,6 +1593,9 @@ Do not instantiate the :class:`Server` class directly. .. versionchanged:: 3.7 Server object is an asynchronous context manager since Python 3.7. + .. versionchanged:: 3.11 + This class was exposed publicly as ``asyncio.Server`` in Python 3.9.11, 3.10.3 and 3.11. + .. method:: close() Stop serving: close listening sockets and set the :attr:`sockets` From webhook-mailer at python.org Wed Jul 19 16:04:00 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 19 Jul 2023 20:04:00 -0000 Subject: [Python-checkins] [3.12] gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (GH-106901) (#106902) Message-ID: https://github.com/python/cpython/commit/c16cf9b6e59a4b9f374644edf463ec187c268c3d commit: c16cf9b6e59a4b9f374644edf463ec187c268c3d branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-19T21:03:56+01:00 summary: [3.12] gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (GH-106901) (#106902) gh-106882: Note that `asyncio.Server` is only publicly exposed on 3.11+ (GH-106901) And later versions of 3.10, 3.9 (cherry picked from commit 1e1f4e91a905bab3103250a3ceadac0693b926d9) Co-authored-by: Jack Nelson files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 38f2e2f510c17..1ef8a5ab832e4 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1593,6 +1593,9 @@ Do not instantiate the :class:`Server` class directly. .. versionchanged:: 3.7 Server object is an asynchronous context manager since Python 3.7. + .. versionchanged:: 3.11 + This class was exposed publicly as ``asyncio.Server`` in Python 3.9.11, 3.10.3 and 3.11. + .. method:: close() Stop serving: close listening sockets and set the :attr:`sockets` From webhook-mailer at python.org Wed Jul 19 20:53:14 2023 From: webhook-mailer at python.org (corona10) Date: Thu, 20 Jul 2023 00:53:14 -0000 Subject: [Python-checkins] gh-106751: Optimize _PolllikeSelector for many iteration case (gh-106884) Message-ID: https://github.com/python/cpython/commit/832c37d42a395d4ea45994daffa5e41bd74ac1bb commit: 832c37d42a395d4ea45994daffa5e41bd74ac1bb branch: main author: Pieter Eendebak committer: corona10 date: 2023-07-20T09:53:11+09:00 summary: gh-106751: Optimize _PolllikeSelector for many iteration case (gh-106884) files: A Misc/NEWS.d/next/Library/2023-07-19-09-11-08.gh-issue-106751.U9nD_B.rst M Lib/selectors.py diff --git a/Lib/selectors.py b/Lib/selectors.py index 13497a2409723..20367c9152f33 100644 --- a/Lib/selectors.py +++ b/Lib/selectors.py @@ -339,11 +339,8 @@ def __init__(self): def register(self, fileobj, events, data=None): key = super().register(fileobj, events, data) - poller_events = 0 - if events & EVENT_READ: - poller_events |= self._EVENT_READ - if events & EVENT_WRITE: - poller_events |= self._EVENT_WRITE + poller_events = ((events & EVENT_READ and self._EVENT_READ) + | (events & EVENT_WRITE and self._EVENT_WRITE) ) try: self._selector.register(key.fd, poller_events) except: @@ -369,11 +366,8 @@ def modify(self, fileobj, events, data=None): changed = False if events != key.events: - selector_events = 0 - if events & EVENT_READ: - selector_events |= self._EVENT_READ - if events & EVENT_WRITE: - selector_events |= self._EVENT_WRITE + selector_events = ((events & EVENT_READ and self._EVENT_READ) + | (events & EVENT_WRITE and self._EVENT_WRITE)) try: self._selector.modify(key.fd, selector_events) except: @@ -404,15 +398,13 @@ def select(self, timeout=None): fd_event_list = self._selector.poll(timeout) except InterruptedError: return ready - for fd, event in fd_event_list: - events = 0 - if event & ~self._EVENT_READ: - events |= EVENT_WRITE - if event & ~self._EVENT_WRITE: - events |= EVENT_READ - key = self._fd_to_key.get(fd) + fd_to_key_get = self._fd_to_key.get + for fd, event in fd_event_list: + key = fd_to_key_get(fd) if key: + events = ((event & ~self._EVENT_READ and EVENT_WRITE) + | (event & ~self._EVENT_WRITE and EVENT_READ)) ready.append((key, events & key.events)) return ready diff --git a/Misc/NEWS.d/next/Library/2023-07-19-09-11-08.gh-issue-106751.U9nD_B.rst b/Misc/NEWS.d/next/Library/2023-07-19-09-11-08.gh-issue-106751.U9nD_B.rst new file mode 100644 index 0000000000000..b9a9b563ad226 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-19-09-11-08.gh-issue-106751.U9nD_B.rst @@ -0,0 +1 @@ +Optimize :meth:`_PollLikeSelector.select` for many iteration case. From webhook-mailer at python.org Thu Jul 20 05:11:36 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Thu, 20 Jul 2023 09:11:36 -0000 Subject: [Python-checkins] gh-102799: use sys.exception() instead of sys.exc_info() in contextlib (#103311) Message-ID: https://github.com/python/cpython/commit/eeff8e72349743f37524221fe9e8f860879b46b3 commit: eeff8e72349743f37524221fe9e8f860879b46b3 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-20T14:41:32+05:30 summary: gh-102799: use sys.exception() instead of sys.exc_info() in contextlib (#103311) Co-authored-by: Kumar Aditya files: M Lib/contextlib.py M Lib/test/test_contextlib.py M Lib/test/test_contextlib_async.py diff --git a/Lib/contextlib.py b/Lib/contextlib.py index 95947aceccc30..f82e7bca35735 100644 --- a/Lib/contextlib.py +++ b/Lib/contextlib.py @@ -557,11 +557,12 @@ def __enter__(self): return self def __exit__(self, *exc_details): - received_exc = exc_details[0] is not None + exc = exc_details[1] + received_exc = exc is not None # We manipulate the exception state so it behaves as though # we were actually nesting multiple with statements - frame_exc = sys.exc_info()[1] + frame_exc = sys.exception() def _fix_exception_context(new_exc, old_exc): # Context may not be correct, so find the end of the chain while 1: @@ -584,24 +585,28 @@ def _fix_exception_context(new_exc, old_exc): is_sync, cb = self._exit_callbacks.pop() assert is_sync try: + if exc is None: + exc_details = None, None, None + else: + exc_details = type(exc), exc, exc.__traceback__ if cb(*exc_details): suppressed_exc = True pending_raise = False - exc_details = (None, None, None) - except: - new_exc_details = sys.exc_info() + exc = None + except BaseException as new_exc: # simulate the stack of exceptions by setting the context - _fix_exception_context(new_exc_details[1], exc_details[1]) + _fix_exception_context(new_exc, exc) pending_raise = True - exc_details = new_exc_details + exc = new_exc + if pending_raise: try: - # bare "raise exc_details[1]" replaces our carefully + # bare "raise exc" replaces our carefully # set-up context - fixed_ctx = exc_details[1].__context__ - raise exc_details[1] + fixed_ctx = exc.__context__ + raise exc except BaseException: - exc_details[1].__context__ = fixed_ctx + exc.__context__ = fixed_ctx raise return received_exc and suppressed_exc @@ -697,11 +702,12 @@ async def __aenter__(self): return self async def __aexit__(self, *exc_details): - received_exc = exc_details[0] is not None + exc = exc_details[1] + received_exc = exc is not None # We manipulate the exception state so it behaves as though # we were actually nesting multiple with statements - frame_exc = sys.exc_info()[1] + frame_exc = sys.exception() def _fix_exception_context(new_exc, old_exc): # Context may not be correct, so find the end of the chain while 1: @@ -723,6 +729,10 @@ def _fix_exception_context(new_exc, old_exc): while self._exit_callbacks: is_sync, cb = self._exit_callbacks.pop() try: + if exc is None: + exc_details = None, None, None + else: + exc_details = type(exc), exc, exc.__traceback__ if is_sync: cb_suppress = cb(*exc_details) else: @@ -731,21 +741,21 @@ def _fix_exception_context(new_exc, old_exc): if cb_suppress: suppressed_exc = True pending_raise = False - exc_details = (None, None, None) - except: - new_exc_details = sys.exc_info() + exc = None + except BaseException as new_exc: # simulate the stack of exceptions by setting the context - _fix_exception_context(new_exc_details[1], exc_details[1]) + _fix_exception_context(new_exc, exc) pending_raise = True - exc_details = new_exc_details + exc = new_exc + if pending_raise: try: - # bare "raise exc_details[1]" replaces our carefully + # bare "raise exc" replaces our carefully # set-up context - fixed_ctx = exc_details[1].__context__ - raise exc_details[1] + fixed_ctx = exc.__context__ + raise exc except BaseException: - exc_details[1].__context__ = fixed_ctx + exc.__context__ = fixed_ctx raise return received_exc and suppressed_exc diff --git a/Lib/test/test_contextlib.py b/Lib/test/test_contextlib.py index ecc5a43dad43d..dbc7dfcc24bf0 100644 --- a/Lib/test/test_contextlib.py +++ b/Lib/test/test_contextlib.py @@ -1085,7 +1085,7 @@ def first(): class TestExitStack(TestBaseExitStack, unittest.TestCase): exit_stack = ExitStack callback_error_internal_frames = [ - ('__exit__', 'raise exc_details[1]'), + ('__exit__', 'raise exc'), ('__exit__', 'if cb(*exc_details):'), ] diff --git a/Lib/test/test_contextlib_async.py b/Lib/test/test_contextlib_async.py index bb72ae74e5845..fa89c23f8edd9 100644 --- a/Lib/test/test_contextlib_async.py +++ b/Lib/test/test_contextlib_async.py @@ -557,7 +557,7 @@ def __exit__(self, *exc_details): ('__exit__', 'return self.run_coroutine(self.__aexit__(*exc_details))'), ('run_coroutine', 'raise exc'), ('run_coroutine', 'raise exc'), - ('__aexit__', 'raise exc_details[1]'), + ('__aexit__', 'raise exc'), ('__aexit__', 'cb_suppress = cb(*exc_details)'), ] From webhook-mailer at python.org Thu Jul 20 05:24:39 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Thu, 20 Jul 2023 09:24:39 -0000 Subject: [Python-checkins] gh-106078: Prepare to isolate decimal module (#106880) Message-ID: https://github.com/python/cpython/commit/a1620dd7b70abfceba6cda76efe5a14468f87f1d commit: a1620dd7b70abfceba6cda76efe5a14468f87f1d branch: main author: Charlie Zhao committer: kumaraditya303 date: 2023-07-20T09:24:35Z summary: gh-106078: Prepare to isolate decimal module (#106880) * move signal_map to global_state * move cond_map to global_state files: M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/ignored.tsv diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index f9dc6e875fa5f..f531def7b3a33 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -46,6 +46,7 @@ #endif struct PyDecContextObject; +struct DecCondMap; typedef struct { PyTypeObject *PyDecContextManager_Type; @@ -82,6 +83,9 @@ typedef struct { PyObject *SignalTuple; + struct DecCondMap *signal_map; + struct DecCondMap *cond_map; + /* External C-API functions */ binaryfunc _py_long_multiply; binaryfunc _py_long_floor_divide; @@ -181,7 +185,7 @@ incr_false(void) #define DEC_ERR_OCCURRED (DEC_INVALID_SIGNALS<<1) #define DEC_ERRORS (DEC_INVALID_SIGNALS|DEC_ERR_OCCURRED) -typedef struct { +typedef struct DecCondMap { const char *name; /* condition or signal name */ const char *fqname; /* fully qualified name */ uint32_t flag; /* libmpdec flag */ @@ -193,7 +197,7 @@ typedef struct { #define INEXACT 6 #define ROUNDED 7 #define SIGNAL_MAP_LEN 9 -static DecCondMap signal_map[] = { +static DecCondMap signal_map_template[] = { {"InvalidOperation", "decimal.InvalidOperation", MPD_IEEE_Invalid_operation, NULL}, {"FloatOperation", "decimal.FloatOperation", MPD_Float_operation, NULL}, {"DivisionByZero", "decimal.DivisionByZero", MPD_Division_by_zero, NULL}, @@ -207,7 +211,7 @@ static DecCondMap signal_map[] = { }; /* Exceptions that inherit from InvalidOperation */ -static DecCondMap cond_map[] = { +static DecCondMap cond_map_template[] = { {"InvalidOperation", "decimal.InvalidOperation", MPD_Invalid_operation, NULL}, {"ConversionSyntax", "decimal.ConversionSyntax", MPD_Conversion_syntax, NULL}, {"DivisionImpossible", "decimal.DivisionImpossible", MPD_Division_impossible, NULL}, @@ -219,6 +223,21 @@ static DecCondMap cond_map[] = { {NULL} }; +/* Return a duplicate of DecCondMap template */ +static inline DecCondMap * +dec_cond_map_init(DecCondMap *template, Py_ssize_t size) +{ + DecCondMap *cm; + cm = PyMem_Malloc(size); + if (cm == NULL) { + PyErr_NoMemory(); + return NULL; + } + + memcpy(cm, template, size); + return cm; +} + static const char *dec_signal_string[MPD_NUM_FLAGS] = { "Clamped", "InvalidOperation", @@ -312,8 +331,9 @@ static PyObject * flags_as_exception(uint32_t flags) { DecCondMap *cm; + decimal_state *state = GLOBAL_STATE(); - for (cm = signal_map; cm->name != NULL; cm++) { + for (cm = state->signal_map; cm->name != NULL; cm++) { if (flags&cm->flag) { return cm->ex; } @@ -326,8 +346,9 @@ Py_LOCAL_INLINE(uint32_t) exception_as_flag(PyObject *ex) { DecCondMap *cm; + decimal_state *state = GLOBAL_STATE(); - for (cm = signal_map; cm->name != NULL; cm++) { + for (cm = state->signal_map; cm->name != NULL; cm++) { if (cm->ex == ex) { return cm->flag; } @@ -342,20 +363,21 @@ flags_as_list(uint32_t flags) { PyObject *list; DecCondMap *cm; + decimal_state *state = GLOBAL_STATE(); list = PyList_New(0); if (list == NULL) { return NULL; } - for (cm = cond_map; cm->name != NULL; cm++) { + for (cm = state->cond_map; cm->name != NULL; cm++) { if (flags&cm->flag) { if (PyList_Append(list, cm->ex) < 0) { goto error; } } } - for (cm = signal_map+1; cm->name != NULL; cm++) { + for (cm = state->signal_map+1; cm->name != NULL; cm++) { if (flags&cm->flag) { if (PyList_Append(list, cm->ex) < 0) { goto error; @@ -375,13 +397,14 @@ signals_as_list(uint32_t flags) { PyObject *list; DecCondMap *cm; + decimal_state *state = GLOBAL_STATE(); list = PyList_New(0); if (list == NULL) { return NULL; } - for (cm = signal_map; cm->name != NULL; cm++) { + for (cm = state->signal_map; cm->name != NULL; cm++) { if (flags&cm->flag) { if (PyList_Append(list, cm->ex) < 0) { Py_DECREF(list); @@ -421,13 +444,14 @@ flags_as_dict(uint32_t flags) { DecCondMap *cm; PyObject *dict; + decimal_state *state = GLOBAL_STATE(); dict = PyDict_New(); if (dict == NULL) { return NULL; } - for (cm = signal_map; cm->name != NULL; cm++) { + for (cm = state->signal_map; cm->name != NULL; cm++) { PyObject *b = flags&cm->flag ? Py_True : Py_False; if (PyDict_SetItem(dict, cm->ex, b) < 0) { Py_DECREF(dict); @@ -445,6 +469,7 @@ dict_as_flags(PyObject *val) DecCondMap *cm; uint32_t flags = 0; int x; + decimal_state *state = GLOBAL_STATE(); if (!PyDict_Check(val)) { PyErr_SetString(PyExc_TypeError, @@ -458,7 +483,7 @@ dict_as_flags(PyObject *val) return DEC_INVALID_SIGNALS; } - for (cm = signal_map; cm->name != NULL; cm++) { + for (cm = state->signal_map; cm->name != NULL; cm++) { b = PyDict_GetItemWithError(val, cm->ex); if (b == NULL) { if (PyErr_Occurred()) { @@ -652,7 +677,8 @@ signaldict_repr(PyObject *self) assert(SIGNAL_MAP_LEN == 9); - for (cm=signal_map, i=0; cm->name != NULL; cm++, i++) { + decimal_state *state = GLOBAL_STATE(); + for (cm=state->signal_map, i=0; cm->name != NULL; cm++, i++) { n[i] = cm->fqname; b[i] = SdFlags(self)&cm->flag ? "True" : "False"; } @@ -5922,10 +5948,12 @@ PyInit__decimal(void) ASSIGN_PTR(state->SignalTuple, PyTuple_New(SIGNAL_MAP_LEN)); /* Add exceptions that correspond to IEEE signals */ + ASSIGN_PTR(state->signal_map, dec_cond_map_init(signal_map_template, + sizeof(signal_map_template))); for (i = SIGNAL_MAP_LEN-1; i >= 0; i--) { PyObject *base; - cm = signal_map + i; + cm = state->signal_map + i; switch (cm->flag) { case MPD_Float_operation: @@ -5936,13 +5964,13 @@ PyInit__decimal(void) PyExc_ZeroDivisionError); break; case MPD_Overflow: - base = PyTuple_Pack(2, signal_map[INEXACT].ex, - signal_map[ROUNDED].ex); + base = PyTuple_Pack(2, state->signal_map[INEXACT].ex, + state->signal_map[ROUNDED].ex); break; case MPD_Underflow: - base = PyTuple_Pack(3, signal_map[INEXACT].ex, - signal_map[ROUNDED].ex, - signal_map[SUBNORMAL].ex); + base = PyTuple_Pack(3, state->signal_map[INEXACT].ex, + state->signal_map[ROUNDED].ex, + state->signal_map[SUBNORMAL].ex); break; default: base = PyTuple_Pack(1, state->DecimalException); @@ -5968,16 +5996,18 @@ PyInit__decimal(void) * several conditions, including InvalidOperation! Naming the * signal IEEEInvalidOperation would prevent the confusion. */ - cond_map[0].ex = signal_map[0].ex; + ASSIGN_PTR(state->cond_map, dec_cond_map_init(cond_map_template, + sizeof(cond_map_template))); + state->cond_map[0].ex = state->signal_map[0].ex; /* Add remaining exceptions, inherit from InvalidOperation */ - for (cm = cond_map+1; cm->name != NULL; cm++) { + for (cm = state->cond_map+1; cm->name != NULL; cm++) { PyObject *base; if (cm->flag == MPD_Division_undefined) { - base = PyTuple_Pack(2, signal_map[0].ex, PyExc_ZeroDivisionError); + base = PyTuple_Pack(2, state->signal_map[0].ex, PyExc_ZeroDivisionError); } else { - base = PyTuple_Pack(1, signal_map[0].ex); + base = PyTuple_Pack(1, state->signal_map[0].ex); } if (base == NULL) { goto error; /* GCOV_NOT_REACHED */ @@ -6057,6 +6087,8 @@ PyInit__decimal(void) Py_CLEAR(collections_abc); /* GCOV_NOT_REACHED */ Py_CLEAR(MutableMapping); /* GCOV_NOT_REACHED */ Py_CLEAR(state->SignalTuple); /* GCOV_NOT_REACHED */ + PyMem_Free(state->signal_map); /* GCOV_NOT_REACHED */ + PyMem_Free(state->cond_map); /* GCOV_NOT_REACHED */ Py_CLEAR(state->DecimalTuple); /* GCOV_NOT_REACHED */ Py_CLEAR(state->default_context_template); /* GCOV_NOT_REACHED */ #ifndef WITH_DECIMAL_CONTEXTVAR diff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsv index 73eec6631d951..228ca8648a2f3 100644 --- a/Tools/c-analyzer/cpython/ignored.tsv +++ b/Tools/c-analyzer/cpython/ignored.tsv @@ -205,13 +205,13 @@ Modules/_datetimemodule.c - max_fold_seconds - Modules/_datetimemodule.c datetime_isoformat specs - Modules/_datetimemodule.c parse_hh_mm_ss_ff correction - Modules/_datetimemodule.c time_isoformat specs - -Modules/_decimal/_decimal.c - cond_map - +Modules/_decimal/_decimal.c - cond_map_template - Modules/_decimal/_decimal.c - dec_signal_string - Modules/_decimal/_decimal.c - dflt_ctx - Modules/_decimal/_decimal.c - int_constants - Modules/_decimal/_decimal.c - invalid_rounding_err - Modules/_decimal/_decimal.c - invalid_signals_err - -Modules/_decimal/_decimal.c - signal_map - +Modules/_decimal/_decimal.c - signal_map_template - Modules/_decimal/_decimal.c - ssize_constants - Modules/_elementtree.c - ExpatMemoryHandler - Modules/_hashopenssl.c - py_hashes - From webhook-mailer at python.org Thu Jul 20 05:28:25 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Thu, 20 Jul 2023 09:28:25 -0000 Subject: [Python-checkins] Fix typo in 3.11.4 changelog: urllib.request.Requst -> Request (#106830) Message-ID: https://github.com/python/cpython/commit/009e8f084c4cbb1f43d40b24b7f71fb189bbe36b commit: 009e8f084c4cbb1f43d40b24b7f71fb189bbe36b branch: main author: Zach Brantmeier committer: kumaraditya303 date: 2023-07-20T09:28:21Z summary: Fix typo in 3.11.4 changelog: urllib.request.Requst -> Request (#106830) files: M Misc/NEWS.d/3.12.0b1.rst diff --git a/Misc/NEWS.d/3.12.0b1.rst b/Misc/NEWS.d/3.12.0b1.rst index 96d76f89fa9c2..7be70de029b1f 100644 --- a/Misc/NEWS.d/3.12.0b1.rst +++ b/Misc/NEWS.d/3.12.0b1.rst @@ -1955,7 +1955,7 @@ introduced in :pep:`692`. .. section: Documentation Clarifying documentation about the url parameter to urllib.request.urlopen -and urllib.request.Requst needing to be encoded properly. +and urllib.request.Request needing to be encoded properly. .. From webhook-mailer at python.org Thu Jul 20 05:29:34 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Thu, 20 Jul 2023 09:29:34 -0000 Subject: [Python-checkins] [3.12] Fix typo in 3.11.4 changelog: urllib.request.Requst -> Request (GH-106830) (#106912) Message-ID: https://github.com/python/cpython/commit/bc107e53d47c4b4ed242b04118bff7bb9ccf01c8 commit: bc107e53d47c4b4ed242b04118bff7bb9ccf01c8 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-20T09:29:31Z summary: [3.12] Fix typo in 3.11.4 changelog: urllib.request.Requst -> Request (GH-106830) (#106912) Fix typo in 3.11.4 changelog: urllib.request.Requst -> Request (GH-106830) (cherry picked from commit 009e8f084c4cbb1f43d40b24b7f71fb189bbe36b) Co-authored-by: Zach Brantmeier files: M Misc/NEWS.d/3.12.0b1.rst diff --git a/Misc/NEWS.d/3.12.0b1.rst b/Misc/NEWS.d/3.12.0b1.rst index a1ea082b3a211..3de3b703e9f4f 100644 --- a/Misc/NEWS.d/3.12.0b1.rst +++ b/Misc/NEWS.d/3.12.0b1.rst @@ -1955,7 +1955,7 @@ introduced in :pep:`692`. .. section: Documentation Clarifying documentation about the url parameter to urllib.request.urlopen -and urllib.request.Requst needing to be encoded properly. +and urllib.request.Request needing to be encoded properly. .. From webhook-mailer at python.org Thu Jul 20 12:35:43 2023 From: webhook-mailer at python.org (brandtbucher) Date: Thu, 20 Jul 2023 16:35:43 -0000 Subject: [Python-checkins] GH-104584: Miscellaneous fixes for -Xuops (GH-106908) Message-ID: https://github.com/python/cpython/commit/214a25dd81dfe5ee0ab843cf665da2a7473a08db commit: 214a25dd81dfe5ee0ab843cf665da2a7473a08db branch: main author: Brandt Bucher committer: brandtbucher date: 2023-07-20T16:35:39Z summary: GH-104584: Miscellaneous fixes for -Xuops (GH-106908) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-20-01-15-58.gh-issue-106908.cDmcVI.rst M Lib/dis.py M Lib/test/test_capi/test_misc.py M Lib/test/test_dis.py M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/optimizer.c M Python/pylifecycle.c M Tools/cases_generator/generate_cases.py diff --git a/Lib/dis.py b/Lib/dis.py index f7a31f2f96b99..184eb14a5b2b2 100644 --- a/Lib/dis.py +++ b/Lib/dis.py @@ -430,7 +430,8 @@ def get_instructions(x, *, first_line=None, show_caches=False, adaptive=False): co.co_names, co.co_consts, linestarts, line_offset, co_positions=co.co_positions(), - show_caches=show_caches) + show_caches=show_caches, + original_code=co.co_code) def _get_const_value(op, arg, co_consts): """Helper to get the value of the const in a hasconst op. @@ -504,7 +505,7 @@ def _get_instructions_bytes(code, varname_from_oparg=None, names=None, co_consts=None, linestarts=None, line_offset=0, exception_entries=(), co_positions=None, - show_caches=False): + show_caches=False, original_code=None): """Iterate over the instructions in a bytecode string. Generates a sequence of Instruction namedtuples giving the details of each @@ -513,14 +514,18 @@ def _get_instructions_bytes(code, varname_from_oparg=None, arguments. """ + # Use the basic, unadaptive code for finding labels and actually walking the + # bytecode, since replacements like ENTER_EXECUTOR and INSTRUMENTED_* can + # mess that logic up pretty badly: + original_code = original_code or code co_positions = co_positions or iter(()) get_name = None if names is None else names.__getitem__ - labels = set(findlabels(code)) + labels = set(findlabels(original_code)) for start, end, target, _, _ in exception_entries: for i in range(start, end): labels.add(target) starts_line = None - for offset, start_offset, op, arg in _unpack_opargs(code): + for offset, start_offset, op, arg in _unpack_opargs(original_code): if linestarts is not None: starts_line = linestarts.get(offset, None) if starts_line is not None: @@ -531,6 +536,7 @@ def _get_instructions_bytes(code, varname_from_oparg=None, positions = Positions(*next(co_positions, ())) deop = _deoptop(op) caches = _inline_cache_entries[deop] + op = code[offset] if arg is not None: # Set argval to the dereferenced value of the argument when # available, and argrepr to the string representation of argval. @@ -591,7 +597,6 @@ def _get_instructions_bytes(code, varname_from_oparg=None, yield Instruction(_all_opname[op], op, arg, argval, argrepr, offset, start_offset, starts_line, is_jump_target, positions) - caches = _inline_cache_entries[deop] if not caches: continue if not show_caches: @@ -622,7 +627,8 @@ def disassemble(co, lasti=-1, *, file=None, show_caches=False, adaptive=False): lasti, co._varname_from_oparg, co.co_names, co.co_consts, linestarts, file=file, exception_entries=exception_entries, - co_positions=co.co_positions(), show_caches=show_caches) + co_positions=co.co_positions(), show_caches=show_caches, + original_code=co.co_code) def _disassemble_recursive(co, *, file=None, depth=None, show_caches=False, adaptive=False): disassemble(co, file=file, show_caches=show_caches, adaptive=adaptive) @@ -640,7 +646,7 @@ def _disassemble_recursive(co, *, file=None, depth=None, show_caches=False, adap def _disassemble_bytes(code, lasti=-1, varname_from_oparg=None, names=None, co_consts=None, linestarts=None, *, file=None, line_offset=0, exception_entries=(), - co_positions=None, show_caches=False): + co_positions=None, show_caches=False, original_code=None): # Omit the line number column entirely if we have no line number info show_lineno = bool(linestarts) if show_lineno: @@ -661,7 +667,8 @@ def _disassemble_bytes(code, lasti=-1, varname_from_oparg=None, line_offset=line_offset, exception_entries=exception_entries, co_positions=co_positions, - show_caches=show_caches): + show_caches=show_caches, + original_code=original_code): new_source_line = (show_lineno and instr.starts_line is not None and instr.offset > 0) @@ -823,7 +830,8 @@ def __iter__(self): line_offset=self._line_offset, exception_entries=self.exception_entries, co_positions=co.co_positions(), - show_caches=self.show_caches) + show_caches=self.show_caches, + original_code=co.co_code) def __repr__(self): return "{}({!r})".format(self.__class__.__name__, @@ -859,7 +867,8 @@ def dis(self): lasti=offset, exception_entries=self.exception_entries, co_positions=co.co_positions(), - show_caches=self.show_caches) + show_caches=self.show_caches, + original_code=co.co_code) return output.getvalue() diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 4e519fa73c50c..e40ffdfd12151 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2368,12 +2368,16 @@ def clear_executors(func): class TestOptimizerAPI(unittest.TestCase): def test_get_set_optimizer(self): - self.assertEqual(_testinternalcapi.get_optimizer(), None) + old = _testinternalcapi.get_optimizer() opt = _testinternalcapi.get_counter_optimizer() - _testinternalcapi.set_optimizer(opt) - self.assertEqual(_testinternalcapi.get_optimizer(), opt) - _testinternalcapi.set_optimizer(None) - self.assertEqual(_testinternalcapi.get_optimizer(), None) + try: + _testinternalcapi.set_optimizer(opt) + self.assertEqual(_testinternalcapi.get_optimizer(), opt) + _testinternalcapi.set_optimizer(None) + self.assertEqual(_testinternalcapi.get_optimizer(), None) + finally: + _testinternalcapi.set_optimizer(old) + def test_counter_optimizer(self): # Generate a new function at each call diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index 8597b8f14ac05..642b26163ab24 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -1244,10 +1244,14 @@ def test_call_specialize(self): @cpython_only @requires_specialization def test_loop_quicken(self): + import _testinternalcapi # Loop can trigger a quicken where the loop is located self.code_quicken(loop_test, 1) got = self.get_disassembly(loop_test, adaptive=True) - self.do_disassembly_compare(got, dis_loop_test_quickened_code) + expected = dis_loop_test_quickened_code + if _testinternalcapi.get_optimizer(): + expected = expected.replace("JUMP_BACKWARD ", "ENTER_EXECUTOR") + self.do_disassembly_compare(got, expected) @cpython_only def test_extended_arg_quick(self): diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-20-01-15-58.gh-issue-106908.cDmcVI.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-01-15-58.gh-issue-106908.cDmcVI.rst new file mode 100644 index 0000000000000..9c9b84599cb55 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-01-15-58.gh-issue-106908.cDmcVI.rst @@ -0,0 +1,3 @@ +Fix various hangs, reference leaks, test failures, and tracing/introspection +bugs when running with :envvar:`PYTHONUOPS` or :option:`-X uops <-X>` +enabled. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index ea136a3fca2e0..81d6f80709a4a 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2182,7 +2182,14 @@ dummy_func( JUMPBY(1-oparg); #if ENABLE_SPECIALIZATION here[1].cache += (1 << OPTIMIZER_BITS_IN_COUNTER); - if (here[1].cache > tstate->interp->optimizer_backedge_threshold) { + if (here[1].cache > tstate->interp->optimizer_backedge_threshold && + // Double-check that the opcode isn't instrumented or something: + here->op.code == JUMP_BACKWARD && + // _PyOptimizer_BackEdge is going to change frame->prev_instr, + // which breaks line event calculations: + next_instr->op.code != INSTRUMENTED_LINE + ) + { OBJECT_STAT_INC(optimization_attempts); frame = _PyOptimizer_BackEdge(frame, here, next_instr, stack_pointer); if (frame == NULL) { diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index e1f8b9f208c76..90607e86c041d 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -2017,6 +2017,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2038,6 +2039,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2089,6 +2091,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2131,6 +2134,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2243,6 +2247,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2281,6 +2286,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2318,6 +2324,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2496,6 +2503,7 @@ case JUMP_TO_TOP: { pc = 0; + CHECK_EVAL_BREAKER(); break; } diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index b2b0aa6ece481..c35b81aee88a0 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -2758,7 +2758,14 @@ JUMPBY(1-oparg); #if ENABLE_SPECIALIZATION here[1].cache += (1 << OPTIMIZER_BITS_IN_COUNTER); - if (here[1].cache > tstate->interp->optimizer_backedge_threshold) { + if (here[1].cache > tstate->interp->optimizer_backedge_threshold && + // Double-check that the opcode isn't instrumented or something: + here->op.code == JUMP_BACKWARD && + // _PyOptimizer_BackEdge is going to change frame->prev_instr, + // which breaks line event calculations: + next_instr->op.code != INSTRUMENTED_LINE + ) + { OBJECT_STAT_INC(optimization_attempts); frame = _PyOptimizer_BackEdge(frame, here, next_instr, stack_pointer); if (frame == NULL) { diff --git a/Python/optimizer.c b/Python/optimizer.c index 3d385a1506cba..09120c33d130c 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -155,6 +155,7 @@ PyUnstable_SetOptimizer(_PyOptimizerObject *optimizer) _PyInterpreterFrame * _PyOptimizer_BackEdge(_PyInterpreterFrame *frame, _Py_CODEUNIT *src, _Py_CODEUNIT *dest, PyObject **stack_pointer) { + assert(src->op.code == JUMP_BACKWARD); PyCodeObject *code = (PyCodeObject *)frame->f_executable; assert(PyCode_Check(code)); PyInterpreterState *interp = _PyInterpreterState_GET(); diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index cf8b4379c1467..f91fac9379ec5 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1192,7 +1192,11 @@ init_interp_main(PyThreadState *tstate) } if (enabled) { PyObject *opt = PyUnstable_Optimizer_NewUOpOptimizer(); + if (opt == NULL) { + return _PyStatus_ERR("can't initialize optimizer"); + } PyUnstable_SetOptimizer((_PyOptimizerObject *)opt); + Py_DECREF(opt); } } diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 33eff548a1880..a18229a57143b 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -1506,6 +1506,8 @@ def write_executor_instructions(self) -> None: self.out.emit("") with self.out.block(f"case {thing.name}:"): instr.write(self.out, tier=TIER_TWO) + if instr.check_eval_breaker: + self.out.emit("CHECK_EVAL_BREAKER();") self.out.emit("break;") # elif instr.kind != "op": # print(f"NOTE: {thing.name} is not a viable uop") From webhook-mailer at python.org Thu Jul 20 12:35:43 2023 From: webhook-mailer at python.org (brandtbucher) Date: Thu, 20 Jul 2023 16:35:43 -0000 Subject: [Python-checkins] GH-104584: Miscellaneous fixes for -Xuops (GH-106908) Message-ID: https://github.com/python/cpython/commit/214a25dd81dfe5ee0ab843cf665da2a7473a08db commit: 214a25dd81dfe5ee0ab843cf665da2a7473a08db branch: main author: Brandt Bucher committer: brandtbucher date: 2023-07-20T16:35:39Z summary: GH-104584: Miscellaneous fixes for -Xuops (GH-106908) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-20-01-15-58.gh-issue-106908.cDmcVI.rst M Lib/dis.py M Lib/test/test_capi/test_misc.py M Lib/test/test_dis.py M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/optimizer.c M Python/pylifecycle.c M Tools/cases_generator/generate_cases.py diff --git a/Lib/dis.py b/Lib/dis.py index f7a31f2f96b99..184eb14a5b2b2 100644 --- a/Lib/dis.py +++ b/Lib/dis.py @@ -430,7 +430,8 @@ def get_instructions(x, *, first_line=None, show_caches=False, adaptive=False): co.co_names, co.co_consts, linestarts, line_offset, co_positions=co.co_positions(), - show_caches=show_caches) + show_caches=show_caches, + original_code=co.co_code) def _get_const_value(op, arg, co_consts): """Helper to get the value of the const in a hasconst op. @@ -504,7 +505,7 @@ def _get_instructions_bytes(code, varname_from_oparg=None, names=None, co_consts=None, linestarts=None, line_offset=0, exception_entries=(), co_positions=None, - show_caches=False): + show_caches=False, original_code=None): """Iterate over the instructions in a bytecode string. Generates a sequence of Instruction namedtuples giving the details of each @@ -513,14 +514,18 @@ def _get_instructions_bytes(code, varname_from_oparg=None, arguments. """ + # Use the basic, unadaptive code for finding labels and actually walking the + # bytecode, since replacements like ENTER_EXECUTOR and INSTRUMENTED_* can + # mess that logic up pretty badly: + original_code = original_code or code co_positions = co_positions or iter(()) get_name = None if names is None else names.__getitem__ - labels = set(findlabels(code)) + labels = set(findlabels(original_code)) for start, end, target, _, _ in exception_entries: for i in range(start, end): labels.add(target) starts_line = None - for offset, start_offset, op, arg in _unpack_opargs(code): + for offset, start_offset, op, arg in _unpack_opargs(original_code): if linestarts is not None: starts_line = linestarts.get(offset, None) if starts_line is not None: @@ -531,6 +536,7 @@ def _get_instructions_bytes(code, varname_from_oparg=None, positions = Positions(*next(co_positions, ())) deop = _deoptop(op) caches = _inline_cache_entries[deop] + op = code[offset] if arg is not None: # Set argval to the dereferenced value of the argument when # available, and argrepr to the string representation of argval. @@ -591,7 +597,6 @@ def _get_instructions_bytes(code, varname_from_oparg=None, yield Instruction(_all_opname[op], op, arg, argval, argrepr, offset, start_offset, starts_line, is_jump_target, positions) - caches = _inline_cache_entries[deop] if not caches: continue if not show_caches: @@ -622,7 +627,8 @@ def disassemble(co, lasti=-1, *, file=None, show_caches=False, adaptive=False): lasti, co._varname_from_oparg, co.co_names, co.co_consts, linestarts, file=file, exception_entries=exception_entries, - co_positions=co.co_positions(), show_caches=show_caches) + co_positions=co.co_positions(), show_caches=show_caches, + original_code=co.co_code) def _disassemble_recursive(co, *, file=None, depth=None, show_caches=False, adaptive=False): disassemble(co, file=file, show_caches=show_caches, adaptive=adaptive) @@ -640,7 +646,7 @@ def _disassemble_recursive(co, *, file=None, depth=None, show_caches=False, adap def _disassemble_bytes(code, lasti=-1, varname_from_oparg=None, names=None, co_consts=None, linestarts=None, *, file=None, line_offset=0, exception_entries=(), - co_positions=None, show_caches=False): + co_positions=None, show_caches=False, original_code=None): # Omit the line number column entirely if we have no line number info show_lineno = bool(linestarts) if show_lineno: @@ -661,7 +667,8 @@ def _disassemble_bytes(code, lasti=-1, varname_from_oparg=None, line_offset=line_offset, exception_entries=exception_entries, co_positions=co_positions, - show_caches=show_caches): + show_caches=show_caches, + original_code=original_code): new_source_line = (show_lineno and instr.starts_line is not None and instr.offset > 0) @@ -823,7 +830,8 @@ def __iter__(self): line_offset=self._line_offset, exception_entries=self.exception_entries, co_positions=co.co_positions(), - show_caches=self.show_caches) + show_caches=self.show_caches, + original_code=co.co_code) def __repr__(self): return "{}({!r})".format(self.__class__.__name__, @@ -859,7 +867,8 @@ def dis(self): lasti=offset, exception_entries=self.exception_entries, co_positions=co.co_positions(), - show_caches=self.show_caches) + show_caches=self.show_caches, + original_code=co.co_code) return output.getvalue() diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 4e519fa73c50c..e40ffdfd12151 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2368,12 +2368,16 @@ def clear_executors(func): class TestOptimizerAPI(unittest.TestCase): def test_get_set_optimizer(self): - self.assertEqual(_testinternalcapi.get_optimizer(), None) + old = _testinternalcapi.get_optimizer() opt = _testinternalcapi.get_counter_optimizer() - _testinternalcapi.set_optimizer(opt) - self.assertEqual(_testinternalcapi.get_optimizer(), opt) - _testinternalcapi.set_optimizer(None) - self.assertEqual(_testinternalcapi.get_optimizer(), None) + try: + _testinternalcapi.set_optimizer(opt) + self.assertEqual(_testinternalcapi.get_optimizer(), opt) + _testinternalcapi.set_optimizer(None) + self.assertEqual(_testinternalcapi.get_optimizer(), None) + finally: + _testinternalcapi.set_optimizer(old) + def test_counter_optimizer(self): # Generate a new function at each call diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index 8597b8f14ac05..642b26163ab24 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -1244,10 +1244,14 @@ def test_call_specialize(self): @cpython_only @requires_specialization def test_loop_quicken(self): + import _testinternalcapi # Loop can trigger a quicken where the loop is located self.code_quicken(loop_test, 1) got = self.get_disassembly(loop_test, adaptive=True) - self.do_disassembly_compare(got, dis_loop_test_quickened_code) + expected = dis_loop_test_quickened_code + if _testinternalcapi.get_optimizer(): + expected = expected.replace("JUMP_BACKWARD ", "ENTER_EXECUTOR") + self.do_disassembly_compare(got, expected) @cpython_only def test_extended_arg_quick(self): diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-20-01-15-58.gh-issue-106908.cDmcVI.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-01-15-58.gh-issue-106908.cDmcVI.rst new file mode 100644 index 0000000000000..9c9b84599cb55 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-01-15-58.gh-issue-106908.cDmcVI.rst @@ -0,0 +1,3 @@ +Fix various hangs, reference leaks, test failures, and tracing/introspection +bugs when running with :envvar:`PYTHONUOPS` or :option:`-X uops <-X>` +enabled. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index ea136a3fca2e0..81d6f80709a4a 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2182,7 +2182,14 @@ dummy_func( JUMPBY(1-oparg); #if ENABLE_SPECIALIZATION here[1].cache += (1 << OPTIMIZER_BITS_IN_COUNTER); - if (here[1].cache > tstate->interp->optimizer_backedge_threshold) { + if (here[1].cache > tstate->interp->optimizer_backedge_threshold && + // Double-check that the opcode isn't instrumented or something: + here->op.code == JUMP_BACKWARD && + // _PyOptimizer_BackEdge is going to change frame->prev_instr, + // which breaks line event calculations: + next_instr->op.code != INSTRUMENTED_LINE + ) + { OBJECT_STAT_INC(optimization_attempts); frame = _PyOptimizer_BackEdge(frame, here, next_instr, stack_pointer); if (frame == NULL) { diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index e1f8b9f208c76..90607e86c041d 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -2017,6 +2017,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2038,6 +2039,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2089,6 +2091,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2131,6 +2134,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2243,6 +2247,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2281,6 +2286,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2318,6 +2324,7 @@ STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; + CHECK_EVAL_BREAKER(); break; } @@ -2496,6 +2503,7 @@ case JUMP_TO_TOP: { pc = 0; + CHECK_EVAL_BREAKER(); break; } diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index b2b0aa6ece481..c35b81aee88a0 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -2758,7 +2758,14 @@ JUMPBY(1-oparg); #if ENABLE_SPECIALIZATION here[1].cache += (1 << OPTIMIZER_BITS_IN_COUNTER); - if (here[1].cache > tstate->interp->optimizer_backedge_threshold) { + if (here[1].cache > tstate->interp->optimizer_backedge_threshold && + // Double-check that the opcode isn't instrumented or something: + here->op.code == JUMP_BACKWARD && + // _PyOptimizer_BackEdge is going to change frame->prev_instr, + // which breaks line event calculations: + next_instr->op.code != INSTRUMENTED_LINE + ) + { OBJECT_STAT_INC(optimization_attempts); frame = _PyOptimizer_BackEdge(frame, here, next_instr, stack_pointer); if (frame == NULL) { diff --git a/Python/optimizer.c b/Python/optimizer.c index 3d385a1506cba..09120c33d130c 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -155,6 +155,7 @@ PyUnstable_SetOptimizer(_PyOptimizerObject *optimizer) _PyInterpreterFrame * _PyOptimizer_BackEdge(_PyInterpreterFrame *frame, _Py_CODEUNIT *src, _Py_CODEUNIT *dest, PyObject **stack_pointer) { + assert(src->op.code == JUMP_BACKWARD); PyCodeObject *code = (PyCodeObject *)frame->f_executable; assert(PyCode_Check(code)); PyInterpreterState *interp = _PyInterpreterState_GET(); diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index cf8b4379c1467..f91fac9379ec5 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1192,7 +1192,11 @@ init_interp_main(PyThreadState *tstate) } if (enabled) { PyObject *opt = PyUnstable_Optimizer_NewUOpOptimizer(); + if (opt == NULL) { + return _PyStatus_ERR("can't initialize optimizer"); + } PyUnstable_SetOptimizer((_PyOptimizerObject *)opt); + Py_DECREF(opt); } } diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 33eff548a1880..a18229a57143b 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -1506,6 +1506,8 @@ def write_executor_instructions(self) -> None: self.out.emit("") with self.out.block(f"case {thing.name}:"): instr.write(self.out, tier=TIER_TWO) + if instr.check_eval_breaker: + self.out.emit("CHECK_EVAL_BREAKER();") self.out.emit("break;") # elif instr.kind != "op": # print(f"NOTE: {thing.name} is not a viable uop") From webhook-mailer at python.org Thu Jul 20 12:46:08 2023 From: webhook-mailer at python.org (iritkatriel) Date: Thu, 20 Jul 2023 16:46:08 -0000 Subject: [Python-checkins] gh-105481: do not auto-generate pycore_intrinsics.h (#106913) Message-ID: https://github.com/python/cpython/commit/9c81fc2dbee3ac8a2f30ad24b0876d80628a94ac commit: 9c81fc2dbee3ac8a2f30ad24b0876d80628a94ac branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-20T17:46:04+01:00 summary: gh-105481: do not auto-generate pycore_intrinsics.h (#106913) files: M Include/cpython/compile.h M Include/internal/pycore_intrinsics.h M Lib/opcode.py M Makefile.pre.in M Modules/_opcode.c M Modules/clinic/_opcode.c.h M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/intrinsics.c M Tools/build/generate_opcode_h.py diff --git a/Include/cpython/compile.h b/Include/cpython/compile.h index fd52697840203..e6cd39af2ba73 100644 --- a/Include/cpython/compile.h +++ b/Include/cpython/compile.h @@ -77,3 +77,5 @@ PyAPI_FUNC(int) PyUnstable_OpcodeHasFree(int opcode); PyAPI_FUNC(int) PyUnstable_OpcodeHasLocal(int opcode); PyAPI_FUNC(int) PyUnstable_OpcodeHasExc(int opcode); +PyAPI_FUNC(PyObject*) _PyUnstable_GetUnaryIntrinsicName(int index); +PyAPI_FUNC(PyObject*) _PyUnstable_GetBinaryIntrinsicName(int index); diff --git a/Include/internal/pycore_intrinsics.h b/Include/internal/pycore_intrinsics.h index 39f15681b7b24..37d4efc12bb77 100644 --- a/Include/internal/pycore_intrinsics.h +++ b/Include/internal/pycore_intrinsics.h @@ -1,4 +1,3 @@ -// Auto-generated by Tools/build/generate_opcode_h.py from Lib/opcode.py /* Unary Functions: */ #define INTRINSIC_1_INVALID 0 @@ -26,7 +25,18 @@ #define MAX_INTRINSIC_2 4 -typedef PyObject *(*instrinsic_func1)(PyThreadState* tstate, PyObject *value); -typedef PyObject *(*instrinsic_func2)(PyThreadState* tstate, PyObject *value1, PyObject *value2); -extern const instrinsic_func1 _PyIntrinsics_UnaryFunctions[]; -extern const instrinsic_func2 _PyIntrinsics_BinaryFunctions[]; +typedef PyObject *(*intrinsic_func1)(PyThreadState* tstate, PyObject *value); +typedef PyObject *(*intrinsic_func2)(PyThreadState* tstate, PyObject *value1, PyObject *value2); + +typedef struct { + intrinsic_func1 func; + const char *name; +} intrinsic_func1_info; + +typedef struct { + intrinsic_func2 func; + const char *name; +} intrinsic_func2_info; + +extern const intrinsic_func1_info _PyIntrinsics_UnaryFunctions[]; +extern const intrinsic_func2_info _PyIntrinsics_BinaryFunctions[]; diff --git a/Lib/opcode.py b/Lib/opcode.py index 08dfd2674dca7..36831d8a122bf 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -257,6 +257,9 @@ def pseudo_op(name, op, real_ops): __all__.extend(["hasarg", "hasconst", "hasname", "hasjump", "hasjrel", "hasjabs", "hasfree", "haslocal", "hasexc"]) + _intrinsic_1_descs = _opcode.get_intrinsic1_descs() + _intrinsic_2_descs = _opcode.get_intrinsic2_descs() + hascompare = [opmap["COMPARE_OP"]] _nb_ops = [ @@ -288,29 +291,6 @@ def pseudo_op(name, op, real_ops): ("NB_INPLACE_XOR", "^="), ] -_intrinsic_1_descs = [ - "INTRINSIC_1_INVALID", - "INTRINSIC_PRINT", - "INTRINSIC_IMPORT_STAR", - "INTRINSIC_STOPITERATION_ERROR", - "INTRINSIC_ASYNC_GEN_WRAP", - "INTRINSIC_UNARY_POSITIVE", - "INTRINSIC_LIST_TO_TUPLE", - "INTRINSIC_TYPEVAR", - "INTRINSIC_PARAMSPEC", - "INTRINSIC_TYPEVARTUPLE", - "INTRINSIC_SUBSCRIPT_GENERIC", - "INTRINSIC_TYPEALIAS", -] - -_intrinsic_2_descs = [ - "INTRINSIC_2_INVALID", - "INTRINSIC_PREP_RERAISE_STAR", - "INTRINSIC_TYPEVAR_WITH_BOUND", - "INTRINSIC_TYPEVAR_WITH_CONSTRAINTS", - "INTRINSIC_SET_FUNCTION_TYPE_PARAMS", -] - _cache_format = { "LOAD_GLOBAL": { diff --git a/Makefile.pre.in b/Makefile.pre.in index 553b2aa480c18..f4f6ff595b0e6 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1427,11 +1427,9 @@ regen-opcode: $(srcdir)/Lib/opcode.py \ $(srcdir)/Lib/_opcode_metadata.py \ $(srcdir)/Include/opcode.h.new \ - $(srcdir)/Include/internal/pycore_opcode.h.new \ - $(srcdir)/Include/internal/pycore_intrinsics.h.new + $(srcdir)/Include/internal/pycore_opcode.h.new $(UPDATE_FILE) $(srcdir)/Include/opcode.h $(srcdir)/Include/opcode.h.new $(UPDATE_FILE) $(srcdir)/Include/internal/pycore_opcode.h $(srcdir)/Include/internal/pycore_opcode.h.new - $(UPDATE_FILE) $(srcdir)/Include/internal/pycore_intrinsics.h $(srcdir)/Include/internal/pycore_intrinsics.h.new .PHONY: regen-token regen-token: diff --git a/Modules/_opcode.c b/Modules/_opcode.c index daabdce165577..b8d95a048ec2c 100644 --- a/Modules/_opcode.c +++ b/Modules/_opcode.c @@ -2,6 +2,7 @@ #include "compile.h" #include "opcode.h" #include "internal/pycore_code.h" +#include "internal/pycore_intrinsics.h" /*[clinic input] module _opcode @@ -220,6 +221,60 @@ _opcode_get_specialization_stats_impl(PyObject *module) #endif } +/*[clinic input] + +_opcode.get_intrinsic1_descs + +Return a list of names of the unary intrinsics. +[clinic start generated code]*/ + +static PyObject * +_opcode_get_intrinsic1_descs_impl(PyObject *module) +/*[clinic end generated code: output=bd1ddb6b4447d18b input=13b51c712618459b]*/ +{ + PyObject *list = PyList_New(MAX_INTRINSIC_1 + 1); + if (list == NULL) { + return NULL; + } + for (int i=0; i <= MAX_INTRINSIC_1; i++) { + PyObject *name = _PyUnstable_GetUnaryIntrinsicName(i); + if (name == NULL) { + Py_DECREF(list); + return NULL; + } + PyList_SET_ITEM(list, i, name); + } + return list; +} + + +/*[clinic input] + +_opcode.get_intrinsic2_descs + +Return a list of names of the binary intrinsics. +[clinic start generated code]*/ + +static PyObject * +_opcode_get_intrinsic2_descs_impl(PyObject *module) +/*[clinic end generated code: output=40e62bc27584c8a0 input=e83068f249f5471b]*/ +{ + PyObject *list = PyList_New(MAX_INTRINSIC_2 + 1); + if (list == NULL) { + return NULL; + } + for (int i=0; i <= MAX_INTRINSIC_2; i++) { + PyObject *name = _PyUnstable_GetBinaryIntrinsicName(i); + if (name == NULL) { + Py_DECREF(list); + return NULL; + } + PyList_SET_ITEM(list, i, name); + } + return list; +} + + static PyMethodDef opcode_functions[] = { _OPCODE_STACK_EFFECT_METHODDEF @@ -232,6 +287,8 @@ opcode_functions[] = { _OPCODE_HAS_LOCAL_METHODDEF _OPCODE_HAS_EXC_METHODDEF _OPCODE_GET_SPECIALIZATION_STATS_METHODDEF + _OPCODE_GET_INTRINSIC1_DESCS_METHODDEF + _OPCODE_GET_INTRINSIC2_DESCS_METHODDEF {NULL, NULL, 0, NULL} }; diff --git a/Modules/clinic/_opcode.c.h b/Modules/clinic/_opcode.c.h index e6381fa73a550..e1fc5ba17f707 100644 --- a/Modules/clinic/_opcode.c.h +++ b/Modules/clinic/_opcode.c.h @@ -612,4 +612,40 @@ _opcode_get_specialization_stats(PyObject *module, PyObject *Py_UNUSED(ignored)) { return _opcode_get_specialization_stats_impl(module); } -/*[clinic end generated code: output=e507bf14fb2796f8 input=a9049054013a1b77]*/ + +PyDoc_STRVAR(_opcode_get_intrinsic1_descs__doc__, +"get_intrinsic1_descs($module, /)\n" +"--\n" +"\n" +"Return a list of names of the unary intrinsics."); + +#define _OPCODE_GET_INTRINSIC1_DESCS_METHODDEF \ + {"get_intrinsic1_descs", (PyCFunction)_opcode_get_intrinsic1_descs, METH_NOARGS, _opcode_get_intrinsic1_descs__doc__}, + +static PyObject * +_opcode_get_intrinsic1_descs_impl(PyObject *module); + +static PyObject * +_opcode_get_intrinsic1_descs(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return _opcode_get_intrinsic1_descs_impl(module); +} + +PyDoc_STRVAR(_opcode_get_intrinsic2_descs__doc__, +"get_intrinsic2_descs($module, /)\n" +"--\n" +"\n" +"Return a list of names of the binary intrinsics."); + +#define _OPCODE_GET_INTRINSIC2_DESCS_METHODDEF \ + {"get_intrinsic2_descs", (PyCFunction)_opcode_get_intrinsic2_descs, METH_NOARGS, _opcode_get_intrinsic2_descs__doc__}, + +static PyObject * +_opcode_get_intrinsic2_descs_impl(PyObject *module); + +static PyObject * +_opcode_get_intrinsic2_descs(PyObject *module, PyObject *Py_UNUSED(ignored)) +{ + return _opcode_get_intrinsic2_descs_impl(module); +} +/*[clinic end generated code: output=d85de5f2887b3661 input=a9049054013a1b77]*/ diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 81d6f80709a4a..b9434390ea489 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -698,14 +698,14 @@ dummy_func( inst(CALL_INTRINSIC_1, (value -- res)) { assert(oparg <= MAX_INTRINSIC_1); - res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value); + res = _PyIntrinsics_UnaryFunctions[oparg].func(tstate, value); DECREF_INPUTS(); ERROR_IF(res == NULL, error); } inst(CALL_INTRINSIC_2, (value2, value1 -- res)) { assert(oparg <= MAX_INTRINSIC_2); - res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1); + res = _PyIntrinsics_BinaryFunctions[oparg].func(tstate, value2, value1); DECREF_INPUTS(); ERROR_IF(res == NULL, error); } diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 90607e86c041d..5b38c6a5fd8a2 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -558,7 +558,7 @@ PyObject *value = stack_pointer[-1]; PyObject *res; assert(oparg <= MAX_INTRINSIC_1); - res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value); + res = _PyIntrinsics_UnaryFunctions[oparg].func(tstate, value); Py_DECREF(value); if (res == NULL) goto pop_1_error; stack_pointer[-1] = res; @@ -570,7 +570,7 @@ PyObject *value2 = stack_pointer[-2]; PyObject *res; assert(oparg <= MAX_INTRINSIC_2); - res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1); + res = _PyIntrinsics_BinaryFunctions[oparg].func(tstate, value2, value1); Py_DECREF(value2); Py_DECREF(value1); if (res == NULL) goto pop_2_error; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index c35b81aee88a0..2d70966aa0913 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -849,7 +849,7 @@ PyObject *value = stack_pointer[-1]; PyObject *res; assert(oparg <= MAX_INTRINSIC_1); - res = _PyIntrinsics_UnaryFunctions[oparg](tstate, value); + res = _PyIntrinsics_UnaryFunctions[oparg].func(tstate, value); Py_DECREF(value); if (res == NULL) goto pop_1_error; stack_pointer[-1] = res; @@ -861,7 +861,7 @@ PyObject *value2 = stack_pointer[-2]; PyObject *res; assert(oparg <= MAX_INTRINSIC_2); - res = _PyIntrinsics_BinaryFunctions[oparg](tstate, value2, value1); + res = _PyIntrinsics_BinaryFunctions[oparg].func(tstate, value2, value1); Py_DECREF(value2); Py_DECREF(value1); if (res == NULL) goto pop_2_error; diff --git a/Python/intrinsics.c b/Python/intrinsics.c index 037b74ca820fa..61a8e75872d2e 100644 --- a/Python/intrinsics.c +++ b/Python/intrinsics.c @@ -14,7 +14,7 @@ /******** Unary functions ********/ static PyObject * -no_intrinsic(PyThreadState* tstate, PyObject *unused) +no_intrinsic1(PyThreadState* tstate, PyObject *unused) { _PyErr_SetString(tstate, PyExc_SystemError, "invalid intrinsic function"); return NULL; @@ -203,25 +203,35 @@ make_typevar(PyThreadState* Py_UNUSED(ignored), PyObject *v) return _Py_make_typevar(v, NULL, NULL); } -const instrinsic_func1 + +#define INTRINSIC_FUNC_ENTRY(N, F) \ + [N] = {F, #N}, + +const intrinsic_func1_info _PyIntrinsics_UnaryFunctions[] = { - [0] = no_intrinsic, - [INTRINSIC_PRINT] = print_expr, - [INTRINSIC_IMPORT_STAR] = import_star, - [INTRINSIC_STOPITERATION_ERROR] = stopiteration_error, - [INTRINSIC_ASYNC_GEN_WRAP] = _PyAsyncGenValueWrapperNew, - [INTRINSIC_UNARY_POSITIVE] = unary_pos, - [INTRINSIC_LIST_TO_TUPLE] = list_to_tuple, - [INTRINSIC_TYPEVAR] = make_typevar, - [INTRINSIC_PARAMSPEC] = _Py_make_paramspec, - [INTRINSIC_TYPEVARTUPLE] = _Py_make_typevartuple, - [INTRINSIC_SUBSCRIPT_GENERIC] = _Py_subscript_generic, - [INTRINSIC_TYPEALIAS] = _Py_make_typealias, + INTRINSIC_FUNC_ENTRY(INTRINSIC_1_INVALID, no_intrinsic1) + INTRINSIC_FUNC_ENTRY(INTRINSIC_PRINT, print_expr) + INTRINSIC_FUNC_ENTRY(INTRINSIC_IMPORT_STAR, import_star) + INTRINSIC_FUNC_ENTRY(INTRINSIC_STOPITERATION_ERROR, stopiteration_error) + INTRINSIC_FUNC_ENTRY(INTRINSIC_ASYNC_GEN_WRAP, _PyAsyncGenValueWrapperNew) + INTRINSIC_FUNC_ENTRY(INTRINSIC_UNARY_POSITIVE, unary_pos) + INTRINSIC_FUNC_ENTRY(INTRINSIC_LIST_TO_TUPLE, list_to_tuple) + INTRINSIC_FUNC_ENTRY(INTRINSIC_TYPEVAR, make_typevar) + INTRINSIC_FUNC_ENTRY(INTRINSIC_PARAMSPEC, _Py_make_paramspec) + INTRINSIC_FUNC_ENTRY(INTRINSIC_TYPEVARTUPLE, _Py_make_typevartuple) + INTRINSIC_FUNC_ENTRY(INTRINSIC_SUBSCRIPT_GENERIC, _Py_subscript_generic) + INTRINSIC_FUNC_ENTRY(INTRINSIC_TYPEALIAS, _Py_make_typealias) }; /******** Binary functions ********/ +static PyObject * +no_intrinsic2(PyThreadState* tstate, PyObject *unused1, PyObject *unused2) +{ + _PyErr_SetString(tstate, PyExc_SystemError, "invalid intrinsic function"); + return NULL; +} static PyObject * prep_reraise_star(PyThreadState* unused, PyObject *orig, PyObject *excs) @@ -246,10 +256,31 @@ make_typevar_with_constraints(PyThreadState* Py_UNUSED(ignored), PyObject *name, return _Py_make_typevar(name, NULL, evaluate_constraints); } -const instrinsic_func2 +const intrinsic_func2_info _PyIntrinsics_BinaryFunctions[] = { - [INTRINSIC_PREP_RERAISE_STAR] = prep_reraise_star, - [INTRINSIC_TYPEVAR_WITH_BOUND] = make_typevar_with_bound, - [INTRINSIC_TYPEVAR_WITH_CONSTRAINTS] = make_typevar_with_constraints, - [INTRINSIC_SET_FUNCTION_TYPE_PARAMS] = _Py_set_function_type_params, + INTRINSIC_FUNC_ENTRY(INTRINSIC_2_INVALID, no_intrinsic2) + INTRINSIC_FUNC_ENTRY(INTRINSIC_PREP_RERAISE_STAR, prep_reraise_star) + INTRINSIC_FUNC_ENTRY(INTRINSIC_TYPEVAR_WITH_BOUND, make_typevar_with_bound) + INTRINSIC_FUNC_ENTRY(INTRINSIC_TYPEVAR_WITH_CONSTRAINTS, make_typevar_with_constraints) + INTRINSIC_FUNC_ENTRY(INTRINSIC_SET_FUNCTION_TYPE_PARAMS, _Py_set_function_type_params) }; + +#undef INTRINSIC_FUNC_ENTRY + +PyObject* +_PyUnstable_GetUnaryIntrinsicName(int index) +{ + if (index < 0 || index > MAX_INTRINSIC_1) { + return NULL; + } + return PyUnicode_FromString(_PyIntrinsics_UnaryFunctions[index].name); +} + +PyObject* +_PyUnstable_GetBinaryIntrinsicName(int index) +{ + if (index < 0 || index > MAX_INTRINSIC_2) { + return NULL; + } + return PyUnicode_FromString(_PyIntrinsics_BinaryFunctions[index].name); +} diff --git a/Tools/build/generate_opcode_h.py b/Tools/build/generate_opcode_h.py index 5b0560e6b21a9..179abcf989e9b 100644 --- a/Tools/build/generate_opcode_h.py +++ b/Tools/build/generate_opcode_h.py @@ -50,18 +50,6 @@ #endif // !Py_INTERNAL_OPCODE_H """ -intrinsic_header = f""" -// Auto-generated by {SCRIPT_NAME} from {PYTHON_OPCODE} - -""".lstrip() - -intrinsic_footer = """ -typedef PyObject *(*instrinsic_func1)(PyThreadState* tstate, PyObject *value); -typedef PyObject *(*instrinsic_func2)(PyThreadState* tstate, PyObject *value1, PyObject *value2); -extern const instrinsic_func1 _PyIntrinsics_UnaryFunctions[]; -extern const instrinsic_func2 _PyIntrinsics_BinaryFunctions[]; -""" - DEFINE = "#define {:<38} {:>3}\n" UINT32_MASK = (1<<32)-1 @@ -76,8 +64,7 @@ def get_python_module_dict(filename): def main(opcode_py, _opcode_metadata_py='Lib/_opcode_metadata.py', outfile='Include/opcode.h', - internaloutfile='Include/internal/pycore_opcode.h', - intrinsicoutfile='Include/internal/pycore_intrinsics.h'): + internaloutfile='Include/internal/pycore_opcode.h'): _opcode_metadata = get_python_module_dict(_opcode_metadata_py) @@ -107,11 +94,9 @@ def main(opcode_py, opname_including_specialized[next_op] = name used[next_op] = True - with open(outfile, 'w') as fobj, open(internaloutfile, 'w') as iobj, open( - intrinsicoutfile, "w") as nobj: + with open(outfile, 'w') as fobj, open(internaloutfile, 'w') as iobj: fobj.write(header) iobj.write(internal_header) - nobj.write(intrinsic_header) for name in opname: if name in opmap: @@ -157,22 +142,6 @@ def main(opcode_py, for i, (op, _) in enumerate(opcode["_nb_ops"]): fobj.write(DEFINE.format(op, i)) - nobj.write("/* Unary Functions: */") - nobj.write("\n") - for i, op in enumerate(opcode["_intrinsic_1_descs"]): - nobj.write(DEFINE.format(op, i)) - nobj.write("\n") - nobj.write(DEFINE.format("MAX_INTRINSIC_1", i)) - - nobj.write("\n\n") - nobj.write("/* Binary Functions: */\n") - for i, op in enumerate(opcode["_intrinsic_2_descs"]): - nobj.write(DEFINE.format(op, i)) - nobj.write("\n") - nobj.write(DEFINE.format("MAX_INTRINSIC_2", i)) - - nobj.write(intrinsic_footer) - fobj.write("\n") fobj.write("/* Defined in Lib/opcode.py */\n") fobj.write(f"#define ENABLE_SPECIALIZATION {int(ENABLE_SPECIALIZATION)}") @@ -203,4 +172,4 @@ def main(opcode_py, if __name__ == '__main__': - main(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5]) + main(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4]) From webhook-mailer at python.org Thu Jul 20 16:37:23 2023 From: webhook-mailer at python.org (brandtbucher) Date: Thu, 20 Jul 2023 20:37:23 -0000 Subject: [Python-checkins] GH-106701: Move _PyUopExecute to Python/executor.c (GH-106924) Message-ID: https://github.com/python/cpython/commit/8f4de57699446f2e0964dffc6639f8156e56c4b3 commit: 8f4de57699446f2e0964dffc6639f8156e56c4b3 branch: main author: Brandt Bucher committer: brandtbucher date: 2023-07-20T20:37:19Z summary: GH-106701: Move _PyUopExecute to Python/executor.c (GH-106924) files: A Python/executor.c M Include/internal/pycore_ceval.h M Makefile.pre.in M PCbuild/_freeze_module.vcxproj M PCbuild/_freeze_module.vcxproj.filters M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters M Python/bytecodes.c M Python/ceval.c M Python/ceval_macros.h M Python/executor_cases.c.h M Python/generated_cases.c.h M Tools/c-analyzer/cpython/ignored.tsv diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h index e729904ff2c4c..ea5b99ebba07b 100644 --- a/Include/internal/pycore_ceval.h +++ b/Include/internal/pycore_ceval.h @@ -158,6 +158,18 @@ extern int _Py_HandlePending(PyThreadState *tstate); extern PyObject * _PyEval_GetFrameLocals(void); +extern const binaryfunc _PyEval_BinaryOps[]; +int _PyEval_CheckExceptStarTypeValid(PyThreadState *tstate, PyObject* right); +int _PyEval_CheckExceptTypeValid(PyThreadState *tstate, PyObject* right); +int _PyEval_ExceptionGroupMatch(PyObject* exc_value, PyObject *match_type, PyObject **match, PyObject **rest); +void _PyEval_FormatAwaitableError(PyThreadState *tstate, PyTypeObject *type, int oparg); +void _PyEval_FormatExcCheckArg(PyThreadState *tstate, PyObject *exc, const char *format_str, PyObject *obj); +void _PyEval_FormatExcUnbound(PyThreadState *tstate, PyCodeObject *co, int oparg); +void _PyEval_FormatKwargsError(PyThreadState *tstate, PyObject *func, PyObject *kwargs); +PyObject *_PyEval_MatchClass(PyThreadState *tstate, PyObject *subject, PyObject *type, Py_ssize_t nargs, PyObject *kwargs); +PyObject *_PyEval_MatchKeys(PyThreadState *tstate, PyObject *map, PyObject *keys); +int _PyEval_UnpackIterable(PyThreadState *tstate, PyObject *v, int argcnt, int argcntafter, PyObject **sp); + #ifdef __cplusplus } diff --git a/Makefile.pre.in b/Makefile.pre.in index f4f6ff595b0e6..f3a0db163ed48 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -381,6 +381,7 @@ PYTHON_OBJS= \ Python/context.o \ Python/dynamic_annotations.o \ Python/errors.o \ + Python/executor.o \ Python/flowgraph.o \ Python/frame.o \ Python/frozenmain.o \ @@ -1562,10 +1563,13 @@ Python/ceval.o: \ $(srcdir)/Python/ceval_macros.h \ $(srcdir)/Python/condvar.h \ $(srcdir)/Python/generated_cases.c.h \ - $(srcdir)/Python/executor_cases.c.h \ - $(srcdir)/Include/internal/pycore_opcode_metadata.h \ $(srcdir)/Python/opcode_targets.h +Python/executor.o: \ + $(srcdir)/Include/internal/pycore_opcode_metadata.h \ + $(srcdir)/Python/ceval_macros.h \ + $(srcdir)/Python/executor_cases.c.h + Python/flowgraph.o: \ $(srcdir)/Include/internal/pycore_opcode_metadata.h diff --git a/PCbuild/_freeze_module.vcxproj b/PCbuild/_freeze_module.vcxproj index 87f08e857d0e2..e247637a0dfe5 100644 --- a/PCbuild/_freeze_module.vcxproj +++ b/PCbuild/_freeze_module.vcxproj @@ -192,6 +192,7 @@ + diff --git a/PCbuild/_freeze_module.vcxproj.filters b/PCbuild/_freeze_module.vcxproj.filters index 3dae8cca7a2d5..2a0e009308022 100644 --- a/PCbuild/_freeze_module.vcxproj.filters +++ b/PCbuild/_freeze_module.vcxproj.filters @@ -124,6 +124,9 @@ Source Files + + Source Files + Source Files diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 760962e4c4b6a..0fe24082ca4cc 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -520,6 +520,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index aaebe1908e30d..2b3793cb0f76e 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -1145,6 +1145,9 @@ Python + + Python + Python diff --git a/Python/bytecodes.c b/Python/bytecodes.c index b9434390ea489..3a191aca730e9 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -77,7 +77,6 @@ dummy_func( PyObject **stack_pointer, PyObject *kwnames, int throwflag, - binaryfunc binary_ops[], PyObject *args[] ) { @@ -893,7 +892,7 @@ dummy_func( iter = _PyCoro_GetAwaitableIter(iterable); if (iter == NULL) { - format_awaitable_error(tstate, Py_TYPE(iterable), oparg); + _PyEval_FormatAwaitableError(tstate, Py_TYPE(iterable), oparg); } DECREF_INPUTS(); @@ -1120,9 +1119,9 @@ dummy_func( err = PyObject_DelItem(ns, name); // Can't use ERROR_IF here. if (err != 0) { - format_exc_check_arg(tstate, PyExc_NameError, - NAME_ERROR_MSG, - name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + NAME_ERROR_MSG, + name); goto error; } } @@ -1145,7 +1144,7 @@ dummy_func( DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ PyObject **top = stack_pointer + oparg - 1; - int res = unpack_iterable(tstate, seq, oparg, -1, top); + int res = _PyEval_UnpackIterable(tstate, seq, oparg, -1, top); DECREF_INPUTS(); ERROR_IF(res == 0, error); } @@ -1185,7 +1184,7 @@ dummy_func( inst(UNPACK_EX, (seq -- unused[oparg & 0xFF], unused, unused[oparg >> 8])) { int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; - int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); + int res = _PyEval_UnpackIterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); DECREF_INPUTS(); ERROR_IF(res == 0, error); } @@ -1235,8 +1234,8 @@ dummy_func( // Can't use ERROR_IF here. if (err != 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg(tstate, PyExc_NameError, - NAME_ERROR_MSG, name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + NAME_ERROR_MSG, name); } goto error; } @@ -1274,7 +1273,7 @@ dummy_func( goto error; } if (v == NULL) { - format_exc_check_arg( + _PyEval_FormatExcCheckArg( tstate, PyExc_NameError, NAME_ERROR_MSG, name); goto error; @@ -1315,8 +1314,8 @@ dummy_func( if (!_PyErr_Occurred(tstate)) { /* _PyDict_LoadGlobal() returns NULL without raising * an exception if the key doesn't exist */ - format_exc_check_arg(tstate, PyExc_NameError, - NAME_ERROR_MSG, name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + NAME_ERROR_MSG, name); } ERROR_IF(true, error); } @@ -1331,7 +1330,7 @@ dummy_func( /* namespace 2: builtins */ ERROR_IF(PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0, error); if (v == NULL) { - format_exc_check_arg( + _PyEval_FormatExcCheckArg( tstate, PyExc_NameError, NAME_ERROR_MSG, name); ERROR_IF(true, error); @@ -1413,7 +1412,7 @@ dummy_func( // Can't use ERROR_IF here. // Fortunately we don't need its superpower. if (oldobj == NULL) { - format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg); + _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg); goto error; } PyCell_SET(cell, NULL); @@ -1434,7 +1433,7 @@ dummy_func( PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { - format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg); + _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg); goto error; } Py_INCREF(value); @@ -1445,7 +1444,7 @@ dummy_func( PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { - format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg); + _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg); ERROR_IF(true, error); } Py_INCREF(value); @@ -1612,7 +1611,7 @@ dummy_func( PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { - format_kwargs_error(tstate, PEEK(3 + oparg), update); + _PyEval_FormatKwargsError(tstate, PEEK(3 + oparg), update); DECREF_INPUTS(); ERROR_IF(true, error); } @@ -2126,15 +2125,15 @@ dummy_func( } inst(CHECK_EG_MATCH, (exc_value, match_type -- rest, match)) { - if (check_except_star_type_valid(tstate, match_type) < 0) { + if (_PyEval_CheckExceptStarTypeValid(tstate, match_type) < 0) { DECREF_INPUTS(); ERROR_IF(true, error); } match = NULL; rest = NULL; - int res = exception_group_match(exc_value, match_type, - &match, &rest); + int res = _PyEval_ExceptionGroupMatch(exc_value, match_type, + &match, &rest); DECREF_INPUTS(); ERROR_IF(res < 0, error); @@ -2148,7 +2147,7 @@ dummy_func( inst(CHECK_EXC_MATCH, (left, right -- left, b)) { assert(PyExceptionInstance_Check(left)); - if (check_except_type_valid(tstate, right) < 0) { + if (_PyEval_CheckExceptTypeValid(tstate, right) < 0) { DECREF_INPUTS(); ERROR_IF(true, error); } @@ -2275,7 +2274,7 @@ dummy_func( // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); - attrs = match_class(tstate, subject, type, oparg, names); + attrs = _PyEval_MatchClass(tstate, subject, type, oparg, names); DECREF_INPUTS(); if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! @@ -2298,7 +2297,7 @@ dummy_func( inst(MATCH_KEYS, (subject, keys -- subject, keys, values_or_none)) { // On successful match, PUSH(values). Otherwise, PUSH(None). - values_or_none = match_keys(tstate, subject, keys); + values_or_none = _PyEval_MatchKeys(tstate, subject, keys); ERROR_IF(values_or_none == NULL, error); } @@ -3617,10 +3616,10 @@ dummy_func( STAT_INC(BINARY_OP, deferred); DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ - assert(0 <= oparg); - assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); - assert(binary_ops[oparg]); - res = binary_ops[oparg](lhs, rhs); + assert(NB_ADD <= oparg); + assert(oparg <= NB_INPLACE_XOR); + assert(_PyEval_BinaryOps[oparg]); + res = _PyEval_BinaryOps[oparg](lhs, rhs); DECREF_INPUTS(); ERROR_IF(res == NULL, error); } diff --git a/Python/ceval.c b/Python/ceval.c index b56ddfb4bd286..9f2855f964ace 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -96,13 +96,6 @@ } while (0) #endif -// GH-89279: Similar to above, force inlining by using a macro. -#if defined(_MSC_VER) && SIZEOF_INT == 4 -#define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) (assert(sizeof((ATOMIC_VAL)->_value) == 4), *((volatile int*)&((ATOMIC_VAL)->_value))) -#else -#define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) _Py_atomic_load_relaxed(ATOMIC_VAL) -#endif - #ifdef LLTRACE static void @@ -206,13 +199,7 @@ static void monitor_throw(PyThreadState *tstate, static PyObject * import_name(PyThreadState *, _PyInterpreterFrame *, PyObject *, PyObject *, PyObject *); static PyObject * import_from(PyThreadState *, PyObject *, PyObject *); -static void format_exc_check_arg(PyThreadState *, PyObject *, const char *, PyObject *); -static void format_exc_unbound(PyThreadState *tstate, PyCodeObject *co, int oparg); static int check_args_iterable(PyThreadState *, PyObject *func, PyObject *vararg); -static int check_except_type_valid(PyThreadState *tstate, PyObject* right); -static int check_except_star_type_valid(PyThreadState *tstate, PyObject* right); -static void format_kwargs_error(PyThreadState *, PyObject *func, PyObject *kwargs); -static void format_awaitable_error(PyThreadState *, PyTypeObject *, int); static int get_exception_handler(PyCodeObject *, int, int*, int*, int*); static _PyInterpreterFrame * _PyEvalFramePushAndInit(PyThreadState *tstate, PyFunctionObject *func, @@ -224,12 +211,6 @@ _PyEvalFramePushAndInit_Ex(PyThreadState *tstate, PyFunctionObject *func, static void _PyEvalFrameClearAndPop(PyThreadState *tstate, _PyInterpreterFrame *frame); -#define UNBOUNDLOCAL_ERROR_MSG \ - "cannot access local variable '%s' where it is not associated with a value" -#define UNBOUNDFREE_ERROR_MSG \ - "cannot access free variable '%s' where it is not associated with a" \ - " value in enclosing scope" - #ifdef HAVE_ERRNO_H #include #endif @@ -286,7 +267,7 @@ _Py_CheckRecursiveCall(PyThreadState *tstate, const char *where) } -static const binaryfunc binary_ops[] = { +const binaryfunc _PyEval_BinaryOps[] = { [NB_ADD] = PyNumber_Add, [NB_AND] = PyNumber_And, [NB_FLOOR_DIVIDE] = PyNumber_FloorDivide, @@ -321,8 +302,8 @@ static const binaryfunc binary_ops[] = { // Return a tuple of values corresponding to keys, with error checks for // duplicate/missing keys. -static PyObject* -match_keys(PyThreadState *tstate, PyObject *map, PyObject *keys) +PyObject * +_PyEval_MatchKeys(PyThreadState *tstate, PyObject *map, PyObject *keys) { assert(PyTuple_CheckExact(keys)); Py_ssize_t nkeys = PyTuple_GET_SIZE(keys); @@ -403,7 +384,7 @@ match_keys(PyThreadState *tstate, PyObject *map, PyObject *keys) // Extract a named attribute from the subject, with additional bookkeeping to // raise TypeErrors for repeated lookups. On failure, return NULL (with no // error set). Use _PyErr_Occurred(tstate) to disambiguate. -static PyObject* +static PyObject * match_class_attr(PyThreadState *tstate, PyObject *subject, PyObject *type, PyObject *name, PyObject *seen) { @@ -425,9 +406,9 @@ match_class_attr(PyThreadState *tstate, PyObject *subject, PyObject *type, // On success (match), return a tuple of extracted attributes. On failure (no // match), return NULL. Use _PyErr_Occurred(tstate) to disambiguate. -static PyObject* -match_class(PyThreadState *tstate, PyObject *subject, PyObject *type, - Py_ssize_t nargs, PyObject *kwargs) +PyObject* +_PyEval_MatchClass(PyThreadState *tstate, PyObject *subject, PyObject *type, + Py_ssize_t nargs, PyObject *kwargs) { if (!PyType_Check(type)) { const char *e = "called match pattern must be a class"; @@ -533,11 +514,6 @@ match_class(PyThreadState *tstate, PyObject *subject, PyObject *type, static int do_raise(PyThreadState *tstate, PyObject *exc, PyObject *cause); -static int exception_group_match( - PyObject* exc_value, PyObject *match_type, - PyObject **match, PyObject **rest); - -static int unpack_iterable(PyThreadState *, PyObject *, int, int, PyObject **); PyObject * PyEval_EvalCode(PyObject *co, PyObject *globals, PyObject *locals) @@ -827,7 +803,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int unbound_local_error: { - format_exc_check_arg(tstate, PyExc_UnboundLocalError, + _PyEval_FormatExcCheckArg(tstate, PyExc_UnboundLocalError, UNBOUNDLOCAL_ERROR_MSG, PyTuple_GetItem(_PyFrame_GetCode(frame)->co_localsplusnames, oparg) ); @@ -1777,9 +1753,9 @@ do_raise(PyThreadState *tstate, PyObject *exc, PyObject *cause) complicated for inlining). */ -static int -exception_group_match(PyObject* exc_value, PyObject *match_type, - PyObject **match, PyObject **rest) +int +_PyEval_ExceptionGroupMatch(PyObject* exc_value, PyObject *match_type, + PyObject **match, PyObject **rest) { if (Py_IsNone(exc_value)) { *match = Py_NewRef(Py_None); @@ -1840,9 +1816,9 @@ exception_group_match(PyObject* exc_value, PyObject *match_type, with a variable target. */ -static int -unpack_iterable(PyThreadState *tstate, PyObject *v, - int argcnt, int argcntafter, PyObject **sp) +int +_PyEval_UnpackIterable(PyThreadState *tstate, PyObject *v, + int argcnt, int argcntafter, PyObject **sp) { int i = 0, j = 0; Py_ssize_t ll = 0; @@ -2487,8 +2463,8 @@ import_from(PyThreadState *tstate, PyObject *v, PyObject *name) #define CANNOT_EXCEPT_STAR_EG "catching ExceptionGroup with except* "\ "is not allowed. Use except instead." -static int -check_except_type_valid(PyThreadState *tstate, PyObject* right) +int +_PyEval_CheckExceptTypeValid(PyThreadState *tstate, PyObject* right) { if (PyTuple_Check(right)) { Py_ssize_t i, length; @@ -2512,10 +2488,10 @@ check_except_type_valid(PyThreadState *tstate, PyObject* right) return 0; } -static int -check_except_star_type_valid(PyThreadState *tstate, PyObject* right) +int +_PyEval_CheckExceptStarTypeValid(PyThreadState *tstate, PyObject* right) { - if (check_except_type_valid(tstate, right) < 0) { + if (_PyEval_CheckExceptTypeValid(tstate, right) < 0) { return -1; } @@ -2569,8 +2545,8 @@ check_args_iterable(PyThreadState *tstate, PyObject *func, PyObject *args) return 0; } -static void -format_kwargs_error(PyThreadState *tstate, PyObject *func, PyObject *kwargs) +void +_PyEval_FormatKwargsError(PyThreadState *tstate, PyObject *func, PyObject *kwargs) { /* _PyDict_MergeEx raises attribute * error (percolated from an attempt @@ -2611,9 +2587,9 @@ format_kwargs_error(PyThreadState *tstate, PyObject *func, PyObject *kwargs) } } -static void -format_exc_check_arg(PyThreadState *tstate, PyObject *exc, - const char *format_str, PyObject *obj) +void +_PyEval_FormatExcCheckArg(PyThreadState *tstate, PyObject *exc, + const char *format_str, PyObject *obj) { const char *obj_str; @@ -2640,8 +2616,8 @@ format_exc_check_arg(PyThreadState *tstate, PyObject *exc, } } -static void -format_exc_unbound(PyThreadState *tstate, PyCodeObject *co, int oparg) +void +_PyEval_FormatExcUnbound(PyThreadState *tstate, PyCodeObject *co, int oparg) { PyObject *name; /* Don't stomp existing exception */ @@ -2649,16 +2625,16 @@ format_exc_unbound(PyThreadState *tstate, PyCodeObject *co, int oparg) return; name = PyTuple_GET_ITEM(co->co_localsplusnames, oparg); if (oparg < PyCode_GetFirstFree(co)) { - format_exc_check_arg(tstate, PyExc_UnboundLocalError, - UNBOUNDLOCAL_ERROR_MSG, name); + _PyEval_FormatExcCheckArg(tstate, PyExc_UnboundLocalError, + UNBOUNDLOCAL_ERROR_MSG, name); } else { - format_exc_check_arg(tstate, PyExc_NameError, - UNBOUNDFREE_ERROR_MSG, name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + UNBOUNDFREE_ERROR_MSG, name); } } -static void -format_awaitable_error(PyThreadState *tstate, PyTypeObject *type, int oparg) +void +_PyEval_FormatAwaitableError(PyThreadState *tstate, PyTypeObject *type, int oparg) { if (type->tp_as_async == NULL || type->tp_as_async->am_await == NULL) { if (oparg == 1) { @@ -2703,110 +2679,3 @@ void Py_LeaveRecursiveCall(void) { _Py_LeaveRecursiveCall(); } - -///////////////////// Experimental UOp Interpreter ///////////////////// - -#undef ASSERT_KWNAMES_IS_NULL -#define ASSERT_KWNAMES_IS_NULL() (void)0 - -#undef DEOPT_IF -#define DEOPT_IF(COND, INSTNAME) \ - if ((COND)) { \ - goto deoptimize; \ - } - -_PyInterpreterFrame * -_PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject **stack_pointer) -{ -#ifdef Py_DEBUG - char *uop_debug = Py_GETENV("PYTHONUOPSDEBUG"); - int lltrace = 0; - if (uop_debug != NULL && *uop_debug >= '0') { - lltrace = *uop_debug - '0'; // TODO: Parse an int and all that - } -#define DPRINTF(level, ...) \ - if (lltrace >= (level)) { fprintf(stderr, __VA_ARGS__); } -#else -#define DPRINTF(level, ...) -#endif - - DPRINTF(3, - "Entering _PyUopExecute for %s (%s:%d) at byte offset %ld\n", - PyUnicode_AsUTF8(_PyFrame_GetCode(frame)->co_qualname), - PyUnicode_AsUTF8(_PyFrame_GetCode(frame)->co_filename), - _PyFrame_GetCode(frame)->co_firstlineno, - 2 * (long)(frame->prev_instr + 1 - - (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive)); - - PyThreadState *tstate = _PyThreadState_GET(); - _PyUOpExecutorObject *self = (_PyUOpExecutorObject *)executor; - - CHECK_EVAL_BREAKER(); - - OBJECT_STAT_INC(optimization_traces_executed); - _Py_CODEUNIT *ip_offset = (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive; - int pc = 0; - int opcode; - int oparg; - uint64_t operand; - - for (;;) { - opcode = self->trace[pc].opcode; - oparg = self->trace[pc].oparg; - operand = self->trace[pc].operand; - DPRINTF(3, - "%4d: uop %s, oparg %d, operand %" PRIu64 ", stack_level %d\n", - pc, - opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode], - oparg, - operand, - (int)(stack_pointer - _PyFrame_Stackbase(frame))); - pc++; - OBJECT_STAT_INC(optimization_uops_executed); - switch (opcode) { - -#undef ENABLE_SPECIALIZATION -#define ENABLE_SPECIALIZATION 0 -#include "executor_cases.c.h" - - default: - { - fprintf(stderr, "Unknown uop %d, operand %" PRIu64 "\n", opcode, operand); - Py_FatalError("Unknown uop"); - } - - } - } - -unbound_local_error: - format_exc_check_arg(tstate, PyExc_UnboundLocalError, - UNBOUNDLOCAL_ERROR_MSG, - PyTuple_GetItem(_PyFrame_GetCode(frame)->co_localsplusnames, oparg) - ); - goto error; - -pop_4_error: - STACK_SHRINK(1); -pop_3_error: - STACK_SHRINK(1); -pop_2_error: - STACK_SHRINK(1); -pop_1_error: - STACK_SHRINK(1); -error: - // On ERROR_IF we return NULL as the frame. - // The caller recovers the frame from cframe.current_frame. - DPRINTF(2, "Error: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand); - _PyFrame_SetStackPointer(frame, stack_pointer); - Py_DECREF(self); - return NULL; - -deoptimize: - // On DEOPT_IF we just repeat the last instruction. - // This presumes nothing was popped from the stack (nor pushed). - DPRINTF(2, "DEOPT: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand); - frame->prev_instr--; // Back up to just before destination - _PyFrame_SetStackPointer(frame, stack_pointer); - Py_DECREF(self); - return frame; -} diff --git a/Python/ceval_macros.h b/Python/ceval_macros.h index c2c323317d10f..8dc8b75448585 100644 --- a/Python/ceval_macros.h +++ b/Python/ceval_macros.h @@ -1,4 +1,4 @@ -// Macros and other things needed by ceval.c and bytecodes.c +// Macros and other things needed by ceval.c, executor.c, and bytecodes.c /* Computed GOTOs, or the-optimization-commonly-but-improperly-known-as-"threaded code" @@ -304,6 +304,11 @@ GETITEM(PyObject *v, Py_ssize_t i) { (COUNTER) += (1 << ADAPTIVE_BACKOFF_BITS); \ } while (0); +#define UNBOUNDLOCAL_ERROR_MSG \ + "cannot access local variable '%s' where it is not associated with a value" +#define UNBOUNDFREE_ERROR_MSG \ + "cannot access free variable '%s' where it is not associated with a value" \ + " in enclosing scope" #define NAME_ERROR_MSG "name '%.200s' is not defined" #define KWNAMES_LEN() \ @@ -352,3 +357,10 @@ static const convertion_func_ptr CONVERSION_FUNCTIONS[4] = { }; #define ASSERT_KWNAMES_IS_NULL() assert(kwnames == NULL) + +// GH-89279: Force inlining by using a macro. +#if defined(_MSC_VER) && SIZEOF_INT == 4 +#define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) (assert(sizeof((ATOMIC_VAL)->_value) == 4), *((volatile int*)&((ATOMIC_VAL)->_value))) +#else +#define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) _Py_atomic_load_relaxed(ATOMIC_VAL) +#endif diff --git a/Python/executor.c b/Python/executor.c new file mode 100644 index 0000000000000..17322526a7bd7 --- /dev/null +++ b/Python/executor.c @@ -0,0 +1,125 @@ +#include "Python.h" + +#include "pycore_call.h" +#include "pycore_ceval.h" +#include "pycore_dict.h" +#include "pycore_emscripten_signal.h" +#include "pycore_intrinsics.h" +#include "pycore_long.h" +#include "pycore_object.h" +#include "pycore_opcode_metadata.h" +#include "pycore_opcode_utils.h" +#include "pycore_pyerrors.h" +#include "pycore_range.h" +#include "pycore_sliceobject.h" +#include "pycore_uops.h" + +#include "ceval_macros.h" + + +#undef ASSERT_KWNAMES_IS_NULL +#define ASSERT_KWNAMES_IS_NULL() (void)0 + +#undef DEOPT_IF +#define DEOPT_IF(COND, INSTNAME) \ + if ((COND)) { \ + goto deoptimize; \ + } + +#undef ENABLE_SPECIALIZATION +#define ENABLE_SPECIALIZATION 0 + + +_PyInterpreterFrame * +_PyUopExecute(_PyExecutorObject *executor, _PyInterpreterFrame *frame, PyObject **stack_pointer) +{ +#ifdef Py_DEBUG + char *uop_debug = Py_GETENV("PYTHONUOPSDEBUG"); + int lltrace = 0; + if (uop_debug != NULL && *uop_debug >= '0') { + lltrace = *uop_debug - '0'; // TODO: Parse an int and all that + } + #define DPRINTF(level, ...) \ + if (lltrace >= (level)) { fprintf(stderr, __VA_ARGS__); } +#else + #define DPRINTF(level, ...) +#endif + + DPRINTF(3, + "Entering _PyUopExecute for %s (%s:%d) at byte offset %ld\n", + PyUnicode_AsUTF8(_PyFrame_GetCode(frame)->co_qualname), + PyUnicode_AsUTF8(_PyFrame_GetCode(frame)->co_filename), + _PyFrame_GetCode(frame)->co_firstlineno, + 2 * (long)(frame->prev_instr + 1 - + (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive)); + + PyThreadState *tstate = _PyThreadState_GET(); + _PyUOpExecutorObject *self = (_PyUOpExecutorObject *)executor; + + CHECK_EVAL_BREAKER(); + + OBJECT_STAT_INC(optimization_traces_executed); + _Py_CODEUNIT *ip_offset = (_Py_CODEUNIT *)_PyFrame_GetCode(frame)->co_code_adaptive; + int pc = 0; + int opcode; + int oparg; + uint64_t operand; + + for (;;) { + opcode = self->trace[pc].opcode; + oparg = self->trace[pc].oparg; + operand = self->trace[pc].operand; + DPRINTF(3, + "%4d: uop %s, oparg %d, operand %" PRIu64 ", stack_level %d\n", + pc, + opcode < 256 ? _PyOpcode_OpName[opcode] : _PyOpcode_uop_name[opcode], + oparg, + operand, + (int)(stack_pointer - _PyFrame_Stackbase(frame))); + pc++; + OBJECT_STAT_INC(optimization_uops_executed); + switch (opcode) { + +#include "executor_cases.c.h" + + default: + { + fprintf(stderr, "Unknown uop %d, operand %" PRIu64 "\n", opcode, operand); + Py_FatalError("Unknown uop"); + } + + } + } + +unbound_local_error: + _PyEval_FormatExcCheckArg(tstate, PyExc_UnboundLocalError, + UNBOUNDLOCAL_ERROR_MSG, + PyTuple_GetItem(_PyFrame_GetCode(frame)->co_localsplusnames, oparg) + ); + goto error; + +pop_4_error: + STACK_SHRINK(1); +pop_3_error: + STACK_SHRINK(1); +pop_2_error: + STACK_SHRINK(1); +pop_1_error: + STACK_SHRINK(1); +error: + // On ERROR_IF we return NULL as the frame. + // The caller recovers the frame from cframe.current_frame. + DPRINTF(2, "Error: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand); + _PyFrame_SetStackPointer(frame, stack_pointer); + Py_DECREF(self); + return NULL; + +deoptimize: + // On DEOPT_IF we just repeat the last instruction. + // This presumes nothing was popped from the stack (nor pushed). + DPRINTF(2, "DEOPT: [Opcode %d, operand %" PRIu64 "]\n", opcode, operand); + frame->prev_instr--; // Back up to just before destination + _PyFrame_SetStackPointer(frame, stack_pointer); + Py_DECREF(self); + return frame; +} diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 5b38c6a5fd8a2..064965a1a1ae7 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -672,7 +672,7 @@ iter = _PyCoro_GetAwaitableIter(iterable); if (iter == NULL) { - format_awaitable_error(tstate, Py_TYPE(iterable), oparg); + _PyEval_FormatAwaitableError(tstate, Py_TYPE(iterable), oparg); } Py_DECREF(iterable); @@ -758,9 +758,9 @@ err = PyObject_DelItem(ns, name); // Can't use ERROR_IF here. if (err != 0) { - format_exc_check_arg(tstate, PyExc_NameError, - NAME_ERROR_MSG, - name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + NAME_ERROR_MSG, + name); goto error; } break; @@ -780,7 +780,7 @@ DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ PyObject **top = stack_pointer + oparg - 1; - int res = unpack_iterable(tstate, seq, oparg, -1, top); + int res = _PyEval_UnpackIterable(tstate, seq, oparg, -1, top); Py_DECREF(seq); if (res == 0) goto pop_1_error; STACK_SHRINK(1); @@ -839,7 +839,7 @@ PyObject *seq = stack_pointer[-1]; int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; - int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); + int res = _PyEval_UnpackIterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); Py_DECREF(seq); if (res == 0) goto pop_1_error; STACK_GROW((oparg & 0xFF) + (oparg >> 8)); @@ -897,8 +897,8 @@ // Can't use ERROR_IF here. if (err != 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg(tstate, PyExc_NameError, - NAME_ERROR_MSG, name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + NAME_ERROR_MSG, name); } goto error; } @@ -941,7 +941,7 @@ goto error; } if (v == NULL) { - format_exc_check_arg( + _PyEval_FormatExcCheckArg( tstate, PyExc_NameError, NAME_ERROR_MSG, name); goto error; @@ -978,8 +978,8 @@ if (!_PyErr_Occurred(tstate)) { /* _PyDict_LoadGlobal() returns NULL without raising * an exception if the key doesn't exist */ - format_exc_check_arg(tstate, PyExc_NameError, - NAME_ERROR_MSG, name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + NAME_ERROR_MSG, name); } if (true) goto error; } @@ -994,7 +994,7 @@ /* namespace 2: builtins */ if (PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0) goto error; if (v == NULL) { - format_exc_check_arg( + _PyEval_FormatExcCheckArg( tstate, PyExc_NameError, NAME_ERROR_MSG, name); if (true) goto error; @@ -1080,7 +1080,7 @@ // Can't use ERROR_IF here. // Fortunately we don't need its superpower. if (oldobj == NULL) { - format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg); + _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg); goto error; } PyCell_SET(cell, NULL); @@ -1104,7 +1104,7 @@ PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { - format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg); + _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg); goto error; } Py_INCREF(value); @@ -1118,7 +1118,7 @@ PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { - format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg); + _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg); if (true) goto error; } Py_INCREF(value); @@ -1348,7 +1348,7 @@ PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { - format_kwargs_error(tstate, PEEK(3 + oparg), update); + _PyEval_FormatKwargsError(tstate, PEEK(3 + oparg), update); Py_DECREF(update); if (true) goto pop_1_error; } @@ -1646,7 +1646,7 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - if (check_except_star_type_valid(tstate, match_type) < 0) { + if (_PyEval_CheckExceptStarTypeValid(tstate, match_type) < 0) { Py_DECREF(exc_value); Py_DECREF(match_type); if (true) goto pop_2_error; @@ -1654,8 +1654,8 @@ match = NULL; rest = NULL; - int res = exception_group_match(exc_value, match_type, - &match, &rest); + int res = _PyEval_ExceptionGroupMatch(exc_value, match_type, + &match, &rest); Py_DECREF(exc_value); Py_DECREF(match_type); if (res < 0) goto pop_2_error; @@ -1676,7 +1676,7 @@ PyObject *left = stack_pointer[-2]; PyObject *b; assert(PyExceptionInstance_Check(left)); - if (check_except_type_valid(tstate, right) < 0) { + if (_PyEval_CheckExceptTypeValid(tstate, right) < 0) { Py_DECREF(right); if (true) goto pop_1_error; } @@ -1723,7 +1723,7 @@ // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); - attrs = match_class(tstate, subject, type, oparg, names); + attrs = _PyEval_MatchClass(tstate, subject, type, oparg, names); Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); @@ -1764,7 +1764,7 @@ PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; // On successful match, PUSH(values). Otherwise, PUSH(None). - values_or_none = match_keys(tstate, subject, keys); + values_or_none = _PyEval_MatchKeys(tstate, subject, keys); if (values_or_none == NULL) goto error; STACK_GROW(1); stack_pointer[-1] = values_or_none; @@ -2462,10 +2462,10 @@ STAT_INC(BINARY_OP, deferred); DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ - assert(0 <= oparg); - assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); - assert(binary_ops[oparg]); - res = binary_ops[oparg](lhs, rhs); + assert(NB_ADD <= oparg); + assert(oparg <= NB_INPLACE_XOR); + assert(_PyEval_BinaryOps[oparg]); + res = _PyEval_BinaryOps[oparg](lhs, rhs); Py_DECREF(lhs); Py_DECREF(rhs); if (res == NULL) goto pop_2_error; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 2d70966aa0913..9d25fc604314a 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -1068,7 +1068,7 @@ iter = _PyCoro_GetAwaitableIter(iterable); if (iter == NULL) { - format_awaitable_error(tstate, Py_TYPE(iterable), oparg); + _PyEval_FormatAwaitableError(tstate, Py_TYPE(iterable), oparg); } Py_DECREF(iterable); @@ -1336,9 +1336,9 @@ err = PyObject_DelItem(ns, name); // Can't use ERROR_IF here. if (err != 0) { - format_exc_check_arg(tstate, PyExc_NameError, - NAME_ERROR_MSG, - name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + NAME_ERROR_MSG, + name); goto error; } DISPATCH(); @@ -1359,7 +1359,7 @@ DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ PyObject **top = stack_pointer + oparg - 1; - int res = unpack_iterable(tstate, seq, oparg, -1, top); + int res = _PyEval_UnpackIterable(tstate, seq, oparg, -1, top); Py_DECREF(seq); if (res == 0) goto pop_1_error; STACK_SHRINK(1); @@ -1422,7 +1422,7 @@ PyObject *seq = stack_pointer[-1]; int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; - int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); + int res = _PyEval_UnpackIterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); Py_DECREF(seq); if (res == 0) goto pop_1_error; STACK_GROW((oparg & 0xFF) + (oparg >> 8)); @@ -1482,8 +1482,8 @@ // Can't use ERROR_IF here. if (err != 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { - format_exc_check_arg(tstate, PyExc_NameError, - NAME_ERROR_MSG, name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + NAME_ERROR_MSG, name); } goto error; } @@ -1543,7 +1543,7 @@ goto error; } if (v == NULL) { - format_exc_check_arg( + _PyEval_FormatExcCheckArg( tstate, PyExc_NameError, NAME_ERROR_MSG, name); goto error; @@ -1581,7 +1581,7 @@ goto error; } if (v == NULL) { - format_exc_check_arg( + _PyEval_FormatExcCheckArg( tstate, PyExc_NameError, NAME_ERROR_MSG, name); goto error; @@ -1621,8 +1621,8 @@ if (!_PyErr_Occurred(tstate)) { /* _PyDict_LoadGlobal() returns NULL without raising * an exception if the key doesn't exist */ - format_exc_check_arg(tstate, PyExc_NameError, - NAME_ERROR_MSG, name); + _PyEval_FormatExcCheckArg(tstate, PyExc_NameError, + NAME_ERROR_MSG, name); } if (true) goto error; } @@ -1637,7 +1637,7 @@ /* namespace 2: builtins */ if (PyMapping_GetOptionalItem(BUILTINS(), name, &v) < 0) goto error; if (v == NULL) { - format_exc_check_arg( + _PyEval_FormatExcCheckArg( tstate, PyExc_NameError, NAME_ERROR_MSG, name); if (true) goto error; @@ -1755,7 +1755,7 @@ // Can't use ERROR_IF here. // Fortunately we don't need its superpower. if (oldobj == NULL) { - format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg); + _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg); goto error; } PyCell_SET(cell, NULL); @@ -1779,7 +1779,7 @@ PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { - format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg); + _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg); goto error; } Py_INCREF(value); @@ -1793,7 +1793,7 @@ PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { - format_exc_unbound(tstate, _PyFrame_GetCode(frame), oparg); + _PyEval_FormatExcUnbound(tstate, _PyFrame_GetCode(frame), oparg); if (true) goto error; } Py_INCREF(value); @@ -2023,7 +2023,7 @@ PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { - format_kwargs_error(tstate, PEEK(3 + oparg), update); + _PyEval_FormatKwargsError(tstate, PEEK(3 + oparg), update); Py_DECREF(update); if (true) goto pop_1_error; } @@ -2679,7 +2679,7 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - if (check_except_star_type_valid(tstate, match_type) < 0) { + if (_PyEval_CheckExceptStarTypeValid(tstate, match_type) < 0) { Py_DECREF(exc_value); Py_DECREF(match_type); if (true) goto pop_2_error; @@ -2687,8 +2687,8 @@ match = NULL; rest = NULL; - int res = exception_group_match(exc_value, match_type, - &match, &rest); + int res = _PyEval_ExceptionGroupMatch(exc_value, match_type, + &match, &rest); Py_DECREF(exc_value); Py_DECREF(match_type); if (res < 0) goto pop_2_error; @@ -2709,7 +2709,7 @@ PyObject *left = stack_pointer[-2]; PyObject *b; assert(PyExceptionInstance_Check(left)); - if (check_except_type_valid(tstate, right) < 0) { + if (_PyEval_CheckExceptTypeValid(tstate, right) < 0) { Py_DECREF(right); if (true) goto pop_1_error; } @@ -2890,7 +2890,7 @@ // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); - attrs = match_class(tstate, subject, type, oparg, names); + attrs = _PyEval_MatchClass(tstate, subject, type, oparg, names); Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); @@ -2931,7 +2931,7 @@ PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; // On successful match, PUSH(values). Otherwise, PUSH(None). - values_or_none = match_keys(tstate, subject, keys); + values_or_none = _PyEval_MatchKeys(tstate, subject, keys); if (values_or_none == NULL) goto error; STACK_GROW(1); stack_pointer[-1] = values_or_none; @@ -4464,10 +4464,10 @@ STAT_INC(BINARY_OP, deferred); DECREMENT_ADAPTIVE_COUNTER(cache->counter); #endif /* ENABLE_SPECIALIZATION */ - assert(0 <= oparg); - assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); - assert(binary_ops[oparg]); - res = binary_ops[oparg](lhs, rhs); + assert(NB_ADD <= oparg); + assert(oparg <= NB_INPLACE_XOR); + assert(_PyEval_BinaryOps[oparg]); + res = _PyEval_BinaryOps[oparg](lhs, rhs); Py_DECREF(lhs); Py_DECREF(rhs); if (res == NULL) goto pop_2_error; diff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsv index 228ca8648a2f3..cf6d3b2443523 100644 --- a/Tools/c-analyzer/cpython/ignored.tsv +++ b/Tools/c-analyzer/cpython/ignored.tsv @@ -326,7 +326,7 @@ Parser/parser.c - reserved_keywords - Parser/parser.c - soft_keywords - Parser/tokenizer.c - type_comment_prefix - Python/ast_opt.c fold_unaryop ops - -Python/ceval.c - binary_ops - +Python/ceval.c - _PyEval_BinaryOps - Python/ceval.c - _Py_INTERPRETER_TRAMPOLINE_INSTRUCTIONS - Python/codecs.c - Py_hexdigits - Python/codecs.c - ucnhash_capi - From webhook-mailer at python.org Thu Jul 20 17:33:37 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Thu, 20 Jul 2023 21:33:37 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Annotate CLanguage.render_option_group_parsing() (#106929) Message-ID: https://github.com/python/cpython/commit/42c6300d4faa1b1582914d84551b83c814ccfdae commit: 42c6300d4faa1b1582914d84551b83c814ccfdae branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-20T21:33:33Z summary: gh-104050: Argument Clinic: Annotate CLanguage.render_option_group_parsing() (#106929) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index bff8935df13bc..6d1796cd6135e 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -530,12 +530,11 @@ class PythonLanguage(Language): checksum_line = "#/*[{dsl_name} end generated code: {arguments}]*/" -ParamGroup = Iterable["Parameter"] ParamTuple = tuple["Parameter", ...] def permute_left_option_groups( - l: Sequence[ParamGroup] + l: Sequence[Iterable[Parameter]] ) -> Iterator[ParamTuple]: """ Given [(1,), (2,), (3,)], should yield: @@ -552,7 +551,7 @@ def permute_left_option_groups( def permute_right_option_groups( - l: Sequence[ParamGroup] + l: Sequence[Iterable[Parameter]] ) -> Iterator[ParamTuple]: """ Given [(1,), (2,), (3,)], should yield: @@ -569,9 +568,9 @@ def permute_right_option_groups( def permute_optional_groups( - left: Sequence[ParamGroup], - required: ParamGroup, - right: Sequence[ParamGroup] + left: Sequence[Iterable[Parameter]], + required: Iterable[Parameter], + right: Sequence[Iterable[Parameter]] ) -> tuple[ParamTuple, ...]: """ Generator function that computes the set of acceptable @@ -1374,7 +1373,11 @@ def group_to_variable_name(group: int) -> str: adjective = "left_" if group < 0 else "right_" return "group_" + adjective + str(abs(group)) - def render_option_group_parsing(self, f, template_dict): + def render_option_group_parsing( + self, + f: Function, + template_dict: TemplateDict + ) -> None: # positional only, grouped, optional arguments! # can be optional on the left or right. # here's an example: @@ -1398,11 +1401,11 @@ def render_option_group_parsing(self, f, template_dict): if isinstance(parameters[0].converter, self_converter): del parameters[0] - group = None + group: list[Parameter] | None = None left = [] right = [] - required = [] - last = unspecified + required: list[Parameter] = [] + last: int | Literal[Sentinels.unspecified] = unspecified for p in parameters: group_id = p.group @@ -1415,6 +1418,7 @@ def render_option_group_parsing(self, f, template_dict): group = required else: right.append(group) + assert group is not None group.append(p) count_min = sys.maxsize @@ -1433,19 +1437,21 @@ def render_option_group_parsing(self, f, template_dict): continue group_ids = {p.group for p in subset} # eliminate duplicates - d = {} + d: dict[str, str | int] = {} d['count'] = count d['name'] = f.name d['format_units'] = "".join(p.converter.format_unit for p in subset) - parse_arguments = [] + parse_arguments: list[str] = [] for p in subset: p.converter.parse_argument(parse_arguments) d['parse_arguments'] = ", ".join(parse_arguments) group_ids.discard(0) - lines = [self.group_to_variable_name(g) + " = 1;" for g in group_ids] - lines = "\n".join(lines) + lines = "\n".join([ + self.group_to_variable_name(g) + " = 1;" + for g in group_ids + ]) s = """\ case {count}: From webhook-mailer at python.org Thu Jul 20 18:45:06 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Thu, 20 Jul 2023 22:45:06 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Increase CConverter typing coverage (#106932) Message-ID: https://github.com/python/cpython/commit/1d5a625eb09ec160d403e526bb6a685525389b99 commit: 1d5a625eb09ec160d403e526bb6a685525389b99 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-20T22:45:02Z summary: gh-104050: Argument Clinic: Increase CConverter typing coverage (#106932) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 6d1796cd6135e..003ecf784d711 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2698,10 +2698,10 @@ class CConverter(metaclass=CConverterAutoRegister): """ # The C name to use for this variable. - name: str | None = None + name: str # The Python name to use for this variable. - py_name: str | None = None + py_name: str # The C type to use for this variable. # 'type' should be a Python string specifying the type, e.g. "int". @@ -2864,7 +2864,11 @@ def _render_self(self, parameter: Parameter, data: CRenderData) -> None: if self.length: data.impl_parameters.append("Py_ssize_t " + self.length_name()) - def _render_non_self(self, parameter, data): + def _render_non_self( + self, + parameter: Parameter, + data: CRenderData + ) -> None: self.parameter = parameter name = self.name @@ -2917,31 +2921,30 @@ def render(self, parameter: Parameter, data: CRenderData) -> None: self._render_self(parameter, data) self._render_non_self(parameter, data) - def length_name(self): + def length_name(self) -> str: """Computes the name of the associated "length" variable.""" - if not self.length: - return None + assert self.length is not None return self.parser_name + "_length" # Why is this one broken out separately? # For "positional-only" function parsing, # which generates a bunch of PyArg_ParseTuple calls. - def parse_argument(self, list): + def parse_argument(self, args: list[str]) -> None: assert not (self.converter and self.encoding) if self.format_unit == 'O&': assert self.converter - list.append(self.converter) + args.append(self.converter) if self.encoding: - list.append(c_repr(self.encoding)) + args.append(c_repr(self.encoding)) elif self.subclass_of: - list.append(self.subclass_of) + args.append(self.subclass_of) s = ("&" if self.parse_by_reference else "") + self.name - list.append(s) + args.append(s) if self.length: - list.append("&" + self.length_name()) + args.append("&" + self.length_name()) # # All the functions after here are intended as extension points. @@ -3066,7 +3069,7 @@ def set_template_dict(self, template_dict: TemplateDict) -> None: pass @property - def parser_name(self): + def parser_name(self) -> str: if self.name in CLINIC_PREFIXED_ARGS: # bpo-39741 return CLINIC_PREFIX + self.name else: From webhook-mailer at python.org Thu Jul 20 19:07:10 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 20 Jul 2023 23:07:10 -0000 Subject: [Python-checkins] Fix typo in tkinter docs (#106936) Message-ID: https://github.com/python/cpython/commit/60e83968d555d53b97de04a0a00b2cdeb3187d39 commit: 60e83968d555d53b97de04a0a00b2cdeb3187d39 branch: main author: Makonede <61922615+Makonede at users.noreply.github.com> committer: AlexWaygood date: 2023-07-21T00:07:06+01:00 summary: Fix typo in tkinter docs (#106936) Signed-off-by: Makonede <61922615+Makonede at users.noreply.github.com> files: M Doc/library/tkinter.ttk.rst diff --git a/Doc/library/tkinter.ttk.rst b/Doc/library/tkinter.ttk.rst index 4ff2b2159c362..9f2f9eb858afd 100644 --- a/Doc/library/tkinter.ttk.rst +++ b/Doc/library/tkinter.ttk.rst @@ -102,7 +102,7 @@ themed widgets and is not supposed to be directly instantiated. Standard Options ^^^^^^^^^^^^^^^^ -All the :mod:`ttk` Widgets accepts the following options: +All the :mod:`ttk` Widgets accept the following options: .. tabularcolumns:: |l|L| From webhook-mailer at python.org Thu Jul 20 19:08:56 2023 From: webhook-mailer at python.org (gvanrossum) Date: Thu, 20 Jul 2023 23:08:56 -0000 Subject: [Python-checkins] gh-105540: Show source files relative to root (#106927) Message-ID: https://github.com/python/cpython/commit/12189108607a1d5d146f32094115297011c8b773 commit: 12189108607a1d5d146f32094115297011c8b773 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-20T23:08:52Z summary: gh-105540: Show source files relative to root (#106927) This restores a corner case: when the generator is run with working directory set to Tools/cases_generator, the source filenames listed in the generated provenance header should be relative to the repo root directory. files: M Tools/cases_generator/generate_cases.py diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index a18229a57143b..3a679b2090f19 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -17,6 +17,7 @@ import parser from parser import StackEffect + HERE = os.path.dirname(__file__) ROOT = os.path.join(HERE, "../..") THIS = os.path.relpath(__file__, ROOT).replace(os.path.sep, posixpath.sep) @@ -1153,10 +1154,15 @@ def write_function( self.out.emit("") def from_source_files(self) -> str: - paths = f"\n{self.out.comment} ".join( - prettify_filename(filename) - for filename in self.input_filenames - ) + filenames = [] + for filename in self.input_filenames: + try: + filename = os.path.relpath(filename, ROOT) + except ValueError: + # May happen on Windows if root and temp on different volumes + pass + filenames.append(filename) + paths = f"\n{self.out.comment} ".join(filenames) return f"{self.out.comment} from:\n{self.out.comment} {paths}\n" def write_provenance_header(self): From webhook-mailer at python.org Thu Jul 20 19:11:56 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 20 Jul 2023 23:11:56 -0000 Subject: [Python-checkins] [3.12] Fix typo in tkinter docs (GH-106936) (#106937) Message-ID: https://github.com/python/cpython/commit/c1fd76e138c0acf4b90b08ded990d6521187fe63 commit: c1fd76e138c0acf4b90b08ded990d6521187fe63 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-21T00:11:52+01:00 summary: [3.12] Fix typo in tkinter docs (GH-106936) (#106937) Fix typo in tkinter docs (GH-106936) (cherry picked from commit 60e83968d555d53b97de04a0a00b2cdeb3187d39) Signed-off-by: Makonede <61922615+Makonede at users.noreply.github.com> Co-authored-by: Makonede <61922615+Makonede at users.noreply.github.com> files: M Doc/library/tkinter.ttk.rst diff --git a/Doc/library/tkinter.ttk.rst b/Doc/library/tkinter.ttk.rst index 4ff2b2159c362..9f2f9eb858afd 100644 --- a/Doc/library/tkinter.ttk.rst +++ b/Doc/library/tkinter.ttk.rst @@ -102,7 +102,7 @@ themed widgets and is not supposed to be directly instantiated. Standard Options ^^^^^^^^^^^^^^^^ -All the :mod:`ttk` Widgets accepts the following options: +All the :mod:`ttk` Widgets accept the following options: .. tabularcolumns:: |l|L| From webhook-mailer at python.org Thu Jul 20 19:12:04 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 20 Jul 2023 23:12:04 -0000 Subject: [Python-checkins] [3.11] Fix typo in tkinter docs (GH-106936) (#106938) Message-ID: https://github.com/python/cpython/commit/a5c8d240c0648ac9f982bbea0fc1b2ab3a959543 commit: a5c8d240c0648ac9f982bbea0fc1b2ab3a959543 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-21T00:12:00+01:00 summary: [3.11] Fix typo in tkinter docs (GH-106936) (#106938) Fix typo in tkinter docs (GH-106936) (cherry picked from commit 60e83968d555d53b97de04a0a00b2cdeb3187d39) Signed-off-by: Makonede <61922615+Makonede at users.noreply.github.com> Co-authored-by: Makonede <61922615+Makonede at users.noreply.github.com> files: M Doc/library/tkinter.ttk.rst diff --git a/Doc/library/tkinter.ttk.rst b/Doc/library/tkinter.ttk.rst index 4ff2b2159c362..9f2f9eb858afd 100644 --- a/Doc/library/tkinter.ttk.rst +++ b/Doc/library/tkinter.ttk.rst @@ -102,7 +102,7 @@ themed widgets and is not supposed to be directly instantiated. Standard Options ^^^^^^^^^^^^^^^^ -All the :mod:`ttk` Widgets accepts the following options: +All the :mod:`ttk` Widgets accept the following options: .. tabularcolumns:: |l|L| From webhook-mailer at python.org Thu Jul 20 19:19:15 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Thu, 20 Jul 2023 23:19:15 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Annotate the IndentStack class (#106934) Message-ID: https://github.com/python/cpython/commit/d81b4f8ff84311fa737e62f2883442ec06d7d5d8 commit: d81b4f8ff84311fa737e62f2883442ec06d7d5d8 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-20T23:19:11Z summary: gh-104050: Argument Clinic: Annotate the IndentStack class (#106934) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 003ecf784d711..a204c1dd0dc2d 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4284,7 +4284,7 @@ def _ensure(self): if not self.indents: fail('IndentStack expected indents, but none are defined.') - def measure(self, line): + def measure(self, line: str) -> int: """ Returns the length of the line's margin. """ @@ -4298,7 +4298,7 @@ def measure(self, line): return self.indents[-1] return len(line) - len(stripped) - def infer(self, line): + def infer(self, line: str) -> int: """ Infer what is now the current margin based on this line. Returns: @@ -4331,19 +4331,19 @@ def infer(self, line): return outdent_count @property - def depth(self): + def depth(self) -> int: """ Returns how many margins are currently defined. """ return len(self.indents) - def indent(self, line): + def indent(self, line: str) -> str: """ Indents a line by the currently defined margin. """ return self.margin + line - def dedent(self, line): + def dedent(self, line: str) -> str: """ Dedents a line by the currently defined margin. (The inverse of 'indent'.) From webhook-mailer at python.org Thu Jul 20 23:30:55 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 21 Jul 2023 03:30:55 -0000 Subject: [Python-checkins] gh-106669: Revert "gh-102988: Detect email address parsing errors ... (#105127)" (#106733) Message-ID: https://github.com/python/cpython/commit/a31dea1feb61793e48fa9aa5014f358352205c1d commit: a31dea1feb61793e48fa9aa5014f358352205c1d branch: main author: Gregory P. Smith committer: gpshead date: 2023-07-20T20:30:52-07:00 summary: gh-106669: Revert "gh-102988: Detect email address parsing errors ... (#105127)" (#106733) This reverts commit 18dfbd035775c15533d13a98e56b1d2bf5c65f00. Adds a regression test from the issue. See https://github.com/python/cpython/issues/106669. files: M Doc/library/email.utils.rst M Doc/whatsnew/3.12.rst M Lib/email/utils.py M Lib/test/test_email/test_email.py M Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst diff --git a/Doc/library/email.utils.rst b/Doc/library/email.utils.rst index a87a0bd2e7de6..345b64001c1ac 100644 --- a/Doc/library/email.utils.rst +++ b/Doc/library/email.utils.rst @@ -65,11 +65,6 @@ of the new API. *email address* parts. Returns a tuple of that information, unless the parse fails, in which case a 2-tuple of ``('', '')`` is returned. - .. versionchanged:: 3.12 - For security reasons, addresses that were ambiguous and could parse into - multiple different addresses now cause ``('', '')`` to be returned - instead of only one of the *potential* addresses. - .. function:: formataddr(pair, charset='utf-8') @@ -92,7 +87,7 @@ of the new API. This method returns a list of 2-tuples of the form returned by ``parseaddr()``. *fieldvalues* is a sequence of header field values as might be returned by :meth:`Message.get_all `. Here's a simple - example that gets all the recipients of a message: + example that gets all the recipients of a message:: from email.utils import getaddresses @@ -102,25 +97,6 @@ of the new API. resent_ccs = msg.get_all('resent-cc', []) all_recipients = getaddresses(tos + ccs + resent_tos + resent_ccs) - When parsing fails for a single fieldvalue, a 2-tuple of ``('', '')`` - is returned in its place. Other errors in parsing the list of - addresses such as a fieldvalue seemingly parsing into multiple - addresses may result in a list containing a single empty 2-tuple - ``[('', '')]`` being returned rather than returning potentially - invalid output. - - Example malformed input parsing: - - .. doctest:: - - >>> from email.utils import getaddresses - >>> getaddresses(['alice at example.com ', 'me at example.com']) - [('', '')] - - .. versionchanged:: 3.12 - The 2-tuple of ``('', '')`` in the returned values when parsing - fails were added as to address a security issue. - .. function:: parsedate(date) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index a6d101bdb9f7a..ab5e5b83e98c0 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -570,14 +570,6 @@ dis :data:`~dis.hasarg` collection instead. (Contributed by Irit Katriel in :gh:`94216`.) -email ------ - -* :func:`email.utils.getaddresses` and :func:`email.utils.parseaddr` now return - ``('', '')`` 2-tuples in more situations where invalid email addresses are - encountered instead of potentially inaccurate values. - (Contributed by Thomas Dwyer for :gh:`102988` to ameliorate CVE-2023-27043.) - fractions --------- diff --git a/Lib/email/utils.py b/Lib/email/utils.py index 11ad75e94e934..81da5394ea169 100644 --- a/Lib/email/utils.py +++ b/Lib/email/utils.py @@ -106,54 +106,12 @@ def formataddr(pair, charset='utf-8'): return address -def _pre_parse_validation(email_header_fields): - accepted_values = [] - for v in email_header_fields: - s = v.replace('\\(', '').replace('\\)', '') - if s.count('(') != s.count(')'): - v = "('', '')" - accepted_values.append(v) - - return accepted_values - - -def _post_parse_validation(parsed_email_header_tuples): - accepted_values = [] - # The parser would have parsed a correctly formatted domain-literal - # The existence of an [ after parsing indicates a parsing failure - for v in parsed_email_header_tuples: - if '[' in v[1]: - v = ('', '') - accepted_values.append(v) - - return accepted_values - def getaddresses(fieldvalues): - """Return a list of (REALNAME, EMAIL) or ('','') for each fieldvalue. - - When parsing fails for a fieldvalue, a 2-tuple of ('', '') is returned in - its place. - - If the resulting list of parsed address is not the same as the number of - fieldvalues in the input list a parsing error has occurred. A list - containing a single empty 2-tuple [('', '')] is returned in its place. - This is done to avoid invalid output. - """ - fieldvalues = [str(v) for v in fieldvalues] - fieldvalues = _pre_parse_validation(fieldvalues) - all = COMMASPACE.join(v for v in fieldvalues) + """Return a list of (REALNAME, EMAIL) for each fieldvalue.""" + all = COMMASPACE.join(str(v) for v in fieldvalues) a = _AddressList(all) - result = _post_parse_validation(a.addresslist) - - n = 0 - for v in fieldvalues: - n += v.count(',') + 1 - - if len(result) != n: - return [('', '')] - - return result + return a.addresslist def _format_timetuple_and_zone(timetuple, zone): @@ -254,18 +212,9 @@ def parseaddr(addr): Return a tuple of realname and email address, unless the parse fails, in which case return a 2-tuple of ('', ''). """ - if isinstance(addr, list): - addr = addr[0] - - if not isinstance(addr, str): - return ('', '') - - addr = _pre_parse_validation([addr])[0] - addrs = _post_parse_validation(_AddressList(addr).addresslist) - - if not addrs or len(addrs) > 1: - return ('', '') - + addrs = _AddressList(addr).addresslist + if not addrs: + return '', '' return addrs[0] diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index 5238944d6b478..b4f3a2481976e 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3319,90 +3319,32 @@ def test_getaddresses(self): [('Al Person', 'aperson at dom.ain'), ('Bud Person', 'bperson at dom.ain')]) - def test_getaddresses_parsing_errors(self): - """Test for parsing errors from CVE-2023-27043""" - eq = self.assertEqual - eq(utils.getaddresses(['alice at example.org(']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org)']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org<']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org>']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org@']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org,']), - [('', 'alice at example.org'), ('', 'bob at example.com')]) - eq(utils.getaddresses(['alice at example.org;']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org:']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org.']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org"']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org[']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org]']), - [('', '')]) - - def test_parseaddr_parsing_errors(self): - """Test for parsing errors from CVE-2023-27043""" - eq = self.assertEqual - eq(utils.parseaddr(['alice at example.org(']), - ('', '')) - eq(utils.parseaddr(['alice at example.org)']), - ('', '')) - eq(utils.parseaddr(['alice at example.org<']), - ('', '')) - eq(utils.parseaddr(['alice at example.org>']), - ('', '')) - eq(utils.parseaddr(['alice at example.org@']), - ('', '')) - eq(utils.parseaddr(['alice at example.org,']), - ('', '')) - eq(utils.parseaddr(['alice at example.org;']), - ('', '')) - eq(utils.parseaddr(['alice at example.org:']), - ('', '')) - eq(utils.parseaddr(['alice at example.org.']), - ('', '')) - eq(utils.parseaddr(['alice at example.org"']), - ('', '')) - eq(utils.parseaddr(['alice at example.org[']), - ('', '')) - eq(utils.parseaddr(['alice at example.org]']), - ('', '')) + def test_getaddresses_comma_in_name(self): + """GH-106669 regression test.""" + self.assertEqual( + utils.getaddresses( + [ + '"Bud, Person" ', + 'aperson at dom.ain (Al Person)', + '"Mariusz Felisiak" ', + ] + ), + [ + ('Bud, Person', 'bperson at dom.ain'), + ('Al Person', 'aperson at dom.ain'), + ('Mariusz Felisiak', 'to at example.com'), + ], + ) def test_getaddresses_nasty(self): eq = self.assertEqual eq(utils.getaddresses(['foo: ;']), [('', '')]) - eq(utils.getaddresses(['[]*-- =~$']), [('', '')]) + eq(utils.getaddresses( + ['[]*-- =~$']), + [('', ''), ('', ''), ('', '*--')]) eq(utils.getaddresses( ['foo: ;', '"Jason R. Mastaler" ']), [('', ''), ('Jason R. Mastaler', 'jason at dom.ain')]) - eq(utils.getaddresses( - [r'Pete(A nice \) chap) ']), - [('Pete (A nice ) chap his account his host)', 'pete at silly.test')]) - eq(utils.getaddresses( - ['(Empty list)(start)Undisclosed recipients :(nobody(I know))']), - [('', '')]) - eq(utils.getaddresses( - ['Mary <@machine.tld:mary at example.net>, , jdoe at test . example']), - [('Mary', 'mary at example.net'), ('', ''), ('', 'jdoe at test.example')]) - eq(utils.getaddresses( - ['John Doe ']), - [('John Doe (comment)', 'jdoe at machine.example')]) - eq(utils.getaddresses( - ['"Mary Smith: Personal Account" ']), - [('Mary Smith: Personal Account', 'smith at home.example')]) - eq(utils.getaddresses( - ['Undisclosed recipients:;']), - [('', '')]) - eq(utils.getaddresses( - [r', "Giant; \"Big\" Box" ']), - [('', 'boss at nil.test'), ('Giant; "Big" Box', 'bob at example.net')]) def test_getaddresses_embedded_comment(self): """Test proper handling of a nested comment""" diff --git a/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst b/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst index e0434ccd2ccab..c67ec45737b53 100644 --- a/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst +++ b/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst @@ -1,4 +1,4 @@ -CVE-2023-27043: Prevent :func:`email.utils.parseaddr` -and :func:`email.utils.getaddresses` from returning the realname portion of an -invalid RFC2822 email header in the email address portion of the 2-tuple -returned after being parsed by :class:`email._parseaddr.AddressList`. +Reverted the :mod:`email.utils` security improvement change released in +3.12beta4 that unintentionally caused :mod:`email.utils.getaddresses` to fail +to parse email addresses with a comma in the quoted name field. +See :gh:`106669`. From webhook-mailer at python.org Fri Jul 21 00:05:50 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 21 Jul 2023 04:05:50 -0000 Subject: [Python-checkins] [3.12] gh-106669: Revert "gh-102988: Detect email address parsing errors ... (GH-105127)" (GH-106733) (#106941) Message-ID: https://github.com/python/cpython/commit/656f62454bff35db8d630ca43c94bf6db44338ba commit: 656f62454bff35db8d630ca43c94bf6db44338ba branch: 3.12 author: Gregory P. Smith committer: gpshead date: 2023-07-21T04:05:46Z summary: [3.12] gh-106669: Revert "gh-102988: Detect email address parsing errors ... (GH-105127)" (GH-106733) (#106941) This reverts commit 18dfbd035775c15533d13a98e56b1d2bf5c65f00. Adds a regression test from the issue. See https://github.com/python/cpython/issues/106669.. (cherry picked from commit a31dea1feb61793e48fa9aa5014f358352205c1d) files: A Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst M Doc/library/email.utils.rst M Doc/whatsnew/3.12.rst M Lib/email/utils.py M Lib/test/test_email/test_email.py diff --git a/Doc/library/email.utils.rst b/Doc/library/email.utils.rst index a87a0bd2e7de6..345b64001c1ac 100644 --- a/Doc/library/email.utils.rst +++ b/Doc/library/email.utils.rst @@ -65,11 +65,6 @@ of the new API. *email address* parts. Returns a tuple of that information, unless the parse fails, in which case a 2-tuple of ``('', '')`` is returned. - .. versionchanged:: 3.12 - For security reasons, addresses that were ambiguous and could parse into - multiple different addresses now cause ``('', '')`` to be returned - instead of only one of the *potential* addresses. - .. function:: formataddr(pair, charset='utf-8') @@ -92,7 +87,7 @@ of the new API. This method returns a list of 2-tuples of the form returned by ``parseaddr()``. *fieldvalues* is a sequence of header field values as might be returned by :meth:`Message.get_all `. Here's a simple - example that gets all the recipients of a message: + example that gets all the recipients of a message:: from email.utils import getaddresses @@ -102,25 +97,6 @@ of the new API. resent_ccs = msg.get_all('resent-cc', []) all_recipients = getaddresses(tos + ccs + resent_tos + resent_ccs) - When parsing fails for a single fieldvalue, a 2-tuple of ``('', '')`` - is returned in its place. Other errors in parsing the list of - addresses such as a fieldvalue seemingly parsing into multiple - addresses may result in a list containing a single empty 2-tuple - ``[('', '')]`` being returned rather than returning potentially - invalid output. - - Example malformed input parsing: - - .. doctest:: - - >>> from email.utils import getaddresses - >>> getaddresses(['alice at example.com ', 'me at example.com']) - [('', '')] - - .. versionchanged:: 3.12 - The 2-tuple of ``('', '')`` in the returned values when parsing - fails were added as to address a security issue. - .. function:: parsedate(date) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index d6d7b7dac9a31..068618fe48e1b 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -570,14 +570,6 @@ dis :data:`~dis.hasarg` collection instead. (Contributed by Irit Katriel in :gh:`94216`.) -email ------ - -* :func:`email.utils.getaddresses` and :func:`email.utils.parseaddr` now return - ``('', '')`` 2-tuples in more situations where invalid email addresses are - encountered instead of potentially inaccurate values. - (Contributed by Thomas Dwyer for :gh:`102988` to ameliorate CVE-2023-27043.) - fractions --------- diff --git a/Lib/email/utils.py b/Lib/email/utils.py index 11ad75e94e934..81da5394ea169 100644 --- a/Lib/email/utils.py +++ b/Lib/email/utils.py @@ -106,54 +106,12 @@ def formataddr(pair, charset='utf-8'): return address -def _pre_parse_validation(email_header_fields): - accepted_values = [] - for v in email_header_fields: - s = v.replace('\\(', '').replace('\\)', '') - if s.count('(') != s.count(')'): - v = "('', '')" - accepted_values.append(v) - - return accepted_values - - -def _post_parse_validation(parsed_email_header_tuples): - accepted_values = [] - # The parser would have parsed a correctly formatted domain-literal - # The existence of an [ after parsing indicates a parsing failure - for v in parsed_email_header_tuples: - if '[' in v[1]: - v = ('', '') - accepted_values.append(v) - - return accepted_values - def getaddresses(fieldvalues): - """Return a list of (REALNAME, EMAIL) or ('','') for each fieldvalue. - - When parsing fails for a fieldvalue, a 2-tuple of ('', '') is returned in - its place. - - If the resulting list of parsed address is not the same as the number of - fieldvalues in the input list a parsing error has occurred. A list - containing a single empty 2-tuple [('', '')] is returned in its place. - This is done to avoid invalid output. - """ - fieldvalues = [str(v) for v in fieldvalues] - fieldvalues = _pre_parse_validation(fieldvalues) - all = COMMASPACE.join(v for v in fieldvalues) + """Return a list of (REALNAME, EMAIL) for each fieldvalue.""" + all = COMMASPACE.join(str(v) for v in fieldvalues) a = _AddressList(all) - result = _post_parse_validation(a.addresslist) - - n = 0 - for v in fieldvalues: - n += v.count(',') + 1 - - if len(result) != n: - return [('', '')] - - return result + return a.addresslist def _format_timetuple_and_zone(timetuple, zone): @@ -254,18 +212,9 @@ def parseaddr(addr): Return a tuple of realname and email address, unless the parse fails, in which case return a 2-tuple of ('', ''). """ - if isinstance(addr, list): - addr = addr[0] - - if not isinstance(addr, str): - return ('', '') - - addr = _pre_parse_validation([addr])[0] - addrs = _post_parse_validation(_AddressList(addr).addresslist) - - if not addrs or len(addrs) > 1: - return ('', '') - + addrs = _AddressList(addr).addresslist + if not addrs: + return '', '' return addrs[0] diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index 5238944d6b478..b4f3a2481976e 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3319,90 +3319,32 @@ def test_getaddresses(self): [('Al Person', 'aperson at dom.ain'), ('Bud Person', 'bperson at dom.ain')]) - def test_getaddresses_parsing_errors(self): - """Test for parsing errors from CVE-2023-27043""" - eq = self.assertEqual - eq(utils.getaddresses(['alice at example.org(']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org)']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org<']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org>']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org@']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org,']), - [('', 'alice at example.org'), ('', 'bob at example.com')]) - eq(utils.getaddresses(['alice at example.org;']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org:']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org.']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org"']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org[']), - [('', '')]) - eq(utils.getaddresses(['alice at example.org]']), - [('', '')]) - - def test_parseaddr_parsing_errors(self): - """Test for parsing errors from CVE-2023-27043""" - eq = self.assertEqual - eq(utils.parseaddr(['alice at example.org(']), - ('', '')) - eq(utils.parseaddr(['alice at example.org)']), - ('', '')) - eq(utils.parseaddr(['alice at example.org<']), - ('', '')) - eq(utils.parseaddr(['alice at example.org>']), - ('', '')) - eq(utils.parseaddr(['alice at example.org@']), - ('', '')) - eq(utils.parseaddr(['alice at example.org,']), - ('', '')) - eq(utils.parseaddr(['alice at example.org;']), - ('', '')) - eq(utils.parseaddr(['alice at example.org:']), - ('', '')) - eq(utils.parseaddr(['alice at example.org.']), - ('', '')) - eq(utils.parseaddr(['alice at example.org"']), - ('', '')) - eq(utils.parseaddr(['alice at example.org[']), - ('', '')) - eq(utils.parseaddr(['alice at example.org]']), - ('', '')) + def test_getaddresses_comma_in_name(self): + """GH-106669 regression test.""" + self.assertEqual( + utils.getaddresses( + [ + '"Bud, Person" ', + 'aperson at dom.ain (Al Person)', + '"Mariusz Felisiak" ', + ] + ), + [ + ('Bud, Person', 'bperson at dom.ain'), + ('Al Person', 'aperson at dom.ain'), + ('Mariusz Felisiak', 'to at example.com'), + ], + ) def test_getaddresses_nasty(self): eq = self.assertEqual eq(utils.getaddresses(['foo: ;']), [('', '')]) - eq(utils.getaddresses(['[]*-- =~$']), [('', '')]) + eq(utils.getaddresses( + ['[]*-- =~$']), + [('', ''), ('', ''), ('', '*--')]) eq(utils.getaddresses( ['foo: ;', '"Jason R. Mastaler" ']), [('', ''), ('Jason R. Mastaler', 'jason at dom.ain')]) - eq(utils.getaddresses( - [r'Pete(A nice \) chap) ']), - [('Pete (A nice ) chap his account his host)', 'pete at silly.test')]) - eq(utils.getaddresses( - ['(Empty list)(start)Undisclosed recipients :(nobody(I know))']), - [('', '')]) - eq(utils.getaddresses( - ['Mary <@machine.tld:mary at example.net>, , jdoe at test . example']), - [('Mary', 'mary at example.net'), ('', ''), ('', 'jdoe at test.example')]) - eq(utils.getaddresses( - ['John Doe ']), - [('John Doe (comment)', 'jdoe at machine.example')]) - eq(utils.getaddresses( - ['"Mary Smith: Personal Account" ']), - [('Mary Smith: Personal Account', 'smith at home.example')]) - eq(utils.getaddresses( - ['Undisclosed recipients:;']), - [('', '')]) - eq(utils.getaddresses( - [r', "Giant; \"Big\" Box" ']), - [('', 'boss at nil.test'), ('Giant; "Big" Box', 'bob at example.net')]) def test_getaddresses_embedded_comment(self): """Test proper handling of a nested comment""" diff --git a/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst b/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst new file mode 100644 index 0000000000000..c67ec45737b53 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2023-06-13-20-52-24.gh-issue-102988.Kei7Vf.rst @@ -0,0 +1,4 @@ +Reverted the :mod:`email.utils` security improvement change released in +3.12beta4 that unintentionally caused :mod:`email.utils.getaddresses` to fail +to parse email addresses with a comma in the quoted name field. +See :gh:`106669`. From webhook-mailer at python.org Fri Jul 21 00:57:02 2023 From: webhook-mailer at python.org (methane) Date: Fri, 21 Jul 2023 04:57:02 -0000 Subject: [Python-checkins] gh-106916: Add missing error check _PyCompile_CleanDoc (#106921) Message-ID: https://github.com/python/cpython/commit/85ed1d24427bf3e000467aab7ee1b0322b0a9013 commit: 85ed1d24427bf3e000467aab7ee1b0322b0a9013 branch: main author: Kirill Podoprigora committer: methane date: 2023-07-21T13:56:58+09:00 summary: gh-106916: Add missing error check _PyCompile_CleanDoc (#106921) Co-authored-by: Serhiy Storchaka files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index d5405b4656182..9a13ab525d8a4 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8053,6 +8053,12 @@ _PyCompile_CleanDoc(PyObject *doc) } char *buff = PyMem_Malloc(doc_size); + if (buff == NULL){ + Py_DECREF(doc); + PyErr_NoMemory(); + return NULL; + } + char *w = buff; while (p < pend) { From webhook-mailer at python.org Fri Jul 21 02:02:43 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Fri, 21 Jul 2023 06:02:43 -0000 Subject: [Python-checkins] gh-106368: Increase Argument Clinic test coverage for IndentStack (#106933) Message-ID: https://github.com/python/cpython/commit/8d228cf66f316803e95685d6553084f3d60cd9c5 commit: 8d228cf66f316803e95685d6553084f3d60cd9c5 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-21T08:02:39+02:00 summary: gh-106368: Increase Argument Clinic test coverage for IndentStack (#106933) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index e925ecca1b9c5..7c725e33f5392 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1035,6 +1035,25 @@ def test_function_not_at_column_0(self): Nested docstring here, goeth. """) + def test_indent_stack_no_tabs(self): + out = self.parse_function_should_fail(""" + module foo + foo.bar + *vararg1: object + \t*vararg2: object + """) + msg = "Tab characters are illegal in the Clinic DSL." + self.assertIn(msg, out) + + def test_indent_stack_illegal_outdent(self): + out = self.parse_function_should_fail(""" + module foo + foo.bar + a: object + b: object + """) + self.assertIn("Illegal outdent", out) + def test_directive(self): c = FakeClinic() parser = DSLParser(c) From webhook-mailer at python.org Fri Jul 21 02:05:45 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Fri, 21 Jul 2023 06:05:45 -0000 Subject: [Python-checkins] Docs: Argument Clinic: Add Background and Tutorial top-level sections (#106904) Message-ID: https://github.com/python/cpython/commit/81861fd90b4ae981e7881cd03a3c370713063525 commit: 81861fd90b4ae981e7881cd03a3c370713063525 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-21T08:05:41+02:00 summary: Docs: Argument Clinic: Add Background and Tutorial top-level sections (#106904) Add Background as a toplevel section with the following subsections: - Background - The goals of Argument Clinic - Basic concepts and usage Rename "Converting your first function" to Tutorial. Add anchors for Background, Tutorial, and How-to Guides: - :ref:`clinic-background` - :ref:`clinic-tutorial` - :ref:`clinic-howtos` Link to these from within the Abstract. Break the compatibility paragraph out of Abstract and make it a note. Co-authored-by: Ezio Melotti files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index efeb22c618b51..f6bf1d2234f88 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -13,12 +13,20 @@ Argument Clinic How-To Argument Clinic is a preprocessor for CPython C files. Its purpose is to automate all the boilerplate involved - with writing argument parsing code for "builtins". - This document shows you how to convert your first C - function to work with Argument Clinic, and then introduces - some advanced topics on Argument Clinic usage. + with writing argument parsing code for "builtins", + module level functions, and class methods. + This document is divided in three major sections: - Currently Argument Clinic is considered internal-only + * :ref:`clinic-background` talks about the basic concepts and goals of + Argument Clinic. + * :ref:`clinic-tutorial` guides you through all the steps required to + adapt an existing C function to Argument Clinic. + * :ref:`clinic-howtos` details how to handle specific tasks. + + +.. note:: + + Argument Clinic is considered internal-only for CPython. Its use is not supported for files outside CPython, and no guarantees are made regarding backwards compatibility for future versions. In other words: if you @@ -28,8 +36,14 @@ Argument Clinic How-To of CPython *could* be totally incompatible and break all your code. +.. _clinic-background: + +Background +========== + + The goals of Argument Clinic -============================ +---------------------------- Argument Clinic's primary goal is to take over responsibility for all argument parsing code @@ -80,7 +94,7 @@ things with all the information you give it. Basic concepts and usage -======================== +------------------------ Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``. If you run that script, specifying a C file as an argument: @@ -142,8 +156,10 @@ For the sake of clarity, here's the terminology we'll use with Argument Clinic: a block.) -Converting your first function -============================== +.. _clinic-tutorial: + +Tutorial +======== The best way to get a sense of how Argument Clinic works is to convert a function to work with it. Here, then, are the bare @@ -560,6 +576,8 @@ Let's dive in! Congratulations, you've ported your first function to work with Argument Clinic! +.. _clinic-howtos: + How-to guides ============= From webhook-mailer at python.org Fri Jul 21 02:18:22 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Fri, 21 Jul 2023 06:18:22 -0000 Subject: [Python-checkins] [3.12] Docs: Argument Clinic: Add Background and Tutorial top-level sections (GH-106904) (#106945) Message-ID: https://github.com/python/cpython/commit/1a3766bb3e5ad5cdd7c3e4c352eb0147ccc9a6ce commit: 1a3766bb3e5ad5cdd7c3e4c352eb0147ccc9a6ce branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-21T08:18:18+02:00 summary: [3.12] Docs: Argument Clinic: Add Background and Tutorial top-level sections (GH-106904) (#106945) Docs: Argument Clinic: Add Background and Tutorial top-level sections (GH-106904) Add Background as a toplevel section with the following subsections: - Background - The goals of Argument Clinic - Basic concepts and usage Rename "Converting your first function" to Tutorial. Add anchors for Background, Tutorial, and How-to Guides: - :ref:`clinic-background` - :ref:`clinic-tutorial` - :ref:`clinic-howtos` Link to these from within the Abstract. Break the compatibility paragraph out of Abstract and make it a note. (cherry picked from commit 81861fd90b4ae981e7881cd03a3c370713063525) Co-authored-by: Erlend E. Aasland Co-authored-by: Ezio Melotti files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index efeb22c618b51..f6bf1d2234f88 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -13,12 +13,20 @@ Argument Clinic How-To Argument Clinic is a preprocessor for CPython C files. Its purpose is to automate all the boilerplate involved - with writing argument parsing code for "builtins". - This document shows you how to convert your first C - function to work with Argument Clinic, and then introduces - some advanced topics on Argument Clinic usage. + with writing argument parsing code for "builtins", + module level functions, and class methods. + This document is divided in three major sections: - Currently Argument Clinic is considered internal-only + * :ref:`clinic-background` talks about the basic concepts and goals of + Argument Clinic. + * :ref:`clinic-tutorial` guides you through all the steps required to + adapt an existing C function to Argument Clinic. + * :ref:`clinic-howtos` details how to handle specific tasks. + + +.. note:: + + Argument Clinic is considered internal-only for CPython. Its use is not supported for files outside CPython, and no guarantees are made regarding backwards compatibility for future versions. In other words: if you @@ -28,8 +36,14 @@ Argument Clinic How-To of CPython *could* be totally incompatible and break all your code. +.. _clinic-background: + +Background +========== + + The goals of Argument Clinic -============================ +---------------------------- Argument Clinic's primary goal is to take over responsibility for all argument parsing code @@ -80,7 +94,7 @@ things with all the information you give it. Basic concepts and usage -======================== +------------------------ Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``. If you run that script, specifying a C file as an argument: @@ -142,8 +156,10 @@ For the sake of clarity, here's the terminology we'll use with Argument Clinic: a block.) -Converting your first function -============================== +.. _clinic-tutorial: + +Tutorial +======== The best way to get a sense of how Argument Clinic works is to convert a function to work with it. Here, then, are the bare @@ -560,6 +576,8 @@ Let's dive in! Congratulations, you've ported your first function to work with Argument Clinic! +.. _clinic-howtos: + How-to guides ============= From webhook-mailer at python.org Fri Jul 21 02:31:04 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Fri, 21 Jul 2023 06:31:04 -0000 Subject: [Python-checkins] [3.11] gh-106368: Increase Argument Clinic test coverage for IndentStack (GH-106933) (#106944) Message-ID: https://github.com/python/cpython/commit/951b08c5006fe6a7daff4a9769964b4017a9b754 commit: 951b08c5006fe6a7daff4a9769964b4017a9b754 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-21T06:31:00Z summary: [3.11] gh-106368: Increase Argument Clinic test coverage for IndentStack (GH-106933) (#106944) (cherry picked from commit 8d228cf66f316803e95685d6553084f3d60cd9c5) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index cef121d9a1ead..605d101b5bef9 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1035,6 +1035,25 @@ def test_function_not_at_column_0(self): Nested docstring here, goeth. """) + def test_indent_stack_no_tabs(self): + out = self.parse_function_should_fail(""" + module foo + foo.bar + *vararg1: object + \t*vararg2: object + """) + msg = "Tab characters are illegal in the Clinic DSL." + self.assertIn(msg, out) + + def test_indent_stack_illegal_outdent(self): + out = self.parse_function_should_fail(""" + module foo + foo.bar + a: object + b: object + """) + self.assertIn("Illegal outdent", out) + def test_directive(self): c = FakeClinic() parser = DSLParser(c) From webhook-mailer at python.org Fri Jul 21 02:32:34 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Fri, 21 Jul 2023 06:32:34 -0000 Subject: [Python-checkins] [3.12] gh-106368: Increase Argument Clinic test coverage for IndentStack (GH-106933) (#106943) Message-ID: https://github.com/python/cpython/commit/807afdac416356c5e4558f11a8e1cfc5f0c86dd7 commit: 807afdac416356c5e4558f11a8e1cfc5f0c86dd7 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-21T06:32:30Z summary: [3.12] gh-106368: Increase Argument Clinic test coverage for IndentStack (GH-106933) (#106943) (cherry picked from commit 8d228cf66f316803e95685d6553084f3d60cd9c5) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index e925ecca1b9c5..7c725e33f5392 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1035,6 +1035,25 @@ def test_function_not_at_column_0(self): Nested docstring here, goeth. """) + def test_indent_stack_no_tabs(self): + out = self.parse_function_should_fail(""" + module foo + foo.bar + *vararg1: object + \t*vararg2: object + """) + msg = "Tab characters are illegal in the Clinic DSL." + self.assertIn(msg, out) + + def test_indent_stack_illegal_outdent(self): + out = self.parse_function_should_fail(""" + module foo + foo.bar + a: object + b: object + """) + self.assertIn("Illegal outdent", out) + def test_directive(self): c = FakeClinic() parser = DSLParser(c) From webhook-mailer at python.org Fri Jul 21 02:33:00 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Fri, 21 Jul 2023 06:33:00 -0000 Subject: [Python-checkins] [3.11] Docs: Argument Clinic: Add Background and Tutorial top-level sections (GH-106904) (#106946) Message-ID: https://github.com/python/cpython/commit/0a57620887f9877398c44062539da3e58235f0e6 commit: 0a57620887f9877398c44062539da3e58235f0e6 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-21T08:32:56+02:00 summary: [3.11] Docs: Argument Clinic: Add Background and Tutorial top-level sections (GH-106904) (#106946) Add Background as a toplevel section with the following subsections: - Background - The goals of Argument Clinic - Basic concepts and usage Rename "Converting your first function" to Tutorial. Add anchors for Background, Tutorial, and How-to Guides: - :ref:`clinic-background` - :ref:`clinic-tutorial` - :ref:`clinic-howtos` Link to these from within the Abstract. Break the compatibility paragraph out of Abstract and make it a note. (cherry picked from commit 81861fd90b4ae981e7881cd03a3c370713063525) Co-authored-by: Erlend E. Aasland Co-authored-by: Ezio Melotti files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index c2e5a68fb9eb2..50703d9b4723d 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -13,12 +13,20 @@ Argument Clinic How-To Argument Clinic is a preprocessor for CPython C files. Its purpose is to automate all the boilerplate involved - with writing argument parsing code for "builtins". - This document shows you how to convert your first C - function to work with Argument Clinic, and then introduces - some advanced topics on Argument Clinic usage. + with writing argument parsing code for "builtins", + module level functions, and class methods. + This document is divided in three major sections: - Currently Argument Clinic is considered internal-only + * :ref:`clinic-background` talks about the basic concepts and goals of + Argument Clinic. + * :ref:`clinic-tutorial` guides you through all the steps required to + adapt an existing C function to Argument Clinic. + * :ref:`clinic-howtos` details how to handle specific tasks. + + +.. note:: + + Argument Clinic is considered internal-only for CPython. Its use is not supported for files outside CPython, and no guarantees are made regarding backwards compatibility for future versions. In other words: if you @@ -28,8 +36,14 @@ Argument Clinic How-To of CPython *could* be totally incompatible and break all your code. +.. _clinic-background: + +Background +========== + + The goals of Argument Clinic -============================ +---------------------------- Argument Clinic's primary goal is to take over responsibility for all argument parsing code @@ -80,7 +94,7 @@ things with all the information you give it. Basic concepts and usage -======================== +------------------------ Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``. If you run that script, specifying a C file as an argument: @@ -142,8 +156,10 @@ For the sake of clarity, here's the terminology we'll use with Argument Clinic: a block.) -Converting your first function -============================== +.. _clinic-tutorial: + +Tutorial +======== The best way to get a sense of how Argument Clinic works is to convert a function to work with it. Here, then, are the bare @@ -552,6 +568,8 @@ Let's dive in! Congratulations, you've ported your first function to work with Argument Clinic! +.. _clinic-howtos: + How-to guides ============= From webhook-mailer at python.org Fri Jul 21 03:52:12 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 07:52:12 -0000 Subject: [Python-checkins] gh-106919: Use role :c:macro: for referencing the C "constants" (GH-106920) Message-ID: https://github.com/python/cpython/commit/fcc816dbff7ca66c26f57a506e4d2330fe41d0ff commit: fcc816dbff7ca66c26f57a506e4d2330fe41d0ff branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T10:52:07+03:00 summary: gh-106919: Use role :c:macro: for referencing the C "constants" (GH-106920) files: M Doc/c-api/arg.rst M Doc/c-api/call.rst M Doc/c-api/complex.rst M Doc/c-api/exceptions.rst M Doc/c-api/file.rst M Doc/c-api/float.rst M Doc/c-api/gcsupport.rst M Doc/c-api/init.rst M Doc/c-api/long.rst M Doc/c-api/memory.rst M Doc/c-api/module.rst M Doc/c-api/object.rst M Doc/c-api/slice.rst M Doc/c-api/stable.rst M Doc/c-api/structures.rst M Doc/c-api/sys.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/c-api/veryhigh.rst M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/howto/isolating-extensions.rst M Doc/library/dis.rst M Doc/library/os.rst M Doc/reference/compound_stmts.rst M Doc/using/cmdline.rst M Doc/whatsnew/2.2.rst M Doc/whatsnew/2.3.rst M Doc/whatsnew/2.4.rst M Doc/whatsnew/2.6.rst M Doc/whatsnew/2.7.rst M Doc/whatsnew/3.10.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.13.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a3.rst M Misc/NEWS.d/3.10.0b1.rst M Misc/NEWS.d/3.11.0a1.rst M Misc/NEWS.d/3.11.0a7.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0a2.rst M Misc/NEWS.d/3.6.0a1.rst M Misc/NEWS.d/3.9.0a1.rst diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index d2ea490732fe5..e8d141f173933 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -343,7 +343,7 @@ Other objects *items*. Format units for sequences may be nested. It is possible to pass "long" integers (integers whose value exceeds the -platform's :const:`LONG_MAX`) however no proper range checking is done --- the +platform's :c:macro:`LONG_MAX`) however no proper range checking is done --- the most significant bits are silently truncated when the receiving field is too small to receive the value (actually, the semantics are inherited from downcasts in C --- your mileage may vary). @@ -463,7 +463,7 @@ API Functions A simpler form of parameter retrieval which does not use a format string to specify the types of the arguments. Functions which use this method to retrieve - their parameters should be declared as :const:`METH_VARARGS` in function or + their parameters should be declared as :c:macro:`METH_VARARGS` in function or method tables. The tuple containing the actual parameters should be passed as *args*; it must actually be a tuple. The length of the tuple must be at least *min* and no more than *max*; *min* and *max* may be equal. Additional diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index ac6242701c504..8be4b5ab80c2d 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -59,12 +59,12 @@ This bears repeating: .. versionchanged:: 3.12 - The :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class + The :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class when the class's :py:meth:`~object.__call__` method is reassigned. (This internally sets :c:member:`~PyTypeObject.tp_call` only, and thus may make it behave differently than the vectorcall function.) In earlier Python versions, vectorcall should only be used with - :const:`immutable ` or static types. + :c:macro:`immutable ` or static types. A class should not implement vectorcall if that would be slower than *tp_call*. For example, if the callee needs to convert @@ -72,7 +72,7 @@ the arguments to an args tuple and kwargs dict anyway, then there is no point in implementing vectorcall. Classes can implement the vectorcall protocol by enabling the -:const:`Py_TPFLAGS_HAVE_VECTORCALL` flag and setting +:c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag and setting :c:member:`~PyTypeObject.tp_vectorcall_offset` to the offset inside the object structure where a *vectorcallfunc* appears. This is a pointer to a function with the following signature: @@ -84,7 +84,7 @@ This is a pointer to a function with the following signature: values of the keyword arguments. This can be *NULL* if there are no arguments. - *nargsf* is the number of positional arguments plus possibly the - :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. + :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. To get the actual number of positional arguments from *nargsf*, use :c:func:`PyVectorcall_NARGS`. - *kwnames* is a tuple containing the names of the keyword arguments; @@ -93,7 +93,7 @@ This is a pointer to a function with the following signature: and they must be unique. If there are no keyword arguments, then *kwnames* can instead be *NULL*. -.. data:: PY_VECTORCALL_ARGUMENTS_OFFSET +.. c:macro:: PY_VECTORCALL_ARGUMENTS_OFFSET If this flag is set in a vectorcall *nargsf* argument, the callee is allowed to temporarily change ``args[-1]``. In other words, *args* points to @@ -104,7 +104,7 @@ This is a pointer to a function with the following signature: ``args[0]`` may be changed. Whenever they can do so cheaply (without additional allocation), callers - are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. + are encouraged to use :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET`. Doing so will allow callables such as bound methods to make their onward calls (which include a prepended *self* argument) very efficiently. @@ -161,7 +161,7 @@ Vectorcall Support API This is a specialized function, intended to be put in the :c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``. - It does not check the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag + It does not check the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag and it does not fall back to ``tp_call``. .. versionadded:: 3.8 @@ -379,11 +379,11 @@ please see individual documentation for details. *args[0]*, and the *args* array starting at *args[1]* represents the arguments of the call. There must be at least one positional argument. *nargsf* is the number of positional arguments including *args[0]*, - plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may + plus :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may temporarily be changed. Keyword arguments can be passed just like in :c:func:`PyObject_Vectorcall`. - If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, + If the object has the :c:macro:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, this will call the unbound method object with the full *args* vector as arguments. diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst index cb8b270fcbab6..6679ce76f1dc6 100644 --- a/Doc/c-api/complex.rst +++ b/Doc/c-api/complex.rst @@ -64,7 +64,7 @@ pointers. This is consistent throughout the API. representation. If *divisor* is null, this method returns zero and sets - :c:data:`errno` to :c:data:`EDOM`. + :c:data:`errno` to :c:macro:`EDOM`. .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp) @@ -73,7 +73,7 @@ pointers. This is consistent throughout the API. representation. If *num* is null and *exp* is not a positive real number, - this method returns zero and sets :c:data:`errno` to :c:data:`EDOM`. + this method returns zero and sets :c:data:`errno` to :c:macro:`EDOM`. Complex Numbers as Python Objects diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index a24ecac861e76..6e09f829da304 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -165,7 +165,7 @@ For convenience, some of these functions will always return a tuple object whose first item is the integer :c:data:`errno` value and whose second item is the corresponding error message (gotten from :c:func:`strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the - :c:data:`errno` value is :const:`EINTR`, indicating an interrupted system call, + :c:data:`errno` value is :c:macro:`EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, leaves it set to that. The function always returns ``NULL``, so a wrapper function around a system call can write ``return PyErr_SetFromErrno(type);`` @@ -631,7 +631,7 @@ Signal Handling be interruptible by user requests (such as by pressing Ctrl-C). .. note:: - The default Python signal handler for :const:`SIGINT` raises the + The default Python signal handler for :c:macro:`SIGINT` raises the :exc:`KeyboardInterrupt` exception. @@ -642,7 +642,7 @@ Signal Handling single: SIGINT single: KeyboardInterrupt (built-in exception) - Simulate the effect of a :const:`SIGINT` signal arriving. + Simulate the effect of a :c:macro:`SIGINT` signal arriving. This is equivalent to ``PyErr_SetInterruptEx(SIGINT)``. .. note:: @@ -754,7 +754,7 @@ Exception Objects .. c:function:: PyObject* PyException_GetCause(PyObject *ex) - Return the cause (either an exception instance, or :const:`None`, + Return the cause (either an exception instance, or ``None``, set by ``raise ... from ...``) associated with the exception as a new reference, as accessible from Python through :attr:`__cause__`. @@ -763,7 +763,7 @@ Exception Objects Set the cause associated with the exception to *cause*. Use ``NULL`` to clear it. There is no type check to make sure that *cause* is either an exception - instance or :const:`None`. This steals a reference to *cause*. + instance or ``None``. This steals a reference to *cause*. :attr:`__suppress_context__` is implicitly set to ``True`` by this function. @@ -874,7 +874,7 @@ because the :ref:`call protocol ` takes care of recursion handling. Marks a point where a recursive C-level call is about to be performed. - If :const:`USE_STACKCHECK` is defined, this function checks if the OS + If :c:macro:`USE_STACKCHECK` is defined, this function checks if the OS stack overflowed using :c:func:`PyOS_CheckStack`. In this is the case, it sets a :exc:`MemoryError` and returns a nonzero value. diff --git a/Doc/c-api/file.rst b/Doc/c-api/file.rst index f32ecba9f2702..b36c800e00444 100644 --- a/Doc/c-api/file.rst +++ b/Doc/c-api/file.rst @@ -93,7 +93,7 @@ the :mod:`io` APIs instead. .. index:: single: Py_PRINT_RAW Write object *obj* to file object *p*. The only supported flag for *flags* is - :const:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written + :c:macro:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written instead of the :func:`repr`. Return ``0`` on success or ``-1`` on failure; the appropriate exception will be set. diff --git a/Doc/c-api/float.rst b/Doc/c-api/float.rst index fd0be1108c630..4f6ac0d8175c6 100644 --- a/Doc/c-api/float.rst +++ b/Doc/c-api/float.rst @@ -109,7 +109,7 @@ Pack functions The pack routines write 2, 4 or 8 bytes, starting at *p*. *le* is an :c:expr:`int` argument, non-zero if you want the bytes string in little-endian format (exponent last, at ``p+1``, ``p+3``, or ``p+6`` ``p+7``), zero if you -want big-endian format (exponent first, at *p*). The :c:data:`PY_BIG_ENDIAN` +want big-endian format (exponent first, at *p*). The :c:macro:`PY_BIG_ENDIAN` constant can be used to use the native endian: it is equal to ``1`` on big endian processor, or ``0`` on little endian processor. @@ -140,7 +140,7 @@ Unpack functions The unpack routines read 2, 4 or 8 bytes, starting at *p*. *le* is an :c:expr:`int` argument, non-zero if the bytes string is in little-endian format (exponent last, at ``p+1``, ``p+3`` or ``p+6`` and ``p+7``), zero if big-endian -(exponent first, at *p*). The :c:data:`PY_BIG_ENDIAN` constant can be used to +(exponent first, at *p*). The :c:macro:`PY_BIG_ENDIAN` constant can be used to use the native endian: it is equal to ``1`` on big endian processor, or ``0`` on little endian processor. diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index c3260a21bc7f8..e56414ab9f754 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -13,14 +13,12 @@ or strings), do not need to provide any explicit support for garbage collection. To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must -include the :const:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the +include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the :c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a :c:member:`~PyTypeObject.tp_clear` implementation must also be provided. -.. data:: Py_TPFLAGS_HAVE_GC - :noindex: - +:c:macro:`Py_TPFLAGS_HAVE_GC` Objects with a type with this flag set must conform with the rules documented here. For convenience these objects will be referred to as container objects. @@ -52,17 +50,17 @@ rules: :c:member:`~PyTypeObject.tp_flags`, :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields if the type inherits from a class that implements the garbage collector protocol and the child class - does *not* include the :const:`Py_TPFLAGS_HAVE_GC` flag. + does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. .. c:function:: TYPE* PyObject_GC_New(TYPE, PyTypeObject *type) Analogous to :c:func:`PyObject_New` but for container objects with the - :const:`Py_TPFLAGS_HAVE_GC` flag set. + :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. .. c:function:: TYPE* PyObject_GC_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) Analogous to :c:func:`PyObject_NewVar` but for container objects with the - :const:`Py_TPFLAGS_HAVE_GC` flag set. + :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. .. c:function:: PyObject* PyUnstable_Object_GC_NewWithExtraData(PyTypeObject *type, size_t extra_size) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index e7b2937d38dcf..48cea69c96940 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1161,7 +1161,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. function does not steal any references to *exc*. To prevent naive misuse, you must write your own C extension to call this. Must be called with the GIL held. Returns the number of thread states modified; this is normally one, but will be - zero if the thread id isn't found. If *exc* is :const:`NULL`, the pending + zero if the thread id isn't found. If *exc* is ``NULL``, the pending exception (if any) for the thread is cleared. This raises no exceptions. .. versionchanged:: 3.7 @@ -1407,32 +1407,32 @@ Python-level trace functions in previous versions. The type of the trace function registered using :c:func:`PyEval_SetProfile` and :c:func:`PyEval_SetTrace`. The first parameter is the object passed to the registration function as *obj*, *frame* is the frame object to which the event - pertains, *what* is one of the constants :const:`PyTrace_CALL`, - :const:`PyTrace_EXCEPTION`, :const:`PyTrace_LINE`, :const:`PyTrace_RETURN`, - :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION`, :const:`PyTrace_C_RETURN`, - or :const:`PyTrace_OPCODE`, and *arg* depends on the value of *what*: - - +------------------------------+----------------------------------------+ - | Value of *what* | Meaning of *arg* | - +==============================+========================================+ - | :const:`PyTrace_CALL` | Always :c:data:`Py_None`. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_EXCEPTION` | Exception information as returned by | - | | :func:`sys.exc_info`. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_LINE` | Always :c:data:`Py_None`. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_RETURN` | Value being returned to the caller, | - | | or ``NULL`` if caused by an exception. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_C_CALL` | Function object being called. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_C_EXCEPTION` | Function object being called. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_C_RETURN` | Function object being called. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_OPCODE` | Always :c:data:`Py_None`. | - +------------------------------+----------------------------------------+ + pertains, *what* is one of the constants :c:data:`PyTrace_CALL`, + :c:data:`PyTrace_EXCEPTION`, :c:data:`PyTrace_LINE`, :c:data:`PyTrace_RETURN`, + :c:data:`PyTrace_C_CALL`, :c:data:`PyTrace_C_EXCEPTION`, :c:data:`PyTrace_C_RETURN`, + or :c:data:`PyTrace_OPCODE`, and *arg* depends on the value of *what*: + + +-------------------------------+----------------------------------------+ + | Value of *what* | Meaning of *arg* | + +===============================+========================================+ + | :c:data:`PyTrace_CALL` | Always :c:data:`Py_None`. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_EXCEPTION` | Exception information as returned by | + | | :func:`sys.exc_info`. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_LINE` | Always :c:data:`Py_None`. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_RETURN` | Value being returned to the caller, | + | | or ``NULL`` if caused by an exception. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_C_CALL` | Function object being called. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_C_EXCEPTION` | Function object being called. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_C_RETURN` | Function object being called. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_OPCODE` | Always :c:data:`Py_None`. | + +-------------------------------+----------------------------------------+ .. c:var:: int PyTrace_CALL @@ -1499,8 +1499,8 @@ Python-level trace functions in previous versions. function as its first parameter, and may be any Python object, or ``NULL``. If the profile function needs to maintain state, using a different value for *obj* for each thread provides a convenient and thread-safe place to store it. The - profile function is called for all monitored events except :const:`PyTrace_LINE` - :const:`PyTrace_OPCODE` and :const:`PyTrace_EXCEPTION`. + profile function is called for all monitored events except :c:data:`PyTrace_LINE` + :c:data:`PyTrace_OPCODE` and :c:data:`PyTrace_EXCEPTION`. See also the :func:`sys.setprofile` function. @@ -1525,8 +1525,8 @@ Python-level trace functions in previous versions. :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number events and per-opcode events, but does not receive any event related to C function objects being called. Any trace function registered using :c:func:`PyEval_SetTrace` - will not receive :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or - :const:`PyTrace_C_RETURN` as a value for the *what* parameter. + will not receive :c:data:`PyTrace_C_CALL`, :c:data:`PyTrace_C_EXCEPTION` or + :c:data:`PyTrace_C_RETURN` as a value for the *what* parameter. See also the :func:`sys.settrace` function. diff --git a/Doc/c-api/long.rst b/Doc/c-api/long.rst index fe379ffe91239..f1354a34f2b2d 100644 --- a/Doc/c-api/long.rst +++ b/Doc/c-api/long.rst @@ -142,8 +142,8 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate. instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. - If the value of *obj* is greater than :const:`LONG_MAX` or less than - :const:`LONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and + If the value of *obj* is greater than :c:macro:`LONG_MAX` or less than + :c:macro:`LONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. @@ -183,8 +183,8 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate. instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. - If the value of *obj* is greater than :const:`LLONG_MAX` or less than - :const:`LLONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, + If the value of *obj* is greater than :c:macro:`LLONG_MAX` or less than + :c:macro:`LLONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 7041c15d23fb8..35c356f25c6c7 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -470,7 +470,7 @@ Customize Memory Allocators The new allocator must return a distinct non-``NULL`` pointer when requesting zero bytes. - For the :c:data:`PYMEM_DOMAIN_RAW` domain, the allocator must be + For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be thread-safe: the :term:`GIL ` is not held when the allocator is called. @@ -536,8 +536,8 @@ Runtime checks: - Detect write before the start of the buffer (buffer underflow). - Detect write after the end of the buffer (buffer overflow). - Check that the :term:`GIL ` is held when - allocator functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: - :c:func:`PyObject_Malloc`) and :c:data:`PYMEM_DOMAIN_MEM` (ex: + allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: + :c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. On error, the debug hooks use the :mod:`tracemalloc` module to get the @@ -557,9 +557,9 @@ that the treatment of negative indices differs from a Python slice): ``p[-S]`` API identifier (ASCII character): - * ``'r'`` for :c:data:`PYMEM_DOMAIN_RAW`. - * ``'m'`` for :c:data:`PYMEM_DOMAIN_MEM`. - * ``'o'`` for :c:data:`PYMEM_DOMAIN_OBJ`. + * ``'r'`` for :c:macro:`PYMEM_DOMAIN_RAW`. + * ``'m'`` for :c:macro:`PYMEM_DOMAIN_MEM`. + * ``'o'`` for :c:macro:`PYMEM_DOMAIN_OBJ`. ``p[-S+1:0]`` Copies of PYMEM_FORBIDDENBYTE. Used to catch under- writes and reads. @@ -601,7 +601,7 @@ PYMEM_CLEANBYTE (meaning uninitialized memory is getting used). compiled in release mode. On error, the debug hooks now use :mod:`tracemalloc` to get the traceback where a memory block was allocated. The debug hooks now also check if the GIL is held when functions of - :c:data:`PYMEM_DOMAIN_OBJ` and :c:data:`PYMEM_DOMAIN_MEM` domains are + :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are called. .. versionchanged:: 3.8 @@ -622,8 +622,8 @@ with a fixed size of 256 KiB. It falls back to :c:func:`PyMem_RawMalloc` and :c:func:`PyMem_RawRealloc` for allocations larger than 512 bytes. *pymalloc* is the :ref:`default allocator ` of the -:c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) and -:c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) domains. +:c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) and +:c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) domains. The arena allocator uses the following functions: diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index d35b302fce6aa..a4f79f8989e96 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -145,7 +145,7 @@ or request "multi-phase initialization" by returning the definition struct itsel .. c:member:: PyModuleDef_Base m_base - Always initialize this member to :const:`PyModuleDef_HEAD_INIT`. + Always initialize this member to :c:data:`PyModuleDef_HEAD_INIT`. .. c:member:: const char *m_name @@ -256,7 +256,7 @@ of the following two module creation functions: Create a new module object, given the definition in *def*. This behaves like :c:func:`PyModule_Create2` with *module_api_version* set to - :const:`PYTHON_API_VERSION`. + :c:macro:`PYTHON_API_VERSION`. .. c:function:: PyObject* PyModule_Create2(PyModuleDef *def, int module_api_version) @@ -390,7 +390,7 @@ objects dynamically. Note that both ``PyModule_FromDefAndSpec`` and Create a new module object, given the definition in *def* and the ModuleSpec *spec*. This behaves like :c:func:`PyModule_FromDefAndSpec2` - with *module_api_version* set to :const:`PYTHON_API_VERSION`. + with *module_api_version* set to :c:macro:`PYTHON_API_VERSION`. .. versionadded:: 3.5 diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 6fc5b2d14dd32..284f75470d524 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -23,7 +23,7 @@ Object Protocol Print an object *o*, on file *fp*. Returns ``-1`` on error. The flags argument is used to enable certain printing options. The only option currently supported - is :const:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written + is :c:macro:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written instead of the :func:`repr`. @@ -199,8 +199,8 @@ Object Protocol .. c:function:: PyObject* PyObject_RichCompare(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, - which must be one of :const:`Py_LT`, :const:`Py_LE`, :const:`Py_EQ`, - :const:`Py_NE`, :const:`Py_GT`, or :const:`Py_GE`, corresponding to ``<``, + which must be one of :c:macro:`Py_LT`, :c:macro:`Py_LE`, :c:macro:`Py_EQ`, + :c:macro:`Py_NE`, :c:macro:`Py_GT`, or :c:macro:`Py_GE`, corresponding to ``<``, ``<=``, ``==``, ``!=``, ``>``, or ``>=`` respectively. This is the equivalent of the Python expression ``o1 op o2``, where ``op`` is the operator corresponding to *opid*. Returns the value of the comparison on success, or ``NULL`` on failure. @@ -209,8 +209,8 @@ Object Protocol .. c:function:: int PyObject_RichCompareBool(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, - which must be one of :const:`Py_LT`, :const:`Py_LE`, :const:`Py_EQ`, - :const:`Py_NE`, :const:`Py_GT`, or :const:`Py_GE`, corresponding to ``<``, + which must be one of :c:macro:`Py_LT`, :c:macro:`Py_LE`, :c:macro:`Py_EQ`, + :c:macro:`Py_NE`, :c:macro:`Py_GT`, or :c:macro:`Py_GE`, corresponding to ``<``, ``<=``, ``==``, ``!=``, ``>``, or ``>=`` respectively. Returns ``-1`` on error, ``0`` if the result is false, ``1`` otherwise. This is the equivalent of the Python expression ``o1 op o2``, where ``op`` is the operator corresponding to @@ -218,7 +218,7 @@ Object Protocol .. note:: If *o1* and *o2* are the same object, :c:func:`PyObject_RichCompareBool` - will always return ``1`` for :const:`Py_EQ` and ``0`` for :const:`Py_NE`. + will always return ``1`` for :c:macro:`Py_EQ` and ``0`` for :c:macro:`Py_NE`. .. c:function:: PyObject* PyObject_Format(PyObject *obj, PyObject *format_spec) @@ -468,10 +468,10 @@ Object Protocol .. c:function:: void *PyObject_GetItemData(PyObject *o) Get a pointer to per-item data for a class with - :const:`Py_TPFLAGS_ITEMS_AT_END`. + :c:macro:`Py_TPFLAGS_ITEMS_AT_END`. On error, set an exception and return ``NULL``. :py:exc:`TypeError` is raised if *o* does not have - :const:`Py_TPFLAGS_ITEMS_AT_END` set. + :c:macro:`Py_TPFLAGS_ITEMS_AT_END` set. .. versionadded:: 3.12 diff --git a/Doc/c-api/slice.rst b/Doc/c-api/slice.rst index c54a659cf2ffd..9e880c6b7f25a 100644 --- a/Doc/c-api/slice.rst +++ b/Doc/c-api/slice.rst @@ -34,7 +34,7 @@ Slice Objects *length* as errors. Returns ``0`` on success and ``-1`` on error with no exception set (unless one of - the indices was not :const:`None` and failed to be converted to an integer, + the indices was not ``None`` and failed to be converted to an integer, in which case ``-1`` is returned with an exception set). You probably do not want to use this function. diff --git a/Doc/c-api/stable.rst b/Doc/c-api/stable.rst index 149d4d6bac3ee..c66b296d304ad 100644 --- a/Doc/c-api/stable.rst +++ b/Doc/c-api/stable.rst @@ -74,7 +74,7 @@ Contents of the Limited API are :ref:`listed below `. Define this macro before including ``Python.h`` to opt in to only use the Limited API, and to select the Limited API version. - Define ``Py_LIMITED_API`` to the value of :c:data:`PY_VERSION_HEX` + Define ``Py_LIMITED_API`` to the value of :c:macro:`PY_VERSION_HEX` corresponding to the lowest Python version your extension supports. The extension will work without recompilation with all Python 3 releases from the specified one onward, and can use Limited API introduced up to that diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 766f881463c00..720ab31f3de85 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -179,7 +179,7 @@ Implementing functions and methods .. c:type:: PyCFunctionWithKeywords Type of the functions used to implement Python callables in C - with signature :const:`METH_VARARGS | METH_KEYWORDS`. + with signature :ref:`METH_VARARGS | METH_KEYWORDS `. The function signature is:: PyObject *PyCFunctionWithKeywords(PyObject *self, @@ -190,7 +190,7 @@ Implementing functions and methods .. c:type:: _PyCFunctionFast Type of the functions used to implement Python callables in C - with signature :const:`METH_FASTCALL`. + with signature :c:macro:`METH_FASTCALL`. The function signature is:: PyObject *_PyCFunctionFast(PyObject *self, @@ -200,7 +200,7 @@ Implementing functions and methods .. c:type:: _PyCFunctionFastWithKeywords Type of the functions used to implement Python callables in C - with signature :const:`METH_FASTCALL | METH_KEYWORDS`. + with signature :ref:`METH_FASTCALL | METH_KEYWORDS `. The function signature is:: PyObject *_PyCFunctionFastWithKeywords(PyObject *self, @@ -211,7 +211,7 @@ Implementing functions and methods .. c:type:: PyCMethod Type of the functions used to implement Python callables in C - with signature :const:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS`. + with signature :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. The function signature is:: PyObject *PyCMethod(PyObject *self, @@ -257,7 +257,7 @@ convention. There are these calling conventions: -.. data:: METH_VARARGS +.. c:macro:: METH_VARARGS This is the typical calling convention, where the methods have the type :c:type:`PyCFunction`. The function expects two :c:expr:`PyObject*` values. @@ -267,8 +267,17 @@ There are these calling conventions: using :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_UnpackTuple`. -.. data:: METH_VARARGS | METH_KEYWORDS +.. c:macro:: METH_KEYWORDS + Can only be used in certain combinations with other flags: + :ref:`METH_VARARGS | METH_KEYWORDS `, + :ref:`METH_FASTCALL | METH_KEYWORDS ` and + :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. + + +.. _METH_VARARGS-METH_KEYWORDS: + +:c:expr:`METH_VARARGS | METH_KEYWORDS` Methods with these flags must be of type :c:type:`PyCFunctionWithKeywords`. The function expects three parameters: *self*, *args*, *kwargs* where *kwargs* is a dictionary of all the keyword arguments or possibly ``NULL`` @@ -276,7 +285,7 @@ There are these calling conventions: using :c:func:`PyArg_ParseTupleAndKeywords`. -.. data:: METH_FASTCALL +.. c:macro:: METH_FASTCALL Fast calling convention supporting only positional arguments. The methods have the type :c:type:`_PyCFunctionFast`. @@ -291,9 +300,10 @@ There are these calling conventions: ``METH_FASTCALL`` is now part of the :ref:`stable ABI `. -.. data:: METH_FASTCALL | METH_KEYWORDS +.. _METH_FASTCALL-METH_KEYWORDS: - Extension of :const:`METH_FASTCALL` supporting also keyword arguments, +:c:expr:`METH_FASTCALL | METH_KEYWORDS` + Extension of :c:macro:`METH_FASTCALL` supporting also keyword arguments, with methods of type :c:type:`_PyCFunctionFastWithKeywords`. Keyword arguments are passed the same way as in the :ref:`vectorcall protocol `: @@ -306,10 +316,18 @@ There are these calling conventions: .. versionadded:: 3.7 -.. data:: METH_METHOD | METH_FASTCALL | METH_KEYWORDS +.. c:macro:: METH_METHOD + + Can only be used in the combination with other flags: + :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. + + +.. _METH_METHOD-METH_FASTCALL-METH_KEYWORDS: - Extension of :const:`METH_FASTCALL | METH_KEYWORDS` supporting the *defining - class*, that is, the class that contains the method in question. +:c:expr:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS` + Extension of :ref:`METH_FASTCALL | METH_KEYWORDS ` + supporting the *defining class*, that is, + the class that contains the method in question. The defining class might be a superclass of ``Py_TYPE(self)``. The method needs to be of type :c:type:`PyCMethod`, the same as for @@ -319,10 +337,10 @@ There are these calling conventions: .. versionadded:: 3.9 -.. data:: METH_NOARGS +.. c:macro:: METH_NOARGS Methods without parameters don't need to check whether arguments are given if - they are listed with the :const:`METH_NOARGS` flag. They need to be of type + they are listed with the :c:macro:`METH_NOARGS` flag. They need to be of type :c:type:`PyCFunction`. The first parameter is typically named *self* and will hold a reference to the module or object instance. In all cases the second parameter will be ``NULL``. @@ -331,9 +349,9 @@ There are these calling conventions: :c:macro:`Py_UNUSED` can be used to prevent a compiler warning. -.. data:: METH_O +.. c:macro:: METH_O - Methods with a single object argument can be listed with the :const:`METH_O` + Methods with a single object argument can be listed with the :c:macro:`METH_O` flag, instead of invoking :c:func:`PyArg_ParseTuple` with a ``"O"`` argument. They have the type :c:type:`PyCFunction`, with the *self* parameter, and a :c:expr:`PyObject*` parameter representing the single argument. @@ -345,7 +363,7 @@ defined for modules. At most one of these flags may be set for any given method. -.. data:: METH_CLASS +.. c:macro:: METH_CLASS .. index:: pair: built-in function; classmethod @@ -355,7 +373,7 @@ method. function. -.. data:: METH_STATIC +.. c:macro:: METH_STATIC .. index:: pair: built-in function; staticmethod @@ -367,7 +385,7 @@ One other constant controls whether a method is loaded in place of another definition with the same method name. -.. data:: METH_COEXIST +.. c:macro:: METH_COEXIST The method will be loaded in place of existing definitions. Without *METH_COEXIST*, the default is to skip repeated definitions. Since slot @@ -440,8 +458,8 @@ Accessing attributes of extension types The legacy offsets :c:member:`~PyTypeObject.tp_dictoffset` and :c:member:`~PyTypeObject.tp_weaklistoffset` can be defined similarly using ``"__dictoffset__"`` and ``"__weaklistoffset__"`` members, but extensions - are strongly encouraged to use :const:`Py_TPFLAGS_MANAGED_DICT` and - :const:`Py_TPFLAGS_MANAGED_WEAKREF` instead. + are strongly encouraged to use :c:macro:`Py_TPFLAGS_MANAGED_DICT` and + :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead. .. versionchanged:: 3.12 @@ -509,19 +527,19 @@ The following flags can be used with :c:member:`PyMemberDef.flags`: .. versionchanged:: 3.10 - The :const:`!RESTRICTED`, :const:`!READ_RESTRICTED` and - :const:`!WRITE_RESTRICTED` macros available with + The :c:macro:`!RESTRICTED`, :c:macro:`!READ_RESTRICTED` and + :c:macro:`!WRITE_RESTRICTED` macros available with ``#include "structmember.h"`` are deprecated. - :const:`!READ_RESTRICTED` and :const:`!RESTRICTED` are equivalent to - :const:`Py_AUDIT_READ`; :const:`!WRITE_RESTRICTED` does nothing. + :c:macro:`!READ_RESTRICTED` and :c:macro:`!RESTRICTED` are equivalent to + :c:macro:`Py_AUDIT_READ`; :c:macro:`!WRITE_RESTRICTED` does nothing. .. index:: single: READONLY .. versionchanged:: 3.12 - The :const:`!READONLY` macro was renamed to :const:`Py_READONLY`. - The :const:`!PY_AUDIT_READ` macro was renamed with the ``Py_`` prefix. + The :c:macro:`!READONLY` macro was renamed to :c:macro:`Py_READONLY`. + The :c:macro:`!PY_AUDIT_READ` macro was renamed with the ``Py_`` prefix. The new names are now always available. Previously, these required ``#include "structmember.h"``. The header is still available and it provides the old names. diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index c4077b2a5620d..433d69e40f947 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -97,9 +97,9 @@ Operating System Utilities .. c:function:: int PyOS_CheckStack() Return true when the interpreter runs out of stack space. This is a reliable - check, but is only available when :const:`USE_STACKCHECK` is defined (currently + check, but is only available when :c:macro:`USE_STACKCHECK` is defined (currently on certain versions of Windows using the Microsoft Visual C++ compiler). - :const:`USE_STACKCHECK` will be defined automatically; you should never + :c:macro:`USE_STACKCHECK` will be defined automatically; you should never change the definition in your own code. diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index a5f333e2a31e0..d9dcd22a24839 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -132,7 +132,7 @@ Type Objects .. c:function:: int PyType_IS_GC(PyTypeObject *o) Return true if the type object includes support for the cycle detector; this - tests the type flag :const:`Py_TPFLAGS_HAVE_GC`. + tests the type flag :c:macro:`Py_TPFLAGS_HAVE_GC`. .. c:function:: int PyType_IsSubtype(PyTypeObject *a, PyTypeObject *b) @@ -165,10 +165,10 @@ Type Objects .. note:: If some of the base classes implements the GC protocol and the provided - type does not include the :const:`Py_TPFLAGS_HAVE_GC` in its flags, then + type does not include the :c:macro:`Py_TPFLAGS_HAVE_GC` in its flags, then the GC protocol will be automatically implemented from its parents. On the contrary, if the type being created does include - :const:`Py_TPFLAGS_HAVE_GC` in its flags then it **must** implement the + :c:macro:`Py_TPFLAGS_HAVE_GC` in its flags then it **must** implement the GC protocol itself by at least implementing the :c:member:`~PyTypeObject.tp_traverse` handle. @@ -268,7 +268,7 @@ The following functions and structs are used to create .. c:function:: PyObject* PyType_FromMetaclass(PyTypeObject *metaclass, PyObject *module, PyType_Spec *spec, PyObject *bases) Create and return a :ref:`heap type ` from the *spec* - (see :const:`Py_TPFLAGS_HEAPTYPE`). + (see :c:macro:`Py_TPFLAGS_HEAPTYPE`). The metaclass *metaclass* is used to construct the resulting type object. When *metaclass* is ``NULL``, the metaclass is derived from *bases* @@ -420,7 +420,7 @@ The following functions and structs are used to create - The requested :c:member:`PyType_Spec.basicsize` is zero, suggesting that the subclass does not access the instance's memory directly. - - With the :const:`Py_TPFLAGS_ITEMS_AT_END` flag. + - With the :c:macro:`Py_TPFLAGS_ITEMS_AT_END` flag. .. c:member:: unsigned int flags @@ -471,9 +471,9 @@ The following functions and structs are used to create * :c:member:`~PyTypeObject.tp_weaklist` * :c:member:`~PyTypeObject.tp_vectorcall` * :c:member:`~PyTypeObject.tp_weaklistoffset` - (use :const:`Py_TPFLAGS_MANAGED_WEAKREF` instead) + (use :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead) * :c:member:`~PyTypeObject.tp_dictoffset` - (use :const:`Py_TPFLAGS_MANAGED_DICT` instead) + (use :c:macro:`Py_TPFLAGS_MANAGED_DICT` instead) * :c:member:`~PyTypeObject.tp_vectorcall_offset` (see :ref:`PyMemberDef `) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 7249cfe79c32e..33591ed14eaf1 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -669,7 +669,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) memory buffers owned by the instance (using the freeing function corresponding to the allocation function used to allocate the buffer), and call the type's :c:member:`~PyTypeObject.tp_free` function. If the type is not subtypable - (doesn't have the :const:`Py_TPFLAGS_BASETYPE` flag bit set), it is + (doesn't have the :c:macro:`Py_TPFLAGS_BASETYPE` flag bit set), it is permissible to call the object deallocator directly instead of via :c:member:`~PyTypeObject.tp_free`. The object deallocator should be the one used to allocate the instance; this is normally :c:func:`PyObject_Del` if the instance was allocated @@ -677,7 +677,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:func:`PyObject_GC_Del` if the instance was allocated using :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar`. - If the type supports garbage collection (has the :const:`Py_TPFLAGS_HAVE_GC` + If the type supports garbage collection (has the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit set), the destructor should call :c:func:`PyObject_GC_UnTrack` before clearing any member fields. @@ -689,7 +689,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) Py_TYPE(self)->tp_free((PyObject *)self); } - Finally, if the type is heap allocated (:const:`Py_TPFLAGS_HEAPTYPE`), the + Finally, if the type is heap allocated (:c:macro:`Py_TPFLAGS_HEAPTYPE`), the deallocator should decrement the reference count for its type object after calling the type deallocator. In order to avoid dangling pointers, the recommended way to achieve this is: @@ -716,12 +716,12 @@ and :c:type:`PyType_Type` effectively act as defaults.) a more efficient alternative of the simpler :c:member:`~PyTypeObject.tp_call`. - This field is only used if the flag :const:`Py_TPFLAGS_HAVE_VECTORCALL` + This field is only used if the flag :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` is set. If so, this must be a positive integer containing the offset in the instance of a :c:type:`vectorcallfunc` pointer. The *vectorcallfunc* pointer may be ``NULL``, in which case the instance behaves - as if :const:`Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance + as if :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance falls back to :c:member:`~PyTypeObject.tp_call`. Any class that sets ``Py_TPFLAGS_HAVE_VECTORCALL`` must also set @@ -743,12 +743,12 @@ and :c:type:`PyType_Type` effectively act as defaults.) When a user sets :attr:`~type.__call__` in Python code, only *tp_call* is updated, likely making it inconsistent with the vectorcall function. Since 3.12, setting ``__call__`` will disable vectorcall optimization - by clearing the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag. + by clearing the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag. **Inheritance:** This field is always inherited. - However, the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is not + However, the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is not always inherited. If it's not set, then the subclass won't use :ref:`vectorcall `, except when :c:func:`PyVectorcall_Call` is explicitly called. @@ -1022,9 +1022,9 @@ and :c:type:`PyType_Type` effectively act as defaults.) this flag bit. The flag bits that pertain to extension structures are strictly inherited if the extension structure is inherited, i.e. the base type's value of the flag bit is copied into the subtype together with a pointer to the extension - structure. The :const:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with + structure. The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the - :const:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have ``NULL`` values. .. XXX are most flag bits *really* inherited individually? @@ -1036,12 +1036,14 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Bit Masks:** + .. c:namespace:: NULL + The following bit masks are currently defined; these can be ORed together using the ``|`` operator to form the value of the :c:member:`~PyTypeObject.tp_flags` field. The macro :c:func:`PyType_HasFeature` takes a type and a flags value, *tp* and *f*, and checks whether ``tp->tp_flags & f`` is non-zero. - .. data:: Py_TPFLAGS_HEAPTYPE + .. c:macro:: Py_TPFLAGS_HEAPTYPE This bit is set when the type object itself is allocated on the heap, for example, types created dynamically using :c:func:`PyType_FromSpec`. In this @@ -1056,7 +1058,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_BASETYPE + .. c:macro:: Py_TPFLAGS_BASETYPE This bit is set when the type can be used as the base type of another type. If this bit is clear, the type cannot be subtyped (similar to a "final" class in @@ -1067,7 +1069,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_READY + .. c:macro:: Py_TPFLAGS_READY This bit is set when the type object has been fully initialized by :c:func:`PyType_Ready`. @@ -1077,7 +1079,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_READYING + .. c:macro:: Py_TPFLAGS_READYING This bit is set while :c:func:`PyType_Ready` is in the process of initializing the type object. @@ -1087,7 +1089,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_HAVE_GC + .. c:macro:: Py_TPFLAGS_HAVE_GC This bit is set when the object supports garbage collection. If this bit is set, instances must be created using :c:func:`PyObject_GC_New` and @@ -1098,28 +1100,28 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :const:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` - The :const:`Py_TPFLAGS_HAVE_GC` flag bit is inherited + The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with the :attr:`tp_traverse` and :attr:`tp_clear` - fields, i.e. if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is + fields, i.e. if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the :attr:`tp_traverse` and :attr:`tp_clear` fields in the subtype exist and have ``NULL`` values. - .. data:: Py_TPFLAGS_DEFAULT + .. c:macro:: Py_TPFLAGS_DEFAULT This is a bitmask of all the bits that pertain to the existence of certain fields in the type object and its extension structures. Currently, it includes - the following bits: :const:`Py_TPFLAGS_HAVE_STACKLESS_EXTENSION`. + the following bits: :c:macro:`Py_TPFLAGS_HAVE_STACKLESS_EXTENSION`. **Inheritance:** ??? - .. data:: Py_TPFLAGS_METHOD_DESCRIPTOR + .. c:macro:: Py_TPFLAGS_METHOD_DESCRIPTOR This bit indicates that objects behave like unbound methods. @@ -1140,15 +1142,15 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** This flag is never inherited by types without the - :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is + :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is inherited whenever :c:member:`~PyTypeObject.tp_descr_get` is inherited. - .. data:: Py_TPFLAGS_MANAGED_DICT + .. c:macro:: Py_TPFLAGS_MANAGED_DICT This bit indicates that instances of the class have a ``__dict__`` attribute, and that the space for the dictionary is managed by the VM. - If this flag is set, :const:`Py_TPFLAGS_HAVE_GC` should also be set. + If this flag is set, :c:macro:`Py_TPFLAGS_HAVE_GC` should also be set. .. versionadded:: 3.12 @@ -1158,7 +1160,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:member:`~PyTypeObject.tp_dictoffset` field is set in a superclass. - .. data:: Py_TPFLAGS_MANAGED_WEAKREF + .. c:macro:: Py_TPFLAGS_MANAGED_WEAKREF This bit indicates that instances of the class should be weakly referenceable. @@ -1171,7 +1173,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:member:`~PyTypeObject.tp_weaklistoffset` field is set in a superclass. - .. data:: Py_TPFLAGS_ITEMS_AT_END + .. c:macro:: Py_TPFLAGS_ITEMS_AT_END Only usable with variable-size types, i.e. ones with non-zero :c:member:`~PyObject.tp_itemsize`. @@ -1194,14 +1196,14 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. XXX Document more flags here? - .. data:: Py_TPFLAGS_LONG_SUBCLASS - .. data:: Py_TPFLAGS_LIST_SUBCLASS - .. data:: Py_TPFLAGS_TUPLE_SUBCLASS - .. data:: Py_TPFLAGS_BYTES_SUBCLASS - .. data:: Py_TPFLAGS_UNICODE_SUBCLASS - .. data:: Py_TPFLAGS_DICT_SUBCLASS - .. data:: Py_TPFLAGS_BASE_EXC_SUBCLASS - .. data:: Py_TPFLAGS_TYPE_SUBCLASS + .. c:macro:: Py_TPFLAGS_LONG_SUBCLASS + .. c:macro:: Py_TPFLAGS_LIST_SUBCLASS + .. c:macro:: Py_TPFLAGS_TUPLE_SUBCLASS + .. c:macro:: Py_TPFLAGS_BYTES_SUBCLASS + .. c:macro:: Py_TPFLAGS_UNICODE_SUBCLASS + .. c:macro:: Py_TPFLAGS_DICT_SUBCLASS + .. c:macro:: Py_TPFLAGS_BASE_EXC_SUBCLASS + .. c:macro:: Py_TPFLAGS_TYPE_SUBCLASS These flags are used by functions such as :c:func:`PyLong_Check` to quickly determine if a type is a subclass @@ -1212,7 +1214,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) will behave differently depending on what kind of check is used. - .. data:: Py_TPFLAGS_HAVE_FINALIZE + .. c:macro:: Py_TPFLAGS_HAVE_FINALIZE This bit is set when the :c:member:`~PyTypeObject.tp_finalize` slot is present in the type structure. @@ -1225,7 +1227,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) type structure. - .. data:: Py_TPFLAGS_HAVE_VECTORCALL + .. c:macro:: Py_TPFLAGS_HAVE_VECTORCALL This bit is set when the class implements the :ref:`vectorcall protocol `. @@ -1245,7 +1247,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) This flag can now be inherited by mutable classes. - .. data:: Py_TPFLAGS_IMMUTABLETYPE + .. c:macro:: Py_TPFLAGS_IMMUTABLETYPE This bit is set for type objects that are immutable: type attributes cannot be set nor deleted. @@ -1258,7 +1260,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_DISALLOW_INSTANTIATION + .. c:macro:: Py_TPFLAGS_DISALLOW_INSTANTIATION Disallow creating instances of the type: set :c:member:`~PyTypeObject.tp_new` to NULL and don't create the ``__new__`` @@ -1289,7 +1291,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_MAPPING + .. c:macro:: Py_TPFLAGS_MAPPING This bit indicates that instances of the class may match mapping patterns when used as the subject of a :keyword:`match` block. It is automatically @@ -1298,20 +1300,20 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: - :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are + :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` are mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** This flag is inherited by types that do not already set - :const:`Py_TPFLAGS_SEQUENCE`. + :c:macro:`Py_TPFLAGS_SEQUENCE`. .. seealso:: :pep:`634` -- Structural Pattern Matching: Specification .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_SEQUENCE + .. c:macro:: Py_TPFLAGS_SEQUENCE This bit indicates that instances of the class may match sequence patterns when used as the subject of a :keyword:`match` block. It is automatically @@ -1320,20 +1322,20 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: - :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are + :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` are mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** This flag is inherited by types that do not already set - :const:`Py_TPFLAGS_MAPPING`. + :c:macro:`Py_TPFLAGS_MAPPING`. .. seealso:: :pep:`634` -- Structural Pattern Matching: Specification .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_VALID_VERSION_TAG + .. c:macro:: Py_TPFLAGS_VALID_VERSION_TAG Internal. Do not set or unset this flag. To indicate that a class has changed call :c:func:`PyType_Modified` @@ -1357,7 +1359,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: traverseproc PyTypeObject.tp_traverse An optional pointer to a traversal function for the garbage collector. This is - only used if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: + only used if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: int tp_traverse(PyObject *self, visitproc visit, void *arg); @@ -1419,10 +1421,10 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :const:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_clear` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. @@ -1430,7 +1432,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: inquiry PyTypeObject.tp_clear An optional pointer to a clear function for the garbage collector. This is only - used if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: + used if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: int tp_clear(PyObject *); @@ -1486,10 +1488,10 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :const:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_traverse` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. @@ -1511,21 +1513,21 @@ and :c:type:`PyType_Type` effectively act as defaults.) The following constants are defined to be used as the third argument for :c:member:`~PyTypeObject.tp_richcompare` and for :c:func:`PyObject_RichCompare`: - +----------------+------------+ - | Constant | Comparison | - +================+============+ - | :const:`Py_LT` | ``<`` | - +----------------+------------+ - | :const:`Py_LE` | ``<=`` | - +----------------+------------+ - | :const:`Py_EQ` | ``==`` | - +----------------+------------+ - | :const:`Py_NE` | ``!=`` | - +----------------+------------+ - | :const:`Py_GT` | ``>`` | - +----------------+------------+ - | :const:`Py_GE` | ``>=`` | - +----------------+------------+ + +------------------+------------+ + | Constant | Comparison | + +==================+============+ + | :c:macro:`Py_LT` | ``<`` | + +------------------+------------+ + | :c:macro:`Py_LE` | ``<=`` | + +------------------+------------+ + | :c:macro:`Py_EQ` | ``==`` | + +------------------+------------+ + | :c:macro:`Py_NE` | ``!=`` | + +------------------+------------+ + | :c:macro:`Py_GT` | ``>`` | + +------------------+------------+ + | :c:macro:`Py_GE` | ``>=`` | + +------------------+------------+ The following macro is defined to ease writing rich comparison functions: @@ -1563,7 +1565,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: Py_ssize_t PyTypeObject.tp_weaklistoffset - While this field is still supported, :const:`Py_TPFLAGS_MANAGED_WEAKREF` + While this field is still supported, :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` should be used instead, if at all possible. If the instances of this type are weakly referenceable, this field is greater @@ -1576,7 +1578,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) Do not confuse this field with :c:member:`~PyTypeObject.tp_weaklist`; that is the list head for weak references to the type object itself. - It is an error to set both the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit and + It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit and :c:member:`~PyTypeObject.tp_weaklist`. **Inheritance:** @@ -1588,7 +1590,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - If the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the + If the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the :c:member:`~PyTypeObject.tp_dict` field, then :c:member:`~PyTypeObject.tp_weaklistoffset` will be set to a negative value, to indicate that it is unsafe to use this field. @@ -1782,7 +1784,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: Py_ssize_t PyTypeObject.tp_dictoffset - While this field is still supported, :const:`Py_TPFLAGS_MANAGED_DICT` should be + While this field is still supported, :c:macro:`Py_TPFLAGS_MANAGED_DICT` should be used instead, if at all possible. If the instances of this type have a dictionary containing instance variables, @@ -1801,7 +1803,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr` when accessing an attribute on the object. - It is an error to set both the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit and + It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit and :c:member:`~PyTypeObject.tp_dictoffset`. **Inheritance:** @@ -1809,14 +1811,14 @@ and :c:type:`PyType_Type` effectively act as defaults.) This field is inherited by subtypes. A subtype should not override this offset; doing so could be unsafe, if C code tries to access the dictionary at the previous offset. - To properly support inheritance, use :const:`Py_TPFLAGS_MANAGED_DICT`. + To properly support inheritance, use :c:macro:`Py_TPFLAGS_MANAGED_DICT`. **Default:** This slot has no default. For :ref:`static types `, if the field is ``NULL`` then no :attr:`__dict__` gets created for instances. - If the :const:`Py_TPFLAGS_MANAGED_DICT` bit is set in the + If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the :c:member:`~PyTypeObject.tp_dict` field, then :c:member:`~PyTypeObject.tp_dictoffset` will be set to ``-1``, to indicate that it is unsafe to use this field. @@ -1903,7 +1905,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) in :c:member:`~PyTypeObject.tp_new`, while for mutable types, most initialization should be deferred to :c:member:`~PyTypeObject.tp_init`. - Set the :const:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag to disallow creating + Set the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag to disallow creating instances of the type in Python. **Inheritance:** @@ -1937,7 +1939,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) In dynamic subtypes, this field is set to a deallocator suitable to match :c:func:`PyType_GenericAlloc` and the value of the - :const:`Py_TPFLAGS_HAVE_GC` flag bit. + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. For static subtypes, :c:type:`PyBaseObject_Type` uses PyObject_Del. @@ -1948,7 +1950,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) The garbage collector needs to know whether a particular object is collectible or not. Normally, it is sufficient to look at the object's type's - :c:member:`~PyTypeObject.tp_flags` field, and check the :const:`Py_TPFLAGS_HAVE_GC` flag bit. But + :c:member:`~PyTypeObject.tp_flags` field, and check the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. But some types have a mixture of statically and dynamically allocated instances, and the statically allocated instances are not collectible. Such types should define this function; it should return ``1`` for a collectible instance, and @@ -1967,7 +1969,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** This slot has no default. If this field is ``NULL``, - :const:`Py_TPFLAGS_HAVE_GC` is used as the functional equivalent. + :c:macro:`Py_TPFLAGS_HAVE_GC` is used as the functional equivalent. .. c:member:: PyObject* PyTypeObject.tp_bases @@ -2114,7 +2116,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionchanged:: 3.8 Before version 3.8 it was necessary to set the - :const:`Py_TPFLAGS_HAVE_FINALIZE` flags bit in order for this field to be + :c:macro:`Py_TPFLAGS_HAVE_FINALIZE` flags bit in order for this field to be used. This is no longer required. .. seealso:: "Safe object finalization" (:pep:`442`) @@ -2173,7 +2175,7 @@ Heap Types An alternative to :ref:`static types ` is *heap-allocated types*, or *heap types* for short, which correspond closely to classes created by -Python's ``class`` statement. Heap types have the :const:`Py_TPFLAGS_HEAPTYPE` +Python's ``class`` statement. Heap types have the :c:macro:`Py_TPFLAGS_HEAPTYPE` flag set. This is done by filling a :c:type:`PyType_Spec` structure and calling @@ -2781,7 +2783,7 @@ A type that supports weakrefs, instance dicts, and hashing:: A str subclass that cannot be subclassed and cannot be called to create instances (e.g. uses a separate factory func) using -:c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag:: +:c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag:: typedef struct { PyUnicodeObject raw; diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 64dcea785d0c6..beb4f42a208fa 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1292,7 +1292,7 @@ the user settings on the machine running the codec. Encode the Unicode object using the specified code page and return a Python bytes object. Return ``NULL`` if an exception was raised by the codec. Use - :c:data:`CP_ACP` code page to get the MBCS encoder. + :c:macro:`CP_ACP` code page to get the MBCS encoder. .. versionadded:: 3.3 @@ -1411,11 +1411,11 @@ They all return ``NULL`` or ``-1`` if an exception occurs. Rich compare two Unicode strings and return one of the following: * ``NULL`` in case an exception was raised - * :const:`Py_True` or :const:`Py_False` for successful comparisons - * :const:`Py_NotImplemented` in case the type combination is unknown + * :c:data:`Py_True` or :c:data:`Py_False` for successful comparisons + * :c:data:`Py_NotImplemented` in case the type combination is unknown - Possible values for *op* are :const:`Py_GT`, :const:`Py_GE`, :const:`Py_EQ`, - :const:`Py_NE`, :const:`Py_LT`, and :const:`Py_LE`. + Possible values for *op* are :c:macro:`Py_GT`, :c:macro:`Py_GE`, :c:macro:`Py_EQ`, + :c:macro:`Py_NE`, :c:macro:`Py_LT`, and :c:macro:`Py_LE`. .. c:function:: PyObject* PyUnicode_Format(PyObject *format, PyObject *args) diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 000a2d3d8790b..1e8a945512d78 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -12,8 +12,8 @@ file or a buffer, but they will not let you interact in a more detailed way with the interpreter. Several of these functions accept a start symbol from the grammar as a -parameter. The available start symbols are :const:`Py_eval_input`, -:const:`Py_file_input`, and :const:`Py_single_input`. These are described +parameter. The available start symbols are :c:data:`Py_eval_input`, +:c:data:`Py_file_input`, and :c:data:`Py_single_input`. These are described following the functions which accept them as parameters. Note also that several of these functions take :c:expr:`FILE*` parameters. One @@ -256,8 +256,8 @@ the same library that the Python runtime is using. Parse and compile the Python source code in *str*, returning the resulting code object. The start token is given by *start*; this can be used to constrain the - code which can be compiled and should be :const:`Py_eval_input`, - :const:`Py_file_input`, or :const:`Py_single_input`. The filename specified by + code which can be compiled and should be :c:data:`Py_eval_input`, + :c:data:`Py_file_input`, or :c:data:`Py_single_input`. The filename specified by *filename* is used to construct the code object and may appear in tracebacks or :exc:`SyntaxError` exception messages. This returns ``NULL`` if the code cannot be parsed or compiled. diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 7d08bb9f6b8dd..0163ba1a5b313 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -337,7 +337,7 @@ When using only ``METH_VARARGS``, the function should expect the Python-level parameters to be passed in as a tuple acceptable for parsing via :c:func:`PyArg_ParseTuple`; more information on this function is provided below. -The :const:`METH_KEYWORDS` bit may be set in the third field if keyword +The :c:macro:`METH_KEYWORDS` bit may be set in the third field if keyword arguments should be passed to the function. In this case, the C function should accept a third ``PyObject *`` parameter which will be a dictionary of keywords. Use :c:func:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a @@ -540,7 +540,7 @@ be part of a module definition:: } This function must be registered with the interpreter using the -:const:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The +:c:macro:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The :c:func:`PyArg_ParseTuple` function and its arguments are documented in section :ref:`parsetuple`. diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index f89934a11f12a..ba09fb1b05c0b 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -151,7 +151,7 @@ only used for variable-sized objects and should otherwise be zero. base type will be :class:`object`, or else you will be adding data members to your base type, and therefore increasing its size. -We set the class flags to :const:`Py_TPFLAGS_DEFAULT`. :: +We set the class flags to :c:macro:`Py_TPFLAGS_DEFAULT`. :: .tp_flags = Py_TPFLAGS_DEFAULT, @@ -498,7 +498,7 @@ definitions:: {NULL} /* Sentinel */ }; -(note that we used the :const:`METH_NOARGS` flag to indicate that the method +(note that we used the :c:macro:`METH_NOARGS` flag to indicate that the method is expecting no arguments other than *self*) and assign it to the :c:member:`~PyTypeObject.tp_methods` slot:: @@ -508,7 +508,7 @@ and assign it to the :c:member:`~PyTypeObject.tp_methods` slot:: Finally, we'll make our type usable as a base class for subclassing. We've written our methods carefully so far so that they don't make any assumptions about the type of the object being created or used, so all we need to do is -to add the :const:`Py_TPFLAGS_BASETYPE` to our class flag definition:: +to add the :c:macro:`Py_TPFLAGS_BASETYPE` to our class flag definition:: .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, @@ -774,7 +774,7 @@ and ``Custom_clear``:: Py_TYPE(self)->tp_free((PyObject *) self); } -Finally, we add the :const:`Py_TPFLAGS_HAVE_GC` flag to the class flags:: +Finally, we add the :c:macro:`Py_TPFLAGS_HAVE_GC` flag to the class flags:: .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index 8adb85f3a8740..f01801ea47212 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -298,10 +298,10 @@ Watch out for the following two points in particular (but note that this is not a comprehensive list): * Unlike static types, heap type objects are mutable by default. - Use the :c:data:`Py_TPFLAGS_IMMUTABLETYPE` flag to prevent mutability. + Use the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag to prevent mutability. * Heap types inherit :c:member:`~PyTypeObject.tp_new` by default, so it may become possible to instantiate them from Python code. - You can prevent this with the :c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. + You can prevent this with the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. Defining Heap Types @@ -333,12 +333,12 @@ To avoid memory leaks, instances of heap types must implement the garbage collection protocol. That is, heap types should: -- Have the :c:data:`Py_TPFLAGS_HAVE_GC` flag. +- Have the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. - Define a traverse function using ``Py_tp_traverse``, which visits the type (e.g. using :c:expr:`Py_VISIT(Py_TYPE(self))`). Please refer to the :ref:`the documentation ` of -:c:data:`Py_TPFLAGS_HAVE_GC` and :c:member:`~PyTypeObject.tp_traverse` +:c:macro:`Py_TPFLAGS_HAVE_GC` and :c:member:`~PyTypeObject.tp_traverse` for additional considerations. If your traverse function delegates to the ``tp_traverse`` of its base class @@ -411,7 +411,7 @@ that subclass, which may be defined in different module than yours. pass For a method to get its "defining class", it must use the -:data:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS` +:ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS ` :c:type:`calling convention ` and the corresponding :c:type:`PyCMethod` signature:: diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 6beaad3825aba..accc652a90649 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -876,7 +876,7 @@ iterations of the loop. .. opcode:: MATCH_MAPPING If ``STACK[-1]`` is an instance of :class:`collections.abc.Mapping` (or, more - technically: if it has the :const:`Py_TPFLAGS_MAPPING` flag set in its + technically: if it has the :c:macro:`Py_TPFLAGS_MAPPING` flag set in its :c:member:`~PyTypeObject.tp_flags`), push ``True`` onto the stack. Otherwise, push ``False``. @@ -887,7 +887,7 @@ iterations of the loop. If ``STACK[-1]`` is an instance of :class:`collections.abc.Sequence` and is *not* an instance of :class:`str`/:class:`bytes`/:class:`bytearray` (or, more technically: if it has - the :const:`Py_TPFLAGS_SEQUENCE` flag set in its :c:member:`~PyTypeObject.tp_flags`), + the :c:macro:`Py_TPFLAGS_SEQUENCE` flag set in its :c:member:`~PyTypeObject.tp_flags`), push ``True`` onto the stack. Otherwise, push ``False``. .. versionadded:: 3.10 diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 83abb5d5ca1e4..b35010498f214 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -4308,7 +4308,7 @@ written in Python, such as a mail server's external command delivery program. specified. If the value specified is 0, the child's process group ID will be made the same as its process ID. If the value of *setpgroup* is not set, the child will inherit the parent's process group ID. This argument corresponds - to the C library :c:data:`POSIX_SPAWN_SETPGROUP` flag. + to the C library :c:macro:`POSIX_SPAWN_SETPGROUP` flag. If the *resetids* argument is ``True`` it will reset the effective UID and GID of the child to the real UID and GID of the parent process. If the @@ -4316,27 +4316,27 @@ written in Python, such as a mail server's external command delivery program. the parent. In either case, if the set-user-ID and set-group-ID permission bits are enabled on the executable file, their effect will override the setting of the effective UID and GID. This argument corresponds to the C - library :c:data:`POSIX_SPAWN_RESETIDS` flag. + library :c:macro:`POSIX_SPAWN_RESETIDS` flag. If the *setsid* argument is ``True``, it will create a new session ID - for ``posix_spawn``. *setsid* requires :c:data:`POSIX_SPAWN_SETSID` - or :c:data:`POSIX_SPAWN_SETSID_NP` flag. Otherwise, :exc:`NotImplementedError` + for ``posix_spawn``. *setsid* requires :c:macro:`POSIX_SPAWN_SETSID` + or :c:macro:`POSIX_SPAWN_SETSID_NP` flag. Otherwise, :exc:`NotImplementedError` is raised. The *setsigmask* argument will set the signal mask to the signal set specified. If the parameter is not used, then the child inherits the parent's signal mask. This argument corresponds to the C library - :c:data:`POSIX_SPAWN_SETSIGMASK` flag. + :c:macro:`POSIX_SPAWN_SETSIGMASK` flag. The *sigdef* argument will reset the disposition of all signals in the set specified. This argument corresponds to the C library - :c:data:`POSIX_SPAWN_SETSIGDEF` flag. + :c:macro:`POSIX_SPAWN_SETSIGDEF` flag. The *scheduler* argument must be a tuple containing the (optional) scheduler policy and an instance of :class:`sched_param` with the scheduler parameters. A value of ``None`` in the place of the scheduler policy indicates that is not being provided. This argument is a combination of the C library - :c:data:`POSIX_SPAWN_SETSCHEDPARAM` and :c:data:`POSIX_SPAWN_SETSCHEDULER` + :c:macro:`POSIX_SPAWN_SETSCHEDPARAM` and :c:macro:`POSIX_SPAWN_SETSCHEDULER` flags. .. audit-event:: os.posix_spawn path,argv,env os.posix_spawn diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 6d30eccab1990..12ad18d411961 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1840,7 +1840,7 @@ like ``TYPE_PARAMS_OF_ListOrSet`` are not actually bound at runtime. * a class that inherits from :class:`collections.abc.Sequence` * a Python class that has been registered as :class:`collections.abc.Sequence` - * a builtin class that has its (CPython) :data:`Py_TPFLAGS_SEQUENCE` bit set + * a builtin class that has its (CPython) :c:macro:`Py_TPFLAGS_SEQUENCE` bit set * a class that inherits from any of the above The following standard library classes are sequences: @@ -1859,7 +1859,7 @@ like ``TYPE_PARAMS_OF_ListOrSet`` are not actually bound at runtime. * a class that inherits from :class:`collections.abc.Mapping` * a Python class that has been registered as :class:`collections.abc.Mapping` - * a builtin class that has its (CPython) :data:`Py_TPFLAGS_MAPPING` bit set + * a builtin class that has its (CPython) :c:macro:`Py_TPFLAGS_MAPPING` bit set * a class that inherits from any of the above The standard library classes :class:`dict` and :class:`types.MappingProxyType` diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 1b470d395d6d5..4bf67eb439ec6 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -897,11 +897,11 @@ conflict. * ``default``: use the :ref:`default memory allocators `. * ``malloc``: use the :c:func:`malloc` function of the C library - for all domains (:c:data:`PYMEM_DOMAIN_RAW`, :c:data:`PYMEM_DOMAIN_MEM`, - :c:data:`PYMEM_DOMAIN_OBJ`). + for all domains (:c:macro:`PYMEM_DOMAIN_RAW`, :c:macro:`PYMEM_DOMAIN_MEM`, + :c:macro:`PYMEM_DOMAIN_OBJ`). * ``pymalloc``: use the :ref:`pymalloc allocator ` for - :c:data:`PYMEM_DOMAIN_MEM` and :c:data:`PYMEM_DOMAIN_OBJ` domains and use - the :c:func:`malloc` function for the :c:data:`PYMEM_DOMAIN_RAW` domain. + :c:macro:`PYMEM_DOMAIN_MEM` and :c:macro:`PYMEM_DOMAIN_OBJ` domains and use + the :c:func:`malloc` function for the :c:macro:`PYMEM_DOMAIN_RAW` domain. Install :ref:`debug hooks `: diff --git a/Doc/whatsnew/2.2.rst b/Doc/whatsnew/2.2.rst index 82aff0be1ed3b..44e9bd8d492bf 100644 --- a/Doc/whatsnew/2.2.rst +++ b/Doc/whatsnew/2.2.rst @@ -1105,11 +1105,11 @@ code, none of the changes described here will affect you very much. expected, and a set of pointers to :c:expr:`PyObject*` variables that will be filled in with argument values. -* Two new flags :const:`METH_NOARGS` and :const:`METH_O` are available in method +* Two new flags :c:macro:`METH_NOARGS` and :c:macro:`METH_O` are available in method definition tables to simplify implementation of methods with no arguments or a single untyped argument. Calling such methods is more efficient than calling a - corresponding method that uses :const:`METH_VARARGS`. Also, the old - :const:`METH_OLDARGS` style of writing C methods is now officially deprecated. + corresponding method that uses :c:macro:`METH_VARARGS`. Also, the old + :c:macro:`METH_OLDARGS` style of writing C methods is now officially deprecated. * Two new wrapper functions, :c:func:`PyOS_snprintf` and :c:func:`PyOS_vsnprintf` were added to provide cross-platform implementations for the relatively new diff --git a/Doc/whatsnew/2.3.rst b/Doc/whatsnew/2.3.rst index 43bf3fa46a29f..c1722e5ab56fc 100644 --- a/Doc/whatsnew/2.3.rst +++ b/Doc/whatsnew/2.3.rst @@ -1474,7 +1474,7 @@ complete list of changes, or look through the CVS logs for all the details. * On Windows, the :mod:`socket` module now ships with Secure Sockets Layer (SSL) support. -* The value of the C :const:`PYTHON_API_VERSION` macro is now exposed at the +* The value of the C :c:macro:`PYTHON_API_VERSION` macro is now exposed at the Python level as ``sys.api_version``. The current exception can be cleared by calling the new :func:`sys.exc_clear` function. @@ -1899,10 +1899,10 @@ Changes to Python's build process and to the C API include: * The :c:func:`PyArg_NoArgs` macro is now deprecated, and code that uses it should be changed. For Python 2.2 and later, the method definition table can - specify the :const:`METH_NOARGS` flag, signalling that there are no arguments, + specify the :c:macro:`METH_NOARGS` flag, signalling that there are no arguments, and the argument checking can then be removed. If compatibility with pre-2.2 versions of Python is important, the code could use ``PyArg_ParseTuple(args, - "")`` instead, but this will be slower than using :const:`METH_NOARGS`. + "")`` instead, but this will be slower than using :c:macro:`METH_NOARGS`. * :c:func:`PyArg_ParseTuple` accepts new format characters for various sizes of unsigned integers: ``B`` for :c:expr:`unsigned char`, ``H`` for :c:expr:`unsigned @@ -1918,7 +1918,7 @@ Changes to Python's build process and to the C API include: seconds, according to one measurement). * It's now possible to define class and static methods for a C extension type by - setting either the :const:`METH_CLASS` or :const:`METH_STATIC` flags in a + setting either the :c:macro:`METH_CLASS` or :c:macro:`METH_STATIC` flags in a method's :c:type:`PyMethodDef` structure. * Python now includes a copy of the Expat XML parser's source code, removing any diff --git a/Doc/whatsnew/2.4.rst b/Doc/whatsnew/2.4.rst index 43c3f01e5af89..ad46637b8497b 100644 --- a/Doc/whatsnew/2.4.rst +++ b/Doc/whatsnew/2.4.rst @@ -1476,7 +1476,7 @@ Some of the changes to Python's build process and to the C API are: :c:func:`PyArg_ParseTupleAndKeywords` but takes a :c:type:`va_list` instead of a number of arguments. (Contributed by Greg Chapman.) -* A new method flag, :const:`METH_COEXISTS`, allows a function defined in slots +* A new method flag, :c:macro:`METH_COEXISTS`, allows a function defined in slots to co-exist with a :c:type:`PyCFunction` having the same name. This can halve the access time for a method such as :meth:`set.__contains__`. (Contributed by Raymond Hettinger.) diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index 69170897ccc50..fb56690ce2511 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -1138,11 +1138,11 @@ indicate that the external caller is done. The *flags* argument to :c:func:`PyObject_GetBuffer` specifies constraints upon the memory returned. Some examples are: - * :const:`PyBUF_WRITABLE` indicates that the memory must be writable. + * :c:macro:`PyBUF_WRITABLE` indicates that the memory must be writable. - * :const:`PyBUF_LOCK` requests a read-only or exclusive lock on the memory. + * :c:macro:`PyBUF_LOCK` requests a read-only or exclusive lock on the memory. - * :const:`PyBUF_C_CONTIGUOUS` and :const:`PyBUF_F_CONTIGUOUS` + * :c:macro:`PyBUF_C_CONTIGUOUS` and :c:macro:`PyBUF_F_CONTIGUOUS` requests a C-contiguous (last dimension varies the fastest) or Fortran-contiguous (first dimension varies the fastest) array layout. diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index f8c7872d7d3f8..1cc6ebd0a6ef0 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -2231,7 +2231,7 @@ Changes to Python's build process and to the C API include: * When using the :c:type:`PyMemberDef` structure to define attributes of a type, Python will no longer let you try to delete or set a - :const:`T_STRING_INPLACE` attribute. + :c:macro:`T_STRING_INPLACE` attribute. .. rev 79644 diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index ab030db5b3ffa..b65e73ce353a5 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -2124,11 +2124,11 @@ New Features These functions allow to activate, deactivate and query the state of the garbage collector from C code without having to import the :mod:`gc` module. -* Add a new :c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow +* Add a new :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow creating type instances. (Contributed by Victor Stinner in :issue:`43916`.) -* Add a new :c:data:`Py_TPFLAGS_IMMUTABLETYPE` type flag for creating immutable +* Add a new :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` type flag for creating immutable type objects: type attributes cannot be set nor deleted. (Contributed by Victor Stinner and Erlend E. Aasland in :issue:`43908`.) @@ -2187,9 +2187,9 @@ Porting to Python 3.10 been included directly, consider including ``Python.h`` instead. (Contributed by Nicholas Sim in :issue:`35134`.) -* Use the :c:data:`Py_TPFLAGS_IMMUTABLETYPE` type flag to create immutable type - objects. Do not rely on :c:data:`Py_TPFLAGS_HEAPTYPE` to decide if a type - object is mutable or not; check if :c:data:`Py_TPFLAGS_IMMUTABLETYPE` is set +* Use the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` type flag to create immutable type + objects. Do not rely on :c:macro:`Py_TPFLAGS_HEAPTYPE` to decide if a type + object is mutable or not; check if :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` is set instead. (Contributed by Victor Stinner and Erlend E. Aasland in :issue:`43908`.) diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 45194130c993a..92e85fc6c1480 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -2347,11 +2347,11 @@ Porting to Python 3.11 #endif * The :c:func:`PyType_Ready` function now raises an error if a type is defined - with the :const:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function + with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function (:c:member:`PyTypeObject.tp_traverse`). (Contributed by Victor Stinner in :issue:`44263`.) -* Heap types with the :const:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit +* Heap types with the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit the :pep:`590` vectorcall protocol. Previously, this was only possible for :ref:`static types `. (Contributed by Erlend E. Aasland in :issue:`43908`) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index ab5e5b83e98c0..b3b9b4134a9bb 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1111,7 +1111,7 @@ Pending Removal in Python 3.14 * :class:`typing.ByteString`, deprecated since Python 3.9, now causes a :exc:`DeprecationWarning` to be emitted when it is used. -* Creating immutable types (:data:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable +* Creating immutable types (:c:macro:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable bases using the C API. * Deprecated the *isdst* parameter in :func:`email.utils.localtime`. @@ -1622,7 +1622,7 @@ New Features inheriting or extending the base class size. - :c:func:`PyObject_GetTypeData` and :c:func:`PyType_GetTypeDataSize` added to allow access to subclass-specific instance data. - - :const:`Py_TPFLAGS_ITEMS_AT_END` and :c:func:`PyObject_GetItemData` + - :c:macro:`Py_TPFLAGS_ITEMS_AT_END` and :c:func:`PyObject_GetItemData` added to allow safely extending certain variable-sized types, including :c:var:`PyType_Type`. - :c:macro:`Py_RELATIVE_OFFSET` added to allow defining @@ -1639,20 +1639,20 @@ New Features :ref:`the vectorcall protocol ` was added to the :ref:`Limited API `: - * :const:`Py_TPFLAGS_HAVE_VECTORCALL` + * :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` * :c:func:`PyVectorcall_NARGS` * :c:func:`PyVectorcall_Call` * :c:type:`vectorcallfunc` - The :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class + The :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class when the class's :py:meth:`~object.__call__` method is reassigned. This makes vectorcall safe to use with mutable types (i.e. heap types - without the immutable flag, :const:`Py_TPFLAGS_IMMUTABLETYPE`). + without the immutable flag, :c:macro:`Py_TPFLAGS_IMMUTABLETYPE`). Mutable types that do not override :c:member:`~PyTypeObject.tp_call` now inherit the ``Py_TPFLAGS_HAVE_VECTORCALL`` flag. (Contributed by Petr Viktorin in :gh:`93274`.) - The :const:`Py_TPFLAGS_MANAGED_DICT` and :const:`Py_TPFLAGS_MANAGED_WEAKREF` + The :c:macro:`Py_TPFLAGS_MANAGED_DICT` and :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` flags have been added. This allows extensions classes to support object ``__dict__`` and weakrefs with less bookkeeping, using less memory and with faster access. @@ -1663,7 +1663,7 @@ New Features * :c:func:`PyObject_Vectorcall` * :c:func:`PyObject_VectorcallMethod` - * :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` + * :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` This means that both the incoming and outgoing ends of the vector call protocol are now available in the :ref:`Limited API `. (Contributed @@ -1785,13 +1785,13 @@ Porting to Python 3.12 (Contributed by Philip Georgi in :gh:`95504`.) * Extension classes wanting to add a ``__dict__`` or weak reference slot - should use :const:`Py_TPFLAGS_MANAGED_DICT` and - :const:`Py_TPFLAGS_MANAGED_WEAKREF` instead of ``tp_dictoffset`` and + should use :c:macro:`Py_TPFLAGS_MANAGED_DICT` and + :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead of ``tp_dictoffset`` and ``tp_weaklistoffset``, respectively. The use of ``tp_dictoffset`` and ``tp_weaklistoffset`` is still supported, but does not fully support multiple inheritance (:gh:`95589`), and performance may be worse. - Classes declaring :const:`Py_TPFLAGS_MANAGED_DICT` should call + Classes declaring :c:macro:`Py_TPFLAGS_MANAGED_DICT` should call :c:func:`!_PyObject_VisitManagedDict` and :c:func:`!_PyObject_ClearManagedDict` to traverse and clear their instance's dictionaries. To clear weakrefs, call :c:func:`PyObject_ClearWeakRefs`, as before. @@ -1845,7 +1845,7 @@ Porting to Python 3.12 :c:member:`~PyTypeObject.tp_init` instead. - If the metaclass doesn't need to be instantiated from Python, set its ``tp_new`` to ``NULL`` using - the :const:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. + the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. This makes it acceptable for ``PyType_From*`` functions. - Avoid ``PyType_From*`` functions: if you don't need C-specific features @@ -1896,7 +1896,7 @@ Deprecated :c:type:`PyConfig` instead. (Contributed by Victor Stinner in :gh:`77782`.) -* Creating immutable types (:const:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable +* Creating immutable types (:c:macro:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable bases is deprecated and will be disabled in Python 3.14. * The ``structmember.h`` header is deprecated, though it continues to be diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 479d08b24b112..1d5e34dac28a5 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -961,7 +961,7 @@ Removed Pending Removal in Python 3.14 ------------------------------ -* Creating immutable types (:data:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable +* Creating immutable types (:c:macro:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable bases using the C API. * Global configuration variables: @@ -1017,7 +1017,7 @@ Pending Removal in Future Versions The following APIs were deprecated in earlier Python versions and will be removed, although there is currently no date scheduled for their removal. -* :const:`Py_TPFLAGS_HAVE_FINALIZE`: no needed since Python 3.8. +* :c:macro:`Py_TPFLAGS_HAVE_FINALIZE`: no needed since Python 3.8. * :c:func:`PyErr_Fetch`: use :c:func:`PyErr_GetRaisedException`. * :c:func:`PyErr_NormalizeException`: use :c:func:`PyErr_GetRaisedException`. * :c:func:`PyErr_Restore`: use :c:func:`PyErr_SetRaisedException`. diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 7d293c634f237..5161a0279452c 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -650,8 +650,8 @@ compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks: * Detect writes before the start of a buffer (buffer underflows) * Detect writes after the end of a buffer (buffer overflows) * Check that the :term:`GIL ` is held when allocator - functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and - :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. + functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and + :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. Checking if the GIL is held is also a new feature of Python 3.6. @@ -1822,7 +1822,7 @@ Optimizations up to 80% faster. (Contributed by Josh Snider in :issue:`26574`). * Allocator functions of the :c:func:`PyMem_Malloc` domain - (:c:data:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator + (:c:macro:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator ` instead of :c:func:`malloc` function of the C library. The pymalloc allocator is optimized for objects smaller or equal to 512 bytes with a short lifetime, and use :c:func:`malloc` for larger memory blocks. @@ -1874,8 +1874,8 @@ Build and C API Changes (Original patch by Alecsandru Patrascu of Intel in :issue:`26359`.) * The :term:`GIL ` must now be held when allocator - functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and - :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. + functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and + :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data failed. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 16762817ab825..b27fcce8178c4 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -2116,7 +2116,7 @@ Changes in the C API extension types across feature releases, anymore. A :c:type:`PyTypeObject` exported by a third-party extension module is supposed to have all the slots expected in the current Python version, including - :c:member:`~PyTypeObject.tp_finalize` (:const:`Py_TPFLAGS_HAVE_FINALIZE` + :c:member:`~PyTypeObject.tp_finalize` (:c:macro:`Py_TPFLAGS_HAVE_FINALIZE` is not checked anymore before reading :c:member:`~PyTypeObject.tp_finalize`). (Contributed by Antoine Pitrou in :issue:`32388`.) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 976d72a034251..d1e7efa079921 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1276,7 +1276,7 @@ New Features * :pep:`573`: Added :c:func:`PyType_FromModuleAndSpec` to associate a module with a class; :c:func:`PyType_GetModule` and :c:func:`PyType_GetModuleState` to retrieve the module and its state; and - :c:data:`PyCMethod` and :c:data:`METH_METHOD` to allow a method to + :c:data:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to access the class it was defined in. (Contributed by Marcel Plch and Petr Viktorin in :issue:`38787`.) diff --git a/Misc/NEWS.d/3.10.0a3.rst b/Misc/NEWS.d/3.10.0a3.rst index f24b6d43e5783..755109cbd376f 100644 --- a/Misc/NEWS.d/3.10.0a3.rst +++ b/Misc/NEWS.d/3.10.0a3.rst @@ -1466,7 +1466,7 @@ success. Patch by Victor Stinner. .. nonce: S3FWTP .. section: C API -The :c:data:`METH_FASTCALL` calling convention is added to the limited API. +The :c:macro:`METH_FASTCALL` calling convention is added to the limited API. The functions :c:func:`PyModule_AddType`, :c:func:`PyType_FromModuleAndSpec`, :c:func:`PyType_GetModule` and :c:func:`PyType_GetModuleState` are added to the limited API on Windows. diff --git a/Misc/NEWS.d/3.10.0b1.rst b/Misc/NEWS.d/3.10.0b1.rst index f29fc6632db26..38e0af17a2c66 100644 --- a/Misc/NEWS.d/3.10.0b1.rst +++ b/Misc/NEWS.d/3.10.0b1.rst @@ -1375,8 +1375,8 @@ Add "Annotations Best Practices" document as a new HOWTO. .. nonce: K5aSl1 .. section: Documentation -Document the new :const:`Py_TPFLAGS_MAPPING` and -:const:`Py_TPFLAGS_SEQUENCE` type flags. +Document the new :c:macro:`Py_TPFLAGS_MAPPING` and +:c:macro:`Py_TPFLAGS_SEQUENCE` type flags. .. @@ -1711,7 +1711,7 @@ IDLE's shell now shows prompts in a separate side-bar. .. nonce: wvWt23 .. section: C API -Add a new :c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow +Add a new :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow creating type instances. Patch by Victor Stinner. .. @@ -1759,7 +1759,7 @@ module. .. nonce: Co3YhZ .. section: C API -Introduce :const:`Py_TPFLAGS_IMMUTABLETYPE` flag for immutable type objects, +Introduce :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag for immutable type objects, and modify :c:func:`PyType_Ready` to set it for static types. Patch by Erlend E. Aasland. diff --git a/Misc/NEWS.d/3.11.0a1.rst b/Misc/NEWS.d/3.11.0a1.rst index 2f40252344f36..1a2d22b7017f4 100644 --- a/Misc/NEWS.d/3.11.0a1.rst +++ b/Misc/NEWS.d/3.11.0a1.rst @@ -888,7 +888,7 @@ zlib.decompress on input data that expands that large. .. nonce: YHuV_s .. section: Core and Builtins -Heap types with the :const:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit +Heap types with the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit the :pep:`590` vectorcall protocol. Previously, this was only possible for :ref:`static types `. Patch by Erlend E. Aasland. @@ -2575,7 +2575,7 @@ E. Aasland. .. nonce: bamAGF .. section: Library -Set the proper :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` +Set the proper :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` flags for subclasses created before a parent has been registered as a :class:`collections.abc.Mapping` or :class:`collections.abc.Sequence`. @@ -2693,7 +2693,7 @@ libgcc_s.so file (ex: EMFILE error). Patch by Victor Stinner. .. section: Library The _thread.RLock type now fully implement the GC protocol: add a traverse -function and the :const:`Py_TPFLAGS_HAVE_GC` flag. Patch by Victor Stinner. +function and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. Patch by Victor Stinner. .. @@ -5014,7 +5014,7 @@ must now be used to set an object type and size. Patch by Victor Stinner. .. section: C API The :c:func:`PyType_Ready` function now raises an error if a type is defined -with the :const:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function +with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function (:c:member:`PyTypeObject.tp_traverse`). Patch by Victor Stinner. .. diff --git a/Misc/NEWS.d/3.11.0a7.rst b/Misc/NEWS.d/3.11.0a7.rst index d3e59a2195669..94c15f1c1f523 100644 --- a/Misc/NEWS.d/3.11.0a7.rst +++ b/Misc/NEWS.d/3.11.0a7.rst @@ -275,7 +275,7 @@ initializing to ``list_extend``. Patch by Jeremiah Pascual. .. nonce: cnaIK3 .. section: Core and Builtins -Speed up throwing exception in generator with :const:`METH_FASTCALL` calling +Speed up throwing exception in generator with :c:macro:`METH_FASTCALL` calling convention. Patch by Kumar Aditya. .. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index d706343adf583..0609270c6805b 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -5856,8 +5856,8 @@ Configuration for the :ref:`integer string conversion length limitation Extensions classes that set ``tp_dictoffset`` and ``tp_weaklistoffset`` lose the support for multiple inheritance, but are now safe. Extension classes -should use :const:`Py_TPFLAGS_MANAGED_DICT` and -:const:`Py_TPFLAGS_MANAGED_WEAKREF` instead. +should use :c:macro:`Py_TPFLAGS_MANAGED_DICT` and +:c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead. .. @@ -5898,7 +5898,7 @@ Support C extensions using managed dictionaries by setting the .. nonce: QoDHEu .. section: C API -API for implementing vectorcall (:c:data:`Py_TPFLAGS_HAVE_VECTORCALL`, +API for implementing vectorcall (:c:macro:`Py_TPFLAGS_HAVE_VECTORCALL`, :c:func:`PyVectorcall_NARGS` and :c:func:`PyVectorcall_Call`) was added to the limited API and stable ABI. @@ -5920,12 +5920,12 @@ Philip Georgi. .. nonce: -DdGEy .. section: C API -The :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class +The :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class when the class's :py:meth:`~object.__call__` method is reassigned. This makes vectorcall safe to use with mutable types (i.e. heap types without the :const:`immutable ` flag). Mutable types that do not override :c:member:`~PyTypeObject.tp_call` now inherit the -:const:`Py_TPFLAGS_HAVE_VECTORCALL` flag. +:c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag. .. diff --git a/Misc/NEWS.d/3.12.0a2.rst b/Misc/NEWS.d/3.12.0a2.rst index d871384903e7c..5a3ccab02016f 100644 --- a/Misc/NEWS.d/3.12.0a2.rst +++ b/Misc/NEWS.d/3.12.0a2.rst @@ -279,7 +279,7 @@ Fix source locations of :keyword:`match` sub-patterns. Added the methods :c:func:`PyObject_Vectorcall` and :c:func:`PyObject_VectorcallMethod` to the :ref:`Limited API ` along -with the auxiliary macro constant :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. +with the auxiliary macro constant :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET`. The availability of these functions enables more efficient :PEP:`590` vector calls from binary extension modules that avoid argument boxing/unboxing diff --git a/Misc/NEWS.d/3.6.0a1.rst b/Misc/NEWS.d/3.6.0a1.rst index 53f09b3dfe336..98f1215fb9187 100644 --- a/Misc/NEWS.d/3.6.0a1.rst +++ b/Misc/NEWS.d/3.6.0a1.rst @@ -125,7 +125,7 @@ Setuptools 19.0. .. section: Core and Builtins Memory functions of the :c:func:`PyMem_Malloc` domain -(:c:data:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc allocator +(:c:macro:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc allocator ` rather than system :c:func:`malloc`. Applications calling :c:func:`PyMem_Malloc` without holding the GIL can now crash: use ``PYTHONMALLOC=debug`` environment variable to validate the usage of memory diff --git a/Misc/NEWS.d/3.9.0a1.rst b/Misc/NEWS.d/3.9.0a1.rst index 5ae14293df3e3..b97b7a506da31 100644 --- a/Misc/NEWS.d/3.9.0a1.rst +++ b/Misc/NEWS.d/3.9.0a1.rst @@ -5706,7 +5706,7 @@ and :c:func:`_PyObject_CallMethodOneArg`. .. nonce: qZC0N_ .. section: C API -The :const:`METH_FASTCALL` calling convention has been documented. +The :c:macro:`METH_FASTCALL` calling convention has been documented. .. From webhook-mailer at python.org Fri Jul 21 05:30:18 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 09:30:18 -0000 Subject: [Python-checkins] gh-47146: Fix reference counting in _testcapi.structmember initializer (GH-106862) Message-ID: https://github.com/python/cpython/commit/8d397ee8259fa0f81598a452438fc335267ca260 commit: 8d397ee8259fa0f81598a452438fc335267ca260 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T12:30:14+03:00 summary: gh-47146: Fix reference counting in _testcapi.structmember initializer (GH-106862) files: M Modules/_testcapi/structmember.c diff --git a/Modules/_testcapi/structmember.c b/Modules/_testcapi/structmember.c index 8522dc962efa4..096eaecd40855 100644 --- a/Modules/_testcapi/structmember.c +++ b/Modules/_testcapi/structmember.c @@ -193,7 +193,7 @@ _PyTestCapi_Init_Structmember(PyObject *m) if (res < 0) { return -1; } - res = PyModule_AddObject( + res = PyModule_AddObjectRef( m, "_test_structmembersType_OldAPI", (PyObject *)&test_structmembersType_OldAPI); From webhook-mailer at python.org Fri Jul 21 05:34:34 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 09:34:34 -0000 Subject: [Python-checkins] gh-106892: Use roles :data: and :const: for referencing module variables (GH-106894) Message-ID: https://github.com/python/cpython/commit/d036db728ea3d54509cbad06df74e2d9a31fbec8 commit: d036db728ea3d54509cbad06df74e2d9a31fbec8 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T12:34:30+03:00 summary: gh-106892: Use roles :data: and :const: for referencing module variables (GH-106894) files: M Doc/c-api/import.rst M Doc/library/__main__.rst M Doc/library/asyncio-subprocess.rst M Doc/library/compileall.rst M Doc/library/devmode.rst M Doc/library/filecmp.rst M Doc/library/ftplib.rst M Doc/library/functions.rst M Doc/library/gc.rst M Doc/library/gzip.rst M Doc/library/importlib.resources.abc.rst M Doc/library/importlib.rst M Doc/library/json.rst M Doc/library/logging.handlers.rst M Doc/library/os.path.rst M Doc/library/os.rst M Doc/library/platform.rst M Doc/library/shutil.rst M Doc/library/stdtypes.rst M Doc/library/subprocess.rst M Doc/library/sys.rst M Doc/library/tarfile.rst M Doc/library/test.rst M Doc/library/tkinter.rst M Doc/library/unittest.rst M Doc/reference/datamodel.rst M Doc/using/windows.rst M Doc/whatsnew/2.5.rst M Doc/whatsnew/3.1.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.2.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.4.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.8.0a4.rst M Misc/NEWS.d/3.9.0a5.rst diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 7aacc219a2bd6..f9c2f4e69ce88 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -142,9 +142,9 @@ Importing Modules read from a Python bytecode file or obtained from the built-in function :func:`compile`, load the module. Return a new reference to the module object, or ``NULL`` with an exception set if an error occurred. *name* - is removed from :attr:`sys.modules` in error cases, even if *name* was already - in :attr:`sys.modules` on entry to :c:func:`PyImport_ExecCodeModule`. Leaving - incompletely initialized modules in :attr:`sys.modules` is dangerous, as imports of + is removed from :data:`sys.modules` in error cases, even if *name* was already + in :data:`sys.modules` on entry to :c:func:`PyImport_ExecCodeModule`. Leaving + incompletely initialized modules in :data:`sys.modules` is dangerous, as imports of such modules have no way to know that the module object is an unknown (and probably damaged with respect to the module author's intents) state. diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index d29cbdff7830c..fd60d92d4eb0f 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -336,12 +336,12 @@ Note that importing ``__main__`` doesn't cause any issues with unintentionally running top-level code meant for script use which is put in the ``if __name__ == "__main__"`` block of the ``start`` module. Why does this work? -Python inserts an empty ``__main__`` module in :attr:`sys.modules` at +Python inserts an empty ``__main__`` module in :data:`sys.modules` at interpreter startup, and populates it by running top-level code. In our example this is the ``start`` module which runs line by line and imports ``namely``. In turn, ``namely`` imports ``__main__`` (which is really ``start``). That's an import cycle! Fortunately, since the partially populated ``__main__`` -module is present in :attr:`sys.modules`, Python passes that to ``namely``. +module is present in :data:`sys.modules`, Python passes that to ``namely``. See :ref:`Special considerations for __main__ ` in the import system's reference for details on how this works. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index b7c83aa04c09f..464a8d6fe7d96 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -68,7 +68,7 @@ Creating Subprocesses The *limit* argument sets the buffer limit for :class:`StreamReader` wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` - (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). + (if :const:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). Return a :class:`~asyncio.subprocess.Process` instance. @@ -86,7 +86,7 @@ Creating Subprocesses The *limit* argument sets the buffer limit for :class:`StreamReader` wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` - (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). + (if :const:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). Return a :class:`~asyncio.subprocess.Process` instance. diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst index 180f5b81c2b61..4226348a17240 100644 --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -141,9 +141,9 @@ There is no command-line option to control the optimization level used by the :func:`compile` function, because the Python interpreter itself already provides the option: :program:`python -O -m compileall`. -Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix` +Similarly, the :func:`compile` function respects the :data:`sys.pycache_prefix` setting. The generated bytecode cache will only be useful if :func:`compile` is -run with the same :attr:`sys.pycache_prefix` (if any) that will be used at +run with the same :data:`sys.pycache_prefix` (if any) that will be used at runtime. Public functions diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst index 80ac13b116c1d..914aa45cf9cbc 100644 --- a/Doc/library/devmode.rst +++ b/Doc/library/devmode.rst @@ -81,7 +81,7 @@ Effects of the Python Development Mode: ignored for empty strings. * The :class:`io.IOBase` destructor logs ``close()`` exceptions. -* Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to +* Set the :attr:`~sys.flags.dev_mode` attribute of :data:`sys.flags` to ``True``. The Python Development Mode does not enable the :mod:`tracemalloc` module by diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst index 83e9e14ddcacd..0efb4897a1eb8 100644 --- a/Doc/library/filecmp.rst +++ b/Doc/library/filecmp.rst @@ -74,7 +74,7 @@ The :class:`dircmp` class Construct a new directory comparison object, to compare the directories *a* and *b*. *ignore* is a list of names to ignore, and defaults to - :attr:`filecmp.DEFAULT_IGNORES`. *hide* is a list of names to hide, and + :const:`filecmp.DEFAULT_IGNORES`. *hide* is a list of names to hide, and defaults to ``[os.curdir, os.pardir]``. The :class:`dircmp` class compares files by doing *shallow* comparisons diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst index e7fb5b1ae2696..b90ead91933ab 100644 --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -431,7 +431,7 @@ FTP_TLS Objects .. attribute:: FTP_TLS.ssl_version - The SSL version to use (defaults to :attr:`ssl.PROTOCOL_SSLv23`). + The SSL version to use (defaults to :data:`ssl.PROTOCOL_SSLv23`). .. method:: FTP_TLS.auth() diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index d8091f0b093aa..2688f43f9b4ff 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1231,7 +1231,7 @@ are always available. They are listed here in alphabetical order. * Binary files are buffered in fixed-size chunks; the size of the buffer is chosen using a heuristic trying to determine the underlying device's "block - size" and falling back on :attr:`io.DEFAULT_BUFFER_SIZE`. On many systems, + size" and falling back on :const:`io.DEFAULT_BUFFER_SIZE`. On many systems, the buffer will typically be 4096 or 8192 bytes long. * "Interactive" text files (files for which :meth:`~io.IOBase.isatty` diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst index 0961ca4aaa942..331c071cda769 100644 --- a/Doc/library/gc.rst +++ b/Doc/library/gc.rst @@ -260,7 +260,7 @@ values but should not rebind them): .. versionchanged:: 3.4 Following :pep:`442`, objects with a :meth:`~object.__del__` method don't end - up in :attr:`gc.garbage` anymore. + up in :data:`gc.garbage` anymore. .. data:: callbacks diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst index 06cbd2567a0bc..979b39a3a5abb 100644 --- a/Doc/library/gzip.rst +++ b/Doc/library/gzip.rst @@ -268,7 +268,7 @@ Command line options .. cmdoption:: file - If *file* is not specified, read from :attr:`sys.stdin`. + If *file* is not specified, read from :data:`sys.stdin`. .. cmdoption:: --fast diff --git a/Doc/library/importlib.resources.abc.rst b/Doc/library/importlib.resources.abc.rst index 2d0f137ffc799..b0e75737137f2 100644 --- a/Doc/library/importlib.resources.abc.rst +++ b/Doc/library/importlib.resources.abc.rst @@ -130,7 +130,7 @@ suitable for reading (same as :attr:`pathlib.Path.open`). When opening as text, accepts encoding parameters such as those - accepted by :attr:`io.TextIOWrapper`. + accepted by :class:`io.TextIOWrapper`. .. method:: read_bytes() diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index 65aaad0df9ee6..a14e5a1a1a350 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -372,7 +372,7 @@ ABC hierarchy:: The list of locations where the package's submodules will be found. Most of the time this is a single directory. The import system passes this attribute to ``__import__()`` and to finders - in the same way as :attr:`sys.path` but just for the package. + in the same way as :data:`sys.path` but just for the package. It is not set on non-package modules so it can be used as an indicator that the module is a package. @@ -609,7 +609,7 @@ ABC hierarchy:: automatically. When writing to the path fails because the path is read-only - (:attr:`errno.EACCES`/:exc:`PermissionError`), do not propagate the + (:const:`errno.EACCES`/:exc:`PermissionError`), do not propagate the exception. .. versionchanged:: 3.4 @@ -843,7 +843,7 @@ find and load modules. .. classmethod:: path_hook(*loader_details) - A class method which returns a closure for use on :attr:`sys.path_hooks`. + A class method which returns a closure for use on :data:`sys.path_hooks`. An instance of :class:`FileFinder` is returned by the closure using the path argument given to the closure directly and *loader_details* indirectly. @@ -1184,10 +1184,10 @@ an :term:`importer`. .. function:: find_spec(name, package=None) Find the :term:`spec ` for a module, optionally relative to - the specified **package** name. If the module is in :attr:`sys.modules`, + the specified **package** name. If the module is in :data:`sys.modules`, then ``sys.modules[name].__spec__`` is returned (unless the spec would be ``None`` or is not set, in which case :exc:`ValueError` is raised). - Otherwise a search using :attr:`sys.meta_path` is done. ``None`` is + Otherwise a search using :data:`sys.meta_path` is done. ``None`` is returned if no spec is found. If **name** is for a submodule (contains a dot), the parent module is @@ -1259,7 +1259,7 @@ an :term:`importer`. :meth:`~importlib.abc.Loader.create_module` method must return ``None`` or a type for which its ``__class__`` attribute can be mutated along with not using :term:`slots <__slots__>`. Finally, modules which substitute the object - placed into :attr:`sys.modules` will not work as there is no way to properly + placed into :data:`sys.modules` will not work as there is no way to properly replace the module references throughout the interpreter safely; :exc:`ValueError` is raised if such a substitution is detected. @@ -1383,9 +1383,9 @@ For deep customizations of import, you typically want to implement an :term:`importer`. This means managing both the :term:`finder` and :term:`loader` side of things. For finders there are two flavours to choose from depending on your needs: a :term:`meta path finder` or a :term:`path entry finder`. The -former is what you would put on :attr:`sys.meta_path` while the latter is what -you create using a :term:`path entry hook` on :attr:`sys.path_hooks` which works -with :attr:`sys.path` entries to potentially create a finder. This example will +former is what you would put on :data:`sys.meta_path` while the latter is what +you create using a :term:`path entry hook` on :data:`sys.path_hooks` which works +with :data:`sys.path` entries to potentially create a finder. This example will show you how to register your own importers so that import will use them (for creating an importer for yourself, read the documentation for the appropriate classes defined within this package):: diff --git a/Doc/library/json.rst b/Doc/library/json.rst index 5383614575c21..35a08995487c1 100644 --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -683,7 +683,7 @@ The :mod:`json.tool` module provides a simple command line interface to validate and pretty-print JSON objects. If the optional ``infile`` and ``outfile`` arguments are not -specified, :attr:`sys.stdin` and :attr:`sys.stdout` will be used respectively: +specified, :data:`sys.stdin` and :data:`sys.stdout` will be used respectively: .. code-block:: shell-session @@ -721,12 +721,12 @@ Command line options } ] - If *infile* is not specified, read from :attr:`sys.stdin`. + If *infile* is not specified, read from :data:`sys.stdin`. .. cmdoption:: outfile Write the output of the *infile* to the given *outfile*. Otherwise, write it - to :attr:`sys.stdout`. + to :data:`sys.stdout`. .. cmdoption:: --sort-keys diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index d4429d3d0a4f7..47bfe4ff7f90e 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -1051,8 +1051,8 @@ possible, while any potentially slow operations (such as sending an email via occur (e.g. because a bounded queue has filled up), the :meth:`~logging.Handler.handleError` method is called to handle the error. This can result in the record silently being dropped (if - :attr:`logging.raiseExceptions` is ``False``) or a message printed to - ``sys.stderr`` (if :attr:`logging.raiseExceptions` is ``True``). + :data:`logging.raiseExceptions` is ``False``) or a message printed to + ``sys.stderr`` (if :data:`logging.raiseExceptions` is ``True``). .. method:: prepare(record) diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst index 3a668e28f2e26..6f9e0853bc894 100644 --- a/Doc/library/os.path.rst +++ b/Doc/library/os.path.rst @@ -410,7 +410,7 @@ the :mod:`glob` module.) *start*. On Windows, :exc:`ValueError` is raised when *path* and *start* are on different drives. - *start* defaults to :attr:`os.curdir`. + *start* defaults to :data:`os.curdir`. .. availability:: Unix, Windows. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index b35010498f214..43c8f50e3c186 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -60,7 +60,7 @@ Notes on the availability of these functions: ``'java'``. .. seealso:: - :attr:`sys.platform` has a finer granularity. :func:`os.uname` gives + :data:`sys.platform` has a finer granularity. :func:`os.uname` gives system-dependent version information. The :mod:`platform` module provides detailed checks for the diff --git a/Doc/library/platform.rst b/Doc/library/platform.rst index 69c4dfc422c98..ec2a7ebd5d6e0 100644 --- a/Doc/library/platform.rst +++ b/Doc/library/platform.rst @@ -46,7 +46,7 @@ Cross Platform universal files containing multiple architectures. To get at the "64-bitness" of the current interpreter, it is more - reliable to query the :attr:`sys.maxsize` attribute:: + reliable to query the :data:`sys.maxsize` attribute:: is_64bits = sys.maxsize > 2**32 diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 7f408be233682..5bd80b10a6580 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -431,7 +431,7 @@ Directory and files operations determining if the file exists and executable. When no *path* is specified, the results of :func:`os.environ` are used, - returning either the "PATH" value or a fallback of :attr:`os.defpath`. + returning either the "PATH" value or a fallback of :data:`os.defpath`. On Windows, the current directory is prepended to the *path* if *mode* does not include ``os.X_OK``. When the *mode* does include ``os.X_OK``, the diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index fd51b1187576b..26266b9eb28b8 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -5632,7 +5632,7 @@ From code, you can inspect the current limit and set a new one using these a getter and setter for the interpreter-wide limit. Subinterpreters have their own limit. -Information about the default and minimum can be found in :attr:`sys.int_info`: +Information about the default and minimum can be found in :data:`sys.int_info`: * :data:`sys.int_info.default_max_str_digits ` is the compiled-in default limit. diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 738e611c05adb..db05cb294d93f 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1610,7 +1610,7 @@ improves performance. If you ever encounter a presumed highly unusual situation where you need to prevent ``vfork()`` from being used by Python, you can set the -:attr:`subprocess._USE_VFORK` attribute to a false value. +:const:`subprocess._USE_VFORK` attribute to a false value. :: @@ -1618,7 +1618,7 @@ prevent ``vfork()`` from being used by Python, you can set the Setting this has no impact on use of ``posix_spawn()`` which could use ``vfork()`` internally within its libc implementation. There is a similar -:attr:`subprocess._USE_POSIX_SPAWN` attribute if you need to prevent use of +:const:`subprocess._USE_POSIX_SPAWN` attribute if you need to prevent use of that. :: diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index bacf8ceac5041..12ba93d9e1847 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -166,7 +166,7 @@ always available. Python interpreter. (This information is not available in any other way --- ``modules.keys()`` only lists the imported modules.) - See also the :attr:`sys.stdlib_module_names` list. + See also the :data:`sys.stdlib_module_names` list. .. function:: call_tracing(func, args) @@ -1287,20 +1287,20 @@ always available. ================ =========================== .. versionchanged:: 3.3 - On Linux, :attr:`sys.platform` doesn't contain the major version anymore. + On Linux, :data:`sys.platform` doesn't contain the major version anymore. It is always ``'linux'``, instead of ``'linux2'`` or ``'linux3'``. Since older Python versions include the version number, it is recommended to always use the ``startswith`` idiom presented above. .. versionchanged:: 3.8 - On AIX, :attr:`sys.platform` doesn't contain the major version anymore. + On AIX, :data:`sys.platform` doesn't contain the major version anymore. It is always ``'aix'``, instead of ``'aix5'`` or ``'aix7'``. Since older Python versions include the version number, it is recommended to always use the ``startswith`` idiom presented above. .. seealso:: - :attr:`os.name` has a coarser granularity. :func:`os.uname` gives + :data:`os.name` has a coarser granularity. :func:`os.uname` gives system-dependent version information. The :mod:`platform` module provides detailed checks for the @@ -1743,7 +1743,7 @@ always available. ``email.mime`` sub-package and the ``email.message`` sub-module are not listed. - See also the :attr:`sys.builtin_module_names` list. + See also the :data:`sys.builtin_module_names` list. .. versionadded:: 3.10 diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst index fd4820e78d68d..00f3070324ec1 100644 --- a/Doc/library/tarfile.rst +++ b/Doc/library/tarfile.rst @@ -938,7 +938,7 @@ reused in custom filters: Implements the ``'tar'`` filter. - - Strip leading slashes (``/`` and :attr:`os.sep`) from filenames. + - Strip leading slashes (``/`` and :data:`os.sep`) from filenames. - :ref:`Refuse ` to extract files with absolute paths (in case the name is absolute even after stripping slashes, e.g. ``C:/foo`` on Windows). @@ -947,7 +947,7 @@ reused in custom filters: path (after following symlinks) would end up outside the destination. This raises :class:`~tarfile.OutsideDestinationError`. - Clear high mode bits (setuid, setgid, sticky) and group/other write bits - (:attr:`~stat.S_IWGRP`|:attr:`~stat.S_IWOTH`). + (:const:`~stat.S_IWGRP`|:const:`~stat.S_IWOTH`). Return the modified ``TarInfo`` member. @@ -972,10 +972,10 @@ reused in custom filters: - For regular files, including hard links: - Set the owner read and write permissions - (:attr:`~stat.S_IRUSR`|:attr:`~stat.S_IWUSR`). + (:const:`~stat.S_IRUSR`|:const:`~stat.S_IWUSR`). - Remove the group & other executable permission - (:attr:`~stat.S_IXGRP`|:attr:`~stat.S_IXOTH`) - if the owner doesn?t have it (:attr:`~stat.S_IXUSR`). + (:const:`~stat.S_IXGRP`|:const:`~stat.S_IXOTH`) + if the owner doesn?t have it (:const:`~stat.S_IXUSR`). - For other files (directories), set ``mode`` to ``None``, so that extraction methods skip applying permission bits. diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 1b045c7de83a8..35cd6d5233c03 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -1040,7 +1040,7 @@ The :mod:`test.support` module defines the following classes: `SetErrorMode `_. On UNIX, :func:`resource.setrlimit` is used to set - :attr:`resource.RLIMIT_CORE`'s soft limit to 0 to prevent coredump file + :const:`resource.RLIMIT_CORE`'s soft limit to 0 to prevent coredump file creation. On both platforms, the old value is restored by :meth:`__exit__`. diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index 988b0cf3d7066..9f6c3e3e862c4 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -163,7 +163,7 @@ the modern themed widget set and API:: interpreter and calls :func:`exec` on the contents of :file:`.{className}.py` and :file:`.{baseName}.py`. The path for the profile files is the :envvar:`HOME` environment variable or, if that - isn't defined, then :attr:`os.curdir`. + isn't defined, then :data:`os.curdir`. .. attribute:: tk diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index b26e6c0e6bc02..518bf6b13bad5 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1134,7 +1134,7 @@ Test cases If given, *level* should be either a numeric logging level or its string equivalent (for example either ``"ERROR"`` or - :attr:`logging.ERROR`). The default is :attr:`logging.INFO`. + :const:`logging.ERROR`). The default is :const:`logging.INFO`. The test passes if at least one message emitted inside the ``with`` block matches the *logger* and *level* conditions, otherwise it fails. @@ -1175,7 +1175,7 @@ Test cases If given, *level* should be either a numeric logging level or its string equivalent (for example either ``"ERROR"`` or - :attr:`logging.ERROR`). The default is :attr:`logging.INFO`. + :const:`logging.ERROR`). The default is :const:`logging.INFO`. Unlike :meth:`assertLogs`, nothing will be returned by the context manager. diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 8a10a34347c2d..cbf854126bd5b 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2499,8 +2499,8 @@ through the object's keys; for sequences, it should iterate through the values. .. impl-detail:: - In CPython, the length is required to be at most :attr:`sys.maxsize`. - If the length is larger than :attr:`!sys.maxsize` some features (such as + In CPython, the length is required to be at most :data:`sys.maxsize`. + If the length is larger than :data:`!sys.maxsize` some features (such as :func:`len`) may raise :exc:`OverflowError`. To prevent raising :exc:`!OverflowError` by truth value testing, an object must define a :meth:`__bool__` method. diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index ac1ba111e6d9b..d24450e2f963f 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -1201,7 +1201,7 @@ non-standard paths in the registry and user site-packages. Modules specified in the registry under ``Modules`` (not ``PythonPath``) may be imported by :class:`importlib.machinery.WindowsRegistryFinder`. This finder is enabled on Windows in 3.6.0 and earlier, but may need to - be explicitly added to :attr:`sys.meta_path` in the future. + be explicitly added to :data:`sys.meta_path` in the future. Additional modules ================== diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index 85ffd170e7d11..ec4a7eb9a2361 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -1448,10 +1448,10 @@ complete list of changes, or look through the SVN logs for all the details. return times that are precise to fractions of a second; not all systems support such precision.) - Constants named :attr:`os.SEEK_SET`, :attr:`os.SEEK_CUR`, and - :attr:`os.SEEK_END` have been added; these are the parameters to the + Constants named :const:`os.SEEK_SET`, :const:`os.SEEK_CUR`, and + :const:`os.SEEK_END` have been added; these are the parameters to the :func:`os.lseek` function. Two new constants for locking are - :attr:`os.O_SHLOCK` and :attr:`os.O_EXLOCK`. + :const:`os.O_SHLOCK` and :const:`os.O_EXLOCK`. Two new functions, :func:`wait3` and :func:`wait4`, were added. They're similar the :func:`waitpid` function which waits for a child process to exit and returns @@ -1602,7 +1602,7 @@ complete list of changes, or look through the SVN logs for all the details. * The :mod:`unicodedata` module has been updated to use version 4.1.0 of the Unicode character database. Version 3.2.0 is required by some specifications, - so it's still available as :attr:`unicodedata.ucd_3_2_0`. + so it's still available as :data:`unicodedata.ucd_3_2_0`. * New module: the :mod:`uuid` module generates universally unique identifiers (UUIDs) according to :rfc:`4122`. The RFC defines several different UUID diff --git a/Doc/whatsnew/3.1.rst b/Doc/whatsnew/3.1.rst index 054762de7e743..c399f007fd63f 100644 --- a/Doc/whatsnew/3.1.rst +++ b/Doc/whatsnew/3.1.rst @@ -370,7 +370,7 @@ New, Improved, and Deprecated Modules * The :mod:`io` module has three new constants for the :meth:`seek` method :data:`SEEK_SET`, :data:`SEEK_CUR`, and :data:`SEEK_END`. -* The :attr:`sys.version_info` tuple is now a named tuple:: +* The :data:`sys.version_info` tuple is now a named tuple:: >>> sys.version_info sys.version_info(major=3, minor=1, micro=0, releaselevel='alpha', serial=2) @@ -486,7 +486,7 @@ Changes to Python's build process and to the C API include: Apart from the performance improvements this change should be invisible to end users, with one exception: for testing and debugging purposes there's a - new :attr:`sys.int_info` that provides information about the + new :data:`sys.int_info` that provides information about the internal format, giving the number of bits per digit and the size in bytes of the C type used to store each digit:: diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 92e85fc6c1480..859ee6e31649f 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -640,7 +640,7 @@ dataclasses datetime -------- -* Add :attr:`datetime.UTC`, a convenience alias for +* Add :const:`datetime.UTC`, a convenience alias for :attr:`datetime.timezone.utc`. (Contributed by Kabir Kwatra in :gh:`91973`.) * :meth:`datetime.date.fromisoformat`, :meth:`datetime.time.fromisoformat` and diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index b3b9b4134a9bb..947ba1a35c12a 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1042,7 +1042,7 @@ Deprecated datetimes in UTC: respectively, call :meth:`~datetime.datetime.now` and :meth:`~datetime.datetime.fromtimestamp` with the *tz* parameter set to - :attr:`datetime.UTC`. + :const:`datetime.UTC`. (Contributed by Paul Ganssle in :gh:`103857`.) Pending Removal in Python 3.13 diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index c3f7ef6e56599..4275503a8633b 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -424,7 +424,7 @@ protocols, the users must to be able access the environment using native strings even though the underlying platform may have a different convention. To bridge this gap, the :mod:`wsgiref` module has a new function, :func:`wsgiref.handlers.read_environ` for transcoding CGI variables from -:attr:`os.environ` into native strings and returning a new dictionary. +:data:`os.environ` into native strings and returning a new dictionary. .. seealso:: @@ -485,7 +485,7 @@ Some smaller changes made to the core Python language are: * The interpreter can now be started with a quiet option, ``-q``, to prevent the copyright and version information from being displayed in the interactive - mode. The option can be introspected using the :attr:`sys.flags` attribute: + mode. The option can be introspected using the :data:`sys.flags` attribute: .. code-block:: shell-session @@ -568,7 +568,7 @@ Some smaller changes made to the core Python language are: * The internal :c:type:`structsequence` tool now creates subclasses of tuple. This means that C structures like those returned by :func:`os.stat`, - :func:`time.gmtime`, and :attr:`sys.version_info` now work like a + :func:`time.gmtime`, and :data:`sys.version_info` now work like a :term:`named tuple` and now work with functions and methods that expect a tuple as an argument. This is a big step forward in making the C structures as flexible as their pure Python counterparts: @@ -598,7 +598,7 @@ Some smaller changes made to the core Python language are: module, or on the command line. A :exc:`ResourceWarning` is issued at interpreter shutdown if the - :data:`gc.garbage` list isn't empty, and if :attr:`gc.DEBUG_UNCOLLECTABLE` is + :data:`gc.garbage` list isn't empty, and if :const:`gc.DEBUG_UNCOLLECTABLE` is set, all uncollectable objects are printed. This is meant to make the programmer aware that their code contains object finalization issues. @@ -623,7 +623,7 @@ Some smaller changes made to the core Python language are: :class:`collections.Sequence` :term:`abstract base class`. As a result, the language will have a more uniform API. In addition, :class:`range` objects now support slicing and negative indices, even with values larger than - :attr:`sys.maxsize`. This makes *range* more interoperable with lists:: + :data:`sys.maxsize`. This makes *range* more interoperable with lists:: >>> range(0, 100, 2).count(10) 1 @@ -1007,13 +1007,13 @@ datetime and time after 1900. The new supported year range is from 1000 to 9999 inclusive. * Whenever a two-digit year is used in a time tuple, the interpretation has been - governed by :attr:`time.accept2dyear`. The default is ``True`` which means that + governed by :data:`time.accept2dyear`. The default is ``True`` which means that for a two-digit year, the century is guessed according to the POSIX rules governing the ``%y`` strptime format. Starting with Py3.2, use of the century guessing heuristic will emit a :exc:`DeprecationWarning`. Instead, it is recommended that - :attr:`time.accept2dyear` be set to ``False`` so that large date ranges + :data:`time.accept2dyear` be set to ``False`` so that large date ranges can be used without guesswork:: >>> import time, warnings @@ -1031,7 +1031,7 @@ datetime and time 'Fri Jan 1 12:34:56 11' Several functions now have significantly expanded date ranges. When - :attr:`time.accept2dyear` is false, the :func:`time.asctime` function will + :data:`time.accept2dyear` is false, the :func:`time.asctime` function will accept any year that fits in a C int, while the :func:`time.mktime` and :func:`time.strftime` functions will accept the full range supported by the corresponding operating system functions. @@ -1194,11 +1194,11 @@ can be set to "$" for the shell-style formatting provided by If no configuration is set-up before a logging event occurs, there is now a default configuration using a :class:`~logging.StreamHandler` directed to -:attr:`sys.stderr` for events of ``WARNING`` level or higher. Formerly, an +:data:`sys.stderr` for events of ``WARNING`` level or higher. Formerly, an event occurring before a configuration was set-up would either raise an exception or silently drop the event depending on the value of -:attr:`logging.raiseExceptions`. The new default handler is stored in -:attr:`logging.lastResort`. +:data:`logging.raiseExceptions`. The new default handler is stored in +:data:`logging.lastResort`. The use of filters has been simplified. Instead of creating a :class:`~logging.Filter` object, the predicate can be any Python callable that @@ -1300,7 +1300,7 @@ values are equal (:issue:`8188`):: hash(Decimal("1.5")) == hash(complex(1.5, 0)) Some of the hashing details are exposed through a new attribute, -:attr:`sys.hash_info`, which describes the bit width of the hash value, the +:data:`sys.hash_info`, which describes the bit width of the hash value, the prime modulus, the hash values for *infinity* and *nan*, and the multiplier used for the imaginary part of a number: @@ -1388,7 +1388,7 @@ select ------ The :mod:`select` module now exposes a new, constant attribute, -:attr:`~select.PIPE_BUF`, which gives the minimum number of bytes which are +:const:`~select.PIPE_BUF`, which gives the minimum number of bytes which are guaranteed not to block when :func:`select.select` says a pipe is ready for writing. @@ -1529,7 +1529,7 @@ filenames: b'Sehensw\xc3\xbcrdigkeiten' Some operating systems allow direct access to encoded bytes in the -environment. If so, the :attr:`os.supports_bytes_environ` constant will be +environment. If so, the :const:`os.supports_bytes_environ` constant will be true. For direct access to encoded environment variables (if available), @@ -2302,7 +2302,7 @@ turtledemo The demonstration code for the :mod:`turtle` module was moved from the *Demo* directory to main library. It includes over a dozen sample scripts with -lively displays. Being on :attr:`sys.path`, it can now be run directly +lively displays. Being on :data:`sys.path`, it can now be run directly from the command-line: .. code-block:: shell-session @@ -2566,7 +2566,7 @@ Changes to Python's build process and to the C API include: (:issue:`2443`). * A new C API function :c:func:`PySys_SetArgvEx` allows an embedded interpreter - to set :attr:`sys.argv` without also modifying :attr:`sys.path` + to set :data:`sys.argv` without also modifying :data:`sys.path` (:issue:`5753`). * :c:macro:`PyEval_CallObject` is now only available in macro form. The diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 3dca7227a91c3..c108510b2aebd 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -648,7 +648,7 @@ PEP 421: Adding sys.implementation A new attribute on the :mod:`sys` module exposes details specific to the implementation of the currently running interpreter. The initial set of -attributes on :attr:`sys.implementation` are ``name``, ``version``, +attributes on :data:`sys.implementation` are ``name``, ``version``, ``hexversion``, and ``cache_tag``. The intention of ``sys.implementation`` is to consolidate into one namespace @@ -719,7 +719,7 @@ and does not enforce any method requirements. In terms of finders, :class:`importlib.machinery.FileFinder` exposes the mechanism used to search for source and bytecode files of a module. Previously -this class was an implicit member of :attr:`sys.path_hooks`. +this class was an implicit member of :data:`sys.path_hooks`. For loaders, the new abstract base class :class:`importlib.abc.FileLoader` helps write a loader that uses the file system as the storage mechanism for a module's @@ -735,7 +735,7 @@ provide the full name of the module now instead of just the tail end of the module's name. The :func:`importlib.invalidate_caches` function will now call the method with -the same name on all finders cached in :attr:`sys.path_importer_cache` to help +the same name on all finders cached in :data:`sys.path_importer_cache` to help clean up any stored state as necessary. Visible Changes @@ -745,8 +745,8 @@ For potential required changes to code, see the `Porting Python code`_ section. Beyond the expanse of what :mod:`importlib` now exposes, there are other -visible changes to import. The biggest is that :attr:`sys.meta_path` and -:attr:`sys.path_hooks` now store all of the meta path finders and path entry +visible changes to import. The biggest is that :data:`sys.meta_path` and +:data:`sys.path_hooks` now store all of the meta path finders and path entry hooks used by import. Previously the finders were implicit and hidden within the C code of import instead of being directly exposed. This means that one can now easily remove or change the order of the various finders to fit one's needs. @@ -761,9 +761,9 @@ Loaders are also now expected to set the ``__package__`` attribute from :pep:`366`. Once again, import itself is already setting this on all loaders from :mod:`importlib` and import itself is setting the attribute post-load. -``None`` is now inserted into :attr:`sys.path_importer_cache` when no finder -can be found on :attr:`sys.path_hooks`. Since :class:`!imp.NullImporter` is not -directly exposed on :attr:`sys.path_hooks` it could no longer be relied upon to +``None`` is now inserted into :data:`sys.path_importer_cache` when no finder +can be found on :data:`sys.path_hooks`. Since :class:`!imp.NullImporter` is not +directly exposed on :data:`sys.path_hooks` it could no longer be relied upon to always be available to use as a value representing no finder found. All other changes relate to semantic changes which should be taken into @@ -1952,7 +1952,7 @@ ssl * You can query the SSL compression algorithm used by an SSL socket, thanks to its new :meth:`~ssl.SSLSocket.compression` method. The new attribute - :attr:`~ssl.OP_NO_COMPRESSION` can be used to disable compression. + :const:`~ssl.OP_NO_COMPRESSION` can be used to disable compression. (Contributed by Antoine Pitrou in :issue:`13634`.) * Support has been added for the Next Protocol Negotiation extension using @@ -1966,7 +1966,7 @@ ssl * The :func:`~ssl.get_server_certificate` function now supports IPv6. (Contributed by Charles-Fran?ois Natali in :issue:`11811`.) -* New attribute :attr:`~ssl.OP_CIPHER_SERVER_PREFERENCE` allows setting +* New attribute :const:`~ssl.OP_CIPHER_SERVER_PREFERENCE` allows setting SSLv3 server sockets to use the server's cipher ordering preference rather than the client's (:issue:`13635`). @@ -2141,7 +2141,7 @@ New attribute :attr:`zlib.Decompress.eof` makes it possible to distinguish between a properly formed compressed stream and an incomplete or truncated one. (Contributed by Nadeem Vawda in :issue:`12646`.) -New attribute :attr:`zlib.ZLIB_RUNTIME_VERSION` reports the version string of +New attribute :const:`zlib.ZLIB_RUNTIME_VERSION` reports the version string of the underlying ``zlib`` library that is loaded at runtime. (Contributed by Torsten Landschoff in :issue:`12306`.) @@ -2378,16 +2378,16 @@ Porting Python code * :func:`__import__` no longer allows one to use an index value other than 0 for top-level modules. E.g. ``__import__('sys', level=1)`` is now an error. -* Because :attr:`sys.meta_path` and :attr:`sys.path_hooks` now have finders on +* Because :data:`sys.meta_path` and :data:`sys.path_hooks` now have finders on them by default, you will most likely want to use :meth:`list.insert` instead of :meth:`list.append` to add to those lists. -* Because ``None`` is now inserted into :attr:`sys.path_importer_cache`, if you +* Because ``None`` is now inserted into :data:`sys.path_importer_cache`, if you are clearing out entries in the dictionary of paths that do not have a finder, you will need to remove keys paired with values of ``None`` **and** :class:`!imp.NullImporter` to be backwards-compatible. This will lead to extra overhead on older versions of Python that re-insert ``None`` into - :attr:`sys.path_importer_cache` where it represents the use of implicit + :data:`sys.path_importer_cache` where it represents the use of implicit finders, but semantically it should not change anything. * :class:`!importlib.abc.Finder` no longer specifies a ``find_module()`` abstract @@ -2445,7 +2445,7 @@ Porting Python code error instead of sleeping forever. It has always raised an error on posix. * The ``ast.__version__`` constant has been removed. If you need to - make decisions affected by the AST version, use :attr:`sys.version_info` + make decisions affected by the AST version, use :data:`sys.version_info` to make the decision. * Code that used to work around the fact that the :mod:`threading` module used diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst index f3a8873747a3e..e753c3d364c2b 100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1323,14 +1323,14 @@ ability to query or set the resource limits for processes other than the one making the call. (Contributed by Christian Heimes in :issue:`16595`.) On Linux kernel version 2.6.36 or later, there are also some new -Linux specific constants: :attr:`~resource.RLIMIT_MSGQUEUE`, -:attr:`~resource.RLIMIT_NICE`, :attr:`~resource.RLIMIT_RTPRIO`, -:attr:`~resource.RLIMIT_RTTIME`, and :attr:`~resource.RLIMIT_SIGPENDING`. +Linux specific constants: :const:`~resource.RLIMIT_MSGQUEUE`, +:const:`~resource.RLIMIT_NICE`, :const:`~resource.RLIMIT_RTPRIO`, +:const:`~resource.RLIMIT_RTTIME`, and :const:`~resource.RLIMIT_SIGPENDING`. (Contributed by Christian Heimes in :issue:`19324`.) On FreeBSD version 9 and later, there some new FreeBSD specific constants: -:attr:`~resource.RLIMIT_SBSIZE`, :attr:`~resource.RLIMIT_SWAP`, and -:attr:`~resource.RLIMIT_NPTS`. (Contributed by Claudiu Popa in +:const:`~resource.RLIMIT_SBSIZE`, :const:`~resource.RLIMIT_SWAP`, and +:const:`~resource.RLIMIT_NPTS`. (Contributed by Claudiu Popa in :issue:`19343`.) @@ -1500,7 +1500,7 @@ implementation is required as most of the values aren't standardized and are platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.) The module supports new :mod:`~stat.ST_MODE` flags, :mod:`~stat.S_IFDOOR`, -:attr:`~stat.S_IFPORT`, and :attr:`~stat.S_IFWHT`. (Contributed by +:const:`~stat.S_IFPORT`, and :const:`~stat.S_IFWHT`. (Contributed by Christian Hiemes in :issue:`11016`.) @@ -1849,7 +1849,7 @@ Python's default implementation to a SipHash implementation on platforms that have a 64 bit data type. Any performance differences in comparison with the older FNV algorithm are trivial. -The PEP adds additional fields to the :attr:`sys.hash_info` named tuple to +The PEP adds additional fields to the :data:`sys.hash_info` named tuple to describe the hash algorithm in use by the currently executing binary. Otherwise, the PEP does not alter any existing CPython APIs. diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 5161a0279452c..ed398f2e8c562 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1388,7 +1388,7 @@ are treated as punctuation. site ---- -When specifying paths to add to :attr:`sys.path` in a ``.pth`` file, +When specifying paths to add to :data:`sys.path` in a ``.pth`` file, you may now specify file paths on top of directories (e.g. zip files). (Contributed by Wolfgang Langner in :issue:`26587`). @@ -2010,7 +2010,7 @@ been deprecated in previous versions of Python in favour of :meth:`importlib.abc.Loader.exec_module`. The :class:`importlib.machinery.WindowsRegistryFinder` class is now -deprecated. As of 3.6.0, it is still added to :attr:`sys.meta_path` by +deprecated. As of 3.6.0, it is still added to :data:`sys.meta_path` by default (on Windows), but this may change in future releases. os diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index b27fcce8178c4..3efb3f49b77ac 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1839,7 +1839,7 @@ Changes in Python behavior classes will affect their string representation. (Contributed by Serhiy Storchaka in :issue:`36793`.) -* On AIX, :attr:`sys.platform` doesn't contain the major version anymore. +* On AIX, :data:`sys.platform` doesn't contain the major version anymore. It is always ``'aix'``, instead of ``'aix3'`` .. ``'aix7'``. Since older Python versions include the version number, so it is recommended to always use ``sys.platform.startswith('aix')``. diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index d1e7efa079921..a8f9774af7add 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -692,13 +692,13 @@ which has nanosecond resolution, rather than sys --- -Added a new :attr:`sys.platlibdir` attribute: name of the platform-specific +Added a new :data:`sys.platlibdir` attribute: name of the platform-specific library directory. It is used to build the path of standard library and the paths of installed extension modules. It is equal to ``"lib"`` on most platforms. On Fedora and SuSE, it is equal to ``"lib64"`` on 64-bit platforms. (Contributed by Jan Mat?jek, Mat?j Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.) -Previously, :attr:`sys.stderr` was block-buffered when non-interactive. Now +Previously, :data:`sys.stderr` was block-buffered when non-interactive. Now ``stderr`` defaults to always being line-buffered. (Contributed by Jendrik Seipp in :issue:`13601`.) @@ -1226,8 +1226,8 @@ Build Changes ============= * Added ``--with-platlibdir`` option to the ``configure`` script: name of the - platform-specific library directory, stored in the new :attr:`sys.platlibdir` - attribute. See :attr:`sys.platlibdir` attribute for more information. + platform-specific library directory, stored in the new :data:`sys.platlibdir` + attribute. See :data:`sys.platlibdir` attribute for more information. (Contributed by Jan Mat?jek, Mat?j Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.) diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index 0609270c6805b..c4dfc6011ed6c 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -4171,7 +4171,7 @@ Add an index_pages parameter to support using non-default index page names. .. nonce: qtT3CE .. section: Library -Drop support for :class:`bytes` on :attr:`sys.path`. +Drop support for :class:`bytes` on :data:`sys.path`. .. diff --git a/Misc/NEWS.d/3.8.0a4.rst b/Misc/NEWS.d/3.8.0a4.rst index 9841195210c9e..a0d60f456789f 100644 --- a/Misc/NEWS.d/3.8.0a4.rst +++ b/Misc/NEWS.d/3.8.0a4.rst @@ -92,7 +92,7 @@ the field. .. nonce: wejLoC .. section: Core and Builtins -On AIX, :attr:`sys.platform` doesn't contain the major version anymore. +On AIX, :data:`sys.platform` doesn't contain the major version anymore. Always return ``'aix'``, instead of ``'aix3'`` .. ``'aix7'``. Since older Python versions include the version number, it is recommended to always use ``sys.platform.startswith('aix')``. Contributed by M. Felt. diff --git a/Misc/NEWS.d/3.9.0a5.rst b/Misc/NEWS.d/3.9.0a5.rst index 25342d21d8f0b..8a1219501e81b 100644 --- a/Misc/NEWS.d/3.9.0a5.rst +++ b/Misc/NEWS.d/3.9.0a5.rst @@ -582,7 +582,7 @@ Fix :mod:`json.tool` to catch :exc:`BrokenPipeError`. Patch by Dong-hee Na. Avoid a possible *"RuntimeError: dictionary changed size during iteration"* from :func:`inspect.getmodule` when it tried to loop through -:attr:`sys.modules`. +:data:`sys.modules`. .. @@ -989,7 +989,7 @@ modules are built. Add ``--with-platlibdir`` option to the configure script: name of the platform-specific library directory, stored in the new -:attr:`sys.platlibdir` attribute. It is used to build the path of +:data:`sys.platlibdir` attribute. It is used to build the path of platform-specific extension modules and the path of the standard library. It is equal to ``"lib"`` on most platforms. On Fedora and SuSE, it is equal to ``"lib64"`` on 64-bit platforms. Patch by Jan Mat?jek, Mat?j Cepl, From webhook-mailer at python.org Fri Jul 21 05:40:42 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 09:40:42 -0000 Subject: [Python-checkins] gh-106909: Use role :const: for referencing module constants (GH-106910) Message-ID: https://github.com/python/cpython/commit/4b9948617f91175783609769aa6160e5b49b9ccc commit: 4b9948617f91175783609769aa6160e5b49b9ccc branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T12:40:37+03:00 summary: gh-106909: Use role :const: for referencing module constants (GH-106910) files: M Doc/c-api/exceptions.rst M Doc/faq/library.rst M Doc/install/index.rst M Doc/library/_thread.rst M Doc/library/asyncio-dev.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-platforms.rst M Doc/library/asyncio-subprocess.rst M Doc/library/exceptions.rst M Doc/library/fcntl.rst M Doc/library/ftplib.rst M Doc/library/http.client.rst M Doc/library/imaplib.rst M Doc/library/io.rst M Doc/library/multiprocessing.rst M Doc/library/optparse.rst M Doc/library/os.rst M Doc/library/poplib.rst M Doc/library/shelve.rst M Doc/library/smtplib.rst M Doc/library/socket.rst M Doc/library/sqlite3.rst M Doc/library/ssl.rst M Doc/library/sys.rst M Doc/library/tempfile.rst M Doc/library/test.rst M Doc/library/unittest.mock.rst M Doc/library/urllib.request.rst M Doc/library/xml.rst M Doc/using/configure.rst M Doc/whatsnew/2.7.rst M Doc/whatsnew/3.10.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.2.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.4.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.7.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a1.rst M Misc/NEWS.d/3.10.0a2.rst M Misc/NEWS.d/3.10.0a6.rst M Misc/NEWS.d/3.10.0a7.rst M Misc/NEWS.d/3.10.0b1.rst M Misc/NEWS.d/3.11.0a1.rst M Misc/NEWS.d/3.11.0a4.rst M Misc/NEWS.d/3.11.0b1.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0a2.rst M Misc/NEWS.d/3.12.0a3.rst M Misc/NEWS.d/3.12.0a4.rst M Misc/NEWS.d/3.12.0a6.rst M Misc/NEWS.d/3.12.0a7.rst M Misc/NEWS.d/3.12.0b1.rst M Misc/NEWS.d/3.6.0rc1.rst M Misc/NEWS.d/3.7.0a1.rst M Misc/NEWS.d/3.7.0a3.rst M Misc/NEWS.d/3.8.0a1.rst M Misc/NEWS.d/3.8.0a4.rst M Misc/NEWS.d/3.9.0a1.rst M Misc/NEWS.d/3.9.0a6.rst diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 6e09f829da304..3d4ceb360f5b1 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -666,7 +666,7 @@ Signal Handling to interrupt an operation). If the given signal isn't handled by Python (it was set to - :data:`signal.SIG_DFL` or :data:`signal.SIG_IGN`), it will be ignored. + :py:const:`signal.SIG_DFL` or :py:const:`signal.SIG_IGN`), it will be ignored. If *signum* is outside of the allowed range of signal numbers, ``-1`` is returned. Otherwise, ``0`` is returned. The error indicator is diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst index 22f7f846d261d..9e3727456bb96 100644 --- a/Doc/faq/library.rst +++ b/Doc/faq/library.rst @@ -566,7 +566,7 @@ use ``p.read(n)``. Note on a bug in popen2: unless your program calls ``wait()`` or ``waitpid()``, finished child processes are never removed, and eventually calls to popen2 will fail because of a limit on the number of child - processes. Calling :func:`os.waitpid` with the :data:`os.WNOHANG` option can + processes. Calling :func:`os.waitpid` with the :const:`os.WNOHANG` option can prevent this; a good place to insert such a call would be before calling ``popen2`` again. diff --git a/Doc/install/index.rst b/Doc/install/index.rst index beb34f0cf21b2..34d47fdb6a2f0 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -313,9 +313,9 @@ install into it. It is enabled with a simple option:: python setup.py install --user -Files will be installed into subdirectories of :data:`site.USER_BASE` (written +Files will be installed into subdirectories of :const:`site.USER_BASE` (written as :file:`{userbase}` hereafter). This scheme installs pure Python modules and -extension modules in the same location (also known as :data:`site.USER_SITE`). +extension modules in the same location (also known as :const:`site.USER_SITE`). Here are the values for UNIX, including macOS: =============== =========================================================== diff --git a/Doc/library/_thread.rst b/Doc/library/_thread.rst index ba9314e46ab6e..3ff6b48b083b5 100644 --- a/Doc/library/_thread.rst +++ b/Doc/library/_thread.rst @@ -70,10 +70,10 @@ This module defines the following constants and functions: there is no guarantee that the interruption will happen immediately. If given, *signum* is the number of the signal to simulate. - If *signum* is not given, :data:`signal.SIGINT` is simulated. + If *signum* is not given, :const:`signal.SIGINT` is simulated. If the given signal isn't handled by Python (it was set to - :data:`signal.SIG_DFL` or :data:`signal.SIG_IGN`), this function does + :const:`signal.SIG_DFL` or :const:`signal.SIG_IGN`), this function does nothing. .. versionchanged:: 3.10 diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index 921a394a59fec..c7d97008fb490 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -34,7 +34,7 @@ There are several ways to enable asyncio debug mode: In addition to enabling the debug mode, consider also: * setting the log level of the :ref:`asyncio logger ` to - :py:data:`logging.DEBUG`, for example the following snippet of code + :py:const:`logging.DEBUG`, for example the following snippet of code can be run at startup of the application:: logging.basicConfig(level=logging.DEBUG) @@ -142,7 +142,7 @@ Logging asyncio uses the :mod:`logging` module and all logging is performed via the ``"asyncio"`` logger. -The default log level is :py:data:`logging.INFO`, which can be easily +The default log level is :py:const:`logging.INFO`, which can be easily adjusted:: logging.getLogger("asyncio").setLevel(logging.WARNING) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 1ef8a5ab832e4..8f2d8f336c82b 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -403,11 +403,11 @@ Opening network connections Open a streaming transport connection to a given address specified by *host* and *port*. - The socket family can be either :py:data:`~socket.AF_INET` or - :py:data:`~socket.AF_INET6` depending on *host* (or the *family* + The socket family can be either :py:const:`~socket.AF_INET` or + :py:const:`~socket.AF_INET6` depending on *host* (or the *family* argument, if provided). - The socket type will be :py:data:`~socket.SOCK_STREAM`. + The socket type will be :py:const:`~socket.SOCK_STREAM`. *protocol_factory* must be a callable returning an :ref:`asyncio protocol ` implementation. @@ -509,7 +509,7 @@ Opening network connections .. versionchanged:: 3.6 - The socket option :py:data:`~socket.TCP_NODELAY` is set by default + The socket option :py:const:`~socket.TCP_NODELAY` is set by default for all TCP connections. .. versionchanged:: 3.7 @@ -552,11 +552,11 @@ Opening network connections Create a datagram connection. - The socket family can be either :py:data:`~socket.AF_INET`, - :py:data:`~socket.AF_INET6`, or :py:data:`~socket.AF_UNIX`, + The socket family can be either :py:const:`~socket.AF_INET`, + :py:const:`~socket.AF_INET6`, or :py:const:`~socket.AF_UNIX`, depending on *host* (or the *family* argument, if provided). - The socket type will be :py:data:`~socket.SOCK_DGRAM`. + The socket type will be :py:const:`~socket.SOCK_DGRAM`. *protocol_factory* must be a callable returning a :ref:`protocol ` implementation. @@ -581,7 +581,7 @@ Opening network connections * *reuse_port* tells the kernel to allow this endpoint to be bound to the same port as other existing endpoints are bound to, so long as they all set this flag when being created. This option is not supported on Windows - and some Unixes. If the :py:data:`~socket.SO_REUSEPORT` constant is not + and some Unixes. If the :py:const:`~socket.SO_REUSEPORT` constant is not defined then this capability is unsupported. * *allow_broadcast* tells the kernel to allow this endpoint to send @@ -607,7 +607,7 @@ Opening network connections .. versionchanged:: 3.8.1 The *reuse_address* parameter is no longer supported, as using - :py:data:`~sockets.SO_REUSEADDR` poses a significant security concern for + :py:const:`~sockets.SO_REUSEADDR` poses a significant security concern for UDP. Explicitly passing ``reuse_address=True`` will raise an exception. When multiple processes with differing UIDs assign sockets to an @@ -616,7 +616,7 @@ Opening network connections For supported platforms, *reuse_port* can be used as a replacement for similar functionality. With *reuse_port*, - :py:data:`~sockets.SO_REUSEPORT` is used instead, which specifically + :py:const:`~sockets.SO_REUSEPORT` is used instead, which specifically prevents processes with differing UIDs from assigning sockets to the same socket address. @@ -634,8 +634,8 @@ Opening network connections Create a Unix connection. - The socket family will be :py:data:`~socket.AF_UNIX`; socket - type will be :py:data:`~socket.SOCK_STREAM`. + The socket family will be :py:const:`~socket.AF_UNIX`; socket + type will be :py:const:`~socket.SOCK_STREAM`. A tuple of ``(transport, protocol)`` is returned on success. @@ -671,7 +671,7 @@ Creating network servers ssl_shutdown_timeout=None, \ start_serving=True) - Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) listening + Create a TCP server (socket type :const:`~socket.SOCK_STREAM`) listening on *port* of the *host* address. Returns a :class:`Server` object. @@ -699,10 +699,10 @@ Creating network servers be selected (note that if *host* resolves to multiple network interfaces, a different random port will be selected for each interface). - * *family* can be set to either :data:`socket.AF_INET` or - :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. + * *family* can be set to either :const:`socket.AF_INET` or + :const:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set, the *family* will be determined from host name - (defaults to :data:`~socket.AF_UNSPEC`). + (defaults to :const:`~socket.AF_UNSPEC`). * *flags* is a bitmask for :meth:`getaddrinfo`. @@ -756,7 +756,7 @@ Creating network servers .. versionchanged:: 3.6 Added *ssl_handshake_timeout* and *start_serving* parameters. - The socket option :py:data:`~socket.TCP_NODELAY` is set by default + The socket option :py:const:`~socket.TCP_NODELAY` is set by default for all TCP connections. .. versionchanged:: 3.11 @@ -777,7 +777,7 @@ Creating network servers start_serving=True) Similar to :meth:`loop.create_server` but works with the - :py:data:`~socket.AF_UNIX` socket family. + :py:const:`~socket.AF_UNIX` socket family. *path* is the name of a Unix domain socket, and is required, unless a *sock* argument is provided. Abstract Unix sockets, diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst index 50ad8a2ab7032..19ec726c1be06 100644 --- a/Doc/library/asyncio-platforms.rst +++ b/Doc/library/asyncio-platforms.rst @@ -37,7 +37,7 @@ All event loops on Windows do not support the following methods: * :meth:`loop.create_unix_connection` and :meth:`loop.create_unix_server` are not supported. - The :data:`socket.AF_UNIX` socket family is specific to Unix. + The :const:`socket.AF_UNIX` socket family is specific to Unix. * :meth:`loop.add_signal_handler` and :meth:`loop.remove_signal_handler` are not supported. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 464a8d6fe7d96..bf35b1cb798ae 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -249,7 +249,7 @@ their completion. Stop the child process. - On POSIX systems this method sends :py:data:`signal.SIGTERM` to the + On POSIX systems this method sends :py:const:`signal.SIGTERM` to the child process. On Windows the Win32 API function :c:func:`TerminateProcess` is diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index 4651eddf84370..fae0cf621323c 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -659,8 +659,8 @@ depending on the system error code. Raised when an operation would block on an object (e.g. socket) set for non-blocking operation. - Corresponds to :c:data:`errno` :py:data:`~errno.EAGAIN`, :py:data:`~errno.EALREADY`, - :py:data:`~errno.EWOULDBLOCK` and :py:data:`~errno.EINPROGRESS`. + Corresponds to :c:data:`errno` :py:const:`~errno.EAGAIN`, :py:const:`~errno.EALREADY`, + :py:const:`~errno.EWOULDBLOCK` and :py:const:`~errno.EINPROGRESS`. In addition to those of :exc:`OSError`, :exc:`BlockingIOError` can have one more attribute: @@ -674,7 +674,7 @@ depending on the system error code. .. exception:: ChildProcessError Raised when an operation on a child process failed. - Corresponds to :c:data:`errno` :py:data:`~errno.ECHILD`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECHILD`. .. exception:: ConnectionError @@ -688,40 +688,40 @@ depending on the system error code. A subclass of :exc:`ConnectionError`, raised when trying to write on a pipe while the other end has been closed, or trying to write on a socket which has been shutdown for writing. - Corresponds to :c:data:`errno` :py:data:`~errno.EPIPE` and :py:data:`~errno.ESHUTDOWN`. + Corresponds to :c:data:`errno` :py:const:`~errno.EPIPE` and :py:const:`~errno.ESHUTDOWN`. .. exception:: ConnectionAbortedError A subclass of :exc:`ConnectionError`, raised when a connection attempt is aborted by the peer. - Corresponds to :c:data:`errno` :py:data:`~errno.ECONNABORTED`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECONNABORTED`. .. exception:: ConnectionRefusedError A subclass of :exc:`ConnectionError`, raised when a connection attempt is refused by the peer. - Corresponds to :c:data:`errno` :py:data:`~errno.ECONNREFUSED`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECONNREFUSED`. .. exception:: ConnectionResetError A subclass of :exc:`ConnectionError`, raised when a connection is reset by the peer. - Corresponds to :c:data:`errno` :py:data:`~errno.ECONNRESET`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECONNRESET`. .. exception:: FileExistsError Raised when trying to create a file or directory which already exists. - Corresponds to :c:data:`errno` :py:data:`~errno.EEXIST`. + Corresponds to :c:data:`errno` :py:const:`~errno.EEXIST`. .. exception:: FileNotFoundError Raised when a file or directory is requested but doesn't exist. - Corresponds to :c:data:`errno` :py:data:`~errno.ENOENT`. + Corresponds to :c:data:`errno` :py:const:`~errno.ENOENT`. .. exception:: InterruptedError Raised when a system call is interrupted by an incoming signal. - Corresponds to :c:data:`errno` :py:data:`~errno.EINTR`. + Corresponds to :c:data:`errno` :py:const:`~errno.EINTR`. .. versionchanged:: 3.5 Python now retries system calls when a syscall is interrupted by a @@ -732,7 +732,7 @@ depending on the system error code. Raised when a file operation (such as :func:`os.remove`) is requested on a directory. - Corresponds to :c:data:`errno` :py:data:`~errno.EISDIR`. + Corresponds to :c:data:`errno` :py:const:`~errno.EISDIR`. .. exception:: NotADirectoryError @@ -740,28 +740,28 @@ depending on the system error code. something which is not a directory. On most POSIX platforms, it may also be raised if an operation attempts to open or traverse a non-directory file as if it were a directory. - Corresponds to :c:data:`errno` :py:data:`~errno.ENOTDIR`. + Corresponds to :c:data:`errno` :py:const:`~errno.ENOTDIR`. .. exception:: PermissionError Raised when trying to run an operation without the adequate access rights - for example filesystem permissions. - Corresponds to :c:data:`errno` :py:data:`~errno.EACCES`, - :py:data:`~errno.EPERM`, and :py:data:`~errno.ENOTCAPABLE`. + Corresponds to :c:data:`errno` :py:const:`~errno.EACCES`, + :py:const:`~errno.EPERM`, and :py:const:`~errno.ENOTCAPABLE`. .. versionchanged:: 3.11.1 - WASI's :py:data:`~errno.ENOTCAPABLE` is now mapped to + WASI's :py:const:`~errno.ENOTCAPABLE` is now mapped to :exc:`PermissionError`. .. exception:: ProcessLookupError Raised when a given process doesn't exist. - Corresponds to :c:data:`errno` :py:data:`~errno.ESRCH`. + Corresponds to :c:data:`errno` :py:const:`~errno.ESRCH`. .. exception:: TimeoutError Raised when a system function timed out at the system level. - Corresponds to :c:data:`errno` :py:data:`~errno.ETIMEDOUT`. + Corresponds to :c:data:`errno` :py:const:`~errno.ETIMEDOUT`. .. versionadded:: 3.3 All the above :exc:`OSError` subclasses were added. diff --git a/Doc/library/fcntl.rst b/Doc/library/fcntl.rst index 997c7ea571fc0..5a27646a96f59 100644 --- a/Doc/library/fcntl.rst +++ b/Doc/library/fcntl.rst @@ -172,9 +172,9 @@ The module defines the following functions: which the lock starts, relative to *whence*, and *whence* is as with :func:`io.IOBase.seek`, specifically: - * :const:`0` -- relative to the start of the file (:data:`os.SEEK_SET`) - * :const:`1` -- relative to the current buffer position (:data:`os.SEEK_CUR`) - * :const:`2` -- relative to the end of the file (:data:`os.SEEK_END`) + * :const:`0` -- relative to the start of the file (:const:`os.SEEK_SET`) + * :const:`1` -- relative to the current buffer position (:const:`os.SEEK_CUR`) + * :const:`2` -- relative to the end of the file (:const:`os.SEEK_END`) The default for *start* is 0, which means to start at the beginning of the file. The default for *len* is 0 which means to lock to the end of the file. The @@ -201,7 +201,7 @@ using the :func:`flock` call may be better. .. seealso:: Module :mod:`os` - If the locking flags :data:`~os.O_SHLOCK` and :data:`~os.O_EXLOCK` are + If the locking flags :const:`~os.O_SHLOCK` and :const:`~os.O_EXLOCK` are present in the :mod:`os` module (on BSD only), the :func:`os.open` function provides an alternative to the :func:`lockf` and :func:`flock` functions. diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst index b90ead91933ab..d1fe6414ea020 100644 --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -105,7 +105,7 @@ The module defines the following items: .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionchanged:: 3.9 If the *timeout* parameter is set to be zero, it will raise a @@ -441,7 +441,7 @@ FTP_TLS Objects .. versionchanged:: 3.4 The method now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. method:: FTP_TLS.ccc() diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst index b9ceab699cef6..c46314fc5e253 100644 --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -83,7 +83,7 @@ The module provides the following classes: .. versionchanged:: 3.2 This class now supports HTTPS virtual hosts if possible (that is, - if :data:`ssl.HAS_SNI` is true). + if :const:`ssl.HAS_SNI` is true). .. versionchanged:: 3.4 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index 59d7711f9cbd3..1f774e64b0eae 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -106,7 +106,7 @@ There's also a subclass for secure connections: .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionchanged:: 3.9 The optional *timeout* parameter was added. @@ -503,7 +503,7 @@ An :class:`IMAP4` instance has the following methods: .. versionchanged:: 3.4 The method now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. method:: IMAP4.status(mailbox, names) diff --git a/Doc/library/io.rst b/Doc/library/io.rst index c9249da1c3c3d..7eec1f87583b8 100644 --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -423,7 +423,7 @@ I/O Base Classes .. versionadded:: 3.3 Some operating systems could support additional values, like - :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. The valid values + :const:`os.SEEK_HOLE` or :const:`os.SEEK_DATA`. The valid values for a file could depend on it being open in text or binary mode. .. method:: seekable() diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 8454296b815b4..2efc08f130af3 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -2707,7 +2707,7 @@ handler type) for messages from different processes to get mixed up. Returns the logger used by :mod:`multiprocessing`. If necessary, a new one will be created. - When first created the logger has level :data:`logging.NOTSET` and no + When first created the logger has level :const:`logging.NOTSET` and no default handler. Messages sent to this logger will not by default propagate to the root logger. diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index 01177a04ab434..015e83ed2ce5f 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -813,7 +813,7 @@ The first step in using :mod:`optparse` is to create an OptionParser instance. help option. When :mod:`optparse` prints the usage string, it expands ``%prog`` to ``os.path.basename(sys.argv[0])`` (or to ``prog`` if you passed that keyword argument). To suppress a usage message, pass the - special value :data:`optparse.SUPPRESS_USAGE`. + special value :const:`optparse.SUPPRESS_USAGE`. ``option_list`` (default: ``[]``) A list of Option objects to populate the parser with. The options in @@ -1079,7 +1079,7 @@ relevant to a particular option, or fail to pass a required option attribute, Help text to print for this option when listing all available options after the user supplies a :attr:`~Option.help` option (such as ``--help``). If no help text is supplied, the option will be listed without help text. To - hide this option, use the special value :data:`optparse.SUPPRESS_HELP`. + hide this option, use the special value :const:`optparse.SUPPRESS_HELP`. .. attribute:: Option.metavar @@ -1251,7 +1251,7 @@ must specify for any option using that action. If no :attr:`~Option.help` string is supplied for an option, it will still be listed in the help message. To omit an option entirely, use the special value - :data:`optparse.SUPPRESS_HELP`. + :const:`optparse.SUPPRESS_HELP`. :mod:`optparse` automatically adds a :attr:`~Option.help` option to all OptionParsers, so you do not normally need to create one. @@ -1522,7 +1522,7 @@ OptionParser supports several other public methods: Set the usage string according to the rules described above for the ``usage`` constructor keyword argument. Passing ``None`` sets the default usage - string; use :data:`optparse.SUPPRESS_USAGE` to suppress a usage message. + string; use :const:`optparse.SUPPRESS_USAGE` to suppress a usage message. .. method:: OptionParser.print_usage(file=None) diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 43c8f50e3c186..127d1616388b3 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -233,7 +233,7 @@ process and user. :data:`environ` and :data:`environb` are synchronized (modifying :data:`environb` updates :data:`environ`, and vice versa). - :data:`environb` is only available if :data:`supports_bytes_environ` is + :data:`environb` is only available if :const:`supports_bytes_environ` is ``True``. .. versionadded:: 3.2 @@ -331,7 +331,7 @@ process and user. future environment changes. - :func:`getenvb` is only available if :data:`supports_bytes_environ` + :func:`getenvb` is only available if :const:`supports_bytes_environ` is ``True``. .. availability:: Unix. @@ -923,7 +923,7 @@ as internal buffering of data. In Linux kernel older than 5.3, the files pointed by *src* and *dst* must reside in the same filesystem, otherwise an :exc:`OSError` is - raised with :attr:`~OSError.errno` set to :data:`errno.EXDEV`. + raised with :attr:`~OSError.errno` set to :const:`errno.EXDEV`. This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, @@ -1181,7 +1181,7 @@ as internal buffering of data. .. versionadded:: 3.3 Some operating systems could support additional values, like - :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. + :const:`os.SEEK_HOLE` or :const:`os.SEEK_DATA`. .. function:: open(path, flags, mode=0o777, *, dir_fd=None) @@ -1422,7 +1422,7 @@ or `the MSDN `_ on Windo If some data was successfully read, it will return the number of bytes read. If no bytes were read, it will return ``-1`` and set errno to - :data:`errno.EAGAIN`. + :const:`errno.EAGAIN`. .. availability:: Linux >= 4.14. @@ -1627,7 +1627,7 @@ or `the MSDN `_ on Windo *offset_dst*. The offset associated to the file descriptor that refers to a pipe must be ``None``. The files pointed by *src* and *dst* must reside in the same filesystem, otherwise an :exc:`OSError` is raised with - :attr:`~OSError.errno` set to :data:`errno.EXDEV`. + :attr:`~OSError.errno` set to :const:`errno.EXDEV`. This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, @@ -1960,18 +1960,18 @@ features: Set the flags of *path* to the numeric *flags*. *flags* may take a combination (bitwise OR) of the following values (as defined in the :mod:`stat` module): - * :data:`stat.UF_NODUMP` - * :data:`stat.UF_IMMUTABLE` - * :data:`stat.UF_APPEND` - * :data:`stat.UF_OPAQUE` - * :data:`stat.UF_NOUNLINK` - * :data:`stat.UF_COMPRESSED` - * :data:`stat.UF_HIDDEN` - * :data:`stat.SF_ARCHIVED` - * :data:`stat.SF_IMMUTABLE` - * :data:`stat.SF_APPEND` - * :data:`stat.SF_NOUNLINK` - * :data:`stat.SF_SNAPSHOT` + * :const:`stat.UF_NODUMP` + * :const:`stat.UF_IMMUTABLE` + * :const:`stat.UF_APPEND` + * :const:`stat.UF_OPAQUE` + * :const:`stat.UF_NOUNLINK` + * :const:`stat.UF_COMPRESSED` + * :const:`stat.UF_HIDDEN` + * :const:`stat.SF_ARCHIVED` + * :const:`stat.SF_IMMUTABLE` + * :const:`stat.SF_APPEND` + * :const:`stat.SF_NOUNLINK` + * :const:`stat.SF_SNAPSHOT` This function can support :ref:`not following symlinks `. @@ -1992,25 +1992,25 @@ features: following values (as defined in the :mod:`stat` module) or bitwise ORed combinations of them: - * :data:`stat.S_ISUID` - * :data:`stat.S_ISGID` - * :data:`stat.S_ENFMT` - * :data:`stat.S_ISVTX` - * :data:`stat.S_IREAD` - * :data:`stat.S_IWRITE` - * :data:`stat.S_IEXEC` - * :data:`stat.S_IRWXU` - * :data:`stat.S_IRUSR` - * :data:`stat.S_IWUSR` - * :data:`stat.S_IXUSR` - * :data:`stat.S_IRWXG` - * :data:`stat.S_IRGRP` - * :data:`stat.S_IWGRP` - * :data:`stat.S_IXGRP` - * :data:`stat.S_IRWXO` - * :data:`stat.S_IROTH` - * :data:`stat.S_IWOTH` - * :data:`stat.S_IXOTH` + * :const:`stat.S_ISUID` + * :const:`stat.S_ISGID` + * :const:`stat.S_ENFMT` + * :const:`stat.S_ISVTX` + * :const:`stat.S_IREAD` + * :const:`stat.S_IWRITE` + * :const:`stat.S_IEXEC` + * :const:`stat.S_IRWXU` + * :const:`stat.S_IRUSR` + * :const:`stat.S_IWUSR` + * :const:`stat.S_IXUSR` + * :const:`stat.S_IRWXG` + * :const:`stat.S_IRGRP` + * :const:`stat.S_IWGRP` + * :const:`stat.S_IXGRP` + * :const:`stat.S_IRWXO` + * :const:`stat.S_IROTH` + * :const:`stat.S_IWOTH` + * :const:`stat.S_IXOTH` This function can support :ref:`specifying a file descriptor `, :ref:`paths relative to directory descriptors ` and :ref:`not @@ -4151,8 +4151,8 @@ written in Python, such as a mail server's external command delivery program. Send signal *sig* to the process *pid*. Constants for the specific signals available on the host platform are defined in the :mod:`signal` module. - Windows: The :data:`signal.CTRL_C_EVENT` and - :data:`signal.CTRL_BREAK_EVENT` signals are special signals which can + Windows: The :const:`signal.CTRL_C_EVENT` and + :const:`signal.CTRL_BREAK_EVENT` signals are special signals which can only be sent to console processes which share a common console window, e.g., some subprocesses. Any other value for *sig* will cause the process to be unconditionally killed by the TerminateProcess API, and the exit code @@ -4205,7 +4205,7 @@ written in Python, such as a mail server's external command delivery program. This flag indicates that the file descriptor will be non-blocking. If the process referred to by the file descriptor has not yet terminated, then an attempt to wait on the file descriptor using :manpage:`waitid(2)` - will immediately return the error :data:`~errno.EAGAIN` rather than blocking. + will immediately return the error :const:`~errno.EAGAIN` rather than blocking. .. availability:: Linux >= 5.10 .. versionadded:: 3.12 @@ -4654,7 +4654,7 @@ written in Python, such as a mail server's external command delivery program. * :attr:`!si_pid` (process ID) * :attr:`!si_uid` (real user ID of the child) - * :attr:`!si_signo` (always :data:`~signal.SIGCHLD`) + * :attr:`!si_signo` (always :const:`~signal.SIGCHLD`) * :attr:`!si_status` (the exit status or signal number, depending on :attr:`!si_code`) * :attr:`!si_code` (see :data:`CLD_EXITED` for possible values) @@ -4892,7 +4892,7 @@ used to determine the disposition of a process. .. function:: WIFCONTINUED(status) Return ``True`` if a stopped child has been resumed by delivery of - :data:`~signal.SIGCONT` (if the process has been continued from a job + :const:`~signal.SIGCONT` (if the process has been continued from a job control stop), otherwise return ``False``. See :data:`WCONTINUED` option. @@ -5264,7 +5264,7 @@ Random numbers ``/dev/urandom`` devices. The flags argument is a bit mask that can contain zero or more of the - following values ORed together: :py:data:`os.GRND_RANDOM` and + following values ORed together: :py:const:`os.GRND_RANDOM` and :py:data:`GRND_NONBLOCK`. See also the `Linux getrandom() manual page diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index 260c4a63d1203..fbd5e150b4cd5 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -77,7 +77,7 @@ The :mod:`poplib` module provides two classes: .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionchanged:: 3.9 If the *timeout* parameter is set to be zero, it will raise a @@ -240,7 +240,7 @@ A :class:`POP3` instance has the following methods: This method supports hostname checking via :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionadded:: 3.4 diff --git a/Doc/library/shelve.rst b/Doc/library/shelve.rst index dc87af398ed75..01314f491f47a 100644 --- a/Doc/library/shelve.rst +++ b/Doc/library/shelve.rst @@ -25,7 +25,7 @@ lots of shared sub-objects. The keys are ordinary strings. database file is opened for reading and writing. The optional *flag* parameter has the same interpretation as the *flag* parameter of :func:`dbm.open`. - By default, pickles created with :data:`pickle.DEFAULT_PROTOCOL` are used + By default, pickles created with :const:`pickle.DEFAULT_PROTOCOL` are used to serialize values. The version of the pickle protocol can be specified with the *protocol* parameter. @@ -42,7 +42,7 @@ lots of shared sub-objects. The keys are ordinary strings. mutated). .. versionchanged:: 3.10 - :data:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle + :const:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle protocol. .. versionchanged:: 3.11 @@ -119,7 +119,7 @@ Restrictions A subclass of :class:`collections.abc.MutableMapping` which stores pickled values in the *dict* object. - By default, pickles created with :data:`pickle.DEFAULT_PROTOCOL` are used + By default, pickles created with :const:`pickle.DEFAULT_PROTOCOL` are used to serialize values. The version of the pickle protocol can be specified with the *protocol* parameter. See the :mod:`pickle` documentation for a discussion of the pickle protocols. @@ -143,7 +143,7 @@ Restrictions Added context manager support. .. versionchanged:: 3.10 - :data:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle + :const:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle protocol. diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index f90274feb6bf9..aaec2aa1ef1db 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -98,7 +98,7 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions). .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionchanged:: 3.9 If the *timeout* parameter is set to be zero, it will raise a @@ -418,7 +418,7 @@ An :class:`SMTP` instance has the following methods: .. versionchanged:: 3.4 The method now supports hostname check with :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see - :data:`~ssl.HAS_SNI`). + :const:`~ssl.HAS_SNI`). .. versionchanged:: 3.5 The error raised for lack of STARTTLS support is now the diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index f2408cb95ff31..4f220e8a09897 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -2252,7 +2252,7 @@ This is because the previous execution has left the socket in a ``TIME_WAIT`` state, and can't be immediately reused. There is a :mod:`socket` flag to set, in order to prevent this, -:data:`socket.SO_REUSEADDR`:: +:const:`socket.SO_REUSEADDR`:: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 356888d64b887..50344058c2604 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -299,7 +299,7 @@ Module functions Can be ``"DEFERRED"`` (default), ``"EXCLUSIVE"`` or ``"IMMEDIATE"``; or ``None`` to disable opening transactions implicitly. Has no effect unless :attr:`Connection.autocommit` is set to - :data:`~sqlite3.LEGACY_TRANSACTION_CONTROL` (the default). + :const:`~sqlite3.LEGACY_TRANSACTION_CONTROL` (the default). :type isolation_level: str | None :param bool check_same_thread: @@ -334,7 +334,7 @@ Module functions See :attr:`Connection.autocommit` and :ref:`sqlite3-transaction-control-autocommit` for more information. *autocommit* currently defaults to - :data:`~sqlite3.LEGACY_TRANSACTION_CONTROL`. + :const:`~sqlite3.LEGACY_TRANSACTION_CONTROL`. The default will change to ``False`` in a future Python release. :type autocommit: bool @@ -1818,9 +1818,9 @@ Blob objects .. method:: seek(offset, origin=os.SEEK_SET, /) Set the current access position of the blob to *offset*. The *origin* - argument defaults to :data:`os.SEEK_SET` (absolute blob positioning). - Other values for *origin* are :data:`os.SEEK_CUR` (seek relative to the - current position) and :data:`os.SEEK_END` (seek relative to the blob?s + argument defaults to :const:`os.SEEK_SET` (absolute blob positioning). + Other values for *origin* are :const:`os.SEEK_CUR` (seek relative to the + current position) and :const:`os.SEEK_END` (seek relative to the blob?s end). diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 18a6c5ab4858a..7f09552632768 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -139,7 +139,7 @@ purposes. The settings are: :data:`PROTOCOL_TLS_CLIENT` or :data:`PROTOCOL_TLS_SERVER`, :data:`OP_NO_SSLv2`, and :data:`OP_NO_SSLv3` with high encryption cipher suites without RC4 and - without unauthenticated cipher suites. Passing :data:`~Purpose.SERVER_AUTH` + without unauthenticated cipher suites. Passing :const:`~Purpose.SERVER_AUTH` as *purpose* sets :data:`~SSLContext.verify_mode` to :data:`CERT_REQUIRED` and either loads CA certificates (when at least one of *cafile*, *capath* or *cadata* is given) or uses :meth:`SSLContext.load_default_certs` to load @@ -1484,9 +1484,9 @@ to speed up repeated connections from the same clients. load CA certificates from other locations, too. The *purpose* flag specifies what kind of CA certificates are loaded. The - default settings :data:`Purpose.SERVER_AUTH` loads certificates, that are + default settings :const:`Purpose.SERVER_AUTH` loads certificates, that are flagged and trusted for TLS web server authentication (client side - sockets). :data:`Purpose.CLIENT_AUTH` loads CA certificates for client + sockets). :const:`Purpose.CLIENT_AUTH` loads CA certificates for client certificate verification on the server side. .. versionadded:: 3.4 @@ -1729,7 +1729,7 @@ to speed up repeated connections from the same clients. Wrap an existing Python socket *sock* and return an instance of :attr:`SSLContext.sslsocket_class` (default :class:`SSLSocket`). The returned SSL socket is tied to the context, its settings and certificates. - *sock* must be a :data:`~socket.SOCK_STREAM` socket; other + *sock* must be a :const:`~socket.SOCK_STREAM` socket; other socket types are unsupported. The parameter ``server_side`` is a boolean which identifies whether diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 12ba93d9e1847..c01d2d4527152 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -697,7 +697,7 @@ always available. Return the current value of the flags that are used for :c:func:`dlopen` calls. Symbolic names for the flag values can be found in the :mod:`os` module (``RTLD_xxx`` constants, e.g. - :data:`os.RTLD_LAZY`). + :const:`os.RTLD_LAZY`). .. availability:: Unix. @@ -1368,7 +1368,7 @@ always available. ``sys.setdlopenflags(0)``. To share symbols across extension modules, call as ``sys.setdlopenflags(os.RTLD_GLOBAL)``. Symbolic names for the flag values can be found in the :mod:`os` module (``RTLD_xxx`` constants, e.g. - :data:`os.RTLD_LAZY`). + :const:`os.RTLD_LAZY`). .. availability:: Unix. diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index fd4c294613fd3..097f7087eccab 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -59,7 +59,7 @@ The module defines the following user-callable items: platforms, it is a file-like object whose :attr:`!file` attribute is the underlying true file object. - The :py:data:`os.O_TMPFILE` flag is used if it is available and works + The :py:const:`os.O_TMPFILE` flag is used if it is available and works (Linux-specific, requires Linux kernel 3.11 or later). On platforms that are neither Posix nor Cygwin, TemporaryFile is an alias @@ -69,7 +69,7 @@ The module defines the following user-callable items: .. versionchanged:: 3.5 - The :py:data:`os.O_TMPFILE` flag is now used if available. + The :py:const:`os.O_TMPFILE` flag is now used if available. .. versionchanged:: 3.8 Added *errors* parameter. diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 35cd6d5233c03..de60151bb32ce 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -472,7 +472,7 @@ The :mod:`test.support` module defines the following functions: .. function:: with_pymalloc() - Return :data:`_testcapi.WITH_PYMALLOC`. + Return :const:`_testcapi.WITH_PYMALLOC`. .. function:: requires(resource, msg=None) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 6d5f17d1c2c5c..836fd42ab71c8 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -2485,7 +2485,7 @@ behaviour you can switch it off by setting the module level switch Alternatively you can just use ``vars(my_mock)`` (instance members) and ``dir(type(my_mock))`` (type members) to bypass the filtering irrespective of -:data:`mock.FILTER_DIR`. +:const:`mock.FILTER_DIR`. mock_open diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst index 7e79871bbd607..a672d8753b7f2 100644 --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -91,7 +91,7 @@ The :mod:`urllib.request` module defines the following functions: .. versionchanged:: 3.2 HTTPS virtual hosts are now supported if possible (that is, if - :data:`ssl.HAS_SNI` is true). + :const:`ssl.HAS_SNI` is true). .. versionadded:: 3.2 *data* can be an iterable object. diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index 20b0905bb1093..cf30de67719b9 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -73,7 +73,7 @@ decompression bomb Safe Safe Safe 1. Expat 2.4.1 and newer is not vulnerable to the "billion laughs" and "quadratic blowup" vulnerabilities. Items still listed as vulnerable due to potential reliance on system-provided libraries. Check - :data:`pyexpat.EXPAT_VERSION`. + :const:`pyexpat.EXPAT_VERSION`. 2. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a :exc:`ParserError` when an entity occurs. 3. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index fbe280d641317..924e73dc54da2 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -97,7 +97,7 @@ General Options .. cmdoption:: --with-tzpath= - Select the default time zone search path for :data:`zoneinfo.TZPATH`. + Select the default time zone search path for :const:`zoneinfo.TZPATH`. See the :ref:`Compile-time configuration ` of the :mod:`zoneinfo` module. @@ -112,7 +112,7 @@ General Options Build the ``_decimal`` extension module using a thread-local context rather than a coroutine-local context (default), see the :mod:`decimal` module. - See :data:`decimal.HAVE_CONTEXTVAR` and the :mod:`contextvars` module. + See :const:`decimal.HAVE_CONTEXTVAR` and the :mod:`contextvars` module. .. versionadded:: 3.9 diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index 1cc6ebd0a6ef0..7bc68b0cc4e66 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -1556,9 +1556,9 @@ changes, or look through the Subversion logs for all the details. :issue:`8484`.) The version of OpenSSL being used is now available as the module - attributes :data:`ssl.OPENSSL_VERSION` (a string), - :data:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and - :data:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Added by Antoine + attributes :const:`ssl.OPENSSL_VERSION` (a string), + :const:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and + :const:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Added by Antoine Pitrou; :issue:`8321`.) * The :mod:`struct` module will no longer silently ignore overflow diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index b65e73ce353a5..d1d2193de4a95 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1253,8 +1253,8 @@ descriptors without copying between kernel address space and user address space, where one of the file descriptors must refer to a pipe. (Contributed by Pablo Galindo in :issue:`41625`.) -Add :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` -and :data:`~os.O_NOFOLLOW_ANY` for macOS. +Add :const:`~os.O_EVTONLY`, :const:`~os.O_FSYNC`, :const:`~os.O_SYMLINK` +and :const:`~os.O_NOFOLLOW_ANY` for macOS. (Contributed by Dong-hee Na in :issue:`43106`.) os.path @@ -1319,7 +1319,7 @@ objects in the tree returned by :func:`pyclbr.readline` and shelve ------ -The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default +The :mod:`shelve` module now uses :const:`pickle.DEFAULT_PROTOCOL` by default instead of :mod:`pickle` protocol ``3`` when creating shelves. (Contributed by Zackery Spytz in :issue:`34204`.) @@ -1356,7 +1356,7 @@ The ssl module requires OpenSSL 1.1.1 or newer. (Contributed by Christian Heimes in :pep:`644` and :issue:`43669`.) The ssl module has preliminary support for OpenSSL 3.0.0 and new option -:data:`~ssl.OP_IGNORE_UNEXPECTED_EOF`. +:const:`~ssl.OP_IGNORE_UNEXPECTED_EOF`. (Contributed by Christian Heimes in :issue:`38820`, :issue:`43794`, :issue:`43788`, :issue:`43791`, :issue:`43799`, :issue:`43920`, :issue:`43789`, and :issue:`43811`.) @@ -1387,7 +1387,7 @@ Add a *timeout* parameter to the :func:`ssl.get_server_certificate` function. The ssl module uses heap-types and multi-phase initialization. (Contributed by Christian Heimes in :issue:`42333`.) -A new verify flag :data:`~ssl.VERIFY_X509_PARTIAL_CHAIN` has been added. +A new verify flag :const:`~ssl.VERIFY_X509_PARTIAL_CHAIN` has been added. (Contributed by l0x in :issue:`40849`.) sqlite3 @@ -1413,7 +1413,7 @@ _thread ------- :func:`_thread.interrupt_main` now takes an optional signal number to -simulate (the default is still :data:`signal.SIGINT`). +simulate (the default is still :const:`signal.SIGINT`). (Contributed by Antoine Pitrou in :issue:`43356`.) threading @@ -1757,8 +1757,8 @@ Deprecated * :data:`~ssl.PROTOCOL_SSLv2`, :data:`~ssl.PROTOCOL_SSLv3`, :data:`~ssl.PROTOCOL_SSLv23`, :data:`~ssl.PROTOCOL_TLSv1`, :data:`~ssl.PROTOCOL_TLSv1_1`, :data:`~ssl.PROTOCOL_TLSv1_2`, and - :data:`~ssl.PROTOCOL_TLS` are deprecated in favor of - :data:`~ssl.PROTOCOL_TLS_CLIENT` and :data:`~ssl.PROTOCOL_TLS_SERVER` + :const:`~ssl.PROTOCOL_TLS` are deprecated in favor of + :const:`~ssl.PROTOCOL_TLS_CLIENT` and :const:`~ssl.PROTOCOL_TLS_SERVER` * :func:`~ssl.wrap_socket` is replaced by :meth:`ssl.SSLContext.wrap_socket` diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 859ee6e31649f..bae78c1d9c205 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -690,7 +690,7 @@ enum * Added the :func:`~enum.global_enum` enum decorator, which adjusts :meth:`~object.__repr__` and :meth:`~object.__str__` to show values as members of their module rather than the enum class. - For example, ``'re.ASCII'`` for the :data:`~re.ASCII` member + For example, ``'re.ASCII'`` for the :const:`~re.ASCII` member of :class:`re.RegexFlag` rather than ``'RegexFlag.ASCII'``. * Enhanced :class:`~enum.Flag` to support @@ -1063,8 +1063,8 @@ threading * On Unix, if the ``sem_clockwait()`` function is available in the C library (glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses - the monotonic clock (:data:`time.CLOCK_MONOTONIC`) for the timeout, rather - than using the system clock (:data:`time.CLOCK_REALTIME`), to not be affected + the monotonic clock (:const:`time.CLOCK_MONOTONIC`) for the timeout, rather + than using the system clock (:const:`time.CLOCK_REALTIME`), to not be affected by system clock changes. (Contributed by Victor Stinner in :issue:`41710`.) @@ -1812,7 +1812,7 @@ Standard Library (Contributed by Serhiy Storchaka in :gh:`91760`.) * In the :mod:`re` module, the :func:`!re.template` function - and the corresponding :data:`!re.TEMPLATE` and :data:`!re.T` flags + and the corresponding :const:`!re.TEMPLATE` and :const:`!re.T` flags are deprecated, as they were undocumented and lacked an obvious purpose. They will be removed in Python 3.13. (Contributed by Serhiy Storchaka and Miro Hron?ok in :gh:`92728`.) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 947ba1a35c12a..1e35427c497c8 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -555,7 +555,7 @@ calendar csv --- -* Add :data:`~csv.QUOTE_NOTNULL` and :data:`~csv.QUOTE_STRINGS` flags to +* Add :const:`~csv.QUOTE_NOTNULL` and :const:`~csv.QUOTE_STRINGS` flags to provide finer grained control of ``None`` and empty strings by :class:`~csv.writer` objects. @@ -612,7 +612,7 @@ math os -- -* Add :data:`os.PIDFD_NONBLOCK` to open a file descriptor +* Add :const:`os.PIDFD_NONBLOCK` to open a file descriptor for a process with :func:`os.pidfd_open` in non-blocking mode. (Contributed by Kumar Aditya in :gh:`93312`.) diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index 4275503a8633b..b9c6541795e44 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -1666,9 +1666,9 @@ for secure (encrypted, authenticated) internet connections: algorithm" error. * The version of OpenSSL being used is now accessible using the module - attributes :data:`ssl.OPENSSL_VERSION` (a string), - :data:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and - :data:`ssl.OPENSSL_VERSION_NUMBER` (an integer). + attributes :const:`ssl.OPENSSL_VERSION` (a string), + :const:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and + :const:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Contributed by Antoine Pitrou in :issue:`8850`, :issue:`1589`, :issue:`8322`, :issue:`5639`, :issue:`4870`, :issue:`8484`, and :issue:`8321`.) diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index c108510b2aebd..dbac74167a126 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -842,7 +842,7 @@ Builtin functions and types * :func:`open` gets a new *opener* parameter: the underlying file descriptor for the file object is then obtained by calling *opener* with (*file*, - *flags*). It can be used to use custom flags like :data:`os.O_CLOEXEC` for + *flags*). It can be used to use custom flags like :const:`os.O_CLOEXEC` for example. The ``'x'`` mode was added: open for exclusive creation, failing if the file already exists. * :func:`print`: added the *flush* keyword argument. If the *flush* keyword @@ -1127,7 +1127,7 @@ Features * If Python is compiled without threads, the C version automatically disables the expensive thread local context machinery. In this case, - the variable :data:`~decimal.HAVE_THREADS` is set to ``False``. + the variable :const:`~decimal.HAVE_THREADS` is set to ``False``. API changes ~~~~~~~~~~~ @@ -1576,8 +1576,8 @@ os -- * The :mod:`os` module has a new :func:`~os.pipe2` function that makes it - possible to create a pipe with :data:`~os.O_CLOEXEC` or - :data:`~os.O_NONBLOCK` flags set atomically. This is especially useful to + possible to create a pipe with :const:`~os.O_CLOEXEC` or + :const:`~os.O_NONBLOCK` flags set atomically. This is especially useful to avoid race conditions in multi-threaded programs. * The :mod:`os` module has a new :func:`~os.sendfile` function which provides @@ -1691,9 +1691,9 @@ os * Some platforms now support additional constants for the :func:`~os.lseek` function, such as ``os.SEEK_HOLE`` and ``os.SEEK_DATA``. -* New constants :data:`~os.RTLD_LAZY`, :data:`~os.RTLD_NOW`, - :data:`~os.RTLD_GLOBAL`, :data:`~os.RTLD_LOCAL`, :data:`~os.RTLD_NODELETE`, - :data:`~os.RTLD_NOLOAD`, and :data:`~os.RTLD_DEEPBIND` are available on +* New constants :const:`~os.RTLD_LAZY`, :const:`~os.RTLD_NOW`, + :const:`~os.RTLD_GLOBAL`, :const:`~os.RTLD_LOCAL`, :const:`~os.RTLD_NODELETE`, + :const:`~os.RTLD_NOLOAD`, and :const:`~os.RTLD_DEEPBIND` are available on platforms that support them. These are for use with the :func:`sys.setdlopenflags` function, and supersede the similar constants defined in :mod:`ctypes` and :mod:`DLFCN`. (Contributed by Victor Stinner @@ -1995,7 +1995,7 @@ subprocess Command strings can now be bytes objects on posix platforms. (Contributed by Victor Stinner in :issue:`8513`.) -A new constant :data:`~subprocess.DEVNULL` allows suppressing output in a +A new constant :const:`~subprocess.DEVNULL` allows suppressing output in a platform-independent fashion. (Contributed by Ross Lagerwall in :issue:`5870`.) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst index e753c3d364c2b..794271f3c32b8 100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -775,7 +775,7 @@ of a given opcode and argument, information that is not otherwise available. doctest ------- -A new :ref:`option flag `, :data:`~doctest.FAIL_FAST`, halts +A new :ref:`option flag `, :const:`~doctest.FAIL_FAST`, halts test running as soon as the first failure is detected. (Contributed by R. David Murray and Daniel Urban in :issue:`16522`.) @@ -841,7 +841,7 @@ for example, if the file might have been changed and re-checked in less time than the resolution of a particular filesystem's file modification time field. (Contributed by Mark Levitt in :issue:`18149`.) -New module attribute :data:`~filecmp.DEFAULT_IGNORES` provides the list of +New module attribute :const:`~filecmp.DEFAULT_IGNORES` provides the list of directories that are used as the default value for the *ignore* parameter of the :func:`~filecmp.dircmp` function. (Contributed by Eli Bendersky in :issue:`15442`.) @@ -1189,7 +1189,7 @@ Windows). (Contributed by Brian Curtin in :issue:`11939`.) root on Windows. (Contributed by Tim Golden in :issue:`9035`.) :func:`os.open` supports two new flags on platforms that provide them, -:data:`~os.O_PATH` (un-opened file descriptor), and :data:`~os.O_TMPFILE` +:const:`~os.O_PATH` (un-opened file descriptor), and :const:`~os.O_TMPFILE` (unnamed temporary file; as of 3.4.0 release available only on Linux systems with a kernel version of 3.11 or newer that have uapi headers). (Contributed by Christian Heimes in :issue:`18673` and Benjamin Peterson, respectively.) @@ -1238,8 +1238,8 @@ plistlib stdlib serialization protocols, with new :func:`~plistlib.load`, :func:`~plistlib.dump`, :func:`~plistlib.loads`, and :func:`~plistlib.dumps` functions. (The older API is now deprecated.) In addition to the already -supported XML plist format (:data:`~plistlib.FMT_XML`), it also now supports -the binary plist format (:data:`~plistlib.FMT_BINARY`). (Contributed by Ronald +supported XML plist format (:const:`~plistlib.FMT_XML`), it also now supports +the binary plist format (:const:`~plistlib.FMT_BINARY`). (Contributed by Ronald Oussoren and others in :issue:`14455`.) @@ -1388,7 +1388,7 @@ try/except statement by code that only cares whether or not an error occurred. socket ------ -The socket module now supports the :data:`~socket.CAN_BCM` protocol on +The socket module now supports the :const:`~socket.CAN_BCM` protocol on platforms that support it. (Contributed by Brian Thorne in :issue:`15359`.) Socket objects have new methods to get or set their :ref:`inheritable flag @@ -1399,7 +1399,7 @@ The ``socket.AF_*`` and ``socket.SOCK_*`` constants are now enumeration values using the new :mod:`enum` module. This allows meaningful names to be printed during debugging, instead of integer "magic numbers". -The :data:`~socket.AF_LINK` constant is now available on BSD and OSX. +The :const:`~socket.AF_LINK` constant is now available on BSD and OSX. :func:`~socket.inet_pton` and :func:`~socket.inet_ntop` are now supported on Windows. (Contributed by Atsuo Ishimoto in :issue:`7171`.) @@ -1460,8 +1460,8 @@ Heimes in :issue:`18147`.) If OpenSSL 0.9.8 or later is available, :class:`~ssl.SSLContext` has a new attribute :attr:`~ssl.SSLContext.verify_flags` that can be used to control the certificate verification process by setting it to some combination of the new -constants :data:`~ssl.VERIFY_DEFAULT`, :data:`~ssl.VERIFY_CRL_CHECK_LEAF`, -:data:`~ssl.VERIFY_CRL_CHECK_CHAIN`, or :data:`~ssl.VERIFY_X509_STRICT`. +constants :const:`~ssl.VERIFY_DEFAULT`, :const:`~ssl.VERIFY_CRL_CHECK_LEAF`, +:const:`~ssl.VERIFY_CRL_CHECK_CHAIN`, or :const:`~ssl.VERIFY_X509_STRICT`. OpenSSL does not do any CRL verification by default. (Contributed by Christien Heimes in :issue:`8813`.) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index ccf71bf08e860..262e4ed32c0a3 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -478,7 +478,7 @@ not make an additional system call:: PEP 475: Retry system calls failing with EINTR ---------------------------------------------- -An :py:data:`errno.EINTR` error code is returned whenever a system call, that +An :py:const:`errno.EINTR` error code is returned whenever a system call, that is waiting for I/O, is interrupted by a signal. Previously, Python would raise :exc:`InterruptedError` in such cases. This meant that, when writing a Python application, the developer had two choices: @@ -527,7 +527,7 @@ by a signal: :func:`~os.writev`; * special cases: :func:`os.close` and :func:`os.dup2` now ignore - :py:data:`~errno.EINTR` errors; the syscall is not retried (see the PEP + :py:const:`~errno.EINTR` errors; the syscall is not retried (see the PEP for the rationale); * :mod:`select` functions: :func:`devpoll.poll() `, @@ -1498,7 +1498,7 @@ use ``/dev/urandom`` and avoiding failures due to potential file descriptor exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow -getting and setting a file descriptor's blocking mode (:data:`~os.O_NONBLOCK`.) +getting and setting a file descriptor's blocking mode (:const:`~os.O_NONBLOCK`.) (Contributed by Victor Stinner in :issue:`22054`.) The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported @@ -1783,7 +1783,7 @@ the TLS handshake. The new :meth:`SSLSocket.selected_alpn_protocol() ` returns the protocol that was selected during the TLS handshake. -The :data:`~ssl.HAS_ALPN` flag indicates whether ALPN support is present. +The :const:`~ssl.HAS_ALPN` flag indicates whether ALPN support is present. Other Changes @@ -2476,7 +2476,7 @@ Changes in the Python API in Python 3.5, all old ``.pyo`` files from previous versions of Python are invalid regardless of this PEP. -* The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES` +* The :mod:`socket` module now exports the :const:`~socket.CAN_RAW_FD_FRAMES` constant on linux 3.6 and greater. * The :func:`ssl.cert_time_to_seconds` function now interprets the input time diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index ed398f2e8c562..4359a9012dd53 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1404,7 +1404,7 @@ socket ------ The :func:`~socket.socket.ioctl` function now supports the -:data:`~socket.SIO_LOOPBACK_FAST_PATH` control code. +:const:`~socket.SIO_LOOPBACK_FAST_PATH` control code. (Contributed by Daniel Stokes in :issue:`26536`.) The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``, @@ -1416,7 +1416,7 @@ The :meth:`~socket.socket.setsockopt` now supports the (Contributed by Christian Heimes in :issue:`27744`.) The socket module now supports the address family -:data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, +:const:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, ``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added. (Contributed by Christian Heimes in :issue:`27744` with support from Victor Stinner.) diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 24244ff17b1ea..09c91fc00dab2 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1280,13 +1280,13 @@ This function should be used instead of :func:`os.close` for better compatibility across platforms. (Contributed by Christian Heimes in :issue:`32454`.) -The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION` -(Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and -:data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. +The :mod:`socket` module now exposes the :const:`socket.TCP_CONGESTION` +(Linux 2.6.13), :const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and +:const:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. (Contributed by Omar Sandoval in :issue:`26273` and Nathaniel J. Smith in :issue:`29728`.) -Support for :data:`socket.AF_VSOCK` sockets has been added to allow +Support for :const:`socket.AF_VSOCK` sockets has been added to allow communication between virtual machines and their hosts. (Contributed by Cathy Avery in :issue:`27584`.) @@ -1394,7 +1394,7 @@ subprocess The :func:`subprocess.run` function accepts the new *capture_output* keyword argument. When true, stdout and stderr will be captured. -This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and +This is equivalent to passing :const:`subprocess.PIPE` as *stdout* and *stderr* arguments. (Contributed by Bo Bayles in :issue:`32102`.) @@ -1453,12 +1453,12 @@ time New clock identifiers have been added: -* :data:`time.CLOCK_BOOTTIME` (Linux): Identical to - :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the +* :const:`time.CLOCK_BOOTTIME` (Linux): Identical to + :const:`time.CLOCK_MONOTONIC`, except it also includes any time that the system is suspended. -* :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution +* :const:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution per-process CPU timer. -* :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is +* :const:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is the time the system has been running and not suspended, providing accurate uptime measurement. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 3efb3f49b77ac..f50c43f724b6b 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1305,7 +1305,7 @@ Zackery Spytz in :issue:`25451`.) time ---- -Added new clock :data:`~time.CLOCK_UPTIME_RAW` for macOS 10.12. +Added new clock :const:`~time.CLOCK_UPTIME_RAW` for macOS 10.12. (Contributed by Joannah Nanjekye in :issue:`35702`.) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index a8f9774af7add..9280f94740005 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -427,8 +427,8 @@ digests. It skips MD5 on platforms that block MD5 digest. fcntl ----- -Added constants :data:`~fcntl.F_OFD_GETLK`, :data:`~fcntl.F_OFD_SETLK` -and :data:`~fcntl.F_OFD_SETLKW`. +Added constants :const:`~fcntl.F_OFD_GETLK`, :const:`~fcntl.F_OFD_SETLK` +and :const:`~fcntl.F_OFD_SETLKW`. (Contributed by Dong-hee Na in :issue:`38602`.) ftplib @@ -593,11 +593,11 @@ a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.) os -- -Added :data:`~os.CLD_KILLED` and :data:`~os.CLD_STOPPED` for :attr:`si_code`. +Added :const:`~os.CLD_KILLED` and :const:`~os.CLD_STOPPED` for :attr:`si_code`. (Contributed by Dong-hee Na in :issue:`38493`.) Exposed the Linux-specific :func:`os.pidfd_open` (:issue:`38692`) and -:data:`os.P_PIDFD` (:issue:`38713`) for process management with file +:const:`os.P_PIDFD` (:issue:`38713`) for process management with file descriptors. The :func:`os.unsetenv` function is now also available on Windows. @@ -669,11 +669,11 @@ a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.) socket ------ -The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_JOIN_FILTERS` +The :mod:`socket` module now exports the :const:`~socket.CAN_RAW_JOIN_FILTERS` constant on Linux 4.1 and greater. (Contributed by Stefan Tatschner and Zackery Spytz in :issue:`25780`.) -The socket module now supports the :data:`~socket.CAN_J1939` protocol on +The socket module now supports the :const:`~socket.CAN_J1939` protocol on platforms that support it. (Contributed by Karl Ding in :issue:`40291`.) The socket module now has the :func:`socket.send_fds` and @@ -1084,7 +1084,7 @@ Changes in the Python API ``__VENV_PROMPT__`` is set to ``""``. * The :meth:`select.epoll.unregister` method no longer ignores the - :data:`~errno.EBADF` error. + :const:`~errno.EBADF` error. (Contributed by Victor Stinner in :issue:`39239`.) * The *compresslevel* parameter of :class:`bz2.BZ2File` became keyword-only, diff --git a/Misc/NEWS.d/3.10.0a1.rst b/Misc/NEWS.d/3.10.0a1.rst index 301612c4307da..572841db7d89d 100644 --- a/Misc/NEWS.d/3.10.0a1.rst +++ b/Misc/NEWS.d/3.10.0a1.rst @@ -2176,7 +2176,7 @@ None. .. nonce: YoYoYo .. section: Library -Add a new :data:`os.RWF_APPEND` flag for :func:`os.pwritev`. +Add a new :const:`os.RWF_APPEND` flag for :func:`os.pwritev`. .. @@ -2304,7 +2304,7 @@ Restored the deprecated :mod:`xml.etree.cElementTree` module. .. nonce: ZCk0_c .. section: Library -:data:`~mmap.MAP_POPULATE` constant has now been added to the list of +:const:`~mmap.MAP_POPULATE` constant has now been added to the list of exported :mod:`mmap` module flags. .. diff --git a/Misc/NEWS.d/3.10.0a2.rst b/Misc/NEWS.d/3.10.0a2.rst index 061a82e90afd6..eeb179a980bb8 100644 --- a/Misc/NEWS.d/3.10.0a2.rst +++ b/Misc/NEWS.d/3.10.0a2.rst @@ -604,7 +604,7 @@ changes the working directory. PR by Anthony Sottile. .. nonce: 9wXTtY .. section: Library -The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default +The :mod:`shelve` module now uses :const:`pickle.DEFAULT_PROTOCOL` by default instead of :mod:`pickle` protocol ``3``. .. diff --git a/Misc/NEWS.d/3.10.0a6.rst b/Misc/NEWS.d/3.10.0a6.rst index 803df6f51ce62..313aa68925404 100644 --- a/Misc/NEWS.d/3.10.0a6.rst +++ b/Misc/NEWS.d/3.10.0a6.rst @@ -294,8 +294,8 @@ actual dictionary. This created problems for introspection tools. .. nonce: SwcSuU .. section: Library -Added :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` and -:data:`~os.O_NOFOLLOW_ANY` for macOS. Patch by Dong-hee Na. +Added :const:`~os.O_EVTONLY`, :const:`~os.O_FSYNC`, :const:`~os.O_SYMLINK` and +:const:`~os.O_NOFOLLOW_ANY` for macOS. Patch by Dong-hee Na. .. @@ -304,7 +304,7 @@ Added :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` and .. nonce: a7Dote .. section: Library -Adds :data:`resource.RLIMIT_KQUEUES` constant from FreeBSD to the +Adds :const:`resource.RLIMIT_KQUEUES` constant from FreeBSD to the :mod:`resource` module. .. diff --git a/Misc/NEWS.d/3.10.0a7.rst b/Misc/NEWS.d/3.10.0a7.rst index 286d0a8a7e919..ff01ee645c0a9 100644 --- a/Misc/NEWS.d/3.10.0a7.rst +++ b/Misc/NEWS.d/3.10.0a7.rst @@ -713,7 +713,7 @@ this situation. Also ensures that the :func:`tempfile.gettempdir()` and .. section: Library Expose ``X509_V_FLAG_ALLOW_PROXY_CERTS`` as -:data:`~ssl.VERIFY_ALLOW_PROXY_CERTS` to allow proxy certificate validation +:const:`~ssl.VERIFY_ALLOW_PROXY_CERTS` to allow proxy certificate validation as explained in https://www.openssl.org/docs/man1.1.1/man7/proxy-certificates.html. diff --git a/Misc/NEWS.d/3.10.0b1.rst b/Misc/NEWS.d/3.10.0b1.rst index 38e0af17a2c66..3c71bc73b812a 100644 --- a/Misc/NEWS.d/3.10.0b1.rst +++ b/Misc/NEWS.d/3.10.0b1.rst @@ -871,7 +871,7 @@ assert_called_once_with) will unconditionally pass. .. nonce: -1XPDH .. section: Library -Add :data:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL 3.0.0) +Add :const:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL 3.0.0) .. diff --git a/Misc/NEWS.d/3.11.0a1.rst b/Misc/NEWS.d/3.11.0a1.rst index 1a2d22b7017f4..17ee5138dbf90 100644 --- a/Misc/NEWS.d/3.11.0a1.rst +++ b/Misc/NEWS.d/3.11.0a1.rst @@ -1468,8 +1468,8 @@ an installed expat library <= 2.2.0. On Unix, if the ``sem_clockwait()`` function is available in the C library (glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses -the monotonic clock (:data:`time.CLOCK_MONOTONIC`) for the timeout, rather -than using the system clock (:data:`time.CLOCK_REALTIME`), to not be +the monotonic clock (:const:`time.CLOCK_MONOTONIC`) for the timeout, rather +than using the system clock (:const:`time.CLOCK_REALTIME`), to not be affected by system clock changes. Patch by Victor Stinner. .. @@ -2087,8 +2087,8 @@ Upgrade bundled pip to 21.2.3 and setuptools to 57.4.0 .. section: Library Fix the :func:`os.set_inheritable` function on FreeBSD 14 for file -descriptor opened with the :data:`~os.O_PATH` flag: ignore the -:data:`~errno.EBADF` error on ``ioctl()``, fallback on the ``fcntl()`` +descriptor opened with the :const:`~os.O_PATH` flag: ignore the +:const:`~errno.EBADF` error on ``ioctl()``, fallback on the ``fcntl()`` implementation. Patch by Victor Stinner. .. diff --git a/Misc/NEWS.d/3.11.0a4.rst b/Misc/NEWS.d/3.11.0a4.rst index bcb6e8b7bdde3..3dd335929d655 100644 --- a/Misc/NEWS.d/3.11.0a4.rst +++ b/Misc/NEWS.d/3.11.0a4.rst @@ -839,7 +839,7 @@ patch by Kumar Aditya. .. nonce: jeiPiX .. section: Library -Added :data:`signal.SIGSTKFLT` on platforms where this signal is defined. +Added :const:`signal.SIGSTKFLT` on platforms where this signal is defined. .. diff --git a/Misc/NEWS.d/3.11.0b1.rst b/Misc/NEWS.d/3.11.0b1.rst index 1338819375bc8..a4f113cddb0f8 100644 --- a/Misc/NEWS.d/3.11.0b1.rst +++ b/Misc/NEWS.d/3.11.0b1.rst @@ -817,8 +817,8 @@ it is ever needed and document the existing mechanism for ``posix_spawn()``. .. nonce: HFtERN .. section: Library -Fix :data:`signal.NSIG` value on FreeBSD to accept signal numbers greater -than 32, like :data:`signal.SIGRTMIN` and :data:`signal.SIGRTMAX`. Patch by +Fix :const:`signal.NSIG` value on FreeBSD to accept signal numbers greater +than 32, like :const:`signal.SIGRTMIN` and :const:`signal.SIGRTMAX`. Patch by Victor Stinner. .. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index c4dfc6011ed6c..9109db537396e 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -736,7 +736,7 @@ new types. .. nonce: 6eoc8k .. section: Core and Builtins -On WASI :data:`~errno.ENOTCAPABLE` is now mapped to :exc:`PermissionError`. +On WASI :const:`~errno.ENOTCAPABLE` is now mapped to :exc:`PermissionError`. The :mod:`errno` modules exposes the new error number. ``getpath.py`` now ignores :exc:`PermissionError` when it cannot open landmark files ``pybuilddir.txt`` and ``pyenv.cfg``. @@ -2649,7 +2649,7 @@ calling any callbacks. Patch by Kumar Aditya. .. nonce: i807-g .. section: Library -Fail gracefully if :data:`~errno.EPERM` or :data:`~errno.ENOSYS` is raised +Fail gracefully if :const:`~errno.EPERM` or :const:`~errno.ENOSYS` is raised when loading :mod:`!crypt` methods. This may happen when trying to load ``MD5`` on a Linux kernel with :abbr:`FIPS (Federal Information Processing Standard)` enabled. @@ -2698,8 +2698,8 @@ Upgrade bundled pip to 22.2. .. nonce: VT34A5 .. section: Library -Fix check for existence of :data:`os.EFD_CLOEXEC`, :data:`os.EFD_NONBLOCK` -and :data:`os.EFD_SEMAPHORE` flags on older kernel versions where these +Fix check for existence of :const:`os.EFD_CLOEXEC`, :const:`os.EFD_NONBLOCK` +and :const:`os.EFD_SEMAPHORE` flags on older kernel versions where these flags are not present. Patch by Kumar Aditya. .. @@ -3553,7 +3553,7 @@ Make :class:`multiprocessing.Pool` raise an exception if .. nonce: HY0Uzj .. section: Library -Add :data:`os.PIDFD_NONBLOCK` flag to open a file descriptor for a process +Add :const:`os.PIDFD_NONBLOCK` flag to open a file descriptor for a process with :func:`os.pidfd_open` in non-blocking mode. Patch by Kumar Aditya. .. diff --git a/Misc/NEWS.d/3.12.0a2.rst b/Misc/NEWS.d/3.12.0a2.rst index 5a3ccab02016f..f781e38665a8e 100644 --- a/Misc/NEWS.d/3.12.0a2.rst +++ b/Misc/NEWS.d/3.12.0a2.rst @@ -397,7 +397,7 @@ longobject.c to speed up some operations. .. nonce: nSGEkG .. section: Core and Builtins -Expose :data:`~socket.ETH_P_ALL` and some of the :ref:`ETHERTYPE_* constants +Expose :const:`~socket.ETH_P_ALL` and some of the :ref:`ETHERTYPE_* constants ` in :mod:`socket`. Patch by Noam Cohen. .. diff --git a/Misc/NEWS.d/3.12.0a3.rst b/Misc/NEWS.d/3.12.0a3.rst index 3d1e43350d136..3e6f8de5d911f 100644 --- a/Misc/NEWS.d/3.12.0a3.rst +++ b/Misc/NEWS.d/3.12.0a3.rst @@ -505,7 +505,7 @@ return True from this method; now they correctly return False. .. nonce: ZoOY5G .. section: Library -Add an :data:`~ssl.OP_ENABLE_KTLS` option for enabling the use of the kernel +Add an :const:`~ssl.OP_ENABLE_KTLS` option for enabling the use of the kernel TLS (kTLS). Patch by Illia Volochii. .. diff --git a/Misc/NEWS.d/3.12.0a4.rst b/Misc/NEWS.d/3.12.0a4.rst index dd26d4d964d6b..8951490f41b94 100644 --- a/Misc/NEWS.d/3.12.0a4.rst +++ b/Misc/NEWS.d/3.12.0a4.rst @@ -317,7 +317,7 @@ Improve performance of ``list.pop`` for small lists. .. nonce: yP4Na0 .. section: Core and Builtins -Add :data:`ssl.OP_LEGACY_SERVER_CONNECT` +Add :const:`ssl.OP_LEGACY_SERVER_CONNECT` .. @@ -356,7 +356,7 @@ arrays. .. nonce: mHRdQn .. section: Library -Add :data:`socket.IP_PKTINFO` constant. +Add :const:`socket.IP_PKTINFO` constant. .. diff --git a/Misc/NEWS.d/3.12.0a6.rst b/Misc/NEWS.d/3.12.0a6.rst index f6beb5b7ec3db..07967028bdee7 100644 --- a/Misc/NEWS.d/3.12.0a6.rst +++ b/Misc/NEWS.d/3.12.0a6.rst @@ -303,7 +303,7 @@ Kim. .. nonce: Vxz0Mr .. section: Library -Add :data:`mmap.MAP_ALIGNED_SUPER` FreeBSD and :data:`mmap.MAP_CONCEAL` +Add :const:`mmap.MAP_ALIGNED_SUPER` FreeBSD and :const:`mmap.MAP_CONCEAL` OpenBSD constants to :mod:`mmap`. Patch by Yeojin Kim. .. diff --git a/Misc/NEWS.d/3.12.0a7.rst b/Misc/NEWS.d/3.12.0a7.rst index 8f078e50823a0..1ef8174755885 100644 --- a/Misc/NEWS.d/3.12.0a7.rst +++ b/Misc/NEWS.d/3.12.0a7.rst @@ -605,7 +605,7 @@ reported unauthenticated EOFs (i.e. without close_notify) as a clean TLS-level EOF. It now raises :exc:`~ssl.SSLEOFError`, matching the behavior in previous versions of OpenSSL. The :attr:`~ssl.SSLContext.options` attribute on :class:`~ssl.SSLContext` also no longer includes -:data:`~ssl.OP_IGNORE_UNEXPECTED_EOF` by default. This option may be set to +:const:`~ssl.OP_IGNORE_UNEXPECTED_EOF` by default. This option may be set to specify the previous OpenSSL 3.0 behavior. .. diff --git a/Misc/NEWS.d/3.12.0b1.rst b/Misc/NEWS.d/3.12.0b1.rst index 7be70de029b1f..51d80711d91ed 100644 --- a/Misc/NEWS.d/3.12.0b1.rst +++ b/Misc/NEWS.d/3.12.0b1.rst @@ -842,7 +842,7 @@ filesystem case. .. section: Library Improve performance of :meth:`pathlib.Path.glob` by using -:data:`re.IGNORECASE` to implement case-insensitive matching. +:const:`re.IGNORECASE` to implement case-insensitive matching. .. @@ -1882,7 +1882,7 @@ both cases. .. nonce: 564i32 .. section: Library -Add :data:`~csv.QUOTE_STRINGS` and :data:`~csv.QUOTE_NOTNULL` to the suite +Add :const:`~csv.QUOTE_STRINGS` and :const:`~csv.QUOTE_NOTNULL` to the suite of :mod:`csv` module quoting styles. .. diff --git a/Misc/NEWS.d/3.6.0rc1.rst b/Misc/NEWS.d/3.6.0rc1.rst index 15769f950db23..658f8c902d870 100644 --- a/Misc/NEWS.d/3.6.0rc1.rst +++ b/Misc/NEWS.d/3.6.0rc1.rst @@ -69,8 +69,8 @@ supported. .. nonce: ilNIWN .. section: Library -Add new :data:`socket.TCP_CONGESTION` (Linux 2.6.13) and -:data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by +Add new :const:`socket.TCP_CONGESTION` (Linux 2.6.13) and +:const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by Omar Sandoval. .. diff --git a/Misc/NEWS.d/3.7.0a1.rst b/Misc/NEWS.d/3.7.0a1.rst index ef93454784b77..712558bf98d01 100644 --- a/Misc/NEWS.d/3.7.0a1.rst +++ b/Misc/NEWS.d/3.7.0a1.rst @@ -3274,7 +3274,7 @@ Added support for bytes paths in os.fwalk(). .. nonce: 37jMwb .. section: Library -Add new :data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constant. Patch by +Add new :const:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constant. Patch by Nathaniel J. Smith. .. @@ -3871,8 +3871,8 @@ as an integer. Function only available on Android. .. nonce: ilNIWN .. section: Library -Add new :data:`socket.TCP_CONGESTION` (Linux 2.6.13) and -:data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by +Add new :const:`socket.TCP_CONGESTION` (Linux 2.6.13) and +:const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by Omar Sandoval. .. diff --git a/Misc/NEWS.d/3.7.0a3.rst b/Misc/NEWS.d/3.7.0a3.rst index 368efb73567c4..52df0e7e82b08 100644 --- a/Misc/NEWS.d/3.7.0a3.rst +++ b/Misc/NEWS.d/3.7.0a3.rst @@ -754,8 +754,8 @@ now accepts characters as arguments. Based on patch by Steve Fink. .. nonce: DYQL0g .. section: Library -Add 3 new clock identifiers: :data:`time.CLOCK_BOOTTIME`, -:data:`time.CLOCK_PROF` and :data:`time.CLOCK_UPTIME`. +Add 3 new clock identifiers: :const:`time.CLOCK_BOOTTIME`, +:const:`time.CLOCK_PROF` and :const:`time.CLOCK_UPTIME`. .. diff --git a/Misc/NEWS.d/3.8.0a1.rst b/Misc/NEWS.d/3.8.0a1.rst index 2634832b78a96..467e992780382 100644 --- a/Misc/NEWS.d/3.8.0a1.rst +++ b/Misc/NEWS.d/3.8.0a1.rst @@ -1934,7 +1934,7 @@ failure. .. nonce: _ct_0H .. section: Library -The :data:`time.CLOCK_UPTIME_RAW` constant is now available for macOS 10.12. +The :const:`time.CLOCK_UPTIME_RAW` constant is now available for macOS 10.12. .. diff --git a/Misc/NEWS.d/3.8.0a4.rst b/Misc/NEWS.d/3.8.0a4.rst index a0d60f456789f..524a05a7ae970 100644 --- a/Misc/NEWS.d/3.8.0a4.rst +++ b/Misc/NEWS.d/3.8.0a4.rst @@ -955,7 +955,7 @@ Add a new :mod:`_testinternalcapi` module to test the internal C API. .. section: Tests Fix ``test_imap4_host_default_value()`` of ``test_imaplib``: catch also -:data:`errno.ENETUNREACH` error. +:const:`errno.ENETUNREACH` error. .. diff --git a/Misc/NEWS.d/3.9.0a1.rst b/Misc/NEWS.d/3.9.0a1.rst index b97b7a506da31..3b8197c5f039e 100644 --- a/Misc/NEWS.d/3.9.0a1.rst +++ b/Misc/NEWS.d/3.9.0a1.rst @@ -1164,7 +1164,7 @@ defines them with eponymous methods. .. nonce: bmhquU .. section: Library -Add :data:`os.P_PIDFD` constant, which may be passed to :func:`os.waitid` to +Add :const:`os.P_PIDFD` constant, which may be passed to :func:`os.waitid` to wait on a Linux process file descriptor. .. @@ -1193,8 +1193,8 @@ Expose the Linux ``pidfd_open`` syscall as :func:`os.pidfd_open`. .. nonce: 7jvYFA .. section: Library -Added constants :data:`~fcntl.F_OFD_GETLK`, :data:`~fcntl.F_OFD_SETLK` and -:data:`~fcntl.F_OFD_SETLKW` to the :mod:`fcntl` module. Patch by Dong-hee +Added constants :const:`~fcntl.F_OFD_GETLK`, :const:`~fcntl.F_OFD_SETLK` and +:const:`~fcntl.F_OFD_SETLKW` to the :mod:`fcntl` module. Patch by Dong-hee Na. .. @@ -1283,7 +1283,7 @@ Fixed erroneous equality comparison in statistics.NormalDist(). .. nonce: 86ExWB .. section: Library -Added :data:`~os.CLD_KILLED` and :data:`~os.CLD_STOPPED` for +Added :const:`~os.CLD_KILLED` and :const:`~os.CLD_STOPPED` for :attr:`si_code`. Patch by Dong-hee Na. .. @@ -1355,8 +1355,8 @@ objects, patch by Samuel Colvin. .. nonce: 9w-IGF .. section: Library -Add missing :data:`stat.S_IFDOOR`, :data:`stat.S_IFPORT`, -:data:`stat.S_IFWHT`, :func:`stat.S_ISDOOR`, :func:`stat.S_ISPORT`, and +Add missing :const:`stat.S_IFDOOR`, :const:`stat.S_IFPORT`, +:const:`stat.S_IFWHT`, :func:`stat.S_ISDOOR`, :func:`stat.S_ISPORT`, and :func:`stat.S_ISWHT` values to the Python implementation of :mod:`stat`. .. @@ -4983,7 +4983,7 @@ set to CP_UTF7 or CP_UTF8. .. nonce: -0g2O3 .. section: Windows -Make :data:`winreg.REG_MULTI_SZ` support zero-length strings. +Make :const:`winreg.REG_MULTI_SZ` support zero-length strings. .. diff --git a/Misc/NEWS.d/3.9.0a6.rst b/Misc/NEWS.d/3.9.0a6.rst index 9594964917f39..519c7f833ebcb 100644 --- a/Misc/NEWS.d/3.9.0a6.rst +++ b/Misc/NEWS.d/3.9.0a6.rst @@ -680,7 +680,7 @@ child process, reset the lock to the unlocked state. Rename also the private .. nonce: kIjVge .. section: Library -Expose :data:`~socket.CAN_RAW_JOIN_FILTERS` in the :mod:`socket` module. +Expose :const:`~socket.CAN_RAW_JOIN_FILTERS` in the :mod:`socket` module. .. @@ -735,7 +735,7 @@ number of groups. For other implementations, double the group list size. .. nonce: HFpHZS .. section: Library -Add :data:`time.CLOCK_TAI` constant if the operating system support it. +Add :const:`time.CLOCK_TAI` constant if the operating system support it. .. From webhook-mailer at python.org Fri Jul 21 06:18:02 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Fri, 21 Jul 2023 10:18:02 -0000 Subject: [Python-checkins] gh-106847: Add -X warn_default_encoding in sys.flags Doc (#106854) Message-ID: https://github.com/python/cpython/commit/fd84ac0ee0a8d5e34e0a106eed7e50539b61c5f8 commit: fd84ac0ee0a8d5e34e0a106eed7e50539b61c5f8 branch: main author: qqwqqw689 <114795525+qqwqqw689 at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-21T15:47:58+05:30 summary: gh-106847: Add -X warn_default_encoding in sys.flags Doc (#106854) Co-authored-by: Nikita Sobolev files: M Doc/library/sys.rst diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index c01d2d4527152..f6b67dc57c187 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -515,27 +515,28 @@ always available. The :term:`named tuple` *flags* exposes the status of command line flags. The attributes are read only. - ============================= ============================================================================================================== - attribute flag - ============================= ============================================================================================================== - :const:`debug` :option:`-d` - :const:`inspect` :option:`-i` - :const:`interactive` :option:`-i` - :const:`isolated` :option:`-I` - :const:`optimize` :option:`-O` or :option:`-OO` - :const:`dont_write_bytecode` :option:`-B` - :const:`no_user_site` :option:`-s` - :const:`no_site` :option:`-S` - :const:`ignore_environment` :option:`-E` - :const:`verbose` :option:`-v` - :const:`bytes_warning` :option:`-b` - :const:`quiet` :option:`-q` - :const:`hash_randomization` :option:`-R` - :const:`dev_mode` :option:`-X dev <-X>` (:ref:`Python Development Mode `) - :const:`utf8_mode` :option:`-X utf8 <-X>` - :const:`safe_path` :option:`-P` - :const:`int_max_str_digits` :option:`-X int_max_str_digits <-X>` (:ref:`integer string conversion length limitation `) - ============================= ============================================================================================================== + ============================== ============================================================================================================== + attribute flag + ============================== ============================================================================================================== + :const:`debug` :option:`-d` + :const:`inspect` :option:`-i` + :const:`interactive` :option:`-i` + :const:`isolated` :option:`-I` + :const:`optimize` :option:`-O` or :option:`-OO` + :const:`dont_write_bytecode` :option:`-B` + :const:`no_user_site` :option:`-s` + :const:`no_site` :option:`-S` + :const:`ignore_environment` :option:`-E` + :const:`verbose` :option:`-v` + :const:`bytes_warning` :option:`-b` + :const:`quiet` :option:`-q` + :const:`hash_randomization` :option:`-R` + :const:`dev_mode` :option:`-X dev <-X>` (:ref:`Python Development Mode `) + :const:`utf8_mode` :option:`-X utf8 <-X>` + :const:`safe_path` :option:`-P` + :const:`int_max_str_digits` :option:`-X int_max_str_digits <-X>` (:ref:`integer string conversion length limitation `) + :const:`warn_default_encoding` :option:`-X warn_default_encoding <-X>` + ============================== ============================================================================================================== .. versionchanged:: 3.2 Added ``quiet`` attribute for the new :option:`-q` flag. @@ -554,6 +555,9 @@ always available. Mode ` and the ``utf8_mode`` attribute for the new :option:`-X` ``utf8`` flag. + .. versionchanged:: 3.10 + Added ``warn_default_encoding`` attribute for :option:`-X` ``warn_default_encoding`` flag. + .. versionchanged:: 3.11 Added the ``safe_path`` attribute for :option:`-P` option. From webhook-mailer at python.org Fri Jul 21 07:48:19 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 11:48:19 -0000 Subject: [Python-checkins] [3.12] gh-106919: Use role :c:macro: for referencing the C "constants" (GH-106920) (GH-106951) Message-ID: https://github.com/python/cpython/commit/ac9aa8a369e03784c5df7f2f8b598959fc9ef5f4 commit: ac9aa8a369e03784c5df7f2f8b598959fc9ef5f4 branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T14:48:15+03:00 summary: [3.12] gh-106919: Use role :c:macro: for referencing the C "constants" (GH-106920) (GH-106951) (cherry picked from commit fcc816dbff7ca66c26f57a506e4d2330fe41d0ff) files: M Doc/c-api/arg.rst M Doc/c-api/call.rst M Doc/c-api/complex.rst M Doc/c-api/exceptions.rst M Doc/c-api/file.rst M Doc/c-api/float.rst M Doc/c-api/gcsupport.rst M Doc/c-api/init.rst M Doc/c-api/long.rst M Doc/c-api/memory.rst M Doc/c-api/module.rst M Doc/c-api/object.rst M Doc/c-api/slice.rst M Doc/c-api/stable.rst M Doc/c-api/structures.rst M Doc/c-api/sys.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/c-api/veryhigh.rst M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/howto/isolating-extensions.rst M Doc/library/dis.rst M Doc/library/os.rst M Doc/reference/compound_stmts.rst M Doc/using/cmdline.rst M Doc/whatsnew/2.2.rst M Doc/whatsnew/2.3.rst M Doc/whatsnew/2.4.rst M Doc/whatsnew/2.6.rst M Doc/whatsnew/2.7.rst M Doc/whatsnew/3.10.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a3.rst M Misc/NEWS.d/3.10.0b1.rst M Misc/NEWS.d/3.11.0a1.rst M Misc/NEWS.d/3.11.0a7.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0a2.rst M Misc/NEWS.d/3.6.0a1.rst M Misc/NEWS.d/3.9.0a1.rst diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index 9713431688d49..eba8f2be70b45 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -343,7 +343,7 @@ Other objects *items*. Format units for sequences may be nested. It is possible to pass "long" integers (integers whose value exceeds the -platform's :const:`LONG_MAX`) however no proper range checking is done --- the +platform's :c:macro:`LONG_MAX`) however no proper range checking is done --- the most significant bits are silently truncated when the receiving field is too small to receive the value (actually, the semantics are inherited from downcasts in C --- your mileage may vary). @@ -455,7 +455,7 @@ API Functions A simpler form of parameter retrieval which does not use a format string to specify the types of the arguments. Functions which use this method to retrieve - their parameters should be declared as :const:`METH_VARARGS` in function or + their parameters should be declared as :c:macro:`METH_VARARGS` in function or method tables. The tuple containing the actual parameters should be passed as *args*; it must actually be a tuple. The length of the tuple must be at least *min* and no more than *max*; *min* and *max* may be equal. Additional diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index 4dc66e318cd12..143fce3cdf477 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -59,12 +59,12 @@ This bears repeating: .. versionchanged:: 3.12 - The :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class + The :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class when the class's :py:meth:`~object.__call__` method is reassigned. (This internally sets :c:member:`~PyTypeObject.tp_call` only, and thus may make it behave differently than the vectorcall function.) In earlier Python versions, vectorcall should only be used with - :const:`immutable ` or static types. + :c:macro:`immutable ` or static types. A class should not implement vectorcall if that would be slower than *tp_call*. For example, if the callee needs to convert @@ -72,7 +72,7 @@ the arguments to an args tuple and kwargs dict anyway, then there is no point in implementing vectorcall. Classes can implement the vectorcall protocol by enabling the -:const:`Py_TPFLAGS_HAVE_VECTORCALL` flag and setting +:c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag and setting :c:member:`~PyTypeObject.tp_vectorcall_offset` to the offset inside the object structure where a *vectorcallfunc* appears. This is a pointer to a function with the following signature: @@ -84,7 +84,7 @@ This is a pointer to a function with the following signature: values of the keyword arguments. This can be *NULL* if there are no arguments. - *nargsf* is the number of positional arguments plus possibly the - :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. + :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. To get the actual number of positional arguments from *nargsf*, use :c:func:`PyVectorcall_NARGS`. - *kwnames* is a tuple containing the names of the keyword arguments; @@ -93,7 +93,7 @@ This is a pointer to a function with the following signature: and they must be unique. If there are no keyword arguments, then *kwnames* can instead be *NULL*. -.. data:: PY_VECTORCALL_ARGUMENTS_OFFSET +.. c:macro:: PY_VECTORCALL_ARGUMENTS_OFFSET If this flag is set in a vectorcall *nargsf* argument, the callee is allowed to temporarily change ``args[-1]``. In other words, *args* points to @@ -104,7 +104,7 @@ This is a pointer to a function with the following signature: ``args[0]`` may be changed. Whenever they can do so cheaply (without additional allocation), callers - are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. + are encouraged to use :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET`. Doing so will allow callables such as bound methods to make their onward calls (which include a prepended *self* argument) very efficiently. @@ -174,7 +174,7 @@ Vectorcall Support API This is a specialized function, intended to be put in the :c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``. - It does not check the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag + It does not check the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag and it does not fall back to ``tp_call``. .. versionadded:: 3.8 @@ -392,11 +392,11 @@ please see individual documentation for details. *args[0]*, and the *args* array starting at *args[1]* represents the arguments of the call. There must be at least one positional argument. *nargsf* is the number of positional arguments including *args[0]*, - plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may + plus :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may temporarily be changed. Keyword arguments can be passed just like in :c:func:`PyObject_Vectorcall`. - If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, + If the object has the :c:macro:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, this will call the unbound method object with the full *args* vector as arguments. diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst index cb8b270fcbab6..6679ce76f1dc6 100644 --- a/Doc/c-api/complex.rst +++ b/Doc/c-api/complex.rst @@ -64,7 +64,7 @@ pointers. This is consistent throughout the API. representation. If *divisor* is null, this method returns zero and sets - :c:data:`errno` to :c:data:`EDOM`. + :c:data:`errno` to :c:macro:`EDOM`. .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp) @@ -73,7 +73,7 @@ pointers. This is consistent throughout the API. representation. If *num* is null and *exp* is not a positive real number, - this method returns zero and sets :c:data:`errno` to :c:data:`EDOM`. + this method returns zero and sets :c:data:`errno` to :c:macro:`EDOM`. Complex Numbers as Python Objects diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index a24ecac861e76..6e09f829da304 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -165,7 +165,7 @@ For convenience, some of these functions will always return a tuple object whose first item is the integer :c:data:`errno` value and whose second item is the corresponding error message (gotten from :c:func:`strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the - :c:data:`errno` value is :const:`EINTR`, indicating an interrupted system call, + :c:data:`errno` value is :c:macro:`EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, leaves it set to that. The function always returns ``NULL``, so a wrapper function around a system call can write ``return PyErr_SetFromErrno(type);`` @@ -631,7 +631,7 @@ Signal Handling be interruptible by user requests (such as by pressing Ctrl-C). .. note:: - The default Python signal handler for :const:`SIGINT` raises the + The default Python signal handler for :c:macro:`SIGINT` raises the :exc:`KeyboardInterrupt` exception. @@ -642,7 +642,7 @@ Signal Handling single: SIGINT single: KeyboardInterrupt (built-in exception) - Simulate the effect of a :const:`SIGINT` signal arriving. + Simulate the effect of a :c:macro:`SIGINT` signal arriving. This is equivalent to ``PyErr_SetInterruptEx(SIGINT)``. .. note:: @@ -754,7 +754,7 @@ Exception Objects .. c:function:: PyObject* PyException_GetCause(PyObject *ex) - Return the cause (either an exception instance, or :const:`None`, + Return the cause (either an exception instance, or ``None``, set by ``raise ... from ...``) associated with the exception as a new reference, as accessible from Python through :attr:`__cause__`. @@ -763,7 +763,7 @@ Exception Objects Set the cause associated with the exception to *cause*. Use ``NULL`` to clear it. There is no type check to make sure that *cause* is either an exception - instance or :const:`None`. This steals a reference to *cause*. + instance or ``None``. This steals a reference to *cause*. :attr:`__suppress_context__` is implicitly set to ``True`` by this function. @@ -874,7 +874,7 @@ because the :ref:`call protocol ` takes care of recursion handling. Marks a point where a recursive C-level call is about to be performed. - If :const:`USE_STACKCHECK` is defined, this function checks if the OS + If :c:macro:`USE_STACKCHECK` is defined, this function checks if the OS stack overflowed using :c:func:`PyOS_CheckStack`. In this is the case, it sets a :exc:`MemoryError` and returns a nonzero value. diff --git a/Doc/c-api/file.rst b/Doc/c-api/file.rst index f32ecba9f2702..b36c800e00444 100644 --- a/Doc/c-api/file.rst +++ b/Doc/c-api/file.rst @@ -93,7 +93,7 @@ the :mod:`io` APIs instead. .. index:: single: Py_PRINT_RAW Write object *obj* to file object *p*. The only supported flag for *flags* is - :const:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written + :c:macro:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written instead of the :func:`repr`. Return ``0`` on success or ``-1`` on failure; the appropriate exception will be set. diff --git a/Doc/c-api/float.rst b/Doc/c-api/float.rst index fd0be1108c630..4f6ac0d8175c6 100644 --- a/Doc/c-api/float.rst +++ b/Doc/c-api/float.rst @@ -109,7 +109,7 @@ Pack functions The pack routines write 2, 4 or 8 bytes, starting at *p*. *le* is an :c:expr:`int` argument, non-zero if you want the bytes string in little-endian format (exponent last, at ``p+1``, ``p+3``, or ``p+6`` ``p+7``), zero if you -want big-endian format (exponent first, at *p*). The :c:data:`PY_BIG_ENDIAN` +want big-endian format (exponent first, at *p*). The :c:macro:`PY_BIG_ENDIAN` constant can be used to use the native endian: it is equal to ``1`` on big endian processor, or ``0`` on little endian processor. @@ -140,7 +140,7 @@ Unpack functions The unpack routines read 2, 4 or 8 bytes, starting at *p*. *le* is an :c:expr:`int` argument, non-zero if the bytes string is in little-endian format (exponent last, at ``p+1``, ``p+3`` or ``p+6`` and ``p+7``), zero if big-endian -(exponent first, at *p*). The :c:data:`PY_BIG_ENDIAN` constant can be used to +(exponent first, at *p*). The :c:macro:`PY_BIG_ENDIAN` constant can be used to use the native endian: it is equal to ``1`` on big endian processor, or ``0`` on little endian processor. diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index c3260a21bc7f8..e56414ab9f754 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -13,14 +13,12 @@ or strings), do not need to provide any explicit support for garbage collection. To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must -include the :const:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the +include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the :c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a :c:member:`~PyTypeObject.tp_clear` implementation must also be provided. -.. data:: Py_TPFLAGS_HAVE_GC - :noindex: - +:c:macro:`Py_TPFLAGS_HAVE_GC` Objects with a type with this flag set must conform with the rules documented here. For convenience these objects will be referred to as container objects. @@ -52,17 +50,17 @@ rules: :c:member:`~PyTypeObject.tp_flags`, :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields if the type inherits from a class that implements the garbage collector protocol and the child class - does *not* include the :const:`Py_TPFLAGS_HAVE_GC` flag. + does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. .. c:function:: TYPE* PyObject_GC_New(TYPE, PyTypeObject *type) Analogous to :c:func:`PyObject_New` but for container objects with the - :const:`Py_TPFLAGS_HAVE_GC` flag set. + :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. .. c:function:: TYPE* PyObject_GC_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) Analogous to :c:func:`PyObject_NewVar` but for container objects with the - :const:`Py_TPFLAGS_HAVE_GC` flag set. + :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. .. c:function:: PyObject* PyUnstable_Object_GC_NewWithExtraData(PyTypeObject *type, size_t extra_size) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 26762969ef8eb..c99377fb45a6b 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1396,7 +1396,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. function does not steal any references to *exc*. To prevent naive misuse, you must write your own C extension to call this. Must be called with the GIL held. Returns the number of thread states modified; this is normally one, but will be - zero if the thread id isn't found. If *exc* is :const:`NULL`, the pending + zero if the thread id isn't found. If *exc* is ``NULL``, the pending exception (if any) for the thread is cleared. This raises no exceptions. .. versionchanged:: 3.7 @@ -1675,32 +1675,32 @@ Python-level trace functions in previous versions. The type of the trace function registered using :c:func:`PyEval_SetProfile` and :c:func:`PyEval_SetTrace`. The first parameter is the object passed to the registration function as *obj*, *frame* is the frame object to which the event - pertains, *what* is one of the constants :const:`PyTrace_CALL`, - :const:`PyTrace_EXCEPTION`, :const:`PyTrace_LINE`, :const:`PyTrace_RETURN`, - :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION`, :const:`PyTrace_C_RETURN`, - or :const:`PyTrace_OPCODE`, and *arg* depends on the value of *what*: - - +------------------------------+----------------------------------------+ - | Value of *what* | Meaning of *arg* | - +==============================+========================================+ - | :const:`PyTrace_CALL` | Always :c:data:`Py_None`. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_EXCEPTION` | Exception information as returned by | - | | :func:`sys.exc_info`. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_LINE` | Always :c:data:`Py_None`. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_RETURN` | Value being returned to the caller, | - | | or ``NULL`` if caused by an exception. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_C_CALL` | Function object being called. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_C_EXCEPTION` | Function object being called. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_C_RETURN` | Function object being called. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_OPCODE` | Always :c:data:`Py_None`. | - +------------------------------+----------------------------------------+ + pertains, *what* is one of the constants :c:data:`PyTrace_CALL`, + :c:data:`PyTrace_EXCEPTION`, :c:data:`PyTrace_LINE`, :c:data:`PyTrace_RETURN`, + :c:data:`PyTrace_C_CALL`, :c:data:`PyTrace_C_EXCEPTION`, :c:data:`PyTrace_C_RETURN`, + or :c:data:`PyTrace_OPCODE`, and *arg* depends on the value of *what*: + + +-------------------------------+----------------------------------------+ + | Value of *what* | Meaning of *arg* | + +===============================+========================================+ + | :c:data:`PyTrace_CALL` | Always :c:data:`Py_None`. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_EXCEPTION` | Exception information as returned by | + | | :func:`sys.exc_info`. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_LINE` | Always :c:data:`Py_None`. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_RETURN` | Value being returned to the caller, | + | | or ``NULL`` if caused by an exception. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_C_CALL` | Function object being called. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_C_EXCEPTION` | Function object being called. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_C_RETURN` | Function object being called. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_OPCODE` | Always :c:data:`Py_None`. | + +-------------------------------+----------------------------------------+ .. c:var:: int PyTrace_CALL @@ -1767,8 +1767,8 @@ Python-level trace functions in previous versions. function as its first parameter, and may be any Python object, or ``NULL``. If the profile function needs to maintain state, using a different value for *obj* for each thread provides a convenient and thread-safe place to store it. The - profile function is called for all monitored events except :const:`PyTrace_LINE` - :const:`PyTrace_OPCODE` and :const:`PyTrace_EXCEPTION`. + profile function is called for all monitored events except :c:data:`PyTrace_LINE` + :c:data:`PyTrace_OPCODE` and :c:data:`PyTrace_EXCEPTION`. See also the :func:`sys.setprofile` function. @@ -1793,8 +1793,8 @@ Python-level trace functions in previous versions. :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number events and per-opcode events, but does not receive any event related to C function objects being called. Any trace function registered using :c:func:`PyEval_SetTrace` - will not receive :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or - :const:`PyTrace_C_RETURN` as a value for the *what* parameter. + will not receive :c:data:`PyTrace_C_CALL`, :c:data:`PyTrace_C_EXCEPTION` or + :c:data:`PyTrace_C_RETURN` as a value for the *what* parameter. See also the :func:`sys.settrace` function. diff --git a/Doc/c-api/long.rst b/Doc/c-api/long.rst index fe379ffe91239..f1354a34f2b2d 100644 --- a/Doc/c-api/long.rst +++ b/Doc/c-api/long.rst @@ -142,8 +142,8 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate. instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. - If the value of *obj* is greater than :const:`LONG_MAX` or less than - :const:`LONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and + If the value of *obj* is greater than :c:macro:`LONG_MAX` or less than + :c:macro:`LONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. @@ -183,8 +183,8 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate. instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. - If the value of *obj* is greater than :const:`LLONG_MAX` or less than - :const:`LLONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, + If the value of *obj* is greater than :c:macro:`LLONG_MAX` or less than + :c:macro:`LLONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 7041c15d23fb8..35c356f25c6c7 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -470,7 +470,7 @@ Customize Memory Allocators The new allocator must return a distinct non-``NULL`` pointer when requesting zero bytes. - For the :c:data:`PYMEM_DOMAIN_RAW` domain, the allocator must be + For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be thread-safe: the :term:`GIL ` is not held when the allocator is called. @@ -536,8 +536,8 @@ Runtime checks: - Detect write before the start of the buffer (buffer underflow). - Detect write after the end of the buffer (buffer overflow). - Check that the :term:`GIL ` is held when - allocator functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: - :c:func:`PyObject_Malloc`) and :c:data:`PYMEM_DOMAIN_MEM` (ex: + allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: + :c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. On error, the debug hooks use the :mod:`tracemalloc` module to get the @@ -557,9 +557,9 @@ that the treatment of negative indices differs from a Python slice): ``p[-S]`` API identifier (ASCII character): - * ``'r'`` for :c:data:`PYMEM_DOMAIN_RAW`. - * ``'m'`` for :c:data:`PYMEM_DOMAIN_MEM`. - * ``'o'`` for :c:data:`PYMEM_DOMAIN_OBJ`. + * ``'r'`` for :c:macro:`PYMEM_DOMAIN_RAW`. + * ``'m'`` for :c:macro:`PYMEM_DOMAIN_MEM`. + * ``'o'`` for :c:macro:`PYMEM_DOMAIN_OBJ`. ``p[-S+1:0]`` Copies of PYMEM_FORBIDDENBYTE. Used to catch under- writes and reads. @@ -601,7 +601,7 @@ PYMEM_CLEANBYTE (meaning uninitialized memory is getting used). compiled in release mode. On error, the debug hooks now use :mod:`tracemalloc` to get the traceback where a memory block was allocated. The debug hooks now also check if the GIL is held when functions of - :c:data:`PYMEM_DOMAIN_OBJ` and :c:data:`PYMEM_DOMAIN_MEM` domains are + :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are called. .. versionchanged:: 3.8 @@ -622,8 +622,8 @@ with a fixed size of 256 KiB. It falls back to :c:func:`PyMem_RawMalloc` and :c:func:`PyMem_RawRealloc` for allocations larger than 512 bytes. *pymalloc* is the :ref:`default allocator ` of the -:c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) and -:c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) domains. +:c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) and +:c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) domains. The arena allocator uses the following functions: diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index 230b471d473be..bc8e3b23b9957 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -145,7 +145,7 @@ or request "multi-phase initialization" by returning the definition struct itsel .. c:member:: PyModuleDef_Base m_base - Always initialize this member to :const:`PyModuleDef_HEAD_INIT`. + Always initialize this member to :c:data:`PyModuleDef_HEAD_INIT`. .. c:member:: const char *m_name @@ -256,7 +256,7 @@ of the following two module creation functions: Create a new module object, given the definition in *def*. This behaves like :c:func:`PyModule_Create2` with *module_api_version* set to - :const:`PYTHON_API_VERSION`. + :c:macro:`PYTHON_API_VERSION`. .. c:function:: PyObject* PyModule_Create2(PyModuleDef *def, int module_api_version) @@ -390,7 +390,7 @@ objects dynamically. Note that both ``PyModule_FromDefAndSpec`` and Create a new module object, given the definition in *def* and the ModuleSpec *spec*. This behaves like :c:func:`PyModule_FromDefAndSpec2` - with *module_api_version* set to :const:`PYTHON_API_VERSION`. + with *module_api_version* set to :c:macro:`PYTHON_API_VERSION`. .. versionadded:: 3.5 diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 22e7458013fb3..3b922f3b810c7 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -23,7 +23,7 @@ Object Protocol Print an object *o*, on file *fp*. Returns ``-1`` on error. The flags argument is used to enable certain printing options. The only option currently supported - is :const:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written + is :c:macro:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written instead of the :func:`repr`. @@ -162,8 +162,8 @@ Object Protocol .. c:function:: PyObject* PyObject_RichCompare(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, - which must be one of :const:`Py_LT`, :const:`Py_LE`, :const:`Py_EQ`, - :const:`Py_NE`, :const:`Py_GT`, or :const:`Py_GE`, corresponding to ``<``, + which must be one of :c:macro:`Py_LT`, :c:macro:`Py_LE`, :c:macro:`Py_EQ`, + :c:macro:`Py_NE`, :c:macro:`Py_GT`, or :c:macro:`Py_GE`, corresponding to ``<``, ``<=``, ``==``, ``!=``, ``>``, or ``>=`` respectively. This is the equivalent of the Python expression ``o1 op o2``, where ``op`` is the operator corresponding to *opid*. Returns the value of the comparison on success, or ``NULL`` on failure. @@ -172,8 +172,8 @@ Object Protocol .. c:function:: int PyObject_RichCompareBool(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, - which must be one of :const:`Py_LT`, :const:`Py_LE`, :const:`Py_EQ`, - :const:`Py_NE`, :const:`Py_GT`, or :const:`Py_GE`, corresponding to ``<``, + which must be one of :c:macro:`Py_LT`, :c:macro:`Py_LE`, :c:macro:`Py_EQ`, + :c:macro:`Py_NE`, :c:macro:`Py_GT`, or :c:macro:`Py_GE`, corresponding to ``<``, ``<=``, ``==``, ``!=``, ``>``, or ``>=`` respectively. Returns ``-1`` on error, ``0`` if the result is false, ``1`` otherwise. This is the equivalent of the Python expression ``o1 op o2``, where ``op`` is the operator corresponding to @@ -181,7 +181,7 @@ Object Protocol .. note:: If *o1* and *o2* are the same object, :c:func:`PyObject_RichCompareBool` - will always return ``1`` for :const:`Py_EQ` and ``0`` for :const:`Py_NE`. + will always return ``1`` for :c:macro:`Py_EQ` and ``0`` for :c:macro:`Py_NE`. .. c:function:: PyObject* PyObject_Format(PyObject *obj, PyObject *format_spec) @@ -431,10 +431,10 @@ Object Protocol .. c:function:: void *PyObject_GetItemData(PyObject *o) Get a pointer to per-item data for a class with - :const:`Py_TPFLAGS_ITEMS_AT_END`. + :c:macro:`Py_TPFLAGS_ITEMS_AT_END`. On error, set an exception and return ``NULL``. :py:exc:`TypeError` is raised if *o* does not have - :const:`Py_TPFLAGS_ITEMS_AT_END` set. + :c:macro:`Py_TPFLAGS_ITEMS_AT_END` set. .. versionadded:: 3.12 diff --git a/Doc/c-api/slice.rst b/Doc/c-api/slice.rst index c54a659cf2ffd..9e880c6b7f25a 100644 --- a/Doc/c-api/slice.rst +++ b/Doc/c-api/slice.rst @@ -34,7 +34,7 @@ Slice Objects *length* as errors. Returns ``0`` on success and ``-1`` on error with no exception set (unless one of - the indices was not :const:`None` and failed to be converted to an integer, + the indices was not ``None`` and failed to be converted to an integer, in which case ``-1`` is returned with an exception set). You probably do not want to use this function. diff --git a/Doc/c-api/stable.rst b/Doc/c-api/stable.rst index 149d4d6bac3ee..c66b296d304ad 100644 --- a/Doc/c-api/stable.rst +++ b/Doc/c-api/stable.rst @@ -74,7 +74,7 @@ Contents of the Limited API are :ref:`listed below `. Define this macro before including ``Python.h`` to opt in to only use the Limited API, and to select the Limited API version. - Define ``Py_LIMITED_API`` to the value of :c:data:`PY_VERSION_HEX` + Define ``Py_LIMITED_API`` to the value of :c:macro:`PY_VERSION_HEX` corresponding to the lowest Python version your extension supports. The extension will work without recompilation with all Python 3 releases from the specified one onward, and can use Limited API introduced up to that diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 766f881463c00..720ab31f3de85 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -179,7 +179,7 @@ Implementing functions and methods .. c:type:: PyCFunctionWithKeywords Type of the functions used to implement Python callables in C - with signature :const:`METH_VARARGS | METH_KEYWORDS`. + with signature :ref:`METH_VARARGS | METH_KEYWORDS `. The function signature is:: PyObject *PyCFunctionWithKeywords(PyObject *self, @@ -190,7 +190,7 @@ Implementing functions and methods .. c:type:: _PyCFunctionFast Type of the functions used to implement Python callables in C - with signature :const:`METH_FASTCALL`. + with signature :c:macro:`METH_FASTCALL`. The function signature is:: PyObject *_PyCFunctionFast(PyObject *self, @@ -200,7 +200,7 @@ Implementing functions and methods .. c:type:: _PyCFunctionFastWithKeywords Type of the functions used to implement Python callables in C - with signature :const:`METH_FASTCALL | METH_KEYWORDS`. + with signature :ref:`METH_FASTCALL | METH_KEYWORDS `. The function signature is:: PyObject *_PyCFunctionFastWithKeywords(PyObject *self, @@ -211,7 +211,7 @@ Implementing functions and methods .. c:type:: PyCMethod Type of the functions used to implement Python callables in C - with signature :const:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS`. + with signature :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. The function signature is:: PyObject *PyCMethod(PyObject *self, @@ -257,7 +257,7 @@ convention. There are these calling conventions: -.. data:: METH_VARARGS +.. c:macro:: METH_VARARGS This is the typical calling convention, where the methods have the type :c:type:`PyCFunction`. The function expects two :c:expr:`PyObject*` values. @@ -267,8 +267,17 @@ There are these calling conventions: using :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_UnpackTuple`. -.. data:: METH_VARARGS | METH_KEYWORDS +.. c:macro:: METH_KEYWORDS + Can only be used in certain combinations with other flags: + :ref:`METH_VARARGS | METH_KEYWORDS `, + :ref:`METH_FASTCALL | METH_KEYWORDS ` and + :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. + + +.. _METH_VARARGS-METH_KEYWORDS: + +:c:expr:`METH_VARARGS | METH_KEYWORDS` Methods with these flags must be of type :c:type:`PyCFunctionWithKeywords`. The function expects three parameters: *self*, *args*, *kwargs* where *kwargs* is a dictionary of all the keyword arguments or possibly ``NULL`` @@ -276,7 +285,7 @@ There are these calling conventions: using :c:func:`PyArg_ParseTupleAndKeywords`. -.. data:: METH_FASTCALL +.. c:macro:: METH_FASTCALL Fast calling convention supporting only positional arguments. The methods have the type :c:type:`_PyCFunctionFast`. @@ -291,9 +300,10 @@ There are these calling conventions: ``METH_FASTCALL`` is now part of the :ref:`stable ABI `. -.. data:: METH_FASTCALL | METH_KEYWORDS +.. _METH_FASTCALL-METH_KEYWORDS: - Extension of :const:`METH_FASTCALL` supporting also keyword arguments, +:c:expr:`METH_FASTCALL | METH_KEYWORDS` + Extension of :c:macro:`METH_FASTCALL` supporting also keyword arguments, with methods of type :c:type:`_PyCFunctionFastWithKeywords`. Keyword arguments are passed the same way as in the :ref:`vectorcall protocol `: @@ -306,10 +316,18 @@ There are these calling conventions: .. versionadded:: 3.7 -.. data:: METH_METHOD | METH_FASTCALL | METH_KEYWORDS +.. c:macro:: METH_METHOD + + Can only be used in the combination with other flags: + :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. + + +.. _METH_METHOD-METH_FASTCALL-METH_KEYWORDS: - Extension of :const:`METH_FASTCALL | METH_KEYWORDS` supporting the *defining - class*, that is, the class that contains the method in question. +:c:expr:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS` + Extension of :ref:`METH_FASTCALL | METH_KEYWORDS ` + supporting the *defining class*, that is, + the class that contains the method in question. The defining class might be a superclass of ``Py_TYPE(self)``. The method needs to be of type :c:type:`PyCMethod`, the same as for @@ -319,10 +337,10 @@ There are these calling conventions: .. versionadded:: 3.9 -.. data:: METH_NOARGS +.. c:macro:: METH_NOARGS Methods without parameters don't need to check whether arguments are given if - they are listed with the :const:`METH_NOARGS` flag. They need to be of type + they are listed with the :c:macro:`METH_NOARGS` flag. They need to be of type :c:type:`PyCFunction`. The first parameter is typically named *self* and will hold a reference to the module or object instance. In all cases the second parameter will be ``NULL``. @@ -331,9 +349,9 @@ There are these calling conventions: :c:macro:`Py_UNUSED` can be used to prevent a compiler warning. -.. data:: METH_O +.. c:macro:: METH_O - Methods with a single object argument can be listed with the :const:`METH_O` + Methods with a single object argument can be listed with the :c:macro:`METH_O` flag, instead of invoking :c:func:`PyArg_ParseTuple` with a ``"O"`` argument. They have the type :c:type:`PyCFunction`, with the *self* parameter, and a :c:expr:`PyObject*` parameter representing the single argument. @@ -345,7 +363,7 @@ defined for modules. At most one of these flags may be set for any given method. -.. data:: METH_CLASS +.. c:macro:: METH_CLASS .. index:: pair: built-in function; classmethod @@ -355,7 +373,7 @@ method. function. -.. data:: METH_STATIC +.. c:macro:: METH_STATIC .. index:: pair: built-in function; staticmethod @@ -367,7 +385,7 @@ One other constant controls whether a method is loaded in place of another definition with the same method name. -.. data:: METH_COEXIST +.. c:macro:: METH_COEXIST The method will be loaded in place of existing definitions. Without *METH_COEXIST*, the default is to skip repeated definitions. Since slot @@ -440,8 +458,8 @@ Accessing attributes of extension types The legacy offsets :c:member:`~PyTypeObject.tp_dictoffset` and :c:member:`~PyTypeObject.tp_weaklistoffset` can be defined similarly using ``"__dictoffset__"`` and ``"__weaklistoffset__"`` members, but extensions - are strongly encouraged to use :const:`Py_TPFLAGS_MANAGED_DICT` and - :const:`Py_TPFLAGS_MANAGED_WEAKREF` instead. + are strongly encouraged to use :c:macro:`Py_TPFLAGS_MANAGED_DICT` and + :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead. .. versionchanged:: 3.12 @@ -509,19 +527,19 @@ The following flags can be used with :c:member:`PyMemberDef.flags`: .. versionchanged:: 3.10 - The :const:`!RESTRICTED`, :const:`!READ_RESTRICTED` and - :const:`!WRITE_RESTRICTED` macros available with + The :c:macro:`!RESTRICTED`, :c:macro:`!READ_RESTRICTED` and + :c:macro:`!WRITE_RESTRICTED` macros available with ``#include "structmember.h"`` are deprecated. - :const:`!READ_RESTRICTED` and :const:`!RESTRICTED` are equivalent to - :const:`Py_AUDIT_READ`; :const:`!WRITE_RESTRICTED` does nothing. + :c:macro:`!READ_RESTRICTED` and :c:macro:`!RESTRICTED` are equivalent to + :c:macro:`Py_AUDIT_READ`; :c:macro:`!WRITE_RESTRICTED` does nothing. .. index:: single: READONLY .. versionchanged:: 3.12 - The :const:`!READONLY` macro was renamed to :const:`Py_READONLY`. - The :const:`!PY_AUDIT_READ` macro was renamed with the ``Py_`` prefix. + The :c:macro:`!READONLY` macro was renamed to :c:macro:`Py_READONLY`. + The :c:macro:`!PY_AUDIT_READ` macro was renamed with the ``Py_`` prefix. The new names are now always available. Previously, these required ``#include "structmember.h"``. The header is still available and it provides the old names. diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 6fc8a3aff9568..3550193c1af82 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -97,9 +97,9 @@ Operating System Utilities .. c:function:: int PyOS_CheckStack() Return true when the interpreter runs out of stack space. This is a reliable - check, but is only available when :const:`USE_STACKCHECK` is defined (currently + check, but is only available when :c:macro:`USE_STACKCHECK` is defined (currently on certain versions of Windows using the Microsoft Visual C++ compiler). - :const:`USE_STACKCHECK` will be defined automatically; you should never + :c:macro:`USE_STACKCHECK` will be defined automatically; you should never change the definition in your own code. diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index a5f333e2a31e0..d9dcd22a24839 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -132,7 +132,7 @@ Type Objects .. c:function:: int PyType_IS_GC(PyTypeObject *o) Return true if the type object includes support for the cycle detector; this - tests the type flag :const:`Py_TPFLAGS_HAVE_GC`. + tests the type flag :c:macro:`Py_TPFLAGS_HAVE_GC`. .. c:function:: int PyType_IsSubtype(PyTypeObject *a, PyTypeObject *b) @@ -165,10 +165,10 @@ Type Objects .. note:: If some of the base classes implements the GC protocol and the provided - type does not include the :const:`Py_TPFLAGS_HAVE_GC` in its flags, then + type does not include the :c:macro:`Py_TPFLAGS_HAVE_GC` in its flags, then the GC protocol will be automatically implemented from its parents. On the contrary, if the type being created does include - :const:`Py_TPFLAGS_HAVE_GC` in its flags then it **must** implement the + :c:macro:`Py_TPFLAGS_HAVE_GC` in its flags then it **must** implement the GC protocol itself by at least implementing the :c:member:`~PyTypeObject.tp_traverse` handle. @@ -268,7 +268,7 @@ The following functions and structs are used to create .. c:function:: PyObject* PyType_FromMetaclass(PyTypeObject *metaclass, PyObject *module, PyType_Spec *spec, PyObject *bases) Create and return a :ref:`heap type ` from the *spec* - (see :const:`Py_TPFLAGS_HEAPTYPE`). + (see :c:macro:`Py_TPFLAGS_HEAPTYPE`). The metaclass *metaclass* is used to construct the resulting type object. When *metaclass* is ``NULL``, the metaclass is derived from *bases* @@ -420,7 +420,7 @@ The following functions and structs are used to create - The requested :c:member:`PyType_Spec.basicsize` is zero, suggesting that the subclass does not access the instance's memory directly. - - With the :const:`Py_TPFLAGS_ITEMS_AT_END` flag. + - With the :c:macro:`Py_TPFLAGS_ITEMS_AT_END` flag. .. c:member:: unsigned int flags @@ -471,9 +471,9 @@ The following functions and structs are used to create * :c:member:`~PyTypeObject.tp_weaklist` * :c:member:`~PyTypeObject.tp_vectorcall` * :c:member:`~PyTypeObject.tp_weaklistoffset` - (use :const:`Py_TPFLAGS_MANAGED_WEAKREF` instead) + (use :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead) * :c:member:`~PyTypeObject.tp_dictoffset` - (use :const:`Py_TPFLAGS_MANAGED_DICT` instead) + (use :c:macro:`Py_TPFLAGS_MANAGED_DICT` instead) * :c:member:`~PyTypeObject.tp_vectorcall_offset` (see :ref:`PyMemberDef `) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 7249cfe79c32e..33591ed14eaf1 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -669,7 +669,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) memory buffers owned by the instance (using the freeing function corresponding to the allocation function used to allocate the buffer), and call the type's :c:member:`~PyTypeObject.tp_free` function. If the type is not subtypable - (doesn't have the :const:`Py_TPFLAGS_BASETYPE` flag bit set), it is + (doesn't have the :c:macro:`Py_TPFLAGS_BASETYPE` flag bit set), it is permissible to call the object deallocator directly instead of via :c:member:`~PyTypeObject.tp_free`. The object deallocator should be the one used to allocate the instance; this is normally :c:func:`PyObject_Del` if the instance was allocated @@ -677,7 +677,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:func:`PyObject_GC_Del` if the instance was allocated using :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar`. - If the type supports garbage collection (has the :const:`Py_TPFLAGS_HAVE_GC` + If the type supports garbage collection (has the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit set), the destructor should call :c:func:`PyObject_GC_UnTrack` before clearing any member fields. @@ -689,7 +689,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) Py_TYPE(self)->tp_free((PyObject *)self); } - Finally, if the type is heap allocated (:const:`Py_TPFLAGS_HEAPTYPE`), the + Finally, if the type is heap allocated (:c:macro:`Py_TPFLAGS_HEAPTYPE`), the deallocator should decrement the reference count for its type object after calling the type deallocator. In order to avoid dangling pointers, the recommended way to achieve this is: @@ -716,12 +716,12 @@ and :c:type:`PyType_Type` effectively act as defaults.) a more efficient alternative of the simpler :c:member:`~PyTypeObject.tp_call`. - This field is only used if the flag :const:`Py_TPFLAGS_HAVE_VECTORCALL` + This field is only used if the flag :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` is set. If so, this must be a positive integer containing the offset in the instance of a :c:type:`vectorcallfunc` pointer. The *vectorcallfunc* pointer may be ``NULL``, in which case the instance behaves - as if :const:`Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance + as if :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance falls back to :c:member:`~PyTypeObject.tp_call`. Any class that sets ``Py_TPFLAGS_HAVE_VECTORCALL`` must also set @@ -743,12 +743,12 @@ and :c:type:`PyType_Type` effectively act as defaults.) When a user sets :attr:`~type.__call__` in Python code, only *tp_call* is updated, likely making it inconsistent with the vectorcall function. Since 3.12, setting ``__call__`` will disable vectorcall optimization - by clearing the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag. + by clearing the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag. **Inheritance:** This field is always inherited. - However, the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is not + However, the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is not always inherited. If it's not set, then the subclass won't use :ref:`vectorcall `, except when :c:func:`PyVectorcall_Call` is explicitly called. @@ -1022,9 +1022,9 @@ and :c:type:`PyType_Type` effectively act as defaults.) this flag bit. The flag bits that pertain to extension structures are strictly inherited if the extension structure is inherited, i.e. the base type's value of the flag bit is copied into the subtype together with a pointer to the extension - structure. The :const:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with + structure. The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the - :const:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have ``NULL`` values. .. XXX are most flag bits *really* inherited individually? @@ -1036,12 +1036,14 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Bit Masks:** + .. c:namespace:: NULL + The following bit masks are currently defined; these can be ORed together using the ``|`` operator to form the value of the :c:member:`~PyTypeObject.tp_flags` field. The macro :c:func:`PyType_HasFeature` takes a type and a flags value, *tp* and *f*, and checks whether ``tp->tp_flags & f`` is non-zero. - .. data:: Py_TPFLAGS_HEAPTYPE + .. c:macro:: Py_TPFLAGS_HEAPTYPE This bit is set when the type object itself is allocated on the heap, for example, types created dynamically using :c:func:`PyType_FromSpec`. In this @@ -1056,7 +1058,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_BASETYPE + .. c:macro:: Py_TPFLAGS_BASETYPE This bit is set when the type can be used as the base type of another type. If this bit is clear, the type cannot be subtyped (similar to a "final" class in @@ -1067,7 +1069,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_READY + .. c:macro:: Py_TPFLAGS_READY This bit is set when the type object has been fully initialized by :c:func:`PyType_Ready`. @@ -1077,7 +1079,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_READYING + .. c:macro:: Py_TPFLAGS_READYING This bit is set while :c:func:`PyType_Ready` is in the process of initializing the type object. @@ -1087,7 +1089,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_HAVE_GC + .. c:macro:: Py_TPFLAGS_HAVE_GC This bit is set when the object supports garbage collection. If this bit is set, instances must be created using :c:func:`PyObject_GC_New` and @@ -1098,28 +1100,28 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :const:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` - The :const:`Py_TPFLAGS_HAVE_GC` flag bit is inherited + The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with the :attr:`tp_traverse` and :attr:`tp_clear` - fields, i.e. if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is + fields, i.e. if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the :attr:`tp_traverse` and :attr:`tp_clear` fields in the subtype exist and have ``NULL`` values. - .. data:: Py_TPFLAGS_DEFAULT + .. c:macro:: Py_TPFLAGS_DEFAULT This is a bitmask of all the bits that pertain to the existence of certain fields in the type object and its extension structures. Currently, it includes - the following bits: :const:`Py_TPFLAGS_HAVE_STACKLESS_EXTENSION`. + the following bits: :c:macro:`Py_TPFLAGS_HAVE_STACKLESS_EXTENSION`. **Inheritance:** ??? - .. data:: Py_TPFLAGS_METHOD_DESCRIPTOR + .. c:macro:: Py_TPFLAGS_METHOD_DESCRIPTOR This bit indicates that objects behave like unbound methods. @@ -1140,15 +1142,15 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** This flag is never inherited by types without the - :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is + :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is inherited whenever :c:member:`~PyTypeObject.tp_descr_get` is inherited. - .. data:: Py_TPFLAGS_MANAGED_DICT + .. c:macro:: Py_TPFLAGS_MANAGED_DICT This bit indicates that instances of the class have a ``__dict__`` attribute, and that the space for the dictionary is managed by the VM. - If this flag is set, :const:`Py_TPFLAGS_HAVE_GC` should also be set. + If this flag is set, :c:macro:`Py_TPFLAGS_HAVE_GC` should also be set. .. versionadded:: 3.12 @@ -1158,7 +1160,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:member:`~PyTypeObject.tp_dictoffset` field is set in a superclass. - .. data:: Py_TPFLAGS_MANAGED_WEAKREF + .. c:macro:: Py_TPFLAGS_MANAGED_WEAKREF This bit indicates that instances of the class should be weakly referenceable. @@ -1171,7 +1173,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:member:`~PyTypeObject.tp_weaklistoffset` field is set in a superclass. - .. data:: Py_TPFLAGS_ITEMS_AT_END + .. c:macro:: Py_TPFLAGS_ITEMS_AT_END Only usable with variable-size types, i.e. ones with non-zero :c:member:`~PyObject.tp_itemsize`. @@ -1194,14 +1196,14 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. XXX Document more flags here? - .. data:: Py_TPFLAGS_LONG_SUBCLASS - .. data:: Py_TPFLAGS_LIST_SUBCLASS - .. data:: Py_TPFLAGS_TUPLE_SUBCLASS - .. data:: Py_TPFLAGS_BYTES_SUBCLASS - .. data:: Py_TPFLAGS_UNICODE_SUBCLASS - .. data:: Py_TPFLAGS_DICT_SUBCLASS - .. data:: Py_TPFLAGS_BASE_EXC_SUBCLASS - .. data:: Py_TPFLAGS_TYPE_SUBCLASS + .. c:macro:: Py_TPFLAGS_LONG_SUBCLASS + .. c:macro:: Py_TPFLAGS_LIST_SUBCLASS + .. c:macro:: Py_TPFLAGS_TUPLE_SUBCLASS + .. c:macro:: Py_TPFLAGS_BYTES_SUBCLASS + .. c:macro:: Py_TPFLAGS_UNICODE_SUBCLASS + .. c:macro:: Py_TPFLAGS_DICT_SUBCLASS + .. c:macro:: Py_TPFLAGS_BASE_EXC_SUBCLASS + .. c:macro:: Py_TPFLAGS_TYPE_SUBCLASS These flags are used by functions such as :c:func:`PyLong_Check` to quickly determine if a type is a subclass @@ -1212,7 +1214,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) will behave differently depending on what kind of check is used. - .. data:: Py_TPFLAGS_HAVE_FINALIZE + .. c:macro:: Py_TPFLAGS_HAVE_FINALIZE This bit is set when the :c:member:`~PyTypeObject.tp_finalize` slot is present in the type structure. @@ -1225,7 +1227,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) type structure. - .. data:: Py_TPFLAGS_HAVE_VECTORCALL + .. c:macro:: Py_TPFLAGS_HAVE_VECTORCALL This bit is set when the class implements the :ref:`vectorcall protocol `. @@ -1245,7 +1247,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) This flag can now be inherited by mutable classes. - .. data:: Py_TPFLAGS_IMMUTABLETYPE + .. c:macro:: Py_TPFLAGS_IMMUTABLETYPE This bit is set for type objects that are immutable: type attributes cannot be set nor deleted. @@ -1258,7 +1260,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_DISALLOW_INSTANTIATION + .. c:macro:: Py_TPFLAGS_DISALLOW_INSTANTIATION Disallow creating instances of the type: set :c:member:`~PyTypeObject.tp_new` to NULL and don't create the ``__new__`` @@ -1289,7 +1291,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_MAPPING + .. c:macro:: Py_TPFLAGS_MAPPING This bit indicates that instances of the class may match mapping patterns when used as the subject of a :keyword:`match` block. It is automatically @@ -1298,20 +1300,20 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: - :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are + :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` are mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** This flag is inherited by types that do not already set - :const:`Py_TPFLAGS_SEQUENCE`. + :c:macro:`Py_TPFLAGS_SEQUENCE`. .. seealso:: :pep:`634` -- Structural Pattern Matching: Specification .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_SEQUENCE + .. c:macro:: Py_TPFLAGS_SEQUENCE This bit indicates that instances of the class may match sequence patterns when used as the subject of a :keyword:`match` block. It is automatically @@ -1320,20 +1322,20 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: - :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are + :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` are mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** This flag is inherited by types that do not already set - :const:`Py_TPFLAGS_MAPPING`. + :c:macro:`Py_TPFLAGS_MAPPING`. .. seealso:: :pep:`634` -- Structural Pattern Matching: Specification .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_VALID_VERSION_TAG + .. c:macro:: Py_TPFLAGS_VALID_VERSION_TAG Internal. Do not set or unset this flag. To indicate that a class has changed call :c:func:`PyType_Modified` @@ -1357,7 +1359,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: traverseproc PyTypeObject.tp_traverse An optional pointer to a traversal function for the garbage collector. This is - only used if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: + only used if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: int tp_traverse(PyObject *self, visitproc visit, void *arg); @@ -1419,10 +1421,10 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :const:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_clear` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. @@ -1430,7 +1432,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: inquiry PyTypeObject.tp_clear An optional pointer to a clear function for the garbage collector. This is only - used if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: + used if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: int tp_clear(PyObject *); @@ -1486,10 +1488,10 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :const:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_traverse` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. @@ -1511,21 +1513,21 @@ and :c:type:`PyType_Type` effectively act as defaults.) The following constants are defined to be used as the third argument for :c:member:`~PyTypeObject.tp_richcompare` and for :c:func:`PyObject_RichCompare`: - +----------------+------------+ - | Constant | Comparison | - +================+============+ - | :const:`Py_LT` | ``<`` | - +----------------+------------+ - | :const:`Py_LE` | ``<=`` | - +----------------+------------+ - | :const:`Py_EQ` | ``==`` | - +----------------+------------+ - | :const:`Py_NE` | ``!=`` | - +----------------+------------+ - | :const:`Py_GT` | ``>`` | - +----------------+------------+ - | :const:`Py_GE` | ``>=`` | - +----------------+------------+ + +------------------+------------+ + | Constant | Comparison | + +==================+============+ + | :c:macro:`Py_LT` | ``<`` | + +------------------+------------+ + | :c:macro:`Py_LE` | ``<=`` | + +------------------+------------+ + | :c:macro:`Py_EQ` | ``==`` | + +------------------+------------+ + | :c:macro:`Py_NE` | ``!=`` | + +------------------+------------+ + | :c:macro:`Py_GT` | ``>`` | + +------------------+------------+ + | :c:macro:`Py_GE` | ``>=`` | + +------------------+------------+ The following macro is defined to ease writing rich comparison functions: @@ -1563,7 +1565,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: Py_ssize_t PyTypeObject.tp_weaklistoffset - While this field is still supported, :const:`Py_TPFLAGS_MANAGED_WEAKREF` + While this field is still supported, :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` should be used instead, if at all possible. If the instances of this type are weakly referenceable, this field is greater @@ -1576,7 +1578,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) Do not confuse this field with :c:member:`~PyTypeObject.tp_weaklist`; that is the list head for weak references to the type object itself. - It is an error to set both the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit and + It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit and :c:member:`~PyTypeObject.tp_weaklist`. **Inheritance:** @@ -1588,7 +1590,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - If the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the + If the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the :c:member:`~PyTypeObject.tp_dict` field, then :c:member:`~PyTypeObject.tp_weaklistoffset` will be set to a negative value, to indicate that it is unsafe to use this field. @@ -1782,7 +1784,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: Py_ssize_t PyTypeObject.tp_dictoffset - While this field is still supported, :const:`Py_TPFLAGS_MANAGED_DICT` should be + While this field is still supported, :c:macro:`Py_TPFLAGS_MANAGED_DICT` should be used instead, if at all possible. If the instances of this type have a dictionary containing instance variables, @@ -1801,7 +1803,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr` when accessing an attribute on the object. - It is an error to set both the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit and + It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit and :c:member:`~PyTypeObject.tp_dictoffset`. **Inheritance:** @@ -1809,14 +1811,14 @@ and :c:type:`PyType_Type` effectively act as defaults.) This field is inherited by subtypes. A subtype should not override this offset; doing so could be unsafe, if C code tries to access the dictionary at the previous offset. - To properly support inheritance, use :const:`Py_TPFLAGS_MANAGED_DICT`. + To properly support inheritance, use :c:macro:`Py_TPFLAGS_MANAGED_DICT`. **Default:** This slot has no default. For :ref:`static types `, if the field is ``NULL`` then no :attr:`__dict__` gets created for instances. - If the :const:`Py_TPFLAGS_MANAGED_DICT` bit is set in the + If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the :c:member:`~PyTypeObject.tp_dict` field, then :c:member:`~PyTypeObject.tp_dictoffset` will be set to ``-1``, to indicate that it is unsafe to use this field. @@ -1903,7 +1905,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) in :c:member:`~PyTypeObject.tp_new`, while for mutable types, most initialization should be deferred to :c:member:`~PyTypeObject.tp_init`. - Set the :const:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag to disallow creating + Set the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag to disallow creating instances of the type in Python. **Inheritance:** @@ -1937,7 +1939,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) In dynamic subtypes, this field is set to a deallocator suitable to match :c:func:`PyType_GenericAlloc` and the value of the - :const:`Py_TPFLAGS_HAVE_GC` flag bit. + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. For static subtypes, :c:type:`PyBaseObject_Type` uses PyObject_Del. @@ -1948,7 +1950,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) The garbage collector needs to know whether a particular object is collectible or not. Normally, it is sufficient to look at the object's type's - :c:member:`~PyTypeObject.tp_flags` field, and check the :const:`Py_TPFLAGS_HAVE_GC` flag bit. But + :c:member:`~PyTypeObject.tp_flags` field, and check the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. But some types have a mixture of statically and dynamically allocated instances, and the statically allocated instances are not collectible. Such types should define this function; it should return ``1`` for a collectible instance, and @@ -1967,7 +1969,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** This slot has no default. If this field is ``NULL``, - :const:`Py_TPFLAGS_HAVE_GC` is used as the functional equivalent. + :c:macro:`Py_TPFLAGS_HAVE_GC` is used as the functional equivalent. .. c:member:: PyObject* PyTypeObject.tp_bases @@ -2114,7 +2116,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionchanged:: 3.8 Before version 3.8 it was necessary to set the - :const:`Py_TPFLAGS_HAVE_FINALIZE` flags bit in order for this field to be + :c:macro:`Py_TPFLAGS_HAVE_FINALIZE` flags bit in order for this field to be used. This is no longer required. .. seealso:: "Safe object finalization" (:pep:`442`) @@ -2173,7 +2175,7 @@ Heap Types An alternative to :ref:`static types ` is *heap-allocated types*, or *heap types* for short, which correspond closely to classes created by -Python's ``class`` statement. Heap types have the :const:`Py_TPFLAGS_HEAPTYPE` +Python's ``class`` statement. Heap types have the :c:macro:`Py_TPFLAGS_HEAPTYPE` flag set. This is done by filling a :c:type:`PyType_Spec` structure and calling @@ -2781,7 +2783,7 @@ A type that supports weakrefs, instance dicts, and hashing:: A str subclass that cannot be subclassed and cannot be called to create instances (e.g. uses a separate factory func) using -:c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag:: +:c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag:: typedef struct { PyUnicodeObject raw; diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 33437b6193919..8a989c2afe871 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1299,7 +1299,7 @@ the user settings on the machine running the codec. Encode the Unicode object using the specified code page and return a Python bytes object. Return ``NULL`` if an exception was raised by the codec. Use - :c:data:`CP_ACP` code page to get the MBCS encoder. + :c:macro:`CP_ACP` code page to get the MBCS encoder. .. versionadded:: 3.3 @@ -1418,11 +1418,11 @@ They all return ``NULL`` or ``-1`` if an exception occurs. Rich compare two Unicode strings and return one of the following: * ``NULL`` in case an exception was raised - * :const:`Py_True` or :const:`Py_False` for successful comparisons - * :const:`Py_NotImplemented` in case the type combination is unknown + * :c:data:`Py_True` or :c:data:`Py_False` for successful comparisons + * :c:data:`Py_NotImplemented` in case the type combination is unknown - Possible values for *op* are :const:`Py_GT`, :const:`Py_GE`, :const:`Py_EQ`, - :const:`Py_NE`, :const:`Py_LT`, and :const:`Py_LE`. + Possible values for *op* are :c:macro:`Py_GT`, :c:macro:`Py_GE`, :c:macro:`Py_EQ`, + :c:macro:`Py_NE`, :c:macro:`Py_LT`, and :c:macro:`Py_LE`. .. c:function:: PyObject* PyUnicode_Format(PyObject *format, PyObject *args) diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 000a2d3d8790b..1e8a945512d78 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -12,8 +12,8 @@ file or a buffer, but they will not let you interact in a more detailed way with the interpreter. Several of these functions accept a start symbol from the grammar as a -parameter. The available start symbols are :const:`Py_eval_input`, -:const:`Py_file_input`, and :const:`Py_single_input`. These are described +parameter. The available start symbols are :c:data:`Py_eval_input`, +:c:data:`Py_file_input`, and :c:data:`Py_single_input`. These are described following the functions which accept them as parameters. Note also that several of these functions take :c:expr:`FILE*` parameters. One @@ -256,8 +256,8 @@ the same library that the Python runtime is using. Parse and compile the Python source code in *str*, returning the resulting code object. The start token is given by *start*; this can be used to constrain the - code which can be compiled and should be :const:`Py_eval_input`, - :const:`Py_file_input`, or :const:`Py_single_input`. The filename specified by + code which can be compiled and should be :c:data:`Py_eval_input`, + :c:data:`Py_file_input`, or :c:data:`Py_single_input`. The filename specified by *filename* is used to construct the code object and may appear in tracebacks or :exc:`SyntaxError` exception messages. This returns ``NULL`` if the code cannot be parsed or compiled. diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index d9bf4fd6c7ae0..76e0490d0d22d 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -335,7 +335,7 @@ When using only ``METH_VARARGS``, the function should expect the Python-level parameters to be passed in as a tuple acceptable for parsing via :c:func:`PyArg_ParseTuple`; more information on this function is provided below. -The :const:`METH_KEYWORDS` bit may be set in the third field if keyword +The :c:macro:`METH_KEYWORDS` bit may be set in the third field if keyword arguments should be passed to the function. In this case, the C function should accept a third ``PyObject *`` parameter which will be a dictionary of keywords. Use :c:func:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a @@ -527,7 +527,7 @@ be part of a module definition:: } This function must be registered with the interpreter using the -:const:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The +:c:macro:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The :c:func:`PyArg_ParseTuple` function and its arguments are documented in section :ref:`parsetuple`. diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index f89934a11f12a..ba09fb1b05c0b 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -151,7 +151,7 @@ only used for variable-sized objects and should otherwise be zero. base type will be :class:`object`, or else you will be adding data members to your base type, and therefore increasing its size. -We set the class flags to :const:`Py_TPFLAGS_DEFAULT`. :: +We set the class flags to :c:macro:`Py_TPFLAGS_DEFAULT`. :: .tp_flags = Py_TPFLAGS_DEFAULT, @@ -498,7 +498,7 @@ definitions:: {NULL} /* Sentinel */ }; -(note that we used the :const:`METH_NOARGS` flag to indicate that the method +(note that we used the :c:macro:`METH_NOARGS` flag to indicate that the method is expecting no arguments other than *self*) and assign it to the :c:member:`~PyTypeObject.tp_methods` slot:: @@ -508,7 +508,7 @@ and assign it to the :c:member:`~PyTypeObject.tp_methods` slot:: Finally, we'll make our type usable as a base class for subclassing. We've written our methods carefully so far so that they don't make any assumptions about the type of the object being created or used, so all we need to do is -to add the :const:`Py_TPFLAGS_BASETYPE` to our class flag definition:: +to add the :c:macro:`Py_TPFLAGS_BASETYPE` to our class flag definition:: .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, @@ -774,7 +774,7 @@ and ``Custom_clear``:: Py_TYPE(self)->tp_free((PyObject *) self); } -Finally, we add the :const:`Py_TPFLAGS_HAVE_GC` flag to the class flags:: +Finally, we add the :c:macro:`Py_TPFLAGS_HAVE_GC` flag to the class flags:: .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index 8adb85f3a8740..f01801ea47212 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -298,10 +298,10 @@ Watch out for the following two points in particular (but note that this is not a comprehensive list): * Unlike static types, heap type objects are mutable by default. - Use the :c:data:`Py_TPFLAGS_IMMUTABLETYPE` flag to prevent mutability. + Use the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag to prevent mutability. * Heap types inherit :c:member:`~PyTypeObject.tp_new` by default, so it may become possible to instantiate them from Python code. - You can prevent this with the :c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. + You can prevent this with the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. Defining Heap Types @@ -333,12 +333,12 @@ To avoid memory leaks, instances of heap types must implement the garbage collection protocol. That is, heap types should: -- Have the :c:data:`Py_TPFLAGS_HAVE_GC` flag. +- Have the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. - Define a traverse function using ``Py_tp_traverse``, which visits the type (e.g. using :c:expr:`Py_VISIT(Py_TYPE(self))`). Please refer to the :ref:`the documentation ` of -:c:data:`Py_TPFLAGS_HAVE_GC` and :c:member:`~PyTypeObject.tp_traverse` +:c:macro:`Py_TPFLAGS_HAVE_GC` and :c:member:`~PyTypeObject.tp_traverse` for additional considerations. If your traverse function delegates to the ``tp_traverse`` of its base class @@ -411,7 +411,7 @@ that subclass, which may be defined in different module than yours. pass For a method to get its "defining class", it must use the -:data:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS` +:ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS ` :c:type:`calling convention ` and the corresponding :c:type:`PyCMethod` signature:: diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 3b31384edeb54..97d5121c4e8a5 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -818,7 +818,7 @@ iterations of the loop. .. opcode:: MATCH_MAPPING If ``STACK[-1]`` is an instance of :class:`collections.abc.Mapping` (or, more - technically: if it has the :const:`Py_TPFLAGS_MAPPING` flag set in its + technically: if it has the :c:macro:`Py_TPFLAGS_MAPPING` flag set in its :c:member:`~PyTypeObject.tp_flags`), push ``True`` onto the stack. Otherwise, push ``False``. @@ -829,7 +829,7 @@ iterations of the loop. If ``STACK[-1]`` is an instance of :class:`collections.abc.Sequence` and is *not* an instance of :class:`str`/:class:`bytes`/:class:`bytearray` (or, more technically: if it has - the :const:`Py_TPFLAGS_SEQUENCE` flag set in its :c:member:`~PyTypeObject.tp_flags`), + the :c:macro:`Py_TPFLAGS_SEQUENCE` flag set in its :c:member:`~PyTypeObject.tp_flags`), push ``True`` onto the stack. Otherwise, push ``False``. .. versionadded:: 3.10 diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 83abb5d5ca1e4..b35010498f214 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -4308,7 +4308,7 @@ written in Python, such as a mail server's external command delivery program. specified. If the value specified is 0, the child's process group ID will be made the same as its process ID. If the value of *setpgroup* is not set, the child will inherit the parent's process group ID. This argument corresponds - to the C library :c:data:`POSIX_SPAWN_SETPGROUP` flag. + to the C library :c:macro:`POSIX_SPAWN_SETPGROUP` flag. If the *resetids* argument is ``True`` it will reset the effective UID and GID of the child to the real UID and GID of the parent process. If the @@ -4316,27 +4316,27 @@ written in Python, such as a mail server's external command delivery program. the parent. In either case, if the set-user-ID and set-group-ID permission bits are enabled on the executable file, their effect will override the setting of the effective UID and GID. This argument corresponds to the C - library :c:data:`POSIX_SPAWN_RESETIDS` flag. + library :c:macro:`POSIX_SPAWN_RESETIDS` flag. If the *setsid* argument is ``True``, it will create a new session ID - for ``posix_spawn``. *setsid* requires :c:data:`POSIX_SPAWN_SETSID` - or :c:data:`POSIX_SPAWN_SETSID_NP` flag. Otherwise, :exc:`NotImplementedError` + for ``posix_spawn``. *setsid* requires :c:macro:`POSIX_SPAWN_SETSID` + or :c:macro:`POSIX_SPAWN_SETSID_NP` flag. Otherwise, :exc:`NotImplementedError` is raised. The *setsigmask* argument will set the signal mask to the signal set specified. If the parameter is not used, then the child inherits the parent's signal mask. This argument corresponds to the C library - :c:data:`POSIX_SPAWN_SETSIGMASK` flag. + :c:macro:`POSIX_SPAWN_SETSIGMASK` flag. The *sigdef* argument will reset the disposition of all signals in the set specified. This argument corresponds to the C library - :c:data:`POSIX_SPAWN_SETSIGDEF` flag. + :c:macro:`POSIX_SPAWN_SETSIGDEF` flag. The *scheduler* argument must be a tuple containing the (optional) scheduler policy and an instance of :class:`sched_param` with the scheduler parameters. A value of ``None`` in the place of the scheduler policy indicates that is not being provided. This argument is a combination of the C library - :c:data:`POSIX_SPAWN_SETSCHEDPARAM` and :c:data:`POSIX_SPAWN_SETSCHEDULER` + :c:macro:`POSIX_SPAWN_SETSCHEDPARAM` and :c:macro:`POSIX_SPAWN_SETSCHEDULER` flags. .. audit-event:: os.posix_spawn path,argv,env os.posix_spawn diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 6d30eccab1990..12ad18d411961 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1840,7 +1840,7 @@ like ``TYPE_PARAMS_OF_ListOrSet`` are not actually bound at runtime. * a class that inherits from :class:`collections.abc.Sequence` * a Python class that has been registered as :class:`collections.abc.Sequence` - * a builtin class that has its (CPython) :data:`Py_TPFLAGS_SEQUENCE` bit set + * a builtin class that has its (CPython) :c:macro:`Py_TPFLAGS_SEQUENCE` bit set * a class that inherits from any of the above The following standard library classes are sequences: @@ -1859,7 +1859,7 @@ like ``TYPE_PARAMS_OF_ListOrSet`` are not actually bound at runtime. * a class that inherits from :class:`collections.abc.Mapping` * a Python class that has been registered as :class:`collections.abc.Mapping` - * a builtin class that has its (CPython) :data:`Py_TPFLAGS_MAPPING` bit set + * a builtin class that has its (CPython) :c:macro:`Py_TPFLAGS_MAPPING` bit set * a class that inherits from any of the above The standard library classes :class:`dict` and :class:`types.MappingProxyType` diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 1b470d395d6d5..4bf67eb439ec6 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -897,11 +897,11 @@ conflict. * ``default``: use the :ref:`default memory allocators `. * ``malloc``: use the :c:func:`malloc` function of the C library - for all domains (:c:data:`PYMEM_DOMAIN_RAW`, :c:data:`PYMEM_DOMAIN_MEM`, - :c:data:`PYMEM_DOMAIN_OBJ`). + for all domains (:c:macro:`PYMEM_DOMAIN_RAW`, :c:macro:`PYMEM_DOMAIN_MEM`, + :c:macro:`PYMEM_DOMAIN_OBJ`). * ``pymalloc``: use the :ref:`pymalloc allocator ` for - :c:data:`PYMEM_DOMAIN_MEM` and :c:data:`PYMEM_DOMAIN_OBJ` domains and use - the :c:func:`malloc` function for the :c:data:`PYMEM_DOMAIN_RAW` domain. + :c:macro:`PYMEM_DOMAIN_MEM` and :c:macro:`PYMEM_DOMAIN_OBJ` domains and use + the :c:func:`malloc` function for the :c:macro:`PYMEM_DOMAIN_RAW` domain. Install :ref:`debug hooks `: diff --git a/Doc/whatsnew/2.2.rst b/Doc/whatsnew/2.2.rst index 82aff0be1ed3b..44e9bd8d492bf 100644 --- a/Doc/whatsnew/2.2.rst +++ b/Doc/whatsnew/2.2.rst @@ -1105,11 +1105,11 @@ code, none of the changes described here will affect you very much. expected, and a set of pointers to :c:expr:`PyObject*` variables that will be filled in with argument values. -* Two new flags :const:`METH_NOARGS` and :const:`METH_O` are available in method +* Two new flags :c:macro:`METH_NOARGS` and :c:macro:`METH_O` are available in method definition tables to simplify implementation of methods with no arguments or a single untyped argument. Calling such methods is more efficient than calling a - corresponding method that uses :const:`METH_VARARGS`. Also, the old - :const:`METH_OLDARGS` style of writing C methods is now officially deprecated. + corresponding method that uses :c:macro:`METH_VARARGS`. Also, the old + :c:macro:`METH_OLDARGS` style of writing C methods is now officially deprecated. * Two new wrapper functions, :c:func:`PyOS_snprintf` and :c:func:`PyOS_vsnprintf` were added to provide cross-platform implementations for the relatively new diff --git a/Doc/whatsnew/2.3.rst b/Doc/whatsnew/2.3.rst index 4eb864f5092d3..14f951174ba54 100644 --- a/Doc/whatsnew/2.3.rst +++ b/Doc/whatsnew/2.3.rst @@ -1474,7 +1474,7 @@ complete list of changes, or look through the CVS logs for all the details. * On Windows, the :mod:`socket` module now ships with Secure Sockets Layer (SSL) support. -* The value of the C :const:`PYTHON_API_VERSION` macro is now exposed at the +* The value of the C :c:macro:`PYTHON_API_VERSION` macro is now exposed at the Python level as ``sys.api_version``. The current exception can be cleared by calling the new :func:`sys.exc_clear` function. @@ -1899,10 +1899,10 @@ Changes to Python's build process and to the C API include: * The :c:func:`PyArg_NoArgs` macro is now deprecated, and code that uses it should be changed. For Python 2.2 and later, the method definition table can - specify the :const:`METH_NOARGS` flag, signalling that there are no arguments, + specify the :c:macro:`METH_NOARGS` flag, signalling that there are no arguments, and the argument checking can then be removed. If compatibility with pre-2.2 versions of Python is important, the code could use ``PyArg_ParseTuple(args, - "")`` instead, but this will be slower than using :const:`METH_NOARGS`. + "")`` instead, but this will be slower than using :c:macro:`METH_NOARGS`. * :c:func:`PyArg_ParseTuple` accepts new format characters for various sizes of unsigned integers: ``B`` for :c:expr:`unsigned char`, ``H`` for :c:expr:`unsigned @@ -1918,7 +1918,7 @@ Changes to Python's build process and to the C API include: seconds, according to one measurement). * It's now possible to define class and static methods for a C extension type by - setting either the :const:`METH_CLASS` or :const:`METH_STATIC` flags in a + setting either the :c:macro:`METH_CLASS` or :c:macro:`METH_STATIC` flags in a method's :c:type:`PyMethodDef` structure. * Python now includes a copy of the Expat XML parser's source code, removing any diff --git a/Doc/whatsnew/2.4.rst b/Doc/whatsnew/2.4.rst index 98dc83fe935d5..d68f600ba44f6 100644 --- a/Doc/whatsnew/2.4.rst +++ b/Doc/whatsnew/2.4.rst @@ -1476,7 +1476,7 @@ Some of the changes to Python's build process and to the C API are: :c:func:`PyArg_ParseTupleAndKeywords` but takes a :c:type:`va_list` instead of a number of arguments. (Contributed by Greg Chapman.) -* A new method flag, :const:`METH_COEXISTS`, allows a function defined in slots +* A new method flag, :c:macro:`METH_COEXISTS`, allows a function defined in slots to co-exist with a :c:type:`PyCFunction` having the same name. This can halve the access time for a method such as :meth:`set.__contains__`. (Contributed by Raymond Hettinger.) diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index 84bb651e68eed..72a273fdd8d12 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -1138,11 +1138,11 @@ indicate that the external caller is done. The *flags* argument to :c:func:`PyObject_GetBuffer` specifies constraints upon the memory returned. Some examples are: - * :const:`PyBUF_WRITABLE` indicates that the memory must be writable. + * :c:macro:`PyBUF_WRITABLE` indicates that the memory must be writable. - * :const:`PyBUF_LOCK` requests a read-only or exclusive lock on the memory. + * :c:macro:`PyBUF_LOCK` requests a read-only or exclusive lock on the memory. - * :const:`PyBUF_C_CONTIGUOUS` and :const:`PyBUF_F_CONTIGUOUS` + * :c:macro:`PyBUF_C_CONTIGUOUS` and :c:macro:`PyBUF_F_CONTIGUOUS` requests a C-contiguous (last dimension varies the fastest) or Fortran-contiguous (first dimension varies the fastest) array layout. diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index 36afcb163f1af..e6bc9a21b365d 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -2231,7 +2231,7 @@ Changes to Python's build process and to the C API include: * When using the :c:type:`PyMemberDef` structure to define attributes of a type, Python will no longer let you try to delete or set a - :const:`T_STRING_INPLACE` attribute. + :c:macro:`T_STRING_INPLACE` attribute. .. rev 79644 diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 661eeaedbfc0d..7f80852c49548 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -2124,11 +2124,11 @@ New Features These functions allow to activate, deactivate and query the state of the garbage collector from C code without having to import the :mod:`gc` module. -* Add a new :c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow +* Add a new :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow creating type instances. (Contributed by Victor Stinner in :issue:`43916`.) -* Add a new :c:data:`Py_TPFLAGS_IMMUTABLETYPE` type flag for creating immutable +* Add a new :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` type flag for creating immutable type objects: type attributes cannot be set nor deleted. (Contributed by Victor Stinner and Erlend E. Aasland in :issue:`43908`.) @@ -2187,9 +2187,9 @@ Porting to Python 3.10 been included directly, consider including ``Python.h`` instead. (Contributed by Nicholas Sim in :issue:`35134`.) -* Use the :c:data:`Py_TPFLAGS_IMMUTABLETYPE` type flag to create immutable type - objects. Do not rely on :c:data:`Py_TPFLAGS_HEAPTYPE` to decide if a type - object is mutable or not; check if :c:data:`Py_TPFLAGS_IMMUTABLETYPE` is set +* Use the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` type flag to create immutable type + objects. Do not rely on :c:macro:`Py_TPFLAGS_HEAPTYPE` to decide if a type + object is mutable or not; check if :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` is set instead. (Contributed by Victor Stinner and Erlend E. Aasland in :issue:`43908`.) diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index c06ce78368985..5ff99fc2d3f61 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -2347,11 +2347,11 @@ Porting to Python 3.11 #endif * The :c:func:`PyType_Ready` function now raises an error if a type is defined - with the :const:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function + with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function (:c:member:`PyTypeObject.tp_traverse`). (Contributed by Victor Stinner in :issue:`44263`.) -* Heap types with the :const:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit +* Heap types with the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit the :pep:`590` vectorcall protocol. Previously, this was only possible for :ref:`static types `. (Contributed by Erlend E. Aasland in :issue:`43908`) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 068618fe48e1b..03d559e455393 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1110,7 +1110,7 @@ Pending Removal in Python 3.14 * :class:`typing.ByteString`, deprecated since Python 3.9, now causes a :exc:`DeprecationWarning` to be emitted when it is used. -* Creating immutable types (:data:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable +* Creating immutable types (:c:macro:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable bases using the C API. * Deprecated the *isdst* parameter in :func:`email.utils.localtime`. @@ -1621,7 +1621,7 @@ New Features inheriting or extending the base class size. - :c:func:`PyObject_GetTypeData` and :c:func:`PyType_GetTypeDataSize` added to allow access to subclass-specific instance data. - - :const:`Py_TPFLAGS_ITEMS_AT_END` and :c:func:`PyObject_GetItemData` + - :c:macro:`Py_TPFLAGS_ITEMS_AT_END` and :c:func:`PyObject_GetItemData` added to allow safely extending certain variable-sized types, including :c:var:`PyType_Type`. - :c:macro:`Py_RELATIVE_OFFSET` added to allow defining @@ -1638,20 +1638,20 @@ New Features :ref:`the vectorcall protocol ` was added to the :ref:`Limited API `: - * :const:`Py_TPFLAGS_HAVE_VECTORCALL` + * :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` * :c:func:`PyVectorcall_NARGS` * :c:func:`PyVectorcall_Call` * :c:type:`vectorcallfunc` - The :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class + The :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class when the class's :py:meth:`~object.__call__` method is reassigned. This makes vectorcall safe to use with mutable types (i.e. heap types - without the immutable flag, :const:`Py_TPFLAGS_IMMUTABLETYPE`). + without the immutable flag, :c:macro:`Py_TPFLAGS_IMMUTABLETYPE`). Mutable types that do not override :c:member:`~PyTypeObject.tp_call` now inherit the ``Py_TPFLAGS_HAVE_VECTORCALL`` flag. (Contributed by Petr Viktorin in :gh:`93274`.) - The :const:`Py_TPFLAGS_MANAGED_DICT` and :const:`Py_TPFLAGS_MANAGED_WEAKREF` + The :c:macro:`Py_TPFLAGS_MANAGED_DICT` and :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` flags have been added. This allows extensions classes to support object ``__dict__`` and weakrefs with less bookkeeping, using less memory and with faster access. @@ -1662,7 +1662,7 @@ New Features * :c:func:`PyObject_Vectorcall` * :c:func:`PyObject_VectorcallMethod` - * :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` + * :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` This means that both the incoming and outgoing ends of the vector call protocol are now available in the :ref:`Limited API `. (Contributed @@ -1784,13 +1784,13 @@ Porting to Python 3.12 (Contributed by Philip Georgi in :gh:`95504`.) * Extension classes wanting to add a ``__dict__`` or weak reference slot - should use :const:`Py_TPFLAGS_MANAGED_DICT` and - :const:`Py_TPFLAGS_MANAGED_WEAKREF` instead of ``tp_dictoffset`` and + should use :c:macro:`Py_TPFLAGS_MANAGED_DICT` and + :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead of ``tp_dictoffset`` and ``tp_weaklistoffset``, respectively. The use of ``tp_dictoffset`` and ``tp_weaklistoffset`` is still supported, but does not fully support multiple inheritance (:gh:`95589`), and performance may be worse. - Classes declaring :const:`Py_TPFLAGS_MANAGED_DICT` should call + Classes declaring :c:macro:`Py_TPFLAGS_MANAGED_DICT` should call :c:func:`!_PyObject_VisitManagedDict` and :c:func:`!_PyObject_ClearManagedDict` to traverse and clear their instance's dictionaries. To clear weakrefs, call :c:func:`PyObject_ClearWeakRefs`, as before. @@ -1844,7 +1844,7 @@ Porting to Python 3.12 :c:member:`~PyTypeObject.tp_init` instead. - If the metaclass doesn't need to be instantiated from Python, set its ``tp_new`` to ``NULL`` using - the :const:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. + the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. This makes it acceptable for ``PyType_From*`` functions. - Avoid ``PyType_From*`` functions: if you don't need C-specific features @@ -1895,7 +1895,7 @@ Deprecated :c:type:`PyConfig` instead. (Contributed by Victor Stinner in :gh:`77782`.) -* Creating immutable types (:const:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable +* Creating immutable types (:c:macro:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable bases is deprecated and will be disabled in Python 3.14. * The ``structmember.h`` header is deprecated, though it continues to be diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 3a681754e25dd..07a65d264c435 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -650,8 +650,8 @@ compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks: * Detect writes before the start of a buffer (buffer underflows) * Detect writes after the end of a buffer (buffer overflows) * Check that the :term:`GIL ` is held when allocator - functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and - :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. + functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and + :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. Checking if the GIL is held is also a new feature of Python 3.6. @@ -1822,7 +1822,7 @@ Optimizations up to 80% faster. (Contributed by Josh Snider in :issue:`26574`). * Allocator functions of the :c:func:`PyMem_Malloc` domain - (:c:data:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator + (:c:macro:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator ` instead of :c:func:`malloc` function of the C library. The pymalloc allocator is optimized for objects smaller or equal to 512 bytes with a short lifetime, and use :c:func:`malloc` for larger memory blocks. @@ -1874,8 +1874,8 @@ Build and C API Changes (Original patch by Alecsandru Patrascu of Intel in :issue:`26359`.) * The :term:`GIL ` must now be held when allocator - functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and - :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. + functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and + :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data failed. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 85e088b64acb2..61249d479fa04 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -2116,7 +2116,7 @@ Changes in the C API extension types across feature releases, anymore. A :c:type:`PyTypeObject` exported by a third-party extension module is supposed to have all the slots expected in the current Python version, including - :c:member:`~PyTypeObject.tp_finalize` (:const:`Py_TPFLAGS_HAVE_FINALIZE` + :c:member:`~PyTypeObject.tp_finalize` (:c:macro:`Py_TPFLAGS_HAVE_FINALIZE` is not checked anymore before reading :c:member:`~PyTypeObject.tp_finalize`). (Contributed by Antoine Pitrou in :issue:`32388`.) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index fd86db9630235..49c8bd2f3e07d 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1276,7 +1276,7 @@ New Features * :pep:`573`: Added :c:func:`PyType_FromModuleAndSpec` to associate a module with a class; :c:func:`PyType_GetModule` and :c:func:`PyType_GetModuleState` to retrieve the module and its state; and - :c:data:`PyCMethod` and :c:data:`METH_METHOD` to allow a method to + :c:data:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to access the class it was defined in. (Contributed by Marcel Plch and Petr Viktorin in :issue:`38787`.) diff --git a/Misc/NEWS.d/3.10.0a3.rst b/Misc/NEWS.d/3.10.0a3.rst index 699a0dd9e8d7c..70b79f5f250a1 100644 --- a/Misc/NEWS.d/3.10.0a3.rst +++ b/Misc/NEWS.d/3.10.0a3.rst @@ -1466,7 +1466,7 @@ success. Patch by Victor Stinner. .. nonce: S3FWTP .. section: C API -The :c:data:`METH_FASTCALL` calling convention is added to the limited API. +The :c:macro:`METH_FASTCALL` calling convention is added to the limited API. The functions :c:func:`PyModule_AddType`, :c:func:`PyType_FromModuleAndSpec`, :c:func:`PyType_GetModule` and :c:func:`PyType_GetModuleState` are added to the limited API on Windows. diff --git a/Misc/NEWS.d/3.10.0b1.rst b/Misc/NEWS.d/3.10.0b1.rst index f29fc6632db26..38e0af17a2c66 100644 --- a/Misc/NEWS.d/3.10.0b1.rst +++ b/Misc/NEWS.d/3.10.0b1.rst @@ -1375,8 +1375,8 @@ Add "Annotations Best Practices" document as a new HOWTO. .. nonce: K5aSl1 .. section: Documentation -Document the new :const:`Py_TPFLAGS_MAPPING` and -:const:`Py_TPFLAGS_SEQUENCE` type flags. +Document the new :c:macro:`Py_TPFLAGS_MAPPING` and +:c:macro:`Py_TPFLAGS_SEQUENCE` type flags. .. @@ -1711,7 +1711,7 @@ IDLE's shell now shows prompts in a separate side-bar. .. nonce: wvWt23 .. section: C API -Add a new :c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow +Add a new :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow creating type instances. Patch by Victor Stinner. .. @@ -1759,7 +1759,7 @@ module. .. nonce: Co3YhZ .. section: C API -Introduce :const:`Py_TPFLAGS_IMMUTABLETYPE` flag for immutable type objects, +Introduce :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag for immutable type objects, and modify :c:func:`PyType_Ready` to set it for static types. Patch by Erlend E. Aasland. diff --git a/Misc/NEWS.d/3.11.0a1.rst b/Misc/NEWS.d/3.11.0a1.rst index 447d5c0e7d903..fb60ac5560102 100644 --- a/Misc/NEWS.d/3.11.0a1.rst +++ b/Misc/NEWS.d/3.11.0a1.rst @@ -888,7 +888,7 @@ zlib.decompress on input data that expands that large. .. nonce: YHuV_s .. section: Core and Builtins -Heap types with the :const:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit +Heap types with the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit the :pep:`590` vectorcall protocol. Previously, this was only possible for :ref:`static types `. Patch by Erlend E. Aasland. @@ -2575,7 +2575,7 @@ E. Aasland. .. nonce: bamAGF .. section: Library -Set the proper :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` +Set the proper :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` flags for subclasses created before a parent has been registered as a :class:`collections.abc.Mapping` or :class:`collections.abc.Sequence`. @@ -2693,7 +2693,7 @@ libgcc_s.so file (ex: EMFILE error). Patch by Victor Stinner. .. section: Library The _thread.RLock type now fully implement the GC protocol: add a traverse -function and the :const:`Py_TPFLAGS_HAVE_GC` flag. Patch by Victor Stinner. +function and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. Patch by Victor Stinner. .. @@ -5014,7 +5014,7 @@ must now be used to set an object type and size. Patch by Victor Stinner. .. section: C API The :c:func:`PyType_Ready` function now raises an error if a type is defined -with the :const:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function +with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function (:c:member:`PyTypeObject.tp_traverse`). Patch by Victor Stinner. .. diff --git a/Misc/NEWS.d/3.11.0a7.rst b/Misc/NEWS.d/3.11.0a7.rst index 8e7ccd4d6771e..2172c02e2fc84 100644 --- a/Misc/NEWS.d/3.11.0a7.rst +++ b/Misc/NEWS.d/3.11.0a7.rst @@ -275,7 +275,7 @@ initializing to ``list_extend``. Patch by Jeremiah Pascual. .. nonce: cnaIK3 .. section: Core and Builtins -Speed up throwing exception in generator with :const:`METH_FASTCALL` calling +Speed up throwing exception in generator with :c:macro:`METH_FASTCALL` calling convention. Patch by Kumar Aditya. .. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index ff5064f89d8dd..9afeba2526351 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -5856,8 +5856,8 @@ Configuration for the :ref:`integer string conversion length limitation Extensions classes that set ``tp_dictoffset`` and ``tp_weaklistoffset`` lose the support for multiple inheritance, but are now safe. Extension classes -should use :const:`Py_TPFLAGS_MANAGED_DICT` and -:const:`Py_TPFLAGS_MANAGED_WEAKREF` instead. +should use :c:macro:`Py_TPFLAGS_MANAGED_DICT` and +:c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead. .. @@ -5898,7 +5898,7 @@ Support C extensions using managed dictionaries by setting the .. nonce: QoDHEu .. section: C API -API for implementing vectorcall (:c:data:`Py_TPFLAGS_HAVE_VECTORCALL`, +API for implementing vectorcall (:c:macro:`Py_TPFLAGS_HAVE_VECTORCALL`, :c:func:`PyVectorcall_NARGS` and :c:func:`PyVectorcall_Call`) was added to the limited API and stable ABI. @@ -5920,12 +5920,12 @@ Philip Georgi. .. nonce: -DdGEy .. section: C API -The :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class +The :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class when the class's :py:meth:`~object.__call__` method is reassigned. This makes vectorcall safe to use with mutable types (i.e. heap types without the :const:`immutable ` flag). Mutable types that do not override :c:member:`~PyTypeObject.tp_call` now inherit the -:const:`Py_TPFLAGS_HAVE_VECTORCALL` flag. +:c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag. .. diff --git a/Misc/NEWS.d/3.12.0a2.rst b/Misc/NEWS.d/3.12.0a2.rst index d871384903e7c..5a3ccab02016f 100644 --- a/Misc/NEWS.d/3.12.0a2.rst +++ b/Misc/NEWS.d/3.12.0a2.rst @@ -279,7 +279,7 @@ Fix source locations of :keyword:`match` sub-patterns. Added the methods :c:func:`PyObject_Vectorcall` and :c:func:`PyObject_VectorcallMethod` to the :ref:`Limited API ` along -with the auxiliary macro constant :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. +with the auxiliary macro constant :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET`. The availability of these functions enables more efficient :PEP:`590` vector calls from binary extension modules that avoid argument boxing/unboxing diff --git a/Misc/NEWS.d/3.6.0a1.rst b/Misc/NEWS.d/3.6.0a1.rst index 53f09b3dfe336..98f1215fb9187 100644 --- a/Misc/NEWS.d/3.6.0a1.rst +++ b/Misc/NEWS.d/3.6.0a1.rst @@ -125,7 +125,7 @@ Setuptools 19.0. .. section: Core and Builtins Memory functions of the :c:func:`PyMem_Malloc` domain -(:c:data:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc allocator +(:c:macro:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc allocator ` rather than system :c:func:`malloc`. Applications calling :c:func:`PyMem_Malloc` without holding the GIL can now crash: use ``PYTHONMALLOC=debug`` environment variable to validate the usage of memory diff --git a/Misc/NEWS.d/3.9.0a1.rst b/Misc/NEWS.d/3.9.0a1.rst index 0888a5c43087b..7af7f2e412053 100644 --- a/Misc/NEWS.d/3.9.0a1.rst +++ b/Misc/NEWS.d/3.9.0a1.rst @@ -5706,7 +5706,7 @@ and :c:func:`_PyObject_CallMethodOneArg`. .. nonce: qZC0N_ .. section: C API -The :const:`METH_FASTCALL` calling convention has been documented. +The :c:macro:`METH_FASTCALL` calling convention has been documented. .. From webhook-mailer at python.org Fri Jul 21 07:48:34 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 11:48:34 -0000 Subject: [Python-checkins] [3.11] gh-106919: Use role :c:macro: for referencing the C "constants" (GH-106920) (GH-106952) Message-ID: https://github.com/python/cpython/commit/6bde1b9f049e35e557068bd8c6034ee9c6fe61dc commit: 6bde1b9f049e35e557068bd8c6034ee9c6fe61dc branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T14:48:30+03:00 summary: [3.11] gh-106919: Use role :c:macro: for referencing the C "constants" (GH-106920) (GH-106952) (cherry picked from commit fcc816dbff7ca66c26f57a506e4d2330fe41d0ff) files: M Doc/c-api/arg.rst M Doc/c-api/call.rst M Doc/c-api/complex.rst M Doc/c-api/exceptions.rst M Doc/c-api/file.rst M Doc/c-api/float.rst M Doc/c-api/gcsupport.rst M Doc/c-api/init.rst M Doc/c-api/long.rst M Doc/c-api/memory.rst M Doc/c-api/module.rst M Doc/c-api/object.rst M Doc/c-api/slice.rst M Doc/c-api/stable.rst M Doc/c-api/structures.rst M Doc/c-api/sys.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/c-api/veryhigh.rst M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/howto/isolating-extensions.rst M Doc/library/dis.rst M Doc/library/os.rst M Doc/reference/compound_stmts.rst M Doc/using/cmdline.rst M Doc/whatsnew/2.2.rst M Doc/whatsnew/2.3.rst M Doc/whatsnew/2.4.rst M Doc/whatsnew/2.6.rst M Doc/whatsnew/2.7.rst M Doc/whatsnew/3.10.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a3.rst M Misc/NEWS.d/3.10.0b1.rst M Misc/NEWS.d/3.11.0a1.rst M Misc/NEWS.d/3.11.0a7.rst M Misc/NEWS.d/3.6.0a1.rst M Misc/NEWS.d/3.9.0a1.rst diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index 6a53c79bd3bec..ee130ab28510b 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -380,7 +380,7 @@ Other objects *items*. Format units for sequences may be nested. It is possible to pass "long" integers (integers whose value exceeds the -platform's :const:`LONG_MAX`) however no proper range checking is done --- the +platform's :c:macro:`LONG_MAX`) however no proper range checking is done --- the most significant bits are silently truncated when the receiving field is too small to receive the value (actually, the semantics are inherited from downcasts in C --- your mileage may vary). @@ -492,7 +492,7 @@ API Functions A simpler form of parameter retrieval which does not use a format string to specify the types of the arguments. Functions which use this method to retrieve - their parameters should be declared as :const:`METH_VARARGS` in function or + their parameters should be declared as :c:macro:`METH_VARARGS` in function or method tables. The tuple containing the actual parameters should be passed as *args*; it must actually be a tuple. The length of the tuple must be at least *min* and no more than *max*; *min* and *max* may be equal. Additional diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index 36149f156c676..a574965bb1260 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -63,7 +63,7 @@ the arguments to an args tuple and kwargs dict anyway, then there is no point in implementing vectorcall. Classes can implement the vectorcall protocol by enabling the -:const:`Py_TPFLAGS_HAVE_VECTORCALL` flag and setting +:c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag and setting :c:member:`~PyTypeObject.tp_vectorcall_offset` to the offset inside the object structure where a *vectorcallfunc* appears. This is a pointer to a function with the following signature: @@ -75,7 +75,7 @@ This is a pointer to a function with the following signature: values of the keyword arguments. This can be *NULL* if there are no arguments. - *nargsf* is the number of positional arguments plus possibly the - :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. + :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. To get the actual number of positional arguments from *nargsf*, use :c:func:`PyVectorcall_NARGS`. - *kwnames* is a tuple containing the names of the keyword arguments; @@ -95,7 +95,7 @@ This is a pointer to a function with the following signature: ``args[0]`` may be changed. Whenever they can do so cheaply (without additional allocation), callers - are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. + are encouraged to use :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET`. Doing so will allow callables such as bound methods to make their onward calls (which include a prepended *self* argument) very efficiently. @@ -165,7 +165,7 @@ Vectorcall Support API This is a specialized function, intended to be put in the :c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``. - It does not check the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag + It does not check the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag and it does not fall back to ``tp_call``. .. versionadded:: 3.8 @@ -383,11 +383,11 @@ please see individual documentation for details. *args[0]*, and the *args* array starting at *args[1]* represents the arguments of the call. There must be at least one positional argument. *nargsf* is the number of positional arguments including *args[0]*, - plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may + plus :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may temporarily be changed. Keyword arguments can be passed just like in :c:func:`PyObject_Vectorcall`. - If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, + If the object has the :c:macro:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, this will call the unbound method object with the full *args* vector as arguments. diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst index cb8b270fcbab6..6679ce76f1dc6 100644 --- a/Doc/c-api/complex.rst +++ b/Doc/c-api/complex.rst @@ -64,7 +64,7 @@ pointers. This is consistent throughout the API. representation. If *divisor* is null, this method returns zero and sets - :c:data:`errno` to :c:data:`EDOM`. + :c:data:`errno` to :c:macro:`EDOM`. .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp) @@ -73,7 +73,7 @@ pointers. This is consistent throughout the API. representation. If *num* is null and *exp* is not a positive real number, - this method returns zero and sets :c:data:`errno` to :c:data:`EDOM`. + this method returns zero and sets :c:data:`errno` to :c:macro:`EDOM`. Complex Numbers as Python Objects diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 092e548fb8fd8..76d32fd1763f5 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -154,7 +154,7 @@ For convenience, some of these functions will always return a tuple object whose first item is the integer :c:data:`errno` value and whose second item is the corresponding error message (gotten from :c:func:`strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the - :c:data:`errno` value is :const:`EINTR`, indicating an interrupted system call, + :c:data:`errno` value is :c:macro:`EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, leaves it set to that. The function always returns ``NULL``, so a wrapper function around a system call can write ``return PyErr_SetFromErrno(type);`` @@ -567,7 +567,7 @@ Signal Handling be interruptible by user requests (such as by pressing Ctrl-C). .. note:: - The default Python signal handler for :const:`SIGINT` raises the + The default Python signal handler for :c:macro:`SIGINT` raises the :exc:`KeyboardInterrupt` exception. @@ -578,7 +578,7 @@ Signal Handling single: SIGINT single: KeyboardInterrupt (built-in exception) - Simulate the effect of a :const:`SIGINT` signal arriving. + Simulate the effect of a :c:macro:`SIGINT` signal arriving. This is equivalent to ``PyErr_SetInterruptEx(SIGINT)``. .. note:: @@ -690,7 +690,7 @@ Exception Objects .. c:function:: PyObject* PyException_GetCause(PyObject *ex) - Return the cause (either an exception instance, or :const:`None`, + Return the cause (either an exception instance, or ``None``, set by ``raise ... from ...``) associated with the exception as a new reference, as accessible from Python through :attr:`__cause__`. @@ -699,7 +699,7 @@ Exception Objects Set the cause associated with the exception to *cause*. Use ``NULL`` to clear it. There is no type check to make sure that *cause* is either an exception - instance or :const:`None`. This steals a reference to *cause*. + instance or ``None``. This steals a reference to *cause*. :attr:`__suppress_context__` is implicitly set to ``True`` by this function. @@ -788,7 +788,7 @@ because the :ref:`call protocol ` takes care of recursion handling. Marks a point where a recursive C-level call is about to be performed. - If :const:`USE_STACKCHECK` is defined, this function checks if the OS + If :c:macro:`USE_STACKCHECK` is defined, this function checks if the OS stack overflowed using :c:func:`PyOS_CheckStack`. In this is the case, it sets a :exc:`MemoryError` and returns a nonzero value. diff --git a/Doc/c-api/file.rst b/Doc/c-api/file.rst index f32ecba9f2702..b36c800e00444 100644 --- a/Doc/c-api/file.rst +++ b/Doc/c-api/file.rst @@ -93,7 +93,7 @@ the :mod:`io` APIs instead. .. index:: single: Py_PRINT_RAW Write object *obj* to file object *p*. The only supported flag for *flags* is - :const:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written + :c:macro:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written instead of the :func:`repr`. Return ``0`` on success or ``-1`` on failure; the appropriate exception will be set. diff --git a/Doc/c-api/float.rst b/Doc/c-api/float.rst index fd0be1108c630..4f6ac0d8175c6 100644 --- a/Doc/c-api/float.rst +++ b/Doc/c-api/float.rst @@ -109,7 +109,7 @@ Pack functions The pack routines write 2, 4 or 8 bytes, starting at *p*. *le* is an :c:expr:`int` argument, non-zero if you want the bytes string in little-endian format (exponent last, at ``p+1``, ``p+3``, or ``p+6`` ``p+7``), zero if you -want big-endian format (exponent first, at *p*). The :c:data:`PY_BIG_ENDIAN` +want big-endian format (exponent first, at *p*). The :c:macro:`PY_BIG_ENDIAN` constant can be used to use the native endian: it is equal to ``1`` on big endian processor, or ``0`` on little endian processor. @@ -140,7 +140,7 @@ Unpack functions The unpack routines read 2, 4 or 8 bytes, starting at *p*. *le* is an :c:expr:`int` argument, non-zero if the bytes string is in little-endian format (exponent last, at ``p+1``, ``p+3`` or ``p+6`` and ``p+7``), zero if big-endian -(exponent first, at *p*). The :c:data:`PY_BIG_ENDIAN` constant can be used to +(exponent first, at *p*). The :c:macro:`PY_BIG_ENDIAN` constant can be used to use the native endian: it is equal to ``1`` on big endian processor, or ``0`` on little endian processor. diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index 8c90d1e8991c1..fc690fd85c9f7 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -13,14 +13,12 @@ or strings), do not need to provide any explicit support for garbage collection. To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must -include the :const:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the +include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the :c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a :c:member:`~PyTypeObject.tp_clear` implementation must also be provided. -.. data:: Py_TPFLAGS_HAVE_GC - :noindex: - +:c:macro:`Py_TPFLAGS_HAVE_GC` Objects with a type with this flag set must conform with the rules documented here. For convenience these objects will be referred to as container objects. @@ -52,18 +50,18 @@ rules: :c:member:`~PyTypeObject.tp_flags`, :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields if the type inherits from a class that implements the garbage collector protocol and the child class - does *not* include the :const:`Py_TPFLAGS_HAVE_GC` flag. + does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. .. c:function:: TYPE* PyObject_GC_New(TYPE, PyTypeObject *type) Analogous to :c:func:`PyObject_New` but for container objects with the - :const:`Py_TPFLAGS_HAVE_GC` flag set. + :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. .. c:function:: TYPE* PyObject_GC_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) Analogous to :c:func:`PyObject_NewVar` but for container objects with the - :const:`Py_TPFLAGS_HAVE_GC` flag set. + :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. .. c:function:: TYPE* PyObject_GC_Resize(TYPE, PyVarObject *op, Py_ssize_t newsize) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index dda87f4e31a42..9a42fc5a25dc2 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1287,7 +1287,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. function does not steal any references to *exc*. To prevent naive misuse, you must write your own C extension to call this. Must be called with the GIL held. Returns the number of thread states modified; this is normally one, but will be - zero if the thread id isn't found. If *exc* is :const:`NULL`, the pending + zero if the thread id isn't found. If *exc* is ``NULL``, the pending exception (if any) for the thread is cleared. This raises no exceptions. .. versionchanged:: 3.7 @@ -1566,32 +1566,32 @@ Python-level trace functions in previous versions. The type of the trace function registered using :c:func:`PyEval_SetProfile` and :c:func:`PyEval_SetTrace`. The first parameter is the object passed to the registration function as *obj*, *frame* is the frame object to which the event - pertains, *what* is one of the constants :const:`PyTrace_CALL`, - :const:`PyTrace_EXCEPTION`, :const:`PyTrace_LINE`, :const:`PyTrace_RETURN`, - :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION`, :const:`PyTrace_C_RETURN`, - or :const:`PyTrace_OPCODE`, and *arg* depends on the value of *what*: - - +------------------------------+----------------------------------------+ - | Value of *what* | Meaning of *arg* | - +==============================+========================================+ - | :const:`PyTrace_CALL` | Always :c:data:`Py_None`. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_EXCEPTION` | Exception information as returned by | - | | :func:`sys.exc_info`. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_LINE` | Always :c:data:`Py_None`. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_RETURN` | Value being returned to the caller, | - | | or ``NULL`` if caused by an exception. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_C_CALL` | Function object being called. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_C_EXCEPTION` | Function object being called. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_C_RETURN` | Function object being called. | - +------------------------------+----------------------------------------+ - | :const:`PyTrace_OPCODE` | Always :c:data:`Py_None`. | - +------------------------------+----------------------------------------+ + pertains, *what* is one of the constants :c:data:`PyTrace_CALL`, + :c:data:`PyTrace_EXCEPTION`, :c:data:`PyTrace_LINE`, :c:data:`PyTrace_RETURN`, + :c:data:`PyTrace_C_CALL`, :c:data:`PyTrace_C_EXCEPTION`, :c:data:`PyTrace_C_RETURN`, + or :c:data:`PyTrace_OPCODE`, and *arg* depends on the value of *what*: + + +-------------------------------+----------------------------------------+ + | Value of *what* | Meaning of *arg* | + +===============================+========================================+ + | :c:data:`PyTrace_CALL` | Always :c:data:`Py_None`. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_EXCEPTION` | Exception information as returned by | + | | :func:`sys.exc_info`. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_LINE` | Always :c:data:`Py_None`. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_RETURN` | Value being returned to the caller, | + | | or ``NULL`` if caused by an exception. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_C_CALL` | Function object being called. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_C_EXCEPTION` | Function object being called. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_C_RETURN` | Function object being called. | + +-------------------------------+----------------------------------------+ + | :c:data:`PyTrace_OPCODE` | Always :c:data:`Py_None`. | + +-------------------------------+----------------------------------------+ .. c:var:: int PyTrace_CALL @@ -1658,8 +1658,8 @@ Python-level trace functions in previous versions. function as its first parameter, and may be any Python object, or ``NULL``. If the profile function needs to maintain state, using a different value for *obj* for each thread provides a convenient and thread-safe place to store it. The - profile function is called for all monitored events except :const:`PyTrace_LINE` - :const:`PyTrace_OPCODE` and :const:`PyTrace_EXCEPTION`. + profile function is called for all monitored events except :c:data:`PyTrace_LINE` + :c:data:`PyTrace_OPCODE` and :c:data:`PyTrace_EXCEPTION`. See also the :func:`sys.setprofile` function. @@ -1672,8 +1672,8 @@ Python-level trace functions in previous versions. :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number events and per-opcode events, but does not receive any event related to C function objects being called. Any trace function registered using :c:func:`PyEval_SetTrace` - will not receive :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or - :const:`PyTrace_C_RETURN` as a value for the *what* parameter. + will not receive :c:data:`PyTrace_C_CALL`, :c:data:`PyTrace_C_EXCEPTION` or + :c:data:`PyTrace_C_RETURN` as a value for the *what* parameter. See also the :func:`sys.settrace` function. diff --git a/Doc/c-api/long.rst b/Doc/c-api/long.rst index de191581561fa..874e96216097f 100644 --- a/Doc/c-api/long.rst +++ b/Doc/c-api/long.rst @@ -141,8 +141,8 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate. instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. - If the value of *obj* is greater than :const:`LONG_MAX` or less than - :const:`LONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and + If the value of *obj* is greater than :c:macro:`LONG_MAX` or less than + :c:macro:`LONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. @@ -182,8 +182,8 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate. instance of :c:type:`PyLongObject`, first call its :meth:`~object.__index__` method (if present) to convert it to a :c:type:`PyLongObject`. - If the value of *obj* is greater than :const:`LLONG_MAX` or less than - :const:`LLONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, + If the value of *obj* is greater than :c:macro:`LLONG_MAX` or less than + :c:macro:`LLONG_MIN`, set *\*overflow* to ``1`` or ``-1``, respectively, and return ``-1``; otherwise, set *\*overflow* to ``0``. If any other exception occurs set *\*overflow* to ``0`` and return ``-1`` as usual. diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 7041c15d23fb8..35c356f25c6c7 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -470,7 +470,7 @@ Customize Memory Allocators The new allocator must return a distinct non-``NULL`` pointer when requesting zero bytes. - For the :c:data:`PYMEM_DOMAIN_RAW` domain, the allocator must be + For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be thread-safe: the :term:`GIL ` is not held when the allocator is called. @@ -536,8 +536,8 @@ Runtime checks: - Detect write before the start of the buffer (buffer underflow). - Detect write after the end of the buffer (buffer overflow). - Check that the :term:`GIL ` is held when - allocator functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: - :c:func:`PyObject_Malloc`) and :c:data:`PYMEM_DOMAIN_MEM` (ex: + allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: + :c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. On error, the debug hooks use the :mod:`tracemalloc` module to get the @@ -557,9 +557,9 @@ that the treatment of negative indices differs from a Python slice): ``p[-S]`` API identifier (ASCII character): - * ``'r'`` for :c:data:`PYMEM_DOMAIN_RAW`. - * ``'m'`` for :c:data:`PYMEM_DOMAIN_MEM`. - * ``'o'`` for :c:data:`PYMEM_DOMAIN_OBJ`. + * ``'r'`` for :c:macro:`PYMEM_DOMAIN_RAW`. + * ``'m'`` for :c:macro:`PYMEM_DOMAIN_MEM`. + * ``'o'`` for :c:macro:`PYMEM_DOMAIN_OBJ`. ``p[-S+1:0]`` Copies of PYMEM_FORBIDDENBYTE. Used to catch under- writes and reads. @@ -601,7 +601,7 @@ PYMEM_CLEANBYTE (meaning uninitialized memory is getting used). compiled in release mode. On error, the debug hooks now use :mod:`tracemalloc` to get the traceback where a memory block was allocated. The debug hooks now also check if the GIL is held when functions of - :c:data:`PYMEM_DOMAIN_OBJ` and :c:data:`PYMEM_DOMAIN_MEM` domains are + :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are called. .. versionchanged:: 3.8 @@ -622,8 +622,8 @@ with a fixed size of 256 KiB. It falls back to :c:func:`PyMem_RawMalloc` and :c:func:`PyMem_RawRealloc` for allocations larger than 512 bytes. *pymalloc* is the :ref:`default allocator ` of the -:c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) and -:c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) domains. +:c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) and +:c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) domains. The arena allocator uses the following functions: diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index 230b471d473be..bc8e3b23b9957 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -145,7 +145,7 @@ or request "multi-phase initialization" by returning the definition struct itsel .. c:member:: PyModuleDef_Base m_base - Always initialize this member to :const:`PyModuleDef_HEAD_INIT`. + Always initialize this member to :c:data:`PyModuleDef_HEAD_INIT`. .. c:member:: const char *m_name @@ -256,7 +256,7 @@ of the following two module creation functions: Create a new module object, given the definition in *def*. This behaves like :c:func:`PyModule_Create2` with *module_api_version* set to - :const:`PYTHON_API_VERSION`. + :c:macro:`PYTHON_API_VERSION`. .. c:function:: PyObject* PyModule_Create2(PyModuleDef *def, int module_api_version) @@ -390,7 +390,7 @@ objects dynamically. Note that both ``PyModule_FromDefAndSpec`` and Create a new module object, given the definition in *def* and the ModuleSpec *spec*. This behaves like :c:func:`PyModule_FromDefAndSpec2` - with *module_api_version* set to :const:`PYTHON_API_VERSION`. + with *module_api_version* set to :c:macro:`PYTHON_API_VERSION`. .. versionadded:: 3.5 diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 7b8463e087374..191a252808bda 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -23,7 +23,7 @@ Object Protocol Print an object *o*, on file *fp*. Returns ``-1`` on error. The flags argument is used to enable certain printing options. The only option currently supported - is :const:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written + is :c:macro:`Py_PRINT_RAW`; if given, the :func:`str` of the object is written instead of the :func:`repr`. @@ -162,8 +162,8 @@ Object Protocol .. c:function:: PyObject* PyObject_RichCompare(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, - which must be one of :const:`Py_LT`, :const:`Py_LE`, :const:`Py_EQ`, - :const:`Py_NE`, :const:`Py_GT`, or :const:`Py_GE`, corresponding to ``<``, + which must be one of :c:macro:`Py_LT`, :c:macro:`Py_LE`, :c:macro:`Py_EQ`, + :c:macro:`Py_NE`, :c:macro:`Py_GT`, or :c:macro:`Py_GE`, corresponding to ``<``, ``<=``, ``==``, ``!=``, ``>``, or ``>=`` respectively. This is the equivalent of the Python expression ``o1 op o2``, where ``op`` is the operator corresponding to *opid*. Returns the value of the comparison on success, or ``NULL`` on failure. @@ -172,8 +172,8 @@ Object Protocol .. c:function:: int PyObject_RichCompareBool(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, - which must be one of :const:`Py_LT`, :const:`Py_LE`, :const:`Py_EQ`, - :const:`Py_NE`, :const:`Py_GT`, or :const:`Py_GE`, corresponding to ``<``, + which must be one of :c:macro:`Py_LT`, :c:macro:`Py_LE`, :c:macro:`Py_EQ`, + :c:macro:`Py_NE`, :c:macro:`Py_GT`, or :c:macro:`Py_GE`, corresponding to ``<``, ``<=``, ``==``, ``!=``, ``>``, or ``>=`` respectively. Returns ``-1`` on error, ``0`` if the result is false, ``1`` otherwise. This is the equivalent of the Python expression ``o1 op o2``, where ``op`` is the operator corresponding to @@ -181,7 +181,7 @@ Object Protocol .. note:: If *o1* and *o2* are the same object, :c:func:`PyObject_RichCompareBool` - will always return ``1`` for :const:`Py_EQ` and ``0`` for :const:`Py_NE`. + will always return ``1`` for :c:macro:`Py_EQ` and ``0`` for :c:macro:`Py_NE`. .. c:function:: PyObject* PyObject_Format(PyObject *obj, PyObject *format_spec) diff --git a/Doc/c-api/slice.rst b/Doc/c-api/slice.rst index 33169ccce8904..af4c540068cbf 100644 --- a/Doc/c-api/slice.rst +++ b/Doc/c-api/slice.rst @@ -34,7 +34,7 @@ Slice Objects *length* as errors. Returns ``0`` on success and ``-1`` on error with no exception set (unless one of - the indices was not :const:`None` and failed to be converted to an integer, + the indices was not ``None`` and failed to be converted to an integer, in which case ``-1`` is returned with an exception set). You probably do not want to use this function. diff --git a/Doc/c-api/stable.rst b/Doc/c-api/stable.rst index 4ae20e93e3678..a697f6cd3edd9 100644 --- a/Doc/c-api/stable.rst +++ b/Doc/c-api/stable.rst @@ -44,7 +44,7 @@ embedding Python.) Define this macro before including ``Python.h`` to opt in to only use the Limited API, and to select the Limited API version. - Define ``Py_LIMITED_API`` to the value of :c:data:`PY_VERSION_HEX` + Define ``Py_LIMITED_API`` to the value of :c:macro:`PY_VERSION_HEX` corresponding to the lowest Python version your extension supports. The extension will work without recompilation with all Python 3 releases from the specified one onward, and can use Limited API introduced up to that diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index cfd6d20a34899..e9f158c176924 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -198,7 +198,7 @@ Implementing functions and methods .. c:type:: PyCFunctionWithKeywords Type of the functions used to implement Python callables in C - with signature :const:`METH_VARARGS | METH_KEYWORDS`. + with signature :ref:`METH_VARARGS | METH_KEYWORDS `. The function signature is:: PyObject *PyCFunctionWithKeywords(PyObject *self, @@ -209,7 +209,7 @@ Implementing functions and methods .. c:type:: _PyCFunctionFast Type of the functions used to implement Python callables in C - with signature :const:`METH_FASTCALL`. + with signature :c:macro:`METH_FASTCALL`. The function signature is:: PyObject *_PyCFunctionFast(PyObject *self, @@ -219,7 +219,7 @@ Implementing functions and methods .. c:type:: _PyCFunctionFastWithKeywords Type of the functions used to implement Python callables in C - with signature :const:`METH_FASTCALL | METH_KEYWORDS`. + with signature :ref:`METH_FASTCALL | METH_KEYWORDS `. The function signature is:: PyObject *_PyCFunctionFastWithKeywords(PyObject *self, @@ -230,7 +230,7 @@ Implementing functions and methods .. c:type:: PyCMethod Type of the functions used to implement Python callables in C - with signature :const:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS`. + with signature :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. The function signature is:: PyObject *PyCMethod(PyObject *self, @@ -276,7 +276,7 @@ convention. There are these calling conventions: -.. data:: METH_VARARGS +.. c:macro:: METH_VARARGS This is the typical calling convention, where the methods have the type :c:type:`PyCFunction`. The function expects two :c:expr:`PyObject*` values. @@ -286,8 +286,17 @@ There are these calling conventions: using :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_UnpackTuple`. -.. data:: METH_VARARGS | METH_KEYWORDS +.. c:macro:: METH_KEYWORDS + Can only be used in certain combinations with other flags: + :ref:`METH_VARARGS | METH_KEYWORDS `, + :ref:`METH_FASTCALL | METH_KEYWORDS ` and + :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. + + +.. _METH_VARARGS-METH_KEYWORDS: + +:c:expr:`METH_VARARGS | METH_KEYWORDS` Methods with these flags must be of type :c:type:`PyCFunctionWithKeywords`. The function expects three parameters: *self*, *args*, *kwargs* where *kwargs* is a dictionary of all the keyword arguments or possibly ``NULL`` @@ -295,7 +304,7 @@ There are these calling conventions: using :c:func:`PyArg_ParseTupleAndKeywords`. -.. data:: METH_FASTCALL +.. c:macro:: METH_FASTCALL Fast calling convention supporting only positional arguments. The methods have the type :c:type:`_PyCFunctionFast`. @@ -310,9 +319,10 @@ There are these calling conventions: ``METH_FASTCALL`` is now part of the stable ABI. -.. data:: METH_FASTCALL | METH_KEYWORDS +.. _METH_FASTCALL-METH_KEYWORDS: - Extension of :const:`METH_FASTCALL` supporting also keyword arguments, +:c:expr:`METH_FASTCALL | METH_KEYWORDS` + Extension of :c:macro:`METH_FASTCALL` supporting also keyword arguments, with methods of type :c:type:`_PyCFunctionFastWithKeywords`. Keyword arguments are passed the same way as in the :ref:`vectorcall protocol `: @@ -325,10 +335,18 @@ There are these calling conventions: .. versionadded:: 3.7 -.. data:: METH_METHOD | METH_FASTCALL | METH_KEYWORDS +.. c:macro:: METH_METHOD + + Can only be used in the combination with other flags: + :ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS `. + + +.. _METH_METHOD-METH_FASTCALL-METH_KEYWORDS: - Extension of :const:`METH_FASTCALL | METH_KEYWORDS` supporting the *defining - class*, that is, the class that contains the method in question. +:c:expr:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS` + Extension of :ref:`METH_FASTCALL | METH_KEYWORDS ` + supporting the *defining class*, that is, + the class that contains the method in question. The defining class might be a superclass of ``Py_TYPE(self)``. The method needs to be of type :c:type:`PyCMethod`, the same as for @@ -338,10 +356,10 @@ There are these calling conventions: .. versionadded:: 3.9 -.. data:: METH_NOARGS +.. c:macro:: METH_NOARGS Methods without parameters don't need to check whether arguments are given if - they are listed with the :const:`METH_NOARGS` flag. They need to be of type + they are listed with the :c:macro:`METH_NOARGS` flag. They need to be of type :c:type:`PyCFunction`. The first parameter is typically named *self* and will hold a reference to the module or object instance. In all cases the second parameter will be ``NULL``. @@ -350,9 +368,9 @@ There are these calling conventions: :c:macro:`Py_UNUSED` can be used to prevent a compiler warning. -.. data:: METH_O +.. c:macro:: METH_O - Methods with a single object argument can be listed with the :const:`METH_O` + Methods with a single object argument can be listed with the :c:macro:`METH_O` flag, instead of invoking :c:func:`PyArg_ParseTuple` with a ``"O"`` argument. They have the type :c:type:`PyCFunction`, with the *self* parameter, and a :c:expr:`PyObject*` parameter representing the single argument. @@ -364,7 +382,7 @@ defined for modules. At most one of these flags may be set for any given method. -.. data:: METH_CLASS +.. c:macro:: METH_CLASS .. index:: pair: built-in function; classmethod @@ -374,7 +392,7 @@ method. function. -.. data:: METH_STATIC +.. c:macro:: METH_STATIC .. index:: pair: built-in function; staticmethod @@ -386,7 +404,7 @@ One other constant controls whether a method is loaded in place of another definition with the same method name. -.. data:: METH_COEXIST +.. c:macro:: METH_COEXIST The method will be loaded in place of existing definitions. Without *METH_COEXIST*, the default is to skip repeated definitions. Since slot diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 517b57b14768b..305a315549d15 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -95,9 +95,9 @@ Operating System Utilities .. c:function:: int PyOS_CheckStack() Return true when the interpreter runs out of stack space. This is a reliable - check, but is only available when :const:`USE_STACKCHECK` is defined (currently + check, but is only available when :c:macro:`USE_STACKCHECK` is defined (currently on certain versions of Windows using the Microsoft Visual C++ compiler). - :const:`USE_STACKCHECK` will be defined automatically; you should never + :c:macro:`USE_STACKCHECK` will be defined automatically; you should never change the definition in your own code. diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index 52eeef0a00863..a4ca724ad671a 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -66,7 +66,7 @@ Type Objects .. c:function:: int PyType_IS_GC(PyTypeObject *o) Return true if the type object includes support for the cycle detector; this - tests the type flag :const:`Py_TPFLAGS_HAVE_GC`. + tests the type flag :c:macro:`Py_TPFLAGS_HAVE_GC`. .. c:function:: int PyType_IsSubtype(PyTypeObject *a, PyTypeObject *b) @@ -99,10 +99,10 @@ Type Objects .. note:: If some of the base classes implements the GC protocol and the provided - type does not include the :const:`Py_TPFLAGS_HAVE_GC` in its flags, then + type does not include the :c:macro:`Py_TPFLAGS_HAVE_GC` in its flags, then the GC protocol will be automatically implemented from its parents. On the contrary, if the type being created does include - :const:`Py_TPFLAGS_HAVE_GC` in its flags then it **must** implement the + :c:macro:`Py_TPFLAGS_HAVE_GC` in its flags then it **must** implement the GC protocol itself by at least implementing the :c:member:`~PyTypeObject.tp_traverse` handle. @@ -193,7 +193,7 @@ The following functions and structs are used to create .. c:function:: PyObject* PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases) Creates and returns a :ref:`heap type ` from the *spec* - (:const:`Py_TPFLAGS_HEAPTYPE`). + (:c:macro:`Py_TPFLAGS_HEAPTYPE`). The *bases* argument can be used to specify base classes; it can either be only one class or a tuple of classes. diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index ce327166f3ab0..32989841c5b1a 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -667,7 +667,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) memory buffers owned by the instance (using the freeing function corresponding to the allocation function used to allocate the buffer), and call the type's :c:member:`~PyTypeObject.tp_free` function. If the type is not subtypable - (doesn't have the :const:`Py_TPFLAGS_BASETYPE` flag bit set), it is + (doesn't have the :c:macro:`Py_TPFLAGS_BASETYPE` flag bit set), it is permissible to call the object deallocator directly instead of via :c:member:`~PyTypeObject.tp_free`. The object deallocator should be the one used to allocate the instance; this is normally :c:func:`PyObject_Del` if the instance was allocated @@ -675,7 +675,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:func:`PyObject_GC_Del` if the instance was allocated using :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar`. - If the type supports garbage collection (has the :const:`Py_TPFLAGS_HAVE_GC` + If the type supports garbage collection (has the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit set), the destructor should call :c:func:`PyObject_GC_UnTrack` before clearing any member fields. @@ -687,7 +687,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) Py_TYPE(self)->tp_free((PyObject *)self); } - Finally, if the type is heap allocated (:const:`Py_TPFLAGS_HEAPTYPE`), the + Finally, if the type is heap allocated (:c:macro:`Py_TPFLAGS_HEAPTYPE`), the deallocator should decrement the reference count for its type object after calling the type deallocator. In order to avoid dangling pointers, the recommended way to achieve this is: @@ -714,12 +714,12 @@ and :c:type:`PyType_Type` effectively act as defaults.) a more efficient alternative of the simpler :c:member:`~PyTypeObject.tp_call`. - This field is only used if the flag :const:`Py_TPFLAGS_HAVE_VECTORCALL` + This field is only used if the flag :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` is set. If so, this must be a positive integer containing the offset in the instance of a :c:type:`vectorcallfunc` pointer. The *vectorcallfunc* pointer may be ``NULL``, in which case the instance behaves - as if :const:`Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance + as if :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance falls back to :c:member:`~PyTypeObject.tp_call`. Any class that sets ``Py_TPFLAGS_HAVE_VECTORCALL`` must also set @@ -743,12 +743,12 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** This field is always inherited. - However, the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is not + However, the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is not always inherited. If it's not, then the subclass won't use :ref:`vectorcall `, except when :c:func:`PyVectorcall_Call` is explicitly called. This is in particular the case for types without the - :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set (including subclasses defined in + :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag set (including subclasses defined in Python). @@ -1020,9 +1020,9 @@ and :c:type:`PyType_Type` effectively act as defaults.) this flag bit. The flag bits that pertain to extension structures are strictly inherited if the extension structure is inherited, i.e. the base type's value of the flag bit is copied into the subtype together with a pointer to the extension - structure. The :const:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with + structure. The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the - :const:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have ``NULL`` values. @@ -1035,12 +1035,14 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Bit Masks:** + .. c:namespace:: NULL + The following bit masks are currently defined; these can be ORed together using the ``|`` operator to form the value of the :c:member:`~PyTypeObject.tp_flags` field. The macro :c:func:`PyType_HasFeature` takes a type and a flags value, *tp* and *f*, and checks whether ``tp->tp_flags & f`` is non-zero. - .. data:: Py_TPFLAGS_HEAPTYPE + .. c:macro:: Py_TPFLAGS_HEAPTYPE This bit is set when the type object itself is allocated on the heap, for example, types created dynamically using :c:func:`PyType_FromSpec`. In this @@ -1055,7 +1057,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_BASETYPE + .. c:macro:: Py_TPFLAGS_BASETYPE This bit is set when the type can be used as the base type of another type. If this bit is clear, the type cannot be subtyped (similar to a "final" class in @@ -1066,7 +1068,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_READY + .. c:macro:: Py_TPFLAGS_READY This bit is set when the type object has been fully initialized by :c:func:`PyType_Ready`. @@ -1076,7 +1078,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_READYING + .. c:macro:: Py_TPFLAGS_READYING This bit is set while :c:func:`PyType_Ready` is in the process of initializing the type object. @@ -1086,7 +1088,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) ??? - .. data:: Py_TPFLAGS_HAVE_GC + .. c:macro:: Py_TPFLAGS_HAVE_GC This bit is set when the object supports garbage collection. If this bit is set, instances must be created using :c:func:`PyObject_GC_New` and @@ -1097,28 +1099,28 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :const:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` - The :const:`Py_TPFLAGS_HAVE_GC` flag bit is inherited + The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited together with the :attr:`tp_traverse` and :attr:`tp_clear` - fields, i.e. if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is + fields, i.e. if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the :attr:`tp_traverse` and :attr:`tp_clear` fields in the subtype exist and have ``NULL`` values. - .. data:: Py_TPFLAGS_DEFAULT + .. c:macro:: Py_TPFLAGS_DEFAULT This is a bitmask of all the bits that pertain to the existence of certain fields in the type object and its extension structures. Currently, it includes - the following bits: :const:`Py_TPFLAGS_HAVE_STACKLESS_EXTENSION`. + the following bits: :c:macro:`Py_TPFLAGS_HAVE_STACKLESS_EXTENSION`. **Inheritance:** ??? - .. data:: Py_TPFLAGS_METHOD_DESCRIPTOR + .. c:macro:: Py_TPFLAGS_METHOD_DESCRIPTOR This bit indicates that objects behave like unbound methods. @@ -1139,21 +1141,21 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** This flag is never inherited by types without the - :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is + :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is inherited whenever :c:member:`~PyTypeObject.tp_descr_get` is inherited. .. XXX Document more flags here? - .. data:: Py_TPFLAGS_LONG_SUBCLASS - .. data:: Py_TPFLAGS_LIST_SUBCLASS - .. data:: Py_TPFLAGS_TUPLE_SUBCLASS - .. data:: Py_TPFLAGS_BYTES_SUBCLASS - .. data:: Py_TPFLAGS_UNICODE_SUBCLASS - .. data:: Py_TPFLAGS_DICT_SUBCLASS - .. data:: Py_TPFLAGS_BASE_EXC_SUBCLASS - .. data:: Py_TPFLAGS_TYPE_SUBCLASS + .. c:macro:: Py_TPFLAGS_LONG_SUBCLASS + .. c:macro:: Py_TPFLAGS_LIST_SUBCLASS + .. c:macro:: Py_TPFLAGS_TUPLE_SUBCLASS + .. c:macro:: Py_TPFLAGS_BYTES_SUBCLASS + .. c:macro:: Py_TPFLAGS_UNICODE_SUBCLASS + .. c:macro:: Py_TPFLAGS_DICT_SUBCLASS + .. c:macro:: Py_TPFLAGS_BASE_EXC_SUBCLASS + .. c:macro:: Py_TPFLAGS_TYPE_SUBCLASS These flags are used by functions such as :c:func:`PyLong_Check` to quickly determine if a type is a subclass @@ -1164,7 +1166,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) will behave differently depending on what kind of check is used. - .. data:: Py_TPFLAGS_HAVE_FINALIZE + .. c:macro:: Py_TPFLAGS_HAVE_FINALIZE This bit is set when the :c:member:`~PyTypeObject.tp_finalize` slot is present in the type structure. @@ -1177,7 +1179,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) type structure. - .. data:: Py_TPFLAGS_HAVE_VECTORCALL + .. c:macro:: Py_TPFLAGS_HAVE_VECTORCALL This bit is set when the class implements the :ref:`vectorcall protocol `. @@ -1186,12 +1188,12 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** This bit is inherited for types with the - :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set, if + :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag set, if :c:member:`~PyTypeObject.tp_call` is also inherited. .. versionadded:: 3.9 - .. data:: Py_TPFLAGS_IMMUTABLETYPE + .. c:macro:: Py_TPFLAGS_IMMUTABLETYPE This bit is set for type objects that are immutable: type attributes cannot be set nor deleted. @@ -1204,7 +1206,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_DISALLOW_INSTANTIATION + .. c:macro:: Py_TPFLAGS_DISALLOW_INSTANTIATION Disallow creating instances of the type: set :c:member:`~PyTypeObject.tp_new` to NULL and don't create the ``__new__`` @@ -1235,7 +1237,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_MAPPING + .. c:macro:: Py_TPFLAGS_MAPPING This bit indicates that instances of the class may match mapping patterns when used as the subject of a :keyword:`match` block. It is automatically @@ -1244,20 +1246,20 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: - :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are + :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` are mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** This flag is inherited by types that do not already set - :const:`Py_TPFLAGS_SEQUENCE`. + :c:macro:`Py_TPFLAGS_SEQUENCE`. .. seealso:: :pep:`634` -- Structural Pattern Matching: Specification .. versionadded:: 3.10 - .. data:: Py_TPFLAGS_SEQUENCE + .. c:macro:: Py_TPFLAGS_SEQUENCE This bit indicates that instances of the class may match sequence patterns when used as the subject of a :keyword:`match` block. It is automatically @@ -1266,13 +1268,13 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: - :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are + :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` are mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** This flag is inherited by types that do not already set - :const:`Py_TPFLAGS_MAPPING`. + :c:macro:`Py_TPFLAGS_MAPPING`. .. seealso:: :pep:`634` -- Structural Pattern Matching: Specification @@ -1293,7 +1295,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: traverseproc PyTypeObject.tp_traverse An optional pointer to a traversal function for the garbage collector. This is - only used if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: + only used if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: int tp_traverse(PyObject *self, visitproc visit, void *arg); @@ -1355,10 +1357,10 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :const:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_clear` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. @@ -1366,7 +1368,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: inquiry PyTypeObject.tp_clear An optional pointer to a clear function for the garbage collector. This is only - used if the :const:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: + used if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is set. The signature is:: int tp_clear(PyObject *); @@ -1422,10 +1424,10 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :const:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_traverse` and the - :const:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and :c:member:`~PyTypeObject.tp_clear` are all inherited from the base type if they are all zero in the subtype. @@ -1447,21 +1449,21 @@ and :c:type:`PyType_Type` effectively act as defaults.) The following constants are defined to be used as the third argument for :c:member:`~PyTypeObject.tp_richcompare` and for :c:func:`PyObject_RichCompare`: - +----------------+------------+ - | Constant | Comparison | - +================+============+ - | :const:`Py_LT` | ``<`` | - +----------------+------------+ - | :const:`Py_LE` | ``<=`` | - +----------------+------------+ - | :const:`Py_EQ` | ``==`` | - +----------------+------------+ - | :const:`Py_NE` | ``!=`` | - +----------------+------------+ - | :const:`Py_GT` | ``>`` | - +----------------+------------+ - | :const:`Py_GE` | ``>=`` | - +----------------+------------+ + +------------------+------------+ + | Constant | Comparison | + +==================+============+ + | :c:macro:`Py_LT` | ``<`` | + +------------------+------------+ + | :c:macro:`Py_LE` | ``<=`` | + +------------------+------------+ + | :c:macro:`Py_EQ` | ``==`` | + +------------------+------------+ + | :c:macro:`Py_NE` | ``!=`` | + +------------------+------------+ + | :c:macro:`Py_GT` | ``>`` | + +------------------+------------+ + | :c:macro:`Py_GE` | ``>=`` | + +------------------+------------+ The following macro is defined to ease writing rich comparison functions: @@ -1839,7 +1841,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) in :c:member:`~PyTypeObject.tp_new`, while for mutable types, most initialization should be deferred to :c:member:`~PyTypeObject.tp_init`. - Set the :const:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag to disallow creating + Set the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag to disallow creating instances of the type in Python. **Inheritance:** @@ -1873,7 +1875,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) In dynamic subtypes, this field is set to a deallocator suitable to match :c:func:`PyType_GenericAlloc` and the value of the - :const:`Py_TPFLAGS_HAVE_GC` flag bit. + :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. For static subtypes, :c:type:`PyBaseObject_Type` uses PyObject_Del. @@ -1884,7 +1886,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) The garbage collector needs to know whether a particular object is collectible or not. Normally, it is sufficient to look at the object's type's - :c:member:`~PyTypeObject.tp_flags` field, and check the :const:`Py_TPFLAGS_HAVE_GC` flag bit. But + :c:member:`~PyTypeObject.tp_flags` field, and check the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. But some types have a mixture of statically and dynamically allocated instances, and the statically allocated instances are not collectible. Such types should define this function; it should return ``1`` for a collectible instance, and @@ -1903,7 +1905,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** This slot has no default. If this field is ``NULL``, - :const:`Py_TPFLAGS_HAVE_GC` is used as the functional equivalent. + :c:macro:`Py_TPFLAGS_HAVE_GC` is used as the functional equivalent. .. c:member:: PyObject* PyTypeObject.tp_bases @@ -2035,7 +2037,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionchanged:: 3.8 Before version 3.8 it was necessary to set the - :const:`Py_TPFLAGS_HAVE_FINALIZE` flags bit in order for this field to be + :c:macro:`Py_TPFLAGS_HAVE_FINALIZE` flags bit in order for this field to be used. This is no longer required. .. seealso:: "Safe object finalization" (:pep:`442`) @@ -2087,7 +2089,7 @@ Heap Types An alternative to :ref:`static types ` is *heap-allocated types*, or *heap types* for short, which correspond closely to classes created by -Python's ``class`` statement. Heap types have the :const:`Py_TPFLAGS_HEAPTYPE` +Python's ``class`` statement. Heap types have the :c:macro:`Py_TPFLAGS_HEAPTYPE` flag set. This is done by filling a :c:type:`PyType_Spec` structure and calling @@ -2697,7 +2699,7 @@ A type that supports weakrefs, instance dicts, and hashing:: A str subclass that cannot be subclassed and cannot be called to create instances (e.g. uses a separate factory func) using -:c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag:: +:c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag:: typedef struct { PyUnicodeObject raw; diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 12f95a2e8a39d..b2241ea8bd2e1 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1391,7 +1391,7 @@ the user settings on the machine running the codec. Encode the Unicode object using the specified code page and return a Python bytes object. Return ``NULL`` if an exception was raised by the codec. Use - :c:data:`CP_ACP` code page to get the MBCS encoder. + :c:macro:`CP_ACP` code page to get the MBCS encoder. .. versionadded:: 3.3 @@ -1510,11 +1510,11 @@ They all return ``NULL`` or ``-1`` if an exception occurs. Rich compare two Unicode strings and return one of the following: * ``NULL`` in case an exception was raised - * :const:`Py_True` or :const:`Py_False` for successful comparisons - * :const:`Py_NotImplemented` in case the type combination is unknown + * :c:data:`Py_True` or :c:data:`Py_False` for successful comparisons + * :c:data:`Py_NotImplemented` in case the type combination is unknown - Possible values for *op* are :const:`Py_GT`, :const:`Py_GE`, :const:`Py_EQ`, - :const:`Py_NE`, :const:`Py_LT`, and :const:`Py_LE`. + Possible values for *op* are :c:macro:`Py_GT`, :c:macro:`Py_GE`, :c:macro:`Py_EQ`, + :c:macro:`Py_NE`, :c:macro:`Py_LT`, and :c:macro:`Py_LE`. .. c:function:: PyObject* PyUnicode_Format(PyObject *format, PyObject *args) diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index bfb14ac912fca..cfbb44f4eb3a6 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -12,8 +12,8 @@ file or a buffer, but they will not let you interact in a more detailed way with the interpreter. Several of these functions accept a start symbol from the grammar as a -parameter. The available start symbols are :const:`Py_eval_input`, -:const:`Py_file_input`, and :const:`Py_single_input`. These are described +parameter. The available start symbols are :c:data:`Py_eval_input`, +:c:data:`Py_file_input`, and :c:data:`Py_single_input`. These are described following the functions which accept them as parameters. Note also that several of these functions take :c:expr:`FILE*` parameters. One @@ -248,8 +248,8 @@ the same library that the Python runtime is using. Parse and compile the Python source code in *str*, returning the resulting code object. The start token is given by *start*; this can be used to constrain the - code which can be compiled and should be :const:`Py_eval_input`, - :const:`Py_file_input`, or :const:`Py_single_input`. The filename specified by + code which can be compiled and should be :c:data:`Py_eval_input`, + :c:data:`Py_file_input`, or :c:data:`Py_single_input`. The filename specified by *filename* is used to construct the code object and may appear in tracebacks or :exc:`SyntaxError` exception messages. This returns ``NULL`` if the code cannot be parsed or compiled. diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index d9bf4fd6c7ae0..76e0490d0d22d 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -335,7 +335,7 @@ When using only ``METH_VARARGS``, the function should expect the Python-level parameters to be passed in as a tuple acceptable for parsing via :c:func:`PyArg_ParseTuple`; more information on this function is provided below. -The :const:`METH_KEYWORDS` bit may be set in the third field if keyword +The :c:macro:`METH_KEYWORDS` bit may be set in the third field if keyword arguments should be passed to the function. In this case, the C function should accept a third ``PyObject *`` parameter which will be a dictionary of keywords. Use :c:func:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a @@ -527,7 +527,7 @@ be part of a module definition:: } This function must be registered with the interpreter using the -:const:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The +:c:macro:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The :c:func:`PyArg_ParseTuple` function and its arguments are documented in section :ref:`parsetuple`. diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index 5d4a3f06dd540..b1bf0535048eb 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -151,7 +151,7 @@ only used for variable-sized objects and should otherwise be zero. base type will be :class:`object`, or else you will be adding data members to your base type, and therefore increasing its size. -We set the class flags to :const:`Py_TPFLAGS_DEFAULT`. :: +We set the class flags to :c:macro:`Py_TPFLAGS_DEFAULT`. :: .tp_flags = Py_TPFLAGS_DEFAULT, @@ -505,7 +505,7 @@ definitions:: {NULL} /* Sentinel */ }; -(note that we used the :const:`METH_NOARGS` flag to indicate that the method +(note that we used the :c:macro:`METH_NOARGS` flag to indicate that the method is expecting no arguments other than *self*) and assign it to the :c:member:`~PyTypeObject.tp_methods` slot:: @@ -515,7 +515,7 @@ and assign it to the :c:member:`~PyTypeObject.tp_methods` slot:: Finally, we'll make our type usable as a base class for subclassing. We've written our methods carefully so far so that they don't make any assumptions about the type of the object being created or used, so all we need to do is -to add the :const:`Py_TPFLAGS_BASETYPE` to our class flag definition:: +to add the :c:macro:`Py_TPFLAGS_BASETYPE` to our class flag definition:: .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, @@ -781,7 +781,7 @@ and ``Custom_clear``:: Py_TYPE(self)->tp_free((PyObject *) self); } -Finally, we add the :const:`Py_TPFLAGS_HAVE_GC` flag to the class flags:: +Finally, we add the :c:macro:`Py_TPFLAGS_HAVE_GC` flag to the class flags:: .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index 2eddb582da7c2..522e0853b496a 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -298,10 +298,10 @@ Watch out for the following two points in particular (but note that this is not a comprehensive list): * Unlike static types, heap type objects are mutable by default. - Use the :c:data:`Py_TPFLAGS_IMMUTABLETYPE` flag to prevent mutability. + Use the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag to prevent mutability. * Heap types inherit :c:member:`~PyTypeObject.tp_new` by default, so it may become possible to instantiate them from Python code. - You can prevent this with the :c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. + You can prevent this with the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag. Defining Heap Types @@ -333,12 +333,12 @@ To avoid memory leaks, instances of heap types must implement the garbage collection protocol. That is, heap types should: -- Have the :c:data:`Py_TPFLAGS_HAVE_GC` flag. +- Have the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. - Define a traverse function using ``Py_tp_traverse``, which visits the type (e.g. using :c:expr:`Py_VISIT(Py_TYPE(self))`). Please refer to the :ref:`the documentation ` of -:c:data:`Py_TPFLAGS_HAVE_GC` and :c:member:`~PyTypeObject.tp_traverse` +:c:macro:`Py_TPFLAGS_HAVE_GC` and :c:member:`~PyTypeObject.tp_traverse` for additional considerations. If your traverse function delegates to the ``tp_traverse`` of its base class @@ -411,7 +411,7 @@ that subclass, which may be defined in different module than yours. pass For a method to get its "defining class", it must use the -:data:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS` +:ref:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS ` :c:type:`calling convention ` and the corresponding :c:type:`PyCMethod` signature:: diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 7467f5620d425..f56784e769a3e 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -747,7 +747,7 @@ iterations of the loop. .. opcode:: MATCH_MAPPING If TOS is an instance of :class:`collections.abc.Mapping` (or, more technically: if - it has the :const:`Py_TPFLAGS_MAPPING` flag set in its + it has the :c:macro:`Py_TPFLAGS_MAPPING` flag set in its :c:member:`~PyTypeObject.tp_flags`), push ``True`` onto the stack. Otherwise, push ``False``. @@ -758,7 +758,7 @@ iterations of the loop. If TOS is an instance of :class:`collections.abc.Sequence` and is *not* an instance of :class:`str`/:class:`bytes`/:class:`bytearray` (or, more technically: if it has - the :const:`Py_TPFLAGS_SEQUENCE` flag set in its :c:member:`~PyTypeObject.tp_flags`), + the :c:macro:`Py_TPFLAGS_SEQUENCE` flag set in its :c:member:`~PyTypeObject.tp_flags`), push ``True`` onto the stack. Otherwise, push ``False``. .. versionadded:: 3.10 diff --git a/Doc/library/os.rst b/Doc/library/os.rst index b4812e25256b6..dcd4d9e4ae2d0 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -4068,7 +4068,7 @@ written in Python, such as a mail server's external command delivery program. specified. If the value specified is 0, the child's process group ID will be made the same as its process ID. If the value of *setpgroup* is not set, the child will inherit the parent's process group ID. This argument corresponds - to the C library :c:data:`POSIX_SPAWN_SETPGROUP` flag. + to the C library :c:macro:`POSIX_SPAWN_SETPGROUP` flag. If the *resetids* argument is ``True`` it will reset the effective UID and GID of the child to the real UID and GID of the parent process. If the @@ -4076,27 +4076,27 @@ written in Python, such as a mail server's external command delivery program. the parent. In either case, if the set-user-ID and set-group-ID permission bits are enabled on the executable file, their effect will override the setting of the effective UID and GID. This argument corresponds to the C - library :c:data:`POSIX_SPAWN_RESETIDS` flag. + library :c:macro:`POSIX_SPAWN_RESETIDS` flag. If the *setsid* argument is ``True``, it will create a new session ID - for ``posix_spawn``. *setsid* requires :c:data:`POSIX_SPAWN_SETSID` - or :c:data:`POSIX_SPAWN_SETSID_NP` flag. Otherwise, :exc:`NotImplementedError` + for ``posix_spawn``. *setsid* requires :c:macro:`POSIX_SPAWN_SETSID` + or :c:macro:`POSIX_SPAWN_SETSID_NP` flag. Otherwise, :exc:`NotImplementedError` is raised. The *setsigmask* argument will set the signal mask to the signal set specified. If the parameter is not used, then the child inherits the parent's signal mask. This argument corresponds to the C library - :c:data:`POSIX_SPAWN_SETSIGMASK` flag. + :c:macro:`POSIX_SPAWN_SETSIGMASK` flag. The *sigdef* argument will reset the disposition of all signals in the set specified. This argument corresponds to the C library - :c:data:`POSIX_SPAWN_SETSIGDEF` flag. + :c:macro:`POSIX_SPAWN_SETSIGDEF` flag. The *scheduler* argument must be a tuple containing the (optional) scheduler policy and an instance of :class:`sched_param` with the scheduler parameters. A value of ``None`` in the place of the scheduler policy indicates that is not being provided. This argument is a combination of the C library - :c:data:`POSIX_SPAWN_SETSCHEDPARAM` and :c:data:`POSIX_SPAWN_SETSCHEDULER` + :c:macro:`POSIX_SPAWN_SETSCHEDPARAM` and :c:macro:`POSIX_SPAWN_SETSCHEDULER` flags. .. audit-event:: os.posix_spawn path,argv,env os.posix_spawn diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index d6492ac18f925..fb0daf89357f9 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -1602,7 +1602,7 @@ body of a coroutine function. * a class that inherits from :class:`collections.abc.Sequence` * a Python class that has been registered as :class:`collections.abc.Sequence` - * a builtin class that has its (CPython) :data:`Py_TPFLAGS_SEQUENCE` bit set + * a builtin class that has its (CPython) :c:macro:`Py_TPFLAGS_SEQUENCE` bit set * a class that inherits from any of the above The following standard library classes are sequences: @@ -1621,7 +1621,7 @@ body of a coroutine function. * a class that inherits from :class:`collections.abc.Mapping` * a Python class that has been registered as :class:`collections.abc.Mapping` - * a builtin class that has its (CPython) :data:`Py_TPFLAGS_MAPPING` bit set + * a builtin class that has its (CPython) :c:macro:`Py_TPFLAGS_MAPPING` bit set * a class that inherits from any of the above The standard library classes :class:`dict` and :class:`types.MappingProxyType` diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index ecc877f5020d5..c42ef78c6b09d 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -883,11 +883,11 @@ conflict. * ``default``: use the :ref:`default memory allocators `. * ``malloc``: use the :c:func:`malloc` function of the C library - for all domains (:c:data:`PYMEM_DOMAIN_RAW`, :c:data:`PYMEM_DOMAIN_MEM`, - :c:data:`PYMEM_DOMAIN_OBJ`). + for all domains (:c:macro:`PYMEM_DOMAIN_RAW`, :c:macro:`PYMEM_DOMAIN_MEM`, + :c:macro:`PYMEM_DOMAIN_OBJ`). * ``pymalloc``: use the :ref:`pymalloc allocator ` for - :c:data:`PYMEM_DOMAIN_MEM` and :c:data:`PYMEM_DOMAIN_OBJ` domains and use - the :c:func:`malloc` function for the :c:data:`PYMEM_DOMAIN_RAW` domain. + :c:macro:`PYMEM_DOMAIN_MEM` and :c:macro:`PYMEM_DOMAIN_OBJ` domains and use + the :c:func:`malloc` function for the :c:macro:`PYMEM_DOMAIN_RAW` domain. Install :ref:`debug hooks `: diff --git a/Doc/whatsnew/2.2.rst b/Doc/whatsnew/2.2.rst index 82aff0be1ed3b..44e9bd8d492bf 100644 --- a/Doc/whatsnew/2.2.rst +++ b/Doc/whatsnew/2.2.rst @@ -1105,11 +1105,11 @@ code, none of the changes described here will affect you very much. expected, and a set of pointers to :c:expr:`PyObject*` variables that will be filled in with argument values. -* Two new flags :const:`METH_NOARGS` and :const:`METH_O` are available in method +* Two new flags :c:macro:`METH_NOARGS` and :c:macro:`METH_O` are available in method definition tables to simplify implementation of methods with no arguments or a single untyped argument. Calling such methods is more efficient than calling a - corresponding method that uses :const:`METH_VARARGS`. Also, the old - :const:`METH_OLDARGS` style of writing C methods is now officially deprecated. + corresponding method that uses :c:macro:`METH_VARARGS`. Also, the old + :c:macro:`METH_OLDARGS` style of writing C methods is now officially deprecated. * Two new wrapper functions, :c:func:`PyOS_snprintf` and :c:func:`PyOS_vsnprintf` were added to provide cross-platform implementations for the relatively new diff --git a/Doc/whatsnew/2.3.rst b/Doc/whatsnew/2.3.rst index af489d7cb45c2..8d5603073005b 100644 --- a/Doc/whatsnew/2.3.rst +++ b/Doc/whatsnew/2.3.rst @@ -1474,7 +1474,7 @@ complete list of changes, or look through the CVS logs for all the details. * On Windows, the :mod:`socket` module now ships with Secure Sockets Layer (SSL) support. -* The value of the C :const:`PYTHON_API_VERSION` macro is now exposed at the +* The value of the C :c:macro:`PYTHON_API_VERSION` macro is now exposed at the Python level as ``sys.api_version``. The current exception can be cleared by calling the new :func:`sys.exc_clear` function. @@ -1899,10 +1899,10 @@ Changes to Python's build process and to the C API include: * The :c:func:`PyArg_NoArgs` macro is now deprecated, and code that uses it should be changed. For Python 2.2 and later, the method definition table can - specify the :const:`METH_NOARGS` flag, signalling that there are no arguments, + specify the :c:macro:`METH_NOARGS` flag, signalling that there are no arguments, and the argument checking can then be removed. If compatibility with pre-2.2 versions of Python is important, the code could use ``PyArg_ParseTuple(args, - "")`` instead, but this will be slower than using :const:`METH_NOARGS`. + "")`` instead, but this will be slower than using :c:macro:`METH_NOARGS`. * :c:func:`PyArg_ParseTuple` accepts new format characters for various sizes of unsigned integers: ``B`` for :c:expr:`unsigned char`, ``H`` for :c:expr:`unsigned @@ -1918,7 +1918,7 @@ Changes to Python's build process and to the C API include: seconds, according to one measurement). * It's now possible to define class and static methods for a C extension type by - setting either the :const:`METH_CLASS` or :const:`METH_STATIC` flags in a + setting either the :c:macro:`METH_CLASS` or :c:macro:`METH_STATIC` flags in a method's :c:type:`PyMethodDef` structure. * Python now includes a copy of the Expat XML parser's source code, removing any diff --git a/Doc/whatsnew/2.4.rst b/Doc/whatsnew/2.4.rst index 98dc83fe935d5..d68f600ba44f6 100644 --- a/Doc/whatsnew/2.4.rst +++ b/Doc/whatsnew/2.4.rst @@ -1476,7 +1476,7 @@ Some of the changes to Python's build process and to the C API are: :c:func:`PyArg_ParseTupleAndKeywords` but takes a :c:type:`va_list` instead of a number of arguments. (Contributed by Greg Chapman.) -* A new method flag, :const:`METH_COEXISTS`, allows a function defined in slots +* A new method flag, :c:macro:`METH_COEXISTS`, allows a function defined in slots to co-exist with a :c:type:`PyCFunction` having the same name. This can halve the access time for a method such as :meth:`set.__contains__`. (Contributed by Raymond Hettinger.) diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index 84bb651e68eed..72a273fdd8d12 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -1138,11 +1138,11 @@ indicate that the external caller is done. The *flags* argument to :c:func:`PyObject_GetBuffer` specifies constraints upon the memory returned. Some examples are: - * :const:`PyBUF_WRITABLE` indicates that the memory must be writable. + * :c:macro:`PyBUF_WRITABLE` indicates that the memory must be writable. - * :const:`PyBUF_LOCK` requests a read-only or exclusive lock on the memory. + * :c:macro:`PyBUF_LOCK` requests a read-only or exclusive lock on the memory. - * :const:`PyBUF_C_CONTIGUOUS` and :const:`PyBUF_F_CONTIGUOUS` + * :c:macro:`PyBUF_C_CONTIGUOUS` and :c:macro:`PyBUF_F_CONTIGUOUS` requests a C-contiguous (last dimension varies the fastest) or Fortran-contiguous (first dimension varies the fastest) array layout. diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index 1acb09cf92571..3cbab8d567bfa 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -2230,7 +2230,7 @@ Changes to Python's build process and to the C API include: * When using the :c:type:`PyMemberDef` structure to define attributes of a type, Python will no longer let you try to delete or set a - :const:`T_STRING_INPLACE` attribute. + :c:macro:`T_STRING_INPLACE` attribute. .. rev 79644 diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index ed72af717f666..b0b570be8947b 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -2123,11 +2123,11 @@ New Features These functions allow to activate, deactivate and query the state of the garbage collector from C code without having to import the :mod:`gc` module. -* Add a new :c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow +* Add a new :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow creating type instances. (Contributed by Victor Stinner in :issue:`43916`.) -* Add a new :c:data:`Py_TPFLAGS_IMMUTABLETYPE` type flag for creating immutable +* Add a new :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` type flag for creating immutable type objects: type attributes cannot be set nor deleted. (Contributed by Victor Stinner and Erlend E. Aasland in :issue:`43908`.) @@ -2186,9 +2186,9 @@ Porting to Python 3.10 been included directly, consider including ``Python.h`` instead. (Contributed by Nicholas Sim in :issue:`35134`.) -* Use the :c:data:`Py_TPFLAGS_IMMUTABLETYPE` type flag to create immutable type - objects. Do not rely on :c:data:`Py_TPFLAGS_HEAPTYPE` to decide if a type - object is mutable or not; check if :c:data:`Py_TPFLAGS_IMMUTABLETYPE` is set +* Use the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` type flag to create immutable type + objects. Do not rely on :c:macro:`Py_TPFLAGS_HEAPTYPE` to decide if a type + object is mutable or not; check if :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` is set instead. (Contributed by Victor Stinner and Erlend E. Aasland in :issue:`43908`.) diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 4e944484da86e..b912b94ebe5a4 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -2349,11 +2349,11 @@ Porting to Python 3.11 #endif * The :c:func:`PyType_Ready` function now raises an error if a type is defined - with the :const:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function + with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function (:c:member:`PyTypeObject.tp_traverse`). (Contributed by Victor Stinner in :issue:`44263`.) -* Heap types with the :const:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit +* Heap types with the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit the :pep:`590` vectorcall protocol. Previously, this was only possible for :ref:`static types `. (Contributed by Erlend E. Aasland in :issue:`43908`) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index e303e6cba966f..79cceec543dcc 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -650,8 +650,8 @@ compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks: * Detect writes before the start of a buffer (buffer underflows) * Detect writes after the end of a buffer (buffer overflows) * Check that the :term:`GIL ` is held when allocator - functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and - :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. + functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and + :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. Checking if the GIL is held is also a new feature of Python 3.6. @@ -1822,7 +1822,7 @@ Optimizations up to 80% faster. (Contributed by Josh Snider in :issue:`26574`). * Allocator functions of the :c:func:`PyMem_Malloc` domain - (:c:data:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator + (:c:macro:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator ` instead of :c:func:`malloc` function of the C library. The pymalloc allocator is optimized for objects smaller or equal to 512 bytes with a short lifetime, and use :c:func:`malloc` for larger memory blocks. @@ -1874,8 +1874,8 @@ Build and C API Changes (Original patch by Alecsandru Patrascu of Intel in :issue:`26359`.) * The :term:`GIL ` must now be held when allocator - functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and - :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. + functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and + :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data failed. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index e99a9f7634c54..54b2a4f8ed84d 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -2113,7 +2113,7 @@ Changes in the C API extension types across feature releases, anymore. A :c:type:`PyTypeObject` exported by a third-party extension module is supposed to have all the slots expected in the current Python version, including - :c:member:`~PyTypeObject.tp_finalize` (:const:`Py_TPFLAGS_HAVE_FINALIZE` + :c:member:`~PyTypeObject.tp_finalize` (:c:macro:`Py_TPFLAGS_HAVE_FINALIZE` is not checked anymore before reading :c:member:`~PyTypeObject.tp_finalize`). (Contributed by Antoine Pitrou in :issue:`32388`.) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index fd86db9630235..49c8bd2f3e07d 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1276,7 +1276,7 @@ New Features * :pep:`573`: Added :c:func:`PyType_FromModuleAndSpec` to associate a module with a class; :c:func:`PyType_GetModule` and :c:func:`PyType_GetModuleState` to retrieve the module and its state; and - :c:data:`PyCMethod` and :c:data:`METH_METHOD` to allow a method to + :c:data:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to access the class it was defined in. (Contributed by Marcel Plch and Petr Viktorin in :issue:`38787`.) diff --git a/Misc/NEWS.d/3.10.0a3.rst b/Misc/NEWS.d/3.10.0a3.rst index 699a0dd9e8d7c..70b79f5f250a1 100644 --- a/Misc/NEWS.d/3.10.0a3.rst +++ b/Misc/NEWS.d/3.10.0a3.rst @@ -1466,7 +1466,7 @@ success. Patch by Victor Stinner. .. nonce: S3FWTP .. section: C API -The :c:data:`METH_FASTCALL` calling convention is added to the limited API. +The :c:macro:`METH_FASTCALL` calling convention is added to the limited API. The functions :c:func:`PyModule_AddType`, :c:func:`PyType_FromModuleAndSpec`, :c:func:`PyType_GetModule` and :c:func:`PyType_GetModuleState` are added to the limited API on Windows. diff --git a/Misc/NEWS.d/3.10.0b1.rst b/Misc/NEWS.d/3.10.0b1.rst index 2a3d358edde90..cf89a550d3854 100644 --- a/Misc/NEWS.d/3.10.0b1.rst +++ b/Misc/NEWS.d/3.10.0b1.rst @@ -1375,8 +1375,8 @@ Add "Annotations Best Practices" document as a new HOWTO. .. nonce: K5aSl1 .. section: Documentation -Document the new :const:`Py_TPFLAGS_MAPPING` and -:const:`Py_TPFLAGS_SEQUENCE` type flags. +Document the new :c:macro:`Py_TPFLAGS_MAPPING` and +:c:macro:`Py_TPFLAGS_SEQUENCE` type flags. .. @@ -1711,7 +1711,7 @@ IDLE's shell now shows prompts in a separate side-bar. .. nonce: wvWt23 .. section: C API -Add a new :c:data:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow +Add a new :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` type flag to disallow creating type instances. Patch by Victor Stinner. .. @@ -1759,7 +1759,7 @@ module. .. nonce: Co3YhZ .. section: C API -Introduce :const:`Py_TPFLAGS_IMMUTABLETYPE` flag for immutable type objects, +Introduce :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag for immutable type objects, and modify :c:func:`PyType_Ready` to set it for static types. Patch by Erlend E. Aasland. diff --git a/Misc/NEWS.d/3.11.0a1.rst b/Misc/NEWS.d/3.11.0a1.rst index 33841d9e4e39b..e5827e2f23b1f 100644 --- a/Misc/NEWS.d/3.11.0a1.rst +++ b/Misc/NEWS.d/3.11.0a1.rst @@ -888,7 +888,7 @@ zlib.decompress on input data that expands that large. .. nonce: YHuV_s .. section: Core and Builtins -Heap types with the :const:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit +Heap types with the :c:macro:`Py_TPFLAGS_IMMUTABLETYPE` flag can now inherit the :pep:`590` vectorcall protocol. Previously, this was only possible for :ref:`static types `. Patch by Erlend E. Aasland. @@ -2575,7 +2575,7 @@ E. Aasland. .. nonce: bamAGF .. section: Library -Set the proper :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` +Set the proper :c:macro:`Py_TPFLAGS_MAPPING` and :c:macro:`Py_TPFLAGS_SEQUENCE` flags for subclasses created before a parent has been registered as a :class:`collections.abc.Mapping` or :class:`collections.abc.Sequence`. @@ -2693,7 +2693,7 @@ libgcc_s.so file (ex: EMFILE error). Patch by Victor Stinner. .. section: Library The _thread.RLock type now fully implement the GC protocol: add a traverse -function and the :const:`Py_TPFLAGS_HAVE_GC` flag. Patch by Victor Stinner. +function and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. Patch by Victor Stinner. .. @@ -5014,7 +5014,7 @@ must now be used to set an object type and size. Patch by Victor Stinner. .. section: C API The :c:func:`PyType_Ready` function now raises an error if a type is defined -with the :const:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function +with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set but has no traverse function (:c:member:`PyTypeObject.tp_traverse`). Patch by Victor Stinner. .. diff --git a/Misc/NEWS.d/3.11.0a7.rst b/Misc/NEWS.d/3.11.0a7.rst index 8e7ccd4d6771e..2172c02e2fc84 100644 --- a/Misc/NEWS.d/3.11.0a7.rst +++ b/Misc/NEWS.d/3.11.0a7.rst @@ -275,7 +275,7 @@ initializing to ``list_extend``. Patch by Jeremiah Pascual. .. nonce: cnaIK3 .. section: Core and Builtins -Speed up throwing exception in generator with :const:`METH_FASTCALL` calling +Speed up throwing exception in generator with :c:macro:`METH_FASTCALL` calling convention. Patch by Kumar Aditya. .. diff --git a/Misc/NEWS.d/3.6.0a1.rst b/Misc/NEWS.d/3.6.0a1.rst index 53f09b3dfe336..98f1215fb9187 100644 --- a/Misc/NEWS.d/3.6.0a1.rst +++ b/Misc/NEWS.d/3.6.0a1.rst @@ -125,7 +125,7 @@ Setuptools 19.0. .. section: Core and Builtins Memory functions of the :c:func:`PyMem_Malloc` domain -(:c:data:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc allocator +(:c:macro:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc allocator ` rather than system :c:func:`malloc`. Applications calling :c:func:`PyMem_Malloc` without holding the GIL can now crash: use ``PYTHONMALLOC=debug`` environment variable to validate the usage of memory diff --git a/Misc/NEWS.d/3.9.0a1.rst b/Misc/NEWS.d/3.9.0a1.rst index 2525696c58a4d..5a929d481a3d5 100644 --- a/Misc/NEWS.d/3.9.0a1.rst +++ b/Misc/NEWS.d/3.9.0a1.rst @@ -5706,7 +5706,7 @@ and :c:func:`_PyObject_CallMethodOneArg`. .. nonce: qZC0N_ .. section: C API -The :const:`METH_FASTCALL` calling convention has been documented. +The :c:macro:`METH_FASTCALL` calling convention has been documented. .. From webhook-mailer at python.org Fri Jul 21 07:48:54 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 11:48:54 -0000 Subject: [Python-checkins] [3.12] gh-47146: Fix reference counting in _testcapi.structmember initializer (GH-106862) (GH-106953) Message-ID: https://github.com/python/cpython/commit/4be0f157ea7000ded8d4a3ae818a74b026f1fed3 commit: 4be0f157ea7000ded8d4a3ae818a74b026f1fed3 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-21T14:48:50+03:00 summary: [3.12] gh-47146: Fix reference counting in _testcapi.structmember initializer (GH-106862) (GH-106953) (cherry picked from commit 8d397ee8259fa0f81598a452438fc335267ca260) Co-authored-by: Serhiy Storchaka files: M Modules/_testcapi/structmember.c diff --git a/Modules/_testcapi/structmember.c b/Modules/_testcapi/structmember.c index 0fb872a4328d6..641b003e56d32 100644 --- a/Modules/_testcapi/structmember.c +++ b/Modules/_testcapi/structmember.c @@ -194,7 +194,7 @@ _PyTestCapi_Init_Structmember(PyObject *m) if (res < 0) { return -1; } - res = PyModule_AddObject( + res = PyModule_AddObjectRef( m, "_test_structmembersType_OldAPI", (PyObject *)&test_structmembersType_OldAPI); From webhook-mailer at python.org Fri Jul 21 07:49:25 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 11:49:25 -0000 Subject: [Python-checkins] [3.11] gh-106892: Use roles :data: and :const: for referencing module variables (GH-106894) (GH-106955) Message-ID: https://github.com/python/cpython/commit/cc76113cf823ebff76d346665bbf1218a40ed4c8 commit: cc76113cf823ebff76d346665bbf1218a40ed4c8 branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T14:49:21+03:00 summary: [3.11] gh-106892: Use roles :data: and :const: for referencing module variables (GH-106894) (GH-106955) (cherry picked from commit d036db728ea3d54509cbad06df74e2d9a31fbec8) files: M Doc/c-api/import.rst M Doc/library/__main__.rst M Doc/library/asyncio-subprocess.rst M Doc/library/compileall.rst M Doc/library/devmode.rst M Doc/library/filecmp.rst M Doc/library/ftplib.rst M Doc/library/functions.rst M Doc/library/gc.rst M Doc/library/gzip.rst M Doc/library/importlib.resources.abc.rst M Doc/library/importlib.rst M Doc/library/json.rst M Doc/library/logging.handlers.rst M Doc/library/os.path.rst M Doc/library/os.rst M Doc/library/platform.rst M Doc/library/shutil.rst M Doc/library/stdtypes.rst M Doc/library/subprocess.rst M Doc/library/sys.rst M Doc/library/tarfile.rst M Doc/library/test.rst M Doc/library/tkinter.rst M Doc/library/unittest.rst M Doc/reference/datamodel.rst M Doc/using/windows.rst M Doc/whatsnew/2.5.rst M Doc/whatsnew/3.1.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.2.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.4.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.8.0a4.rst M Misc/NEWS.d/3.9.0a5.rst diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 57328fc16198c..4ca1517efcb38 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -126,9 +126,9 @@ Importing Modules read from a Python bytecode file or obtained from the built-in function :func:`compile`, load the module. Return a new reference to the module object, or ``NULL`` with an exception set if an error occurred. *name* - is removed from :attr:`sys.modules` in error cases, even if *name* was already - in :attr:`sys.modules` on entry to :c:func:`PyImport_ExecCodeModule`. Leaving - incompletely initialized modules in :attr:`sys.modules` is dangerous, as imports of + is removed from :data:`sys.modules` in error cases, even if *name* was already + in :data:`sys.modules` on entry to :c:func:`PyImport_ExecCodeModule`. Leaving + incompletely initialized modules in :data:`sys.modules` is dangerous, as imports of such modules have no way to know that the module object is an unknown (and probably damaged with respect to the module author's intents) state. diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 363596782d8e3..90973d0058e88 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -336,12 +336,12 @@ Note that importing ``__main__`` doesn't cause any issues with unintentionally running top-level code meant for script use which is put in the ``if __name__ == "__main__"`` block of the ``start`` module. Why does this work? -Python inserts an empty ``__main__`` module in :attr:`sys.modules` at +Python inserts an empty ``__main__`` module in :data:`sys.modules` at interpreter startup, and populates it by running top-level code. In our example this is the ``start`` module which runs line by line and imports ``namely``. In turn, ``namely`` imports ``__main__`` (which is really ``start``). That's an import cycle! Fortunately, since the partially populated ``__main__`` -module is present in :attr:`sys.modules`, Python passes that to ``namely``. +module is present in :data:`sys.modules`, Python passes that to ``namely``. See :ref:`Special considerations for __main__ ` in the import system's reference for details on how this works. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 4274638c5e862..6e04817b1818d 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -68,7 +68,7 @@ Creating Subprocesses The *limit* argument sets the buffer limit for :class:`StreamReader` wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` - (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). + (if :const:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). Return a :class:`~asyncio.subprocess.Process` instance. @@ -86,7 +86,7 @@ Creating Subprocesses The *limit* argument sets the buffer limit for :class:`StreamReader` wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` - (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). + (if :const:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). Return a :class:`~asyncio.subprocess.Process` instance. diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst index 180f5b81c2b61..4226348a17240 100644 --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -141,9 +141,9 @@ There is no command-line option to control the optimization level used by the :func:`compile` function, because the Python interpreter itself already provides the option: :program:`python -O -m compileall`. -Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix` +Similarly, the :func:`compile` function respects the :data:`sys.pycache_prefix` setting. The generated bytecode cache will only be useful if :func:`compile` is -run with the same :attr:`sys.pycache_prefix` (if any) that will be used at +run with the same :data:`sys.pycache_prefix` (if any) that will be used at runtime. Public functions diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst index 90138dd2a75f9..bc66815d57f34 100644 --- a/Doc/library/devmode.rst +++ b/Doc/library/devmode.rst @@ -81,7 +81,7 @@ Effects of the Python Development Mode: ignored for empty strings. * The :class:`io.IOBase` destructor logs ``close()`` exceptions. -* Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to +* Set the :attr:`~sys.flags.dev_mode` attribute of :data:`sys.flags` to ``True``. The Python Development Mode does not enable the :mod:`tracemalloc` module by diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst index 83e9e14ddcacd..0efb4897a1eb8 100644 --- a/Doc/library/filecmp.rst +++ b/Doc/library/filecmp.rst @@ -74,7 +74,7 @@ The :class:`dircmp` class Construct a new directory comparison object, to compare the directories *a* and *b*. *ignore* is a list of names to ignore, and defaults to - :attr:`filecmp.DEFAULT_IGNORES`. *hide* is a list of names to hide, and + :const:`filecmp.DEFAULT_IGNORES`. *hide* is a list of names to hide, and defaults to ``[os.curdir, os.pardir]``. The :class:`dircmp` class compares files by doing *shallow* comparisons diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst index e5ba9eef4074b..517ea4796c0a4 100644 --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -438,7 +438,7 @@ FTP_TLS Objects .. attribute:: FTP_TLS.ssl_version - The SSL version to use (defaults to :attr:`ssl.PROTOCOL_SSLv23`). + The SSL version to use (defaults to :data:`ssl.PROTOCOL_SSLv23`). .. method:: FTP_TLS.auth() diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 3d7e6603a7f35..e00d079ce4387 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1230,7 +1230,7 @@ are always available. They are listed here in alphabetical order. * Binary files are buffered in fixed-size chunks; the size of the buffer is chosen using a heuristic trying to determine the underlying device's "block - size" and falling back on :attr:`io.DEFAULT_BUFFER_SIZE`. On many systems, + size" and falling back on :const:`io.DEFAULT_BUFFER_SIZE`. On many systems, the buffer will typically be 4096 or 8192 bytes long. * "Interactive" text files (files for which :meth:`~io.IOBase.isatty` diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst index 5c2d0bbe0035a..0ecd93813c919 100644 --- a/Doc/library/gc.rst +++ b/Doc/library/gc.rst @@ -260,7 +260,7 @@ values but should not rebind them): .. versionchanged:: 3.4 Following :pep:`442`, objects with a :meth:`__del__` method don't end - up in :attr:`gc.garbage` anymore. + up in :data:`gc.garbage` anymore. .. data:: callbacks diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst index 9afaf86cb1328..2f26295d057f8 100644 --- a/Doc/library/gzip.rst +++ b/Doc/library/gzip.rst @@ -264,7 +264,7 @@ Command line options .. cmdoption:: file - If *file* is not specified, read from :attr:`sys.stdin`. + If *file* is not specified, read from :data:`sys.stdin`. .. cmdoption:: --fast diff --git a/Doc/library/importlib.resources.abc.rst b/Doc/library/importlib.resources.abc.rst index 4a563c8a5390d..b91e37510fe11 100644 --- a/Doc/library/importlib.resources.abc.rst +++ b/Doc/library/importlib.resources.abc.rst @@ -124,7 +124,7 @@ suitable for reading (same as :attr:`pathlib.Path.open`). When opening as text, accepts encoding parameters such as those - accepted by :attr:`io.TextIOWrapper`. + accepted by :class:`io.TextIOWrapper`. .. method:: read_bytes() diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index 55668dacfe49f..386a47b05f93d 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -466,7 +466,7 @@ ABC hierarchy:: The list of locations where the package's submodules will be found. Most of the time this is a single directory. The import system passes this attribute to ``__import__()`` and to finders - in the same way as :attr:`sys.path` but just for the package. + in the same way as :data:`sys.path` but just for the package. It is not set on non-package modules so it can be used as an indicator that the module is a package. @@ -717,7 +717,7 @@ ABC hierarchy:: automatically. When writing to the path fails because the path is read-only - (:attr:`errno.EACCES`/:exc:`PermissionError`), do not propagate the + (:const:`errno.EACCES`/:exc:`PermissionError`), do not propagate the exception. .. versionchanged:: 3.4 @@ -965,7 +965,7 @@ find and load modules. .. classmethod:: path_hook(*loader_details) - A class method which returns a closure for use on :attr:`sys.path_hooks`. + A class method which returns a closure for use on :data:`sys.path_hooks`. An instance of :class:`FileFinder` is returned by the closure using the path argument given to the closure directly and *loader_details* indirectly. @@ -1306,10 +1306,10 @@ an :term:`importer`. .. function:: find_spec(name, package=None) Find the :term:`spec ` for a module, optionally relative to - the specified **package** name. If the module is in :attr:`sys.modules`, + the specified **package** name. If the module is in :data:`sys.modules`, then ``sys.modules[name].__spec__`` is returned (unless the spec would be ``None`` or is not set, in which case :exc:`ValueError` is raised). - Otherwise a search using :attr:`sys.meta_path` is done. ``None`` is + Otherwise a search using :data:`sys.meta_path` is done. ``None`` is returned if no spec is found. If **name** is for a submodule (contains a dot), the parent module is @@ -1442,7 +1442,7 @@ an :term:`importer`. :meth:`~importlib.abc.Loader.create_module` method must return ``None`` or a type for which its ``__class__`` attribute can be mutated along with not using :term:`slots <__slots__>`. Finally, modules which substitute the object - placed into :attr:`sys.modules` will not work as there is no way to properly + placed into :data:`sys.modules` will not work as there is no way to properly replace the module references throughout the interpreter safely; :exc:`ValueError` is raised if such a substitution is detected. @@ -1566,9 +1566,9 @@ For deep customizations of import, you typically want to implement an :term:`importer`. This means managing both the :term:`finder` and :term:`loader` side of things. For finders there are two flavours to choose from depending on your needs: a :term:`meta path finder` or a :term:`path entry finder`. The -former is what you would put on :attr:`sys.meta_path` while the latter is what -you create using a :term:`path entry hook` on :attr:`sys.path_hooks` which works -with :attr:`sys.path` entries to potentially create a finder. This example will +former is what you would put on :data:`sys.meta_path` while the latter is what +you create using a :term:`path entry hook` on :data:`sys.path_hooks` which works +with :data:`sys.path` entries to potentially create a finder. This example will show you how to register your own importers so that import will use them (for creating an importer for yourself, read the documentation for the appropriate classes defined within this package):: diff --git a/Doc/library/json.rst b/Doc/library/json.rst index 5383614575c21..35a08995487c1 100644 --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -683,7 +683,7 @@ The :mod:`json.tool` module provides a simple command line interface to validate and pretty-print JSON objects. If the optional ``infile`` and ``outfile`` arguments are not -specified, :attr:`sys.stdin` and :attr:`sys.stdout` will be used respectively: +specified, :data:`sys.stdin` and :data:`sys.stdout` will be used respectively: .. code-block:: shell-session @@ -721,12 +721,12 @@ Command line options } ] - If *infile* is not specified, read from :attr:`sys.stdin`. + If *infile* is not specified, read from :data:`sys.stdin`. .. cmdoption:: outfile Write the output of the *infile* to the given *outfile*. Otherwise, write it - to :attr:`sys.stdout`. + to :data:`sys.stdout`. .. cmdoption:: --sort-keys diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index b645529827874..7f638251602c7 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -1051,8 +1051,8 @@ possible, while any potentially slow operations (such as sending an email via occur (e.g. because a bounded queue has filled up), the :meth:`~logging.Handler.handleError` method is called to handle the error. This can result in the record silently being dropped (if - :attr:`logging.raiseExceptions` is ``False``) or a message printed to - ``sys.stderr`` (if :attr:`logging.raiseExceptions` is ``True``). + :data:`logging.raiseExceptions` is ``False``) or a message printed to + ``sys.stderr`` (if :data:`logging.raiseExceptions` is ``True``). .. method:: prepare(record) diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst index 1a6b121de186a..8f6dffd04fa54 100644 --- a/Doc/library/os.path.rst +++ b/Doc/library/os.path.rst @@ -383,7 +383,7 @@ the :mod:`glob` module.) *start*. On Windows, :exc:`ValueError` is raised when *path* and *start* are on different drives. - *start* defaults to :attr:`os.curdir`. + *start* defaults to :data:`os.curdir`. .. availability:: Unix, Windows. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index dcd4d9e4ae2d0..31946f1a12b9d 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -60,7 +60,7 @@ Notes on the availability of these functions: ``'java'``. .. seealso:: - :attr:`sys.platform` has a finer granularity. :func:`os.uname` gives + :data:`sys.platform` has a finer granularity. :func:`os.uname` gives system-dependent version information. The :mod:`platform` module provides detailed checks for the diff --git a/Doc/library/platform.rst b/Doc/library/platform.rst index dc2d871b47d5e..60cd6b880847e 100644 --- a/Doc/library/platform.rst +++ b/Doc/library/platform.rst @@ -46,7 +46,7 @@ Cross Platform universal files containing multiple architectures. To get at the "64-bitness" of the current interpreter, it is more - reliable to query the :attr:`sys.maxsize` attribute:: + reliable to query the :data:`sys.maxsize` attribute:: is_64bits = sys.maxsize > 2**32 diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index ef3fd26518010..26a5cd531fa14 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -425,7 +425,7 @@ Directory and files operations determining if the file exists and executable. When no *path* is specified, the results of :func:`os.environ` are used, - returning either the "PATH" value or a fallback of :attr:`os.defpath`. + returning either the "PATH" value or a fallback of :data:`os.defpath`. On Windows, the current directory is always prepended to the *path* whether or not you use the default or provide your own, which is the behavior the diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index f47ed686136ff..4437bc496ab9b 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -5591,7 +5591,7 @@ From code, you can inspect the current limit and set a new one using these a getter and setter for the interpreter-wide limit. Subinterpreters have their own limit. -Information about the default and minimum can be found in :attr:`sys.int_info`: +Information about the default and minimum can be found in :data:`sys.int_info`: * :data:`sys.int_info.default_max_str_digits ` is the compiled-in default limit. diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 4c7fb27631146..78718f7983e04 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1609,7 +1609,7 @@ improves performance. If you ever encounter a presumed highly unusual situation where you need to prevent ``vfork()`` from being used by Python, you can set the -:attr:`subprocess._USE_VFORK` attribute to a false value. +:const:`subprocess._USE_VFORK` attribute to a false value. :: @@ -1617,7 +1617,7 @@ prevent ``vfork()`` from being used by Python, you can set the Setting this has no impact on use of ``posix_spawn()`` which could use ``vfork()`` internally within its libc implementation. There is a similar -:attr:`subprocess._USE_POSIX_SPAWN` attribute if you need to prevent use of +:const:`subprocess._USE_POSIX_SPAWN` attribute if you need to prevent use of that. :: diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 32713b2a8879d..643cdb876ee3c 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -166,7 +166,7 @@ always available. Python interpreter. (This information is not available in any other way --- ``modules.keys()`` only lists the imported modules.) - See also the :attr:`sys.stdlib_module_names` list. + See also the :data:`sys.stdlib_module_names` list. .. function:: call_tracing(func, args) @@ -1261,20 +1261,20 @@ always available. ================ =========================== .. versionchanged:: 3.3 - On Linux, :attr:`sys.platform` doesn't contain the major version anymore. + On Linux, :data:`sys.platform` doesn't contain the major version anymore. It is always ``'linux'``, instead of ``'linux2'`` or ``'linux3'``. Since older Python versions include the version number, it is recommended to always use the ``startswith`` idiom presented above. .. versionchanged:: 3.8 - On AIX, :attr:`sys.platform` doesn't contain the major version anymore. + On AIX, :data:`sys.platform` doesn't contain the major version anymore. It is always ``'aix'``, instead of ``'aix5'`` or ``'aix7'``. Since older Python versions include the version number, it is recommended to always use the ``startswith`` idiom presented above. .. seealso:: - :attr:`os.name` has a coarser granularity. :func:`os.uname` gives + :data:`os.name` has a coarser granularity. :func:`os.uname` gives system-dependent version information. The :mod:`platform` module provides detailed checks for the @@ -1685,7 +1685,7 @@ always available. ``email.mime`` sub-package and the ``email.message`` sub-module are not listed. - See also the :attr:`sys.builtin_module_names` list. + See also the :data:`sys.builtin_module_names` list. .. versionadded:: 3.10 diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst index 41428740ba3b2..b7b089a73e648 100644 --- a/Doc/library/tarfile.rst +++ b/Doc/library/tarfile.rst @@ -931,7 +931,7 @@ reused in custom filters: Implements the ``'tar'`` filter. - - Strip leading slashes (``/`` and :attr:`os.sep`) from filenames. + - Strip leading slashes (``/`` and :data:`os.sep`) from filenames. - :ref:`Refuse ` to extract files with absolute paths (in case the name is absolute even after stripping slashes, e.g. ``C:/foo`` on Windows). @@ -940,7 +940,7 @@ reused in custom filters: path (after following symlinks) would end up outside the destination. This raises :class:`~tarfile.OutsideDestinationError`. - Clear high mode bits (setuid, setgid, sticky) and group/other write bits - (:attr:`~stat.S_IWGRP`|:attr:`~stat.S_IWOTH`). + (:const:`~stat.S_IWGRP`|:const:`~stat.S_IWOTH`). Return the modified ``TarInfo`` member. @@ -965,10 +965,10 @@ reused in custom filters: - For regular files, including hard links: - Set the owner read and write permissions - (:attr:`~stat.S_IRUSR`|:attr:`~stat.S_IWUSR`). + (:const:`~stat.S_IRUSR`|:const:`~stat.S_IWUSR`). - Remove the group & other executable permission - (:attr:`~stat.S_IXGRP`|:attr:`~stat.S_IXOTH`) - if the owner doesn?t have it (:attr:`~stat.S_IXUSR`). + (:const:`~stat.S_IXGRP`|:const:`~stat.S_IXOTH`) + if the owner doesn?t have it (:const:`~stat.S_IXUSR`). - For other files (directories), set ``mode`` to ``None``, so that extraction methods skip applying permission bits. diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 0bddd31ae259d..73d1178906a8c 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -973,7 +973,7 @@ The :mod:`test.support` module defines the following classes: `SetErrorMode `_. On UNIX, :func:`resource.setrlimit` is used to set - :attr:`resource.RLIMIT_CORE`'s soft limit to 0 to prevent coredump file + :const:`resource.RLIMIT_CORE`'s soft limit to 0 to prevent coredump file creation. On both platforms, the old value is restored by :meth:`__exit__`. diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index c8e4317be7587..2aa3442270387 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -163,7 +163,7 @@ the modern themed widget set and API:: interpreter and calls :func:`exec` on the contents of :file:`.{className}.py` and :file:`.{baseName}.py`. The path for the profile files is the :envvar:`HOME` environment variable or, if that - isn't defined, then :attr:`os.curdir`. + isn't defined, then :data:`os.curdir`. .. attribute:: tk diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index c2ed7acaf5f0d..2c6316d66eadf 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1128,7 +1128,7 @@ Test cases If given, *level* should be either a numeric logging level or its string equivalent (for example either ``"ERROR"`` or - :attr:`logging.ERROR`). The default is :attr:`logging.INFO`. + :const:`logging.ERROR`). The default is :const:`logging.INFO`. The test passes if at least one message emitted inside the ``with`` block matches the *logger* and *level* conditions, otherwise it fails. @@ -1169,7 +1169,7 @@ Test cases If given, *level* should be either a numeric logging level or its string equivalent (for example either ``"ERROR"`` or - :attr:`logging.ERROR`). The default is :attr:`logging.INFO`. + :const:`logging.ERROR`). The default is :const:`logging.INFO`. Unlike :meth:`assertLogs`, nothing will be returned by the context manager. diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 112d5f8e8a84d..d2a39c57adfc5 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2482,8 +2482,8 @@ through the object's keys; for sequences, it should iterate through the values. .. impl-detail:: - In CPython, the length is required to be at most :attr:`sys.maxsize`. - If the length is larger than :attr:`!sys.maxsize` some features (such as + In CPython, the length is required to be at most :data:`sys.maxsize`. + If the length is larger than :data:`!sys.maxsize` some features (such as :func:`len`) may raise :exc:`OverflowError`. To prevent raising :exc:`!OverflowError` by truth value testing, an object must define a :meth:`__bool__` method. diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index f51a5247b6392..2a07e42f8443c 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -1188,7 +1188,7 @@ non-standard paths in the registry and user site-packages. Modules specified in the registry under ``Modules`` (not ``PythonPath``) may be imported by :class:`importlib.machinery.WindowsRegistryFinder`. This finder is enabled on Windows in 3.6.0 and earlier, but may need to - be explicitly added to :attr:`sys.meta_path` in the future. + be explicitly added to :data:`sys.meta_path` in the future. Additional modules ================== diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index dcfaef6ed2949..b410fe1ce3f08 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -1448,10 +1448,10 @@ complete list of changes, or look through the SVN logs for all the details. return times that are precise to fractions of a second; not all systems support such precision.) - Constants named :attr:`os.SEEK_SET`, :attr:`os.SEEK_CUR`, and - :attr:`os.SEEK_END` have been added; these are the parameters to the + Constants named :const:`os.SEEK_SET`, :const:`os.SEEK_CUR`, and + :const:`os.SEEK_END` have been added; these are the parameters to the :func:`os.lseek` function. Two new constants for locking are - :attr:`os.O_SHLOCK` and :attr:`os.O_EXLOCK`. + :const:`os.O_SHLOCK` and :const:`os.O_EXLOCK`. Two new functions, :func:`wait3` and :func:`wait4`, were added. They're similar the :func:`waitpid` function which waits for a child process to exit and returns @@ -1602,7 +1602,7 @@ complete list of changes, or look through the SVN logs for all the details. * The :mod:`unicodedata` module has been updated to use version 4.1.0 of the Unicode character database. Version 3.2.0 is required by some specifications, - so it's still available as :attr:`unicodedata.ucd_3_2_0`. + so it's still available as :data:`unicodedata.ucd_3_2_0`. * New module: the :mod:`uuid` module generates universally unique identifiers (UUIDs) according to :rfc:`4122`. The RFC defines several different UUID diff --git a/Doc/whatsnew/3.1.rst b/Doc/whatsnew/3.1.rst index fba8816bb243a..e4365d44928b5 100644 --- a/Doc/whatsnew/3.1.rst +++ b/Doc/whatsnew/3.1.rst @@ -370,7 +370,7 @@ New, Improved, and Deprecated Modules * The :mod:`io` module has three new constants for the :meth:`seek` method :data:`SEEK_SET`, :data:`SEEK_CUR`, and :data:`SEEK_END`. -* The :attr:`sys.version_info` tuple is now a named tuple:: +* The :data:`sys.version_info` tuple is now a named tuple:: >>> sys.version_info sys.version_info(major=3, minor=1, micro=0, releaselevel='alpha', serial=2) @@ -486,7 +486,7 @@ Changes to Python's build process and to the C API include: Apart from the performance improvements this change should be invisible to end users, with one exception: for testing and debugging purposes there's a - new :attr:`sys.int_info` that provides information about the + new :data:`sys.int_info` that provides information about the internal format, giving the number of bits per digit and the size in bytes of the C type used to store each digit:: diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index b912b94ebe5a4..973ce154bbdad 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -642,7 +642,7 @@ dataclasses datetime -------- -* Add :attr:`datetime.UTC`, a convenience alias for +* Add :const:`datetime.UTC`, a convenience alias for :attr:`datetime.timezone.utc`. (Contributed by Kabir Kwatra in :gh:`91973`.) * :meth:`datetime.date.fromisoformat`, :meth:`datetime.time.fromisoformat` and diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index 8e2f9e2716c4f..2e4c8ee45df84 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -424,7 +424,7 @@ protocols, the users must to be able access the environment using native strings even though the underlying platform may have a different convention. To bridge this gap, the :mod:`wsgiref` module has a new function, :func:`wsgiref.handlers.read_environ` for transcoding CGI variables from -:attr:`os.environ` into native strings and returning a new dictionary. +:data:`os.environ` into native strings and returning a new dictionary. .. seealso:: @@ -483,7 +483,7 @@ Some smaller changes made to the core Python language are: * The interpreter can now be started with a quiet option, ``-q``, to prevent the copyright and version information from being displayed in the interactive - mode. The option can be introspected using the :attr:`sys.flags` attribute: + mode. The option can be introspected using the :data:`sys.flags` attribute: .. code-block:: shell-session @@ -566,7 +566,7 @@ Some smaller changes made to the core Python language are: * The internal :c:type:`structsequence` tool now creates subclasses of tuple. This means that C structures like those returned by :func:`os.stat`, - :func:`time.gmtime`, and :attr:`sys.version_info` now work like a + :func:`time.gmtime`, and :data:`sys.version_info` now work like a :term:`named tuple` and now work with functions and methods that expect a tuple as an argument. This is a big step forward in making the C structures as flexible as their pure Python counterparts: @@ -596,7 +596,7 @@ Some smaller changes made to the core Python language are: module, or on the command line. A :exc:`ResourceWarning` is issued at interpreter shutdown if the - :data:`gc.garbage` list isn't empty, and if :attr:`gc.DEBUG_UNCOLLECTABLE` is + :data:`gc.garbage` list isn't empty, and if :const:`gc.DEBUG_UNCOLLECTABLE` is set, all uncollectable objects are printed. This is meant to make the programmer aware that their code contains object finalization issues. @@ -621,7 +621,7 @@ Some smaller changes made to the core Python language are: :class:`collections.Sequence` :term:`abstract base class`. As a result, the language will have a more uniform API. In addition, :class:`range` objects now support slicing and negative indices, even with values larger than - :attr:`sys.maxsize`. This makes *range* more interoperable with lists:: + :data:`sys.maxsize`. This makes *range* more interoperable with lists:: >>> range(0, 100, 2).count(10) 1 @@ -1005,13 +1005,13 @@ datetime and time after 1900. The new supported year range is from 1000 to 9999 inclusive. * Whenever a two-digit year is used in a time tuple, the interpretation has been - governed by :attr:`time.accept2dyear`. The default is ``True`` which means that + governed by :data:`time.accept2dyear`. The default is ``True`` which means that for a two-digit year, the century is guessed according to the POSIX rules governing the ``%y`` strptime format. Starting with Py3.2, use of the century guessing heuristic will emit a :exc:`DeprecationWarning`. Instead, it is recommended that - :attr:`time.accept2dyear` be set to ``False`` so that large date ranges + :data:`time.accept2dyear` be set to ``False`` so that large date ranges can be used without guesswork:: >>> import time, warnings @@ -1029,7 +1029,7 @@ datetime and time 'Fri Jan 1 12:34:56 11' Several functions now have significantly expanded date ranges. When - :attr:`time.accept2dyear` is false, the :func:`time.asctime` function will + :data:`time.accept2dyear` is false, the :func:`time.asctime` function will accept any year that fits in a C int, while the :func:`time.mktime` and :func:`time.strftime` functions will accept the full range supported by the corresponding operating system functions. @@ -1192,11 +1192,11 @@ can be set to "$" for the shell-style formatting provided by If no configuration is set-up before a logging event occurs, there is now a default configuration using a :class:`~logging.StreamHandler` directed to -:attr:`sys.stderr` for events of ``WARNING`` level or higher. Formerly, an +:data:`sys.stderr` for events of ``WARNING`` level or higher. Formerly, an event occurring before a configuration was set-up would either raise an exception or silently drop the event depending on the value of -:attr:`logging.raiseExceptions`. The new default handler is stored in -:attr:`logging.lastResort`. +:data:`logging.raiseExceptions`. The new default handler is stored in +:data:`logging.lastResort`. The use of filters has been simplified. Instead of creating a :class:`~logging.Filter` object, the predicate can be any Python callable that @@ -1298,7 +1298,7 @@ values are equal (:issue:`8188`):: hash(Decimal("1.5")) == hash(complex(1.5, 0)) Some of the hashing details are exposed through a new attribute, -:attr:`sys.hash_info`, which describes the bit width of the hash value, the +:data:`sys.hash_info`, which describes the bit width of the hash value, the prime modulus, the hash values for *infinity* and *nan*, and the multiplier used for the imaginary part of a number: @@ -1386,7 +1386,7 @@ select ------ The :mod:`select` module now exposes a new, constant attribute, -:attr:`~select.PIPE_BUF`, which gives the minimum number of bytes which are +:const:`~select.PIPE_BUF`, which gives the minimum number of bytes which are guaranteed not to block when :func:`select.select` says a pipe is ready for writing. @@ -1527,7 +1527,7 @@ filenames: b'Sehensw\xc3\xbcrdigkeiten' Some operating systems allow direct access to encoded bytes in the -environment. If so, the :attr:`os.supports_bytes_environ` constant will be +environment. If so, the :const:`os.supports_bytes_environ` constant will be true. For direct access to encoded environment variables (if available), @@ -2300,7 +2300,7 @@ turtledemo The demonstration code for the :mod:`turtle` module was moved from the *Demo* directory to main library. It includes over a dozen sample scripts with -lively displays. Being on :attr:`sys.path`, it can now be run directly +lively displays. Being on :data:`sys.path`, it can now be run directly from the command-line: .. code-block:: shell-session @@ -2564,7 +2564,7 @@ Changes to Python's build process and to the C API include: (:issue:`2443`). * A new C API function :c:func:`PySys_SetArgvEx` allows an embedded interpreter - to set :attr:`sys.argv` without also modifying :attr:`sys.path` + to set :data:`sys.argv` without also modifying :data:`sys.path` (:issue:`5753`). * :c:macro:`PyEval_CallObject` is now only available in macro form. The diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 74cfa8385c4c9..9bbf52c555178 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -647,7 +647,7 @@ PEP 421: Adding sys.implementation A new attribute on the :mod:`sys` module exposes details specific to the implementation of the currently running interpreter. The initial set of -attributes on :attr:`sys.implementation` are ``name``, ``version``, +attributes on :data:`sys.implementation` are ``name``, ``version``, ``hexversion``, and ``cache_tag``. The intention of ``sys.implementation`` is to consolidate into one namespace @@ -718,7 +718,7 @@ and does not enforce any method requirements. In terms of finders, :class:`importlib.machinery.FileFinder` exposes the mechanism used to search for source and bytecode files of a module. Previously -this class was an implicit member of :attr:`sys.path_hooks`. +this class was an implicit member of :data:`sys.path_hooks`. For loaders, the new abstract base class :class:`importlib.abc.FileLoader` helps write a loader that uses the file system as the storage mechanism for a module's @@ -734,7 +734,7 @@ provide the full name of the module now instead of just the tail end of the module's name. The :func:`importlib.invalidate_caches` function will now call the method with -the same name on all finders cached in :attr:`sys.path_importer_cache` to help +the same name on all finders cached in :data:`sys.path_importer_cache` to help clean up any stored state as necessary. Visible Changes @@ -744,8 +744,8 @@ For potential required changes to code, see the `Porting Python code`_ section. Beyond the expanse of what :mod:`importlib` now exposes, there are other -visible changes to import. The biggest is that :attr:`sys.meta_path` and -:attr:`sys.path_hooks` now store all of the meta path finders and path entry +visible changes to import. The biggest is that :data:`sys.meta_path` and +:data:`sys.path_hooks` now store all of the meta path finders and path entry hooks used by import. Previously the finders were implicit and hidden within the C code of import instead of being directly exposed. This means that one can now easily remove or change the order of the various finders to fit one's needs. @@ -760,9 +760,9 @@ Loaders are also now expected to set the ``__package__`` attribute from :pep:`366`. Once again, import itself is already setting this on all loaders from :mod:`importlib` and import itself is setting the attribute post-load. -``None`` is now inserted into :attr:`sys.path_importer_cache` when no finder -can be found on :attr:`sys.path_hooks`. Since :class:`imp.NullImporter` is not -directly exposed on :attr:`sys.path_hooks` it could no longer be relied upon to +``None`` is now inserted into :data:`sys.path_importer_cache` when no finder +can be found on :data:`sys.path_hooks`. Since :class:`imp.NullImporter` is not +directly exposed on :data:`sys.path_hooks` it could no longer be relied upon to always be available to use as a value representing no finder found. All other changes relate to semantic changes which should be taken into @@ -1951,7 +1951,7 @@ ssl * You can query the SSL compression algorithm used by an SSL socket, thanks to its new :meth:`~ssl.SSLSocket.compression` method. The new attribute - :attr:`~ssl.OP_NO_COMPRESSION` can be used to disable compression. + :const:`~ssl.OP_NO_COMPRESSION` can be used to disable compression. (Contributed by Antoine Pitrou in :issue:`13634`.) * Support has been added for the Next Protocol Negotiation extension using @@ -1965,7 +1965,7 @@ ssl * The :func:`~ssl.get_server_certificate` function now supports IPv6. (Contributed by Charles-Fran?ois Natali in :issue:`11811`.) -* New attribute :attr:`~ssl.OP_CIPHER_SERVER_PREFERENCE` allows setting +* New attribute :const:`~ssl.OP_CIPHER_SERVER_PREFERENCE` allows setting SSLv3 server sockets to use the server's cipher ordering preference rather than the client's (:issue:`13635`). @@ -2140,7 +2140,7 @@ New attribute :attr:`zlib.Decompress.eof` makes it possible to distinguish between a properly formed compressed stream and an incomplete or truncated one. (Contributed by Nadeem Vawda in :issue:`12646`.) -New attribute :attr:`zlib.ZLIB_RUNTIME_VERSION` reports the version string of +New attribute :const:`zlib.ZLIB_RUNTIME_VERSION` reports the version string of the underlying ``zlib`` library that is loaded at runtime. (Contributed by Torsten Landschoff in :issue:`12306`.) @@ -2377,16 +2377,16 @@ Porting Python code * :func:`__import__` no longer allows one to use an index value other than 0 for top-level modules. E.g. ``__import__('sys', level=1)`` is now an error. -* Because :attr:`sys.meta_path` and :attr:`sys.path_hooks` now have finders on +* Because :data:`sys.meta_path` and :data:`sys.path_hooks` now have finders on them by default, you will most likely want to use :meth:`list.insert` instead of :meth:`list.append` to add to those lists. -* Because ``None`` is now inserted into :attr:`sys.path_importer_cache`, if you +* Because ``None`` is now inserted into :data:`sys.path_importer_cache`, if you are clearing out entries in the dictionary of paths that do not have a finder, you will need to remove keys paired with values of ``None`` **and** :class:`imp.NullImporter` to be backwards-compatible. This will lead to extra overhead on older versions of Python that re-insert ``None`` into - :attr:`sys.path_importer_cache` where it represents the use of implicit + :data:`sys.path_importer_cache` where it represents the use of implicit finders, but semantically it should not change anything. * :class:`importlib.abc.Finder` no longer specifies a ``find_module()`` abstract @@ -2444,7 +2444,7 @@ Porting Python code error instead of sleeping forever. It has always raised an error on posix. * The ``ast.__version__`` constant has been removed. If you need to - make decisions affected by the AST version, use :attr:`sys.version_info` + make decisions affected by the AST version, use :data:`sys.version_info` to make the decision. * Code that used to work around the fact that the :mod:`threading` module used diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst index 9f894b40e2396..7436a552f529e 100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1323,14 +1323,14 @@ ability to query or set the resource limits for processes other than the one making the call. (Contributed by Christian Heimes in :issue:`16595`.) On Linux kernel version 2.6.36 or later, there are also some new -Linux specific constants: :attr:`~resource.RLIMIT_MSGQUEUE`, -:attr:`~resource.RLIMIT_NICE`, :attr:`~resource.RLIMIT_RTPRIO`, -:attr:`~resource.RLIMIT_RTTIME`, and :attr:`~resource.RLIMIT_SIGPENDING`. +Linux specific constants: :const:`~resource.RLIMIT_MSGQUEUE`, +:const:`~resource.RLIMIT_NICE`, :const:`~resource.RLIMIT_RTPRIO`, +:const:`~resource.RLIMIT_RTTIME`, and :const:`~resource.RLIMIT_SIGPENDING`. (Contributed by Christian Heimes in :issue:`19324`.) On FreeBSD version 9 and later, there some new FreeBSD specific constants: -:attr:`~resource.RLIMIT_SBSIZE`, :attr:`~resource.RLIMIT_SWAP`, and -:attr:`~resource.RLIMIT_NPTS`. (Contributed by Claudiu Popa in +:const:`~resource.RLIMIT_SBSIZE`, :const:`~resource.RLIMIT_SWAP`, and +:const:`~resource.RLIMIT_NPTS`. (Contributed by Claudiu Popa in :issue:`19343`.) @@ -1500,7 +1500,7 @@ implementation is required as most of the values aren't standardized and are platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.) The module supports new :mod:`~stat.ST_MODE` flags, :mod:`~stat.S_IFDOOR`, -:attr:`~stat.S_IFPORT`, and :attr:`~stat.S_IFWHT`. (Contributed by +:const:`~stat.S_IFPORT`, and :const:`~stat.S_IFWHT`. (Contributed by Christian Hiemes in :issue:`11016`.) @@ -1849,7 +1849,7 @@ Python's default implementation to a SipHash implementation on platforms that have a 64 bit data type. Any performance differences in comparison with the older FNV algorithm are trivial. -The PEP adds additional fields to the :attr:`sys.hash_info` named tuple to +The PEP adds additional fields to the :data:`sys.hash_info` named tuple to describe the hash algorithm in use by the currently executing binary. Otherwise, the PEP does not alter any existing CPython APIs. diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 79cceec543dcc..20c43e0e04a3c 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1388,7 +1388,7 @@ are treated as punctuation. site ---- -When specifying paths to add to :attr:`sys.path` in a ``.pth`` file, +When specifying paths to add to :data:`sys.path` in a ``.pth`` file, you may now specify file paths on top of directories (e.g. zip files). (Contributed by Wolfgang Langner in :issue:`26587`). @@ -2010,7 +2010,7 @@ been deprecated in previous versions of Python in favour of :meth:`importlib.abc.Loader.exec_module`. The :class:`importlib.machinery.WindowsRegistryFinder` class is now -deprecated. As of 3.6.0, it is still added to :attr:`sys.meta_path` by +deprecated. As of 3.6.0, it is still added to :data:`sys.meta_path` by default (on Windows), but this may change in future releases. os diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 54b2a4f8ed84d..90220dfb0c054 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1836,7 +1836,7 @@ Changes in Python behavior classes will affect their string representation. (Contributed by Serhiy Storchaka in :issue:`36793`.) -* On AIX, :attr:`sys.platform` doesn't contain the major version anymore. +* On AIX, :data:`sys.platform` doesn't contain the major version anymore. It is always ``'aix'``, instead of ``'aix3'`` .. ``'aix7'``. Since older Python versions include the version number, so it is recommended to always use ``sys.platform.startswith('aix')``. diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 49c8bd2f3e07d..c7466c13a4048 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -692,13 +692,13 @@ which has nanosecond resolution, rather than sys --- -Added a new :attr:`sys.platlibdir` attribute: name of the platform-specific +Added a new :data:`sys.platlibdir` attribute: name of the platform-specific library directory. It is used to build the path of standard library and the paths of installed extension modules. It is equal to ``"lib"`` on most platforms. On Fedora and SuSE, it is equal to ``"lib64"`` on 64-bit platforms. (Contributed by Jan Mat?jek, Mat?j Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.) -Previously, :attr:`sys.stderr` was block-buffered when non-interactive. Now +Previously, :data:`sys.stderr` was block-buffered when non-interactive. Now ``stderr`` defaults to always being line-buffered. (Contributed by Jendrik Seipp in :issue:`13601`.) @@ -1226,8 +1226,8 @@ Build Changes ============= * Added ``--with-platlibdir`` option to the ``configure`` script: name of the - platform-specific library directory, stored in the new :attr:`sys.platlibdir` - attribute. See :attr:`sys.platlibdir` attribute for more information. + platform-specific library directory, stored in the new :data:`sys.platlibdir` + attribute. See :data:`sys.platlibdir` attribute for more information. (Contributed by Jan Mat?jek, Mat?j Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.) diff --git a/Misc/NEWS.d/3.8.0a4.rst b/Misc/NEWS.d/3.8.0a4.rst index 5250d82f65e7b..d6fde41b7b11c 100644 --- a/Misc/NEWS.d/3.8.0a4.rst +++ b/Misc/NEWS.d/3.8.0a4.rst @@ -92,7 +92,7 @@ the field. .. nonce: wejLoC .. section: Core and Builtins -On AIX, :attr:`sys.platform` doesn't contain the major version anymore. +On AIX, :data:`sys.platform` doesn't contain the major version anymore. Always return ``'aix'``, instead of ``'aix3'`` .. ``'aix7'``. Since older Python versions include the version number, it is recommended to always use ``sys.platform.startswith('aix')``. Contributed by M. Felt. diff --git a/Misc/NEWS.d/3.9.0a5.rst b/Misc/NEWS.d/3.9.0a5.rst index 49a118ad7e430..a129e721a4cf9 100644 --- a/Misc/NEWS.d/3.9.0a5.rst +++ b/Misc/NEWS.d/3.9.0a5.rst @@ -582,7 +582,7 @@ Fix :mod:`json.tool` to catch :exc:`BrokenPipeError`. Patch by Dong-hee Na. Avoid a possible *"RuntimeError: dictionary changed size during iteration"* from :func:`inspect.getmodule` when it tried to loop through -:attr:`sys.modules`. +:data:`sys.modules`. .. @@ -989,7 +989,7 @@ modules are built. Add ``--with-platlibdir`` option to the configure script: name of the platform-specific library directory, stored in the new -:attr:`sys.platlibdir` attribute. It is used to build the path of +:data:`sys.platlibdir` attribute. It is used to build the path of platform-specific extension modules and the path of the standard library. It is equal to ``"lib"`` on most platforms. On Fedora and SuSE, it is equal to ``"lib64"`` on 64-bit platforms. Patch by Jan Mat?jek, Mat?j Cepl, From webhook-mailer at python.org Fri Jul 21 07:49:43 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 11:49:43 -0000 Subject: [Python-checkins] [3.12] gh-106909: Use role :const: for referencing module constants (GH-106910) (GH-106956) Message-ID: https://github.com/python/cpython/commit/84e52171b541ecc01f2d738cf82f5d4199a4bce7 commit: 84e52171b541ecc01f2d738cf82f5d4199a4bce7 branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T14:49:39+03:00 summary: [3.12] gh-106909: Use role :const: for referencing module constants (GH-106910) (GH-106956) (cherry picked from commit 4b9948617f91175783609769aa6160e5b49b9ccc) files: M Doc/c-api/exceptions.rst M Doc/faq/library.rst M Doc/install/index.rst M Doc/library/_thread.rst M Doc/library/asyncio-dev.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-platforms.rst M Doc/library/asyncio-subprocess.rst M Doc/library/exceptions.rst M Doc/library/fcntl.rst M Doc/library/ftplib.rst M Doc/library/http.client.rst M Doc/library/imaplib.rst M Doc/library/io.rst M Doc/library/multiprocessing.rst M Doc/library/optparse.rst M Doc/library/os.rst M Doc/library/poplib.rst M Doc/library/shelve.rst M Doc/library/smtplib.rst M Doc/library/socket.rst M Doc/library/sqlite3.rst M Doc/library/ssl.rst M Doc/library/sys.rst M Doc/library/tempfile.rst M Doc/library/test.rst M Doc/library/unittest.mock.rst M Doc/library/urllib.request.rst M Doc/library/xml.rst M Doc/using/configure.rst M Doc/whatsnew/2.7.rst M Doc/whatsnew/3.10.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.2.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.4.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.7.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a1.rst M Misc/NEWS.d/3.10.0a2.rst M Misc/NEWS.d/3.10.0a6.rst M Misc/NEWS.d/3.10.0a7.rst M Misc/NEWS.d/3.10.0b1.rst M Misc/NEWS.d/3.11.0a1.rst M Misc/NEWS.d/3.11.0a4.rst M Misc/NEWS.d/3.11.0b1.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0a2.rst M Misc/NEWS.d/3.12.0a3.rst M Misc/NEWS.d/3.12.0a4.rst M Misc/NEWS.d/3.12.0a6.rst M Misc/NEWS.d/3.12.0a7.rst M Misc/NEWS.d/3.12.0b1.rst M Misc/NEWS.d/3.6.0rc1.rst M Misc/NEWS.d/3.7.0a1.rst M Misc/NEWS.d/3.7.0a3.rst M Misc/NEWS.d/3.8.0a1.rst M Misc/NEWS.d/3.8.0a4.rst M Misc/NEWS.d/3.9.0a1.rst M Misc/NEWS.d/3.9.0a6.rst diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 6e09f829da304..3d4ceb360f5b1 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -666,7 +666,7 @@ Signal Handling to interrupt an operation). If the given signal isn't handled by Python (it was set to - :data:`signal.SIG_DFL` or :data:`signal.SIG_IGN`), it will be ignored. + :py:const:`signal.SIG_DFL` or :py:const:`signal.SIG_IGN`), it will be ignored. If *signum* is outside of the allowed range of signal numbers, ``-1`` is returned. Otherwise, ``0`` is returned. The error indicator is diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst index 597caaa778e1c..b43c7505c0401 100644 --- a/Doc/faq/library.rst +++ b/Doc/faq/library.rst @@ -566,7 +566,7 @@ use ``p.read(n)``. Note on a bug in popen2: unless your program calls ``wait()`` or ``waitpid()``, finished child processes are never removed, and eventually calls to popen2 will fail because of a limit on the number of child - processes. Calling :func:`os.waitpid` with the :data:`os.WNOHANG` option can + processes. Calling :func:`os.waitpid` with the :const:`os.WNOHANG` option can prevent this; a good place to insert such a call would be before calling ``popen2`` again. diff --git a/Doc/install/index.rst b/Doc/install/index.rst index beb34f0cf21b2..34d47fdb6a2f0 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -313,9 +313,9 @@ install into it. It is enabled with a simple option:: python setup.py install --user -Files will be installed into subdirectories of :data:`site.USER_BASE` (written +Files will be installed into subdirectories of :const:`site.USER_BASE` (written as :file:`{userbase}` hereafter). This scheme installs pure Python modules and -extension modules in the same location (also known as :data:`site.USER_SITE`). +extension modules in the same location (also known as :const:`site.USER_SITE`). Here are the values for UNIX, including macOS: =============== =========================================================== diff --git a/Doc/library/_thread.rst b/Doc/library/_thread.rst index ba9314e46ab6e..3ff6b48b083b5 100644 --- a/Doc/library/_thread.rst +++ b/Doc/library/_thread.rst @@ -70,10 +70,10 @@ This module defines the following constants and functions: there is no guarantee that the interruption will happen immediately. If given, *signum* is the number of the signal to simulate. - If *signum* is not given, :data:`signal.SIGINT` is simulated. + If *signum* is not given, :const:`signal.SIGINT` is simulated. If the given signal isn't handled by Python (it was set to - :data:`signal.SIG_DFL` or :data:`signal.SIG_IGN`), this function does + :const:`signal.SIG_DFL` or :const:`signal.SIG_IGN`), this function does nothing. .. versionchanged:: 3.10 diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index 921a394a59fec..c7d97008fb490 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -34,7 +34,7 @@ There are several ways to enable asyncio debug mode: In addition to enabling the debug mode, consider also: * setting the log level of the :ref:`asyncio logger ` to - :py:data:`logging.DEBUG`, for example the following snippet of code + :py:const:`logging.DEBUG`, for example the following snippet of code can be run at startup of the application:: logging.basicConfig(level=logging.DEBUG) @@ -142,7 +142,7 @@ Logging asyncio uses the :mod:`logging` module and all logging is performed via the ``"asyncio"`` logger. -The default log level is :py:data:`logging.INFO`, which can be easily +The default log level is :py:const:`logging.INFO`, which can be easily adjusted:: logging.getLogger("asyncio").setLevel(logging.WARNING) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 1ef8a5ab832e4..8f2d8f336c82b 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -403,11 +403,11 @@ Opening network connections Open a streaming transport connection to a given address specified by *host* and *port*. - The socket family can be either :py:data:`~socket.AF_INET` or - :py:data:`~socket.AF_INET6` depending on *host* (or the *family* + The socket family can be either :py:const:`~socket.AF_INET` or + :py:const:`~socket.AF_INET6` depending on *host* (or the *family* argument, if provided). - The socket type will be :py:data:`~socket.SOCK_STREAM`. + The socket type will be :py:const:`~socket.SOCK_STREAM`. *protocol_factory* must be a callable returning an :ref:`asyncio protocol ` implementation. @@ -509,7 +509,7 @@ Opening network connections .. versionchanged:: 3.6 - The socket option :py:data:`~socket.TCP_NODELAY` is set by default + The socket option :py:const:`~socket.TCP_NODELAY` is set by default for all TCP connections. .. versionchanged:: 3.7 @@ -552,11 +552,11 @@ Opening network connections Create a datagram connection. - The socket family can be either :py:data:`~socket.AF_INET`, - :py:data:`~socket.AF_INET6`, or :py:data:`~socket.AF_UNIX`, + The socket family can be either :py:const:`~socket.AF_INET`, + :py:const:`~socket.AF_INET6`, or :py:const:`~socket.AF_UNIX`, depending on *host* (or the *family* argument, if provided). - The socket type will be :py:data:`~socket.SOCK_DGRAM`. + The socket type will be :py:const:`~socket.SOCK_DGRAM`. *protocol_factory* must be a callable returning a :ref:`protocol ` implementation. @@ -581,7 +581,7 @@ Opening network connections * *reuse_port* tells the kernel to allow this endpoint to be bound to the same port as other existing endpoints are bound to, so long as they all set this flag when being created. This option is not supported on Windows - and some Unixes. If the :py:data:`~socket.SO_REUSEPORT` constant is not + and some Unixes. If the :py:const:`~socket.SO_REUSEPORT` constant is not defined then this capability is unsupported. * *allow_broadcast* tells the kernel to allow this endpoint to send @@ -607,7 +607,7 @@ Opening network connections .. versionchanged:: 3.8.1 The *reuse_address* parameter is no longer supported, as using - :py:data:`~sockets.SO_REUSEADDR` poses a significant security concern for + :py:const:`~sockets.SO_REUSEADDR` poses a significant security concern for UDP. Explicitly passing ``reuse_address=True`` will raise an exception. When multiple processes with differing UIDs assign sockets to an @@ -616,7 +616,7 @@ Opening network connections For supported platforms, *reuse_port* can be used as a replacement for similar functionality. With *reuse_port*, - :py:data:`~sockets.SO_REUSEPORT` is used instead, which specifically + :py:const:`~sockets.SO_REUSEPORT` is used instead, which specifically prevents processes with differing UIDs from assigning sockets to the same socket address. @@ -634,8 +634,8 @@ Opening network connections Create a Unix connection. - The socket family will be :py:data:`~socket.AF_UNIX`; socket - type will be :py:data:`~socket.SOCK_STREAM`. + The socket family will be :py:const:`~socket.AF_UNIX`; socket + type will be :py:const:`~socket.SOCK_STREAM`. A tuple of ``(transport, protocol)`` is returned on success. @@ -671,7 +671,7 @@ Creating network servers ssl_shutdown_timeout=None, \ start_serving=True) - Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) listening + Create a TCP server (socket type :const:`~socket.SOCK_STREAM`) listening on *port* of the *host* address. Returns a :class:`Server` object. @@ -699,10 +699,10 @@ Creating network servers be selected (note that if *host* resolves to multiple network interfaces, a different random port will be selected for each interface). - * *family* can be set to either :data:`socket.AF_INET` or - :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. + * *family* can be set to either :const:`socket.AF_INET` or + :const:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set, the *family* will be determined from host name - (defaults to :data:`~socket.AF_UNSPEC`). + (defaults to :const:`~socket.AF_UNSPEC`). * *flags* is a bitmask for :meth:`getaddrinfo`. @@ -756,7 +756,7 @@ Creating network servers .. versionchanged:: 3.6 Added *ssl_handshake_timeout* and *start_serving* parameters. - The socket option :py:data:`~socket.TCP_NODELAY` is set by default + The socket option :py:const:`~socket.TCP_NODELAY` is set by default for all TCP connections. .. versionchanged:: 3.11 @@ -777,7 +777,7 @@ Creating network servers start_serving=True) Similar to :meth:`loop.create_server` but works with the - :py:data:`~socket.AF_UNIX` socket family. + :py:const:`~socket.AF_UNIX` socket family. *path* is the name of a Unix domain socket, and is required, unless a *sock* argument is provided. Abstract Unix sockets, diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst index 50ad8a2ab7032..19ec726c1be06 100644 --- a/Doc/library/asyncio-platforms.rst +++ b/Doc/library/asyncio-platforms.rst @@ -37,7 +37,7 @@ All event loops on Windows do not support the following methods: * :meth:`loop.create_unix_connection` and :meth:`loop.create_unix_server` are not supported. - The :data:`socket.AF_UNIX` socket family is specific to Unix. + The :const:`socket.AF_UNIX` socket family is specific to Unix. * :meth:`loop.add_signal_handler` and :meth:`loop.remove_signal_handler` are not supported. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index b7c83aa04c09f..afade9b0b5070 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -249,7 +249,7 @@ their completion. Stop the child process. - On POSIX systems this method sends :py:data:`signal.SIGTERM` to the + On POSIX systems this method sends :py:const:`signal.SIGTERM` to the child process. On Windows the Win32 API function :c:func:`TerminateProcess` is diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index d54c49fff6b62..dcfbc486eeb35 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -659,8 +659,8 @@ depending on the system error code. Raised when an operation would block on an object (e.g. socket) set for non-blocking operation. - Corresponds to :c:data:`errno` :py:data:`~errno.EAGAIN`, :py:data:`~errno.EALREADY`, - :py:data:`~errno.EWOULDBLOCK` and :py:data:`~errno.EINPROGRESS`. + Corresponds to :c:data:`errno` :py:const:`~errno.EAGAIN`, :py:const:`~errno.EALREADY`, + :py:const:`~errno.EWOULDBLOCK` and :py:const:`~errno.EINPROGRESS`. In addition to those of :exc:`OSError`, :exc:`BlockingIOError` can have one more attribute: @@ -674,7 +674,7 @@ depending on the system error code. .. exception:: ChildProcessError Raised when an operation on a child process failed. - Corresponds to :c:data:`errno` :py:data:`~errno.ECHILD`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECHILD`. .. exception:: ConnectionError @@ -688,40 +688,40 @@ depending on the system error code. A subclass of :exc:`ConnectionError`, raised when trying to write on a pipe while the other end has been closed, or trying to write on a socket which has been shutdown for writing. - Corresponds to :c:data:`errno` :py:data:`~errno.EPIPE` and :py:data:`~errno.ESHUTDOWN`. + Corresponds to :c:data:`errno` :py:const:`~errno.EPIPE` and :py:const:`~errno.ESHUTDOWN`. .. exception:: ConnectionAbortedError A subclass of :exc:`ConnectionError`, raised when a connection attempt is aborted by the peer. - Corresponds to :c:data:`errno` :py:data:`~errno.ECONNABORTED`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECONNABORTED`. .. exception:: ConnectionRefusedError A subclass of :exc:`ConnectionError`, raised when a connection attempt is refused by the peer. - Corresponds to :c:data:`errno` :py:data:`~errno.ECONNREFUSED`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECONNREFUSED`. .. exception:: ConnectionResetError A subclass of :exc:`ConnectionError`, raised when a connection is reset by the peer. - Corresponds to :c:data:`errno` :py:data:`~errno.ECONNRESET`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECONNRESET`. .. exception:: FileExistsError Raised when trying to create a file or directory which already exists. - Corresponds to :c:data:`errno` :py:data:`~errno.EEXIST`. + Corresponds to :c:data:`errno` :py:const:`~errno.EEXIST`. .. exception:: FileNotFoundError Raised when a file or directory is requested but doesn't exist. - Corresponds to :c:data:`errno` :py:data:`~errno.ENOENT`. + Corresponds to :c:data:`errno` :py:const:`~errno.ENOENT`. .. exception:: InterruptedError Raised when a system call is interrupted by an incoming signal. - Corresponds to :c:data:`errno` :py:data:`~errno.EINTR`. + Corresponds to :c:data:`errno` :py:const:`~errno.EINTR`. .. versionchanged:: 3.5 Python now retries system calls when a syscall is interrupted by a @@ -732,7 +732,7 @@ depending on the system error code. Raised when a file operation (such as :func:`os.remove`) is requested on a directory. - Corresponds to :c:data:`errno` :py:data:`~errno.EISDIR`. + Corresponds to :c:data:`errno` :py:const:`~errno.EISDIR`. .. exception:: NotADirectoryError @@ -740,28 +740,28 @@ depending on the system error code. something which is not a directory. On most POSIX platforms, it may also be raised if an operation attempts to open or traverse a non-directory file as if it were a directory. - Corresponds to :c:data:`errno` :py:data:`~errno.ENOTDIR`. + Corresponds to :c:data:`errno` :py:const:`~errno.ENOTDIR`. .. exception:: PermissionError Raised when trying to run an operation without the adequate access rights - for example filesystem permissions. - Corresponds to :c:data:`errno` :py:data:`~errno.EACCES`, - :py:data:`~errno.EPERM`, and :py:data:`~errno.ENOTCAPABLE`. + Corresponds to :c:data:`errno` :py:const:`~errno.EACCES`, + :py:const:`~errno.EPERM`, and :py:const:`~errno.ENOTCAPABLE`. .. versionchanged:: 3.11.1 - WASI's :py:data:`~errno.ENOTCAPABLE` is now mapped to + WASI's :py:const:`~errno.ENOTCAPABLE` is now mapped to :exc:`PermissionError`. .. exception:: ProcessLookupError Raised when a given process doesn't exist. - Corresponds to :c:data:`errno` :py:data:`~errno.ESRCH`. + Corresponds to :c:data:`errno` :py:const:`~errno.ESRCH`. .. exception:: TimeoutError Raised when a system function timed out at the system level. - Corresponds to :c:data:`errno` :py:data:`~errno.ETIMEDOUT`. + Corresponds to :c:data:`errno` :py:const:`~errno.ETIMEDOUT`. .. versionadded:: 3.3 All the above :exc:`OSError` subclasses were added. diff --git a/Doc/library/fcntl.rst b/Doc/library/fcntl.rst index 997c7ea571fc0..5a27646a96f59 100644 --- a/Doc/library/fcntl.rst +++ b/Doc/library/fcntl.rst @@ -172,9 +172,9 @@ The module defines the following functions: which the lock starts, relative to *whence*, and *whence* is as with :func:`io.IOBase.seek`, specifically: - * :const:`0` -- relative to the start of the file (:data:`os.SEEK_SET`) - * :const:`1` -- relative to the current buffer position (:data:`os.SEEK_CUR`) - * :const:`2` -- relative to the end of the file (:data:`os.SEEK_END`) + * :const:`0` -- relative to the start of the file (:const:`os.SEEK_SET`) + * :const:`1` -- relative to the current buffer position (:const:`os.SEEK_CUR`) + * :const:`2` -- relative to the end of the file (:const:`os.SEEK_END`) The default for *start* is 0, which means to start at the beginning of the file. The default for *len* is 0 which means to lock to the end of the file. The @@ -201,7 +201,7 @@ using the :func:`flock` call may be better. .. seealso:: Module :mod:`os` - If the locking flags :data:`~os.O_SHLOCK` and :data:`~os.O_EXLOCK` are + If the locking flags :const:`~os.O_SHLOCK` and :const:`~os.O_EXLOCK` are present in the :mod:`os` module (on BSD only), the :func:`os.open` function provides an alternative to the :func:`lockf` and :func:`flock` functions. diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst index e7fb5b1ae2696..cd6e2d38b0286 100644 --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -105,7 +105,7 @@ The module defines the following items: .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionchanged:: 3.9 If the *timeout* parameter is set to be zero, it will raise a @@ -441,7 +441,7 @@ FTP_TLS Objects .. versionchanged:: 3.4 The method now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. method:: FTP_TLS.ccc() diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst index b9ceab699cef6..c46314fc5e253 100644 --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -83,7 +83,7 @@ The module provides the following classes: .. versionchanged:: 3.2 This class now supports HTTPS virtual hosts if possible (that is, - if :data:`ssl.HAS_SNI` is true). + if :const:`ssl.HAS_SNI` is true). .. versionchanged:: 3.4 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index 59d7711f9cbd3..1f774e64b0eae 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -106,7 +106,7 @@ There's also a subclass for secure connections: .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionchanged:: 3.9 The optional *timeout* parameter was added. @@ -503,7 +503,7 @@ An :class:`IMAP4` instance has the following methods: .. versionchanged:: 3.4 The method now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. method:: IMAP4.status(mailbox, names) diff --git a/Doc/library/io.rst b/Doc/library/io.rst index c9249da1c3c3d..7eec1f87583b8 100644 --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -423,7 +423,7 @@ I/O Base Classes .. versionadded:: 3.3 Some operating systems could support additional values, like - :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. The valid values + :const:`os.SEEK_HOLE` or :const:`os.SEEK_DATA`. The valid values for a file could depend on it being open in text or binary mode. .. method:: seekable() diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 8454296b815b4..2efc08f130af3 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -2707,7 +2707,7 @@ handler type) for messages from different processes to get mixed up. Returns the logger used by :mod:`multiprocessing`. If necessary, a new one will be created. - When first created the logger has level :data:`logging.NOTSET` and no + When first created the logger has level :const:`logging.NOTSET` and no default handler. Messages sent to this logger will not by default propagate to the root logger. diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index 0cff381745236..846b8e031f749 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -812,7 +812,7 @@ The first step in using :mod:`optparse` is to create an OptionParser instance. help option. When :mod:`optparse` prints the usage string, it expands ``%prog`` to ``os.path.basename(sys.argv[0])`` (or to ``prog`` if you passed that keyword argument). To suppress a usage message, pass the - special value :data:`optparse.SUPPRESS_USAGE`. + special value :const:`optparse.SUPPRESS_USAGE`. ``option_list`` (default: ``[]``) A list of Option objects to populate the parser with. The options in @@ -1078,7 +1078,7 @@ relevant to a particular option, or fail to pass a required option attribute, Help text to print for this option when listing all available options after the user supplies a :attr:`~Option.help` option (such as ``--help``). If no help text is supplied, the option will be listed without help text. To - hide this option, use the special value :data:`optparse.SUPPRESS_HELP`. + hide this option, use the special value :const:`optparse.SUPPRESS_HELP`. .. attribute:: Option.metavar @@ -1250,7 +1250,7 @@ must specify for any option using that action. If no :attr:`~Option.help` string is supplied for an option, it will still be listed in the help message. To omit an option entirely, use the special value - :data:`optparse.SUPPRESS_HELP`. + :const:`optparse.SUPPRESS_HELP`. :mod:`optparse` automatically adds a :attr:`~Option.help` option to all OptionParsers, so you do not normally need to create one. @@ -1521,7 +1521,7 @@ OptionParser supports several other public methods: Set the usage string according to the rules described above for the ``usage`` constructor keyword argument. Passing ``None`` sets the default usage - string; use :data:`optparse.SUPPRESS_USAGE` to suppress a usage message. + string; use :const:`optparse.SUPPRESS_USAGE` to suppress a usage message. .. method:: OptionParser.print_usage(file=None) diff --git a/Doc/library/os.rst b/Doc/library/os.rst index b35010498f214..76f797ac2c4cb 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -233,7 +233,7 @@ process and user. :data:`environ` and :data:`environb` are synchronized (modifying :data:`environb` updates :data:`environ`, and vice versa). - :data:`environb` is only available if :data:`supports_bytes_environ` is + :data:`environb` is only available if :const:`supports_bytes_environ` is ``True``. .. versionadded:: 3.2 @@ -331,7 +331,7 @@ process and user. future environment changes. - :func:`getenvb` is only available if :data:`supports_bytes_environ` + :func:`getenvb` is only available if :const:`supports_bytes_environ` is ``True``. .. availability:: Unix. @@ -923,7 +923,7 @@ as internal buffering of data. In Linux kernel older than 5.3, the files pointed by *src* and *dst* must reside in the same filesystem, otherwise an :exc:`OSError` is - raised with :attr:`~OSError.errno` set to :data:`errno.EXDEV`. + raised with :attr:`~OSError.errno` set to :const:`errno.EXDEV`. This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, @@ -1181,7 +1181,7 @@ as internal buffering of data. .. versionadded:: 3.3 Some operating systems could support additional values, like - :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. + :const:`os.SEEK_HOLE` or :const:`os.SEEK_DATA`. .. function:: open(path, flags, mode=0o777, *, dir_fd=None) @@ -1422,7 +1422,7 @@ or `the MSDN `_ on Windo If some data was successfully read, it will return the number of bytes read. If no bytes were read, it will return ``-1`` and set errno to - :data:`errno.EAGAIN`. + :const:`errno.EAGAIN`. .. availability:: Linux >= 4.14. @@ -1627,7 +1627,7 @@ or `the MSDN `_ on Windo *offset_dst*. The offset associated to the file descriptor that refers to a pipe must be ``None``. The files pointed by *src* and *dst* must reside in the same filesystem, otherwise an :exc:`OSError` is raised with - :attr:`~OSError.errno` set to :data:`errno.EXDEV`. + :attr:`~OSError.errno` set to :const:`errno.EXDEV`. This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, @@ -1960,18 +1960,18 @@ features: Set the flags of *path* to the numeric *flags*. *flags* may take a combination (bitwise OR) of the following values (as defined in the :mod:`stat` module): - * :data:`stat.UF_NODUMP` - * :data:`stat.UF_IMMUTABLE` - * :data:`stat.UF_APPEND` - * :data:`stat.UF_OPAQUE` - * :data:`stat.UF_NOUNLINK` - * :data:`stat.UF_COMPRESSED` - * :data:`stat.UF_HIDDEN` - * :data:`stat.SF_ARCHIVED` - * :data:`stat.SF_IMMUTABLE` - * :data:`stat.SF_APPEND` - * :data:`stat.SF_NOUNLINK` - * :data:`stat.SF_SNAPSHOT` + * :const:`stat.UF_NODUMP` + * :const:`stat.UF_IMMUTABLE` + * :const:`stat.UF_APPEND` + * :const:`stat.UF_OPAQUE` + * :const:`stat.UF_NOUNLINK` + * :const:`stat.UF_COMPRESSED` + * :const:`stat.UF_HIDDEN` + * :const:`stat.SF_ARCHIVED` + * :const:`stat.SF_IMMUTABLE` + * :const:`stat.SF_APPEND` + * :const:`stat.SF_NOUNLINK` + * :const:`stat.SF_SNAPSHOT` This function can support :ref:`not following symlinks `. @@ -1992,25 +1992,25 @@ features: following values (as defined in the :mod:`stat` module) or bitwise ORed combinations of them: - * :data:`stat.S_ISUID` - * :data:`stat.S_ISGID` - * :data:`stat.S_ENFMT` - * :data:`stat.S_ISVTX` - * :data:`stat.S_IREAD` - * :data:`stat.S_IWRITE` - * :data:`stat.S_IEXEC` - * :data:`stat.S_IRWXU` - * :data:`stat.S_IRUSR` - * :data:`stat.S_IWUSR` - * :data:`stat.S_IXUSR` - * :data:`stat.S_IRWXG` - * :data:`stat.S_IRGRP` - * :data:`stat.S_IWGRP` - * :data:`stat.S_IXGRP` - * :data:`stat.S_IRWXO` - * :data:`stat.S_IROTH` - * :data:`stat.S_IWOTH` - * :data:`stat.S_IXOTH` + * :const:`stat.S_ISUID` + * :const:`stat.S_ISGID` + * :const:`stat.S_ENFMT` + * :const:`stat.S_ISVTX` + * :const:`stat.S_IREAD` + * :const:`stat.S_IWRITE` + * :const:`stat.S_IEXEC` + * :const:`stat.S_IRWXU` + * :const:`stat.S_IRUSR` + * :const:`stat.S_IWUSR` + * :const:`stat.S_IXUSR` + * :const:`stat.S_IRWXG` + * :const:`stat.S_IRGRP` + * :const:`stat.S_IWGRP` + * :const:`stat.S_IXGRP` + * :const:`stat.S_IRWXO` + * :const:`stat.S_IROTH` + * :const:`stat.S_IWOTH` + * :const:`stat.S_IXOTH` This function can support :ref:`specifying a file descriptor `, :ref:`paths relative to directory descriptors ` and :ref:`not @@ -4151,8 +4151,8 @@ written in Python, such as a mail server's external command delivery program. Send signal *sig* to the process *pid*. Constants for the specific signals available on the host platform are defined in the :mod:`signal` module. - Windows: The :data:`signal.CTRL_C_EVENT` and - :data:`signal.CTRL_BREAK_EVENT` signals are special signals which can + Windows: The :const:`signal.CTRL_C_EVENT` and + :const:`signal.CTRL_BREAK_EVENT` signals are special signals which can only be sent to console processes which share a common console window, e.g., some subprocesses. Any other value for *sig* will cause the process to be unconditionally killed by the TerminateProcess API, and the exit code @@ -4205,7 +4205,7 @@ written in Python, such as a mail server's external command delivery program. This flag indicates that the file descriptor will be non-blocking. If the process referred to by the file descriptor has not yet terminated, then an attempt to wait on the file descriptor using :manpage:`waitid(2)` - will immediately return the error :data:`~errno.EAGAIN` rather than blocking. + will immediately return the error :const:`~errno.EAGAIN` rather than blocking. .. availability:: Linux >= 5.10 .. versionadded:: 3.12 @@ -4654,7 +4654,7 @@ written in Python, such as a mail server's external command delivery program. * :attr:`!si_pid` (process ID) * :attr:`!si_uid` (real user ID of the child) - * :attr:`!si_signo` (always :data:`~signal.SIGCHLD`) + * :attr:`!si_signo` (always :const:`~signal.SIGCHLD`) * :attr:`!si_status` (the exit status or signal number, depending on :attr:`!si_code`) * :attr:`!si_code` (see :data:`CLD_EXITED` for possible values) @@ -4892,7 +4892,7 @@ used to determine the disposition of a process. .. function:: WIFCONTINUED(status) Return ``True`` if a stopped child has been resumed by delivery of - :data:`~signal.SIGCONT` (if the process has been continued from a job + :const:`~signal.SIGCONT` (if the process has been continued from a job control stop), otherwise return ``False``. See :data:`WCONTINUED` option. @@ -5264,7 +5264,7 @@ Random numbers ``/dev/urandom`` devices. The flags argument is a bit mask that can contain zero or more of the - following values ORed together: :py:data:`os.GRND_RANDOM` and + following values ORed together: :py:const:`os.GRND_RANDOM` and :py:data:`GRND_NONBLOCK`. See also the `Linux getrandom() manual page diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index 260c4a63d1203..fbd5e150b4cd5 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -77,7 +77,7 @@ The :mod:`poplib` module provides two classes: .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionchanged:: 3.9 If the *timeout* parameter is set to be zero, it will raise a @@ -240,7 +240,7 @@ A :class:`POP3` instance has the following methods: This method supports hostname checking via :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionadded:: 3.4 diff --git a/Doc/library/shelve.rst b/Doc/library/shelve.rst index dc87af398ed75..01314f491f47a 100644 --- a/Doc/library/shelve.rst +++ b/Doc/library/shelve.rst @@ -25,7 +25,7 @@ lots of shared sub-objects. The keys are ordinary strings. database file is opened for reading and writing. The optional *flag* parameter has the same interpretation as the *flag* parameter of :func:`dbm.open`. - By default, pickles created with :data:`pickle.DEFAULT_PROTOCOL` are used + By default, pickles created with :const:`pickle.DEFAULT_PROTOCOL` are used to serialize values. The version of the pickle protocol can be specified with the *protocol* parameter. @@ -42,7 +42,7 @@ lots of shared sub-objects. The keys are ordinary strings. mutated). .. versionchanged:: 3.10 - :data:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle + :const:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle protocol. .. versionchanged:: 3.11 @@ -119,7 +119,7 @@ Restrictions A subclass of :class:`collections.abc.MutableMapping` which stores pickled values in the *dict* object. - By default, pickles created with :data:`pickle.DEFAULT_PROTOCOL` are used + By default, pickles created with :const:`pickle.DEFAULT_PROTOCOL` are used to serialize values. The version of the pickle protocol can be specified with the *protocol* parameter. See the :mod:`pickle` documentation for a discussion of the pickle protocols. @@ -143,7 +143,7 @@ Restrictions Added context manager support. .. versionchanged:: 3.10 - :data:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle + :const:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle protocol. diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index f90274feb6bf9..aaec2aa1ef1db 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -98,7 +98,7 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions). .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionchanged:: 3.9 If the *timeout* parameter is set to be zero, it will raise a @@ -418,7 +418,7 @@ An :class:`SMTP` instance has the following methods: .. versionchanged:: 3.4 The method now supports hostname check with :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see - :data:`~ssl.HAS_SNI`). + :const:`~ssl.HAS_SNI`). .. versionchanged:: 3.5 The error raised for lack of STARTTLS support is now the diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index f2408cb95ff31..4f220e8a09897 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -2252,7 +2252,7 @@ This is because the previous execution has left the socket in a ``TIME_WAIT`` state, and can't be immediately reused. There is a :mod:`socket` flag to set, in order to prevent this, -:data:`socket.SO_REUSEADDR`:: +:const:`socket.SO_REUSEADDR`:: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 2e2c28b1e5fe7..23a22e637c0a9 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -299,7 +299,7 @@ Module functions Can be ``"DEFERRED"`` (default), ``"EXCLUSIVE"`` or ``"IMMEDIATE"``; or ``None`` to disable opening transactions implicitly. Has no effect unless :attr:`Connection.autocommit` is set to - :data:`~sqlite3.LEGACY_TRANSACTION_CONTROL` (the default). + :const:`~sqlite3.LEGACY_TRANSACTION_CONTROL` (the default). :type isolation_level: str | None :param bool check_same_thread: @@ -334,7 +334,7 @@ Module functions See :attr:`Connection.autocommit` and :ref:`sqlite3-transaction-control-autocommit` for more information. *autocommit* currently defaults to - :data:`~sqlite3.LEGACY_TRANSACTION_CONTROL`. + :const:`~sqlite3.LEGACY_TRANSACTION_CONTROL`. The default will change to ``False`` in a future Python release. :type autocommit: bool @@ -1821,9 +1821,9 @@ Blob objects .. method:: seek(offset, origin=os.SEEK_SET, /) Set the current access position of the blob to *offset*. The *origin* - argument defaults to :data:`os.SEEK_SET` (absolute blob positioning). - Other values for *origin* are :data:`os.SEEK_CUR` (seek relative to the - current position) and :data:`os.SEEK_END` (seek relative to the blob?s + argument defaults to :const:`os.SEEK_SET` (absolute blob positioning). + Other values for *origin* are :const:`os.SEEK_CUR` (seek relative to the + current position) and :const:`os.SEEK_END` (seek relative to the blob?s end). diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 18a6c5ab4858a..7f09552632768 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -139,7 +139,7 @@ purposes. The settings are: :data:`PROTOCOL_TLS_CLIENT` or :data:`PROTOCOL_TLS_SERVER`, :data:`OP_NO_SSLv2`, and :data:`OP_NO_SSLv3` with high encryption cipher suites without RC4 and - without unauthenticated cipher suites. Passing :data:`~Purpose.SERVER_AUTH` + without unauthenticated cipher suites. Passing :const:`~Purpose.SERVER_AUTH` as *purpose* sets :data:`~SSLContext.verify_mode` to :data:`CERT_REQUIRED` and either loads CA certificates (when at least one of *cafile*, *capath* or *cadata* is given) or uses :meth:`SSLContext.load_default_certs` to load @@ -1484,9 +1484,9 @@ to speed up repeated connections from the same clients. load CA certificates from other locations, too. The *purpose* flag specifies what kind of CA certificates are loaded. The - default settings :data:`Purpose.SERVER_AUTH` loads certificates, that are + default settings :const:`Purpose.SERVER_AUTH` loads certificates, that are flagged and trusted for TLS web server authentication (client side - sockets). :data:`Purpose.CLIENT_AUTH` loads CA certificates for client + sockets). :const:`Purpose.CLIENT_AUTH` loads CA certificates for client certificate verification on the server side. .. versionadded:: 3.4 @@ -1729,7 +1729,7 @@ to speed up repeated connections from the same clients. Wrap an existing Python socket *sock* and return an instance of :attr:`SSLContext.sslsocket_class` (default :class:`SSLSocket`). The returned SSL socket is tied to the context, its settings and certificates. - *sock* must be a :data:`~socket.SOCK_STREAM` socket; other + *sock* must be a :const:`~socket.SOCK_STREAM` socket; other socket types are unsupported. The parameter ``server_side`` is a boolean which identifies whether diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index bacf8ceac5041..911d73b546b5c 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -697,7 +697,7 @@ always available. Return the current value of the flags that are used for :c:func:`dlopen` calls. Symbolic names for the flag values can be found in the :mod:`os` module (``RTLD_xxx`` constants, e.g. - :data:`os.RTLD_LAZY`). + :const:`os.RTLD_LAZY`). .. availability:: Unix. @@ -1368,7 +1368,7 @@ always available. ``sys.setdlopenflags(0)``. To share symbols across extension modules, call as ``sys.setdlopenflags(os.RTLD_GLOBAL)``. Symbolic names for the flag values can be found in the :mod:`os` module (``RTLD_xxx`` constants, e.g. - :data:`os.RTLD_LAZY`). + :const:`os.RTLD_LAZY`). .. availability:: Unix. diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index fd4c294613fd3..097f7087eccab 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -59,7 +59,7 @@ The module defines the following user-callable items: platforms, it is a file-like object whose :attr:`!file` attribute is the underlying true file object. - The :py:data:`os.O_TMPFILE` flag is used if it is available and works + The :py:const:`os.O_TMPFILE` flag is used if it is available and works (Linux-specific, requires Linux kernel 3.11 or later). On platforms that are neither Posix nor Cygwin, TemporaryFile is an alias @@ -69,7 +69,7 @@ The module defines the following user-callable items: .. versionchanged:: 3.5 - The :py:data:`os.O_TMPFILE` flag is now used if available. + The :py:const:`os.O_TMPFILE` flag is now used if available. .. versionchanged:: 3.8 Added *errors* parameter. diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 1b045c7de83a8..d69a3e326a411 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -472,7 +472,7 @@ The :mod:`test.support` module defines the following functions: .. function:: with_pymalloc() - Return :data:`_testcapi.WITH_PYMALLOC`. + Return :const:`_testcapi.WITH_PYMALLOC`. .. function:: requires(resource, msg=None) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 6c4d801f69f5a..13cd593f9549a 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -2438,7 +2438,7 @@ behaviour you can switch it off by setting the module level switch Alternatively you can just use ``vars(my_mock)`` (instance members) and ``dir(type(my_mock))`` (type members) to bypass the filtering irrespective of -:data:`mock.FILTER_DIR`. +:const:`mock.FILTER_DIR`. mock_open diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst index 1b05458280d89..35b8f5b471dd9 100644 --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -99,7 +99,7 @@ The :mod:`urllib.request` module defines the following functions: .. versionchanged:: 3.2 HTTPS virtual hosts are now supported if possible (that is, if - :data:`ssl.HAS_SNI` is true). + :const:`ssl.HAS_SNI` is true). .. versionadded:: 3.2 *data* can be an iterable object. diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index 20b0905bb1093..cf30de67719b9 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -73,7 +73,7 @@ decompression bomb Safe Safe Safe 1. Expat 2.4.1 and newer is not vulnerable to the "billion laughs" and "quadratic blowup" vulnerabilities. Items still listed as vulnerable due to potential reliance on system-provided libraries. Check - :data:`pyexpat.EXPAT_VERSION`. + :const:`pyexpat.EXPAT_VERSION`. 2. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a :exc:`ParserError` when an entity occurs. 3. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index fbe280d641317..924e73dc54da2 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -97,7 +97,7 @@ General Options .. cmdoption:: --with-tzpath= - Select the default time zone search path for :data:`zoneinfo.TZPATH`. + Select the default time zone search path for :const:`zoneinfo.TZPATH`. See the :ref:`Compile-time configuration ` of the :mod:`zoneinfo` module. @@ -112,7 +112,7 @@ General Options Build the ``_decimal`` extension module using a thread-local context rather than a coroutine-local context (default), see the :mod:`decimal` module. - See :data:`decimal.HAVE_CONTEXTVAR` and the :mod:`contextvars` module. + See :const:`decimal.HAVE_CONTEXTVAR` and the :mod:`contextvars` module. .. versionadded:: 3.9 diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index e6bc9a21b365d..a1d522b7ce6da 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -1556,9 +1556,9 @@ changes, or look through the Subversion logs for all the details. :issue:`8484`.) The version of OpenSSL being used is now available as the module - attributes :data:`ssl.OPENSSL_VERSION` (a string), - :data:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and - :data:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Added by Antoine + attributes :const:`ssl.OPENSSL_VERSION` (a string), + :const:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and + :const:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Added by Antoine Pitrou; :issue:`8321`.) * The :mod:`struct` module will no longer silently ignore overflow diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 7f80852c49548..bb271d0a45568 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1253,8 +1253,8 @@ descriptors without copying between kernel address space and user address space, where one of the file descriptors must refer to a pipe. (Contributed by Pablo Galindo in :issue:`41625`.) -Add :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` -and :data:`~os.O_NOFOLLOW_ANY` for macOS. +Add :const:`~os.O_EVTONLY`, :const:`~os.O_FSYNC`, :const:`~os.O_SYMLINK` +and :const:`~os.O_NOFOLLOW_ANY` for macOS. (Contributed by Dong-hee Na in :issue:`43106`.) os.path @@ -1319,7 +1319,7 @@ objects in the tree returned by :func:`pyclbr.readline` and shelve ------ -The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default +The :mod:`shelve` module now uses :const:`pickle.DEFAULT_PROTOCOL` by default instead of :mod:`pickle` protocol ``3`` when creating shelves. (Contributed by Zackery Spytz in :issue:`34204`.) @@ -1356,7 +1356,7 @@ The ssl module requires OpenSSL 1.1.1 or newer. (Contributed by Christian Heimes in :pep:`644` and :issue:`43669`.) The ssl module has preliminary support for OpenSSL 3.0.0 and new option -:data:`~ssl.OP_IGNORE_UNEXPECTED_EOF`. +:const:`~ssl.OP_IGNORE_UNEXPECTED_EOF`. (Contributed by Christian Heimes in :issue:`38820`, :issue:`43794`, :issue:`43788`, :issue:`43791`, :issue:`43799`, :issue:`43920`, :issue:`43789`, and :issue:`43811`.) @@ -1387,7 +1387,7 @@ Add a *timeout* parameter to the :func:`ssl.get_server_certificate` function. The ssl module uses heap-types and multi-phase initialization. (Contributed by Christian Heimes in :issue:`42333`.) -A new verify flag :data:`~ssl.VERIFY_X509_PARTIAL_CHAIN` has been added. +A new verify flag :const:`~ssl.VERIFY_X509_PARTIAL_CHAIN` has been added. (Contributed by l0x in :issue:`40849`.) sqlite3 @@ -1413,7 +1413,7 @@ _thread ------- :func:`_thread.interrupt_main` now takes an optional signal number to -simulate (the default is still :data:`signal.SIGINT`). +simulate (the default is still :const:`signal.SIGINT`). (Contributed by Antoine Pitrou in :issue:`43356`.) threading @@ -1757,8 +1757,8 @@ Deprecated * :data:`~ssl.PROTOCOL_SSLv2`, :data:`~ssl.PROTOCOL_SSLv3`, :data:`~ssl.PROTOCOL_SSLv23`, :data:`~ssl.PROTOCOL_TLSv1`, :data:`~ssl.PROTOCOL_TLSv1_1`, :data:`~ssl.PROTOCOL_TLSv1_2`, and - :data:`~ssl.PROTOCOL_TLS` are deprecated in favor of - :data:`~ssl.PROTOCOL_TLS_CLIENT` and :data:`~ssl.PROTOCOL_TLS_SERVER` + :const:`~ssl.PROTOCOL_TLS` are deprecated in favor of + :const:`~ssl.PROTOCOL_TLS_CLIENT` and :const:`~ssl.PROTOCOL_TLS_SERVER` * :func:`~ssl.wrap_socket` is replaced by :meth:`ssl.SSLContext.wrap_socket` diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 5ff99fc2d3f61..0585eccf198e8 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -690,7 +690,7 @@ enum * Added the :func:`~enum.global_enum` enum decorator, which adjusts :meth:`~object.__repr__` and :meth:`~object.__str__` to show values as members of their module rather than the enum class. - For example, ``'re.ASCII'`` for the :data:`~re.ASCII` member + For example, ``'re.ASCII'`` for the :const:`~re.ASCII` member of :class:`re.RegexFlag` rather than ``'RegexFlag.ASCII'``. * Enhanced :class:`~enum.Flag` to support @@ -1063,8 +1063,8 @@ threading * On Unix, if the ``sem_clockwait()`` function is available in the C library (glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses - the monotonic clock (:data:`time.CLOCK_MONOTONIC`) for the timeout, rather - than using the system clock (:data:`time.CLOCK_REALTIME`), to not be affected + the monotonic clock (:const:`time.CLOCK_MONOTONIC`) for the timeout, rather + than using the system clock (:const:`time.CLOCK_REALTIME`), to not be affected by system clock changes. (Contributed by Victor Stinner in :issue:`41710`.) @@ -1812,7 +1812,7 @@ Standard Library (Contributed by Serhiy Storchaka in :gh:`91760`.) * In the :mod:`re` module, the :func:`!re.template` function - and the corresponding :data:`!re.TEMPLATE` and :data:`!re.T` flags + and the corresponding :const:`!re.TEMPLATE` and :const:`!re.T` flags are deprecated, as they were undocumented and lacked an obvious purpose. They will be removed in Python 3.13. (Contributed by Serhiy Storchaka and Miro Hron?ok in :gh:`92728`.) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 03d559e455393..b4a3fd5c98291 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -555,7 +555,7 @@ calendar csv --- -* Add :data:`~csv.QUOTE_NOTNULL` and :data:`~csv.QUOTE_STRINGS` flags to +* Add :const:`~csv.QUOTE_NOTNULL` and :const:`~csv.QUOTE_STRINGS` flags to provide finer grained control of ``None`` and empty strings by :class:`~csv.writer` objects. @@ -612,7 +612,7 @@ math os -- -* Add :data:`os.PIDFD_NONBLOCK` to open a file descriptor +* Add :const:`os.PIDFD_NONBLOCK` to open a file descriptor for a process with :func:`os.pidfd_open` in non-blocking mode. (Contributed by Kumar Aditya in :gh:`93312`.) diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index 7af0c0288376f..504bbf6deda7c 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -1666,9 +1666,9 @@ for secure (encrypted, authenticated) internet connections: algorithm" error. * The version of OpenSSL being used is now accessible using the module - attributes :data:`ssl.OPENSSL_VERSION` (a string), - :data:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and - :data:`ssl.OPENSSL_VERSION_NUMBER` (an integer). + attributes :const:`ssl.OPENSSL_VERSION` (a string), + :const:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and + :const:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Contributed by Antoine Pitrou in :issue:`8850`, :issue:`1589`, :issue:`8322`, :issue:`5639`, :issue:`4870`, :issue:`8484`, and :issue:`8321`.) diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index f121652ba51cb..b3f4e9ac743fb 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -842,7 +842,7 @@ Builtin functions and types * :func:`open` gets a new *opener* parameter: the underlying file descriptor for the file object is then obtained by calling *opener* with (*file*, - *flags*). It can be used to use custom flags like :data:`os.O_CLOEXEC` for + *flags*). It can be used to use custom flags like :const:`os.O_CLOEXEC` for example. The ``'x'`` mode was added: open for exclusive creation, failing if the file already exists. * :func:`print`: added the *flush* keyword argument. If the *flush* keyword @@ -1127,7 +1127,7 @@ Features * If Python is compiled without threads, the C version automatically disables the expensive thread local context machinery. In this case, - the variable :data:`~decimal.HAVE_THREADS` is set to ``False``. + the variable :const:`~decimal.HAVE_THREADS` is set to ``False``. API changes ~~~~~~~~~~~ @@ -1576,8 +1576,8 @@ os -- * The :mod:`os` module has a new :func:`~os.pipe2` function that makes it - possible to create a pipe with :data:`~os.O_CLOEXEC` or - :data:`~os.O_NONBLOCK` flags set atomically. This is especially useful to + possible to create a pipe with :const:`~os.O_CLOEXEC` or + :const:`~os.O_NONBLOCK` flags set atomically. This is especially useful to avoid race conditions in multi-threaded programs. * The :mod:`os` module has a new :func:`~os.sendfile` function which provides @@ -1691,9 +1691,9 @@ os * Some platforms now support additional constants for the :func:`~os.lseek` function, such as ``os.SEEK_HOLE`` and ``os.SEEK_DATA``. -* New constants :data:`~os.RTLD_LAZY`, :data:`~os.RTLD_NOW`, - :data:`~os.RTLD_GLOBAL`, :data:`~os.RTLD_LOCAL`, :data:`~os.RTLD_NODELETE`, - :data:`~os.RTLD_NOLOAD`, and :data:`~os.RTLD_DEEPBIND` are available on +* New constants :const:`~os.RTLD_LAZY`, :const:`~os.RTLD_NOW`, + :const:`~os.RTLD_GLOBAL`, :const:`~os.RTLD_LOCAL`, :const:`~os.RTLD_NODELETE`, + :const:`~os.RTLD_NOLOAD`, and :const:`~os.RTLD_DEEPBIND` are available on platforms that support them. These are for use with the :func:`sys.setdlopenflags` function, and supersede the similar constants defined in :mod:`ctypes` and :mod:`DLFCN`. (Contributed by Victor Stinner @@ -1995,7 +1995,7 @@ subprocess Command strings can now be bytes objects on posix platforms. (Contributed by Victor Stinner in :issue:`8513`.) -A new constant :data:`~subprocess.DEVNULL` allows suppressing output in a +A new constant :const:`~subprocess.DEVNULL` allows suppressing output in a platform-independent fashion. (Contributed by Ross Lagerwall in :issue:`5870`.) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst index 45bb91833a352..d72163b249041 100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -775,7 +775,7 @@ of a given opcode and argument, information that is not otherwise available. doctest ------- -A new :ref:`option flag `, :data:`~doctest.FAIL_FAST`, halts +A new :ref:`option flag `, :const:`~doctest.FAIL_FAST`, halts test running as soon as the first failure is detected. (Contributed by R. David Murray and Daniel Urban in :issue:`16522`.) @@ -841,7 +841,7 @@ for example, if the file might have been changed and re-checked in less time than the resolution of a particular filesystem's file modification time field. (Contributed by Mark Levitt in :issue:`18149`.) -New module attribute :data:`~filecmp.DEFAULT_IGNORES` provides the list of +New module attribute :const:`~filecmp.DEFAULT_IGNORES` provides the list of directories that are used as the default value for the *ignore* parameter of the :func:`~filecmp.dircmp` function. (Contributed by Eli Bendersky in :issue:`15442`.) @@ -1189,7 +1189,7 @@ Windows). (Contributed by Brian Curtin in :issue:`11939`.) root on Windows. (Contributed by Tim Golden in :issue:`9035`.) :func:`os.open` supports two new flags on platforms that provide them, -:data:`~os.O_PATH` (un-opened file descriptor), and :data:`~os.O_TMPFILE` +:const:`~os.O_PATH` (un-opened file descriptor), and :const:`~os.O_TMPFILE` (unnamed temporary file; as of 3.4.0 release available only on Linux systems with a kernel version of 3.11 or newer that have uapi headers). (Contributed by Christian Heimes in :issue:`18673` and Benjamin Peterson, respectively.) @@ -1238,8 +1238,8 @@ plistlib stdlib serialization protocols, with new :func:`~plistlib.load`, :func:`~plistlib.dump`, :func:`~plistlib.loads`, and :func:`~plistlib.dumps` functions. (The older API is now deprecated.) In addition to the already -supported XML plist format (:data:`~plistlib.FMT_XML`), it also now supports -the binary plist format (:data:`~plistlib.FMT_BINARY`). (Contributed by Ronald +supported XML plist format (:const:`~plistlib.FMT_XML`), it also now supports +the binary plist format (:const:`~plistlib.FMT_BINARY`). (Contributed by Ronald Oussoren and others in :issue:`14455`.) @@ -1388,7 +1388,7 @@ try/except statement by code that only cares whether or not an error occurred. socket ------ -The socket module now supports the :data:`~socket.CAN_BCM` protocol on +The socket module now supports the :const:`~socket.CAN_BCM` protocol on platforms that support it. (Contributed by Brian Thorne in :issue:`15359`.) Socket objects have new methods to get or set their :ref:`inheritable flag @@ -1399,7 +1399,7 @@ The ``socket.AF_*`` and ``socket.SOCK_*`` constants are now enumeration values using the new :mod:`enum` module. This allows meaningful names to be printed during debugging, instead of integer "magic numbers". -The :data:`~socket.AF_LINK` constant is now available on BSD and OSX. +The :const:`~socket.AF_LINK` constant is now available on BSD and OSX. :func:`~socket.inet_pton` and :func:`~socket.inet_ntop` are now supported on Windows. (Contributed by Atsuo Ishimoto in :issue:`7171`.) @@ -1460,8 +1460,8 @@ Heimes in :issue:`18147`.) If OpenSSL 0.9.8 or later is available, :class:`~ssl.SSLContext` has a new attribute :attr:`~ssl.SSLContext.verify_flags` that can be used to control the certificate verification process by setting it to some combination of the new -constants :data:`~ssl.VERIFY_DEFAULT`, :data:`~ssl.VERIFY_CRL_CHECK_LEAF`, -:data:`~ssl.VERIFY_CRL_CHECK_CHAIN`, or :data:`~ssl.VERIFY_X509_STRICT`. +constants :const:`~ssl.VERIFY_DEFAULT`, :const:`~ssl.VERIFY_CRL_CHECK_LEAF`, +:const:`~ssl.VERIFY_CRL_CHECK_CHAIN`, or :const:`~ssl.VERIFY_X509_STRICT`. OpenSSL does not do any CRL verification by default. (Contributed by Christien Heimes in :issue:`8813`.) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 14b6425cea699..86c2905e5fad9 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -478,7 +478,7 @@ not make an additional system call:: PEP 475: Retry system calls failing with EINTR ---------------------------------------------- -An :py:data:`errno.EINTR` error code is returned whenever a system call, that +An :py:const:`errno.EINTR` error code is returned whenever a system call, that is waiting for I/O, is interrupted by a signal. Previously, Python would raise :exc:`InterruptedError` in such cases. This meant that, when writing a Python application, the developer had two choices: @@ -527,7 +527,7 @@ by a signal: :func:`~os.writev`; * special cases: :func:`os.close` and :func:`os.dup2` now ignore - :py:data:`~errno.EINTR` errors; the syscall is not retried (see the PEP + :py:const:`~errno.EINTR` errors; the syscall is not retried (see the PEP for the rationale); * :mod:`select` functions: :func:`devpoll.poll() `, @@ -1498,7 +1498,7 @@ use ``/dev/urandom`` and avoiding failures due to potential file descriptor exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow -getting and setting a file descriptor's blocking mode (:data:`~os.O_NONBLOCK`.) +getting and setting a file descriptor's blocking mode (:const:`~os.O_NONBLOCK`.) (Contributed by Victor Stinner in :issue:`22054`.) The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported @@ -1783,7 +1783,7 @@ the TLS handshake. The new :meth:`SSLSocket.selected_alpn_protocol() ` returns the protocol that was selected during the TLS handshake. -The :data:`~ssl.HAS_ALPN` flag indicates whether ALPN support is present. +The :const:`~ssl.HAS_ALPN` flag indicates whether ALPN support is present. Other Changes @@ -2476,7 +2476,7 @@ Changes in the Python API in Python 3.5, all old ``.pyo`` files from previous versions of Python are invalid regardless of this PEP. -* The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES` +* The :mod:`socket` module now exports the :const:`~socket.CAN_RAW_FD_FRAMES` constant on linux 3.6 and greater. * The :func:`ssl.cert_time_to_seconds` function now interprets the input time diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 07a65d264c435..84b043856ed01 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1404,7 +1404,7 @@ socket ------ The :func:`~socket.socket.ioctl` function now supports the -:data:`~socket.SIO_LOOPBACK_FAST_PATH` control code. +:const:`~socket.SIO_LOOPBACK_FAST_PATH` control code. (Contributed by Daniel Stokes in :issue:`26536`.) The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``, @@ -1416,7 +1416,7 @@ The :meth:`~socket.socket.setsockopt` now supports the (Contributed by Christian Heimes in :issue:`27744`.) The socket module now supports the address family -:data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, +:const:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, ``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added. (Contributed by Christian Heimes in :issue:`27744` with support from Victor Stinner.) diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 28f22836d8d09..eda8bf0137bff 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1280,13 +1280,13 @@ This function should be used instead of :func:`os.close` for better compatibility across platforms. (Contributed by Christian Heimes in :issue:`32454`.) -The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION` -(Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and -:data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. +The :mod:`socket` module now exposes the :const:`socket.TCP_CONGESTION` +(Linux 2.6.13), :const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and +:const:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. (Contributed by Omar Sandoval in :issue:`26273` and Nathaniel J. Smith in :issue:`29728`.) -Support for :data:`socket.AF_VSOCK` sockets has been added to allow +Support for :const:`socket.AF_VSOCK` sockets has been added to allow communication between virtual machines and their hosts. (Contributed by Cathy Avery in :issue:`27584`.) @@ -1394,7 +1394,7 @@ subprocess The :func:`subprocess.run` function accepts the new *capture_output* keyword argument. When true, stdout and stderr will be captured. -This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and +This is equivalent to passing :const:`subprocess.PIPE` as *stdout* and *stderr* arguments. (Contributed by Bo Bayles in :issue:`32102`.) @@ -1453,12 +1453,12 @@ time New clock identifiers have been added: -* :data:`time.CLOCK_BOOTTIME` (Linux): Identical to - :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the +* :const:`time.CLOCK_BOOTTIME` (Linux): Identical to + :const:`time.CLOCK_MONOTONIC`, except it also includes any time that the system is suspended. -* :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution +* :const:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution per-process CPU timer. -* :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is +* :const:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is the time the system has been running and not suspended, providing accurate uptime measurement. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 61249d479fa04..ba3d86118b926 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1305,7 +1305,7 @@ Zackery Spytz in :issue:`25451`.) time ---- -Added new clock :data:`~time.CLOCK_UPTIME_RAW` for macOS 10.12. +Added new clock :const:`~time.CLOCK_UPTIME_RAW` for macOS 10.12. (Contributed by Joannah Nanjekye in :issue:`35702`.) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 49c8bd2f3e07d..36d942e8c064f 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -427,8 +427,8 @@ digests. It skips MD5 on platforms that block MD5 digest. fcntl ----- -Added constants :data:`~fcntl.F_OFD_GETLK`, :data:`~fcntl.F_OFD_SETLK` -and :data:`~fcntl.F_OFD_SETLKW`. +Added constants :const:`~fcntl.F_OFD_GETLK`, :const:`~fcntl.F_OFD_SETLK` +and :const:`~fcntl.F_OFD_SETLKW`. (Contributed by Dong-hee Na in :issue:`38602`.) ftplib @@ -593,11 +593,11 @@ a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.) os -- -Added :data:`~os.CLD_KILLED` and :data:`~os.CLD_STOPPED` for :attr:`si_code`. +Added :const:`~os.CLD_KILLED` and :const:`~os.CLD_STOPPED` for :attr:`si_code`. (Contributed by Dong-hee Na in :issue:`38493`.) Exposed the Linux-specific :func:`os.pidfd_open` (:issue:`38692`) and -:data:`os.P_PIDFD` (:issue:`38713`) for process management with file +:const:`os.P_PIDFD` (:issue:`38713`) for process management with file descriptors. The :func:`os.unsetenv` function is now also available on Windows. @@ -669,11 +669,11 @@ a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.) socket ------ -The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_JOIN_FILTERS` +The :mod:`socket` module now exports the :const:`~socket.CAN_RAW_JOIN_FILTERS` constant on Linux 4.1 and greater. (Contributed by Stefan Tatschner and Zackery Spytz in :issue:`25780`.) -The socket module now supports the :data:`~socket.CAN_J1939` protocol on +The socket module now supports the :const:`~socket.CAN_J1939` protocol on platforms that support it. (Contributed by Karl Ding in :issue:`40291`.) The socket module now has the :func:`socket.send_fds` and @@ -1084,7 +1084,7 @@ Changes in the Python API ``__VENV_PROMPT__`` is set to ``""``. * The :meth:`select.epoll.unregister` method no longer ignores the - :data:`~errno.EBADF` error. + :const:`~errno.EBADF` error. (Contributed by Victor Stinner in :issue:`39239`.) * The *compresslevel* parameter of :class:`bz2.BZ2File` became keyword-only, diff --git a/Misc/NEWS.d/3.10.0a1.rst b/Misc/NEWS.d/3.10.0a1.rst index 471811662644b..f7eb8036bc5cf 100644 --- a/Misc/NEWS.d/3.10.0a1.rst +++ b/Misc/NEWS.d/3.10.0a1.rst @@ -2176,7 +2176,7 @@ None. .. nonce: YoYoYo .. section: Library -Add a new :data:`os.RWF_APPEND` flag for :func:`os.pwritev`. +Add a new :const:`os.RWF_APPEND` flag for :func:`os.pwritev`. .. @@ -2304,7 +2304,7 @@ Restored the deprecated :mod:`xml.etree.cElementTree` module. .. nonce: ZCk0_c .. section: Library -:data:`~mmap.MAP_POPULATE` constant has now been added to the list of +:const:`~mmap.MAP_POPULATE` constant has now been added to the list of exported :mod:`mmap` module flags. .. diff --git a/Misc/NEWS.d/3.10.0a2.rst b/Misc/NEWS.d/3.10.0a2.rst index 061a82e90afd6..eeb179a980bb8 100644 --- a/Misc/NEWS.d/3.10.0a2.rst +++ b/Misc/NEWS.d/3.10.0a2.rst @@ -604,7 +604,7 @@ changes the working directory. PR by Anthony Sottile. .. nonce: 9wXTtY .. section: Library -The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default +The :mod:`shelve` module now uses :const:`pickle.DEFAULT_PROTOCOL` by default instead of :mod:`pickle` protocol ``3``. .. diff --git a/Misc/NEWS.d/3.10.0a6.rst b/Misc/NEWS.d/3.10.0a6.rst index 803df6f51ce62..313aa68925404 100644 --- a/Misc/NEWS.d/3.10.0a6.rst +++ b/Misc/NEWS.d/3.10.0a6.rst @@ -294,8 +294,8 @@ actual dictionary. This created problems for introspection tools. .. nonce: SwcSuU .. section: Library -Added :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` and -:data:`~os.O_NOFOLLOW_ANY` for macOS. Patch by Dong-hee Na. +Added :const:`~os.O_EVTONLY`, :const:`~os.O_FSYNC`, :const:`~os.O_SYMLINK` and +:const:`~os.O_NOFOLLOW_ANY` for macOS. Patch by Dong-hee Na. .. @@ -304,7 +304,7 @@ Added :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` and .. nonce: a7Dote .. section: Library -Adds :data:`resource.RLIMIT_KQUEUES` constant from FreeBSD to the +Adds :const:`resource.RLIMIT_KQUEUES` constant from FreeBSD to the :mod:`resource` module. .. diff --git a/Misc/NEWS.d/3.10.0a7.rst b/Misc/NEWS.d/3.10.0a7.rst index 286d0a8a7e919..ff01ee645c0a9 100644 --- a/Misc/NEWS.d/3.10.0a7.rst +++ b/Misc/NEWS.d/3.10.0a7.rst @@ -713,7 +713,7 @@ this situation. Also ensures that the :func:`tempfile.gettempdir()` and .. section: Library Expose ``X509_V_FLAG_ALLOW_PROXY_CERTS`` as -:data:`~ssl.VERIFY_ALLOW_PROXY_CERTS` to allow proxy certificate validation +:const:`~ssl.VERIFY_ALLOW_PROXY_CERTS` to allow proxy certificate validation as explained in https://www.openssl.org/docs/man1.1.1/man7/proxy-certificates.html. diff --git a/Misc/NEWS.d/3.10.0b1.rst b/Misc/NEWS.d/3.10.0b1.rst index 38e0af17a2c66..3c71bc73b812a 100644 --- a/Misc/NEWS.d/3.10.0b1.rst +++ b/Misc/NEWS.d/3.10.0b1.rst @@ -871,7 +871,7 @@ assert_called_once_with) will unconditionally pass. .. nonce: -1XPDH .. section: Library -Add :data:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL 3.0.0) +Add :const:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL 3.0.0) .. diff --git a/Misc/NEWS.d/3.11.0a1.rst b/Misc/NEWS.d/3.11.0a1.rst index fb60ac5560102..a9ff21149a87d 100644 --- a/Misc/NEWS.d/3.11.0a1.rst +++ b/Misc/NEWS.d/3.11.0a1.rst @@ -1468,8 +1468,8 @@ an installed expat library <= 2.2.0. On Unix, if the ``sem_clockwait()`` function is available in the C library (glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses -the monotonic clock (:data:`time.CLOCK_MONOTONIC`) for the timeout, rather -than using the system clock (:data:`time.CLOCK_REALTIME`), to not be +the monotonic clock (:const:`time.CLOCK_MONOTONIC`) for the timeout, rather +than using the system clock (:const:`time.CLOCK_REALTIME`), to not be affected by system clock changes. Patch by Victor Stinner. .. @@ -2087,8 +2087,8 @@ Upgrade bundled pip to 21.2.3 and setuptools to 57.4.0 .. section: Library Fix the :func:`os.set_inheritable` function on FreeBSD 14 for file -descriptor opened with the :data:`~os.O_PATH` flag: ignore the -:data:`~errno.EBADF` error on ``ioctl()``, fallback on the ``fcntl()`` +descriptor opened with the :const:`~os.O_PATH` flag: ignore the +:const:`~errno.EBADF` error on ``ioctl()``, fallback on the ``fcntl()`` implementation. Patch by Victor Stinner. .. diff --git a/Misc/NEWS.d/3.11.0a4.rst b/Misc/NEWS.d/3.11.0a4.rst index bcb6e8b7bdde3..3dd335929d655 100644 --- a/Misc/NEWS.d/3.11.0a4.rst +++ b/Misc/NEWS.d/3.11.0a4.rst @@ -839,7 +839,7 @@ patch by Kumar Aditya. .. nonce: jeiPiX .. section: Library -Added :data:`signal.SIGSTKFLT` on platforms where this signal is defined. +Added :const:`signal.SIGSTKFLT` on platforms where this signal is defined. .. diff --git a/Misc/NEWS.d/3.11.0b1.rst b/Misc/NEWS.d/3.11.0b1.rst index d8c2ec0a79971..ad52bd14dbfd1 100644 --- a/Misc/NEWS.d/3.11.0b1.rst +++ b/Misc/NEWS.d/3.11.0b1.rst @@ -817,8 +817,8 @@ it is ever needed and document the existing mechanism for ``posix_spawn()``. .. nonce: HFtERN .. section: Library -Fix :data:`signal.NSIG` value on FreeBSD to accept signal numbers greater -than 32, like :data:`signal.SIGRTMIN` and :data:`signal.SIGRTMAX`. Patch by +Fix :const:`signal.NSIG` value on FreeBSD to accept signal numbers greater +than 32, like :const:`signal.SIGRTMIN` and :const:`signal.SIGRTMAX`. Patch by Victor Stinner. .. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index 9afeba2526351..5b6aff4d51fd6 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -736,7 +736,7 @@ new types. .. nonce: 6eoc8k .. section: Core and Builtins -On WASI :data:`~errno.ENOTCAPABLE` is now mapped to :exc:`PermissionError`. +On WASI :const:`~errno.ENOTCAPABLE` is now mapped to :exc:`PermissionError`. The :mod:`errno` modules exposes the new error number. ``getpath.py`` now ignores :exc:`PermissionError` when it cannot open landmark files ``pybuilddir.txt`` and ``pyenv.cfg``. @@ -2649,7 +2649,7 @@ calling any callbacks. Patch by Kumar Aditya. .. nonce: i807-g .. section: Library -Fail gracefully if :data:`~errno.EPERM` or :data:`~errno.ENOSYS` is raised +Fail gracefully if :const:`~errno.EPERM` or :const:`~errno.ENOSYS` is raised when loading :mod:`crypt` methods. This may happen when trying to load ``MD5`` on a Linux kernel with :abbr:`FIPS (Federal Information Processing Standard)` enabled. @@ -2698,8 +2698,8 @@ Upgrade bundled pip to 22.2. .. nonce: VT34A5 .. section: Library -Fix check for existence of :data:`os.EFD_CLOEXEC`, :data:`os.EFD_NONBLOCK` -and :data:`os.EFD_SEMAPHORE` flags on older kernel versions where these +Fix check for existence of :const:`os.EFD_CLOEXEC`, :const:`os.EFD_NONBLOCK` +and :const:`os.EFD_SEMAPHORE` flags on older kernel versions where these flags are not present. Patch by Kumar Aditya. .. @@ -3553,7 +3553,7 @@ Make :class:`multiprocessing.Pool` raise an exception if .. nonce: HY0Uzj .. section: Library -Add :data:`os.PIDFD_NONBLOCK` flag to open a file descriptor for a process +Add :const:`os.PIDFD_NONBLOCK` flag to open a file descriptor for a process with :func:`os.pidfd_open` in non-blocking mode. Patch by Kumar Aditya. .. diff --git a/Misc/NEWS.d/3.12.0a2.rst b/Misc/NEWS.d/3.12.0a2.rst index 5a3ccab02016f..f781e38665a8e 100644 --- a/Misc/NEWS.d/3.12.0a2.rst +++ b/Misc/NEWS.d/3.12.0a2.rst @@ -397,7 +397,7 @@ longobject.c to speed up some operations. .. nonce: nSGEkG .. section: Core and Builtins -Expose :data:`~socket.ETH_P_ALL` and some of the :ref:`ETHERTYPE_* constants +Expose :const:`~socket.ETH_P_ALL` and some of the :ref:`ETHERTYPE_* constants ` in :mod:`socket`. Patch by Noam Cohen. .. diff --git a/Misc/NEWS.d/3.12.0a3.rst b/Misc/NEWS.d/3.12.0a3.rst index 3d1e43350d136..3e6f8de5d911f 100644 --- a/Misc/NEWS.d/3.12.0a3.rst +++ b/Misc/NEWS.d/3.12.0a3.rst @@ -505,7 +505,7 @@ return True from this method; now they correctly return False. .. nonce: ZoOY5G .. section: Library -Add an :data:`~ssl.OP_ENABLE_KTLS` option for enabling the use of the kernel +Add an :const:`~ssl.OP_ENABLE_KTLS` option for enabling the use of the kernel TLS (kTLS). Patch by Illia Volochii. .. diff --git a/Misc/NEWS.d/3.12.0a4.rst b/Misc/NEWS.d/3.12.0a4.rst index dd26d4d964d6b..8951490f41b94 100644 --- a/Misc/NEWS.d/3.12.0a4.rst +++ b/Misc/NEWS.d/3.12.0a4.rst @@ -317,7 +317,7 @@ Improve performance of ``list.pop`` for small lists. .. nonce: yP4Na0 .. section: Core and Builtins -Add :data:`ssl.OP_LEGACY_SERVER_CONNECT` +Add :const:`ssl.OP_LEGACY_SERVER_CONNECT` .. @@ -356,7 +356,7 @@ arrays. .. nonce: mHRdQn .. section: Library -Add :data:`socket.IP_PKTINFO` constant. +Add :const:`socket.IP_PKTINFO` constant. .. diff --git a/Misc/NEWS.d/3.12.0a6.rst b/Misc/NEWS.d/3.12.0a6.rst index f6beb5b7ec3db..07967028bdee7 100644 --- a/Misc/NEWS.d/3.12.0a6.rst +++ b/Misc/NEWS.d/3.12.0a6.rst @@ -303,7 +303,7 @@ Kim. .. nonce: Vxz0Mr .. section: Library -Add :data:`mmap.MAP_ALIGNED_SUPER` FreeBSD and :data:`mmap.MAP_CONCEAL` +Add :const:`mmap.MAP_ALIGNED_SUPER` FreeBSD and :const:`mmap.MAP_CONCEAL` OpenBSD constants to :mod:`mmap`. Patch by Yeojin Kim. .. diff --git a/Misc/NEWS.d/3.12.0a7.rst b/Misc/NEWS.d/3.12.0a7.rst index 8f078e50823a0..1ef8174755885 100644 --- a/Misc/NEWS.d/3.12.0a7.rst +++ b/Misc/NEWS.d/3.12.0a7.rst @@ -605,7 +605,7 @@ reported unauthenticated EOFs (i.e. without close_notify) as a clean TLS-level EOF. It now raises :exc:`~ssl.SSLEOFError`, matching the behavior in previous versions of OpenSSL. The :attr:`~ssl.SSLContext.options` attribute on :class:`~ssl.SSLContext` also no longer includes -:data:`~ssl.OP_IGNORE_UNEXPECTED_EOF` by default. This option may be set to +:const:`~ssl.OP_IGNORE_UNEXPECTED_EOF` by default. This option may be set to specify the previous OpenSSL 3.0 behavior. .. diff --git a/Misc/NEWS.d/3.12.0b1.rst b/Misc/NEWS.d/3.12.0b1.rst index 3de3b703e9f4f..8cd88e9b0f55e 100644 --- a/Misc/NEWS.d/3.12.0b1.rst +++ b/Misc/NEWS.d/3.12.0b1.rst @@ -842,7 +842,7 @@ filesystem case. .. section: Library Improve performance of :meth:`pathlib.Path.glob` by using -:data:`re.IGNORECASE` to implement case-insensitive matching. +:const:`re.IGNORECASE` to implement case-insensitive matching. .. @@ -1882,7 +1882,7 @@ both cases. .. nonce: 564i32 .. section: Library -Add :data:`~csv.QUOTE_STRINGS` and :data:`~csv.QUOTE_NOTNULL` to the suite +Add :const:`~csv.QUOTE_STRINGS` and :const:`~csv.QUOTE_NOTNULL` to the suite of :mod:`csv` module quoting styles. .. diff --git a/Misc/NEWS.d/3.6.0rc1.rst b/Misc/NEWS.d/3.6.0rc1.rst index 15769f950db23..658f8c902d870 100644 --- a/Misc/NEWS.d/3.6.0rc1.rst +++ b/Misc/NEWS.d/3.6.0rc1.rst @@ -69,8 +69,8 @@ supported. .. nonce: ilNIWN .. section: Library -Add new :data:`socket.TCP_CONGESTION` (Linux 2.6.13) and -:data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by +Add new :const:`socket.TCP_CONGESTION` (Linux 2.6.13) and +:const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by Omar Sandoval. .. diff --git a/Misc/NEWS.d/3.7.0a1.rst b/Misc/NEWS.d/3.7.0a1.rst index ef93454784b77..712558bf98d01 100644 --- a/Misc/NEWS.d/3.7.0a1.rst +++ b/Misc/NEWS.d/3.7.0a1.rst @@ -3274,7 +3274,7 @@ Added support for bytes paths in os.fwalk(). .. nonce: 37jMwb .. section: Library -Add new :data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constant. Patch by +Add new :const:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constant. Patch by Nathaniel J. Smith. .. @@ -3871,8 +3871,8 @@ as an integer. Function only available on Android. .. nonce: ilNIWN .. section: Library -Add new :data:`socket.TCP_CONGESTION` (Linux 2.6.13) and -:data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by +Add new :const:`socket.TCP_CONGESTION` (Linux 2.6.13) and +:const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by Omar Sandoval. .. diff --git a/Misc/NEWS.d/3.7.0a3.rst b/Misc/NEWS.d/3.7.0a3.rst index 6576c1fadbff6..92b0f32885120 100644 --- a/Misc/NEWS.d/3.7.0a3.rst +++ b/Misc/NEWS.d/3.7.0a3.rst @@ -754,8 +754,8 @@ now accepts characters as arguments. Based on patch by Steve Fink. .. nonce: DYQL0g .. section: Library -Add 3 new clock identifiers: :data:`time.CLOCK_BOOTTIME`, -:data:`time.CLOCK_PROF` and :data:`time.CLOCK_UPTIME`. +Add 3 new clock identifiers: :const:`time.CLOCK_BOOTTIME`, +:const:`time.CLOCK_PROF` and :const:`time.CLOCK_UPTIME`. .. diff --git a/Misc/NEWS.d/3.8.0a1.rst b/Misc/NEWS.d/3.8.0a1.rst index 854458f2d1a99..a3bf6ff3c54b4 100644 --- a/Misc/NEWS.d/3.8.0a1.rst +++ b/Misc/NEWS.d/3.8.0a1.rst @@ -1934,7 +1934,7 @@ failure. .. nonce: _ct_0H .. section: Library -The :data:`time.CLOCK_UPTIME_RAW` constant is now available for macOS 10.12. +The :const:`time.CLOCK_UPTIME_RAW` constant is now available for macOS 10.12. .. diff --git a/Misc/NEWS.d/3.8.0a4.rst b/Misc/NEWS.d/3.8.0a4.rst index 9841195210c9e..efe7b4d2ff0dd 100644 --- a/Misc/NEWS.d/3.8.0a4.rst +++ b/Misc/NEWS.d/3.8.0a4.rst @@ -955,7 +955,7 @@ Add a new :mod:`_testinternalcapi` module to test the internal C API. .. section: Tests Fix ``test_imap4_host_default_value()`` of ``test_imaplib``: catch also -:data:`errno.ENETUNREACH` error. +:const:`errno.ENETUNREACH` error. .. diff --git a/Misc/NEWS.d/3.9.0a1.rst b/Misc/NEWS.d/3.9.0a1.rst index 7af7f2e412053..5e6fa6759d577 100644 --- a/Misc/NEWS.d/3.9.0a1.rst +++ b/Misc/NEWS.d/3.9.0a1.rst @@ -1164,7 +1164,7 @@ defines them with eponymous methods. .. nonce: bmhquU .. section: Library -Add :data:`os.P_PIDFD` constant, which may be passed to :func:`os.waitid` to +Add :const:`os.P_PIDFD` constant, which may be passed to :func:`os.waitid` to wait on a Linux process file descriptor. .. @@ -1193,8 +1193,8 @@ Expose the Linux ``pidfd_open`` syscall as :func:`os.pidfd_open`. .. nonce: 7jvYFA .. section: Library -Added constants :data:`~fcntl.F_OFD_GETLK`, :data:`~fcntl.F_OFD_SETLK` and -:data:`~fcntl.F_OFD_SETLKW` to the :mod:`fcntl` module. Patch by Dong-hee +Added constants :const:`~fcntl.F_OFD_GETLK`, :const:`~fcntl.F_OFD_SETLK` and +:const:`~fcntl.F_OFD_SETLKW` to the :mod:`fcntl` module. Patch by Dong-hee Na. .. @@ -1283,7 +1283,7 @@ Fixed erroneous equality comparison in statistics.NormalDist(). .. nonce: 86ExWB .. section: Library -Added :data:`~os.CLD_KILLED` and :data:`~os.CLD_STOPPED` for +Added :const:`~os.CLD_KILLED` and :const:`~os.CLD_STOPPED` for :attr:`si_code`. Patch by Dong-hee Na. .. @@ -1355,8 +1355,8 @@ objects, patch by Samuel Colvin. .. nonce: 9w-IGF .. section: Library -Add missing :data:`stat.S_IFDOOR`, :data:`stat.S_IFPORT`, -:data:`stat.S_IFWHT`, :func:`stat.S_ISDOOR`, :func:`stat.S_ISPORT`, and +Add missing :const:`stat.S_IFDOOR`, :const:`stat.S_IFPORT`, +:const:`stat.S_IFWHT`, :func:`stat.S_ISDOOR`, :func:`stat.S_ISPORT`, and :func:`stat.S_ISWHT` values to the Python implementation of :mod:`stat`. .. @@ -4983,7 +4983,7 @@ set to CP_UTF7 or CP_UTF8. .. nonce: -0g2O3 .. section: Windows -Make :data:`winreg.REG_MULTI_SZ` support zero-length strings. +Make :const:`winreg.REG_MULTI_SZ` support zero-length strings. .. diff --git a/Misc/NEWS.d/3.9.0a6.rst b/Misc/NEWS.d/3.9.0a6.rst index af2cc7c3e9788..fec792a998bf9 100644 --- a/Misc/NEWS.d/3.9.0a6.rst +++ b/Misc/NEWS.d/3.9.0a6.rst @@ -680,7 +680,7 @@ child process, reset the lock to the unlocked state. Rename also the private .. nonce: kIjVge .. section: Library -Expose :data:`~socket.CAN_RAW_JOIN_FILTERS` in the :mod:`socket` module. +Expose :const:`~socket.CAN_RAW_JOIN_FILTERS` in the :mod:`socket` module. .. @@ -735,7 +735,7 @@ number of groups. For other implementations, double the group list size. .. nonce: HFpHZS .. section: Library -Add :data:`time.CLOCK_TAI` constant if the operating system support it. +Add :const:`time.CLOCK_TAI` constant if the operating system support it. .. From webhook-mailer at python.org Fri Jul 21 07:50:01 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 11:50:01 -0000 Subject: [Python-checkins] [3.11] gh-106909: Use role :const: for referencing module constants (GH-106910) (GH-106957) Message-ID: https://github.com/python/cpython/commit/b338ac75405be26b1df6c35e1360a23f0e88f591 commit: b338ac75405be26b1df6c35e1360a23f0e88f591 branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T14:49:57+03:00 summary: [3.11] gh-106909: Use role :const: for referencing module constants (GH-106910) (GH-106957) (cherry picked from commit 4b9948617f91175783609769aa6160e5b49b9ccc) files: M Doc/c-api/exceptions.rst M Doc/faq/library.rst M Doc/install/index.rst M Doc/library/_thread.rst M Doc/library/asyncio-dev.rst M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-platforms.rst M Doc/library/asyncio-subprocess.rst M Doc/library/exceptions.rst M Doc/library/fcntl.rst M Doc/library/ftplib.rst M Doc/library/http.client.rst M Doc/library/imaplib.rst M Doc/library/io.rst M Doc/library/multiprocessing.rst M Doc/library/optparse.rst M Doc/library/os.rst M Doc/library/poplib.rst M Doc/library/shelve.rst M Doc/library/smtplib.rst M Doc/library/socket.rst M Doc/library/sqlite3.rst M Doc/library/ssl.rst M Doc/library/sys.rst M Doc/library/tempfile.rst M Doc/library/test.rst M Doc/library/unittest.mock.rst M Doc/library/urllib.request.rst M Doc/library/xml.rst M Doc/using/configure.rst M Doc/whatsnew/2.7.rst M Doc/whatsnew/3.10.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.2.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.4.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.7.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a1.rst M Misc/NEWS.d/3.10.0a2.rst M Misc/NEWS.d/3.10.0a6.rst M Misc/NEWS.d/3.10.0a7.rst M Misc/NEWS.d/3.10.0b1.rst M Misc/NEWS.d/3.11.0a1.rst M Misc/NEWS.d/3.11.0a4.rst M Misc/NEWS.d/3.11.0b1.rst M Misc/NEWS.d/3.6.0rc1.rst M Misc/NEWS.d/3.7.0a1.rst M Misc/NEWS.d/3.7.0a3.rst M Misc/NEWS.d/3.8.0a1.rst M Misc/NEWS.d/3.8.0a4.rst M Misc/NEWS.d/3.9.0a1.rst M Misc/NEWS.d/3.9.0a6.rst diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 76d32fd1763f5..5dad596f6197f 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -602,7 +602,7 @@ Signal Handling to interrupt an operation). If the given signal isn't handled by Python (it was set to - :data:`signal.SIG_DFL` or :data:`signal.SIG_IGN`), it will be ignored. + :py:const:`signal.SIG_DFL` or :py:const:`signal.SIG_IGN`), it will be ignored. If *signum* is outside of the allowed range of signal numbers, ``-1`` is returned. Otherwise, ``0`` is returned. The error indicator is diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst index 597caaa778e1c..b43c7505c0401 100644 --- a/Doc/faq/library.rst +++ b/Doc/faq/library.rst @@ -566,7 +566,7 @@ use ``p.read(n)``. Note on a bug in popen2: unless your program calls ``wait()`` or ``waitpid()``, finished child processes are never removed, and eventually calls to popen2 will fail because of a limit on the number of child - processes. Calling :func:`os.waitpid` with the :data:`os.WNOHANG` option can + processes. Calling :func:`os.waitpid` with the :const:`os.WNOHANG` option can prevent this; a good place to insert such a call would be before calling ``popen2`` again. diff --git a/Doc/install/index.rst b/Doc/install/index.rst index b9588df611c59..6c401c4c13d1c 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -309,9 +309,9 @@ install into it. It is enabled with a simple option:: python setup.py install --user -Files will be installed into subdirectories of :data:`site.USER_BASE` (written +Files will be installed into subdirectories of :const:`site.USER_BASE` (written as :file:`{userbase}` hereafter). This scheme installs pure Python modules and -extension modules in the same location (also known as :data:`site.USER_SITE`). +extension modules in the same location (also known as :const:`site.USER_SITE`). Here are the values for UNIX, including macOS: =============== =========================================================== diff --git a/Doc/library/_thread.rst b/Doc/library/_thread.rst index 3d7e354d1e368..21f7847d679af 100644 --- a/Doc/library/_thread.rst +++ b/Doc/library/_thread.rst @@ -68,10 +68,10 @@ This module defines the following constants and functions: there is no guarantee that the interruption will happen immediately. If given, *signum* is the number of the signal to simulate. - If *signum* is not given, :data:`signal.SIGINT` is simulated. + If *signum* is not given, :const:`signal.SIGINT` is simulated. If the given signal isn't handled by Python (it was set to - :data:`signal.SIG_DFL` or :data:`signal.SIG_IGN`), this function does + :const:`signal.SIG_DFL` or :const:`signal.SIG_IGN`), this function does nothing. .. versionchanged:: 3.10 diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index 921a394a59fec..c7d97008fb490 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -34,7 +34,7 @@ There are several ways to enable asyncio debug mode: In addition to enabling the debug mode, consider also: * setting the log level of the :ref:`asyncio logger ` to - :py:data:`logging.DEBUG`, for example the following snippet of code + :py:const:`logging.DEBUG`, for example the following snippet of code can be run at startup of the application:: logging.basicConfig(level=logging.DEBUG) @@ -142,7 +142,7 @@ Logging asyncio uses the :mod:`logging` module and all logging is performed via the ``"asyncio"`` logger. -The default log level is :py:data:`logging.INFO`, which can be easily +The default log level is :py:const:`logging.INFO`, which can be easily adjusted:: logging.getLogger("asyncio").setLevel(logging.WARNING) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 83fb27060b214..07be0d0eea088 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -397,11 +397,11 @@ Opening network connections Open a streaming transport connection to a given address specified by *host* and *port*. - The socket family can be either :py:data:`~socket.AF_INET` or - :py:data:`~socket.AF_INET6` depending on *host* (or the *family* + The socket family can be either :py:const:`~socket.AF_INET` or + :py:const:`~socket.AF_INET6` depending on *host* (or the *family* argument, if provided). - The socket type will be :py:data:`~socket.SOCK_STREAM`. + The socket type will be :py:const:`~socket.SOCK_STREAM`. *protocol_factory* must be a callable returning an :ref:`asyncio protocol ` implementation. @@ -495,7 +495,7 @@ Opening network connections .. versionchanged:: 3.6 - The socket option :py:data:`~socket.TCP_NODELAY` is set by default + The socket option :py:const:`~socket.TCP_NODELAY` is set by default for all TCP connections. .. versionchanged:: 3.7 @@ -535,11 +535,11 @@ Opening network connections Create a datagram connection. - The socket family can be either :py:data:`~socket.AF_INET`, - :py:data:`~socket.AF_INET6`, or :py:data:`~socket.AF_UNIX`, + The socket family can be either :py:const:`~socket.AF_INET`, + :py:const:`~socket.AF_INET6`, or :py:const:`~socket.AF_UNIX`, depending on *host* (or the *family* argument, if provided). - The socket type will be :py:data:`~socket.SOCK_DGRAM`. + The socket type will be :py:const:`~socket.SOCK_DGRAM`. *protocol_factory* must be a callable returning a :ref:`protocol ` implementation. @@ -564,7 +564,7 @@ Opening network connections * *reuse_port* tells the kernel to allow this endpoint to be bound to the same port as other existing endpoints are bound to, so long as they all set this flag when being created. This option is not supported on Windows - and some Unixes. If the :py:data:`~socket.SO_REUSEPORT` constant is not + and some Unixes. If the :py:const:`~socket.SO_REUSEPORT` constant is not defined then this capability is unsupported. * *allow_broadcast* tells the kernel to allow this endpoint to send @@ -590,7 +590,7 @@ Opening network connections .. versionchanged:: 3.8.1 The *reuse_address* parameter is no longer supported, as using - :py:data:`~sockets.SO_REUSEADDR` poses a significant security concern for + :py:const:`~sockets.SO_REUSEADDR` poses a significant security concern for UDP. Explicitly passing ``reuse_address=True`` will raise an exception. When multiple processes with differing UIDs assign sockets to an @@ -599,7 +599,7 @@ Opening network connections For supported platforms, *reuse_port* can be used as a replacement for similar functionality. With *reuse_port*, - :py:data:`~sockets.SO_REUSEPORT` is used instead, which specifically + :py:const:`~sockets.SO_REUSEPORT` is used instead, which specifically prevents processes with differing UIDs from assigning sockets to the same socket address. @@ -617,8 +617,8 @@ Opening network connections Create a Unix connection. - The socket family will be :py:data:`~socket.AF_UNIX`; socket - type will be :py:data:`~socket.SOCK_STREAM`. + The socket family will be :py:const:`~socket.AF_UNIX`; socket + type will be :py:const:`~socket.SOCK_STREAM`. A tuple of ``(transport, protocol)`` is returned on success. @@ -654,7 +654,7 @@ Creating network servers ssl_shutdown_timeout=None, \ start_serving=True) - Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) listening + Create a TCP server (socket type :const:`~socket.SOCK_STREAM`) listening on *port* of the *host* address. Returns a :class:`Server` object. @@ -682,10 +682,10 @@ Creating network servers be selected (note that if *host* resolves to multiple network interfaces, a different random port will be selected for each interface). - * *family* can be set to either :data:`socket.AF_INET` or - :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. + * *family* can be set to either :const:`socket.AF_INET` or + :const:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set, the *family* will be determined from host name - (defaults to :data:`~socket.AF_UNSPEC`). + (defaults to :const:`~socket.AF_UNSPEC`). * *flags* is a bitmask for :meth:`getaddrinfo`. @@ -739,7 +739,7 @@ Creating network servers .. versionchanged:: 3.6 Added *ssl_handshake_timeout* and *start_serving* parameters. - The socket option :py:data:`~socket.TCP_NODELAY` is set by default + The socket option :py:const:`~socket.TCP_NODELAY` is set by default for all TCP connections. .. versionchanged:: 3.11 @@ -760,7 +760,7 @@ Creating network servers start_serving=True) Similar to :meth:`loop.create_server` but works with the - :py:data:`~socket.AF_UNIX` socket family. + :py:const:`~socket.AF_UNIX` socket family. *path* is the name of a Unix domain socket, and is required, unless a *sock* argument is provided. Abstract Unix sockets, diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst index 50ad8a2ab7032..19ec726c1be06 100644 --- a/Doc/library/asyncio-platforms.rst +++ b/Doc/library/asyncio-platforms.rst @@ -37,7 +37,7 @@ All event loops on Windows do not support the following methods: * :meth:`loop.create_unix_connection` and :meth:`loop.create_unix_server` are not supported. - The :data:`socket.AF_UNIX` socket family is specific to Unix. + The :const:`socket.AF_UNIX` socket family is specific to Unix. * :meth:`loop.add_signal_handler` and :meth:`loop.remove_signal_handler` are not supported. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 6e04817b1818d..77c3aae43099c 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -244,7 +244,7 @@ their completion. Stop the child process. - On POSIX systems this method sends :py:data:`signal.SIGTERM` to the + On POSIX systems this method sends :py:const:`signal.SIGTERM` to the child process. On Windows the Win32 API function :c:func:`TerminateProcess` is diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index d54c49fff6b62..dcfbc486eeb35 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -659,8 +659,8 @@ depending on the system error code. Raised when an operation would block on an object (e.g. socket) set for non-blocking operation. - Corresponds to :c:data:`errno` :py:data:`~errno.EAGAIN`, :py:data:`~errno.EALREADY`, - :py:data:`~errno.EWOULDBLOCK` and :py:data:`~errno.EINPROGRESS`. + Corresponds to :c:data:`errno` :py:const:`~errno.EAGAIN`, :py:const:`~errno.EALREADY`, + :py:const:`~errno.EWOULDBLOCK` and :py:const:`~errno.EINPROGRESS`. In addition to those of :exc:`OSError`, :exc:`BlockingIOError` can have one more attribute: @@ -674,7 +674,7 @@ depending on the system error code. .. exception:: ChildProcessError Raised when an operation on a child process failed. - Corresponds to :c:data:`errno` :py:data:`~errno.ECHILD`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECHILD`. .. exception:: ConnectionError @@ -688,40 +688,40 @@ depending on the system error code. A subclass of :exc:`ConnectionError`, raised when trying to write on a pipe while the other end has been closed, or trying to write on a socket which has been shutdown for writing. - Corresponds to :c:data:`errno` :py:data:`~errno.EPIPE` and :py:data:`~errno.ESHUTDOWN`. + Corresponds to :c:data:`errno` :py:const:`~errno.EPIPE` and :py:const:`~errno.ESHUTDOWN`. .. exception:: ConnectionAbortedError A subclass of :exc:`ConnectionError`, raised when a connection attempt is aborted by the peer. - Corresponds to :c:data:`errno` :py:data:`~errno.ECONNABORTED`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECONNABORTED`. .. exception:: ConnectionRefusedError A subclass of :exc:`ConnectionError`, raised when a connection attempt is refused by the peer. - Corresponds to :c:data:`errno` :py:data:`~errno.ECONNREFUSED`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECONNREFUSED`. .. exception:: ConnectionResetError A subclass of :exc:`ConnectionError`, raised when a connection is reset by the peer. - Corresponds to :c:data:`errno` :py:data:`~errno.ECONNRESET`. + Corresponds to :c:data:`errno` :py:const:`~errno.ECONNRESET`. .. exception:: FileExistsError Raised when trying to create a file or directory which already exists. - Corresponds to :c:data:`errno` :py:data:`~errno.EEXIST`. + Corresponds to :c:data:`errno` :py:const:`~errno.EEXIST`. .. exception:: FileNotFoundError Raised when a file or directory is requested but doesn't exist. - Corresponds to :c:data:`errno` :py:data:`~errno.ENOENT`. + Corresponds to :c:data:`errno` :py:const:`~errno.ENOENT`. .. exception:: InterruptedError Raised when a system call is interrupted by an incoming signal. - Corresponds to :c:data:`errno` :py:data:`~errno.EINTR`. + Corresponds to :c:data:`errno` :py:const:`~errno.EINTR`. .. versionchanged:: 3.5 Python now retries system calls when a syscall is interrupted by a @@ -732,7 +732,7 @@ depending on the system error code. Raised when a file operation (such as :func:`os.remove`) is requested on a directory. - Corresponds to :c:data:`errno` :py:data:`~errno.EISDIR`. + Corresponds to :c:data:`errno` :py:const:`~errno.EISDIR`. .. exception:: NotADirectoryError @@ -740,28 +740,28 @@ depending on the system error code. something which is not a directory. On most POSIX platforms, it may also be raised if an operation attempts to open or traverse a non-directory file as if it were a directory. - Corresponds to :c:data:`errno` :py:data:`~errno.ENOTDIR`. + Corresponds to :c:data:`errno` :py:const:`~errno.ENOTDIR`. .. exception:: PermissionError Raised when trying to run an operation without the adequate access rights - for example filesystem permissions. - Corresponds to :c:data:`errno` :py:data:`~errno.EACCES`, - :py:data:`~errno.EPERM`, and :py:data:`~errno.ENOTCAPABLE`. + Corresponds to :c:data:`errno` :py:const:`~errno.EACCES`, + :py:const:`~errno.EPERM`, and :py:const:`~errno.ENOTCAPABLE`. .. versionchanged:: 3.11.1 - WASI's :py:data:`~errno.ENOTCAPABLE` is now mapped to + WASI's :py:const:`~errno.ENOTCAPABLE` is now mapped to :exc:`PermissionError`. .. exception:: ProcessLookupError Raised when a given process doesn't exist. - Corresponds to :c:data:`errno` :py:data:`~errno.ESRCH`. + Corresponds to :c:data:`errno` :py:const:`~errno.ESRCH`. .. exception:: TimeoutError Raised when a system function timed out at the system level. - Corresponds to :c:data:`errno` :py:data:`~errno.ETIMEDOUT`. + Corresponds to :c:data:`errno` :py:const:`~errno.ETIMEDOUT`. .. versionadded:: 3.3 All the above :exc:`OSError` subclasses were added. diff --git a/Doc/library/fcntl.rst b/Doc/library/fcntl.rst index c5a7ba0150eb7..13de0c7bc33bf 100644 --- a/Doc/library/fcntl.rst +++ b/Doc/library/fcntl.rst @@ -166,9 +166,9 @@ The module defines the following functions: which the lock starts, relative to *whence*, and *whence* is as with :func:`io.IOBase.seek`, specifically: - * :const:`0` -- relative to the start of the file (:data:`os.SEEK_SET`) - * :const:`1` -- relative to the current buffer position (:data:`os.SEEK_CUR`) - * :const:`2` -- relative to the end of the file (:data:`os.SEEK_END`) + * :const:`0` -- relative to the start of the file (:const:`os.SEEK_SET`) + * :const:`1` -- relative to the current buffer position (:const:`os.SEEK_CUR`) + * :const:`2` -- relative to the end of the file (:const:`os.SEEK_END`) The default for *start* is 0, which means to start at the beginning of the file. The default for *len* is 0 which means to lock to the end of the file. The @@ -195,7 +195,7 @@ using the :func:`flock` call may be better. .. seealso:: Module :mod:`os` - If the locking flags :data:`~os.O_SHLOCK` and :data:`~os.O_EXLOCK` are + If the locking flags :const:`~os.O_SHLOCK` and :const:`~os.O_EXLOCK` are present in the :mod:`os` module (on BSD only), the :func:`os.open` function provides an alternative to the :func:`lockf` and :func:`flock` functions. diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst index 517ea4796c0a4..dcac83bbe777f 100644 --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -108,7 +108,7 @@ The module defines the following items: .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. deprecated:: 3.6 @@ -448,7 +448,7 @@ FTP_TLS Objects .. versionchanged:: 3.4 The method now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. method:: FTP_TLS.ccc() diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst index df99ab99826e9..87433ddbd7497 100644 --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -84,7 +84,7 @@ The module provides the following classes: .. versionchanged:: 3.2 This class now supports HTTPS virtual hosts if possible (that is, - if :data:`ssl.HAS_SNI` is true). + if :const:`ssl.HAS_SNI` is true). .. versionchanged:: 3.4 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index 0c10e7afee401..594c6b1a3803a 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -112,7 +112,7 @@ There's also a subclass for secure connections: .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. deprecated:: 3.6 @@ -513,7 +513,7 @@ An :class:`IMAP4` instance has the following methods: .. versionchanged:: 3.4 The method now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. method:: IMAP4.status(mailbox, names) diff --git a/Doc/library/io.rst b/Doc/library/io.rst index c9249da1c3c3d..7eec1f87583b8 100644 --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -423,7 +423,7 @@ I/O Base Classes .. versionadded:: 3.3 Some operating systems could support additional values, like - :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. The valid values + :const:`os.SEEK_HOLE` or :const:`os.SEEK_DATA`. The valid values for a file could depend on it being open in text or binary mode. .. method:: seekable() diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 26b057f297bf7..6cbbcbf1b804a 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -2682,7 +2682,7 @@ handler type) for messages from different processes to get mixed up. Returns the logger used by :mod:`multiprocessing`. If necessary, a new one will be created. - When first created the logger has level :data:`logging.NOTSET` and no + When first created the logger has level :const:`logging.NOTSET` and no default handler. Messages sent to this logger will not by default propagate to the root logger. diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index f3b3a43f3c44a..6ab61ba1d13e0 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -812,7 +812,7 @@ The first step in using :mod:`optparse` is to create an OptionParser instance. help option. When :mod:`optparse` prints the usage string, it expands ``%prog`` to ``os.path.basename(sys.argv[0])`` (or to ``prog`` if you passed that keyword argument). To suppress a usage message, pass the - special value :data:`optparse.SUPPRESS_USAGE`. + special value :const:`optparse.SUPPRESS_USAGE`. ``option_list`` (default: ``[]``) A list of Option objects to populate the parser with. The options in @@ -1078,7 +1078,7 @@ relevant to a particular option, or fail to pass a required option attribute, Help text to print for this option when listing all available options after the user supplies a :attr:`~Option.help` option (such as ``--help``). If no help text is supplied, the option will be listed without help text. To - hide this option, use the special value :data:`optparse.SUPPRESS_HELP`. + hide this option, use the special value :const:`optparse.SUPPRESS_HELP`. .. attribute:: Option.metavar @@ -1250,7 +1250,7 @@ must specify for any option using that action. If no :attr:`~Option.help` string is supplied for an option, it will still be listed in the help message. To omit an option entirely, use the special value - :data:`optparse.SUPPRESS_HELP`. + :const:`optparse.SUPPRESS_HELP`. :mod:`optparse` automatically adds a :attr:`~Option.help` option to all OptionParsers, so you do not normally need to create one. @@ -1521,7 +1521,7 @@ OptionParser supports several other public methods: Set the usage string according to the rules described above for the ``usage`` constructor keyword argument. Passing ``None`` sets the default usage - string; use :data:`optparse.SUPPRESS_USAGE` to suppress a usage message. + string; use :const:`optparse.SUPPRESS_USAGE` to suppress a usage message. .. method:: OptionParser.print_usage(file=None) diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 31946f1a12b9d..283dc5ad1541a 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -233,7 +233,7 @@ process and user. :data:`environ` and :data:`environb` are synchronized (modifying :data:`environb` updates :data:`environ`, and vice versa). - :data:`environb` is only available if :data:`supports_bytes_environ` is + :data:`environb` is only available if :const:`supports_bytes_environ` is ``True``. .. versionadded:: 3.2 @@ -331,7 +331,7 @@ process and user. future environment changes. - :func:`getenvb` is only available if :data:`supports_bytes_environ` + :func:`getenvb` is only available if :const:`supports_bytes_environ` is ``True``. .. availability:: Unix. @@ -829,7 +829,7 @@ as internal buffering of data. If *offset_src* is None, then *src* is read from the current position; respectively for *offset_dst*. The files pointed by *src* and *dst* must reside in the same filesystem, otherwise an :exc:`OSError` is - raised with :attr:`~OSError.errno` set to :data:`errno.EXDEV`. + raised with :attr:`~OSError.errno` set to :const:`errno.EXDEV`. This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, @@ -1071,7 +1071,7 @@ as internal buffering of data. .. versionadded:: 3.3 Some operating systems could support additional values, like - :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. + :const:`os.SEEK_HOLE` or :const:`os.SEEK_DATA`. .. function:: open(path, flags, mode=0o777, *, dir_fd=None) @@ -1312,7 +1312,7 @@ or `the MSDN `_ on Windo If some data was successfully read, it will return the number of bytes read. If no bytes were read, it will return ``-1`` and set errno to - :data:`errno.EAGAIN`. + :const:`errno.EAGAIN`. .. availability:: Linux >= 4.14. @@ -1513,7 +1513,7 @@ or `the MSDN `_ on Windo *offset_dst*. The offset associated to the file descriptor that refers to a pipe must be ``None``. The files pointed by *src* and *dst* must reside in the same filesystem, otherwise an :exc:`OSError` is raised with - :attr:`~OSError.errno` set to :data:`errno.EXDEV`. + :attr:`~OSError.errno` set to :const:`errno.EXDEV`. This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, @@ -1846,18 +1846,18 @@ features: Set the flags of *path* to the numeric *flags*. *flags* may take a combination (bitwise OR) of the following values (as defined in the :mod:`stat` module): - * :data:`stat.UF_NODUMP` - * :data:`stat.UF_IMMUTABLE` - * :data:`stat.UF_APPEND` - * :data:`stat.UF_OPAQUE` - * :data:`stat.UF_NOUNLINK` - * :data:`stat.UF_COMPRESSED` - * :data:`stat.UF_HIDDEN` - * :data:`stat.SF_ARCHIVED` - * :data:`stat.SF_IMMUTABLE` - * :data:`stat.SF_APPEND` - * :data:`stat.SF_NOUNLINK` - * :data:`stat.SF_SNAPSHOT` + * :const:`stat.UF_NODUMP` + * :const:`stat.UF_IMMUTABLE` + * :const:`stat.UF_APPEND` + * :const:`stat.UF_OPAQUE` + * :const:`stat.UF_NOUNLINK` + * :const:`stat.UF_COMPRESSED` + * :const:`stat.UF_HIDDEN` + * :const:`stat.SF_ARCHIVED` + * :const:`stat.SF_IMMUTABLE` + * :const:`stat.SF_APPEND` + * :const:`stat.SF_NOUNLINK` + * :const:`stat.SF_SNAPSHOT` This function can support :ref:`not following symlinks `. @@ -1878,25 +1878,25 @@ features: following values (as defined in the :mod:`stat` module) or bitwise ORed combinations of them: - * :data:`stat.S_ISUID` - * :data:`stat.S_ISGID` - * :data:`stat.S_ENFMT` - * :data:`stat.S_ISVTX` - * :data:`stat.S_IREAD` - * :data:`stat.S_IWRITE` - * :data:`stat.S_IEXEC` - * :data:`stat.S_IRWXU` - * :data:`stat.S_IRUSR` - * :data:`stat.S_IWUSR` - * :data:`stat.S_IXUSR` - * :data:`stat.S_IRWXG` - * :data:`stat.S_IRGRP` - * :data:`stat.S_IWGRP` - * :data:`stat.S_IXGRP` - * :data:`stat.S_IRWXO` - * :data:`stat.S_IROTH` - * :data:`stat.S_IWOTH` - * :data:`stat.S_IXOTH` + * :const:`stat.S_ISUID` + * :const:`stat.S_ISGID` + * :const:`stat.S_ENFMT` + * :const:`stat.S_ISVTX` + * :const:`stat.S_IREAD` + * :const:`stat.S_IWRITE` + * :const:`stat.S_IEXEC` + * :const:`stat.S_IRWXU` + * :const:`stat.S_IRUSR` + * :const:`stat.S_IWUSR` + * :const:`stat.S_IXUSR` + * :const:`stat.S_IRWXG` + * :const:`stat.S_IRGRP` + * :const:`stat.S_IWGRP` + * :const:`stat.S_IXGRP` + * :const:`stat.S_IRWXO` + * :const:`stat.S_IROTH` + * :const:`stat.S_IWOTH` + * :const:`stat.S_IXOTH` This function can support :ref:`specifying a file descriptor `, :ref:`paths relative to directory descriptors ` and :ref:`not @@ -3920,8 +3920,8 @@ written in Python, such as a mail server's external command delivery program. Send signal *sig* to the process *pid*. Constants for the specific signals available on the host platform are defined in the :mod:`signal` module. - Windows: The :data:`signal.CTRL_C_EVENT` and - :data:`signal.CTRL_BREAK_EVENT` signals are special signals which can + Windows: The :const:`signal.CTRL_C_EVENT` and + :const:`signal.CTRL_BREAK_EVENT` signals are special signals which can only be sent to console processes which share a common console window, e.g., some subprocesses. Any other value for *sig* will cause the process to be unconditionally killed by the TerminateProcess API, and the exit code @@ -4414,7 +4414,7 @@ written in Python, such as a mail server's external command delivery program. * :attr:`!si_pid` (process ID) * :attr:`!si_uid` (real user ID of the child) - * :attr:`!si_signo` (always :data:`~signal.SIGCHLD`) + * :attr:`!si_signo` (always :const:`~signal.SIGCHLD`) * :attr:`!si_status` (the exit status or signal number, depending on :attr:`!si_code`) * :attr:`!si_code` (see :data:`CLD_EXITED` for possible values) @@ -4652,7 +4652,7 @@ used to determine the disposition of a process. .. function:: WIFCONTINUED(status) Return ``True`` if a stopped child has been resumed by delivery of - :data:`~signal.SIGCONT` (if the process has been continued from a job + :const:`~signal.SIGCONT` (if the process has been continued from a job control stop), otherwise return ``False``. See :data:`WCONTINUED` option. @@ -5024,7 +5024,7 @@ Random numbers ``/dev/urandom`` devices. The flags argument is a bit mask that can contain zero or more of the - following values ORed together: :py:data:`os.GRND_RANDOM` and + following values ORed together: :py:const:`os.GRND_RANDOM` and :py:data:`GRND_NONBLOCK`. See also the `Linux getrandom() manual page diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index e22a2e1455e7f..c9ada42447e8a 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -81,7 +81,7 @@ The :mod:`poplib` module provides two classes: .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. deprecated:: 3.6 @@ -248,7 +248,7 @@ A :class:`POP3` instance has the following methods: This method supports hostname checking via :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. versionadded:: 3.4 diff --git a/Doc/library/shelve.rst b/Doc/library/shelve.rst index dc87af398ed75..01314f491f47a 100644 --- a/Doc/library/shelve.rst +++ b/Doc/library/shelve.rst @@ -25,7 +25,7 @@ lots of shared sub-objects. The keys are ordinary strings. database file is opened for reading and writing. The optional *flag* parameter has the same interpretation as the *flag* parameter of :func:`dbm.open`. - By default, pickles created with :data:`pickle.DEFAULT_PROTOCOL` are used + By default, pickles created with :const:`pickle.DEFAULT_PROTOCOL` are used to serialize values. The version of the pickle protocol can be specified with the *protocol* parameter. @@ -42,7 +42,7 @@ lots of shared sub-objects. The keys are ordinary strings. mutated). .. versionchanged:: 3.10 - :data:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle + :const:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle protocol. .. versionchanged:: 3.11 @@ -119,7 +119,7 @@ Restrictions A subclass of :class:`collections.abc.MutableMapping` which stores pickled values in the *dict* object. - By default, pickles created with :data:`pickle.DEFAULT_PROTOCOL` are used + By default, pickles created with :const:`pickle.DEFAULT_PROTOCOL` are used to serialize values. The version of the pickle protocol can be specified with the *protocol* parameter. See the :mod:`pickle` documentation for a discussion of the pickle protocols. @@ -143,7 +143,7 @@ Restrictions Added context manager support. .. versionchanged:: 3.10 - :data:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle + :const:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle protocol. diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index a2815059ad66e..67cd5b000afb7 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -103,7 +103,7 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions). .. versionchanged:: 3.4 The class now supports hostname check with :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see - :data:`ssl.HAS_SNI`). + :const:`ssl.HAS_SNI`). .. deprecated:: 3.6 @@ -431,7 +431,7 @@ An :class:`SMTP` instance has the following methods: .. versionchanged:: 3.4 The method now supports hostname check with :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see - :data:`~ssl.HAS_SNI`). + :const:`~ssl.HAS_SNI`). .. versionchanged:: 3.5 The error raised for lack of STARTTLS support is now the diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index 19b4a1ee59c28..2e129175636f2 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -2153,7 +2153,7 @@ This is because the previous execution has left the socket in a ``TIME_WAIT`` state, and can't be immediately reused. There is a :mod:`socket` flag to set, in order to prevent this, -:data:`socket.SO_REUSEADDR`:: +:const:`socket.SO_REUSEADDR`:: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 06f5293fbdbd1..13e61b3d68006 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1643,9 +1643,9 @@ Blob objects .. method:: seek(offset, origin=os.SEEK_SET, /) Set the current access position of the blob to *offset*. The *origin* - argument defaults to :data:`os.SEEK_SET` (absolute blob positioning). - Other values for *origin* are :data:`os.SEEK_CUR` (seek relative to the - current position) and :data:`os.SEEK_END` (seek relative to the blob?s + argument defaults to :const:`os.SEEK_SET` (absolute blob positioning). + Other values for *origin* are :const:`os.SEEK_CUR` (seek relative to the + current position) and :const:`os.SEEK_END` (seek relative to the blob?s end). diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 289dd0b51dea1..a0936747db5a0 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -142,7 +142,7 @@ purposes. The settings are: :data:`PROTOCOL_TLS_CLIENT` or :data:`PROTOCOL_TLS_SERVER`, :data:`OP_NO_SSLv2`, and :data:`OP_NO_SSLv3` with high encryption cipher suites without RC4 and - without unauthenticated cipher suites. Passing :data:`~Purpose.SERVER_AUTH` + without unauthenticated cipher suites. Passing :const:`~Purpose.SERVER_AUTH` as *purpose* sets :data:`~SSLContext.verify_mode` to :data:`CERT_REQUIRED` and either loads CA certificates (when at least one of *cafile*, *capath* or *cadata* is given) or uses :meth:`SSLContext.load_default_certs` to load @@ -1579,9 +1579,9 @@ to speed up repeated connections from the same clients. load CA certificates from other locations, too. The *purpose* flag specifies what kind of CA certificates are loaded. The - default settings :data:`Purpose.SERVER_AUTH` loads certificates, that are + default settings :const:`Purpose.SERVER_AUTH` loads certificates, that are flagged and trusted for TLS web server authentication (client side - sockets). :data:`Purpose.CLIENT_AUTH` loads CA certificates for client + sockets). :const:`Purpose.CLIENT_AUTH` loads CA certificates for client certificate verification on the server side. .. versionadded:: 3.4 @@ -1824,7 +1824,7 @@ to speed up repeated connections from the same clients. Wrap an existing Python socket *sock* and return an instance of :attr:`SSLContext.sslsocket_class` (default :class:`SSLSocket`). The returned SSL socket is tied to the context, its settings and certificates. - *sock* must be a :data:`~socket.SOCK_STREAM` socket; other + *sock* must be a :const:`~socket.SOCK_STREAM` socket; other socket types are unsupported. The parameter ``server_side`` is a boolean which identifies whether diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 643cdb876ee3c..718dc48e804c0 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -686,7 +686,7 @@ always available. Return the current value of the flags that are used for :c:func:`dlopen` calls. Symbolic names for the flag values can be found in the :mod:`os` module (``RTLD_xxx`` constants, e.g. - :data:`os.RTLD_LAZY`). + :const:`os.RTLD_LAZY`). .. availability:: Unix. @@ -1342,7 +1342,7 @@ always available. ``sys.setdlopenflags(0)``. To share symbols across extension modules, call as ``sys.setdlopenflags(os.RTLD_GLOBAL)``. Symbolic names for the flag values can be found in the :mod:`os` module (``RTLD_xxx`` constants, e.g. - :data:`os.RTLD_LAZY`). + :const:`os.RTLD_LAZY`). .. availability:: Unix. diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index c17ead1510e7e..84a6f8887954c 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -59,7 +59,7 @@ The module defines the following user-callable items: platforms, it is a file-like object whose :attr:`!file` attribute is the underlying true file object. - The :py:data:`os.O_TMPFILE` flag is used if it is available and works + The :py:const:`os.O_TMPFILE` flag is used if it is available and works (Linux-specific, requires Linux kernel 3.11 or later). On platforms that are neither Posix nor Cygwin, TemporaryFile is an alias @@ -69,7 +69,7 @@ The module defines the following user-callable items: .. versionchanged:: 3.5 - The :py:data:`os.O_TMPFILE` flag is now used if available. + The :py:const:`os.O_TMPFILE` flag is now used if available. .. versionchanged:: 3.8 Added *errors* parameter. diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 73d1178906a8c..427953e0077aa 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -418,7 +418,7 @@ The :mod:`test.support` module defines the following functions: .. function:: with_pymalloc() - Return :data:`_testcapi.WITH_PYMALLOC`. + Return :const:`_testcapi.WITH_PYMALLOC`. .. function:: requires(resource, msg=None) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 4f06e836d5728..8d877165a0974 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -2436,7 +2436,7 @@ behaviour you can switch it off by setting the module level switch Alternatively you can just use ``vars(my_mock)`` (instance members) and ``dir(type(my_mock))`` (type members) to bypass the filtering irrespective of -:data:`mock.FILTER_DIR`. +:const:`mock.FILTER_DIR`. mock_open diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst index 1b05458280d89..35b8f5b471dd9 100644 --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -99,7 +99,7 @@ The :mod:`urllib.request` module defines the following functions: .. versionchanged:: 3.2 HTTPS virtual hosts are now supported if possible (that is, if - :data:`ssl.HAS_SNI` is true). + :const:`ssl.HAS_SNI` is true). .. versionadded:: 3.2 *data* can be an iterable object. diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index 20b0905bb1093..cf30de67719b9 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -73,7 +73,7 @@ decompression bomb Safe Safe Safe 1. Expat 2.4.1 and newer is not vulnerable to the "billion laughs" and "quadratic blowup" vulnerabilities. Items still listed as vulnerable due to potential reliance on system-provided libraries. Check - :data:`pyexpat.EXPAT_VERSION`. + :const:`pyexpat.EXPAT_VERSION`. 2. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a :exc:`ParserError` when an entity occurs. 3. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index 2f31910a5958c..468228f12cfa0 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -62,7 +62,7 @@ General Options .. cmdoption:: --with-tzpath= - Select the default time zone search path for :data:`zoneinfo.TZPATH`. + Select the default time zone search path for :const:`zoneinfo.TZPATH`. See the :ref:`Compile-time configuration ` of the :mod:`zoneinfo` module. @@ -77,7 +77,7 @@ General Options Build the ``_decimal`` extension module using a thread-local context rather than a coroutine-local context (default), see the :mod:`decimal` module. - See :data:`decimal.HAVE_CONTEXTVAR` and the :mod:`contextvars` module. + See :const:`decimal.HAVE_CONTEXTVAR` and the :mod:`contextvars` module. .. versionadded:: 3.9 diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index 3cbab8d567bfa..ba6aebafd59d9 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -1555,9 +1555,9 @@ changes, or look through the Subversion logs for all the details. :issue:`8484`.) The version of OpenSSL being used is now available as the module - attributes :data:`ssl.OPENSSL_VERSION` (a string), - :data:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and - :data:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Added by Antoine + attributes :const:`ssl.OPENSSL_VERSION` (a string), + :const:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and + :const:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Added by Antoine Pitrou; :issue:`8321`.) * The :mod:`struct` module will no longer silently ignore overflow diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index b0b570be8947b..56b8ff36bee77 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1252,8 +1252,8 @@ descriptors without copying between kernel address space and user address space, where one of the file descriptors must refer to a pipe. (Contributed by Pablo Galindo in :issue:`41625`.) -Add :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` -and :data:`~os.O_NOFOLLOW_ANY` for macOS. +Add :const:`~os.O_EVTONLY`, :const:`~os.O_FSYNC`, :const:`~os.O_SYMLINK` +and :const:`~os.O_NOFOLLOW_ANY` for macOS. (Contributed by Dong-hee Na in :issue:`43106`.) os.path @@ -1318,7 +1318,7 @@ objects in the tree returned by :func:`pyclbr.readline` and shelve ------ -The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default +The :mod:`shelve` module now uses :const:`pickle.DEFAULT_PROTOCOL` by default instead of :mod:`pickle` protocol ``3`` when creating shelves. (Contributed by Zackery Spytz in :issue:`34204`.) @@ -1355,7 +1355,7 @@ The ssl module requires OpenSSL 1.1.1 or newer. (Contributed by Christian Heimes in :pep:`644` and :issue:`43669`.) The ssl module has preliminary support for OpenSSL 3.0.0 and new option -:data:`~ssl.OP_IGNORE_UNEXPECTED_EOF`. +:const:`~ssl.OP_IGNORE_UNEXPECTED_EOF`. (Contributed by Christian Heimes in :issue:`38820`, :issue:`43794`, :issue:`43788`, :issue:`43791`, :issue:`43799`, :issue:`43920`, :issue:`43789`, and :issue:`43811`.) @@ -1386,7 +1386,7 @@ Add a *timeout* parameter to the :func:`ssl.get_server_certificate` function. The ssl module uses heap-types and multi-phase initialization. (Contributed by Christian Heimes in :issue:`42333`.) -A new verify flag :data:`~ssl.VERIFY_X509_PARTIAL_CHAIN` has been added. +A new verify flag :const:`~ssl.VERIFY_X509_PARTIAL_CHAIN` has been added. (Contributed by l0x in :issue:`40849`.) sqlite3 @@ -1412,7 +1412,7 @@ _thread ------- :func:`_thread.interrupt_main` now takes an optional signal number to -simulate (the default is still :data:`signal.SIGINT`). +simulate (the default is still :const:`signal.SIGINT`). (Contributed by Antoine Pitrou in :issue:`43356`.) threading @@ -1756,8 +1756,8 @@ Deprecated * :data:`~ssl.PROTOCOL_SSLv2`, :data:`~ssl.PROTOCOL_SSLv3`, :data:`~ssl.PROTOCOL_SSLv23`, :data:`~ssl.PROTOCOL_TLSv1`, :data:`~ssl.PROTOCOL_TLSv1_1`, :data:`~ssl.PROTOCOL_TLSv1_2`, and - :data:`~ssl.PROTOCOL_TLS` are deprecated in favor of - :data:`~ssl.PROTOCOL_TLS_CLIENT` and :data:`~ssl.PROTOCOL_TLS_SERVER` + :const:`~ssl.PROTOCOL_TLS` are deprecated in favor of + :const:`~ssl.PROTOCOL_TLS_CLIENT` and :const:`~ssl.PROTOCOL_TLS_SERVER` * :func:`~ssl.wrap_socket` is replaced by :meth:`ssl.SSLContext.wrap_socket` diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 973ce154bbdad..c437c5bfb6120 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -692,7 +692,7 @@ enum * Added the :func:`~enum.global_enum` enum decorator, which adjusts :meth:`~object.__repr__` and :meth:`~object.__str__` to show values as members of their module rather than the enum class. - For example, ``'re.ASCII'`` for the :data:`~re.ASCII` member + For example, ``'re.ASCII'`` for the :const:`~re.ASCII` member of :class:`re.RegexFlag` rather than ``'RegexFlag.ASCII'``. * Enhanced :class:`~enum.Flag` to support @@ -1065,8 +1065,8 @@ threading * On Unix, if the ``sem_clockwait()`` function is available in the C library (glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses - the monotonic clock (:data:`time.CLOCK_MONOTONIC`) for the timeout, rather - than using the system clock (:data:`time.CLOCK_REALTIME`), to not be affected + the monotonic clock (:const:`time.CLOCK_MONOTONIC`) for the timeout, rather + than using the system clock (:const:`time.CLOCK_REALTIME`), to not be affected by system clock changes. (Contributed by Victor Stinner in :issue:`41710`.) @@ -1814,7 +1814,7 @@ Standard Library (Contributed by Serhiy Storchaka in :gh:`91760`.) * In the :mod:`re` module, the :func:`!re.template` function - and the corresponding :data:`!re.TEMPLATE` and :data:`!re.T` flags + and the corresponding :const:`!re.TEMPLATE` and :const:`!re.T` flags are deprecated, as they were undocumented and lacked an obvious purpose. They will be removed in Python 3.13. (Contributed by Serhiy Storchaka and Miro Hron?ok in :gh:`92728`.) diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index 2e4c8ee45df84..2c176926e55e1 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -1664,9 +1664,9 @@ for secure (encrypted, authenticated) internet connections: algorithm" error. * The version of OpenSSL being used is now accessible using the module - attributes :data:`ssl.OPENSSL_VERSION` (a string), - :data:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and - :data:`ssl.OPENSSL_VERSION_NUMBER` (an integer). + attributes :const:`ssl.OPENSSL_VERSION` (a string), + :const:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and + :const:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Contributed by Antoine Pitrou in :issue:`8850`, :issue:`1589`, :issue:`8322`, :issue:`5639`, :issue:`4870`, :issue:`8484`, and :issue:`8321`.) diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 9bbf52c555178..782048c397f97 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -841,7 +841,7 @@ Builtin functions and types * :func:`open` gets a new *opener* parameter: the underlying file descriptor for the file object is then obtained by calling *opener* with (*file*, - *flags*). It can be used to use custom flags like :data:`os.O_CLOEXEC` for + *flags*). It can be used to use custom flags like :const:`os.O_CLOEXEC` for example. The ``'x'`` mode was added: open for exclusive creation, failing if the file already exists. * :func:`print`: added the *flush* keyword argument. If the *flush* keyword @@ -1126,7 +1126,7 @@ Features * If Python is compiled without threads, the C version automatically disables the expensive thread local context machinery. In this case, - the variable :data:`~decimal.HAVE_THREADS` is set to ``False``. + the variable :const:`~decimal.HAVE_THREADS` is set to ``False``. API changes ~~~~~~~~~~~ @@ -1575,8 +1575,8 @@ os -- * The :mod:`os` module has a new :func:`~os.pipe2` function that makes it - possible to create a pipe with :data:`~os.O_CLOEXEC` or - :data:`~os.O_NONBLOCK` flags set atomically. This is especially useful to + possible to create a pipe with :const:`~os.O_CLOEXEC` or + :const:`~os.O_NONBLOCK` flags set atomically. This is especially useful to avoid race conditions in multi-threaded programs. * The :mod:`os` module has a new :func:`~os.sendfile` function which provides @@ -1690,9 +1690,9 @@ os * Some platforms now support additional constants for the :func:`~os.lseek` function, such as ``os.SEEK_HOLE`` and ``os.SEEK_DATA``. -* New constants :data:`~os.RTLD_LAZY`, :data:`~os.RTLD_NOW`, - :data:`~os.RTLD_GLOBAL`, :data:`~os.RTLD_LOCAL`, :data:`~os.RTLD_NODELETE`, - :data:`~os.RTLD_NOLOAD`, and :data:`~os.RTLD_DEEPBIND` are available on +* New constants :const:`~os.RTLD_LAZY`, :const:`~os.RTLD_NOW`, + :const:`~os.RTLD_GLOBAL`, :const:`~os.RTLD_LOCAL`, :const:`~os.RTLD_NODELETE`, + :const:`~os.RTLD_NOLOAD`, and :const:`~os.RTLD_DEEPBIND` are available on platforms that support them. These are for use with the :func:`sys.setdlopenflags` function, and supersede the similar constants defined in :mod:`ctypes` and :mod:`DLFCN`. (Contributed by Victor Stinner @@ -1994,7 +1994,7 @@ subprocess Command strings can now be bytes objects on posix platforms. (Contributed by Victor Stinner in :issue:`8513`.) -A new constant :data:`~subprocess.DEVNULL` allows suppressing output in a +A new constant :const:`~subprocess.DEVNULL` allows suppressing output in a platform-independent fashion. (Contributed by Ross Lagerwall in :issue:`5870`.) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst index 7436a552f529e..b0d0417447691 100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -775,7 +775,7 @@ of a given opcode and argument, information that is not otherwise available. doctest ------- -A new :ref:`option flag `, :data:`~doctest.FAIL_FAST`, halts +A new :ref:`option flag `, :const:`~doctest.FAIL_FAST`, halts test running as soon as the first failure is detected. (Contributed by R. David Murray and Daniel Urban in :issue:`16522`.) @@ -841,7 +841,7 @@ for example, if the file might have been changed and re-checked in less time than the resolution of a particular filesystem's file modification time field. (Contributed by Mark Levitt in :issue:`18149`.) -New module attribute :data:`~filecmp.DEFAULT_IGNORES` provides the list of +New module attribute :const:`~filecmp.DEFAULT_IGNORES` provides the list of directories that are used as the default value for the *ignore* parameter of the :func:`~filecmp.dircmp` function. (Contributed by Eli Bendersky in :issue:`15442`.) @@ -1189,7 +1189,7 @@ Windows). (Contributed by Brian Curtin in :issue:`11939`.) root on Windows. (Contributed by Tim Golden in :issue:`9035`.) :func:`os.open` supports two new flags on platforms that provide them, -:data:`~os.O_PATH` (un-opened file descriptor), and :data:`~os.O_TMPFILE` +:const:`~os.O_PATH` (un-opened file descriptor), and :const:`~os.O_TMPFILE` (unnamed temporary file; as of 3.4.0 release available only on Linux systems with a kernel version of 3.11 or newer that have uapi headers). (Contributed by Christian Heimes in :issue:`18673` and Benjamin Peterson, respectively.) @@ -1238,8 +1238,8 @@ plistlib stdlib serialization protocols, with new :func:`~plistlib.load`, :func:`~plistlib.dump`, :func:`~plistlib.loads`, and :func:`~plistlib.dumps` functions. (The older API is now deprecated.) In addition to the already -supported XML plist format (:data:`~plistlib.FMT_XML`), it also now supports -the binary plist format (:data:`~plistlib.FMT_BINARY`). (Contributed by Ronald +supported XML plist format (:const:`~plistlib.FMT_XML`), it also now supports +the binary plist format (:const:`~plistlib.FMT_BINARY`). (Contributed by Ronald Oussoren and others in :issue:`14455`.) @@ -1388,7 +1388,7 @@ try/except statement by code that only cares whether or not an error occurred. socket ------ -The socket module now supports the :data:`~socket.CAN_BCM` protocol on +The socket module now supports the :const:`~socket.CAN_BCM` protocol on platforms that support it. (Contributed by Brian Thorne in :issue:`15359`.) Socket objects have new methods to get or set their :ref:`inheritable flag @@ -1399,7 +1399,7 @@ The ``socket.AF_*`` and ``socket.SOCK_*`` constants are now enumeration values using the new :mod:`enum` module. This allows meaningful names to be printed during debugging, instead of integer "magic numbers". -The :data:`~socket.AF_LINK` constant is now available on BSD and OSX. +The :const:`~socket.AF_LINK` constant is now available on BSD and OSX. :func:`~socket.inet_pton` and :func:`~socket.inet_ntop` are now supported on Windows. (Contributed by Atsuo Ishimoto in :issue:`7171`.) @@ -1460,8 +1460,8 @@ Heimes in :issue:`18147`.) If OpenSSL 0.9.8 or later is available, :class:`~ssl.SSLContext` has a new attribute :attr:`~ssl.SSLContext.verify_flags` that can be used to control the certificate verification process by setting it to some combination of the new -constants :data:`~ssl.VERIFY_DEFAULT`, :data:`~ssl.VERIFY_CRL_CHECK_LEAF`, -:data:`~ssl.VERIFY_CRL_CHECK_CHAIN`, or :data:`~ssl.VERIFY_X509_STRICT`. +constants :const:`~ssl.VERIFY_DEFAULT`, :const:`~ssl.VERIFY_CRL_CHECK_LEAF`, +:const:`~ssl.VERIFY_CRL_CHECK_CHAIN`, or :const:`~ssl.VERIFY_X509_STRICT`. OpenSSL does not do any CRL verification by default. (Contributed by Christien Heimes in :issue:`8813`.) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index fb64135e1ee38..23d2f6562aab8 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -478,7 +478,7 @@ not make an additional system call:: PEP 475: Retry system calls failing with EINTR ---------------------------------------------- -An :py:data:`errno.EINTR` error code is returned whenever a system call, that +An :py:const:`errno.EINTR` error code is returned whenever a system call, that is waiting for I/O, is interrupted by a signal. Previously, Python would raise :exc:`InterruptedError` in such cases. This meant that, when writing a Python application, the developer had two choices: @@ -527,7 +527,7 @@ by a signal: :func:`~os.writev`; * special cases: :func:`os.close` and :func:`os.dup2` now ignore - :py:data:`~errno.EINTR` errors; the syscall is not retried (see the PEP + :py:const:`~errno.EINTR` errors; the syscall is not retried (see the PEP for the rationale); * :mod:`select` functions: :func:`devpoll.poll() `, @@ -1498,7 +1498,7 @@ use ``/dev/urandom`` and avoiding failures due to potential file descriptor exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow -getting and setting a file descriptor's blocking mode (:data:`~os.O_NONBLOCK`.) +getting and setting a file descriptor's blocking mode (:const:`~os.O_NONBLOCK`.) (Contributed by Victor Stinner in :issue:`22054`.) The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported @@ -1783,7 +1783,7 @@ the TLS handshake. The new :meth:`SSLSocket.selected_alpn_protocol() ` returns the protocol that was selected during the TLS handshake. -The :data:`~ssl.HAS_ALPN` flag indicates whether ALPN support is present. +The :const:`~ssl.HAS_ALPN` flag indicates whether ALPN support is present. Other Changes @@ -2476,7 +2476,7 @@ Changes in the Python API in Python 3.5, all old ``.pyo`` files from previous versions of Python are invalid regardless of this PEP. -* The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES` +* The :mod:`socket` module now exports the :const:`~socket.CAN_RAW_FD_FRAMES` constant on linux 3.6 and greater. * The :func:`ssl.cert_time_to_seconds` function now interprets the input time diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 20c43e0e04a3c..1b7cef14d912e 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1404,7 +1404,7 @@ socket ------ The :func:`~socket.socket.ioctl` function now supports the -:data:`~socket.SIO_LOOPBACK_FAST_PATH` control code. +:const:`~socket.SIO_LOOPBACK_FAST_PATH` control code. (Contributed by Daniel Stokes in :issue:`26536`.) The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``, @@ -1416,7 +1416,7 @@ The :meth:`~socket.socket.setsockopt` now supports the (Contributed by Christian Heimes in :issue:`27744`.) The socket module now supports the address family -:data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, +:const:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, ``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added. (Contributed by Christian Heimes in :issue:`27744` with support from Victor Stinner.) diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index de03e5bedeaa3..d00c55a776613 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1280,13 +1280,13 @@ This function should be used instead of :func:`os.close` for better compatibility across platforms. (Contributed by Christian Heimes in :issue:`32454`.) -The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION` -(Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and -:data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. +The :mod:`socket` module now exposes the :const:`socket.TCP_CONGESTION` +(Linux 2.6.13), :const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and +:const:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. (Contributed by Omar Sandoval in :issue:`26273` and Nathaniel J. Smith in :issue:`29728`.) -Support for :data:`socket.AF_VSOCK` sockets has been added to allow +Support for :const:`socket.AF_VSOCK` sockets has been added to allow communication between virtual machines and their hosts. (Contributed by Cathy Avery in :issue:`27584`.) @@ -1394,7 +1394,7 @@ subprocess The :func:`subprocess.run` function accepts the new *capture_output* keyword argument. When true, stdout and stderr will be captured. -This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and +This is equivalent to passing :const:`subprocess.PIPE` as *stdout* and *stderr* arguments. (Contributed by Bo Bayles in :issue:`32102`.) @@ -1453,12 +1453,12 @@ time New clock identifiers have been added: -* :data:`time.CLOCK_BOOTTIME` (Linux): Identical to - :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the +* :const:`time.CLOCK_BOOTTIME` (Linux): Identical to + :const:`time.CLOCK_MONOTONIC`, except it also includes any time that the system is suspended. -* :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution +* :const:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution per-process CPU timer. -* :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is +* :const:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is the time the system has been running and not suspended, providing accurate uptime measurement. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 90220dfb0c054..99291ba48390c 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1302,7 +1302,7 @@ Zackery Spytz in :issue:`25451`.) time ---- -Added new clock :data:`~time.CLOCK_UPTIME_RAW` for macOS 10.12. +Added new clock :const:`~time.CLOCK_UPTIME_RAW` for macOS 10.12. (Contributed by Joannah Nanjekye in :issue:`35702`.) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index c7466c13a4048..1f0fbbc219666 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -427,8 +427,8 @@ digests. It skips MD5 on platforms that block MD5 digest. fcntl ----- -Added constants :data:`~fcntl.F_OFD_GETLK`, :data:`~fcntl.F_OFD_SETLK` -and :data:`~fcntl.F_OFD_SETLKW`. +Added constants :const:`~fcntl.F_OFD_GETLK`, :const:`~fcntl.F_OFD_SETLK` +and :const:`~fcntl.F_OFD_SETLKW`. (Contributed by Dong-hee Na in :issue:`38602`.) ftplib @@ -593,11 +593,11 @@ a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.) os -- -Added :data:`~os.CLD_KILLED` and :data:`~os.CLD_STOPPED` for :attr:`si_code`. +Added :const:`~os.CLD_KILLED` and :const:`~os.CLD_STOPPED` for :attr:`si_code`. (Contributed by Dong-hee Na in :issue:`38493`.) Exposed the Linux-specific :func:`os.pidfd_open` (:issue:`38692`) and -:data:`os.P_PIDFD` (:issue:`38713`) for process management with file +:const:`os.P_PIDFD` (:issue:`38713`) for process management with file descriptors. The :func:`os.unsetenv` function is now also available on Windows. @@ -669,11 +669,11 @@ a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.) socket ------ -The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_JOIN_FILTERS` +The :mod:`socket` module now exports the :const:`~socket.CAN_RAW_JOIN_FILTERS` constant on Linux 4.1 and greater. (Contributed by Stefan Tatschner and Zackery Spytz in :issue:`25780`.) -The socket module now supports the :data:`~socket.CAN_J1939` protocol on +The socket module now supports the :const:`~socket.CAN_J1939` protocol on platforms that support it. (Contributed by Karl Ding in :issue:`40291`.) The socket module now has the :func:`socket.send_fds` and @@ -1084,7 +1084,7 @@ Changes in the Python API ``__VENV_PROMPT__`` is set to ``""``. * The :meth:`select.epoll.unregister` method no longer ignores the - :data:`~errno.EBADF` error. + :const:`~errno.EBADF` error. (Contributed by Victor Stinner in :issue:`39239`.) * The *compresslevel* parameter of :class:`bz2.BZ2File` became keyword-only, diff --git a/Misc/NEWS.d/3.10.0a1.rst b/Misc/NEWS.d/3.10.0a1.rst index 1c1c2d54e8c20..58e57a951ae60 100644 --- a/Misc/NEWS.d/3.10.0a1.rst +++ b/Misc/NEWS.d/3.10.0a1.rst @@ -2176,7 +2176,7 @@ None. .. nonce: YoYoYo .. section: Library -Add a new :data:`os.RWF_APPEND` flag for :func:`os.pwritev`. +Add a new :const:`os.RWF_APPEND` flag for :func:`os.pwritev`. .. @@ -2304,7 +2304,7 @@ Restored the deprecated :mod:`xml.etree.cElementTree` module. .. nonce: ZCk0_c .. section: Library -:data:`~mmap.MAP_POPULATE` constant has now been added to the list of +:const:`~mmap.MAP_POPULATE` constant has now been added to the list of exported :mod:`mmap` module flags. .. diff --git a/Misc/NEWS.d/3.10.0a2.rst b/Misc/NEWS.d/3.10.0a2.rst index 61a291914f933..cafd4804a1ee6 100644 --- a/Misc/NEWS.d/3.10.0a2.rst +++ b/Misc/NEWS.d/3.10.0a2.rst @@ -604,7 +604,7 @@ changes the working directory. PR by Anthony Sottile. .. nonce: 9wXTtY .. section: Library -The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default +The :mod:`shelve` module now uses :const:`pickle.DEFAULT_PROTOCOL` by default instead of :mod:`pickle` protocol ``3``. .. diff --git a/Misc/NEWS.d/3.10.0a6.rst b/Misc/NEWS.d/3.10.0a6.rst index 803df6f51ce62..313aa68925404 100644 --- a/Misc/NEWS.d/3.10.0a6.rst +++ b/Misc/NEWS.d/3.10.0a6.rst @@ -294,8 +294,8 @@ actual dictionary. This created problems for introspection tools. .. nonce: SwcSuU .. section: Library -Added :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` and -:data:`~os.O_NOFOLLOW_ANY` for macOS. Patch by Dong-hee Na. +Added :const:`~os.O_EVTONLY`, :const:`~os.O_FSYNC`, :const:`~os.O_SYMLINK` and +:const:`~os.O_NOFOLLOW_ANY` for macOS. Patch by Dong-hee Na. .. @@ -304,7 +304,7 @@ Added :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK` and .. nonce: a7Dote .. section: Library -Adds :data:`resource.RLIMIT_KQUEUES` constant from FreeBSD to the +Adds :const:`resource.RLIMIT_KQUEUES` constant from FreeBSD to the :mod:`resource` module. .. diff --git a/Misc/NEWS.d/3.10.0a7.rst b/Misc/NEWS.d/3.10.0a7.rst index 286d0a8a7e919..ff01ee645c0a9 100644 --- a/Misc/NEWS.d/3.10.0a7.rst +++ b/Misc/NEWS.d/3.10.0a7.rst @@ -713,7 +713,7 @@ this situation. Also ensures that the :func:`tempfile.gettempdir()` and .. section: Library Expose ``X509_V_FLAG_ALLOW_PROXY_CERTS`` as -:data:`~ssl.VERIFY_ALLOW_PROXY_CERTS` to allow proxy certificate validation +:const:`~ssl.VERIFY_ALLOW_PROXY_CERTS` to allow proxy certificate validation as explained in https://www.openssl.org/docs/man1.1.1/man7/proxy-certificates.html. diff --git a/Misc/NEWS.d/3.10.0b1.rst b/Misc/NEWS.d/3.10.0b1.rst index cf89a550d3854..4f33420c144c9 100644 --- a/Misc/NEWS.d/3.10.0b1.rst +++ b/Misc/NEWS.d/3.10.0b1.rst @@ -871,7 +871,7 @@ assert_called_once_with) will unconditionally pass. .. nonce: -1XPDH .. section: Library -Add :data:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL 3.0.0) +Add :const:`ssl.OP_IGNORE_UNEXPECTED_EOF` constants (OpenSSL 3.0.0) .. diff --git a/Misc/NEWS.d/3.11.0a1.rst b/Misc/NEWS.d/3.11.0a1.rst index e5827e2f23b1f..17cc1fa7cb7ac 100644 --- a/Misc/NEWS.d/3.11.0a1.rst +++ b/Misc/NEWS.d/3.11.0a1.rst @@ -1468,8 +1468,8 @@ an installed expat library <= 2.2.0. On Unix, if the ``sem_clockwait()`` function is available in the C library (glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses -the monotonic clock (:data:`time.CLOCK_MONOTONIC`) for the timeout, rather -than using the system clock (:data:`time.CLOCK_REALTIME`), to not be +the monotonic clock (:const:`time.CLOCK_MONOTONIC`) for the timeout, rather +than using the system clock (:const:`time.CLOCK_REALTIME`), to not be affected by system clock changes. Patch by Victor Stinner. .. @@ -2087,8 +2087,8 @@ Upgrade bundled pip to 21.2.3 and setuptools to 57.4.0 .. section: Library Fix the :func:`os.set_inheritable` function on FreeBSD 14 for file -descriptor opened with the :data:`~os.O_PATH` flag: ignore the -:data:`~errno.EBADF` error on ``ioctl()``, fallback on the ``fcntl()`` +descriptor opened with the :const:`~os.O_PATH` flag: ignore the +:const:`~errno.EBADF` error on ``ioctl()``, fallback on the ``fcntl()`` implementation. Patch by Victor Stinner. .. diff --git a/Misc/NEWS.d/3.11.0a4.rst b/Misc/NEWS.d/3.11.0a4.rst index bcb6e8b7bdde3..3dd335929d655 100644 --- a/Misc/NEWS.d/3.11.0a4.rst +++ b/Misc/NEWS.d/3.11.0a4.rst @@ -839,7 +839,7 @@ patch by Kumar Aditya. .. nonce: jeiPiX .. section: Library -Added :data:`signal.SIGSTKFLT` on platforms where this signal is defined. +Added :const:`signal.SIGSTKFLT` on platforms where this signal is defined. .. diff --git a/Misc/NEWS.d/3.11.0b1.rst b/Misc/NEWS.d/3.11.0b1.rst index d8c2ec0a79971..ad52bd14dbfd1 100644 --- a/Misc/NEWS.d/3.11.0b1.rst +++ b/Misc/NEWS.d/3.11.0b1.rst @@ -817,8 +817,8 @@ it is ever needed and document the existing mechanism for ``posix_spawn()``. .. nonce: HFtERN .. section: Library -Fix :data:`signal.NSIG` value on FreeBSD to accept signal numbers greater -than 32, like :data:`signal.SIGRTMIN` and :data:`signal.SIGRTMAX`. Patch by +Fix :const:`signal.NSIG` value on FreeBSD to accept signal numbers greater +than 32, like :const:`signal.SIGRTMIN` and :const:`signal.SIGRTMAX`. Patch by Victor Stinner. .. diff --git a/Misc/NEWS.d/3.6.0rc1.rst b/Misc/NEWS.d/3.6.0rc1.rst index 15769f950db23..658f8c902d870 100644 --- a/Misc/NEWS.d/3.6.0rc1.rst +++ b/Misc/NEWS.d/3.6.0rc1.rst @@ -69,8 +69,8 @@ supported. .. nonce: ilNIWN .. section: Library -Add new :data:`socket.TCP_CONGESTION` (Linux 2.6.13) and -:data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by +Add new :const:`socket.TCP_CONGESTION` (Linux 2.6.13) and +:const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by Omar Sandoval. .. diff --git a/Misc/NEWS.d/3.7.0a1.rst b/Misc/NEWS.d/3.7.0a1.rst index 24a1c4c83e646..2a851fad5fb2e 100644 --- a/Misc/NEWS.d/3.7.0a1.rst +++ b/Misc/NEWS.d/3.7.0a1.rst @@ -3274,7 +3274,7 @@ Added support for bytes paths in os.fwalk(). .. nonce: 37jMwb .. section: Library -Add new :data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constant. Patch by +Add new :const:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constant. Patch by Nathaniel J. Smith. .. @@ -3871,8 +3871,8 @@ as an integer. Function only available on Android. .. nonce: ilNIWN .. section: Library -Add new :data:`socket.TCP_CONGESTION` (Linux 2.6.13) and -:data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by +Add new :const:`socket.TCP_CONGESTION` (Linux 2.6.13) and +:const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37) constants. Patch written by Omar Sandoval. .. diff --git a/Misc/NEWS.d/3.7.0a3.rst b/Misc/NEWS.d/3.7.0a3.rst index 6576c1fadbff6..92b0f32885120 100644 --- a/Misc/NEWS.d/3.7.0a3.rst +++ b/Misc/NEWS.d/3.7.0a3.rst @@ -754,8 +754,8 @@ now accepts characters as arguments. Based on patch by Steve Fink. .. nonce: DYQL0g .. section: Library -Add 3 new clock identifiers: :data:`time.CLOCK_BOOTTIME`, -:data:`time.CLOCK_PROF` and :data:`time.CLOCK_UPTIME`. +Add 3 new clock identifiers: :const:`time.CLOCK_BOOTTIME`, +:const:`time.CLOCK_PROF` and :const:`time.CLOCK_UPTIME`. .. diff --git a/Misc/NEWS.d/3.8.0a1.rst b/Misc/NEWS.d/3.8.0a1.rst index c9ca4b8404e67..5f1a0142ae778 100644 --- a/Misc/NEWS.d/3.8.0a1.rst +++ b/Misc/NEWS.d/3.8.0a1.rst @@ -1934,7 +1934,7 @@ failure. .. nonce: _ct_0H .. section: Library -The :data:`time.CLOCK_UPTIME_RAW` constant is now available for macOS 10.12. +The :const:`time.CLOCK_UPTIME_RAW` constant is now available for macOS 10.12. .. diff --git a/Misc/NEWS.d/3.8.0a4.rst b/Misc/NEWS.d/3.8.0a4.rst index d6fde41b7b11c..17bbac6238913 100644 --- a/Misc/NEWS.d/3.8.0a4.rst +++ b/Misc/NEWS.d/3.8.0a4.rst @@ -955,7 +955,7 @@ Add a new :mod:`_testinternalcapi` module to test the internal C API. .. section: Tests Fix ``test_imap4_host_default_value()`` of ``test_imaplib``: catch also -:data:`errno.ENETUNREACH` error. +:const:`errno.ENETUNREACH` error. .. diff --git a/Misc/NEWS.d/3.9.0a1.rst b/Misc/NEWS.d/3.9.0a1.rst index 5a929d481a3d5..97e07a643e59f 100644 --- a/Misc/NEWS.d/3.9.0a1.rst +++ b/Misc/NEWS.d/3.9.0a1.rst @@ -1164,7 +1164,7 @@ defines them with eponymous methods. .. nonce: bmhquU .. section: Library -Add :data:`os.P_PIDFD` constant, which may be passed to :func:`os.waitid` to +Add :const:`os.P_PIDFD` constant, which may be passed to :func:`os.waitid` to wait on a Linux process file descriptor. .. @@ -1193,8 +1193,8 @@ Expose the Linux ``pidfd_open`` syscall as :func:`os.pidfd_open`. .. nonce: 7jvYFA .. section: Library -Added constants :data:`~fcntl.F_OFD_GETLK`, :data:`~fcntl.F_OFD_SETLK` and -:data:`~fcntl.F_OFD_SETLKW` to the :mod:`fcntl` module. Patch by Dong-hee +Added constants :const:`~fcntl.F_OFD_GETLK`, :const:`~fcntl.F_OFD_SETLK` and +:const:`~fcntl.F_OFD_SETLKW` to the :mod:`fcntl` module. Patch by Dong-hee Na. .. @@ -1283,7 +1283,7 @@ Fixed erroneous equality comparison in statistics.NormalDist(). .. nonce: 86ExWB .. section: Library -Added :data:`~os.CLD_KILLED` and :data:`~os.CLD_STOPPED` for +Added :const:`~os.CLD_KILLED` and :const:`~os.CLD_STOPPED` for :attr:`si_code`. Patch by Dong-hee Na. .. @@ -1355,8 +1355,8 @@ objects, patch by Samuel Colvin. .. nonce: 9w-IGF .. section: Library -Add missing :data:`stat.S_IFDOOR`, :data:`stat.S_IFPORT`, -:data:`stat.S_IFWHT`, :func:`stat.S_ISDOOR`, :func:`stat.S_ISPORT`, and +Add missing :const:`stat.S_IFDOOR`, :const:`stat.S_IFPORT`, +:const:`stat.S_IFWHT`, :func:`stat.S_ISDOOR`, :func:`stat.S_ISPORT`, and :func:`stat.S_ISWHT` values to the Python implementation of :mod:`stat`. .. @@ -4983,7 +4983,7 @@ set to CP_UTF7 or CP_UTF8. .. nonce: -0g2O3 .. section: Windows -Make :data:`winreg.REG_MULTI_SZ` support zero-length strings. +Make :const:`winreg.REG_MULTI_SZ` support zero-length strings. .. diff --git a/Misc/NEWS.d/3.9.0a6.rst b/Misc/NEWS.d/3.9.0a6.rst index af2cc7c3e9788..fec792a998bf9 100644 --- a/Misc/NEWS.d/3.9.0a6.rst +++ b/Misc/NEWS.d/3.9.0a6.rst @@ -680,7 +680,7 @@ child process, reset the lock to the unlocked state. Rename also the private .. nonce: kIjVge .. section: Library -Expose :data:`~socket.CAN_RAW_JOIN_FILTERS` in the :mod:`socket` module. +Expose :const:`~socket.CAN_RAW_JOIN_FILTERS` in the :mod:`socket` module. .. @@ -735,7 +735,7 @@ number of groups. For other implementations, double the group list size. .. nonce: HFpHZS .. section: Library -Add :data:`time.CLOCK_TAI` constant if the operating system support it. +Add :const:`time.CLOCK_TAI` constant if the operating system support it. .. From webhook-mailer at python.org Fri Jul 21 07:51:04 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 21 Jul 2023 11:51:04 -0000 Subject: [Python-checkins] [3.12] gh-106892: Use roles :data: and :const: for referencing module variables (GH-106894) (GH-106954) Message-ID: https://github.com/python/cpython/commit/ffc7678f4683a63180c1321334c76e9b3e09b0a5 commit: ffc7678f4683a63180c1321334c76e9b3e09b0a5 branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-21T14:51:00+03:00 summary: [3.12] gh-106892: Use roles :data: and :const: for referencing module variables (GH-106894) (GH-106954) (cherry picked from commit d036db728ea3d54509cbad06df74e2d9a31fbec8) files: M Doc/c-api/import.rst M Doc/library/__main__.rst M Doc/library/asyncio-subprocess.rst M Doc/library/compileall.rst M Doc/library/devmode.rst M Doc/library/filecmp.rst M Doc/library/ftplib.rst M Doc/library/functions.rst M Doc/library/gc.rst M Doc/library/gzip.rst M Doc/library/importlib.resources.abc.rst M Doc/library/importlib.rst M Doc/library/json.rst M Doc/library/logging.handlers.rst M Doc/library/os.path.rst M Doc/library/os.rst M Doc/library/platform.rst M Doc/library/shutil.rst M Doc/library/stdtypes.rst M Doc/library/subprocess.rst M Doc/library/sys.rst M Doc/library/tarfile.rst M Doc/library/test.rst M Doc/library/tkinter.rst M Doc/library/unittest.rst M Doc/reference/datamodel.rst M Doc/using/windows.rst M Doc/whatsnew/2.5.rst M Doc/whatsnew/3.1.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.2.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.4.rst M Doc/whatsnew/3.6.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.8.0a4.rst M Misc/NEWS.d/3.9.0a5.rst diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 79843ba521ab9..d32c09ef06c9e 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -126,9 +126,9 @@ Importing Modules read from a Python bytecode file or obtained from the built-in function :func:`compile`, load the module. Return a new reference to the module object, or ``NULL`` with an exception set if an error occurred. *name* - is removed from :attr:`sys.modules` in error cases, even if *name* was already - in :attr:`sys.modules` on entry to :c:func:`PyImport_ExecCodeModule`. Leaving - incompletely initialized modules in :attr:`sys.modules` is dangerous, as imports of + is removed from :data:`sys.modules` in error cases, even if *name* was already + in :data:`sys.modules` on entry to :c:func:`PyImport_ExecCodeModule`. Leaving + incompletely initialized modules in :data:`sys.modules` is dangerous, as imports of such modules have no way to know that the module object is an unknown (and probably damaged with respect to the module author's intents) state. diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index d29cbdff7830c..fd60d92d4eb0f 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -336,12 +336,12 @@ Note that importing ``__main__`` doesn't cause any issues with unintentionally running top-level code meant for script use which is put in the ``if __name__ == "__main__"`` block of the ``start`` module. Why does this work? -Python inserts an empty ``__main__`` module in :attr:`sys.modules` at +Python inserts an empty ``__main__`` module in :data:`sys.modules` at interpreter startup, and populates it by running top-level code. In our example this is the ``start`` module which runs line by line and imports ``namely``. In turn, ``namely`` imports ``__main__`` (which is really ``start``). That's an import cycle! Fortunately, since the partially populated ``__main__`` -module is present in :attr:`sys.modules`, Python passes that to ``namely``. +module is present in :data:`sys.modules`, Python passes that to ``namely``. See :ref:`Special considerations for __main__ ` in the import system's reference for details on how this works. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index afade9b0b5070..bf35b1cb798ae 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -68,7 +68,7 @@ Creating Subprocesses The *limit* argument sets the buffer limit for :class:`StreamReader` wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` - (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). + (if :const:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). Return a :class:`~asyncio.subprocess.Process` instance. @@ -86,7 +86,7 @@ Creating Subprocesses The *limit* argument sets the buffer limit for :class:`StreamReader` wrappers for :attr:`Process.stdout` and :attr:`Process.stderr` - (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). + (if :const:`subprocess.PIPE` is passed to *stdout* and *stderr* arguments). Return a :class:`~asyncio.subprocess.Process` instance. diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst index 180f5b81c2b61..4226348a17240 100644 --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -141,9 +141,9 @@ There is no command-line option to control the optimization level used by the :func:`compile` function, because the Python interpreter itself already provides the option: :program:`python -O -m compileall`. -Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix` +Similarly, the :func:`compile` function respects the :data:`sys.pycache_prefix` setting. The generated bytecode cache will only be useful if :func:`compile` is -run with the same :attr:`sys.pycache_prefix` (if any) that will be used at +run with the same :data:`sys.pycache_prefix` (if any) that will be used at runtime. Public functions diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst index 80ac13b116c1d..914aa45cf9cbc 100644 --- a/Doc/library/devmode.rst +++ b/Doc/library/devmode.rst @@ -81,7 +81,7 @@ Effects of the Python Development Mode: ignored for empty strings. * The :class:`io.IOBase` destructor logs ``close()`` exceptions. -* Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to +* Set the :attr:`~sys.flags.dev_mode` attribute of :data:`sys.flags` to ``True``. The Python Development Mode does not enable the :mod:`tracemalloc` module by diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst index 83e9e14ddcacd..0efb4897a1eb8 100644 --- a/Doc/library/filecmp.rst +++ b/Doc/library/filecmp.rst @@ -74,7 +74,7 @@ The :class:`dircmp` class Construct a new directory comparison object, to compare the directories *a* and *b*. *ignore* is a list of names to ignore, and defaults to - :attr:`filecmp.DEFAULT_IGNORES`. *hide* is a list of names to hide, and + :const:`filecmp.DEFAULT_IGNORES`. *hide* is a list of names to hide, and defaults to ``[os.curdir, os.pardir]``. The :class:`dircmp` class compares files by doing *shallow* comparisons diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst index cd6e2d38b0286..d1fe6414ea020 100644 --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -431,7 +431,7 @@ FTP_TLS Objects .. attribute:: FTP_TLS.ssl_version - The SSL version to use (defaults to :attr:`ssl.PROTOCOL_SSLv23`). + The SSL version to use (defaults to :data:`ssl.PROTOCOL_SSLv23`). .. method:: FTP_TLS.auth() diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index d8091f0b093aa..2688f43f9b4ff 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1231,7 +1231,7 @@ are always available. They are listed here in alphabetical order. * Binary files are buffered in fixed-size chunks; the size of the buffer is chosen using a heuristic trying to determine the underlying device's "block - size" and falling back on :attr:`io.DEFAULT_BUFFER_SIZE`. On many systems, + size" and falling back on :const:`io.DEFAULT_BUFFER_SIZE`. On many systems, the buffer will typically be 4096 or 8192 bytes long. * "Interactive" text files (files for which :meth:`~io.IOBase.isatty` diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst index 0961ca4aaa942..331c071cda769 100644 --- a/Doc/library/gc.rst +++ b/Doc/library/gc.rst @@ -260,7 +260,7 @@ values but should not rebind them): .. versionchanged:: 3.4 Following :pep:`442`, objects with a :meth:`~object.__del__` method don't end - up in :attr:`gc.garbage` anymore. + up in :data:`gc.garbage` anymore. .. data:: callbacks diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst index 06cbd2567a0bc..979b39a3a5abb 100644 --- a/Doc/library/gzip.rst +++ b/Doc/library/gzip.rst @@ -268,7 +268,7 @@ Command line options .. cmdoption:: file - If *file* is not specified, read from :attr:`sys.stdin`. + If *file* is not specified, read from :data:`sys.stdin`. .. cmdoption:: --fast diff --git a/Doc/library/importlib.resources.abc.rst b/Doc/library/importlib.resources.abc.rst index 2d0f137ffc799..b0e75737137f2 100644 --- a/Doc/library/importlib.resources.abc.rst +++ b/Doc/library/importlib.resources.abc.rst @@ -130,7 +130,7 @@ suitable for reading (same as :attr:`pathlib.Path.open`). When opening as text, accepts encoding parameters such as those - accepted by :attr:`io.TextIOWrapper`. + accepted by :class:`io.TextIOWrapper`. .. method:: read_bytes() diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index 65aaad0df9ee6..a14e5a1a1a350 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -372,7 +372,7 @@ ABC hierarchy:: The list of locations where the package's submodules will be found. Most of the time this is a single directory. The import system passes this attribute to ``__import__()`` and to finders - in the same way as :attr:`sys.path` but just for the package. + in the same way as :data:`sys.path` but just for the package. It is not set on non-package modules so it can be used as an indicator that the module is a package. @@ -609,7 +609,7 @@ ABC hierarchy:: automatically. When writing to the path fails because the path is read-only - (:attr:`errno.EACCES`/:exc:`PermissionError`), do not propagate the + (:const:`errno.EACCES`/:exc:`PermissionError`), do not propagate the exception. .. versionchanged:: 3.4 @@ -843,7 +843,7 @@ find and load modules. .. classmethod:: path_hook(*loader_details) - A class method which returns a closure for use on :attr:`sys.path_hooks`. + A class method which returns a closure for use on :data:`sys.path_hooks`. An instance of :class:`FileFinder` is returned by the closure using the path argument given to the closure directly and *loader_details* indirectly. @@ -1184,10 +1184,10 @@ an :term:`importer`. .. function:: find_spec(name, package=None) Find the :term:`spec ` for a module, optionally relative to - the specified **package** name. If the module is in :attr:`sys.modules`, + the specified **package** name. If the module is in :data:`sys.modules`, then ``sys.modules[name].__spec__`` is returned (unless the spec would be ``None`` or is not set, in which case :exc:`ValueError` is raised). - Otherwise a search using :attr:`sys.meta_path` is done. ``None`` is + Otherwise a search using :data:`sys.meta_path` is done. ``None`` is returned if no spec is found. If **name** is for a submodule (contains a dot), the parent module is @@ -1259,7 +1259,7 @@ an :term:`importer`. :meth:`~importlib.abc.Loader.create_module` method must return ``None`` or a type for which its ``__class__`` attribute can be mutated along with not using :term:`slots <__slots__>`. Finally, modules which substitute the object - placed into :attr:`sys.modules` will not work as there is no way to properly + placed into :data:`sys.modules` will not work as there is no way to properly replace the module references throughout the interpreter safely; :exc:`ValueError` is raised if such a substitution is detected. @@ -1383,9 +1383,9 @@ For deep customizations of import, you typically want to implement an :term:`importer`. This means managing both the :term:`finder` and :term:`loader` side of things. For finders there are two flavours to choose from depending on your needs: a :term:`meta path finder` or a :term:`path entry finder`. The -former is what you would put on :attr:`sys.meta_path` while the latter is what -you create using a :term:`path entry hook` on :attr:`sys.path_hooks` which works -with :attr:`sys.path` entries to potentially create a finder. This example will +former is what you would put on :data:`sys.meta_path` while the latter is what +you create using a :term:`path entry hook` on :data:`sys.path_hooks` which works +with :data:`sys.path` entries to potentially create a finder. This example will show you how to register your own importers so that import will use them (for creating an importer for yourself, read the documentation for the appropriate classes defined within this package):: diff --git a/Doc/library/json.rst b/Doc/library/json.rst index 5383614575c21..35a08995487c1 100644 --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -683,7 +683,7 @@ The :mod:`json.tool` module provides a simple command line interface to validate and pretty-print JSON objects. If the optional ``infile`` and ``outfile`` arguments are not -specified, :attr:`sys.stdin` and :attr:`sys.stdout` will be used respectively: +specified, :data:`sys.stdin` and :data:`sys.stdout` will be used respectively: .. code-block:: shell-session @@ -721,12 +721,12 @@ Command line options } ] - If *infile* is not specified, read from :attr:`sys.stdin`. + If *infile* is not specified, read from :data:`sys.stdin`. .. cmdoption:: outfile Write the output of the *infile* to the given *outfile*. Otherwise, write it - to :attr:`sys.stdout`. + to :data:`sys.stdout`. .. cmdoption:: --sort-keys diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index d4429d3d0a4f7..47bfe4ff7f90e 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -1051,8 +1051,8 @@ possible, while any potentially slow operations (such as sending an email via occur (e.g. because a bounded queue has filled up), the :meth:`~logging.Handler.handleError` method is called to handle the error. This can result in the record silently being dropped (if - :attr:`logging.raiseExceptions` is ``False``) or a message printed to - ``sys.stderr`` (if :attr:`logging.raiseExceptions` is ``True``). + :data:`logging.raiseExceptions` is ``False``) or a message printed to + ``sys.stderr`` (if :data:`logging.raiseExceptions` is ``True``). .. method:: prepare(record) diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst index 3a668e28f2e26..6f9e0853bc894 100644 --- a/Doc/library/os.path.rst +++ b/Doc/library/os.path.rst @@ -410,7 +410,7 @@ the :mod:`glob` module.) *start*. On Windows, :exc:`ValueError` is raised when *path* and *start* are on different drives. - *start* defaults to :attr:`os.curdir`. + *start* defaults to :data:`os.curdir`. .. availability:: Unix, Windows. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 76f797ac2c4cb..127d1616388b3 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -60,7 +60,7 @@ Notes on the availability of these functions: ``'java'``. .. seealso:: - :attr:`sys.platform` has a finer granularity. :func:`os.uname` gives + :data:`sys.platform` has a finer granularity. :func:`os.uname` gives system-dependent version information. The :mod:`platform` module provides detailed checks for the diff --git a/Doc/library/platform.rst b/Doc/library/platform.rst index 69c4dfc422c98..ec2a7ebd5d6e0 100644 --- a/Doc/library/platform.rst +++ b/Doc/library/platform.rst @@ -46,7 +46,7 @@ Cross Platform universal files containing multiple architectures. To get at the "64-bitness" of the current interpreter, it is more - reliable to query the :attr:`sys.maxsize` attribute:: + reliable to query the :data:`sys.maxsize` attribute:: is_64bits = sys.maxsize > 2**32 diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 7f408be233682..5bd80b10a6580 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -431,7 +431,7 @@ Directory and files operations determining if the file exists and executable. When no *path* is specified, the results of :func:`os.environ` are used, - returning either the "PATH" value or a fallback of :attr:`os.defpath`. + returning either the "PATH" value or a fallback of :data:`os.defpath`. On Windows, the current directory is prepended to the *path* if *mode* does not include ``os.X_OK``. When the *mode* does include ``os.X_OK``, the diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 092f2719379e8..d0233e3f35394 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -5629,7 +5629,7 @@ From code, you can inspect the current limit and set a new one using these a getter and setter for the interpreter-wide limit. Subinterpreters have their own limit. -Information about the default and minimum can be found in :attr:`sys.int_info`: +Information about the default and minimum can be found in :data:`sys.int_info`: * :data:`sys.int_info.default_max_str_digits ` is the compiled-in default limit. diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 738e611c05adb..db05cb294d93f 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1610,7 +1610,7 @@ improves performance. If you ever encounter a presumed highly unusual situation where you need to prevent ``vfork()`` from being used by Python, you can set the -:attr:`subprocess._USE_VFORK` attribute to a false value. +:const:`subprocess._USE_VFORK` attribute to a false value. :: @@ -1618,7 +1618,7 @@ prevent ``vfork()`` from being used by Python, you can set the Setting this has no impact on use of ``posix_spawn()`` which could use ``vfork()`` internally within its libc implementation. There is a similar -:attr:`subprocess._USE_POSIX_SPAWN` attribute if you need to prevent use of +:const:`subprocess._USE_POSIX_SPAWN` attribute if you need to prevent use of that. :: diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 911d73b546b5c..c01d2d4527152 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -166,7 +166,7 @@ always available. Python interpreter. (This information is not available in any other way --- ``modules.keys()`` only lists the imported modules.) - See also the :attr:`sys.stdlib_module_names` list. + See also the :data:`sys.stdlib_module_names` list. .. function:: call_tracing(func, args) @@ -1287,20 +1287,20 @@ always available. ================ =========================== .. versionchanged:: 3.3 - On Linux, :attr:`sys.platform` doesn't contain the major version anymore. + On Linux, :data:`sys.platform` doesn't contain the major version anymore. It is always ``'linux'``, instead of ``'linux2'`` or ``'linux3'``. Since older Python versions include the version number, it is recommended to always use the ``startswith`` idiom presented above. .. versionchanged:: 3.8 - On AIX, :attr:`sys.platform` doesn't contain the major version anymore. + On AIX, :data:`sys.platform` doesn't contain the major version anymore. It is always ``'aix'``, instead of ``'aix5'`` or ``'aix7'``. Since older Python versions include the version number, it is recommended to always use the ``startswith`` idiom presented above. .. seealso:: - :attr:`os.name` has a coarser granularity. :func:`os.uname` gives + :data:`os.name` has a coarser granularity. :func:`os.uname` gives system-dependent version information. The :mod:`platform` module provides detailed checks for the @@ -1743,7 +1743,7 @@ always available. ``email.mime`` sub-package and the ``email.message`` sub-module are not listed. - See also the :attr:`sys.builtin_module_names` list. + See also the :data:`sys.builtin_module_names` list. .. versionadded:: 3.10 diff --git a/Doc/library/tarfile.rst b/Doc/library/tarfile.rst index 6a5947b788fda..7a5bdaaf065fe 100644 --- a/Doc/library/tarfile.rst +++ b/Doc/library/tarfile.rst @@ -933,7 +933,7 @@ reused in custom filters: Implements the ``'tar'`` filter. - - Strip leading slashes (``/`` and :attr:`os.sep`) from filenames. + - Strip leading slashes (``/`` and :data:`os.sep`) from filenames. - :ref:`Refuse ` to extract files with absolute paths (in case the name is absolute even after stripping slashes, e.g. ``C:/foo`` on Windows). @@ -942,7 +942,7 @@ reused in custom filters: path (after following symlinks) would end up outside the destination. This raises :class:`~tarfile.OutsideDestinationError`. - Clear high mode bits (setuid, setgid, sticky) and group/other write bits - (:attr:`~stat.S_IWGRP`|:attr:`~stat.S_IWOTH`). + (:const:`~stat.S_IWGRP`|:const:`~stat.S_IWOTH`). Return the modified ``TarInfo`` member. @@ -967,10 +967,10 @@ reused in custom filters: - For regular files, including hard links: - Set the owner read and write permissions - (:attr:`~stat.S_IRUSR`|:attr:`~stat.S_IWUSR`). + (:const:`~stat.S_IRUSR`|:const:`~stat.S_IWUSR`). - Remove the group & other executable permission - (:attr:`~stat.S_IXGRP`|:attr:`~stat.S_IXOTH`) - if the owner doesn?t have it (:attr:`~stat.S_IXUSR`). + (:const:`~stat.S_IXGRP`|:const:`~stat.S_IXOTH`) + if the owner doesn?t have it (:const:`~stat.S_IXUSR`). - For other files (directories), set ``mode`` to ``None``, so that extraction methods skip applying permission bits. diff --git a/Doc/library/test.rst b/Doc/library/test.rst index d69a3e326a411..de60151bb32ce 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -1040,7 +1040,7 @@ The :mod:`test.support` module defines the following classes: `SetErrorMode `_. On UNIX, :func:`resource.setrlimit` is used to set - :attr:`resource.RLIMIT_CORE`'s soft limit to 0 to prevent coredump file + :const:`resource.RLIMIT_CORE`'s soft limit to 0 to prevent coredump file creation. On both platforms, the old value is restored by :meth:`__exit__`. diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index c8e4317be7587..2aa3442270387 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -163,7 +163,7 @@ the modern themed widget set and API:: interpreter and calls :func:`exec` on the contents of :file:`.{className}.py` and :file:`.{baseName}.py`. The path for the profile files is the :envvar:`HOME` environment variable or, if that - isn't defined, then :attr:`os.curdir`. + isn't defined, then :data:`os.curdir`. .. attribute:: tk diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index b26e6c0e6bc02..518bf6b13bad5 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1134,7 +1134,7 @@ Test cases If given, *level* should be either a numeric logging level or its string equivalent (for example either ``"ERROR"`` or - :attr:`logging.ERROR`). The default is :attr:`logging.INFO`. + :const:`logging.ERROR`). The default is :const:`logging.INFO`. The test passes if at least one message emitted inside the ``with`` block matches the *logger* and *level* conditions, otherwise it fails. @@ -1175,7 +1175,7 @@ Test cases If given, *level* should be either a numeric logging level or its string equivalent (for example either ``"ERROR"`` or - :attr:`logging.ERROR`). The default is :attr:`logging.INFO`. + :const:`logging.ERROR`). The default is :const:`logging.INFO`. Unlike :meth:`assertLogs`, nothing will be returned by the context manager. diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index e8f9775dd33ce..ce36bff89424c 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2499,8 +2499,8 @@ through the object's keys; for sequences, it should iterate through the values. .. impl-detail:: - In CPython, the length is required to be at most :attr:`sys.maxsize`. - If the length is larger than :attr:`!sys.maxsize` some features (such as + In CPython, the length is required to be at most :data:`sys.maxsize`. + If the length is larger than :data:`!sys.maxsize` some features (such as :func:`len`) may raise :exc:`OverflowError`. To prevent raising :exc:`!OverflowError` by truth value testing, an object must define a :meth:`__bool__` method. diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index 43e3c72f3e1cd..b7e4b2b48a601 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -1201,7 +1201,7 @@ non-standard paths in the registry and user site-packages. Modules specified in the registry under ``Modules`` (not ``PythonPath``) may be imported by :class:`importlib.machinery.WindowsRegistryFinder`. This finder is enabled on Windows in 3.6.0 and earlier, but may need to - be explicitly added to :attr:`sys.meta_path` in the future. + be explicitly added to :data:`sys.meta_path` in the future. Additional modules ================== diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index dcfaef6ed2949..b410fe1ce3f08 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -1448,10 +1448,10 @@ complete list of changes, or look through the SVN logs for all the details. return times that are precise to fractions of a second; not all systems support such precision.) - Constants named :attr:`os.SEEK_SET`, :attr:`os.SEEK_CUR`, and - :attr:`os.SEEK_END` have been added; these are the parameters to the + Constants named :const:`os.SEEK_SET`, :const:`os.SEEK_CUR`, and + :const:`os.SEEK_END` have been added; these are the parameters to the :func:`os.lseek` function. Two new constants for locking are - :attr:`os.O_SHLOCK` and :attr:`os.O_EXLOCK`. + :const:`os.O_SHLOCK` and :const:`os.O_EXLOCK`. Two new functions, :func:`wait3` and :func:`wait4`, were added. They're similar the :func:`waitpid` function which waits for a child process to exit and returns @@ -1602,7 +1602,7 @@ complete list of changes, or look through the SVN logs for all the details. * The :mod:`unicodedata` module has been updated to use version 4.1.0 of the Unicode character database. Version 3.2.0 is required by some specifications, - so it's still available as :attr:`unicodedata.ucd_3_2_0`. + so it's still available as :data:`unicodedata.ucd_3_2_0`. * New module: the :mod:`uuid` module generates universally unique identifiers (UUIDs) according to :rfc:`4122`. The RFC defines several different UUID diff --git a/Doc/whatsnew/3.1.rst b/Doc/whatsnew/3.1.rst index fba8816bb243a..e4365d44928b5 100644 --- a/Doc/whatsnew/3.1.rst +++ b/Doc/whatsnew/3.1.rst @@ -370,7 +370,7 @@ New, Improved, and Deprecated Modules * The :mod:`io` module has three new constants for the :meth:`seek` method :data:`SEEK_SET`, :data:`SEEK_CUR`, and :data:`SEEK_END`. -* The :attr:`sys.version_info` tuple is now a named tuple:: +* The :data:`sys.version_info` tuple is now a named tuple:: >>> sys.version_info sys.version_info(major=3, minor=1, micro=0, releaselevel='alpha', serial=2) @@ -486,7 +486,7 @@ Changes to Python's build process and to the C API include: Apart from the performance improvements this change should be invisible to end users, with one exception: for testing and debugging purposes there's a - new :attr:`sys.int_info` that provides information about the + new :data:`sys.int_info` that provides information about the internal format, giving the number of bits per digit and the size in bytes of the C type used to store each digit:: diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 0585eccf198e8..4d9aed0d0f9cd 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -640,7 +640,7 @@ dataclasses datetime -------- -* Add :attr:`datetime.UTC`, a convenience alias for +* Add :const:`datetime.UTC`, a convenience alias for :attr:`datetime.timezone.utc`. (Contributed by Kabir Kwatra in :gh:`91973`.) * :meth:`datetime.date.fromisoformat`, :meth:`datetime.time.fromisoformat` and diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index b4a3fd5c98291..9d05352f3b887 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1042,7 +1042,7 @@ Deprecated datetimes in UTC: respectively, call :meth:`~datetime.datetime.now` and :meth:`~datetime.datetime.fromtimestamp` with the *tz* parameter set to - :attr:`datetime.UTC`. + :const:`datetime.UTC`. (Contributed by Paul Ganssle in :gh:`103857`.) Pending Removal in Python 3.13 diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index 504bbf6deda7c..a42775804637e 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -424,7 +424,7 @@ protocols, the users must to be able access the environment using native strings even though the underlying platform may have a different convention. To bridge this gap, the :mod:`wsgiref` module has a new function, :func:`wsgiref.handlers.read_environ` for transcoding CGI variables from -:attr:`os.environ` into native strings and returning a new dictionary. +:data:`os.environ` into native strings and returning a new dictionary. .. seealso:: @@ -485,7 +485,7 @@ Some smaller changes made to the core Python language are: * The interpreter can now be started with a quiet option, ``-q``, to prevent the copyright and version information from being displayed in the interactive - mode. The option can be introspected using the :attr:`sys.flags` attribute: + mode. The option can be introspected using the :data:`sys.flags` attribute: .. code-block:: shell-session @@ -568,7 +568,7 @@ Some smaller changes made to the core Python language are: * The internal :c:type:`structsequence` tool now creates subclasses of tuple. This means that C structures like those returned by :func:`os.stat`, - :func:`time.gmtime`, and :attr:`sys.version_info` now work like a + :func:`time.gmtime`, and :data:`sys.version_info` now work like a :term:`named tuple` and now work with functions and methods that expect a tuple as an argument. This is a big step forward in making the C structures as flexible as their pure Python counterparts: @@ -598,7 +598,7 @@ Some smaller changes made to the core Python language are: module, or on the command line. A :exc:`ResourceWarning` is issued at interpreter shutdown if the - :data:`gc.garbage` list isn't empty, and if :attr:`gc.DEBUG_UNCOLLECTABLE` is + :data:`gc.garbage` list isn't empty, and if :const:`gc.DEBUG_UNCOLLECTABLE` is set, all uncollectable objects are printed. This is meant to make the programmer aware that their code contains object finalization issues. @@ -623,7 +623,7 @@ Some smaller changes made to the core Python language are: :class:`collections.Sequence` :term:`abstract base class`. As a result, the language will have a more uniform API. In addition, :class:`range` objects now support slicing and negative indices, even with values larger than - :attr:`sys.maxsize`. This makes *range* more interoperable with lists:: + :data:`sys.maxsize`. This makes *range* more interoperable with lists:: >>> range(0, 100, 2).count(10) 1 @@ -1007,13 +1007,13 @@ datetime and time after 1900. The new supported year range is from 1000 to 9999 inclusive. * Whenever a two-digit year is used in a time tuple, the interpretation has been - governed by :attr:`time.accept2dyear`. The default is ``True`` which means that + governed by :data:`time.accept2dyear`. The default is ``True`` which means that for a two-digit year, the century is guessed according to the POSIX rules governing the ``%y`` strptime format. Starting with Py3.2, use of the century guessing heuristic will emit a :exc:`DeprecationWarning`. Instead, it is recommended that - :attr:`time.accept2dyear` be set to ``False`` so that large date ranges + :data:`time.accept2dyear` be set to ``False`` so that large date ranges can be used without guesswork:: >>> import time, warnings @@ -1031,7 +1031,7 @@ datetime and time 'Fri Jan 1 12:34:56 11' Several functions now have significantly expanded date ranges. When - :attr:`time.accept2dyear` is false, the :func:`time.asctime` function will + :data:`time.accept2dyear` is false, the :func:`time.asctime` function will accept any year that fits in a C int, while the :func:`time.mktime` and :func:`time.strftime` functions will accept the full range supported by the corresponding operating system functions. @@ -1194,11 +1194,11 @@ can be set to "$" for the shell-style formatting provided by If no configuration is set-up before a logging event occurs, there is now a default configuration using a :class:`~logging.StreamHandler` directed to -:attr:`sys.stderr` for events of ``WARNING`` level or higher. Formerly, an +:data:`sys.stderr` for events of ``WARNING`` level or higher. Formerly, an event occurring before a configuration was set-up would either raise an exception or silently drop the event depending on the value of -:attr:`logging.raiseExceptions`. The new default handler is stored in -:attr:`logging.lastResort`. +:data:`logging.raiseExceptions`. The new default handler is stored in +:data:`logging.lastResort`. The use of filters has been simplified. Instead of creating a :class:`~logging.Filter` object, the predicate can be any Python callable that @@ -1300,7 +1300,7 @@ values are equal (:issue:`8188`):: hash(Decimal("1.5")) == hash(complex(1.5, 0)) Some of the hashing details are exposed through a new attribute, -:attr:`sys.hash_info`, which describes the bit width of the hash value, the +:data:`sys.hash_info`, which describes the bit width of the hash value, the prime modulus, the hash values for *infinity* and *nan*, and the multiplier used for the imaginary part of a number: @@ -1388,7 +1388,7 @@ select ------ The :mod:`select` module now exposes a new, constant attribute, -:attr:`~select.PIPE_BUF`, which gives the minimum number of bytes which are +:const:`~select.PIPE_BUF`, which gives the minimum number of bytes which are guaranteed not to block when :func:`select.select` says a pipe is ready for writing. @@ -1529,7 +1529,7 @@ filenames: b'Sehensw\xc3\xbcrdigkeiten' Some operating systems allow direct access to encoded bytes in the -environment. If so, the :attr:`os.supports_bytes_environ` constant will be +environment. If so, the :const:`os.supports_bytes_environ` constant will be true. For direct access to encoded environment variables (if available), @@ -2302,7 +2302,7 @@ turtledemo The demonstration code for the :mod:`turtle` module was moved from the *Demo* directory to main library. It includes over a dozen sample scripts with -lively displays. Being on :attr:`sys.path`, it can now be run directly +lively displays. Being on :data:`sys.path`, it can now be run directly from the command-line: .. code-block:: shell-session @@ -2566,7 +2566,7 @@ Changes to Python's build process and to the C API include: (:issue:`2443`). * A new C API function :c:func:`PySys_SetArgvEx` allows an embedded interpreter - to set :attr:`sys.argv` without also modifying :attr:`sys.path` + to set :data:`sys.argv` without also modifying :data:`sys.path` (:issue:`5753`). * :c:macro:`PyEval_CallObject` is now only available in macro form. The diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index b3f4e9ac743fb..765ba60d28c3a 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -648,7 +648,7 @@ PEP 421: Adding sys.implementation A new attribute on the :mod:`sys` module exposes details specific to the implementation of the currently running interpreter. The initial set of -attributes on :attr:`sys.implementation` are ``name``, ``version``, +attributes on :data:`sys.implementation` are ``name``, ``version``, ``hexversion``, and ``cache_tag``. The intention of ``sys.implementation`` is to consolidate into one namespace @@ -719,7 +719,7 @@ and does not enforce any method requirements. In terms of finders, :class:`importlib.machinery.FileFinder` exposes the mechanism used to search for source and bytecode files of a module. Previously -this class was an implicit member of :attr:`sys.path_hooks`. +this class was an implicit member of :data:`sys.path_hooks`. For loaders, the new abstract base class :class:`importlib.abc.FileLoader` helps write a loader that uses the file system as the storage mechanism for a module's @@ -735,7 +735,7 @@ provide the full name of the module now instead of just the tail end of the module's name. The :func:`importlib.invalidate_caches` function will now call the method with -the same name on all finders cached in :attr:`sys.path_importer_cache` to help +the same name on all finders cached in :data:`sys.path_importer_cache` to help clean up any stored state as necessary. Visible Changes @@ -745,8 +745,8 @@ For potential required changes to code, see the `Porting Python code`_ section. Beyond the expanse of what :mod:`importlib` now exposes, there are other -visible changes to import. The biggest is that :attr:`sys.meta_path` and -:attr:`sys.path_hooks` now store all of the meta path finders and path entry +visible changes to import. The biggest is that :data:`sys.meta_path` and +:data:`sys.path_hooks` now store all of the meta path finders and path entry hooks used by import. Previously the finders were implicit and hidden within the C code of import instead of being directly exposed. This means that one can now easily remove or change the order of the various finders to fit one's needs. @@ -761,9 +761,9 @@ Loaders are also now expected to set the ``__package__`` attribute from :pep:`366`. Once again, import itself is already setting this on all loaders from :mod:`importlib` and import itself is setting the attribute post-load. -``None`` is now inserted into :attr:`sys.path_importer_cache` when no finder -can be found on :attr:`sys.path_hooks`. Since :class:`!imp.NullImporter` is not -directly exposed on :attr:`sys.path_hooks` it could no longer be relied upon to +``None`` is now inserted into :data:`sys.path_importer_cache` when no finder +can be found on :data:`sys.path_hooks`. Since :class:`!imp.NullImporter` is not +directly exposed on :data:`sys.path_hooks` it could no longer be relied upon to always be available to use as a value representing no finder found. All other changes relate to semantic changes which should be taken into @@ -1952,7 +1952,7 @@ ssl * You can query the SSL compression algorithm used by an SSL socket, thanks to its new :meth:`~ssl.SSLSocket.compression` method. The new attribute - :attr:`~ssl.OP_NO_COMPRESSION` can be used to disable compression. + :const:`~ssl.OP_NO_COMPRESSION` can be used to disable compression. (Contributed by Antoine Pitrou in :issue:`13634`.) * Support has been added for the Next Protocol Negotiation extension using @@ -1966,7 +1966,7 @@ ssl * The :func:`~ssl.get_server_certificate` function now supports IPv6. (Contributed by Charles-Fran?ois Natali in :issue:`11811`.) -* New attribute :attr:`~ssl.OP_CIPHER_SERVER_PREFERENCE` allows setting +* New attribute :const:`~ssl.OP_CIPHER_SERVER_PREFERENCE` allows setting SSLv3 server sockets to use the server's cipher ordering preference rather than the client's (:issue:`13635`). @@ -2141,7 +2141,7 @@ New attribute :attr:`zlib.Decompress.eof` makes it possible to distinguish between a properly formed compressed stream and an incomplete or truncated one. (Contributed by Nadeem Vawda in :issue:`12646`.) -New attribute :attr:`zlib.ZLIB_RUNTIME_VERSION` reports the version string of +New attribute :const:`zlib.ZLIB_RUNTIME_VERSION` reports the version string of the underlying ``zlib`` library that is loaded at runtime. (Contributed by Torsten Landschoff in :issue:`12306`.) @@ -2378,16 +2378,16 @@ Porting Python code * :func:`__import__` no longer allows one to use an index value other than 0 for top-level modules. E.g. ``__import__('sys', level=1)`` is now an error. -* Because :attr:`sys.meta_path` and :attr:`sys.path_hooks` now have finders on +* Because :data:`sys.meta_path` and :data:`sys.path_hooks` now have finders on them by default, you will most likely want to use :meth:`list.insert` instead of :meth:`list.append` to add to those lists. -* Because ``None`` is now inserted into :attr:`sys.path_importer_cache`, if you +* Because ``None`` is now inserted into :data:`sys.path_importer_cache`, if you are clearing out entries in the dictionary of paths that do not have a finder, you will need to remove keys paired with values of ``None`` **and** :class:`!imp.NullImporter` to be backwards-compatible. This will lead to extra overhead on older versions of Python that re-insert ``None`` into - :attr:`sys.path_importer_cache` where it represents the use of implicit + :data:`sys.path_importer_cache` where it represents the use of implicit finders, but semantically it should not change anything. * :class:`!importlib.abc.Finder` no longer specifies a ``find_module()`` abstract @@ -2445,7 +2445,7 @@ Porting Python code error instead of sleeping forever. It has always raised an error on posix. * The ``ast.__version__`` constant has been removed. If you need to - make decisions affected by the AST version, use :attr:`sys.version_info` + make decisions affected by the AST version, use :data:`sys.version_info` to make the decision. * Code that used to work around the fact that the :mod:`threading` module used diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst index d72163b249041..acb662188b05a 100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1323,14 +1323,14 @@ ability to query or set the resource limits for processes other than the one making the call. (Contributed by Christian Heimes in :issue:`16595`.) On Linux kernel version 2.6.36 or later, there are also some new -Linux specific constants: :attr:`~resource.RLIMIT_MSGQUEUE`, -:attr:`~resource.RLIMIT_NICE`, :attr:`~resource.RLIMIT_RTPRIO`, -:attr:`~resource.RLIMIT_RTTIME`, and :attr:`~resource.RLIMIT_SIGPENDING`. +Linux specific constants: :const:`~resource.RLIMIT_MSGQUEUE`, +:const:`~resource.RLIMIT_NICE`, :const:`~resource.RLIMIT_RTPRIO`, +:const:`~resource.RLIMIT_RTTIME`, and :const:`~resource.RLIMIT_SIGPENDING`. (Contributed by Christian Heimes in :issue:`19324`.) On FreeBSD version 9 and later, there some new FreeBSD specific constants: -:attr:`~resource.RLIMIT_SBSIZE`, :attr:`~resource.RLIMIT_SWAP`, and -:attr:`~resource.RLIMIT_NPTS`. (Contributed by Claudiu Popa in +:const:`~resource.RLIMIT_SBSIZE`, :const:`~resource.RLIMIT_SWAP`, and +:const:`~resource.RLIMIT_NPTS`. (Contributed by Claudiu Popa in :issue:`19343`.) @@ -1500,7 +1500,7 @@ implementation is required as most of the values aren't standardized and are platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.) The module supports new :mod:`~stat.ST_MODE` flags, :mod:`~stat.S_IFDOOR`, -:attr:`~stat.S_IFPORT`, and :attr:`~stat.S_IFWHT`. (Contributed by +:const:`~stat.S_IFPORT`, and :const:`~stat.S_IFWHT`. (Contributed by Christian Hiemes in :issue:`11016`.) @@ -1849,7 +1849,7 @@ Python's default implementation to a SipHash implementation on platforms that have a 64 bit data type. Any performance differences in comparison with the older FNV algorithm are trivial. -The PEP adds additional fields to the :attr:`sys.hash_info` named tuple to +The PEP adds additional fields to the :data:`sys.hash_info` named tuple to describe the hash algorithm in use by the currently executing binary. Otherwise, the PEP does not alter any existing CPython APIs. diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index 84b043856ed01..1093d24531505 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1388,7 +1388,7 @@ are treated as punctuation. site ---- -When specifying paths to add to :attr:`sys.path` in a ``.pth`` file, +When specifying paths to add to :data:`sys.path` in a ``.pth`` file, you may now specify file paths on top of directories (e.g. zip files). (Contributed by Wolfgang Langner in :issue:`26587`). @@ -2010,7 +2010,7 @@ been deprecated in previous versions of Python in favour of :meth:`importlib.abc.Loader.exec_module`. The :class:`importlib.machinery.WindowsRegistryFinder` class is now -deprecated. As of 3.6.0, it is still added to :attr:`sys.meta_path` by +deprecated. As of 3.6.0, it is still added to :data:`sys.meta_path` by default (on Windows), but this may change in future releases. os diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index ba3d86118b926..5df2cef7e898a 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1839,7 +1839,7 @@ Changes in Python behavior classes will affect their string representation. (Contributed by Serhiy Storchaka in :issue:`36793`.) -* On AIX, :attr:`sys.platform` doesn't contain the major version anymore. +* On AIX, :data:`sys.platform` doesn't contain the major version anymore. It is always ``'aix'``, instead of ``'aix3'`` .. ``'aix7'``. Since older Python versions include the version number, so it is recommended to always use ``sys.platform.startswith('aix')``. diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 36d942e8c064f..1f0fbbc219666 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -692,13 +692,13 @@ which has nanosecond resolution, rather than sys --- -Added a new :attr:`sys.platlibdir` attribute: name of the platform-specific +Added a new :data:`sys.platlibdir` attribute: name of the platform-specific library directory. It is used to build the path of standard library and the paths of installed extension modules. It is equal to ``"lib"`` on most platforms. On Fedora and SuSE, it is equal to ``"lib64"`` on 64-bit platforms. (Contributed by Jan Mat?jek, Mat?j Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.) -Previously, :attr:`sys.stderr` was block-buffered when non-interactive. Now +Previously, :data:`sys.stderr` was block-buffered when non-interactive. Now ``stderr`` defaults to always being line-buffered. (Contributed by Jendrik Seipp in :issue:`13601`.) @@ -1226,8 +1226,8 @@ Build Changes ============= * Added ``--with-platlibdir`` option to the ``configure`` script: name of the - platform-specific library directory, stored in the new :attr:`sys.platlibdir` - attribute. See :attr:`sys.platlibdir` attribute for more information. + platform-specific library directory, stored in the new :data:`sys.platlibdir` + attribute. See :data:`sys.platlibdir` attribute for more information. (Contributed by Jan Mat?jek, Mat?j Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.) diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index 5b6aff4d51fd6..d37ce84eadbcb 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -4171,7 +4171,7 @@ Add an index_pages parameter to support using non-default index page names. .. nonce: qtT3CE .. section: Library -Drop support for :class:`bytes` on :attr:`sys.path`. +Drop support for :class:`bytes` on :data:`sys.path`. .. diff --git a/Misc/NEWS.d/3.8.0a4.rst b/Misc/NEWS.d/3.8.0a4.rst index efe7b4d2ff0dd..524a05a7ae970 100644 --- a/Misc/NEWS.d/3.8.0a4.rst +++ b/Misc/NEWS.d/3.8.0a4.rst @@ -92,7 +92,7 @@ the field. .. nonce: wejLoC .. section: Core and Builtins -On AIX, :attr:`sys.platform` doesn't contain the major version anymore. +On AIX, :data:`sys.platform` doesn't contain the major version anymore. Always return ``'aix'``, instead of ``'aix3'`` .. ``'aix7'``. Since older Python versions include the version number, it is recommended to always use ``sys.platform.startswith('aix')``. Contributed by M. Felt. diff --git a/Misc/NEWS.d/3.9.0a5.rst b/Misc/NEWS.d/3.9.0a5.rst index 25342d21d8f0b..8a1219501e81b 100644 --- a/Misc/NEWS.d/3.9.0a5.rst +++ b/Misc/NEWS.d/3.9.0a5.rst @@ -582,7 +582,7 @@ Fix :mod:`json.tool` to catch :exc:`BrokenPipeError`. Patch by Dong-hee Na. Avoid a possible *"RuntimeError: dictionary changed size during iteration"* from :func:`inspect.getmodule` when it tried to loop through -:attr:`sys.modules`. +:data:`sys.modules`. .. @@ -989,7 +989,7 @@ modules are built. Add ``--with-platlibdir`` option to the configure script: name of the platform-specific library directory, stored in the new -:attr:`sys.platlibdir` attribute. It is used to build the path of +:data:`sys.platlibdir` attribute. It is used to build the path of platform-specific extension modules and the path of the standard library. It is equal to ``"lib"`` on most platforms. On Fedora and SuSE, it is equal to ``"lib64"`` on 64-bit platforms. Patch by Jan Mat?jek, Mat?j Cepl, From webhook-mailer at python.org Fri Jul 21 10:32:46 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Fri, 21 Jul 2023 14:32:46 -0000 Subject: [Python-checkins] gh-105699: Fix an Interned Strings Crasher (gh-106930) Message-ID: https://github.com/python/cpython/commit/87e7cb09e4258c4deb01a07dc52c1021907195d7 commit: 87e7cb09e4258c4deb01a07dc52c1021907195d7 branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-21T08:32:42-06:00 summary: gh-105699: Fix an Interned Strings Crasher (gh-106930) A static (process-global) str object must only have its "interned" state cleared when no longer interned in any interpreters. They are the only ones that can be shared by interpreters so we don't have to worry about any other str objects. We trigger clearing the state with the main interpreter, since no other interpreters may exist at that point and _PyUnicode_ClearInterned() is only called during interpreter finalization. We do not address here the fact that a string will only be interned in the first interpreter that interns it. In any subsequent interpreters str.state.interned is already set so _PyUnicode_InternInPlace() will skip it. That needs to be addressed separately from fixing the crasher. files: A Misc/NEWS.d/next/Core and Builtins/2023-07-20-15-15-57.gh-issue-105699.DdqHFg.rst M Objects/unicodeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-20-15-15-57.gh-issue-105699.DdqHFg.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-15-15-57.gh-issue-105699.DdqHFg.rst new file mode 100644 index 0000000000000..4a257c6282220 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-15-15-57.gh-issue-105699.DdqHFg.rst @@ -0,0 +1,3 @@ +Python no longer crashes due an infrequent race when initialzing +per-interpreter interned strings. The crash would manifest when the +interpreter was finalized. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index f543c0a65b49f..e8e8cab99c814 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -14818,6 +14818,7 @@ _PyUnicode_ClearInterned(PyInterpreterState *interp) PyObject *s, *ignored_value; while (PyDict_Next(interned, &pos, &s, &ignored_value)) { assert(PyUnicode_IS_READY(s)); + int shared = 0; switch (PyUnicode_CHECK_INTERNED(s)) { case SSTATE_INTERNED_IMMORTAL: // Skip the Immortal Instance check and restore @@ -14829,6 +14830,14 @@ _PyUnicode_ClearInterned(PyInterpreterState *interp) #endif break; case SSTATE_INTERNED_IMMORTAL_STATIC: + /* It is shared between interpreters, so we should unmark it + only when this is the last interpreter in which it's + interned. We immortalize all the statically initialized + strings during startup, so we can rely on the + main interpreter to be the last one. */ + if (!_Py_IsMainInterpreter(interp)) { + shared = 1; + } break; case SSTATE_INTERNED_MORTAL: /* fall through */ @@ -14837,7 +14846,9 @@ _PyUnicode_ClearInterned(PyInterpreterState *interp) default: Py_UNREACHABLE(); } - _PyUnicode_STATE(s).interned = SSTATE_NOT_INTERNED; + if (!shared) { + _PyUnicode_STATE(s).interned = SSTATE_NOT_INTERNED; + } } #ifdef INTERNED_STATS fprintf(stderr, From webhook-mailer at python.org Fri Jul 21 10:34:13 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Fri, 21 Jul 2023 14:34:13 -0000 Subject: [Python-checkins] gh-105699: Fix a Crasher Related to a Deprecated Global Variable (gh-106923) Message-ID: https://github.com/python/cpython/commit/0ba07b2108d4763273f3fb85544dde34c5acd40a commit: 0ba07b2108d4763273f3fb85544dde34c5acd40a branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-21T08:34:09-06:00 summary: gh-105699: Fix a Crasher Related to a Deprecated Global Variable (gh-106923) There was a slight race in _Py_ClearFileSystemEncoding() (when called from _Py_SetFileSystemEncoding()), between freeing the value and setting the variable to NULL, which occasionally caused crashes when multiple isolated interpreters were used. (Notably, I saw at least 10 different, seemingly unrelated spooky-action-at-a-distance, ways this crashed. Yay, free threading!) We avoid the problem by only setting the global variables with the main interpreter (i.e. runtime init). files: A Misc/NEWS.d/next/Core and Builtins/2023-07-20-12-21-37.gh-issue-105699.08ywGV.rst M Objects/unicodeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-20-12-21-37.gh-issue-105699.08ywGV.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-12-21-37.gh-issue-105699.08ywGV.rst new file mode 100644 index 0000000000000..82312718cd047 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-12-21-37.gh-issue-105699.08ywGV.rst @@ -0,0 +1,4 @@ +Python no longer crashes due to an infrequent race in setting +``Py_FileSystemDefaultEncoding`` and ``Py_FileSystemDefaultEncodeErrors`` +(both deprecated), when simultaneously initializing two isolated +subinterpreters. Now they are only set during runtime initialization. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index e8e8cab99c814..fe2660c6ce605 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -15189,10 +15189,13 @@ init_fs_codec(PyInterpreterState *interp) /* Set Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors global configuration variables. */ - if (_Py_SetFileSystemEncoding(fs_codec->encoding, - fs_codec->errors) < 0) { - PyErr_NoMemory(); - return -1; + if (_Py_IsMainInterpreter(interp)) { + + if (_Py_SetFileSystemEncoding(fs_codec->encoding, + fs_codec->errors) < 0) { + PyErr_NoMemory(); + return -1; + } } return 0; } From webhook-mailer at python.org Fri Jul 21 16:27:56 2023 From: webhook-mailer at python.org (Yhg1s) Date: Fri, 21 Jul 2023 20:27:56 -0000 Subject: [Python-checkins] [3.12] gh-105699: Fix a Crasher Related to a Deprecated Global Variable (gh-106923) (#106964) Message-ID: https://github.com/python/cpython/commit/957f14d0decb7aa5723d5ed8c4d16abad40547b0 commit: 957f14d0decb7aa5723d5ed8c4d16abad40547b0 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Yhg1s date: 2023-07-21T22:27:52+02:00 summary: [3.12] gh-105699: Fix a Crasher Related to a Deprecated Global Variable (gh-106923) (#106964) gh-105699: Fix a Crasher Related to a Deprecated Global Variable (gh-106923) There was a slight race in _Py_ClearFileSystemEncoding() (when called from _Py_SetFileSystemEncoding()), between freeing the value and setting the variable to NULL, which occasionally caused crashes when multiple isolated interpreters were used. (Notably, I saw at least 10 different, seemingly unrelated spooky-action-at-a-distance, ways this crashed. Yay, free threading!) We avoid the problem by only setting the global variables with the main interpreter (i.e. runtime init). (cherry picked from commit 0ba07b2108d4763273f3fb85544dde34c5acd40a) Co-authored-by: Eric Snow files: A Misc/NEWS.d/next/Core and Builtins/2023-07-20-12-21-37.gh-issue-105699.08ywGV.rst M Objects/unicodeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-20-12-21-37.gh-issue-105699.08ywGV.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-12-21-37.gh-issue-105699.08ywGV.rst new file mode 100644 index 0000000000000..82312718cd047 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-12-21-37.gh-issue-105699.08ywGV.rst @@ -0,0 +1,4 @@ +Python no longer crashes due to an infrequent race in setting +``Py_FileSystemDefaultEncoding`` and ``Py_FileSystemDefaultEncodeErrors`` +(both deprecated), when simultaneously initializing two isolated +subinterpreters. Now they are only set during runtime initialization. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index a0be85718119d..cd196f5b11cd9 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -15177,10 +15177,13 @@ init_fs_codec(PyInterpreterState *interp) /* Set Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors global configuration variables. */ - if (_Py_SetFileSystemEncoding(fs_codec->encoding, - fs_codec->errors) < 0) { - PyErr_NoMemory(); - return -1; + if (_Py_IsMainInterpreter(interp)) { + + if (_Py_SetFileSystemEncoding(fs_codec->encoding, + fs_codec->errors) < 0) { + PyErr_NoMemory(); + return -1; + } } return 0; } From webhook-mailer at python.org Fri Jul 21 16:28:25 2023 From: webhook-mailer at python.org (Yhg1s) Date: Fri, 21 Jul 2023 20:28:25 -0000 Subject: [Python-checkins] [3.12] gh-105699: Fix an Interned Strings Crasher (gh-106930) (#106963) Message-ID: https://github.com/python/cpython/commit/d0176ed911e7c1717b21e276d1ff2c109e62356a commit: d0176ed911e7c1717b21e276d1ff2c109e62356a branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Yhg1s date: 2023-07-21T22:28:22+02:00 summary: [3.12] gh-105699: Fix an Interned Strings Crasher (gh-106930) (#106963) gh-105699: Fix an Interned Strings Crasher (gh-106930) A static (process-global) str object must only have its "interned" state cleared when no longer interned in any interpreters. They are the only ones that can be shared by interpreters so we don't have to worry about any other str objects. We trigger clearing the state with the main interpreter, since no other interpreters may exist at that point and _PyUnicode_ClearInterned() is only called during interpreter finalization. We do not address here the fact that a string will only be interned in the first interpreter that interns it. In any subsequent interpreters str.state.interned is already set so _PyUnicode_InternInPlace() will skip it. That needs to be addressed separately from fixing the crasher. (cherry picked from commit 87e7cb09e4258c4deb01a07dc52c1021907195d7) Co-authored-by: Eric Snow files: A Misc/NEWS.d/next/Core and Builtins/2023-07-20-15-15-57.gh-issue-105699.DdqHFg.rst M Objects/unicodeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-20-15-15-57.gh-issue-105699.DdqHFg.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-15-15-57.gh-issue-105699.DdqHFg.rst new file mode 100644 index 0000000000000..4a257c6282220 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-20-15-15-57.gh-issue-105699.DdqHFg.rst @@ -0,0 +1,3 @@ +Python no longer crashes due an infrequent race when initialzing +per-interpreter interned strings. The crash would manifest when the +interpreter was finalized. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index cd196f5b11cd9..7ead65bf98d1f 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -14817,6 +14817,7 @@ _PyUnicode_ClearInterned(PyInterpreterState *interp) PyObject *s, *ignored_value; while (PyDict_Next(interned, &pos, &s, &ignored_value)) { assert(PyUnicode_IS_READY(s)); + int shared = 0; switch (PyUnicode_CHECK_INTERNED(s)) { case SSTATE_INTERNED_IMMORTAL: // Skip the Immortal Instance check and restore @@ -14828,6 +14829,14 @@ _PyUnicode_ClearInterned(PyInterpreterState *interp) #endif break; case SSTATE_INTERNED_IMMORTAL_STATIC: + /* It is shared between interpreters, so we should unmark it + only when this is the last interpreter in which it's + interned. We immortalize all the statically initialized + strings during startup, so we can rely on the + main interpreter to be the last one. */ + if (!_Py_IsMainInterpreter(interp)) { + shared = 1; + } break; case SSTATE_INTERNED_MORTAL: /* fall through */ @@ -14836,7 +14845,9 @@ _PyUnicode_ClearInterned(PyInterpreterState *interp) default: Py_UNREACHABLE(); } - _PyUnicode_STATE(s).interned = SSTATE_NOT_INTERNED; + if (!shared) { + _PyUnicode_STATE(s).interned = SSTATE_NOT_INTERNED; + } } #ifdef INTERNED_STATS fprintf(stderr, From webhook-mailer at python.org Fri Jul 21 17:10:55 2023 From: webhook-mailer at python.org (vstinner) Date: Fri, 21 Jul 2023 21:10:55 -0000 Subject: [Python-checkins] gh-106004: Add PyDict_GetItemRef() function (#106005) Message-ID: https://github.com/python/cpython/commit/41ca16455188db806bfc7037058e8ecff2755e6c commit: 41ca16455188db806bfc7037058e8ecff2755e6c branch: main author: Victor Stinner committer: vstinner date: 2023-07-21T23:10:51+02:00 summary: gh-106004: Add PyDict_GetItemRef() function (#106005) * Add PyDict_GetItemRef() and PyDict_GetItemStringRef() functions. Add these functions to the stable ABI version 3.13. * Add unit tests on the PyDict C API in test_capi. files: A Misc/NEWS.d/next/C API/2023-06-23-02-57-15.gh-issue-106004.-OToh6.rst M Doc/c-api/dict.rst M Doc/data/stable_abi.dat M Doc/whatsnew/3.13.rst M Include/dictobject.h M Lib/test/test_stable_abi_ctypes.py M Misc/stable_abi.toml M Modules/_testcapimodule.c M Objects/dictobject.c M PC/python3dll.c diff --git a/Doc/c-api/dict.rst b/Doc/c-api/dict.rst index bd0c36a217e2c..e4c1d71a413a6 100644 --- a/Doc/c-api/dict.rst +++ b/Doc/c-api/dict.rst @@ -93,10 +93,26 @@ Dictionary Objects Return ``0`` on success or ``-1`` on failure. +.. c:function:: int PyDict_GetItemRef(PyObject *p, PyObject *key, PyObject **result) + + Return a new :term:`strong reference` to the object from dictionary *p* + which has a key *key*: + + * If the key is present, set *\*result* to a new :term:`strong reference` + to the value and return ``1``. + * If the key is missing, set *\*result* to ``NULL`` and return ``0``. + * On error, raise an exception and return ``-1``. + + .. versionadded:: 3.13 + + See also the :c:func:`PyObject_GetItem` function. + + .. c:function:: PyObject* PyDict_GetItem(PyObject *p, PyObject *key) - Return the object from dictionary *p* which has a key *key*. Return ``NULL`` - if the key *key* is not present, but *without* setting an exception. + Return a :term:`borrowed reference` to the object from dictionary *p* which + has a key *key*. Return ``NULL`` if the key *key* is missing *without* + setting an exception. .. note:: @@ -131,6 +147,14 @@ Dictionary Objects :c:func:`PyUnicode_FromString` *key* instead. +.. c:function:: int PyDict_GetItemStringRef(PyObject *p, const char *key, PyObject **result) + + Similar than :c:func:`PyDict_GetItemRef`, but *key* is specified as a + :c:expr:`const char*`, rather than a :c:expr:`PyObject*`. + + .. versionadded:: 3.13 + + .. c:function:: PyObject* PyDict_SetDefault(PyObject *p, PyObject *key, PyObject *defaultobj) This is the same as the Python-level :meth:`dict.setdefault`. If present, it diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index aa1edf5463705..ed415a4dc644a 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -111,7 +111,9 @@ function,PyDict_Copy,3.2,, function,PyDict_DelItem,3.2,, function,PyDict_DelItemString,3.2,, function,PyDict_GetItem,3.2,, +function,PyDict_GetItemRef,3.13,, function,PyDict_GetItemString,3.2,, +function,PyDict_GetItemStringRef,3.13,, function,PyDict_GetItemWithError,3.2,, function,PyDict_Items,3.2,, function,PyDict_Keys,3.2,, diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 1d5e34dac28a5..0203e3d18738d 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -786,6 +786,13 @@ New Features always steals a reference to the value. (Contributed by Serhiy Storchaka in :gh:`86493`.) +* Added :c:func:`PyDict_GetItemRef` and :c:func:`PyDict_GetItemStringRef` + functions: similar to :c:func:`PyDict_GetItemWithError` but returning a + :term:`strong reference` instead of a :term:`borrowed reference`. Moreover, + these functions return -1 on error and so checking ``PyErr_Occurred()`` is + not needed. + (Contributed by Victor Stinner in :gh:`106004`.) + Porting to Python 3.13 ---------------------- diff --git a/Include/dictobject.h b/Include/dictobject.h index e7fcb44d0cf9a..26434e49f9140 100644 --- a/Include/dictobject.h +++ b/Include/dictobject.h @@ -57,6 +57,15 @@ PyAPI_FUNC(int) PyDict_MergeFromSeq2(PyObject *d, PyAPI_FUNC(PyObject *) PyDict_GetItemString(PyObject *dp, const char *key); PyAPI_FUNC(int) PyDict_SetItemString(PyObject *dp, const char *key, PyObject *item); PyAPI_FUNC(int) PyDict_DelItemString(PyObject *dp, const char *key); + +// Return the object from dictionary *op* which has a key *key*. +// - If the key is present, set *result to a new strong reference to the value +// and return 1. +// - If the key is missing, set *result to NULL and return 0 . +// - On error, raise an exception and return -1. +PyAPI_FUNC(int) PyDict_GetItemRef(PyObject *mp, PyObject *key, PyObject **result); +PyAPI_FUNC(int) PyDict_GetItemStringRef(PyObject *mp, const char *key, PyObject **result); + #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030A0000 PyAPI_FUNC(PyObject *) PyObject_GenericGetDict(PyObject *, void *); #endif diff --git a/Lib/test/test_stable_abi_ctypes.py b/Lib/test/test_stable_abi_ctypes.py index 4e74bb374c93b..566d36a3f5ba1 100644 --- a/Lib/test/test_stable_abi_ctypes.py +++ b/Lib/test/test_stable_abi_ctypes.py @@ -144,7 +144,9 @@ def test_windows_feature_macros(self): "PyDict_DelItem", "PyDict_DelItemString", "PyDict_GetItem", + "PyDict_GetItemRef", "PyDict_GetItemString", + "PyDict_GetItemStringRef", "PyDict_GetItemWithError", "PyDict_Items", "PyDict_Keys", diff --git a/Misc/NEWS.d/next/C API/2023-06-23-02-57-15.gh-issue-106004.-OToh6.rst b/Misc/NEWS.d/next/C API/2023-06-23-02-57-15.gh-issue-106004.-OToh6.rst new file mode 100644 index 0000000000000..c7a006b2bc075 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-06-23-02-57-15.gh-issue-106004.-OToh6.rst @@ -0,0 +1,4 @@ +Adds :c:func:`PyDict_GetItemRef` and :c:func:`PyDict_GetItemStringRef` +functions: similar to :c:func:`PyDict_GetItemWithError` but returning a +:term:`strong reference` instead of a :term:`borrowed reference`. Patch by +Victor Stinner. diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml index dd2c9910b83cc..16d5c1a07ae3e 100644 --- a/Misc/stable_abi.toml +++ b/Misc/stable_abi.toml @@ -2446,3 +2446,7 @@ added = '3.13' [function.PyModule_Add] added = '3.13' +[function.PyDict_GetItemRef] + added = '3.13' +[function.PyDict_GetItemStringRef] + added = '3.13' diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index dd2c9c72e5378..065a7fb733d43 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -3464,6 +3464,196 @@ test_weakref_capi(PyObject *Py_UNUSED(module), PyObject *Py_UNUSED(args)) } +static PyObject * +test_dict_capi(PyObject *Py_UNUSED(module), PyObject *Py_UNUSED(args)) +{ + assert(!PyErr_Occurred()); + + PyObject *dict= NULL, *key = NULL, *missing_key = NULL, *value = NULL; + PyObject *invalid_key = NULL; + int res; + + // test PyDict_New() + dict = PyDict_New(); + if (dict == NULL) { + goto error; + } + + key = PyUnicode_FromString("key"); + if (key == NULL) { + goto error; + } + + missing_key = PyUnicode_FromString("missing_key"); + if (missing_key == NULL) { + goto error; + } + + value = PyUnicode_FromString("value"); + if (value == NULL) { + goto error; + } + + // test PyDict_SetItem() + Py_ssize_t key_refcnt = Py_REFCNT(key); + Py_ssize_t value_refcnt = Py_REFCNT(value); + res = PyDict_SetItem(dict, key, value); + if (res < 0) { + goto error; + } + assert(res == 0); + assert(Py_REFCNT(key) == (key_refcnt + 1)); + assert(Py_REFCNT(value) == (value_refcnt + 1)); + + // test PyDict_SetItemString() + res = PyDict_SetItemString(dict, "key", value); + if (res < 0) { + goto error; + } + assert(res == 0); + assert(Py_REFCNT(key) == (key_refcnt + 1)); + assert(Py_REFCNT(value) == (value_refcnt + 1)); + + // test PyDict_Size() + assert(PyDict_Size(dict) == 1); + + // test PyDict_Contains(), key is present + assert(PyDict_Contains(dict, key) == 1); + + // test PyDict_GetItem(), key is present + assert(PyDict_GetItem(dict, key) == value); + + // test PyDict_GetItemString(), key is present + assert(PyDict_GetItemString(dict, "key") == value); + + // test PyDict_GetItemWithError(), key is present + assert(PyDict_GetItemWithError(dict, key) == value); + assert(!PyErr_Occurred()); + + // test PyDict_GetItemRef(), key is present + PyObject *get_value = Py_Ellipsis; // marker value + assert(PyDict_GetItemRef(dict, key, &get_value) == 1); + assert(get_value == value); + Py_DECREF(get_value); + + // test PyDict_GetItemStringRef(), key is present + get_value = Py_Ellipsis; // marker value + assert(PyDict_GetItemStringRef(dict, "key", &get_value) == 1); + assert(get_value == value); + Py_DECREF(get_value); + + // test PyDict_Contains(), missing key + assert(PyDict_Contains(dict, missing_key) == 0); + + // test PyDict_GetItem(), missing key + assert(PyDict_GetItem(dict, missing_key) == NULL); + assert(!PyErr_Occurred()); + + // test PyDict_GetItemString(), missing key + assert(PyDict_GetItemString(dict, "missing_key") == NULL); + assert(!PyErr_Occurred()); + + // test PyDict_GetItemWithError(), missing key + assert(PyDict_GetItem(dict, missing_key) == NULL); + assert(!PyErr_Occurred()); + + // test PyDict_GetItemRef(), missing key + get_value = Py_Ellipsis; // marker value + assert(PyDict_GetItemRef(dict, missing_key, &get_value) == 0); + assert(!PyErr_Occurred()); + assert(get_value == NULL); + + // test PyDict_GetItemStringRef(), missing key + get_value = Py_Ellipsis; // marker value + assert(PyDict_GetItemStringRef(dict, "missing_key", &get_value) == 0); + assert(!PyErr_Occurred()); + assert(get_value == NULL); + + // test PyDict_GetItem(), invalid dict + PyObject *invalid_dict = key; // borrowed reference + assert(PyDict_GetItem(invalid_dict, key) == NULL); + assert(!PyErr_Occurred()); + + // test PyDict_GetItemWithError(), invalid dict + assert(PyDict_GetItemWithError(invalid_dict, key) == NULL); + assert(PyErr_ExceptionMatches(PyExc_SystemError)); + PyErr_Clear(); + + // test PyDict_GetItemRef(), invalid dict + get_value = Py_Ellipsis; // marker value + assert(PyDict_GetItemRef(invalid_dict, key, &get_value) == -1); + assert(PyErr_ExceptionMatches(PyExc_SystemError)); + PyErr_Clear(); + assert(get_value == NULL); + + // test PyDict_GetItemStringRef(), invalid dict + get_value = Py_Ellipsis; // marker value + assert(PyDict_GetItemStringRef(invalid_dict, "key", &get_value) == -1); + assert(PyErr_ExceptionMatches(PyExc_SystemError)); + PyErr_Clear(); + assert(get_value == NULL); + + invalid_key = PyList_New(0); + if (invalid_key == NULL) { + goto error; + } + + // test PyDict_Contains(), invalid key + assert(PyDict_Contains(dict, invalid_key) == -1); + assert(PyErr_ExceptionMatches(PyExc_TypeError)); + PyErr_Clear(); + + // test PyDict_GetItem(), invalid key + assert(PyDict_GetItem(dict, invalid_key) == NULL); + assert(!PyErr_Occurred()); + + // test PyDict_GetItemWithError(), invalid key + assert(PyDict_GetItemWithError(dict, invalid_key) == NULL); + assert(PyErr_ExceptionMatches(PyExc_TypeError)); + PyErr_Clear(); + + // test PyDict_GetItemRef(), invalid key + get_value = Py_Ellipsis; // marker value + assert(PyDict_GetItemRef(dict, invalid_key, &get_value) == -1); + assert(PyErr_ExceptionMatches(PyExc_TypeError)); + PyErr_Clear(); + assert(get_value == NULL); + + // test PyDict_DelItem(), key is present + assert(PyDict_DelItem(dict, key) == 0); + assert(PyDict_Size(dict) == 0); + + // test PyDict_DelItem(), missing key + assert(PyDict_DelItem(dict, missing_key) == -1); + assert(PyErr_ExceptionMatches(PyExc_KeyError)); + PyErr_Clear(); + + // test PyDict_DelItem(), invalid key + assert(PyDict_DelItem(dict, invalid_key) == -1); + assert(PyErr_ExceptionMatches(PyExc_TypeError)); + PyErr_Clear(); + + // test PyDict_Clear() + PyDict_Clear(dict); + + Py_DECREF(dict); + Py_DECREF(key); + Py_DECREF(missing_key); + Py_DECREF(value); + Py_DECREF(invalid_key); + + Py_RETURN_NONE; + +error: + Py_XDECREF(dict); + Py_XDECREF(key); + Py_XDECREF(missing_key); + Py_XDECREF(value); + Py_XDECREF(invalid_key); + return NULL; +} + + static PyMethodDef TestMethods[] = { {"set_errno", set_errno, METH_VARARGS}, {"test_config", test_config, METH_NOARGS}, @@ -3609,6 +3799,7 @@ static PyMethodDef TestMethods[] = { {"function_set_kw_defaults", function_set_kw_defaults, METH_VARARGS, NULL}, {"check_pyimport_addmodule", check_pyimport_addmodule, METH_VARARGS}, {"test_weakref_capi", test_weakref_capi, METH_NOARGS}, + {"test_dict_capi", test_dict_capi, METH_NOARGS}, {NULL, NULL} /* sentinel */ }; diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 013c21884032a..3f9297c8a411f 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -1696,9 +1696,8 @@ PyDict_GetItem(PyObject *op, PyObject *key) /* Ignore any exception raised by the lookup */ _PyErr_SetRaisedException(tstate, exc); - assert(ix >= 0 || value == NULL); - return value; + return value; // borrowed reference } Py_ssize_t @@ -1737,9 +1736,46 @@ _PyDict_GetItem_KnownHash(PyObject *op, PyObject *key, Py_hash_t hash) ix = _Py_dict_lookup(mp, key, hash, &value); assert(ix >= 0 || value == NULL); - return value; + return value; // borrowed reference } + +int +PyDict_GetItemRef(PyObject *op, PyObject *key, PyObject **result) +{ + if (!PyDict_Check(op)) { + PyErr_BadInternalCall(); + *result = NULL; + return -1; + } + PyDictObject*mp = (PyDictObject *)op; + + Py_hash_t hash; + if (!PyUnicode_CheckExact(key) || (hash = unicode_get_hash(key)) == -1) + { + hash = PyObject_Hash(key); + if (hash == -1) { + *result = NULL; + return -1; + } + } + + PyObject *value; + Py_ssize_t ix = _Py_dict_lookup(mp, key, hash, &value); + assert(ix >= 0 || value == NULL); + if (ix == DKIX_ERROR) { + *result = NULL; + return -1; + } + if (value == NULL) { + *result = NULL; + return 0; // missing key + } + *result = Py_NewRef(value); + return 1; // key is present +} + + /* Variant of PyDict_GetItem() that doesn't suppress exceptions. This returns NULL *with* an exception set if an exception occurred. It returns NULL *without* an exception set if the key wasn't present. @@ -1766,7 +1802,7 @@ PyDict_GetItemWithError(PyObject *op, PyObject *key) ix = _Py_dict_lookup(mp, key, hash, &value); assert(ix >= 0 || value == NULL); - return value; + return value; // borrowed reference } PyObject * @@ -1777,7 +1813,7 @@ _PyDict_GetItemWithError(PyObject *dp, PyObject *kv) if (hash == -1) { return NULL; } - return _PyDict_GetItem_KnownHash(dp, kv, hash); + return _PyDict_GetItem_KnownHash(dp, kv, hash); // borrowed reference } PyObject * @@ -1789,7 +1825,7 @@ _PyDict_GetItemIdWithError(PyObject *dp, _Py_Identifier *key) return NULL; Py_hash_t hash = unicode_get_hash(kv); assert (hash != -1); /* interned strings have their hash value initialised */ - return _PyDict_GetItem_KnownHash(dp, kv, hash); + return _PyDict_GetItem_KnownHash(dp, kv, hash); // borrowed reference } PyObject * @@ -1802,7 +1838,7 @@ _PyDict_GetItemStringWithError(PyObject *v, const char *key) } rv = PyDict_GetItemWithError(v, kv); Py_DECREF(kv); - return rv; + return rv; // borrowed reference } /* Fast version of global value lookup (LOAD_GLOBAL). @@ -3894,7 +3930,20 @@ PyDict_GetItemString(PyObject *v, const char *key) } rv = PyDict_GetItem(v, kv); Py_DECREF(kv); - return rv; + return rv; // borrowed reference +} + +int +PyDict_GetItemStringRef(PyObject *v, const char *key, PyObject **result) +{ + PyObject *key_obj = PyUnicode_FromString(key); + if (key_obj == NULL) { + *result = NULL; + return -1; + } + int res = PyDict_GetItemRef(v, key_obj, result); + Py_DECREF(key_obj); + return res; } int @@ -5146,15 +5195,11 @@ dictitems_contains(_PyDictViewObject *dv, PyObject *obj) return 0; key = PyTuple_GET_ITEM(obj, 0); value = PyTuple_GET_ITEM(obj, 1); - found = PyDict_GetItemWithError((PyObject *)dv->dv_dict, key); - if (found == NULL) { - if (PyErr_Occurred()) - return -1; - return 0; + result = PyDict_GetItemRef((PyObject *)dv->dv_dict, key, &found); + if (result == 1) { + result = PyObject_RichCompareBool(found, value, Py_EQ); + Py_DECREF(found); } - Py_INCREF(found); - result = PyObject_RichCompareBool(found, value, Py_EQ); - Py_DECREF(found); return result; } diff --git a/PC/python3dll.c b/PC/python3dll.c index 0b54c5a707231..64dfbba3e424a 100755 --- a/PC/python3dll.c +++ b/PC/python3dll.c @@ -172,7 +172,9 @@ EXPORT_FUNC(PyDict_Copy) EXPORT_FUNC(PyDict_DelItem) EXPORT_FUNC(PyDict_DelItemString) EXPORT_FUNC(PyDict_GetItem) +EXPORT_FUNC(PyDict_GetItemRef) EXPORT_FUNC(PyDict_GetItemString) +EXPORT_FUNC(PyDict_GetItemStringRef) EXPORT_FUNC(PyDict_GetItemWithError) EXPORT_FUNC(PyDict_Items) EXPORT_FUNC(PyDict_Keys) From webhook-mailer at python.org Fri Jul 21 20:24:29 2023 From: webhook-mailer at python.org (gvanrossum) Date: Sat, 22 Jul 2023 00:24:29 -0000 Subject: [Python-checkins] gh-96663: Add a better error message for __dict__-less classes setattr (#103232) Message-ID: https://github.com/python/cpython/commit/cdeb1a6caad5e3067f01d6058238803b8517f9de commit: cdeb1a6caad5e3067f01d6058238803b8517f9de branch: main author: James Hilton-Balfe committer: gvanrossum date: 2023-07-21T17:24:26-07:00 summary: gh-96663: Add a better error message for __dict__-less classes setattr (#103232) files: A Misc/NEWS.d/next/Core and Builtins/2023-04-04-00-40-04.gh-issue-96663.PdR9hK.rst M Lib/test/test_class.py M Lib/test/test_descrtut.py M Objects/object.c diff --git a/Lib/test/test_class.py b/Lib/test/test_class.py index 894e0ca67deab..bb35baea508c7 100644 --- a/Lib/test/test_class.py +++ b/Lib/test/test_class.py @@ -641,6 +641,14 @@ class A: class B: y = 0 __slots__ = ('z',) + class C: + __slots__ = ("y",) + + def __setattr__(self, name, value) -> None: + if name == "z": + super().__setattr__("y", 1) + else: + super().__setattr__(name, value) error_msg = "'A' object has no attribute 'x'" with self.assertRaisesRegex(AttributeError, error_msg): @@ -653,8 +661,16 @@ class B: B().x with self.assertRaisesRegex(AttributeError, error_msg): del B().x - with self.assertRaisesRegex(AttributeError, error_msg): + with self.assertRaisesRegex( + AttributeError, + "'B' object has no attribute 'x' and no __dict__ for setting new attributes" + ): B().x = 0 + with self.assertRaisesRegex( + AttributeError, + "'C' object has no attribute 'x'" + ): + C().x = 0 error_msg = "'B' object attribute 'y' is read-only" with self.assertRaisesRegex(AttributeError, error_msg): diff --git a/Lib/test/test_descrtut.py b/Lib/test/test_descrtut.py index 7796031ed0602..13e3ea41bdb76 100644 --- a/Lib/test/test_descrtut.py +++ b/Lib/test/test_descrtut.py @@ -139,7 +139,7 @@ def merge(self, other): >>> a.x1 = 1 Traceback (most recent call last): File "", line 1, in ? - AttributeError: 'defaultdict2' object has no attribute 'x1' + AttributeError: 'defaultdict2' object has no attribute 'x1' and no __dict__ for setting new attributes >>> """ diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-04-04-00-40-04.gh-issue-96663.PdR9hK.rst b/Misc/NEWS.d/next/Core and Builtins/2023-04-04-00-40-04.gh-issue-96663.PdR9hK.rst new file mode 100644 index 0000000000000..cb806b5ea7a9f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-04-04-00-40-04.gh-issue-96663.PdR9hK.rst @@ -0,0 +1 @@ +Add a better, more introspect-able error message when setting attributes on classes without a ``__dict__`` and no slot member for the attribute. diff --git a/Objects/object.c b/Objects/object.c index d30e048335ab6..bfbc87198f5b3 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -1576,9 +1576,18 @@ _PyObject_GenericSetAttrWithDict(PyObject *obj, PyObject *name, } if (dictptr == NULL) { if (descr == NULL) { - PyErr_Format(PyExc_AttributeError, - "'%.100s' object has no attribute '%U'", - tp->tp_name, name); + if (tp->tp_setattro == PyObject_GenericSetAttr) { + PyErr_Format(PyExc_AttributeError, + "'%.100s' object has no attribute '%U' and no " + "__dict__ for setting new attributes", + tp->tp_name, name); + } + else { + PyErr_Format(PyExc_AttributeError, + "'%.100s' object has no attribute '%U'", + tp->tp_name, name); + } + set_attribute_error_context(obj, name); } else { PyErr_Format(PyExc_AttributeError, @@ -1611,6 +1620,7 @@ _PyObject_GenericSetAttrWithDict(PyObject *obj, PyObject *name, "'%.100s' object has no attribute '%U'", tp->tp_name, name); } + set_attribute_error_context(obj, name); } done: Py_XDECREF(descr); From webhook-mailer at python.org Sat Jul 22 03:43:16 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 22 Jul 2023 07:43:16 -0000 Subject: [Python-checkins] gh-106970: Fix Argument Clinic 'destination clear' command (#106972) Message-ID: https://github.com/python/cpython/commit/3372bcba9893030e4063a9264ec0b4d1b6166883 commit: 3372bcba9893030e4063a9264ec0b4d1b6166883 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-22T09:43:13+02:00 summary: gh-106970: Fix Argument Clinic 'destination clear' command (#106972) Add test for the 'destination clear' command, and the 'destination' directive in general. Fix two bugs in 'destination clear' command: 1. The text attribute of the allocator is called 'text', not '_text' 2. Return after processing the 'clear' command, instead of proceeding directly to the fail(). files: A Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst M Lib/test/clinic.test.c M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index da99e58c77f02..a660ccf8876e2 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -4948,3 +4948,59 @@ Test_meth_coexist(TestObj *self, PyObject *Py_UNUSED(ignored)) static PyObject * Test_meth_coexist_impl(TestObj *self) /*[clinic end generated code: output=808a293d0cd27439 input=2a1d75b5e6fec6dd]*/ + + +/*[clinic input] +output push +output preset buffer +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=5bff3376ee0df0b5]*/ + +/*[clinic input] +buffer_clear + a: int +We'll call 'destination buffer clear' after this. + +Argument Clinic's buffer preset puts most generated code into the +'buffer' destination, except from 'impl_definition', which is put into +the 'block' destination, so we should expect everything but +'impl_definition' to be cleared. +[clinic start generated code]*/ + +static PyObject * +buffer_clear_impl(PyObject *module, int a) +/*[clinic end generated code: output=f14bba74677e1846 input=a4c308a6fdab043c]*/ + +/*[clinic input] +destination buffer clear +output pop +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=f20d06adb8252084]*/ + + +/*[clinic input] +output push +destination test1 new buffer +output everything suppress +output docstring_definition test1 +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=5a77c454970992fc]*/ + +/*[clinic input] +new_dest + a: int +Only this docstring should be outputted to test1. +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=da5af421ed8996ed]*/ + +/*[clinic input] +dump test1 +output pop +[clinic start generated code]*/ + +PyDoc_STRVAR(new_dest__doc__, +"new_dest($module, /, a)\n" +"--\n" +"\n" +"Only this docstring should be outputted to test1."); +/*[clinic end generated code: output=9cac703f51d90e84 input=090db8df4945576d]*/ diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 7c725e33f5392..e40579b0041cf 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -249,6 +249,15 @@ def test_cpp_monitor_fail_no_matching_if(self): out = self.expect_failure(raw) self.assertEqual(out, msg) + def test_unknown_destination_command(self): + out = self.expect_failure(""" + /*[clinic input] + destination buffer nosuchcommand + [clinic start generated code]*/ + """) + msg = "unknown destination command 'nosuchcommand'" + self.assertIn(msg, out) + class ClinicGroupPermuterTest(TestCase): def _test(self, l, m, r, output): diff --git a/Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst b/Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst new file mode 100644 index 0000000000000..194e3351b0470 --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst @@ -0,0 +1,4 @@ +Fix bugs in the Argument Clinic ``destination clear`` command; the +destination buffers would never be cleared, and the ``destination`` +directive parser would simply continue to the fault handler after processing +the command. Patch by Erlend E. Aasland. diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index a204c1dd0dc2d..8acf513a6c1fc 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1983,7 +1983,7 @@ def __getitem__(self, i): def clear(self): for ta in self._array: - ta._text.clear() + ta.text.clear() def dump(self): texts = [ta.output() for ta in self._array] @@ -4487,13 +4487,13 @@ def directive_destination( command: str, *args ) -> None: - if command == 'new': - self.clinic.add_destination(name, *args) - return - - if command == 'clear': - self.clinic.get_destination(name).clear() - fail("unknown destination command", repr(command)) + match command: + case "new": + self.clinic.add_destination(name, *args) + case "clear": + self.clinic.get_destination(name).clear() + case _: + fail("unknown destination command", repr(command)) def directive_output( From webhook-mailer at python.org Sat Jul 22 04:12:47 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 22 Jul 2023 08:12:47 -0000 Subject: [Python-checkins] gh-101100: Docs: Check Sphinx warnings and fail if improved (#106460) Message-ID: https://github.com/python/cpython/commit/806d7c98a5da5c1fd2e52a5b666f36ca4f545092 commit: 806d7c98a5da5c1fd2e52a5b666f36ca4f545092 branch: main author: Hugo van Kemenade committer: hugovk date: 2023-07-22T08:12:43Z summary: gh-101100: Docs: Check Sphinx warnings and fail if improved (#106460) files: A Doc/tools/check-warnings.py D Doc/tools/touch-clean-files.py D Doc/tools/warnings-to-gh-actions.py M .github/workflows/reusable-docs.yml M Doc/tools/.nitignore diff --git a/.github/workflows/reusable-docs.yml b/.github/workflows/reusable-docs.yml index b39d8cea6421e..56932c4860573 100644 --- a/.github/workflows/reusable-docs.yml +++ b/.github/workflows/reusable-docs.yml @@ -28,10 +28,8 @@ jobs: cache-dependency-path: 'Doc/requirements.txt' - name: 'Install build dependencies' run: make -C Doc/ venv - - name: 'Build HTML documentation' - run: make -C Doc/ SPHINXOPTS="-q" SPHINXERRORHANDLING="-W --keep-going" html - # Add pull request annotations for Sphinx nitpicks (missing references) + # To annotate PRs with Sphinx nitpicks (missing references) - name: 'Get list of changed files' if: github.event_name == 'pull_request' id: changed_files @@ -39,24 +37,19 @@ jobs: with: filter: "Doc/**" format: csv # works for paths with spaces - - name: 'Build changed files in nit-picky mode' - if: github.event_name == 'pull_request' + - name: 'Build HTML documentation' continue-on-error: true run: | set -Eeuo pipefail - # Mark files the pull request modified - python Doc/tools/touch-clean-files.py --clean '${{ steps.changed_files.outputs.added_modified }}' - # Build docs with the '-n' (nit-picky) option; convert warnings to annotations - make -C Doc/ PYTHON=../python SPHINXOPTS="-q -n --keep-going" html 2>&1 | - python Doc/tools/warnings-to-gh-actions.py - - # Ensure some files always pass Sphinx nit-picky mode (no missing references) - - name: 'Build known-good files in nit-picky mode' + # Build docs with the '-n' (nit-picky) option; write warnings to file + make -C Doc/ PYTHON=../python SPHINXOPTS="-q -n -W --keep-going -w sphinx-warnings.txt" html + - name: 'Check warnings' + if: github.event_name == 'pull_request' run: | - # Mark files that must pass nit-picky - python Doc/tools/touch-clean-files.py - # Build docs with the '-n' (nit-picky) option, convert warnings to errors (-W) - make -C Doc/ PYTHON=../python SPHINXOPTS="-q -n -W --keep-going" html 2>&1 + python Doc/tools/check-warnings.py \ + --check-and-annotate '${{ steps.changed_files.outputs.added_modified }}' \ + --fail-if-regression \ + --fail-if-improved # This build doesn't use problem matchers or check annotations build_doc_oldest_supported_sphinx: diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 23aa30c956b3b..ca6f32c55acd6 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -1,7 +1,6 @@ # All RST files under Doc/ -- except these -- must pass Sphinx nit-picky mode, -# as tested on the CI via touch-clean-files.py in doc.yml. -# Add blank lines between files and keep them sorted lexicographically -# to help avoid merge conflicts. +# as tested on the CI via check-warnings.py in reusable-docs.yml. +# Keep lines sorted lexicographically to help avoid merge conflicts. Doc/c-api/allocation.rst Doc/c-api/apiabiversion.rst @@ -18,7 +17,6 @@ Doc/c-api/complex.rst Doc/c-api/conversion.rst Doc/c-api/datetime.rst Doc/c-api/descriptor.rst -Doc/c-api/dict.rst Doc/c-api/exceptions.rst Doc/c-api/file.rst Doc/c-api/float.rst @@ -48,7 +46,6 @@ Doc/c-api/typehints.rst Doc/c-api/typeobj.rst Doc/c-api/unicode.rst Doc/c-api/veryhigh.rst -Doc/c-api/weakref.rst Doc/extending/embedding.rst Doc/extending/extending.rst Doc/extending/newtypes.rst @@ -88,8 +85,6 @@ Doc/library/bdb.rst Doc/library/bisect.rst Doc/library/bz2.rst Doc/library/calendar.rst -Doc/library/cgi.rst -Doc/library/cmath.rst Doc/library/cmd.rst Doc/library/code.rst Doc/library/codecs.rst @@ -105,7 +100,6 @@ Doc/library/contextlib.rst Doc/library/copy.rst Doc/library/csv.rst Doc/library/ctypes.rst -Doc/library/curses.ascii.rst Doc/library/curses.rst Doc/library/datetime.rst Doc/library/dbm.rst @@ -134,7 +128,6 @@ Doc/library/fractions.rst Doc/library/ftplib.rst Doc/library/functions.rst Doc/library/functools.rst -Doc/library/getopt.rst Doc/library/getpass.rst Doc/library/gettext.rst Doc/library/graphlib.rst diff --git a/Doc/tools/check-warnings.py b/Doc/tools/check-warnings.py new file mode 100644 index 0000000000000..c17d0f51cd127 --- /dev/null +++ b/Doc/tools/check-warnings.py @@ -0,0 +1,149 @@ +#!/usr/bin/env python3 +""" +Check the output of running Sphinx in nit-picky mode (missing references). +""" +import argparse +import csv +import os +import re +import sys +from pathlib import Path + +# Exclude these whether they're dirty or clean, +# because they trigger a rebuild of dirty files. +EXCLUDE_FILES = { + "Doc/whatsnew/changelog.rst", +} + +# Subdirectories of Doc/ to exclude. +EXCLUDE_SUBDIRS = { + ".env", + ".venv", + "env", + "includes", + "venv", +} + +PATTERN = re.compile(r"(?P[^:]+):(?P\d+): WARNING: (?P.+)") + + +def check_and_annotate(warnings: list[str], files_to_check: str) -> None: + """ + Convert Sphinx warning messages to GitHub Actions. + + Converts lines like: + .../Doc/library/cgi.rst:98: WARNING: reference target not found + to: + ::warning file=.../Doc/library/cgi.rst,line=98::reference target not found + + Non-matching lines are echoed unchanged. + + see: + https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-a-warning-message + """ + files_to_check = next(csv.reader([files_to_check])) + for warning in warnings: + if any(filename in warning for filename in files_to_check): + if match := PATTERN.fullmatch(warning): + print("::warning file={file},line={line}::{msg}".format_map(match)) + + +def fail_if_regression( + warnings: list[str], files_with_expected_nits: set[str], files_with_nits: set[str] +) -> int: + """ + Ensure some files always pass Sphinx nit-picky mode (no missing references). + These are files which are *not* in .nitignore. + """ + all_rst = { + str(rst) + for rst in Path("Doc/").rglob("*.rst") + if rst.parts[1] not in EXCLUDE_SUBDIRS + } + should_be_clean = all_rst - files_with_expected_nits - EXCLUDE_FILES + problem_files = sorted(should_be_clean & files_with_nits) + if problem_files: + print("\nError: must not contain warnings:\n") + for filename in problem_files: + print(filename) + for warning in warnings: + if filename in warning: + if match := PATTERN.fullmatch(warning): + print(" {line}: {msg}".format_map(match)) + return -1 + return 0 + + +def fail_if_improved( + files_with_expected_nits: set[str], files_with_nits: set[str] +) -> int: + """ + We may have fixed warnings in some files so that the files are now completely clean. + Good news! Let's add them to .nitignore to prevent regression. + """ + files_with_no_nits = files_with_expected_nits - files_with_nits + if files_with_no_nits: + print("\nCongratulations! You improved:\n") + for filename in sorted(files_with_no_nits): + print(filename) + print("\nPlease remove from Doc/tools/.nitignore\n") + return -1 + return 0 + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument( + "--check-and-annotate", + help="Comma-separated list of files to check, " + "and annotate those with warnings on GitHub Actions", + ) + parser.add_argument( + "--fail-if-regression", + action="store_true", + help="Fail if known-good files have warnings", + ) + parser.add_argument( + "--fail-if-improved", + action="store_true", + help="Fail if new files with no nits are found", + ) + args = parser.parse_args() + exit_code = 0 + + wrong_directory_msg = "Must run this script from the repo root" + assert Path("Doc").exists() and Path("Doc").is_dir(), wrong_directory_msg + + with Path("Doc/sphinx-warnings.txt").open() as f: + warnings = f.read().splitlines() + + cwd = str(Path.cwd()) + os.path.sep + files_with_nits = { + warning.removeprefix(cwd).split(":")[0] + for warning in warnings + if "Doc/" in warning + } + + with Path("Doc/tools/.nitignore").open() as clean_files: + files_with_expected_nits = { + filename.strip() + for filename in clean_files + if filename.strip() and not filename.startswith("#") + } + + if args.check_and_annotate: + check_and_annotate(warnings, args.check_and_annotate) + + if args.fail_if_regression: + exit_code += fail_if_regression( + warnings, files_with_expected_nits, files_with_nits + ) + + if args.fail_if_improved: + exit_code += fail_if_improved(files_with_expected_nits, files_with_nits) + + return exit_code + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/Doc/tools/touch-clean-files.py b/Doc/tools/touch-clean-files.py deleted file mode 100644 index 2b045bd68a0cf..0000000000000 --- a/Doc/tools/touch-clean-files.py +++ /dev/null @@ -1,65 +0,0 @@ -#!/usr/bin/env python3 -""" -Touch files that must pass Sphinx nit-picky mode -so they are rebuilt and we can catch regressions. -""" -import argparse -import csv -import sys -from pathlib import Path - -wrong_directory_msg = "Must run this script from the repo root" -assert Path("Doc").exists() and Path("Doc").is_dir(), wrong_directory_msg - -# Exclude these whether they're dirty or clean, -# because they trigger a rebuild of dirty files. -EXCLUDE_FILES = { - Path("Doc/whatsnew/changelog.rst"), -} - -# Subdirectories of Doc/ to exclude. -EXCLUDE_SUBDIRS = { - ".env", - ".venv", - "env", - "includes", - "venv", -} - -ALL_RST = { - rst for rst in Path("Doc/").rglob("*.rst") if rst.parts[1] not in EXCLUDE_SUBDIRS -} - - -parser = argparse.ArgumentParser( - description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter -) -parser.add_argument("-c", "--clean", help="Comma-separated list of clean files") -args = parser.parse_args() - -if args.clean: - clean_files = next(csv.reader([args.clean])) - CLEAN = { - Path(filename.strip()) - for filename in clean_files - if Path(filename.strip()).is_file() - } -elif args.clean is not None: - print( - "Not touching any files: an empty string `--clean` arg value passed.", - ) - sys.exit(0) -else: - with Path("Doc/tools/.nitignore").open() as ignored_files: - IGNORED = { - Path(filename.strip()) - for filename in ignored_files - if filename.strip() and not filename.startswith("#") - } - CLEAN = ALL_RST - IGNORED - EXCLUDE_FILES - -print("Touching:") -for filename in sorted(CLEAN): - print(filename) - filename.touch() -print(f"Touched {len(CLEAN)} files") diff --git a/Doc/tools/warnings-to-gh-actions.py b/Doc/tools/warnings-to-gh-actions.py deleted file mode 100644 index da33a4ede07ab..0000000000000 --- a/Doc/tools/warnings-to-gh-actions.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python3 - -""" -Convert Sphinx warning messages to GitHub Actions. - -Converts lines like: - .../Doc/library/cgi.rst:98: WARNING: reference target not found -to: - ::warning file=.../Doc/library/cgi.rst,line=98::reference target not found - -Non-matching lines are echoed unchanged. - -see: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-a-warning-message -""" - -import re -import sys - -pattern = re.compile(r'(?P[^:]+):(?P\d+): WARNING: (?P.+)') - -for line in sys.stdin: - if match := pattern.fullmatch(line.strip()): - print('::warning file={file},line={line}::{msg}'.format_map(match)) - else: - print(line) From webhook-mailer at python.org Sat Jul 22 04:27:51 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 22 Jul 2023 08:27:51 -0000 Subject: [Python-checkins] gh-106973: Change non-integral to non-integer in "3.12 What's New" (#106984) Message-ID: https://github.com/python/cpython/commit/d55b4da10c56f3299998b5b8fee3a8a744fac76c commit: d55b4da10c56f3299998b5b8fee3a8a744fac76c branch: main author: Sebastiaan Zeeff <33516116+SebastiaanZ at users.noreply.github.com> committer: hugovk date: 2023-07-22T02:27:48-06:00 summary: gh-106973: Change non-integral to non-integer in "3.12 What's New" (#106984) files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 1e35427c497c8..45093122eb7b7 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1456,7 +1456,7 @@ Changes in the Python API * Removed ``randrange()`` functionality deprecated since Python 3.10. Formerly, ``randrange(10.0)`` losslessly converted to ``randrange(10)``. Now, it raises a - :exc:`TypeError`. Also, the exception raised for non-integral values such as + :exc:`TypeError`. Also, the exception raised for non-integer values such as ``randrange(10.5)`` or ``randrange('10')`` has been changed from :exc:`ValueError` to :exc:`TypeError`. This also prevents bugs where ``randrange(1e25)`` would silently select from a larger range than ``randrange(10**25)``. From webhook-mailer at python.org Sat Jul 22 04:34:52 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 08:34:52 -0000 Subject: [Python-checkins] gh-106976:alphabetise bullets by module name task1 (#106982) Message-ID: https://github.com/python/cpython/commit/443d9b3033bc6189e7f1ae936806779c02a46c43 commit: 443d9b3033bc6189e7f1ae936806779c02a46c43 branch: main author: littlebutt's workshop committer: ambv date: 2023-07-22T10:34:48+02:00 summary: gh-106976:alphabetise bullets by module name task1 (#106982) files: M Doc/whatsnew/3.13.rst diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 0203e3d18738d..a778e12aba1bc 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -150,30 +150,7 @@ Optimizations Deprecated ========== -* :mod:`wave`: Deprecate the ``getmark()``, ``setmark()`` and ``getmarkers()`` - methods of the :class:`wave.Wave_read` and :class:`wave.Wave_write` classes. - They will be removed in Python 3.15. - (Contributed by Victor Stinner in :gh:`105096`.) -* :mod:`typing`: Creating a :class:`typing.NamedTuple` class using keyword arguments to denote - the fields (``NT = NamedTuple("NT", x=int, y=int)``) is deprecated, and will - be disallowed in Python 3.15. Use the class-based syntax or the functional - syntax instead. (Contributed by Alex Waygood in :gh:`105566`.) -* :mod:`typing`: When using the functional syntax to create a :class:`typing.NamedTuple` - class or a :class:`typing.TypedDict` class, failing to pass a value to the - 'fields' parameter (``NT = NamedTuple("NT")`` or ``TD = TypedDict("TD")``) is - deprecated. Passing ``None`` to the 'fields' parameter - (``NT = NamedTuple("NT", None)`` or ``TD = TypedDict("TD", None)``) is also - deprecated. Both will be disallowed in Python 3.15. To create a NamedTuple - class with 0 fields, use ``class NT(NamedTuple): pass`` or - ``NT = NamedTuple("NT", [])``. To create a TypedDict class with 0 fields, use - ``class TD(TypedDict): pass`` or ``TD = TypedDict("TD", {})``. - (Contributed by Alex Waygood in :gh:`105566` and :gh:`105570`.) -* :func:`typing.no_type_check_decorator` is deprecated, and scheduled for - removal in Python 3.15. After eight years in the :mod:`typing` module, it - has yet to be supported by any major type checkers. - (Contributed by Alex Waygood in :gh:`106309`.) - -* :mod:`array`'s ``'u'`` format code, deprecated in docs since Python 3.3, +* :mod:`array`: :mod:`array`'s ``'u'`` format code, deprecated in docs since Python 3.3, emits :exc:`DeprecationWarning` since 3.13 and will be removed in Python 3.16. Use the ``'w'`` format code instead. @@ -184,13 +161,39 @@ Deprecated Replace ``ctypes.ARRAY(item_type, size)`` with ``item_type * size``. (Contributed by Victor Stinner in :gh:`105733`.) -* The :mod:`getopt` and :mod:`optparse` modules are now +* :mod:`getopt` and :mod:`optparse` modules: They are now :term:`soft deprecated`: the :mod:`argparse` should be used for new projects. Previously, the :mod:`optparse` module was already deprecated, its removal was not scheduled, and no warnings was emitted: so there is no change in practice. (Contributed by Victor Stinner in :gh:`106535`.) +* :mod:`typing`: Creating a :class:`typing.NamedTuple` class using keyword arguments to denote + the fields (``NT = NamedTuple("NT", x=int, y=int)``) is deprecated, and will + be disallowed in Python 3.15. Use the class-based syntax or the functional + syntax instead. (Contributed by Alex Waygood in :gh:`105566`.) + + * When using the functional syntax to create a :class:`typing.NamedTuple` + class or a :class:`typing.TypedDict` class, failing to pass a value to the + 'fields' parameter (``NT = NamedTuple("NT")`` or ``TD = TypedDict("TD")``) is + deprecated. Passing ``None`` to the 'fields' parameter + (``NT = NamedTuple("NT", None)`` or ``TD = TypedDict("TD", None)``) is also + deprecated. Both will be disallowed in Python 3.15. To create a NamedTuple + class with 0 fields, use ``class NT(NamedTuple): pass`` or + ``NT = NamedTuple("NT", [])``. To create a TypedDict class with 0 fields, use + ``class TD(TypedDict): pass`` or ``TD = TypedDict("TD", {})``. + (Contributed by Alex Waygood in :gh:`105566` and :gh:`105570`.) + + * :func:`typing.no_type_check_decorator` is deprecated, and scheduled for + removal in Python 3.15. After eight years in the :mod:`typing` module, it + has yet to be supported by any major type checkers. + (Contributed by Alex Waygood in :gh:`106309`.) + +* :mod:`wave`: Deprecate the ``getmark()``, ``setmark()`` and ``getmarkers()`` + methods of the :class:`wave.Wave_read` and :class:`wave.Wave_write` classes. + They will be removed in Python 3.15. + (Contributed by Victor Stinner in :gh:`105096`.) + Pending Removal in Python 3.14 ------------------------------ From webhook-mailer at python.org Sat Jul 22 04:37:19 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 22 Jul 2023 08:37:19 -0000 Subject: [Python-checkins] [3.11] Change non-integral to non-integer in random docs (GH-106975) (#106985) Message-ID: https://github.com/python/cpython/commit/4f7c23e4dfb4b932b3916f8f79895e2a1bf3ae43 commit: 4f7c23e4dfb4b932b3916f8f79895e2a1bf3ae43 branch: 3.11 author: Sebastiaan Zeeff <33516116+SebastiaanZ at users.noreply.github.com> committer: hugovk date: 2023-07-22T02:37:16-06:00 summary: [3.11] Change non-integral to non-integer in random docs (GH-106975) (#106985) files: M Doc/library/random.rst diff --git a/Doc/library/random.rst b/Doc/library/random.rst index 3109343a3c52d..4af4f83e280ca 100644 --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -141,7 +141,7 @@ Functions for integers ``randrange(10)``. In the future, this will raise a :exc:`TypeError`. .. deprecated:: 3.10 - The exception raised for non-integral values such as ``randrange(10.5)`` + The exception raised for non-integer values such as ``randrange(10.5)`` or ``randrange('10')`` will be changed from :exc:`ValueError` to :exc:`TypeError`. From webhook-mailer at python.org Sat Jul 22 04:40:41 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 22 Jul 2023 08:40:41 -0000 Subject: [Python-checkins] [3.12] gh-106973: Change non-integral to non-integer in "3.12 What's New" (GH-106984) (#106986) Message-ID: https://github.com/python/cpython/commit/9cbde7c6ce6f7b93301a37f03dfa0c0d45e00a39 commit: 9cbde7c6ce6f7b93301a37f03dfa0c0d45e00a39 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-22T02:40:37-06:00 summary: [3.12] gh-106973: Change non-integral to non-integer in "3.12 What's New" (GH-106984) (#106986) Co-authored-by: Sebastiaan Zeeff <33516116+SebastiaanZ at users.noreply.github.com> files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 9d05352f3b887..a119f77bdceb4 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1455,7 +1455,7 @@ Changes in the Python API * Removed ``randrange()`` functionality deprecated since Python 3.10. Formerly, ``randrange(10.0)`` losslessly converted to ``randrange(10)``. Now, it raises a - :exc:`TypeError`. Also, the exception raised for non-integral values such as + :exc:`TypeError`. Also, the exception raised for non-integer values such as ``randrange(10.5)`` or ``randrange('10')`` has been changed from :exc:`ValueError` to :exc:`TypeError`. This also prevents bugs where ``randrange(1e25)`` would silently select from a larger range than ``randrange(10**25)``. From webhook-mailer at python.org Sat Jul 22 05:06:50 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 22 Jul 2023 09:06:50 -0000 Subject: [Python-checkins] [3.12] gh-106970: Fix Argument Clinic 'destination clear' command (GH-106972) (#106983) Message-ID: https://github.com/python/cpython/commit/67748f18c9b53def23f7614759fb632165f50d6a commit: 67748f18c9b53def23f7614759fb632165f50d6a branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-22T11:06:46+02:00 summary: [3.12] gh-106970: Fix Argument Clinic 'destination clear' command (GH-106972) (#106983) Add test for the 'destination clear' command, and the 'destination' directive in general. Fix two bugs in 'destination clear' command: 1. The text attribute of the allocator is called 'text', not '_text' 2. Return after processing the 'clear' command, instead of proceeding directly to the fail(). (cherry picked from commit 3372bcba9893030e4063a9264ec0b4d1b6166883) Co-authored-by: Erlend E. Aasland files: A Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst M Lib/test/clinic.test.c M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index 5748bbc71df5a..04227e41a4d50 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -4954,3 +4954,59 @@ Test_meth_coexist(TestObj *self, PyObject *Py_UNUSED(ignored)) static PyObject * Test_meth_coexist_impl(TestObj *self) /*[clinic end generated code: output=808a293d0cd27439 input=2a1d75b5e6fec6dd]*/ + + +/*[clinic input] +output push +output preset buffer +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=5bff3376ee0df0b5]*/ + +/*[clinic input] +buffer_clear + a: int +We'll call 'destination buffer clear' after this. + +Argument Clinic's buffer preset puts most generated code into the +'buffer' destination, except from 'impl_definition', which is put into +the 'block' destination, so we should expect everything but +'impl_definition' to be cleared. +[clinic start generated code]*/ + +static PyObject * +buffer_clear_impl(PyObject *module, int a) +/*[clinic end generated code: output=f14bba74677e1846 input=a4c308a6fdab043c]*/ + +/*[clinic input] +destination buffer clear +output pop +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=f20d06adb8252084]*/ + + +/*[clinic input] +output push +destination test1 new buffer +output everything suppress +output docstring_definition test1 +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=5a77c454970992fc]*/ + +/*[clinic input] +new_dest + a: int +Only this docstring should be outputted to test1. +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=da5af421ed8996ed]*/ + +/*[clinic input] +dump test1 +output pop +[clinic start generated code]*/ + +PyDoc_STRVAR(new_dest__doc__, +"new_dest($module, /, a)\n" +"--\n" +"\n" +"Only this docstring should be outputted to test1."); +/*[clinic end generated code: output=9cac703f51d90e84 input=090db8df4945576d]*/ diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 7c725e33f5392..e40579b0041cf 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -249,6 +249,15 @@ def test_cpp_monitor_fail_no_matching_if(self): out = self.expect_failure(raw) self.assertEqual(out, msg) + def test_unknown_destination_command(self): + out = self.expect_failure(""" + /*[clinic input] + destination buffer nosuchcommand + [clinic start generated code]*/ + """) + msg = "unknown destination command 'nosuchcommand'" + self.assertIn(msg, out) + class ClinicGroupPermuterTest(TestCase): def _test(self, l, m, r, output): diff --git a/Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst b/Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst new file mode 100644 index 0000000000000..194e3351b0470 --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst @@ -0,0 +1,4 @@ +Fix bugs in the Argument Clinic ``destination clear`` command; the +destination buffers would never be cleared, and the ``destination`` +directive parser would simply continue to the fault handler after processing +the command. Patch by Erlend E. Aasland. diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 04951e97bbd80..3e60328082d66 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1906,7 +1906,7 @@ def __getitem__(self, i): def clear(self): for ta in self._array: - ta._text.clear() + ta.text.clear() def dump(self): texts = [ta.output() for ta in self._array] @@ -4366,13 +4366,13 @@ def directive_destination( command: str, *args ) -> None: - if command == 'new': - self.clinic.add_destination(name, *args) - return - - if command == 'clear': - self.clinic.get_destination(name).clear() - fail("unknown destination command", repr(command)) + match command: + case "new": + self.clinic.add_destination(name, *args) + case "clear": + self.clinic.get_destination(name).clear() + case _: + fail("unknown destination command", repr(command)) def directive_output( From webhook-mailer at python.org Sat Jul 22 05:32:14 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 22 Jul 2023 09:32:14 -0000 Subject: [Python-checkins] gh-106368: Increase coverage for Argument Clinic output directive (#106979) Message-ID: https://github.com/python/cpython/commit/ee5c01b473eeadb007b9f330db3143e34e46038b commit: ee5c01b473eeadb007b9f330db3143e34e46038b branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-22T09:32:10Z summary: gh-106368: Increase coverage for Argument Clinic output directive (#106979) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index e40579b0041cf..c24b8cd3899f2 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -249,6 +249,59 @@ def test_cpp_monitor_fail_no_matching_if(self): out = self.expect_failure(raw) self.assertEqual(out, msg) + def test_directive_output_unknown_preset(self): + out = self.expect_failure(""" + /*[clinic input] + output preset nosuchpreset + [clinic start generated code]*/ + """) + msg = "Unknown preset 'nosuchpreset'" + self.assertIn(msg, out) + + def test_directive_output_cant_pop(self): + out = self.expect_failure(""" + /*[clinic input] + output pop + [clinic start generated code]*/ + """) + msg = "Can't 'output pop', stack is empty" + self.assertIn(msg, out) + + def test_directive_output_print(self): + raw = dedent(""" + /*[clinic input] + output print 'I told you once.' + [clinic start generated code]*/ + """) + out = self.clinic.parse(raw) + # The generated output will differ for every run, but we can check that + # it starts with the clinic block, we check that it contains all the + # expected fields, and we check that it contains the checksum line. + self.assertTrue(out.startswith(dedent(""" + /*[clinic input] + output print 'I told you once.' + [clinic start generated code]*/ + """))) + fields = { + "cpp_endif", + "cpp_if", + "docstring_definition", + "docstring_prototype", + "impl_definition", + "impl_prototype", + "methoddef_define", + "methoddef_ifndef", + "parser_definition", + "parser_prototype", + } + for field in fields: + with self.subTest(field=field): + self.assertIn(field, out) + last_line = out.rstrip().split("\n")[-1] + self.assertTrue( + last_line.startswith("/*[clinic end generated code: output=") + ) + def test_unknown_destination_command(self): out = self.expect_failure(""" /*[clinic input] From webhook-mailer at python.org Sat Jul 22 05:38:51 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 22 Jul 2023 09:38:51 -0000 Subject: [Python-checkins] gh-106978: Bump sphinx-lint to 0.6.8 (#106990) Message-ID: https://github.com/python/cpython/commit/6acd85d91063380d1afb86c116a66e0aadd4f909 commit: 6acd85d91063380d1afb86c116a66e0aadd4f909 branch: main author: Sven Arends <88426829+picnic-sven at users.noreply.github.com> committer: hugovk date: 2023-07-22T09:38:47Z summary: gh-106978: Bump sphinx-lint to 0.6.8 (#106990) files: M .pre-commit-config.yaml diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index d62c57c044728..85a6de4abe014 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -10,7 +10,7 @@ repos: types_or: [c, python, rst] - repo: https://github.com/sphinx-contrib/sphinx-lint - rev: v0.6.7 + rev: v0.6.8 hooks: - id: sphinx-lint args: [--enable=default-role] From webhook-mailer at python.org Sat Jul 22 06:31:48 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 22 Jul 2023 10:31:48 -0000 Subject: [Python-checkins] gh-106996: Amend the introduction to the turtle graphics documentation (#106997) Message-ID: https://github.com/python/cpython/commit/f8f16d0cfcbe287deaaf1a6eb6461b11b7837a1c commit: f8f16d0cfcbe287deaaf1a6eb6461b11b7837a1c branch: main author: Daniele Procida committer: hugovk date: 2023-07-22T04:31:44-06:00 summary: gh-106996: Amend the introduction to the turtle graphics documentation (#106997) Co-authored-by: Hugo van Kemenade files: M Doc/library/turtle.rst diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index c9ce955a6d2ba..f14a677b7dd88 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -19,9 +19,27 @@ Introduction ============ -Turtle graphics is a popular way for introducing programming to kids. It was -part of the original Logo programming language developed by Wally Feurzeig, -Seymour Papert and Cynthia Solomon in 1967. +Turtle graphics is an implementation of `the popular geometric drawing tools +introduced in Logo `_, developed by Wally Feurzeig, Seymour Papert and Cynthia Solomon +in 1967. + +In Python, turtle graphics provides a representation of a physical "turtle" +(a little robot with a pen) that draws on a sheet of paper on the floor. + +It's an effective and well-proven way for learners to encounter +programming concepts and interaction with software, as it provides instant, +visible feedback. It also provides convenient access to graphical output +in general. + +Turtle drawing was originally created as an educational tool, to be used by +teachers in the classroom. For the programmer who needs to produce some +graphical output it can be a way to do that without the overhead of +introducing more complex or external libraries into their work. + + +Get started +=========== Imagine a robotic turtle starting at (0, 0) in the x-y plane. After an ``import turtle``, give it the command ``turtle.forward(15)``, and it moves (on-screen!) 15 pixels in the From webhook-mailer at python.org Sat Jul 22 06:46:46 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 22 Jul 2023 10:46:46 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Annotate the BufferSeries class (#106935) Message-ID: https://github.com/python/cpython/commit/c5adf26b1867767781af30ada6f05a29449fc650 commit: c5adf26b1867767781af30ada6f05a29449fc650 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-22T12:46:42+02:00 summary: gh-104050: Argument Clinic: Annotate the BufferSeries class (#106935) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 8acf513a6c1fc..7a4c9c4cacf55 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1965,12 +1965,12 @@ class BufferSeries: e.g. o[-1] is an element immediately preceding o[0]. """ - def __init__(self): + def __init__(self) -> None: self._start = 0 - self._array = [] + self._array: list[_TextAccumulator] = [] self._constructor = _text_accumulator - def __getitem__(self, i): + def __getitem__(self, i: int) -> _TextAccumulator: i -= self._start if i < 0: self._start += i @@ -1981,11 +1981,11 @@ def __getitem__(self, i): self._array.append(self._constructor()) return self._array[i] - def clear(self): + def clear(self) -> None: for ta in self._array: ta.text.clear() - def dump(self): + def dump(self) -> str: texts = [ta.output() for ta in self._array] return "".join(texts) From webhook-mailer at python.org Sat Jul 22 06:48:38 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 22 Jul 2023 10:48:38 -0000 Subject: [Python-checkins] [3.12] gh-106368: Increase coverage for Argument Clinic output directive (GH-106979) (#106994) Message-ID: https://github.com/python/cpython/commit/76fd98a675d0c1a4f61735f9038e1f8282b60177 commit: 76fd98a675d0c1a4f61735f9038e1f8282b60177 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-22T12:48:35+02:00 summary: [3.12] gh-106368: Increase coverage for Argument Clinic output directive (GH-106979) (#106994) (cherry picked from commit ee5c01b473eeadb007b9f330db3143e34e46038b) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index e40579b0041cf..c24b8cd3899f2 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -249,6 +249,59 @@ def test_cpp_monitor_fail_no_matching_if(self): out = self.expect_failure(raw) self.assertEqual(out, msg) + def test_directive_output_unknown_preset(self): + out = self.expect_failure(""" + /*[clinic input] + output preset nosuchpreset + [clinic start generated code]*/ + """) + msg = "Unknown preset 'nosuchpreset'" + self.assertIn(msg, out) + + def test_directive_output_cant_pop(self): + out = self.expect_failure(""" + /*[clinic input] + output pop + [clinic start generated code]*/ + """) + msg = "Can't 'output pop', stack is empty" + self.assertIn(msg, out) + + def test_directive_output_print(self): + raw = dedent(""" + /*[clinic input] + output print 'I told you once.' + [clinic start generated code]*/ + """) + out = self.clinic.parse(raw) + # The generated output will differ for every run, but we can check that + # it starts with the clinic block, we check that it contains all the + # expected fields, and we check that it contains the checksum line. + self.assertTrue(out.startswith(dedent(""" + /*[clinic input] + output print 'I told you once.' + [clinic start generated code]*/ + """))) + fields = { + "cpp_endif", + "cpp_if", + "docstring_definition", + "docstring_prototype", + "impl_definition", + "impl_prototype", + "methoddef_define", + "methoddef_ifndef", + "parser_definition", + "parser_prototype", + } + for field in fields: + with self.subTest(field=field): + self.assertIn(field, out) + last_line = out.rstrip().split("\n")[-1] + self.assertTrue( + last_line.startswith("/*[clinic end generated code: output=") + ) + def test_unknown_destination_command(self): out = self.expect_failure(""" /*[clinic input] From webhook-mailer at python.org Sat Jul 22 08:13:47 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 12:13:47 -0000 Subject: [Python-checkins] gh-106969: Indicate no modules were added in 3.10 & 3.12 (#106988) Message-ID: https://github.com/python/cpython/commit/6dbffaed17d59079d6a2788d686009f762a3278f commit: 6dbffaed17d59079d6a2788d686009f762a3278f branch: main author: Sebastiaan Zeeff <33516116+SebastiaanZ at users.noreply.github.com> committer: ambv date: 2023-07-22T14:13:44+02:00 summary: gh-106969: Indicate no modules were added in 3.10 & 3.12 (#106988) The "New Modules" section was left in place to ensure that the anchor link for new modules will still exist: /whatsnew/3.12.html#new-modules /whatsnew/3.10.html#new-modules This means that existing links to this section don't break. files: M Doc/whatsnew/3.10.rst M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index d1d2193de4a95..e4ca3c4c81ba5 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -887,7 +887,7 @@ Other Language Changes New Modules =========== -* None yet. +* None. Improved Modules diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 45093122eb7b7..a46621071b3d8 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -486,7 +486,7 @@ Other Language Changes New Modules =========== -* None yet. +* None. Improved Modules From webhook-mailer at python.org Sat Jul 22 08:15:21 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 12:15:21 -0000 Subject: [Python-checkins] [3.11] gh-106368: Increase coverage for Argument Clinic output directive (GH-106979) (#107002) Message-ID: https://github.com/python/cpython/commit/06f8a439956fdf5f2d68bc3b5b6e2499981ce21f commit: 06f8a439956fdf5f2d68bc3b5b6e2499981ce21f branch: 3.11 author: ?ukasz Langa committer: ambv date: 2023-07-22T14:15:18+02:00 summary: [3.11] gh-106368: Increase coverage for Argument Clinic output directive (GH-106979) (#107002) (cherry picked from commit ee5c01b473eeadb007b9f330db3143e34e46038b) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 605d101b5bef9..6520966b345d1 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -249,6 +249,68 @@ def test_cpp_monitor_fail_no_matching_if(self): out = self.expect_failure(raw) self.assertEqual(out, msg) + def test_directive_output_unknown_preset(self): + out = self.expect_failure(""" + /*[clinic input] + output preset nosuchpreset + [clinic start generated code]*/ + """) + msg = "Unknown preset 'nosuchpreset'" + self.assertIn(msg, out) + + def test_directive_output_cant_pop(self): + out = self.expect_failure(""" + /*[clinic input] + output pop + [clinic start generated code]*/ + """) + msg = "Can't 'output pop', stack is empty" + self.assertIn(msg, out) + + def test_directive_output_print(self): + raw = dedent(""" + /*[clinic input] + output print 'I told you once.' + [clinic start generated code]*/ + """) + out = self.clinic.parse(raw) + # The generated output will differ for every run, but we can check that + # it starts with the clinic block, we check that it contains all the + # expected fields, and we check that it contains the checksum line. + self.assertTrue(out.startswith(dedent(""" + /*[clinic input] + output print 'I told you once.' + [clinic start generated code]*/ + """))) + fields = { + "cpp_endif", + "cpp_if", + "docstring_definition", + "docstring_prototype", + "impl_definition", + "impl_prototype", + "methoddef_define", + "methoddef_ifndef", + "parser_definition", + "parser_prototype", + } + for field in fields: + with self.subTest(field=field): + self.assertIn(field, out) + last_line = out.rstrip().split("\n")[-1] + self.assertTrue( + last_line.startswith("/*[clinic end generated code: output=") + ) + + def test_unknown_destination_command(self): + out = self.expect_failure(""" + /*[clinic input] + destination buffer nosuchcommand + [clinic start generated code]*/ + """) + msg = "unknown destination command 'nosuchcommand'" + self.assertIn(msg, out) + class ClinicGroupPermuterTest(TestCase): def _test(self, l, m, r, output): From webhook-mailer at python.org Sat Jul 22 08:17:28 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 12:17:28 -0000 Subject: [Python-checkins] gh-106714: Fix test_capi to not write a coredump (#107007) Message-ID: https://github.com/python/cpython/commit/4a1026d7647c084b0dc80dd49163d16ba12a2e55 commit: 4a1026d7647c084b0dc80dd49163d16ba12a2e55 branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T12:17:25Z summary: gh-106714: Fix test_capi to not write a coredump (#107007) test_capi: Fix test_no_FatalError_infinite_loop() to no longer write a coredump, by using test.support.SuppressCrashReport. files: A Misc/NEWS.d/next/Tests/2023-07-22-13-49-40.gh-issue-106714.btYI5S.rst M Lib/test/test_capi/test_misc.py diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index e40ffdfd12151..dc155cc99961c 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -85,9 +85,15 @@ def test_instancemethod(self): @support.requires_subprocess() def test_no_FatalError_infinite_loop(self): - run_result, _cmd_line = run_python_until_end( - '-c', 'import _testcapi; _testcapi.crash_no_current_thread()', - ) + code = textwrap.dedent(""" + import _testcapi + from test import support + + with support.SuppressCrashReport(): + _testcapi.crash_no_current_thread() + """) + + run_result, _cmd_line = run_python_until_end('-c', code) _rc, out, err = run_result self.assertEqual(out, b'') # This used to cause an infinite loop. diff --git a/Misc/NEWS.d/next/Tests/2023-07-22-13-49-40.gh-issue-106714.btYI5S.rst b/Misc/NEWS.d/next/Tests/2023-07-22-13-49-40.gh-issue-106714.btYI5S.rst new file mode 100644 index 0000000000000..955620521c8f6 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-22-13-49-40.gh-issue-106714.btYI5S.rst @@ -0,0 +1,3 @@ +test_capi: Fix test_no_FatalError_infinite_loop() to no longer write a +coredump, by using test.support.SuppressCrashReport. Patch by Victor +Stinner. From webhook-mailer at python.org Sat Jul 22 08:19:34 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 12:19:34 -0000 Subject: [Python-checkins] Reformat code block to make it easier to read (#106965) Message-ID: https://github.com/python/cpython/commit/ed491d9f782480fb00535abcf667027e0e323287 commit: ed491d9f782480fb00535abcf667027e0e323287 branch: main author: Joe Kaufeld committer: ambv date: 2023-07-22T14:19:30+02:00 summary: Reformat code block to make it easier to read (#106965) files: M Doc/tutorial/errors.rst diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index 6419ff621f1b3..8a207c385c6ab 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -535,11 +535,20 @@ of a certain type while letting all other exceptions propagate to other clauses and eventually to be reraised. :: >>> def f(): - ... raise ExceptionGroup("group1", - ... [OSError(1), - ... SystemError(2), - ... ExceptionGroup("group2", - ... [OSError(3), RecursionError(4)])]) + ... raise ExceptionGroup( + ... "group1", + ... [ + ... OSError(1), + ... SystemError(2), + ... ExceptionGroup( + ... "group2", + ... [ + ... OSError(3), + ... RecursionError(4) + ... ] + ... ) + ... ] + ... ) ... >>> try: ... f() From webhook-mailer at python.org Sat Jul 22 08:21:58 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 12:21:58 -0000 Subject: [Python-checkins] gh-105699: Add some stress tests for subinterpreter creation (#106966) Message-ID: https://github.com/python/cpython/commit/adda43dc0bcea853cbfa33126e5549c584cef8be commit: adda43dc0bcea853cbfa33126e5549c584cef8be branch: main author: Eric Snow committer: ambv date: 2023-07-22T14:21:55+02:00 summary: gh-105699: Add some stress tests for subinterpreter creation (#106966) files: M Lib/test/test_interpreters.py diff --git a/Lib/test/test_interpreters.py b/Lib/test/test_interpreters.py index d1bebe4715832..5981d96de8de0 100644 --- a/Lib/test/test_interpreters.py +++ b/Lib/test/test_interpreters.py @@ -7,6 +7,7 @@ from test import support from test.support import import_helper +from test.support import threading_helper _interpreters = import_helper.import_module('_xxsubinterpreters') _channels = import_helper.import_module('_xxinterpchannels') from test.support import interpreters @@ -463,6 +464,27 @@ def test_bytes_for_script(self): # test_xxsubinterpreters covers the remaining Interpreter.run() behavior. +class StressTests(TestBase): + + # In these tests we generally want a lot of interpreters, + # but not so many that any test takes too long. + + def test_create_many_sequential(self): + alive = [] + for _ in range(100): + interp = interpreters.create() + alive.append(interp) + + def test_create_many_threaded(self): + alive = [] + def task(): + interp = interpreters.create() + alive.append(interp) + threads = (threading.Thread(target=task) for _ in range(200)) + with threading_helper.start_threads(threads): + pass + + class TestIsShareable(TestBase): def test_default_shareables(self): From webhook-mailer at python.org Sat Jul 22 08:23:27 2023 From: webhook-mailer at python.org (pablogsal) Date: Sat, 22 Jul 2023 12:23:27 -0000 Subject: [Python-checkins] gh-106989: Remove tok report warnings (#106993) Message-ID: https://github.com/python/cpython/commit/76e20c361c8d6bc20d939d436a1c3d4077a58186 commit: 76e20c361c8d6bc20d939d436a1c3d4077a58186 branch: main author: Menelaos Kotoglou committer: pablogsal date: 2023-07-22T14:23:23+02:00 summary: gh-106989: Remove tok report warnings (#106993) files: M Parser/tokenizer.c M Parser/tokenizer.h diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index f19198600fa01..ccff16045233d 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -116,7 +116,6 @@ tok_new(void) tok->implicit_newline = 0; tok->tok_mode_stack[0] = (tokenizer_mode){.kind =TOK_REGULAR_MODE, .f_string_quote='\0', .f_string_quote_size = 0, .f_string_debug=0}; tok->tok_mode_stack_index = 0; - tok->tok_report_warnings = 1; #ifdef Py_DEBUG tok->debug = _Py_GetConfig()->parser_debug; #endif @@ -1545,10 +1544,6 @@ static int warn_invalid_escape_sequence(struct tok_state *tok, int first_invalid_escape_char) { - if (!tok->tok_report_warnings) { - return 0; - } - PyObject *msg = PyUnicode_FromFormat( "invalid escape sequence '\\%c'", (char) first_invalid_escape_char diff --git a/Parser/tokenizer.h b/Parser/tokenizer.h index cb44845c1d306..1e1daa3648f5d 100644 --- a/Parser/tokenizer.h +++ b/Parser/tokenizer.h @@ -128,7 +128,6 @@ struct tok_state { // TODO: Factor this into its own thing tokenizer_mode tok_mode_stack[MAXFSTRINGLEVEL]; int tok_mode_stack_index; - int tok_report_warnings; int tok_extra_tokens; int comment_newline; int implicit_newline; From webhook-mailer at python.org Sat Jul 22 08:43:47 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 22 Jul 2023 12:43:47 -0000 Subject: [Python-checkins] [3.11] gh-100700 Remove Date and Release fields in past whatsnews (GH-100728) (#106999) Message-ID: https://github.com/python/cpython/commit/a7a973e9c7c50d37681ebfaf67a44f4d08c202df commit: a7a973e9c7c50d37681ebfaf67a44f4d08c202df branch: 3.11 author: Oliver Rew <12968411+oliver-rew at users.noreply.github.com> committer: hugovk date: 2023-07-22T12:43:43Z summary: [3.11] gh-100700 Remove Date and Release fields in past whatsnews (GH-100728) (#106999) Co-authored-by: Zachary Ware files: M Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index c437c5bfb6120..f31b439ed2fd4 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -2,8 +2,6 @@ What's New In Python 3.11 **************************** -:Release: |release| -:Date: |today| :Editor: Pablo Galindo Salgado .. Rules for maintenance: From webhook-mailer at python.org Sat Jul 22 08:48:45 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 12:48:45 -0000 Subject: [Python-checkins] [3.12] gh-106714: Fix test_capi to not write a coredump (GH-107007) (#107009) Message-ID: https://github.com/python/cpython/commit/713590f9b2bef0641511b5195bdbb4c9253e9b8c commit: 713590f9b2bef0641511b5195bdbb4c9253e9b8c branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-22T12:48:41Z summary: [3.12] gh-106714: Fix test_capi to not write a coredump (GH-107007) (#107009) gh-106714: Fix test_capi to not write a coredump (GH-107007) test_capi: Fix test_no_FatalError_infinite_loop() to no longer write a coredump, by using test.support.SuppressCrashReport. (cherry picked from commit 4a1026d7647c084b0dc80dd49163d16ba12a2e55) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Tests/2023-07-22-13-49-40.gh-issue-106714.btYI5S.rst M Lib/test/test_capi/test_misc.py diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 61947cb794519..cd37fc71aa966 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -85,9 +85,15 @@ def test_instancemethod(self): @support.requires_subprocess() def test_no_FatalError_infinite_loop(self): - run_result, _cmd_line = run_python_until_end( - '-c', 'import _testcapi; _testcapi.crash_no_current_thread()', - ) + code = textwrap.dedent(""" + import _testcapi + from test import support + + with support.SuppressCrashReport(): + _testcapi.crash_no_current_thread() + """) + + run_result, _cmd_line = run_python_until_end('-c', code) _rc, out, err = run_result self.assertEqual(out, b'') # This used to cause an infinite loop. diff --git a/Misc/NEWS.d/next/Tests/2023-07-22-13-49-40.gh-issue-106714.btYI5S.rst b/Misc/NEWS.d/next/Tests/2023-07-22-13-49-40.gh-issue-106714.btYI5S.rst new file mode 100644 index 0000000000000..955620521c8f6 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-22-13-49-40.gh-issue-106714.btYI5S.rst @@ -0,0 +1,3 @@ +test_capi: Fix test_no_FatalError_infinite_loop() to no longer write a +coredump, by using test.support.SuppressCrashReport. Patch by Victor +Stinner. From webhook-mailer at python.org Sat Jul 22 08:54:17 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 12:54:17 -0000 Subject: [Python-checkins] [3.12] Reformat code block to make it easier to read (GH-106965) (#107010) Message-ID: https://github.com/python/cpython/commit/e7757ab9e13b88a37c4490751782d010b0b44015 commit: e7757ab9e13b88a37c4490751782d010b0b44015 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-22T14:54:13+02:00 summary: [3.12] Reformat code block to make it easier to read (GH-106965) (#107010) (cherry picked from commit ed491d9f782480fb00535abcf667027e0e323287) Co-authored-by: Joe Kaufeld files: M Doc/tutorial/errors.rst diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index 6419ff621f1b3..8a207c385c6ab 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -535,11 +535,20 @@ of a certain type while letting all other exceptions propagate to other clauses and eventually to be reraised. :: >>> def f(): - ... raise ExceptionGroup("group1", - ... [OSError(1), - ... SystemError(2), - ... ExceptionGroup("group2", - ... [OSError(3), RecursionError(4)])]) + ... raise ExceptionGroup( + ... "group1", + ... [ + ... OSError(1), + ... SystemError(2), + ... ExceptionGroup( + ... "group2", + ... [ + ... OSError(3), + ... RecursionError(4) + ... ] + ... ) + ... ] + ... ) ... >>> try: ... f() From webhook-mailer at python.org Sat Jul 22 09:01:39 2023 From: webhook-mailer at python.org (pablogsal) Date: Sat, 22 Jul 2023 13:01:39 -0000 Subject: [Python-checkins] [3.12] gh-106989: Remove tok report warnings (GH-106993) (#107013) Message-ID: https://github.com/python/cpython/commit/b6ace7516b032068a8b539a6b312c50bdde7c012 commit: b6ace7516b032068a8b539a6b312c50bdde7c012 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: pablogsal date: 2023-07-22T13:01:36Z summary: [3.12] gh-106989: Remove tok report warnings (GH-106993) (#107013) Co-authored-by: Menelaos Kotoglou files: M Parser/tokenizer.c M Parser/tokenizer.h diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 115b497cc2409..7e246d2f56481 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -117,7 +117,6 @@ tok_new(void) tok->implicit_newline = 0; tok->tok_mode_stack[0] = (tokenizer_mode){.kind =TOK_REGULAR_MODE, .f_string_quote='\0', .f_string_quote_size = 0, .f_string_debug=0}; tok->tok_mode_stack_index = 0; - tok->tok_report_warnings = 1; #ifdef Py_DEBUG tok->debug = _Py_GetConfig()->parser_debug; #endif @@ -1546,10 +1545,6 @@ static int warn_invalid_escape_sequence(struct tok_state *tok, int first_invalid_escape_char) { - if (!tok->tok_report_warnings) { - return 0; - } - PyObject *msg = PyUnicode_FromFormat( "invalid escape sequence '\\%c'", (char) first_invalid_escape_char diff --git a/Parser/tokenizer.h b/Parser/tokenizer.h index cb44845c1d306..1e1daa3648f5d 100644 --- a/Parser/tokenizer.h +++ b/Parser/tokenizer.h @@ -128,7 +128,6 @@ struct tok_state { // TODO: Factor this into its own thing tokenizer_mode tok_mode_stack[MAXFSTRINGLEVEL]; int tok_mode_stack_index; - int tok_report_warnings; int tok_extra_tokens; int comment_newline; int implicit_newline; From webhook-mailer at python.org Sat Jul 22 09:15:08 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 13:15:08 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyUnicode_AsString() alias (#107021) Message-ID: https://github.com/python/cpython/commit/463b56da1265a17e4207f651d6d5835a8d6de2dd commit: 463b56da1265a17e4207f651d6d5835a8d6de2dd branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T13:15:05Z summary: gh-106320: Remove private _PyUnicode_AsString() alias (#107021) Remove private _PyUnicode_AsString() alias to PyUnicode_AsUTF8(). It was kept for backward compatibility with Python 3.0 - 3.2. The PyUnicode_AsUTF8() is available since Python 3.3. The PyUnicode_AsUTF8String() function can be used to keep compatibility with Python 3.2 and older. files: A Misc/NEWS.d/next/C API/2023-07-22-14-40-48.gh-issue-106320.H3u7x4.rst M Include/cpython/unicodeobject.h diff --git a/Include/cpython/unicodeobject.h b/Include/cpython/unicodeobject.h index e75b5e154943d..51eb998db4ab6 100644 --- a/Include/cpython/unicodeobject.h +++ b/Include/cpython/unicodeobject.h @@ -446,17 +446,12 @@ PyAPI_FUNC(PyObject*) PyUnicode_FromKindAndData( Like PyUnicode_AsUTF8AndSize(), this also caches the UTF-8 representation in the unicodeobject. - _PyUnicode_AsString is a #define for PyUnicode_AsUTF8 to - support the previous internal function with the same behaviour. - Use of this API is DEPRECATED since no size information can be extracted from the returned data. */ PyAPI_FUNC(const char *) PyUnicode_AsUTF8(PyObject *unicode); -#define _PyUnicode_AsString PyUnicode_AsUTF8 - /* === Characters Type APIs =============================================== */ /* These should not be used directly. Use the Py_UNICODE_IS* and diff --git a/Misc/NEWS.d/next/C API/2023-07-22-14-40-48.gh-issue-106320.H3u7x4.rst b/Misc/NEWS.d/next/C API/2023-07-22-14-40-48.gh-issue-106320.H3u7x4.rst new file mode 100644 index 0000000000000..1e0ba0d71e755 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-22-14-40-48.gh-issue-106320.H3u7x4.rst @@ -0,0 +1,5 @@ +Remove private ``_PyUnicode_AsString()`` alias to +:c:func:`PyUnicode_AsUTF8`. It was kept for backward compatibility with +Python 3.0 - 3.2. The :c:func:`PyUnicode_AsUTF8` is available since Python +3.3. The :c:func:`PyUnicode_AsUTF8String` function can be used to keep +compatibility with Python 3.2 and older. Patch by Victor Stinner. From webhook-mailer at python.org Sat Jul 22 09:20:44 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 13:20:44 -0000 Subject: [Python-checkins] gh-69714: Make `calendar` module fully tested (#93655) Message-ID: https://github.com/python/cpython/commit/2aba047f0a2be57cc33a57756707452fdd6a1b3f commit: 2aba047f0a2be57cc33a57756707452fdd6a1b3f branch: main author: Bart Skowron committer: ambv date: 2023-07-22T15:20:40+02:00 summary: gh-69714: Make `calendar` module fully tested (#93655) There are 3 paths to use `locale` argument in `calendar.Locale{Text|HTML}Calendar.__init__(..., locale=None)`: (1) `locale=None` -- denotes the "default locale"[1] (2) `locale=""` -- denotes the native environment (3) `locale=other_valid_locale` -- denotes a custom locale So far case (2) is covered and case (1) is in 78935daf5a (same branch). This commit adds a remaining case (3). [1] In the current implementation, this translates into the following approach: GET current locale IF current locale == "C" THEN SET current locale TO "" GET current locale ENDIF * Remove unreachable code (and increase test coverage) This condition cannot be true. `_locale.setlocale()` from the C module raises `locale.Error` instead of returning `None` for `different_locale.__enter__` (where `self.oldlocale` is set). * Expand the try clause to calls to `LocaleTextCalendar.formatmonthname()`. This method temporarily changes the current locale to the given locale, so `_locale.setlocale()` may raise `local.Error`. Co-authored-by: Rohit Mediratta Co-authored-by: Jessica McKellar Co-authored-by: Adam Turner <9087854+AA-Turner at users.noreply.github.com> Co-authored-by: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> files: A Misc/NEWS.d/next/Tests/2022-06-09-21-27-38.gh-issue-69714.49tyHW.rst M Lib/calendar.py M Lib/test/test_calendar.py M Misc/ACKS diff --git a/Lib/calendar.py b/Lib/calendar.py index ea56f12ccc41d..e43ba4a078bca 100644 --- a/Lib/calendar.py +++ b/Lib/calendar.py @@ -585,8 +585,6 @@ def __enter__(self): _locale.setlocale(_locale.LC_TIME, self.locale) def __exit__(self, *args): - if self.oldlocale is None: - return _locale.setlocale(_locale.LC_TIME, self.oldlocale) @@ -690,7 +688,7 @@ def timegm(tuple): return seconds -def main(args): +def main(args=None): import argparse parser = argparse.ArgumentParser() textgroup = parser.add_argument_group('text only arguments') @@ -747,7 +745,7 @@ def main(args): help="month number (1-12, text only)" ) - options = parser.parse_args(args[1:]) + options = parser.parse_args(args) if options.locale and not options.encoding: parser.error("if --locale is specified --encoding is required") @@ -756,6 +754,9 @@ def main(args): locale = options.locale, options.encoding if options.type == "html": + if options.month: + parser.error("incorrect number of arguments") + sys.exit(1) if options.locale: cal = LocaleHTMLCalendar(locale=locale) else: @@ -767,11 +768,8 @@ def main(args): write = sys.stdout.buffer.write if options.year is None: write(cal.formatyearpage(datetime.date.today().year, **optdict)) - elif options.month is None: - write(cal.formatyearpage(options.year, **optdict)) else: - parser.error("incorrect number of arguments") - sys.exit(1) + write(cal.formatyearpage(options.year, **optdict)) else: if options.locale: cal = LocaleTextCalendar(locale=locale) @@ -795,4 +793,4 @@ def main(args): if __name__ == "__main__": - main(sys.argv) + main() diff --git a/Lib/test/test_calendar.py b/Lib/test/test_calendar.py index 0df084e17a3c8..1f9ffc5e9a5c3 100644 --- a/Lib/test/test_calendar.py +++ b/Lib/test/test_calendar.py @@ -3,11 +3,13 @@ from test import support from test.support.script_helper import assert_python_ok, assert_python_failure -import time -import locale -import sys +import contextlib import datetime +import io +import locale import os +import sys +import time # From https://en.wikipedia.org/wiki/Leap_year_starting_on_Saturday result_0_02_text = """\ @@ -549,26 +551,92 @@ def test_months(self): # verify it "acts like a sequence" in two forms of iteration self.assertEqual(value[::-1], list(reversed(value))) - def test_locale_calendars(self): + def test_locale_text_calendar(self): + try: + cal = calendar.LocaleTextCalendar(locale='') + local_weekday = cal.formatweekday(1, 10) + local_weekday_abbr = cal.formatweekday(1, 3) + local_month = cal.formatmonthname(2010, 10, 10) + except locale.Error: + # cannot set the system default locale -- skip rest of test + raise unittest.SkipTest('cannot set the system default locale') + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_weekday_abbr, str) + self.assertIsInstance(local_month, str) + self.assertEqual(len(local_weekday), 10) + self.assertEqual(len(local_weekday_abbr), 3) + self.assertGreaterEqual(len(local_month), 10) + + cal = calendar.LocaleTextCalendar(locale=None) + local_weekday = cal.formatweekday(1, 10) + local_weekday_abbr = cal.formatweekday(1, 3) + local_month = cal.formatmonthname(2010, 10, 10) + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_weekday_abbr, str) + self.assertIsInstance(local_month, str) + self.assertEqual(len(local_weekday), 10) + self.assertEqual(len(local_weekday_abbr), 3) + self.assertGreaterEqual(len(local_month), 10) + + cal = calendar.LocaleTextCalendar(locale='C') + local_weekday = cal.formatweekday(1, 10) + local_weekday_abbr = cal.formatweekday(1, 3) + local_month = cal.formatmonthname(2010, 10, 10) + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_weekday_abbr, str) + self.assertIsInstance(local_month, str) + self.assertEqual(len(local_weekday), 10) + self.assertEqual(len(local_weekday_abbr), 3) + self.assertGreaterEqual(len(local_month), 10) + + def test_locale_html_calendar(self): + try: + cal = calendar.LocaleHTMLCalendar(locale='') + local_weekday = cal.formatweekday(1) + local_month = cal.formatmonthname(2010, 10) + except locale.Error: + # cannot set the system default locale -- skip rest of test + raise unittest.SkipTest('cannot set the system default locale') + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_month, str) + + cal = calendar.LocaleHTMLCalendar(locale=None) + local_weekday = cal.formatweekday(1) + local_month = cal.formatmonthname(2010, 10) + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_month, str) + + cal = calendar.LocaleHTMLCalendar(locale='C') + local_weekday = cal.formatweekday(1) + local_month = cal.formatmonthname(2010, 10) + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_month, str) + + def test_locale_calendars_reset_locale_properly(self): # ensure that Locale{Text,HTML}Calendar resets the locale properly # (it is still not thread-safe though) old_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) try: cal = calendar.LocaleTextCalendar(locale='') local_weekday = cal.formatweekday(1, 10) + local_weekday_abbr = cal.formatweekday(1, 3) local_month = cal.formatmonthname(2010, 10, 10) except locale.Error: # cannot set the system default locale -- skip rest of test raise unittest.SkipTest('cannot set the system default locale') self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_weekday_abbr, str) self.assertIsInstance(local_month, str) self.assertEqual(len(local_weekday), 10) + self.assertEqual(len(local_weekday_abbr), 3) self.assertGreaterEqual(len(local_month), 10) + cal = calendar.LocaleHTMLCalendar(locale='') local_weekday = cal.formatweekday(1) local_month = cal.formatmonthname(2010, 10) self.assertIsInstance(local_weekday, str) self.assertIsInstance(local_month, str) + new_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) self.assertEqual(old_october, new_october) @@ -589,6 +657,21 @@ def test_locale_calendar_formatweekday(self): except locale.Error: raise unittest.SkipTest('cannot set the en_US locale') + def test_locale_calendar_formatmonthname(self): + try: + # formatmonthname uses the same month names regardless of the width argument. + cal = calendar.LocaleTextCalendar(locale='en_US') + # For too short widths, a full name (with year) is used. + self.assertEqual(cal.formatmonthname(2022, 6, 2, withyear=False), "June") + self.assertEqual(cal.formatmonthname(2022, 6, 2, withyear=True), "June 2022") + self.assertEqual(cal.formatmonthname(2022, 6, 3, withyear=False), "June") + self.assertEqual(cal.formatmonthname(2022, 6, 3, withyear=True), "June 2022") + # For long widths, a centered name is used. + self.assertEqual(cal.formatmonthname(2022, 6, 10, withyear=False), " June ") + self.assertEqual(cal.formatmonthname(2022, 6, 15, withyear=True), " June 2022 ") + except locale.Error: + raise unittest.SkipTest('cannot set the en_US locale') + def test_locale_html_calendar_custom_css_class_month_name(self): try: cal = calendar.LocaleHTMLCalendar(locale='') @@ -847,46 +930,104 @@ def conv(s): return s.replace('\n', os.linesep).encode() class CommandLineTestCase(unittest.TestCase): - def run_ok(self, *args): + def setUp(self): + self.runners = [self.run_cli_ok, self.run_cmd_ok] + + @contextlib.contextmanager + def captured_stdout_with_buffer(self): + orig_stdout = sys.stdout + buffer = io.BytesIO() + sys.stdout = io.TextIOWrapper(buffer) + try: + yield sys.stdout + finally: + sys.stdout.flush() + sys.stdout.buffer.seek(0) + sys.stdout = orig_stdout + + @contextlib.contextmanager + def captured_stderr_with_buffer(self): + orig_stderr = sys.stderr + buffer = io.BytesIO() + sys.stderr = io.TextIOWrapper(buffer) + try: + yield sys.stderr + finally: + sys.stderr.flush() + sys.stderr.buffer.seek(0) + sys.stderr = orig_stderr + + def run_cli_ok(self, *args): + with self.captured_stdout_with_buffer() as stdout: + calendar.main(args) + return stdout.buffer.read() + + def run_cmd_ok(self, *args): return assert_python_ok('-m', 'calendar', *args)[1] - def assertFailure(self, *args): + def assertCLIFails(self, *args): + with self.captured_stderr_with_buffer() as stderr: + self.assertRaises(SystemExit, calendar.main, args) + stderr = stderr.buffer.read() + self.assertIn(b'usage:', stderr) + return stderr + + def assertCmdFails(self, *args): rc, stdout, stderr = assert_python_failure('-m', 'calendar', *args) self.assertIn(b'usage:', stderr) self.assertEqual(rc, 2) + return rc, stdout, stderr + + def assertFailure(self, *args): + self.assertCLIFails(*args) + self.assertCmdFails(*args) def test_help(self): - stdout = self.run_ok('-h') + stdout = self.run_cmd_ok('-h') self.assertIn(b'usage:', stdout) self.assertIn(b'calendar.py', stdout) self.assertIn(b'--help', stdout) + # special case: stdout but sys.exit() + with self.captured_stdout_with_buffer() as output: + self.assertRaises(SystemExit, calendar.main, ['-h']) + output = output.buffer.read() + self.assertIn(b'usage:', output) + self.assertIn(b'--help', output) + def test_illegal_arguments(self): self.assertFailure('-z') self.assertFailure('spam') self.assertFailure('2004', 'spam') + self.assertFailure('2004', '1', 'spam') + self.assertFailure('2004', '1', '1') + self.assertFailure('2004', '1', '1', 'spam') self.assertFailure('-t', 'html', '2004', '1') def test_output_current_year(self): - stdout = self.run_ok() - year = datetime.datetime.now().year - self.assertIn((' %s' % year).encode(), stdout) - self.assertIn(b'January', stdout) - self.assertIn(b'Mo Tu We Th Fr Sa Su', stdout) + for run in self.runners: + output = run() + year = datetime.datetime.now().year + self.assertIn(conv(' %s' % year), output) + self.assertIn(b'January', output) + self.assertIn(b'Mo Tu We Th Fr Sa Su', output) def test_output_year(self): - stdout = self.run_ok('2004') - self.assertEqual(stdout, conv(result_2004_text)) + for run in self.runners: + output = run('2004') + self.assertEqual(output, conv(result_2004_text)) def test_output_month(self): - stdout = self.run_ok('2004', '1') - self.assertEqual(stdout, conv(result_2004_01_text)) + for run in self.runners: + output = run('2004', '1') + self.assertEqual(output, conv(result_2004_01_text)) def test_option_encoding(self): self.assertFailure('-e') self.assertFailure('--encoding') - stdout = self.run_ok('--encoding', 'utf-16-le', '2004') - self.assertEqual(stdout, result_2004_text.encode('utf-16-le')) + for run in self.runners: + output = run('--encoding', 'utf-16-le', '2004') + self.assertEqual(output, result_2004_text.encode('utf-16-le')) def test_option_locale(self): self.assertFailure('-L') @@ -904,66 +1045,75 @@ def test_option_locale(self): locale.setlocale(locale.LC_TIME, oldlocale) except (locale.Error, ValueError): self.skipTest('cannot set the system default locale') - stdout = self.run_ok('--locale', lang, '--encoding', enc, '2004') - self.assertIn('2004'.encode(enc), stdout) + for run in self.runners: + for type in ('text', 'html'): + output = run( + '--type', type, '--locale', lang, '--encoding', enc, '2004' + ) + self.assertIn('2004'.encode(enc), output) def test_option_width(self): self.assertFailure('-w') self.assertFailure('--width') self.assertFailure('-w', 'spam') - stdout = self.run_ok('--width', '3', '2004') - self.assertIn(b'Mon Tue Wed Thu Fri Sat Sun', stdout) + for run in self.runners: + output = run('--width', '3', '2004') + self.assertIn(b'Mon Tue Wed Thu Fri Sat Sun', output) def test_option_lines(self): self.assertFailure('-l') self.assertFailure('--lines') self.assertFailure('-l', 'spam') - stdout = self.run_ok('--lines', '2', '2004') - self.assertIn(conv('December\n\nMo Tu We'), stdout) + for run in self.runners: + output = run('--lines', '2', '2004') + self.assertIn(conv('December\n\nMo Tu We'), output) def test_option_spacing(self): self.assertFailure('-s') self.assertFailure('--spacing') self.assertFailure('-s', 'spam') - stdout = self.run_ok('--spacing', '8', '2004') - self.assertIn(b'Su Mo', stdout) + for run in self.runners: + output = run('--spacing', '8', '2004') + self.assertIn(b'Su Mo', output) def test_option_months(self): self.assertFailure('-m') self.assertFailure('--month') self.assertFailure('-m', 'spam') - stdout = self.run_ok('--months', '1', '2004') - self.assertIn(conv('\nMo Tu We Th Fr Sa Su\n'), stdout) + for run in self.runners: + output = run('--months', '1', '2004') + self.assertIn(conv('\nMo Tu We Th Fr Sa Su\n'), output) def test_option_type(self): self.assertFailure('-t') self.assertFailure('--type') self.assertFailure('-t', 'spam') - stdout = self.run_ok('--type', 'text', '2004') - self.assertEqual(stdout, conv(result_2004_text)) - stdout = self.run_ok('--type', 'html', '2004') - self.assertEqual(stdout[:6], b'Calendar for 2004', stdout) + for run in self.runners: + output = run('--type', 'text', '2004') + self.assertEqual(output, conv(result_2004_text)) + output = run('--type', 'html', '2004') + self.assertEqual(output[:6], b'Calendar for 2004', output) def test_html_output_current_year(self): - stdout = self.run_ok('--type', 'html') - year = datetime.datetime.now().year - self.assertIn(('Calendar for %s' % year).encode(), - stdout) - self.assertIn(b'January', - stdout) + for run in self.runners: + output = run('--type', 'html') + year = datetime.datetime.now().year + self.assertIn(('Calendar for %s' % year).encode(), output) + self.assertIn(b'January', output) def test_html_output_year_encoding(self): - stdout = self.run_ok('-t', 'html', '--encoding', 'ascii', '2004') - self.assertEqual(stdout, - result_2004_html.format(**default_format).encode('ascii')) + for run in self.runners: + output = run('-t', 'html', '--encoding', 'ascii', '2004') + self.assertEqual(output, result_2004_html.format(**default_format).encode('ascii')) def test_html_output_year_css(self): self.assertFailure('-t', 'html', '-c') self.assertFailure('-t', 'html', '--css') - stdout = self.run_ok('-t', 'html', '--css', 'custom.css', '2004') - self.assertIn(b'', stdout) + for run in self.runners: + output = run('-t', 'html', '--css', 'custom.css', '2004') + self.assertIn(b'', output) class MiscTestCase(unittest.TestCase): diff --git a/Misc/ACKS b/Misc/ACKS index 645ad5b700baa..fadf488888aa8 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1700,6 +1700,7 @@ Ngalim Siregar Kragen Sitaker Kaartic Sivaraam Stanis?aw Skonieczny +Bart Skowron Roman Skurikhin Ville Skytt? Michael Sloan diff --git a/Misc/NEWS.d/next/Tests/2022-06-09-21-27-38.gh-issue-69714.49tyHW.rst b/Misc/NEWS.d/next/Tests/2022-06-09-21-27-38.gh-issue-69714.49tyHW.rst new file mode 100644 index 0000000000000..e28b94a171c40 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-06-09-21-27-38.gh-issue-69714.49tyHW.rst @@ -0,0 +1 @@ +Add additional tests to :mod:`calendar` to achieve full test coverage. From webhook-mailer at python.org Sat Jul 22 09:30:16 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 13:30:16 -0000 Subject: [Python-checkins] gh-106320: Document private C API removal in What's New 3.13 (#107027) Message-ID: https://github.com/python/cpython/commit/756add081ead8fa9aeb11ac178e581ba2450f296 commit: 756add081ead8fa9aeb11ac178e581ba2450f296 branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T15:30:13+02:00 summary: gh-106320: Document private C API removal in What's New 3.13 (#107027) files: M Doc/whatsnew/3.13.rst diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index a778e12aba1bc..85f0ca1ae1a67 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -838,6 +838,13 @@ Deprecated Removed ------- +* Remove many APIs (functions, macros, variables) with names prefixed by + ``_Py`` or ``_PY`` (considered as private API). If your project is affected + by one of these removals and you consider that the removed API should remain + available, please open a new issue to request a public C API and + add ``cc @vstinner`` to the issue to notify Victor Stinner. + (Contributed by Victor Stinner in :gh:`106320`.) + * Remove functions deprecated in Python 3.9. * ``PyEval_CallObject()``, ``PyEval_CallObjectWithKeywords()``: use From webhook-mailer at python.org Sat Jul 22 09:41:42 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 22 Jul 2023 13:41:42 -0000 Subject: [Python-checkins] [3.12] Bump sphinx-lint to 0.6.8 (gh-106978) (#107023) Message-ID: https://github.com/python/cpython/commit/beb5e4fba7fa8118e47523999e1a129382984614 commit: beb5e4fba7fa8118e47523999e1a129382984614 branch: 3.12 author: Sven Arends <88426829+picnic-sven at users.noreply.github.com> committer: hugovk date: 2023-07-22T13:41:38Z summary: [3.12] Bump sphinx-lint to 0.6.8 (gh-106978) (#107023) files: M .pre-commit-config.yaml diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index d62c57c044728..85a6de4abe014 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -10,7 +10,7 @@ repos: types_or: [c, python, rst] - repo: https://github.com/sphinx-contrib/sphinx-lint - rev: v0.6.7 + rev: v0.6.8 hooks: - id: sphinx-lint args: [--enable=default-role] From webhook-mailer at python.org Sat Jul 22 09:49:41 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 13:49:41 -0000 Subject: [Python-checkins] gh-106320: Move private _PyHash API to the internal C API (#107026) Message-ID: https://github.com/python/cpython/commit/89f987544860d1360e6232b6f52b8ced92a4d690 commit: 89f987544860d1360e6232b6f52b8ced92a4d690 branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T13:49:37Z summary: gh-106320: Move private _PyHash API to the internal C API (#107026) * No longer export most private _PyHash symbols, only export the ones which are needed by shared extensions. * Modules/_xxtestfuzz/fuzzer.c now uses the internal C API. files: M Include/internal/pycore_pyhash.h M Include/pyhash.h M Modules/_elementtree.c M Modules/_hashopenssl.c M Modules/_xxtestfuzz/fuzzer.c M Modules/pyexpat.c M Python/hashtable.c M Python/pyhash.c diff --git a/Include/internal/pycore_pyhash.h b/Include/internal/pycore_pyhash.h index 34dfa53771288..59e416ca18a72 100644 --- a/Include/internal/pycore_pyhash.h +++ b/Include/internal/pycore_pyhash.h @@ -1,10 +1,86 @@ -#ifndef Py_INTERNAL_HASH_H -#define Py_INTERNAL_HASH_H +#ifndef Py_INTERNAL_PYHASH_H +#define Py_INTERNAL_PYHASH_H #ifndef Py_BUILD_CORE # error "this header requires Py_BUILD_CORE define" #endif +/* Helpers for hash functions */ +extern Py_hash_t _Py_HashDouble(PyObject *, double); +// _decimal shared extensions uses _Py_HashPointer() +PyAPI_FUNC(Py_hash_t) _Py_HashPointer(const void*); +// Similar to _Py_HashPointer(), but don't replace -1 with -2 +extern Py_hash_t _Py_HashPointerRaw(const void*); +// _datetime shared extension uses _Py_HashBytes() +PyAPI_FUNC(Py_hash_t) _Py_HashBytes(const void*, Py_ssize_t); + +/* Prime multiplier used in string and various other hashes. */ +#define _PyHASH_MULTIPLIER 1000003UL /* 0xf4243 */ + +/* Parameters used for the numeric hash implementation. See notes for + _Py_HashDouble in Python/pyhash.c. Numeric hashes are based on + reduction modulo the prime 2**_PyHASH_BITS - 1. */ + +#if SIZEOF_VOID_P >= 8 +# define _PyHASH_BITS 61 +#else +# define _PyHASH_BITS 31 +#endif + +#define _PyHASH_MODULUS (((size_t)1 << _PyHASH_BITS) - 1) +#define _PyHASH_INF 314159 +#define _PyHASH_IMAG _PyHASH_MULTIPLIER + +/* Hash secret + * + * memory layout on 64 bit systems + * cccccccc cccccccc cccccccc uc -- unsigned char[24] + * pppppppp ssssssss ........ fnv -- two Py_hash_t + * k0k0k0k0 k1k1k1k1 ........ siphash -- two uint64_t + * ........ ........ ssssssss djbx33a -- 16 bytes padding + one Py_hash_t + * ........ ........ eeeeeeee pyexpat XML hash salt + * + * memory layout on 32 bit systems + * cccccccc cccccccc cccccccc uc + * ppppssss ........ ........ fnv -- two Py_hash_t + * k0k0k0k0 k1k1k1k1 ........ siphash -- two uint64_t (*) + * ........ ........ ssss.... djbx33a -- 16 bytes padding + one Py_hash_t + * ........ ........ eeee.... pyexpat XML hash salt + * + * (*) The siphash member may not be available on 32 bit platforms without + * an unsigned int64 data type. + */ +typedef union { + /* ensure 24 bytes */ + unsigned char uc[24]; + /* two Py_hash_t for FNV */ + struct { + Py_hash_t prefix; + Py_hash_t suffix; + } fnv; + /* two uint64 for SipHash24 */ + struct { + uint64_t k0; + uint64_t k1; + } siphash; + /* a different (!) Py_hash_t for small string optimization */ + struct { + unsigned char padding[16]; + Py_hash_t suffix; + } djbx33a; + struct { + unsigned char padding[16]; + Py_hash_t hashsalt; + } expat; +} _Py_HashSecret_t; + +// _elementtree shared extension uses _Py_HashSecret.expat +PyAPI_DATA(_Py_HashSecret_t) _Py_HashSecret; + +#ifdef Py_DEBUG +extern int _Py_HashSecret_Initialized; +#endif + struct pyhash_runtime_state { struct { @@ -34,7 +110,6 @@ struct pyhash_runtime_state { } -uint64_t _Py_KeyedHash(uint64_t, const char *, Py_ssize_t); - +extern uint64_t _Py_KeyedHash(uint64_t key, const void *src, Py_ssize_t src_sz); -#endif // Py_INTERNAL_HASH_H +#endif // !Py_INTERNAL_PYHASH_H diff --git a/Include/pyhash.h b/Include/pyhash.h index 182d223fab1ca..6e969f86fa262 100644 --- a/Include/pyhash.h +++ b/Include/pyhash.h @@ -1,87 +1,10 @@ #ifndef Py_HASH_H - #define Py_HASH_H #ifdef __cplusplus extern "C" { #endif -/* Helpers for hash functions */ #ifndef Py_LIMITED_API -PyAPI_FUNC(Py_hash_t) _Py_HashDouble(PyObject *, double); -PyAPI_FUNC(Py_hash_t) _Py_HashPointer(const void*); -// Similar to _Py_HashPointer(), but don't replace -1 with -2 -PyAPI_FUNC(Py_hash_t) _Py_HashPointerRaw(const void*); -PyAPI_FUNC(Py_hash_t) _Py_HashBytes(const void*, Py_ssize_t); -#endif - -/* Prime multiplier used in string and various other hashes. */ -#define _PyHASH_MULTIPLIER 1000003UL /* 0xf4243 */ - -/* Parameters used for the numeric hash implementation. See notes for - _Py_HashDouble in Python/pyhash.c. Numeric hashes are based on - reduction modulo the prime 2**_PyHASH_BITS - 1. */ - -#if SIZEOF_VOID_P >= 8 -# define _PyHASH_BITS 61 -#else -# define _PyHASH_BITS 31 -#endif - -#define _PyHASH_MODULUS (((size_t)1 << _PyHASH_BITS) - 1) -#define _PyHASH_INF 314159 -#define _PyHASH_IMAG _PyHASH_MULTIPLIER - - -/* hash secret - * - * memory layout on 64 bit systems - * cccccccc cccccccc cccccccc uc -- unsigned char[24] - * pppppppp ssssssss ........ fnv -- two Py_hash_t - * k0k0k0k0 k1k1k1k1 ........ siphash -- two uint64_t - * ........ ........ ssssssss djbx33a -- 16 bytes padding + one Py_hash_t - * ........ ........ eeeeeeee pyexpat XML hash salt - * - * memory layout on 32 bit systems - * cccccccc cccccccc cccccccc uc - * ppppssss ........ ........ fnv -- two Py_hash_t - * k0k0k0k0 k1k1k1k1 ........ siphash -- two uint64_t (*) - * ........ ........ ssss.... djbx33a -- 16 bytes padding + one Py_hash_t - * ........ ........ eeee.... pyexpat XML hash salt - * - * (*) The siphash member may not be available on 32 bit platforms without - * an unsigned int64 data type. - */ -#ifndef Py_LIMITED_API -typedef union { - /* ensure 24 bytes */ - unsigned char uc[24]; - /* two Py_hash_t for FNV */ - struct { - Py_hash_t prefix; - Py_hash_t suffix; - } fnv; - /* two uint64 for SipHash24 */ - struct { - uint64_t k0; - uint64_t k1; - } siphash; - /* a different (!) Py_hash_t for small string optimization */ - struct { - unsigned char padding[16]; - Py_hash_t suffix; - } djbx33a; - struct { - unsigned char padding[16]; - Py_hash_t hashsalt; - } expat; -} _Py_HashSecret_t; -PyAPI_DATA(_Py_HashSecret_t) _Py_HashSecret; - -#ifdef Py_DEBUG -PyAPI_DATA(int) _Py_HashSecret_Initialized; -#endif - - /* hash function definition */ typedef struct { Py_hash_t (*const hash)(const void *, Py_ssize_t); @@ -94,7 +17,7 @@ PyAPI_FUNC(PyHash_FuncDef*) PyHash_GetFuncDef(void); #endif -/* cutoff for small string DJBX33A optimization in range [1, cutoff). +/* Cutoff for small string DJBX33A optimization in range [1, cutoff). * * About 50% of the strings in a typical Python application are smaller than * 6 to 7 chars. However DJBX33A is vulnerable to hash collision attacks. @@ -112,7 +35,7 @@ PyAPI_FUNC(PyHash_FuncDef*) PyHash_GetFuncDef(void); #endif /* Py_HASH_CUTOFF */ -/* hash algorithm selection +/* Hash algorithm selection * * The values for Py_HASH_* are hard-coded in the * configure script. @@ -140,5 +63,4 @@ PyAPI_FUNC(PyHash_FuncDef*) PyHash_GetFuncDef(void); #ifdef __cplusplus } #endif - -#endif /* !Py_HASH_H */ +#endif // !Py_HASH_H diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index a8d68d68420d3..30327e0b6fc31 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -17,6 +17,7 @@ #include "Python.h" #include "pycore_import.h" // _PyImport_GetModuleAttrString() +#include "pycore_pyhash.h" // _Py_HashSecret #include "structmember.h" // PyMemberDef #include "expat.h" #include "pyexpat.h" diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c index 246eea7409882..6d60037cd2054 100644 --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -24,8 +24,9 @@ #include "Python.h" #include "pycore_hashtable.h" -#include "hashlib.h" +#include "pycore_pyhash.h" // _Py_HashBytes() #include "pycore_strhex.h" // _Py_strhex() +#include "hashlib.h" /* EVP is the preferred interface to hashing in OpenSSL */ #include diff --git a/Modules/_xxtestfuzz/fuzzer.c b/Modules/_xxtestfuzz/fuzzer.c index 37d402824853f..54f8a42273401 100644 --- a/Modules/_xxtestfuzz/fuzzer.c +++ b/Modules/_xxtestfuzz/fuzzer.c @@ -10,7 +10,12 @@ See the source code for LLVMFuzzerTestOneInput for details. */ +#ifndef Py_BUILD_CORE +# define Py_BUILD_CORE 1 +#endif + #include +#include "pycore_pyhash.h" // _Py_HashBytes() #include #include diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index bd8a98a46579a..33a56c57a876e 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -4,6 +4,7 @@ #include "Python.h" #include "pycore_import.h" // _PyImport_SetModule() +#include "pycore_pyhash.h" // _Py_HashSecret #include #include "structmember.h" // PyMemberDef diff --git a/Python/hashtable.c b/Python/hashtable.c index 09501de199b0e..0f07cd20b1a95 100644 --- a/Python/hashtable.c +++ b/Python/hashtable.c @@ -46,6 +46,7 @@ #include "Python.h" #include "pycore_hashtable.h" +#include "pycore_pyhash.h" // _Py_HashPointerRaw() #define HASHTABLE_MIN_SIZE 16 #define HASHTABLE_HIGH 0.50 diff --git a/Python/pyhash.c b/Python/pyhash.c index d5ac9f83be61c..b2bdab5099d86 100644 --- a/Python/pyhash.c +++ b/Python/pyhash.c @@ -4,6 +4,7 @@ All the utility functions (_Py_Hash*()) return "-1" to signify an error. */ #include "Python.h" +#include "pycore_pyhash.h" // _Py_HashSecret_t #ifdef __APPLE__ # include From webhook-mailer at python.org Sat Jul 22 10:12:20 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 14:12:20 -0000 Subject: [Python-checkins] gh-106320: Move _PyNone_Type to the internal C API (#107030) Message-ID: https://github.com/python/cpython/commit/eda9ce148714947c164ff72b720e2645ca2cd5fb commit: eda9ce148714947c164ff72b720e2645ca2cd5fb branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T14:12:17Z summary: gh-106320: Move _PyNone_Type to the internal C API (#107030) Move private types _PyNone_Type and _PyNotImplemented_Type to internal C API. files: M Include/cpython/object.h M Include/internal/pycore_object.h M Modules/_pickle.c diff --git a/Include/cpython/object.h b/Include/cpython/object.h index cd421b4f7e0d4..0e5b6acb43b41 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -377,9 +377,6 @@ PyAPI_FUNC(PyObject *) _PyObject_FunctionStr(PyObject *); #endif -PyAPI_DATA(PyTypeObject) _PyNone_Type; -PyAPI_DATA(PyTypeObject) _PyNotImplemented_Type; - /* Maps Py_LT to Py_GT, ..., Py_GE to Py_LE. * Defined in object.c. */ diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h index 2358f48738a90..90588daa64cc3 100644 --- a/Include/internal/pycore_object.h +++ b/Include/internal/pycore_object.h @@ -438,6 +438,10 @@ extern PyObject* _PyCFunctionWithKeywords_TrampolineCall( (meth)((self), (args), (kw)) #endif // __EMSCRIPTEN__ && PY_CALL_TRAMPOLINE +// _pickle shared extension uses _PyNone_Type and _PyNotImplemented_Type +PyAPI_DATA(PyTypeObject) _PyNone_Type; +PyAPI_DATA(PyTypeObject) _PyNotImplemented_Type; + #ifdef __cplusplus } #endif diff --git a/Modules/_pickle.c b/Modules/_pickle.c index ea44b494cdd7c..f2e98c10ef718 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -12,6 +12,7 @@ #include "pycore_bytesobject.h" // _PyBytesWriter #include "pycore_ceval.h" // _Py_EnterRecursiveCall() #include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_object.h" // _PyNone_Type #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_runtime.h" // _Py_ID() #include "structmember.h" // PyMemberDef From webhook-mailer at python.org Sat Jul 22 10:14:28 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 14:14:28 -0000 Subject: [Python-checkins] [3.11] Reformat code block to make it easier to read (GH-106965) (#107022) Message-ID: https://github.com/python/cpython/commit/c65ca7097aeea267a57c516707b6d1095870c6f8 commit: c65ca7097aeea267a57c516707b6d1095870c6f8 branch: 3.11 author: ?ukasz Langa committer: ambv date: 2023-07-22T16:14:24+02:00 summary: [3.11] Reformat code block to make it easier to read (GH-106965) (#107022) (cherry picked from commit ed491d9f782480fb00535abcf667027e0e323287) Co-authored-by: Joe Kaufeld files: M Doc/tutorial/errors.rst diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index 6419ff621f1b3..8a207c385c6ab 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -535,11 +535,20 @@ of a certain type while letting all other exceptions propagate to other clauses and eventually to be reraised. :: >>> def f(): - ... raise ExceptionGroup("group1", - ... [OSError(1), - ... SystemError(2), - ... ExceptionGroup("group2", - ... [OSError(3), RecursionError(4)])]) + ... raise ExceptionGroup( + ... "group1", + ... [ + ... OSError(1), + ... SystemError(2), + ... ExceptionGroup( + ... "group2", + ... [ + ... OSError(3), + ... RecursionError(4) + ... ] + ... ) + ... ] + ... ) ... >>> try: ... f() From webhook-mailer at python.org Sat Jul 22 10:21:20 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 14:21:20 -0000 Subject: [Python-checkins] gh-106320: Move private _PyGen API to the internal C API (#107032) Message-ID: https://github.com/python/cpython/commit/ae8b114c5b9d211f47bf521fcfdbf40ef72a1bac commit: ae8b114c5b9d211f47bf521fcfdbf40ef72a1bac branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T14:21:16Z summary: gh-106320: Move private _PyGen API to the internal C API (#107032) Move private _PyGen API to internal C API: * _PyAsyncGenAThrow_Type * _PyAsyncGenWrappedValue_Type * _PyCoroWrapper_Type * _PyGen_FetchStopIterationValue() * _PyGen_Finalize() * _PyGen_SetStopIterationValue() No longer these symbols, except of the ones by the _asyncio shared extensions. files: M Include/cpython/genobject.h M Include/internal/pycore_genobject.h diff --git a/Include/cpython/genobject.h b/Include/cpython/genobject.h index 7856481b5db30..49e46c277d75a 100644 --- a/Include/cpython/genobject.h +++ b/Include/cpython/genobject.h @@ -41,9 +41,6 @@ PyAPI_DATA(PyTypeObject) PyGen_Type; PyAPI_FUNC(PyObject *) PyGen_New(PyFrameObject *); PyAPI_FUNC(PyObject *) PyGen_NewWithQualName(PyFrameObject *, PyObject *name, PyObject *qualname); -PyAPI_FUNC(int) _PyGen_SetStopIterationValue(PyObject *); -PyAPI_FUNC(int) _PyGen_FetchStopIterationValue(PyObject **); -PyAPI_FUNC(void) _PyGen_Finalize(PyObject *self); PyAPI_FUNC(PyCodeObject *) PyGen_GetCode(PyGenObject *gen); @@ -54,7 +51,6 @@ typedef struct { } PyCoroObject; PyAPI_DATA(PyTypeObject) PyCoro_Type; -PyAPI_DATA(PyTypeObject) _PyCoroWrapper_Type; #define PyCoro_CheckExact(op) Py_IS_TYPE((op), &PyCoro_Type) PyAPI_FUNC(PyObject *) PyCoro_New(PyFrameObject *, @@ -69,8 +65,6 @@ typedef struct { PyAPI_DATA(PyTypeObject) PyAsyncGen_Type; PyAPI_DATA(PyTypeObject) _PyAsyncGenASend_Type; -PyAPI_DATA(PyTypeObject) _PyAsyncGenWrappedValue_Type; -PyAPI_DATA(PyTypeObject) _PyAsyncGenAThrow_Type; PyAPI_FUNC(PyObject *) PyAsyncGen_New(PyFrameObject *, PyObject *name, PyObject *qualname); diff --git a/Include/internal/pycore_genobject.h b/Include/internal/pycore_genobject.h index dc60b4ca70511..fb63d9c3537a6 100644 --- a/Include/internal/pycore_genobject.h +++ b/Include/internal/pycore_genobject.h @@ -9,9 +9,20 @@ extern "C" { #endif extern PyObject *_PyGen_yf(PyGenObject *); +extern void _PyGen_Finalize(PyObject *self); + +// _asyncio shared extensions uses _PyGen_SetStopIterationValue() and +// _PyGen_FetchStopIterationValue() +PyAPI_FUNC(int) _PyGen_SetStopIterationValue(PyObject *); +PyAPI_FUNC(int) _PyGen_FetchStopIterationValue(PyObject **); + extern PyObject *_PyCoro_GetAwaitableIter(PyObject *o); extern PyObject *_PyAsyncGenValueWrapperNew(PyThreadState *state, PyObject *); +extern PyTypeObject _PyCoroWrapper_Type; +extern PyTypeObject _PyAsyncGenWrappedValue_Type; +extern PyTypeObject _PyAsyncGenAThrow_Type; + /* runtime lifecycle */ extern void _PyAsyncGen_Fini(PyInterpreterState *); From webhook-mailer at python.org Sat Jul 22 10:29:13 2023 From: webhook-mailer at python.org (pablogsal) Date: Sat, 22 Jul 2023 14:29:13 -0000 Subject: [Python-checkins] gh-105090: Replace incorrect TLSv1.2 with TLSv1.3 (#105404) Message-ID: https://github.com/python/cpython/commit/e5252c6127ad788b39221982964f935bd1513028 commit: e5252c6127ad788b39221982964f935bd1513028 branch: main author: Jocelyn Castellano committer: pablogsal date: 2023-07-22T14:29:08Z summary: gh-105090: Replace incorrect TLSv1.2 with TLSv1.3 (#105404) files: M Doc/library/ssl.rst diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 7f09552632768..1f8bbe1e3de3e 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -2592,7 +2592,7 @@ disabled by default. >>> client_context.maximum_version = ssl.TLSVersion.TLSv1_3 -The SSL context created above will only allow TLSv1.2 and later (if +The SSL context created above will only allow TLSv1.3 and later (if supported by your system) connections to a server. :const:`PROTOCOL_TLS_CLIENT` implies certificate validation and hostname checks by default. You have to load certificates into the context. From webhook-mailer at python.org Sat Jul 22 10:29:57 2023 From: webhook-mailer at python.org (pablogsal) Date: Sat, 22 Jul 2023 14:29:57 -0000 Subject: [Python-checkins] [3.11] gh-105090: Replace incorrect TLSv1.2 with TLSv1.3 (GH-105404) (#107038) Message-ID: https://github.com/python/cpython/commit/755166e864e7ff451c2a30b7cdc3a68cd4e9ec3d commit: 755166e864e7ff451c2a30b7cdc3a68cd4e9ec3d branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: pablogsal date: 2023-07-22T14:29:54Z summary: [3.11] gh-105090: Replace incorrect TLSv1.2 with TLSv1.3 (GH-105404) (#107038) Co-authored-by: Jocelyn Castellano files: M Doc/library/ssl.rst diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index a0936747db5a0..fe896be2faeb5 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -2688,7 +2688,7 @@ disabled by default. >>> client_context.maximum_version = ssl.TLSVersion.TLSv1_3 -The SSL context created above will only allow TLSv1.2 and later (if +The SSL context created above will only allow TLSv1.3 and later (if supported by your system) connections to a server. :const:`PROTOCOL_TLS_CLIENT` implies certificate validation and hostname checks by default. You have to load certificates into the context. From webhook-mailer at python.org Sat Jul 22 10:30:12 2023 From: webhook-mailer at python.org (pablogsal) Date: Sat, 22 Jul 2023 14:30:12 -0000 Subject: [Python-checkins] [3.10] gh-105090: Replace incorrect TLSv1.2 with TLSv1.3 (GH-105404) (#107039) Message-ID: https://github.com/python/cpython/commit/a9e5e59b7d912b4a90c550fad6a2d208bf8a1ed9 commit: a9e5e59b7d912b4a90c550fad6a2d208bf8a1ed9 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: pablogsal date: 2023-07-22T16:30:09+02:00 summary: [3.10] gh-105090: Replace incorrect TLSv1.2 with TLSv1.3 (GH-105404) (#107039) Co-authored-by: Jocelyn Castellano files: M Doc/library/ssl.rst diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 36c852974da91..2eefefc32eb0d 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -2690,7 +2690,7 @@ disabled by default. >>> client_context.maximum_version = ssl.TLSVersion.TLSv1_3 -The SSL context created above will only allow TLSv1.2 and later (if +The SSL context created above will only allow TLSv1.3 and later (if supported by your system) connections to a server. :const:`PROTOCOL_TLS_CLIENT` implies certificate validation and hostname checks by default. You have to load certificates into the context. From webhook-mailer at python.org Sat Jul 22 10:30:32 2023 From: webhook-mailer at python.org (pablogsal) Date: Sat, 22 Jul 2023 14:30:32 -0000 Subject: [Python-checkins] [3.12] gh-105090: Replace incorrect TLSv1.2 with TLSv1.3 (GH-105404) (#107040) Message-ID: https://github.com/python/cpython/commit/c8a0296901f0266fad269306b56c66f3a9d5479d commit: c8a0296901f0266fad269306b56c66f3a9d5479d branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: pablogsal date: 2023-07-22T14:30:28Z summary: [3.12] gh-105090: Replace incorrect TLSv1.2 with TLSv1.3 (GH-105404) (#107040) Co-authored-by: Jocelyn Castellano files: M Doc/library/ssl.rst diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 7f09552632768..1f8bbe1e3de3e 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -2592,7 +2592,7 @@ disabled by default. >>> client_context.maximum_version = ssl.TLSVersion.TLSv1_3 -The SSL context created above will only allow TLSv1.2 and later (if +The SSL context created above will only allow TLSv1.3 and later (if supported by your system) connections to a server. :const:`PROTOCOL_TLS_CLIENT` implies certificate validation and hostname checks by default. You have to load certificates into the context. From webhook-mailer at python.org Sat Jul 22 10:45:59 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 14:45:59 -0000 Subject: [Python-checkins] gh-106320: Remove _PyOS_ReadlineTState API (#107034) Message-ID: https://github.com/python/cpython/commit/d228825e08883fc13f35eb91435f95d32524931c commit: d228825e08883fc13f35eb91435f95d32524931c branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T14:45:56Z summary: gh-106320: Remove _PyOS_ReadlineTState API (#107034) Remove _PyOS_ReadlineTState variable from the public C API. The symbol is still exported for the readline shared extension. files: M Include/cpython/pythonrun.h M Modules/readline.c M Parser/myreadline.c diff --git a/Include/cpython/pythonrun.h b/Include/cpython/pythonrun.h index fb61765537402..3b2537e01b83b 100644 --- a/Include/cpython/pythonrun.h +++ b/Include/cpython/pythonrun.h @@ -117,5 +117,4 @@ PyAPI_FUNC(PyObject *) PyRun_FileFlags(FILE *fp, const char *p, int s, PyObject /* Stuff with no proper home (yet) */ PyAPI_FUNC(char *) PyOS_Readline(FILE *, FILE *, const char *); -PyAPI_DATA(PyThreadState*) _PyOS_ReadlineTState; PyAPI_DATA(char) *(*PyOS_ReadlineFunctionPointer)(FILE *, FILE *, const char *); diff --git a/Modules/readline.c b/Modules/readline.c index a592919692cb8..6729a09cb0da5 100644 --- a/Modules/readline.c +++ b/Modules/readline.c @@ -1313,6 +1313,9 @@ rlhandler(char *text) static char * readline_until_enter_or_signal(const char *prompt, int *signal) { + // Defined in Parser/myreadline.c + extern PyThreadState *_PyOS_ReadlineTState; + char * not_done_reading = ""; fd_set selectset; diff --git a/Parser/myreadline.c b/Parser/myreadline.c index 7074aba74b728..815387388218c 100644 --- a/Parser/myreadline.c +++ b/Parser/myreadline.c @@ -20,7 +20,9 @@ #endif /* MS_WINDOWS */ -PyThreadState* _PyOS_ReadlineTState = NULL; +// Export the symbol since it's used by the readline shared extension +PyAPI_DATA(PyThreadState*) _PyOS_ReadlineTState; +PyThreadState *_PyOS_ReadlineTState = NULL; static PyThread_type_lock _PyOS_ReadlineLock = NULL; From webhook-mailer at python.org Sat Jul 22 10:47:03 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 14:47:03 -0000 Subject: [Python-checkins] gh-65495: Use lowercase `mail from` and `rcpt to` in `smtplib.SMTP` (#107019) Message-ID: https://github.com/python/cpython/commit/3782def5a2b4d24ab5d356f89da181e99a9a59b2 commit: 3782def5a2b4d24ab5d356f89da181e99a9a59b2 branch: main author: Matthieu Caneill committer: ambv date: 2023-07-22T16:46:59+02:00 summary: gh-65495: Use lowercase `mail from` and `rcpt to` in `smtplib.SMTP` (#107019) Use lowercase `mail from` and `rcpt to` in `smtplib.SMTP` SMTP commands are case-insensitive. `smtplib` uses lowercase commands, however it writes `mail FROM` and `rcpt TO`, lacking consistency. files: A Misc/NEWS.d/next/Library/2023-07-22-14-29-34.gh-issue-65495.fw84qM.rst M Lib/smtplib.py M Lib/test/test_smtplib.py diff --git a/Lib/smtplib.py b/Lib/smtplib.py index 18c91746fd7bf..b3cc68a789a7d 100755 --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -542,7 +542,7 @@ def mail(self, sender, options=()): raise SMTPNotSupportedError( 'SMTPUTF8 not supported by server') optionlist = ' ' + ' '.join(options) - self.putcmd("mail", "FROM:%s%s" % (quoteaddr(sender), optionlist)) + self.putcmd("mail", "from:%s%s" % (quoteaddr(sender), optionlist)) return self.getreply() def rcpt(self, recip, options=()): @@ -550,7 +550,7 @@ def rcpt(self, recip, options=()): optionlist = '' if options and self.does_esmtp: optionlist = ' ' + ' '.join(options) - self.putcmd("rcpt", "TO:%s%s" % (quoteaddr(recip), optionlist)) + self.putcmd("rcpt", "to:%s%s" % (quoteaddr(recip), optionlist)) return self.getreply() def data(self, msg): diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py index b6d5b8c3d8258..f2e02dab1c3ca 100644 --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -831,6 +831,7 @@ class SimSMTPChannel(smtpd.SMTPChannel): def __init__(self, extra_features, *args, **kw): self._extrafeatures = ''.join( [ "250-{0}\r\n".format(x) for x in extra_features ]) + self.all_received_lines = [] super(SimSMTPChannel, self).__init__(*args, **kw) # AUTH related stuff. It would be nice if support for this were in smtpd. @@ -845,6 +846,7 @@ def found_terminator(self): self.smtp_state = self.COMMAND self.push('%s %s' % (e.smtp_code, e.smtp_error)) return + self.all_received_lines.append(self.received_lines) super().found_terminator() @@ -1349,6 +1351,18 @@ def test_name_field_not_included_in_envelop_addresses(self): self.assertEqual(self.serv._addresses['from'], 'michael at example.com') self.assertEqual(self.serv._addresses['tos'], ['rene at example.com']) + def test_lowercase_mail_from_rcpt_to(self): + m = 'A test message' + smtp = smtplib.SMTP( + HOST, self.port, local_hostname='localhost', + timeout=support.LOOPBACK_TIMEOUT) + self.addCleanup(smtp.close) + + smtp.sendmail('John', 'Sally', m) + + self.assertIn(['mail from: size=14'], self.serv._SMTPchannel.all_received_lines) + self.assertIn(['rcpt to:'], self.serv._SMTPchannel.all_received_lines) + class SimSMTPUTF8Server(SimSMTPServer): diff --git a/Misc/NEWS.d/next/Library/2023-07-22-14-29-34.gh-issue-65495.fw84qM.rst b/Misc/NEWS.d/next/Library/2023-07-22-14-29-34.gh-issue-65495.fw84qM.rst new file mode 100644 index 0000000000000..e75b6c0f1d675 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-14-29-34.gh-issue-65495.fw84qM.rst @@ -0,0 +1 @@ +Use lowercase ``mail from`` and ``rcpt to`` in :class:`smptlib.SMTP`. From webhook-mailer at python.org Sat Jul 22 10:57:55 2023 From: webhook-mailer at python.org (hugovk) Date: Sat, 22 Jul 2023 14:57:55 -0000 Subject: [Python-checkins] gh-106967: remove Release and Date fields from whatsnew for 3.12 and 3.13 (#107000) Message-ID: https://github.com/python/cpython/commit/c92ef6fe0e1384c090b94143cdc01e5e114a8747 commit: c92ef6fe0e1384c090b94143cdc01e5e114a8747 branch: main author: Oliver Rew <12968411+oliver-rew at users.noreply.github.com> committer: hugovk date: 2023-07-22T14:57:51Z summary: gh-106967: remove Release and Date fields from whatsnew for 3.12 and 3.13 (#107000) files: M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.13.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index a46621071b3d8..ac03af52db071 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -3,8 +3,7 @@ What's New In Python 3.12 **************************** -:Release: |release| -:Date: |today| +:Editor: TBD .. Rules for maintenance: diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 85f0ca1ae1a67..5d4fa44aab695 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -3,8 +3,7 @@ What's New In Python 3.13 **************************** -:Release: |release| -:Date: |today| +:Editor: TBD .. Rules for maintenance: From webhook-mailer at python.org Sat Jul 22 11:04:37 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 15:04:37 -0000 Subject: [Python-checkins] gh-106320: Move private _PySet API to the internal API (#107041) Message-ID: https://github.com/python/cpython/commit/5e4af2a3e90d78fdf6a4cc42e0472576b1f2b975 commit: 5e4af2a3e90d78fdf6a4cc42e0472576b1f2b975 branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T17:04:34+02:00 summary: gh-106320: Move private _PySet API to the internal API (#107041) * Add pycore_setobject.h header file. * Move the following API to the internal C API: * _PySet_Dummy * _PySet_NextEntry() * _PySet_Update() files: A Include/internal/pycore_setobject.h M Include/cpython/setobject.h M Include/internal/pycore_pymem.h M Makefile.pre.in M Modules/_abc.c M Modules/_pickle.c M Objects/codeobject.c M Objects/dictobject.c M Objects/setobject.c M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters M Python/ast_opt.c M Python/bytecodes.c M Python/ceval.c M Python/compile.c M Python/executor.c M Python/marshal.c M Python/pylifecycle.c diff --git a/Include/cpython/setobject.h b/Include/cpython/setobject.h index 20fd63eaae56e..1778c778a0532 100644 --- a/Include/cpython/setobject.h +++ b/Include/cpython/setobject.h @@ -65,8 +65,3 @@ static inline Py_ssize_t PySet_GET_SIZE(PyObject *so) { return _PySet_CAST(so)->used; } #define PySet_GET_SIZE(so) PySet_GET_SIZE(_PyObject_CAST(so)) - -PyAPI_DATA(PyObject *) _PySet_Dummy; - -PyAPI_FUNC(int) _PySet_NextEntry(PyObject *set, Py_ssize_t *pos, PyObject **key, Py_hash_t *hash); -PyAPI_FUNC(int) _PySet_Update(PyObject *set, PyObject *iterable); diff --git a/Include/internal/pycore_pymem.h b/Include/internal/pycore_pymem.h index 81a707a0a5ddf..c2f03254bb876 100644 --- a/Include/internal/pycore_pymem.h +++ b/Include/internal/pycore_pymem.h @@ -95,4 +95,4 @@ PyAPI_FUNC(int) _PyMem_SetupAllocators(PyMemAllocatorName allocator); #ifdef __cplusplus } #endif -#endif /* !Py_INTERNAL_PYMEM_H */ +#endif // !Py_INTERNAL_PYMEM_H diff --git a/Include/internal/pycore_setobject.h b/Include/internal/pycore_setobject.h new file mode 100644 index 0000000000000..96f9aea7be9fa --- /dev/null +++ b/Include/internal/pycore_setobject.h @@ -0,0 +1,25 @@ +#ifndef Py_INTERNAL_SETOBJECT_H +#define Py_INTERNAL_SETOBJECT_H +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef Py_BUILD_CORE +# error "this header requires Py_BUILD_CORE define" +#endif + +// _pickle shared extension uses _PySet_NextEntry() and _PySet_Update() +PyAPI_FUNC(int) _PySet_NextEntry( + PyObject *set, + Py_ssize_t *pos, + PyObject **key, + Py_hash_t *hash); +PyAPI_FUNC(int) _PySet_Update(PyObject *set, PyObject *iterable); + +// Export _PySet_Dummy for the gdb plugin's benefit +PyAPI_DATA(PyObject *) _PySet_Dummy; + +#ifdef __cplusplus +} +#endif +#endif // !Py_INTERNAL_SETOBJECT_H diff --git a/Makefile.pre.in b/Makefile.pre.in index f3a0db163ed48..a4f76eaaaba52 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1798,6 +1798,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/internal/pycore_runtime.h \ $(srcdir)/Include/internal/pycore_runtime_init_generated.h \ $(srcdir)/Include/internal/pycore_runtime_init.h \ + $(srcdir)/Include/internal/pycore_setobject.h \ $(srcdir)/Include/internal/pycore_signal.h \ $(srcdir)/Include/internal/pycore_sliceobject.h \ $(srcdir)/Include/internal/pycore_strhex.h \ diff --git a/Modules/_abc.c b/Modules/_abc.c index 8a3aa9cb88880..9473905243d43 100644 --- a/Modules/_abc.c +++ b/Modules/_abc.c @@ -7,6 +7,7 @@ #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_object.h" // _PyType_GetSubclasses() #include "pycore_runtime.h" // _Py_ID() +#include "pycore_setobject.h" // _PySet_NextEntry() #include "pycore_typeobject.h" // _PyType_GetMRO() #include "pycore_weakref.h" // _PyWeakref_GET_REF() #include "clinic/_abc.c.h" diff --git a/Modules/_pickle.c b/Modules/_pickle.c index f2e98c10ef718..d9395e35daf8b 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -15,6 +15,7 @@ #include "pycore_object.h" // _PyNone_Type #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_runtime.h" // _Py_ID() +#include "pycore_setobject.h" // _PySet_NextEntry() #include "structmember.h" // PyMemberDef #include // strtol() diff --git a/Objects/codeobject.c b/Objects/codeobject.c index d2670c71caa44..977e5b05b3d2a 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -8,6 +8,7 @@ #include "pycore_interp.h" // PyInterpreterState.co_extra_freefuncs #include "pycore_opcode.h" // _PyOpcode_Deopt #include "pycore_pystate.h" // _PyInterpreterState_GET() +#include "pycore_setobject.h" // _PySet_NextEntry() #include "pycore_tuple.h" // _PyTuple_ITEMS() #include "clinic/codeobject.c.h" diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 3f9297c8a411f..26e2dc31e3664 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -121,6 +121,7 @@ As a consequence of this, split keys have a maximum size of 16. #include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_pyerrors.h" // _PyErr_GetRaisedException() #include "pycore_pystate.h" // _PyThreadState_GET() +#include "pycore_setobject.h" // _PySet_NextEntry() #include "stringlib/eq.h" // unicode_eq() #include diff --git a/Objects/setobject.c b/Objects/setobject.c index 4ac541b950975..2f12320254ec9 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -34,6 +34,7 @@ #include "Python.h" #include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _PyObject_GC_UNTRACK() +#include "pycore_setobject.h" // _PySet_NextEntry() definition #include // offsetof() /* Object used as dummy key to fill deleted entries */ diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 0fe24082ca4cc..bfe59acf12a69 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -261,6 +261,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 2b3793cb0f76e..0a8b0c3faf51e 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -687,6 +687,9 @@ Include\internal + + Include\internal + Include\internal diff --git a/Python/ast_opt.c b/Python/ast_opt.c index 276e910089a27..ad1e312b084b7 100644 --- a/Python/ast_opt.c +++ b/Python/ast_opt.c @@ -1,9 +1,10 @@ /* AST Optimizer */ #include "Python.h" #include "pycore_ast.h" // _PyAST_GetDocString() -#include "pycore_long.h" // _PyLong -#include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_format.h" // F_LJUST +#include "pycore_long.h" // _PyLong +#include "pycore_pystate.h" // _PyThreadState_GET() +#include "pycore_setobject.h" // _PySet_NextEntry() typedef struct { diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 3a191aca730e9..7ce8978e998b9 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -10,23 +10,24 @@ #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_ceval.h" // _PyEval_SignalAsyncExc() #include "pycore_code.h" +#include "pycore_emscripten_signal.h" // _Py_CHECK_EMSCRIPTEN_SIGNALS #include "pycore_function.h" +#include "pycore_instruments.h" #include "pycore_intrinsics.h" #include "pycore_long.h" // _PyLong_GetZero() -#include "pycore_instruments.h" -#include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_moduleobject.h" // PyModuleObject +#include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_opcode.h" // EXTRA_CASES #include "pycore_opcode_metadata.h" // uop names #include "pycore_opcode_utils.h" // MAKE_FUNCTION_* #include "pycore_pyerrors.h" // _PyErr_GetRaisedException() #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_range.h" // _PyRangeIterObject +#include "pycore_setobject.h" // _PySet_NextEntry() #include "pycore_sliceobject.h" // _PyBuildSlice_ConsumeRefs #include "pycore_sysmodule.h" // _PySys_Audit() #include "pycore_tuple.h" // _PyTuple_ITEMS() #include "pycore_typeobject.h" // _PySuper_Lookup() -#include "pycore_emscripten_signal.h" // _Py_CHECK_EMSCRIPTEN_SIGNALS #include "pycore_dict.h" #include "dictobject.h" diff --git a/Python/ceval.c b/Python/ceval.c index 9f2855f964ace..e9f082f85bd1b 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -7,24 +7,25 @@ #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_ceval.h" // _PyEval_SignalAsyncExc() #include "pycore_code.h" +#include "pycore_emscripten_signal.h" // _Py_CHECK_EMSCRIPTEN_SIGNALS #include "pycore_function.h" +#include "pycore_instruments.h" #include "pycore_intrinsics.h" #include "pycore_long.h" // _PyLong_GetZero() -#include "pycore_instruments.h" -#include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_moduleobject.h" // PyModuleObject +#include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_opcode.h" // EXTRA_CASES #include "pycore_opcode_metadata.h" #include "pycore_opcode_utils.h" // MAKE_FUNCTION_* #include "pycore_pyerrors.h" // _PyErr_GetRaisedException() #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_range.h" // _PyRangeIterObject +#include "pycore_setobject.h" // _PySet_Update() #include "pycore_sliceobject.h" // _PyBuildSlice_ConsumeRefs #include "pycore_sysmodule.h" // _PySys_Audit() #include "pycore_tuple.h" // _PyTuple_ITEMS() #include "pycore_typeobject.h" // _PySuper_Lookup() #include "pycore_uops.h" // _PyUOpExecutorObject -#include "pycore_emscripten_signal.h" // _Py_CHECK_EMSCRIPTEN_SIGNALS #include "pycore_dict.h" #include "dictobject.h" diff --git a/Python/compile.c b/Python/compile.c index 9a13ab525d8a4..600118768e639 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -28,12 +28,13 @@ #define NEED_OPCODE_TABLES #include "pycore_opcode_utils.h" #undef NEED_OPCODE_TABLES -#include "pycore_flowgraph.h" #include "pycore_code.h" // _PyCode_New() #include "pycore_compile.h" +#include "pycore_flowgraph.h" #include "pycore_intrinsics.h" #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_pystate.h" // _Py_GetConfig() +#include "pycore_setobject.h" // _PySet_NextEntry() #include "pycore_symtable.h" // PySTEntryObject, _PyFuture_FromAST() #define NEED_OPCODE_METADATA diff --git a/Python/executor.c b/Python/executor.c index 17322526a7bd7..57525df202d86 100644 --- a/Python/executor.c +++ b/Python/executor.c @@ -11,6 +11,7 @@ #include "pycore_opcode_utils.h" #include "pycore_pyerrors.h" #include "pycore_range.h" +#include "pycore_setobject.h" // _PySet_Update() #include "pycore_sliceobject.h" #include "pycore_uops.h" diff --git a/Python/marshal.c b/Python/marshal.c index 517220a4463cf..8940582c7f532 100644 --- a/Python/marshal.c +++ b/Python/marshal.c @@ -9,8 +9,9 @@ #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_code.h" // _PyCode_New() -#include "pycore_long.h" // _PyLong_DigitCount #include "pycore_hashtable.h" // _Py_hashtable_t +#include "pycore_long.h" // _PyLong_DigitCount +#include "pycore_setobject.h" // _PySet_NextEntry() #include "marshal.h" // Py_MARSHAL_VERSION /*[clinic input] diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index f91fac9379ec5..ceca7776a276e 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -24,6 +24,7 @@ #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_runtime.h" // _Py_ID() #include "pycore_runtime_init.h" // _PyRuntimeState_INIT +#include "pycore_setobject.h" // _PySet_NextEntry() #include "pycore_sliceobject.h" // _PySlice_Fini() #include "pycore_sysmodule.h" // _PySys_ClearAuditHooks() #include "pycore_traceback.h" // _Py_DumpTracebackThreads() From webhook-mailer at python.org Sat Jul 22 11:44:48 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 22 Jul 2023 15:44:48 -0000 Subject: [Python-checkins] Fix Sphinx warnings in `re` module docs (#107044) Message-ID: https://github.com/python/cpython/commit/149748ea4f552e6fe43a1d6d69bd65910a7c4813 commit: 149748ea4f552e6fe43a1d6d69bd65910a7c4813 branch: main author: wulmer committer: AlexWaygood date: 2023-07-22T16:44:44+01:00 summary: Fix Sphinx warnings in `re` module docs (#107044) files: M Doc/library/re.rst M Doc/tools/.nitignore diff --git a/Doc/library/re.rst b/Doc/library/re.rst index b7510b93d7542..629ee472cca68 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -501,6 +501,8 @@ The special characters are: in the ASCII range (``b'\x00'``-``b'\x7f'``). +.. _re-special-sequences: + The special sequences consist of ``'\'`` and a character from the list below. If the ordinary character is not an ASCII digit or an ASCII letter, then the resulting RE will match the second character. For example, ``\$`` matches the @@ -779,6 +781,17 @@ Flags Corresponds to the inline flag ``(?s)``. +.. data:: U + UNICODE + + In Python 2, this flag made :ref:`special sequences ` + include Unicode characters in matches. Since Python 3, Unicode characters + are matched by default. + + See :const:`A` for restricting matching on ASCII characters instead. + + This flag is only kept for backward compatibility. + .. data:: X VERBOSE @@ -1520,14 +1533,14 @@ Simulating scanf() .. index:: single: scanf() -Python does not currently have an equivalent to :c:func:`scanf`. Regular +Python does not currently have an equivalent to :c:func:`!scanf`. Regular expressions are generally more powerful, though also more verbose, than -:c:func:`scanf` format strings. The table below offers some more-or-less -equivalent mappings between :c:func:`scanf` format tokens and regular +:c:func:`!scanf` format strings. The table below offers some more-or-less +equivalent mappings between :c:func:`!scanf` format tokens and regular expressions. +--------------------------------+---------------------------------------------+ -| :c:func:`scanf` Token | Regular Expression | +| :c:func:`!scanf` Token | Regular Expression | +================================+=============================================+ | ``%c`` | ``.`` | +--------------------------------+---------------------------------------------+ @@ -1552,7 +1565,7 @@ To extract the filename and numbers from a string like :: /usr/sbin/sendmail - 0 errors, 4 warnings -you would use a :c:func:`scanf` format like :: +you would use a :c:func:`!scanf` format like :: %s - %d errors, %d warnings diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index ca6f32c55acd6..87157f0eff4c4 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -174,7 +174,6 @@ Doc/library/pyclbr.rst Doc/library/pydoc.rst Doc/library/pyexpat.rst Doc/library/random.rst -Doc/library/re.rst Doc/library/readline.rst Doc/library/reprlib.rst Doc/library/resource.rst From webhook-mailer at python.org Sat Jul 22 11:54:05 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 22 Jul 2023 15:54:05 -0000 Subject: [Python-checkins] [3.12] Fix Sphinx warnings in `re` module docs (GH-107044) (#107046) Message-ID: https://github.com/python/cpython/commit/844bdce712574feb339c3f91696930a38ad656a9 commit: 844bdce712574feb339c3f91696930a38ad656a9 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-22T16:54:02+01:00 summary: [3.12] Fix Sphinx warnings in `re` module docs (GH-107044) (#107046) Fix Sphinx warnings in `re` module docs (GH-107044) (cherry picked from commit 149748ea4f552e6fe43a1d6d69bd65910a7c4813) Co-authored-by: wulmer files: M Doc/library/re.rst M Doc/tools/.nitignore diff --git a/Doc/library/re.rst b/Doc/library/re.rst index b7510b93d7542..629ee472cca68 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -501,6 +501,8 @@ The special characters are: in the ASCII range (``b'\x00'``-``b'\x7f'``). +.. _re-special-sequences: + The special sequences consist of ``'\'`` and a character from the list below. If the ordinary character is not an ASCII digit or an ASCII letter, then the resulting RE will match the second character. For example, ``\$`` matches the @@ -779,6 +781,17 @@ Flags Corresponds to the inline flag ``(?s)``. +.. data:: U + UNICODE + + In Python 2, this flag made :ref:`special sequences ` + include Unicode characters in matches. Since Python 3, Unicode characters + are matched by default. + + See :const:`A` for restricting matching on ASCII characters instead. + + This flag is only kept for backward compatibility. + .. data:: X VERBOSE @@ -1520,14 +1533,14 @@ Simulating scanf() .. index:: single: scanf() -Python does not currently have an equivalent to :c:func:`scanf`. Regular +Python does not currently have an equivalent to :c:func:`!scanf`. Regular expressions are generally more powerful, though also more verbose, than -:c:func:`scanf` format strings. The table below offers some more-or-less -equivalent mappings between :c:func:`scanf` format tokens and regular +:c:func:`!scanf` format strings. The table below offers some more-or-less +equivalent mappings between :c:func:`!scanf` format tokens and regular expressions. +--------------------------------+---------------------------------------------+ -| :c:func:`scanf` Token | Regular Expression | +| :c:func:`!scanf` Token | Regular Expression | +================================+=============================================+ | ``%c`` | ``.`` | +--------------------------------+---------------------------------------------+ @@ -1552,7 +1565,7 @@ To extract the filename and numbers from a string like :: /usr/sbin/sendmail - 0 errors, 4 warnings -you would use a :c:func:`scanf` format like :: +you would use a :c:func:`!scanf` format like :: %s - %d errors, %d warnings diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 3a34c0b2cbfff..66b68305c5c69 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -188,7 +188,6 @@ Doc/library/pyclbr.rst Doc/library/pydoc.rst Doc/library/pyexpat.rst Doc/library/random.rst -Doc/library/re.rst Doc/library/readline.rst Doc/library/reprlib.rst Doc/library/resource.rst From webhook-mailer at python.org Sat Jul 22 11:58:09 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 15:58:09 -0000 Subject: [Python-checkins] gh-83006: Document behavior of `shutil.disk_usage` for non-mounted filesystems on Unix (#107031) Message-ID: https://github.com/python/cpython/commit/6e5f2235f3754307292c7d8d3698958136b5e311 commit: 6e5f2235f3754307292c7d8d3698958136b5e311 branch: main author: Matthieu Caneill committer: ambv date: 2023-07-22T17:58:06+02:00 summary: gh-83006: Document behavior of `shutil.disk_usage` for non-mounted filesystems on Unix (#107031) files: A Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst M Doc/library/shutil.rst diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 5bd80b10a6580..7699d22a72aaf 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -399,6 +399,12 @@ Directory and files operations total, used and free space, in bytes. *path* may be a file or a directory. + .. note:: + + On Unix filesystems, *path* must point to a path within a **mounted** + filesystem partition. On those platforms, CPython doesn't attempt to + retrieve disk usage information from non-mounted filesystems. + .. versionadded:: 3.3 .. versionchanged:: 3.8 diff --git a/Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst b/Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst new file mode 100644 index 0000000000000..e64d186082843 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst @@ -0,0 +1,2 @@ +Document behavior of :func:`shutil.disk_usage` for non-mounted filesystems +on Unix. From webhook-mailer at python.org Sat Jul 22 11:59:14 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 15:59:14 -0000 Subject: [Python-checkins] gh-75371: reformat Makefile.pre.in to accommodate for empty FRAMEWORKALTINSTALLLAST (#107035) Message-ID: https://github.com/python/cpython/commit/9c38206925246ab919cf558ac069ae9458720ba7 commit: 9c38206925246ab919cf558ac069ae9458720ba7 branch: main author: Moritz Neeb committer: ambv date: 2023-07-22T17:59:11+02:00 summary: gh-75371: reformat Makefile.pre.in to accommodate for empty FRAMEWORKALTINSTALLLAST (#107035) in the case of an empty FRAMEWORKALTINSTALLLAST, this patch prevents leaving an astray linebreak and two tabs in the resulting Makefile. Before change: ``` .PHONY: commoninstall commoninstall: check-clean-src \ altbininstall libinstall inclinstall libainstall \ sharedinstall altmaninstall \ ``` After change (with empty FRAMEWORKALTINSTALLLAST): ``` .PHONY: commoninstall commoninstall: check-clean-src \ altbininstall libinstall inclinstall libainstall \ sharedinstall altmaninstall ``` files: M Makefile.pre.in diff --git a/Makefile.pre.in b/Makefile.pre.in index a4f76eaaaba52..3725feaca66ce 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1940,8 +1940,7 @@ altinstall: commoninstall .PHONY: commoninstall commoninstall: check-clean-src @FRAMEWORKALTINSTALLFIRST@ \ altbininstall libinstall inclinstall libainstall \ - sharedinstall altmaninstall \ - @FRAMEWORKALTINSTALLLAST@ + sharedinstall altmaninstall @FRAMEWORKALTINSTALLLAST@ # Install shared libraries enabled by Setup DESTDIRS= $(exec_prefix) $(LIBDIR) $(BINLIBDEST) $(DESTSHARED) From webhook-mailer at python.org Sat Jul 22 12:07:11 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 16:07:11 -0000 Subject: [Python-checkins] gh-106320: Remove _Py_SwappedOp from the C API (#107036) Message-ID: https://github.com/python/cpython/commit/6fbc717214210e06313a283b2f3ec8ce67209609 commit: 6fbc717214210e06313a283b2f3ec8ce67209609 branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T16:07:07Z summary: gh-106320: Remove _Py_SwappedOp from the C API (#107036) Move _Py_SwappedOp to the internal C API (pycore_object.h). files: M Include/cpython/object.h M Include/internal/pycore_object.h M Objects/object.c diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 0e5b6acb43b41..49101c1875686 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -377,11 +377,6 @@ PyAPI_FUNC(PyObject *) _PyObject_FunctionStr(PyObject *); #endif -/* Maps Py_LT to Py_GT, ..., Py_GE to Py_LE. - * Defined in object.c. - */ -PyAPI_DATA(int) _Py_SwappedOp[]; - PyAPI_FUNC(void) _PyDebugAllocatorStats(FILE *out, const char *block_name, int num_blocks, size_t sizeof_block); diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h index 90588daa64cc3..a9c996557430a 100644 --- a/Include/internal/pycore_object.h +++ b/Include/internal/pycore_object.h @@ -442,6 +442,9 @@ extern PyObject* _PyCFunctionWithKeywords_TrampolineCall( PyAPI_DATA(PyTypeObject) _PyNone_Type; PyAPI_DATA(PyTypeObject) _PyNotImplemented_Type; +/* Maps Py_LT to Py_GT, ..., Py_GE to Py_LE. Defined in Objects/object.c. */ +PyAPI_DATA(int) _Py_SwappedOp[]; + #ifdef __cplusplus } #endif diff --git a/Objects/object.c b/Objects/object.c index bfbc87198f5b3..3b8839b877261 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -9,7 +9,7 @@ #include "pycore_floatobject.h" // _PyFloat_DebugMallocStats() #include "pycore_initconfig.h" // _PyStatus_EXCEPTION() #include "pycore_namespace.h" // _PyNamespace_Type -#include "pycore_object.h" // _PyType_CheckConsistency(), _Py_FatalRefcountError() +#include "pycore_object.h" // PyAPI_DATA() _Py_SwappedOp definition #include "pycore_pyerrors.h" // _PyErr_Occurred() #include "pycore_pymem.h" // _PyMem_IsPtrFreed() #include "pycore_pystate.h" // _PyThreadState_GET() From webhook-mailer at python.org Sat Jul 22 12:23:34 2023 From: webhook-mailer at python.org (methane) Date: Sat, 22 Jul 2023 16:23:34 -0000 Subject: [Python-checkins] gh-106847: Add -X warn_default_encoding in sys.flags Doc (GH-106854) Message-ID: https://github.com/python/cpython/commit/c332117b522df8b7aaf5cde81f965405dbb4ae7a commit: c332117b522df8b7aaf5cde81f965405dbb4ae7a branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: methane date: 2023-07-23T01:23:31+09:00 summary: gh-106847: Add -X warn_default_encoding in sys.flags Doc (GH-106854) gh-106847: Add -X warn_default_encoding in sys.flags Doc (GH-106854) (cherry picked from commit fd84ac0ee0a8d5e34e0a106eed7e50539b61c5f8) Co-authored-by: qqwqqw689 <114795525+qqwqqw689 at users.noreply.github.com> Co-authored-by: Nikita Sobolev files: M Doc/library/sys.rst diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 718dc48e804c0..dca35cc731b69 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -511,27 +511,28 @@ always available. The :term:`named tuple` *flags* exposes the status of command line flags. The attributes are read only. - ============================= ============================================================================================================== - attribute flag - ============================= ============================================================================================================== - :const:`debug` :option:`-d` - :const:`inspect` :option:`-i` - :const:`interactive` :option:`-i` - :const:`isolated` :option:`-I` - :const:`optimize` :option:`-O` or :option:`-OO` - :const:`dont_write_bytecode` :option:`-B` - :const:`no_user_site` :option:`-s` - :const:`no_site` :option:`-S` - :const:`ignore_environment` :option:`-E` - :const:`verbose` :option:`-v` - :const:`bytes_warning` :option:`-b` - :const:`quiet` :option:`-q` - :const:`hash_randomization` :option:`-R` - :const:`dev_mode` :option:`-X dev <-X>` (:ref:`Python Development Mode `) - :const:`utf8_mode` :option:`-X utf8 <-X>` - :const:`safe_path` :option:`-P` - :const:`int_max_str_digits` :option:`-X int_max_str_digits <-X>` (:ref:`integer string conversion length limitation `) - ============================= ============================================================================================================== + ============================== ============================================================================================================== + attribute flag + ============================== ============================================================================================================== + :const:`debug` :option:`-d` + :const:`inspect` :option:`-i` + :const:`interactive` :option:`-i` + :const:`isolated` :option:`-I` + :const:`optimize` :option:`-O` or :option:`-OO` + :const:`dont_write_bytecode` :option:`-B` + :const:`no_user_site` :option:`-s` + :const:`no_site` :option:`-S` + :const:`ignore_environment` :option:`-E` + :const:`verbose` :option:`-v` + :const:`bytes_warning` :option:`-b` + :const:`quiet` :option:`-q` + :const:`hash_randomization` :option:`-R` + :const:`dev_mode` :option:`-X dev <-X>` (:ref:`Python Development Mode `) + :const:`utf8_mode` :option:`-X utf8 <-X>` + :const:`safe_path` :option:`-P` + :const:`int_max_str_digits` :option:`-X int_max_str_digits <-X>` (:ref:`integer string conversion length limitation `) + :const:`warn_default_encoding` :option:`-X warn_default_encoding <-X>` + ============================== ============================================================================================================== .. versionchanged:: 3.2 Added ``quiet`` attribute for the new :option:`-q` flag. @@ -550,6 +551,9 @@ always available. Mode ` and the ``utf8_mode`` attribute for the new :option:`-X` ``utf8`` flag. + .. versionchanged:: 3.10 + Added ``warn_default_encoding`` attribute for :option:`-X` ``warn_default_encoding`` flag. + .. versionchanged:: 3.11 Added the ``safe_path`` attribute for :option:`-P` option. From webhook-mailer at python.org Sat Jul 22 12:39:52 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 22 Jul 2023 16:39:52 -0000 Subject: [Python-checkins] [3.11] Fix Sphinx warnings in `re` module docs (GH-107044). (#107055) Message-ID: https://github.com/python/cpython/commit/b71144193b5e7aec022a946a2e36f902280d7c30 commit: b71144193b5e7aec022a946a2e36f902280d7c30 branch: 3.11 author: wulmer committer: AlexWaygood date: 2023-07-22T17:39:46+01:00 summary: [3.11] Fix Sphinx warnings in `re` module docs (GH-107044). (#107055) (cherry picked from commit 149748ea4f552e6fe43a1d6d69bd65910a7c4813) files: M Doc/library/re.rst diff --git a/Doc/library/re.rst b/Doc/library/re.rst index 1f42cd28e65ec..798b56aa5d0c3 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -500,6 +500,8 @@ The special characters are: (``b'\x00'``-``b'\x7f'``) in :class:`bytes` replacement strings. +.. _re-special-sequences: + The special sequences consist of ``'\'`` and a character from the list below. If the ordinary character is not an ASCII digit or an ASCII letter, then the resulting RE will match the second character. For example, ``\$`` matches the @@ -778,6 +780,17 @@ Flags Corresponds to the inline flag ``(?s)``. +.. data:: U + UNICODE + + In Python 2, this flag made :ref:`special sequences ` + include Unicode characters in matches. Since Python 3, Unicode characters + are matched by default. + + See :const:`A` for restricting matching on ASCII characters instead. + + This flag is only kept for backward compatibility. + .. data:: X VERBOSE @@ -1518,14 +1531,14 @@ Simulating scanf() .. index:: single: scanf() -Python does not currently have an equivalent to :c:func:`scanf`. Regular +Python does not currently have an equivalent to :c:func:`!scanf`. Regular expressions are generally more powerful, though also more verbose, than -:c:func:`scanf` format strings. The table below offers some more-or-less -equivalent mappings between :c:func:`scanf` format tokens and regular +:c:func:`!scanf` format strings. The table below offers some more-or-less +equivalent mappings between :c:func:`!scanf` format tokens and regular expressions. +--------------------------------+---------------------------------------------+ -| :c:func:`scanf` Token | Regular Expression | +| :c:func:`!scanf` Token | Regular Expression | +================================+=============================================+ | ``%c`` | ``.`` | +--------------------------------+---------------------------------------------+ @@ -1550,7 +1563,7 @@ To extract the filename and numbers from a string like :: /usr/sbin/sendmail - 0 errors, 4 warnings -you would use a :c:func:`scanf` format like :: +you would use a :c:func:`!scanf` format like :: %s - %d errors, %d warnings From webhook-mailer at python.org Sat Jul 22 12:54:54 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 22 Jul 2023 16:54:54 -0000 Subject: [Python-checkins] gh-107008: Document the curses module variables LINES and COLS (GH-107011) Message-ID: https://github.com/python/cpython/commit/26e08dfdd7ac1b3d567d30cd35e4898121580390 commit: 26e08dfdd7ac1b3d567d30cd35e4898121580390 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-22T19:54:50+03:00 summary: gh-107008: Document the curses module variables LINES and COLS (GH-107011) LINES and COLS referred in curses.update_lines_cols() documentations are the module variables, not the environment variables. files: A Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst M Doc/library/curses.rst M Doc/whatsnew/3.5.rst diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst index cf208f3ba0db3..391c81a844d3e 100644 --- a/Doc/library/curses.rst +++ b/Doc/library/curses.rst @@ -641,7 +641,8 @@ The module :mod:`curses` defines the following functions: .. function:: update_lines_cols() - Update :envvar:`LINES` and :envvar:`COLS`. Useful for detecting manual screen resize. + Update the :const:`LINES` and :const:`COLS` module variables. + Useful for detecting manual screen resize. .. versionadded:: 3.5 @@ -1342,10 +1343,27 @@ The :mod:`curses` module defines the following data members: .. data:: COLORS The maximum number of colors the terminal can support. + It is defined only after the call to :func:`start_color`. .. data:: COLOR_PAIRS The maximum number of color pairs the terminal can support. + It is defined only after the call to :func:`start_color`. + +.. data:: COLS + + The width of the screen, i.e., the number of columns. + It is defined only after the call to :func:`initscr`. + Updated by :func:`update_lines_cols`, :func:`resizeterm` and + :func:`resize_term`. + +.. data:: LINES + + The height of the screen, i.e., the number of lines. + It is defined only after the call to :func:`initscr`. + Updated by :func:`update_lines_cols`, :func:`resizeterm` and + :func:`resize_term`. + Some constants are available to specify character cell attributes. The exact constants available are system dependent. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 262e4ed32c0a3..66610fa70efe9 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1045,8 +1045,8 @@ not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) curses ------ -The new :func:`~curses.update_lines_cols` function updates the :envvar:`LINES` -and :envvar:`COLS` environment variables. This is useful for detecting +The new :func:`~curses.update_lines_cols` function updates the :data:`LINES` +and :data:`COLS` module variables. This is useful for detecting manual screen resizing. (Contributed by Arnon Yaari in :issue:`4254`.) diff --git a/Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst b/Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst new file mode 100644 index 0000000000000..a0fa27ec10303 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst @@ -0,0 +1,2 @@ +Document the :mod:`curses` module variables :const:`~curses.LINES` and +:const:`~curses.COLS`. From webhook-mailer at python.org Sat Jul 22 13:44:15 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 22 Jul 2023 17:44:15 -0000 Subject: [Python-checkins] [3.11] gh-106970: Fix Argument Clinic 'destination clear' command (#106972) (#107059) Message-ID: https://github.com/python/cpython/commit/63c945a3f540a30288f21cb730a9da8bd4bf0ab6 commit: 63c945a3f540a30288f21cb730a9da8bd4bf0ab6 branch: 3.11 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-22T17:44:11Z summary: [3.11] gh-106970: Fix Argument Clinic 'destination clear' command (#106972) (#107059) Add test for the 'destination clear' command, and the 'destination' directive in general. Fix two bugs in 'destination clear' command: 1. The text attribute of the allocator is called 'text', not '_text' 2. Return after processing the 'clear' command, instead of proceeding directly to the fail(). (cherry picked from commit 3372bcba9893030e4063a9264ec0b4d1b6166883) files: A Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst M Lib/test/clinic.test.c M Tools/clinic/clinic.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index e4d02f83bc636..feb30458b462a 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -4192,3 +4192,59 @@ Test_meth_coexist(TestObj *self, PyObject *Py_UNUSED(ignored)) static PyObject * Test_meth_coexist_impl(TestObj *self) /*[clinic end generated code: output=808a293d0cd27439 input=2a1d75b5e6fec6dd]*/ + + +/*[clinic input] +output push +output preset buffer +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=5bff3376ee0df0b5]*/ + +/*[clinic input] +buffer_clear + a: int +We'll call 'destination buffer clear' after this. + +Argument Clinic's buffer preset puts most generated code into the +'buffer' destination, except from 'impl_definition', which is put into +the 'block' destination, so we should expect everything but +'impl_definition' to be cleared. +[clinic start generated code]*/ + +static PyObject * +buffer_clear_impl(PyObject *module, int a) +/*[clinic end generated code: output=f14bba74677e1846 input=a4c308a6fdab043c]*/ + +/*[clinic input] +destination buffer clear +output pop +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=f20d06adb8252084]*/ + + +/*[clinic input] +output push +destination test1 new buffer +output everything suppress +output docstring_definition test1 +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=5a77c454970992fc]*/ + +/*[clinic input] +new_dest + a: int +Only this docstring should be outputted to test1. +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=da5af421ed8996ed]*/ + +/*[clinic input] +dump test1 +output pop +[clinic start generated code]*/ + +PyDoc_STRVAR(new_dest__doc__, +"new_dest($module, /, a)\n" +"--\n" +"\n" +"Only this docstring should be outputted to test1."); +/*[clinic end generated code: output=9cac703f51d90e84 input=090db8df4945576d]*/ diff --git a/Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst b/Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst new file mode 100644 index 0000000000000..194e3351b0470 --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2023-07-21-23-16-05.gh-issue-106970.NLRnml.rst @@ -0,0 +1,4 @@ +Fix bugs in the Argument Clinic ``destination clear`` command; the +destination buffers would never be cleared, and the ``destination`` +directive parser would simply continue to the fault handler after processing +the command. Patch by Erlend E. Aasland. diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index b18c0205a8559..f8e89bc74f177 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1798,7 +1798,7 @@ def __getitem__(self, i): def clear(self): for ta in self._array: - ta._text.clear() + ta.text.clear() def dump(self): texts = [ta.output() for ta in self._array] @@ -4121,14 +4121,19 @@ def directive_set(self, name, value): self.clinic.__dict__[name] = value - def directive_destination(self, name, command, *args): - if command == 'new': - self.clinic.add_destination(name, *args) - return - - if command == 'clear': - self.clinic.get_destination(name).clear() - fail("unknown destination command", repr(command)) + def directive_destination( + self, + name: str, + command: str, + *args + ) -> None: + match command: + case "new": + self.clinic.add_destination(name, *args) + case "clear": + self.clinic.get_destination(name).clear() + case _: + fail("unknown destination command", repr(command)) def directive_output(self, command_or_name, destination=''): From webhook-mailer at python.org Sat Jul 22 14:13:01 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 22 Jul 2023 18:13:01 -0000 Subject: [Python-checkins] [3.12] gh-107008: Document the curses module variables LINES and COLS (GH-107011) (GH-107057) Message-ID: https://github.com/python/cpython/commit/3889d39471d626baf35b12511ba949a233a36106 commit: 3889d39471d626baf35b12511ba949a233a36106 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-22T21:12:58+03:00 summary: [3.12] gh-107008: Document the curses module variables LINES and COLS (GH-107011) (GH-107057) LINES and COLS referred in curses.update_lines_cols() documentations are the module variables, not the environment variables. (cherry picked from commit 26e08dfdd7ac1b3d567d30cd35e4898121580390) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst M Doc/library/curses.rst M Doc/whatsnew/3.5.rst diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst index cf208f3ba0db3..391c81a844d3e 100644 --- a/Doc/library/curses.rst +++ b/Doc/library/curses.rst @@ -641,7 +641,8 @@ The module :mod:`curses` defines the following functions: .. function:: update_lines_cols() - Update :envvar:`LINES` and :envvar:`COLS`. Useful for detecting manual screen resize. + Update the :const:`LINES` and :const:`COLS` module variables. + Useful for detecting manual screen resize. .. versionadded:: 3.5 @@ -1342,10 +1343,27 @@ The :mod:`curses` module defines the following data members: .. data:: COLORS The maximum number of colors the terminal can support. + It is defined only after the call to :func:`start_color`. .. data:: COLOR_PAIRS The maximum number of color pairs the terminal can support. + It is defined only after the call to :func:`start_color`. + +.. data:: COLS + + The width of the screen, i.e., the number of columns. + It is defined only after the call to :func:`initscr`. + Updated by :func:`update_lines_cols`, :func:`resizeterm` and + :func:`resize_term`. + +.. data:: LINES + + The height of the screen, i.e., the number of lines. + It is defined only after the call to :func:`initscr`. + Updated by :func:`update_lines_cols`, :func:`resizeterm` and + :func:`resize_term`. + Some constants are available to specify character cell attributes. The exact constants available are system dependent. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 86c2905e5fad9..83ce591c30ee6 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1045,8 +1045,8 @@ not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) curses ------ -The new :func:`~curses.update_lines_cols` function updates the :envvar:`LINES` -and :envvar:`COLS` environment variables. This is useful for detecting +The new :func:`~curses.update_lines_cols` function updates the :data:`LINES` +and :data:`COLS` module variables. This is useful for detecting manual screen resizing. (Contributed by Arnon Yaari in :issue:`4254`.) diff --git a/Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst b/Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst new file mode 100644 index 0000000000000..a0fa27ec10303 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst @@ -0,0 +1,2 @@ +Document the :mod:`curses` module variables :const:`~curses.LINES` and +:const:`~curses.COLS`. From webhook-mailer at python.org Sat Jul 22 14:13:41 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 22 Jul 2023 18:13:41 -0000 Subject: [Python-checkins] [3.11] gh-107008: Document the curses module variables LINES and COLS (GH-107011) (GH-107058) Message-ID: https://github.com/python/cpython/commit/e5d2a19ac5ad9d1130d778c960c4b966c32cac72 commit: e5d2a19ac5ad9d1130d778c960c4b966c32cac72 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-22T21:13:38+03:00 summary: [3.11] gh-107008: Document the curses module variables LINES and COLS (GH-107011) (GH-107058) LINES and COLS referred in curses.update_lines_cols() documentations are the module variables, not the environment variables. (cherry picked from commit 26e08dfdd7ac1b3d567d30cd35e4898121580390) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst M Doc/library/curses.rst M Doc/whatsnew/3.5.rst diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst index 44dd9d32f148e..e0bc462fb1c95 100644 --- a/Doc/library/curses.rst +++ b/Doc/library/curses.rst @@ -644,7 +644,8 @@ The module :mod:`curses` defines the following functions: .. function:: update_lines_cols() - Update :envvar:`LINES` and :envvar:`COLS`. Useful for detecting manual screen resize. + Update the :const:`LINES` and :const:`COLS` module variables. + Useful for detecting manual screen resize. .. versionadded:: 3.5 @@ -1345,10 +1346,27 @@ The :mod:`curses` module defines the following data members: .. data:: COLORS The maximum number of colors the terminal can support. + It is defined only after the call to :func:`start_color`. .. data:: COLOR_PAIRS The maximum number of color pairs the terminal can support. + It is defined only after the call to :func:`start_color`. + +.. data:: COLS + + The width of the screen, i.e., the number of columns. + It is defined only after the call to :func:`initscr`. + Updated by :func:`update_lines_cols`, :func:`resizeterm` and + :func:`resize_term`. + +.. data:: LINES + + The height of the screen, i.e., the number of lines. + It is defined only after the call to :func:`initscr`. + Updated by :func:`update_lines_cols`, :func:`resizeterm` and + :func:`resize_term`. + Some constants are available to specify character cell attributes. The exact constants available are system dependent. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 23d2f6562aab8..69d9147524cae 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1045,8 +1045,8 @@ not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) curses ------ -The new :func:`~curses.update_lines_cols` function updates the :envvar:`LINES` -and :envvar:`COLS` environment variables. This is useful for detecting +The new :func:`~curses.update_lines_cols` function updates the :data:`LINES` +and :data:`COLS` module variables. This is useful for detecting manual screen resizing. (Contributed by Arnon Yaari in :issue:`4254`.) diff --git a/Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst b/Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst new file mode 100644 index 0000000000000..a0fa27ec10303 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-07-22-15-14-13.gh-issue-107008.3JQ1Vt.rst @@ -0,0 +1,2 @@ +Document the :mod:`curses` module variables :const:`~curses.LINES` and +:const:`~curses.COLS`. From webhook-mailer at python.org Sat Jul 22 14:25:06 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 18:25:06 -0000 Subject: [Python-checkins] [3.12] gh-83006: Document behavior of `shutil.disk_usage` for non-mounted filesystems on Unix (GH-107031) (#107047) Message-ID: https://github.com/python/cpython/commit/0fff068181d1ad69907c06c1e0b30d5467a80f39 commit: 0fff068181d1ad69907c06c1e0b30d5467a80f39 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-22T20:25:03+02:00 summary: [3.12] gh-83006: Document behavior of `shutil.disk_usage` for non-mounted filesystems on Unix (GH-107031) (#107047) (cherry picked from commit 6e5f2235f3754307292c7d8d3698958136b5e311) Co-authored-by: Matthieu Caneill files: A Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst M Doc/library/shutil.rst diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 5bd80b10a6580..7699d22a72aaf 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -399,6 +399,12 @@ Directory and files operations total, used and free space, in bytes. *path* may be a file or a directory. + .. note:: + + On Unix filesystems, *path* must point to a path within a **mounted** + filesystem partition. On those platforms, CPython doesn't attempt to + retrieve disk usage information from non-mounted filesystems. + .. versionadded:: 3.3 .. versionchanged:: 3.8 diff --git a/Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst b/Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst new file mode 100644 index 0000000000000..e64d186082843 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst @@ -0,0 +1,2 @@ +Document behavior of :func:`shutil.disk_usage` for non-mounted filesystems +on Unix. From webhook-mailer at python.org Sat Jul 22 14:25:18 2023 From: webhook-mailer at python.org (ambv) Date: Sat, 22 Jul 2023 18:25:18 -0000 Subject: [Python-checkins] [3.11] gh-83006: Document behavior of `shutil.disk_usage` for non-mounted filesystems on Unix (GH-107031) (#107048) Message-ID: https://github.com/python/cpython/commit/2895dbd0f661566829d6dc4eadabb46d0395fde9 commit: 2895dbd0f661566829d6dc4eadabb46d0395fde9 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-22T20:25:15+02:00 summary: [3.11] gh-83006: Document behavior of `shutil.disk_usage` for non-mounted filesystems on Unix (GH-107031) (#107048) (cherry picked from commit 6e5f2235f3754307292c7d8d3698958136b5e311) Co-authored-by: Matthieu Caneill files: A Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst M Doc/library/shutil.rst diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 26a5cd531fa14..dcf421f162b15 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -393,6 +393,12 @@ Directory and files operations total, used and free space, in bytes. *path* may be a file or a directory. + .. note:: + + On Unix filesystems, *path* must point to a path within a **mounted** + filesystem partition. On those platforms, CPython doesn't attempt to + retrieve disk usage information from non-mounted filesystems. + .. versionadded:: 3.3 .. versionchanged:: 3.8 diff --git a/Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst b/Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst new file mode 100644 index 0000000000000..e64d186082843 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-15-51-33.gh-issue-83006.21zaCz.rst @@ -0,0 +1,2 @@ +Document behavior of :func:`shutil.disk_usage` for non-mounted filesystems +on Unix. From webhook-mailer at python.org Sat Jul 22 14:35:25 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 22 Jul 2023 18:35:25 -0000 Subject: [Python-checkins] gh-106948: Add standard external names to nitpick_ignore (GH-106949) Message-ID: https://github.com/python/cpython/commit/f8b7fe2f2647813ae8249675a80e59c117d30fe1 commit: f8b7fe2f2647813ae8249675a80e59c117d30fe1 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-22T21:35:22+03:00 summary: gh-106948: Add standard external names to nitpick_ignore (GH-106949) It includes standard C types, macros and variables like "size_t", "LONG_MAX" and "errno", and standard environment variables like "PATH". files: A Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst M Doc/c-api/arg.rst M Doc/c-api/memory.rst M Doc/c-api/unicode.rst M Doc/c-api/veryhigh.rst M Doc/conf.py M Doc/library/array.rst M Doc/library/ctypes.rst M Doc/library/os.rst M Doc/library/struct.rst M Doc/library/venv.rst M Doc/library/webbrowser.rst M Doc/tools/.nitignore M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.13.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a5.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0b1.rst M Misc/NEWS.d/next/C API/2023-05-31-18-37-57.gh-issue-105156.R4El5V.rst diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index e8d141f173933..9d744a12e374b 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -555,7 +555,7 @@ Building values Same as ``s#``. ``u`` (:class:`str`) [const wchar_t \*] - Convert a null-terminated :c:expr:`wchar_t` buffer of Unicode (UTF-16 or UCS-4) + Convert a null-terminated :c:type:`wchar_t` buffer of Unicode (UTF-16 or UCS-4) data to a Python Unicode object. If the Unicode buffer pointer is ``NULL``, ``None`` is returned. diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 35c356f25c6c7..4ca3b8804427f 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -581,7 +581,7 @@ that the treatment of negative indices differs from a Python slice): default). A serial number, incremented by 1 on each call to a malloc-like or - realloc-like function. Big-endian ``size_t``. If "bad memory" is detected + realloc-like function. Big-endian :c:type:`size_t`. If "bad memory" is detected later, the serial number gives an excellent way to set a breakpoint on the next run, to capture the instant at which this block was passed out. The static function bumpserialno() in obmalloc.c is the only place the serial diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index beb4f42a208fa..cf97b11bdbbf4 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -44,7 +44,7 @@ Python: .. c:type:: Py_UNICODE - This is a typedef of :c:expr:`wchar_t`, which is a 16-bit type or 32-bit type + This is a typedef of :c:type:`wchar_t`, which is a 16-bit type or 32-bit type depending on the platform. .. versionchanged:: 3.3 @@ -437,11 +437,11 @@ APIs: +----------+-----------------------------------------------------+ | ``ll`` | :c:expr:`long long` or :c:expr:`unsigned long long` | +----------+-----------------------------------------------------+ - | ``j`` | :c:expr:`intmax_t` or :c:expr:`uintmax_t` | + | ``j`` | :c:type:`intmax_t` or :c:type:`uintmax_t` | +----------+-----------------------------------------------------+ - | ``z`` | :c:expr:`size_t` or :c:expr:`ssize_t` | + | ``z`` | :c:type:`size_t` or :c:type:`ssize_t` | +----------+-----------------------------------------------------+ - | ``t`` | :c:expr:`ptrdiff_t` | + | ``t`` | :c:type:`ptrdiff_t` | +----------+-----------------------------------------------------+ The length modifier ``l`` for following conversions ``s`` or ``V`` specify @@ -520,7 +520,7 @@ APIs: .. note:: The width formatter unit is number of characters rather than bytes. - The precision formatter unit is number of bytes or :c:expr:`wchar_t` + The precision formatter unit is number of bytes or :c:type:`wchar_t` items (if the length modifier ``l`` is used) for ``"%s"`` and ``"%V"`` (if the ``PyObject*`` argument is ``NULL``), and a number of characters for ``"%A"``, ``"%U"``, ``"%S"``, ``"%R"`` and ``"%V"`` @@ -839,11 +839,11 @@ conversion function: wchar_t Support """"""""""""""" -:c:expr:`wchar_t` support for platforms which support it: +:c:type:`wchar_t` support for platforms which support it: .. c:function:: PyObject* PyUnicode_FromWideChar(const wchar_t *w, Py_ssize_t size) - Create a Unicode object from the :c:expr:`wchar_t` buffer *w* of the given *size*. + Create a Unicode object from the :c:type:`wchar_t` buffer *w* of the given *size*. Passing ``-1`` as the *size* indicates that the function must itself compute the length, using wcslen. Return ``NULL`` on failure. @@ -851,9 +851,9 @@ wchar_t Support .. c:function:: Py_ssize_t PyUnicode_AsWideChar(PyObject *unicode, wchar_t *w, Py_ssize_t size) - Copy the Unicode object contents into the :c:expr:`wchar_t` buffer *w*. At most - *size* :c:expr:`wchar_t` characters are copied (excluding a possibly trailing - null termination character). Return the number of :c:expr:`wchar_t` characters + Copy the Unicode object contents into the :c:type:`wchar_t` buffer *w*. At most + *size* :c:type:`wchar_t` characters are copied (excluding a possibly trailing + null termination character). Return the number of :c:type:`wchar_t` characters copied or ``-1`` in case of an error. Note that the resulting :c:expr:`wchar_t*` string may or may not be null-terminated. It is the responsibility of the caller to make sure that the :c:expr:`wchar_t*` string is null-terminated in case this is @@ -867,7 +867,7 @@ wchar_t Support Convert the Unicode object to a wide character string. The output string always ends with a null character. If *size* is not ``NULL``, write the number of wide characters (excluding the trailing null termination character) into - *\*size*. Note that the resulting :c:expr:`wchar_t` string might contain + *\*size*. Note that the resulting :c:type:`wchar_t` string might contain null characters, which would cause the string to be truncated when used with most C functions. If *size* is ``NULL`` and the :c:expr:`wchar_t*` string contains null characters a :exc:`ValueError` is raised. diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 1e8a945512d78..56fa2d6abd913 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -17,7 +17,7 @@ parameter. The available start symbols are :c:data:`Py_eval_input`, following the functions which accept them as parameters. Note also that several of these functions take :c:expr:`FILE*` parameters. One -particular issue which needs to be handled carefully is that the :c:expr:`FILE` +particular issue which needs to be handled carefully is that the :c:type:`FILE` structure for different C libraries can be different and incompatible. Under Windows (at least), it is possible for dynamically linked extensions to actually use different libraries, so care should be taken that :c:expr:`FILE*` parameters diff --git a/Doc/conf.py b/Doc/conf.py index 09e12e245891d..bd01850589b27 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -77,6 +77,58 @@ exclude_patterns.append(venvdir + '/*') nitpick_ignore = [ + # Standard C types + ('c:type', 'FILE'), + ('c:type', '__int'), + ('c:type', 'intmax_t'), + ('c:type', 'off_t'), + ('c:type', 'ptrdiff_t'), + ('c:type', 'siginfo_t'), + ('c:type', 'size_t'), + ('c:type', 'ssize_t'), + ('c:type', 'time_t'), + ('c:type', 'uintmax_t'), + ('c:type', 'va_list'), + ('c:type', 'wchar_t'), + # Standard C macros + ('c:macro', 'LLONG_MAX'), + ('c:macro', 'LLONG_MIN'), + ('c:macro', 'LONG_MAX'), + ('c:macro', 'LONG_MIN'), + # Standard C variables + ('c:data', 'errno'), + # Standard environment variables + ('envvar', 'BROWSER'), + ('envvar', 'COLUMNS'), + ('envvar', 'COMSPEC'), + ('envvar', 'DISPLAY'), + ('envvar', 'HOME'), + ('envvar', 'HOMEDRIVE'), + ('envvar', 'HOMEPATH'), + ('envvar', 'IDLESTARTUP'), + ('envvar', 'LANG'), + ('envvar', 'LANGUAGE'), + ('envvar', 'LC_ALL'), + ('envvar', 'LC_CTYPE'), + ('envvar', 'LC_COLLATE'), + ('envvar', 'LC_MESSAGES'), + ('envvar', 'LC_MONETARY'), + ('envvar', 'LC_NUMERIC'), + ('envvar', 'LC_TIME'), + ('envvar', 'LINES'), + ('envvar', 'LOGNAME'), + ('envvar', 'PAGER'), + ('envvar', 'PATH'), + ('envvar', 'PATHEXT'), + ('envvar', 'SOURCE_DATE_EPOCH'), + ('envvar', 'TEMP'), + ('envvar', 'TERM'), + ('envvar', 'TMP'), + ('envvar', 'TMPDIR'), + ('envvar', 'TZ'), + ('envvar', 'USER'), + ('envvar', 'USERNAME'), + ('envvar', 'USERPROFILE'), # Do not error nit-picky mode builds when _SubParsersAction.add_parser cannot # be resolved, as the method is currently undocumented. For context, see # https://github.com/python/cpython/pull/103289. diff --git a/Doc/library/array.rst b/Doc/library/array.rst index 0afc217642a75..ad62262772421 100644 --- a/Doc/library/array.rst +++ b/Doc/library/array.rst @@ -53,9 +53,9 @@ Notes: It can be 16 bits or 32 bits depending on the platform. .. versionchanged:: 3.9 - ``array('u')`` now uses ``wchar_t`` as C type instead of deprecated + ``array('u')`` now uses :c:type:`wchar_t` as C type instead of deprecated ``Py_UNICODE``. This change doesn't affect its behavior because - ``Py_UNICODE`` is alias of ``wchar_t`` since Python 3.3. + ``Py_UNICODE`` is alias of :c:type:`wchar_t` since Python 3.3. .. deprecated-removed:: 3.3 3.16 Please migrate to ``'w'`` typecode. diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst index 81509c0920bb6..c253a45e1a8b5 100644 --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -220,7 +220,7 @@ Fundamental data types +----------------------+------------------------------------------+----------------------------+ | :class:`c_char` | :c:expr:`char` | 1-character bytes object | +----------------------+------------------------------------------+----------------------------+ -| :class:`c_wchar` | :c:expr:`wchar_t` | 1-character string | +| :class:`c_wchar` | :c:type:`wchar_t` | 1-character string | +----------------------+------------------------------------------+----------------------------+ | :class:`c_byte` | :c:expr:`char` | int | +----------------------+------------------------------------------+----------------------------+ @@ -243,9 +243,9 @@ Fundamental data types | :class:`c_ulonglong` | :c:expr:`unsigned __int64` or | int | | | :c:expr:`unsigned long long` | | +----------------------+------------------------------------------+----------------------------+ -| :class:`c_size_t` | :c:expr:`size_t` | int | +| :class:`c_size_t` | :c:type:`size_t` | int | +----------------------+------------------------------------------+----------------------------+ -| :class:`c_ssize_t` | :c:expr:`ssize_t` or | int | +| :class:`c_ssize_t` | :c:type:`ssize_t` or | int | | | :c:expr:`Py_ssize_t` | | +----------------------+------------------------------------------+----------------------------+ | :class:`c_time_t` | :c:type:`time_t` | int | @@ -335,7 +335,7 @@ property:: The :func:`create_string_buffer` function replaces the old :func:`c_buffer` function (which is still available as an alias). To create a mutable memory -block containing unicode characters of the C type :c:expr:`wchar_t`, use the +block containing unicode characters of the C type :c:type:`wchar_t`, use the :func:`create_unicode_buffer` function. @@ -478,7 +478,7 @@ By default functions are assumed to return the C :c:expr:`int` type. Other return types can be specified by setting the :attr:`restype` attribute of the function object. -The C prototype of ``time()`` is ``time_t time(time_t *)``. Because ``time_t`` +The C prototype of ``time()`` is ``time_t time(time_t *)``. Because :c:type:`time_t` might be of a different type than the default return type ``int``, you should specify the ``restype``:: @@ -2407,7 +2407,7 @@ These are the fundamental ctypes data types: .. class:: c_wchar - Represents the C :c:expr:`wchar_t` datatype, and interprets the value as a + Represents the C :c:type:`wchar_t` datatype, and interprets the value as a single character unicode string. The constructor accepts an optional string initializer, the length of the string must be exactly one character. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 127d1616388b3..57405916ed7b3 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -4650,7 +4650,7 @@ written in Python, such as a mail server's external command delivery program. :data:`WNOHANG` and :data:`WNOWAIT` are additional optional flags. The return value is an object representing the data contained in the - :c:type:`!siginfo_t` structure with the following attributes: + :c:type:`siginfo_t` structure with the following attributes: * :attr:`!si_pid` (process ID) * :attr:`!si_uid` (real user ID of the child) diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index 6d2739b4557fb..c94dfde4d5576 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -231,9 +231,9 @@ platform-dependent. | ``Q`` | :c:expr:`unsigned long | integer | 8 | \(2) | | | long` | | | | +--------+--------------------------+--------------------+----------------+------------+ -| ``n`` | :c:expr:`ssize_t` | integer | | \(3) | +| ``n`` | :c:type:`ssize_t` | integer | | \(3) | +--------+--------------------------+--------------------+----------------+------------+ -| ``N`` | :c:expr:`size_t` | integer | | \(3) | +| ``N`` | :c:type:`size_t` | integer | | \(3) | +--------+--------------------------+--------------------+----------------+------------+ | ``e`` | \(6) | float | 2 | \(4) | +--------+--------------------------+--------------------+----------------+------------+ diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst index 9e5672545dea3..2482441d64979 100644 --- a/Doc/library/venv.rst +++ b/Doc/library/venv.rst @@ -60,7 +60,7 @@ running from a virtual environment. A virtual environment may be "activated" using a script in its binary directory (``bin`` on POSIX; ``Scripts`` on Windows). -This will prepend that directory to your :envvar:`!PATH`, so that running +This will prepend that directory to your :envvar:`PATH`, so that running :program:`python` will invoke the environment's Python interpreter and you can run installed scripts without having to use their full path. The invocation of the activation script is platform-specific @@ -100,10 +100,10 @@ In order to achieve this, scripts installed into virtual environments have a "shebang" line which points to the environment's Python interpreter, i.e. :samp:`#!/{}/bin/python`. This means that the script will run with that interpreter regardless of the -value of :envvar:`!PATH`. On Windows, "shebang" line processing is supported if +value of :envvar:`PATH`. On Windows, "shebang" line processing is supported if you have the :ref:`launcher` installed. Thus, double-clicking an installed script in a Windows Explorer window should run it with the correct interpreter -without the environment needing to be activated or on the :envvar:`!PATH`. +without the environment needing to be activated or on the :envvar:`PATH`. When a virtual environment has been activated, the :envvar:`!VIRTUAL_ENV` environment variable is set to the path of the environment. diff --git a/Doc/library/webbrowser.rst b/Doc/library/webbrowser.rst index b6762f78830a5..4667b81e38ada 100644 --- a/Doc/library/webbrowser.rst +++ b/Doc/library/webbrowser.rst @@ -20,7 +20,7 @@ will be used if graphical browsers are not available or an X11 display isn't available. If text-mode browsers are used, the calling process will block until the user exits the browser. -If the environment variable :envvar:`!BROWSER` exists, it is interpreted as the +If the environment variable :envvar:`BROWSER` exists, it is interpreted as the :data:`os.pathsep`-separated list of browsers to try ahead of the platform defaults. When the value of a list part contains the string ``%s``, then it is interpreted as a literal browser command line to be used with the argument URL @@ -97,7 +97,7 @@ The following functions are defined: Setting *preferred* to ``True`` makes this browser a preferred result for a :func:`get` call with no argument. Otherwise, this entry point is only - useful if you plan to either set the :envvar:`!BROWSER` variable or call + useful if you plan to either set the :envvar:`BROWSER` variable or call :func:`get` with a nonempty argument matching the name of a handler you declare. @@ -224,4 +224,4 @@ module-level convenience functions: .. rubric:: Footnotes .. [1] Executables named here without a full path will be searched in the - directories given in the :envvar:`!PATH` environment variable. + directories given in the :envvar:`PATH` environment variable. diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 87157f0eff4c4..14d9b2e121d3b 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -91,7 +91,6 @@ Doc/library/codecs.rst Doc/library/codeop.rst Doc/library/collections.abc.rst Doc/library/collections.rst -Doc/library/compileall.rst Doc/library/concurrent.futures.rst Doc/library/concurrent.rst Doc/library/configparser.rst @@ -100,7 +99,6 @@ Doc/library/contextlib.rst Doc/library/copy.rst Doc/library/csv.rst Doc/library/ctypes.rst -Doc/library/curses.rst Doc/library/datetime.rst Doc/library/dbm.rst Doc/library/decimal.rst @@ -137,7 +135,6 @@ Doc/library/http.client.rst Doc/library/http.cookiejar.rst Doc/library/http.cookies.rst Doc/library/http.server.rst -Doc/library/idle.rst Doc/library/importlib.resources.abc.rst Doc/library/importlib.resources.rst Doc/library/importlib.rst @@ -165,11 +162,9 @@ Doc/library/pickletools.rst Doc/library/platform.rst Doc/library/plistlib.rst Doc/library/poplib.rst -Doc/library/posix.rst Doc/library/pprint.rst Doc/library/profile.rst Doc/library/pty.rst -Doc/library/py_compile.rst Doc/library/pyclbr.rst Doc/library/pydoc.rst Doc/library/pyexpat.rst @@ -192,7 +187,6 @@ Doc/library/ssl.rst Doc/library/stat.rst Doc/library/stdtypes.rst Doc/library/string.rst -Doc/library/struct.rst Doc/library/subprocess.rst Doc/library/sys.rst Doc/library/sys_path_init.rst @@ -254,7 +248,6 @@ Doc/tutorial/modules.rst Doc/tutorial/stdlib2.rst Doc/using/cmdline.rst Doc/using/configure.rst -Doc/using/unix.rst Doc/using/windows.rst Doc/whatsnew/2.0.rst Doc/whatsnew/2.1.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index ac03af52db071..ea9806987f687 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1768,7 +1768,7 @@ Porting to Python 3.12 for example). * Add support of more formatting options (left aligning, octals, uppercase - hexadecimals, ``intmax_t``, ``ptrdiff_t``, ``wchar_t`` C + hexadecimals, :c:type:`intmax_t`, :c:type:`ptrdiff_t`, :c:type:`wchar_t` C strings, variable width and precision) in :c:func:`PyUnicode_FromFormat` and :c:func:`PyUnicode_FromFormatV`. (Contributed by Serhiy Storchaka in :gh:`98836`.) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 5d4fa44aab695..1d34d8a0fa463 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -329,7 +329,7 @@ Pending Removal in Python 3.15 Pending Removal in Python 3.16 ------------------------------ -* :class:`array.array` ``'u'`` type (``wchar_t``): +* :class:`array.array` ``'u'`` type (:c:type:`wchar_t`): use the ``'w'`` type instead (``Py_UCS4``). Pending Removal in Future Versions @@ -802,8 +802,8 @@ Deprecated ---------- * Deprecate the old ``Py_UNICODE`` and ``PY_UNICODE_TYPE`` types: use directly - the ``wchar_t`` type instead. Since Python 3.3, ``Py_UNICODE`` and - ``PY_UNICODE_TYPE`` are just aliases to ``wchar_t``. + the :c:type:`wchar_t` type instead. Since Python 3.3, ``Py_UNICODE`` and + ``PY_UNICODE_TYPE`` are just aliases to :c:type:`wchar_t`. (Contributed by Victor Stinner in :gh:`105156`.) * Deprecate old Python initialization functions: @@ -1013,8 +1013,8 @@ Pending Removal in Python 3.15 * :c:func:`PyImport_ImportModuleNoBlock`: use :c:func:`PyImport_ImportModule`. * :c:func:`PyWeakref_GET_OBJECT`: use :c:func:`PyWeakref_GetRef` instead. * :c:func:`PyWeakref_GetObject`: use :c:func:`PyWeakref_GetRef` instead. -* :c:type:`!Py_UNICODE_WIDE` type: use ``wchar_t`` instead. -* :c:type:`Py_UNICODE` type: use ``wchar_t`` instead. +* :c:type:`!Py_UNICODE_WIDE` type: use :c:type:`wchar_t` instead. +* :c:type:`Py_UNICODE` type: use :c:type:`wchar_t` instead. * Python initialization functions: * :c:func:`PySys_ResetWarnOptions`: clear :data:`sys.warnoptions` and diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index dbac74167a126..911e5ba47df41 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -1984,7 +1984,7 @@ the form '-rwxrwxrwx'. struct ------ -The :mod:`struct` module now supports ``ssize_t`` and ``size_t`` via the +The :mod:`struct` module now supports :c:type:`ssize_t` and :c:type:`size_t` via the new codes ``n`` and ``N``, respectively. (Contributed by Antoine Pitrou in :issue:`3163`.) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 66610fa70efe9..1e30569c559b9 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2192,7 +2192,7 @@ encode error with ``\N{...}`` escapes. (Contributed by Serhiy Storchaka in :issue:`19676`.) A new :c:func:`PyErr_FormatV` function similar to :c:func:`PyErr_Format`, -but accepts a ``va_list`` argument. +but accepts a :c:type:`va_list` argument. (Contributed by Antoine Pitrou in :issue:`18711`.) A new :c:data:`PyExc_RecursionError` exception. diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 9280f94740005..bf889b7ffbadf 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1115,9 +1115,9 @@ Changes in the Python API ``PyCF_ALLOW_TOP_LEVEL_AWAIT`` was clashing with ``CO_FUTURE_DIVISION``. (Contributed by Batuhan Taskaya in :issue:`39562`) -* ``array('u')`` now uses ``wchar_t`` as C type instead of ``Py_UNICODE``. +* ``array('u')`` now uses :c:type:`wchar_t` as C type instead of ``Py_UNICODE``. This change doesn't affect to its behavior because ``Py_UNICODE`` is alias - of ``wchar_t`` since Python 3.3. + of :c:type:`wchar_t` since Python 3.3. (Contributed by Inada Naoki in :issue:`34538`.) * The :func:`logging.getLogger` API now returns the root logger when passed diff --git a/Misc/NEWS.d/3.10.0a5.rst b/Misc/NEWS.d/3.10.0a5.rst index 497e384917183..dc95e8ce072fd 100644 --- a/Misc/NEWS.d/3.10.0a5.rst +++ b/Misc/NEWS.d/3.10.0a5.rst @@ -667,4 +667,4 @@ exception (if an exception is set). Patch by Victor Stinner. .. section: C API Fixed a compiler warning in :c:func:`Py_UNICODE_ISSPACE()` on platforms with -signed ``wchar_t``. +signed :c:type:`wchar_t`. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index 9109db537396e..209d079feb514 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -5308,7 +5308,7 @@ parameter. Patch by Kumar Aditya. .. section: Build Python now always use the ``%zu`` and ``%zd`` printf formats to format a -``size_t`` or ``Py_ssize_t`` number. Building Python 3.12 requires a C11 +:c:type:`size_t` or ``Py_ssize_t`` number. Building Python 3.12 requires a C11 compiler, so these printf formats are now always supported. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/3.12.0b1.rst b/Misc/NEWS.d/3.12.0b1.rst index 51d80711d91ed..89af6efdae048 100644 --- a/Misc/NEWS.d/3.12.0b1.rst +++ b/Misc/NEWS.d/3.12.0b1.rst @@ -2382,7 +2382,7 @@ Patch by Dong-hee Na. .. section: C API Add support of more formatting options (left aligning, octals, uppercase -hexadecimals, :c:expr:`intmax_t`, :c:expr:`ptrdiff_t`, :c:expr:`wchar_t` C +hexadecimals, :c:type:`intmax_t`, :c:type:`ptrdiff_t`, :c:type:`wchar_t` C strings, variable width and precision) in :c:func:`PyUnicode_FromFormat` and :c:func:`PyUnicode_FromFormatV`. diff --git a/Misc/NEWS.d/next/C API/2023-05-31-18-37-57.gh-issue-105156.R4El5V.rst b/Misc/NEWS.d/next/C API/2023-05-31-18-37-57.gh-issue-105156.R4El5V.rst index cbdb8379f24cc..536e484116690 100644 --- a/Misc/NEWS.d/next/C API/2023-05-31-18-37-57.gh-issue-105156.R4El5V.rst +++ b/Misc/NEWS.d/next/C API/2023-05-31-18-37-57.gh-issue-105156.R4El5V.rst @@ -1,4 +1,4 @@ Deprecate the old ``Py_UNICODE`` and ``PY_UNICODE_TYPE`` types: use directly -the ``wchar_t`` type instead. Since Python 3.3, ``Py_UNICODE`` and -``PY_UNICODE_TYPE`` are just aliases to ``wchar_t``. Patch by Victor +the :c:type:`wchar_t` type instead. Since Python 3.3, ``Py_UNICODE`` and +``PY_UNICODE_TYPE`` are just aliases to :c:type:`wchar_t`. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst b/Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst new file mode 100644 index 0000000000000..42b6348153b56 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst @@ -0,0 +1 @@ +Add a number of standard external names to ``nitpick_ignore``. From webhook-mailer at python.org Sat Jul 22 15:31:59 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 19:31:59 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyInterpreterID C API (#107053) Message-ID: https://github.com/python/cpython/commit/22422e9d1a50b4970bc6957a88569618d579c47f commit: 22422e9d1a50b4970bc6957a88569618d579c47f branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T19:31:55Z summary: gh-106320: Remove private _PyInterpreterID C API (#107053) Move the private _PyInterpreterID C API to the internal C API: add a new pycore_interp_id.h header file. Remove Include/interpreteridobject.h and Include/cpython/interpreteridobject.h header files. files: A Include/internal/pycore_interp_id.h D Include/cpython/interpreteridobject.h D Include/interpreteridobject.h M Makefile.pre.in M Modules/_testinternalcapi.c M Modules/_xxinterpchannelsmodule.c M Modules/_xxsubinterpretersmodule.c M Objects/interpreteridobject.c M Objects/object.c M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters M Tools/c-analyzer/cpython/_parser.py diff --git a/Include/cpython/interpreteridobject.h b/Include/cpython/interpreteridobject.h deleted file mode 100644 index 5076584209b90..0000000000000 --- a/Include/cpython/interpreteridobject.h +++ /dev/null @@ -1,11 +0,0 @@ -#ifndef Py_CPYTHON_INTERPRETERIDOBJECT_H -# error "this header file must not be included directly" -#endif - -/* Interpreter ID Object */ - -PyAPI_DATA(PyTypeObject) _PyInterpreterID_Type; - -PyAPI_FUNC(PyObject *) _PyInterpreterID_New(int64_t); -PyAPI_FUNC(PyObject *) _PyInterpreterState_GetIDObject(PyInterpreterState *); -PyAPI_FUNC(PyInterpreterState *) _PyInterpreterID_LookUp(PyObject *); diff --git a/Include/internal/pycore_interp_id.h b/Include/internal/pycore_interp_id.h new file mode 100644 index 0000000000000..34c96df98ef73 --- /dev/null +++ b/Include/internal/pycore_interp_id.h @@ -0,0 +1,28 @@ +/* Interpreter ID Object */ + +#ifndef Py_INTERNAL_INTERPRETERIDOBJECT_H +#define Py_INTERNAL_INTERPRETERIDOBJECT_H +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef Py_BUILD_CORE +# error "this header requires Py_BUILD_CORE define" +#endif + +// Export for the _xxsubinterpreters shared extension +PyAPI_DATA(PyTypeObject) _PyInterpreterID_Type; + +// Export for the _xxsubinterpreters shared extension +PyAPI_FUNC(PyObject*) _PyInterpreterID_New(int64_t); + +// Export for the _xxinterpchannels shared extension +PyAPI_FUNC(PyObject*) _PyInterpreterState_GetIDObject(PyInterpreterState *); + +// Export for the _testinternalcapi shared extension +PyAPI_FUNC(PyInterpreterState*) _PyInterpreterID_LookUp(PyObject *); + +#ifdef __cplusplus +} +#endif +#endif // !Py_INTERNAL_INTERPRETERIDOBJECT_H diff --git a/Include/interpreteridobject.h b/Include/interpreteridobject.h deleted file mode 100644 index 8432632f339e9..0000000000000 --- a/Include/interpreteridobject.h +++ /dev/null @@ -1,17 +0,0 @@ -#ifndef Py_INTERPRETERIDOBJECT_H -#define Py_INTERPRETERIDOBJECT_H - -#ifdef __cplusplus -extern "C" { -#endif - -#ifndef Py_LIMITED_API -# define Py_CPYTHON_INTERPRETERIDOBJECT_H -# include "cpython/interpreteridobject.h" -# undef Py_CPYTHON_INTERPRETERIDOBJECT_H -#endif - -#ifdef __cplusplus -} -#endif -#endif /* !Py_INTERPRETERIDOBJECT_H */ diff --git a/Makefile.pre.in b/Makefile.pre.in index 3725feaca66ce..5f1988b0d1721 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1631,7 +1631,6 @@ PYTHON_HEADERS= \ $(srcdir)/Include/floatobject.h \ $(srcdir)/Include/frameobject.h \ $(srcdir)/Include/import.h \ - $(srcdir)/Include/interpreteridobject.h \ $(srcdir)/Include/intrcheck.h \ $(srcdir)/Include/iterobject.h \ $(srcdir)/Include/listobject.h \ @@ -1701,7 +1700,6 @@ PYTHON_HEADERS= \ $(srcdir)/Include/cpython/genobject.h \ $(srcdir)/Include/cpython/import.h \ $(srcdir)/Include/cpython/initconfig.h \ - $(srcdir)/Include/cpython/interpreteridobject.h \ $(srcdir)/Include/cpython/listobject.h \ $(srcdir)/Include/cpython/longintrepr.h \ $(srcdir)/Include/cpython/longobject.h \ @@ -1773,6 +1771,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/internal/pycore_import.h \ $(srcdir)/Include/internal/pycore_initconfig.h \ $(srcdir)/Include/internal/pycore_interp.h \ + $(srcdir)/Include/internal/pycore_interp_id.h \ $(srcdir)/Include/internal/pycore_intrinsics.h \ $(srcdir)/Include/internal/pycore_list.h \ $(srcdir)/Include/internal/pycore_long.h \ diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 74c932fa921cd..ecc2721a4988f 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -11,18 +11,18 @@ #include "Python.h" #include "frameobject.h" -#include "interpreteridobject.h" // _PyInterpreterID_LookUp() #include "pycore_atomic_funcs.h" // _Py_atomic_int_get() #include "pycore_bitutils.h" // _Py_bswap32() #include "pycore_bytesobject.h" // _PyBytes_Find() -#include "pycore_compile.h" // _PyCompile_CodeGen, _PyCompile_OptimizeCfg, _PyCompile_Assemble, _PyCompile_CleanDoc #include "pycore_ceval.h" // _PyEval_AddPendingCall +#include "pycore_compile.h" // _PyCompile_CodeGen, _PyCompile_OptimizeCfg, _PyCompile_Assemble, _PyCompile_CleanDoc #include "pycore_fileutils.h" // _Py_normpath #include "pycore_frame.h" // _PyInterpreterFrame #include "pycore_gc.h" // PyGC_Head #include "pycore_hashtable.h" // _Py_hashtable_new() #include "pycore_initconfig.h" // _Py_GetConfigsAsDict() #include "pycore_interp.h" // _PyInterpreterState_GetConfigCopy() +#include "pycore_interp_id.h" // _PyInterpreterID_LookUp() #include "pycore_pathconfig.h" // _PyPathConfig_ClearGlobal() #include "pycore_pyerrors.h" // _Py_UTF8_Edit_Cost() #include "pycore_pystate.h" // _PyThreadState_GET() diff --git a/Modules/_xxinterpchannelsmodule.c b/Modules/_xxinterpchannelsmodule.c index 82472555ec7d6..bdffdba52aa02 100644 --- a/Modules/_xxinterpchannelsmodule.c +++ b/Modules/_xxinterpchannelsmodule.c @@ -6,8 +6,8 @@ #endif #include "Python.h" -#include "interpreteridobject.h" #include "pycore_atexit.h" // _Py_AtExit() +#include "pycore_interp_id.h" // _PyInterpreterState_GetIDObject() /* diff --git a/Modules/_xxsubinterpretersmodule.c b/Modules/_xxsubinterpretersmodule.c index d2e0593872c5f..bf01aaf6ed040 100644 --- a/Modules/_xxsubinterpretersmodule.c +++ b/Modules/_xxsubinterpretersmodule.c @@ -7,7 +7,7 @@ #include "Python.h" #include "pycore_interp.h" // _PyInterpreterState_GetMainModule() -#include "interpreteridobject.h" +#include "pycore_interp_id.h" // _PyInterpreterState_GetIDObject() #define MODULE_NAME "_xxsubinterpreters" diff --git a/Objects/interpreteridobject.c b/Objects/interpreteridobject.c index 46239100dcb7b..7a3e245ce3357 100644 --- a/Objects/interpreteridobject.c +++ b/Objects/interpreteridobject.c @@ -3,7 +3,7 @@ #include "Python.h" #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_interp.h" // _PyInterpreterState_LookUpID() -#include "interpreteridobject.h" +#include "pycore_interp_id.h" // _PyInterpreterID_Type typedef struct interpid { diff --git a/Objects/object.c b/Objects/object.c index 3b8839b877261..93396bf5491d3 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -8,16 +8,16 @@ #include "pycore_dict.h" // _PyObject_MakeDictFromInstanceAttributes() #include "pycore_floatobject.h" // _PyFloat_DebugMallocStats() #include "pycore_initconfig.h" // _PyStatus_EXCEPTION() +#include "pycore_interp_id.h" // _PyInterpreterID_Type #include "pycore_namespace.h" // _PyNamespace_Type #include "pycore_object.h" // PyAPI_DATA() _Py_SwappedOp definition #include "pycore_pyerrors.h" // _PyErr_Occurred() #include "pycore_pymem.h" // _PyMem_IsPtrFreed() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_symtable.h" // PySTEntry_Type -#include "pycore_typevarobject.h" // _PyTypeAlias_Type, _Py_initialize_generic #include "pycore_typeobject.h" // _PyBufferWrapper_Type +#include "pycore_typevarobject.h" // _PyTypeAlias_Type, _Py_initialize_generic #include "pycore_unionobject.h" // _PyUnion_Type -#include "interpreteridobject.h" // _PyInterpreterID_Type #ifdef Py_LIMITED_API // Prevent recursive call _Py_IncRef() <=> Py_INCREF() diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index bfe59acf12a69..5ccc895833065 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -152,7 +152,6 @@ - @@ -238,6 +237,7 @@ + @@ -281,7 +281,6 @@ - diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 0a8b0c3faf51e..54a77f81a9a1a 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -312,9 +312,6 @@ Include - - Include - Modules @@ -462,9 +459,6 @@ Include - - Include\cpython - Include\cpython @@ -618,6 +612,9 @@ Include\internal + + Include\internal + Include\cpython diff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.py index 9bc7285e18b2f..64ce4cda7d6ed 100644 --- a/Tools/c-analyzer/cpython/_parser.py +++ b/Tools/c-analyzer/cpython/_parser.py @@ -227,7 +227,6 @@ def clean_lines(text): Include/cpython/fileutils.h Py_CPYTHON_FILEUTILS_H 1 Include/cpython/frameobject.h Py_CPYTHON_FRAMEOBJECT_H 1 Include/cpython/import.h Py_CPYTHON_IMPORT_H 1 -Include/cpython/interpreteridobject.h Py_CPYTHON_INTERPRETERIDOBJECT_H 1 Include/cpython/listobject.h Py_CPYTHON_LISTOBJECT_H 1 Include/cpython/methodobject.h Py_CPYTHON_METHODOBJECT_H 1 Include/cpython/object.h Py_CPYTHON_OBJECT_H 1 From webhook-mailer at python.org Sat Jul 22 16:43:22 2023 From: webhook-mailer at python.org (vsajip) Date: Sat, 22 Jul 2023 20:43:22 -0000 Subject: [Python-checkins] gh-107028: tiny textual changes in logging docs and docstrings (GH-107029) Message-ID: https://github.com/python/cpython/commit/5e5a34ac3a827e040cd89426b1774fec2123336a commit: 5e5a34ac3a827e040cd89426b1774fec2123336a branch: main author: Jochem Boersma committer: vsajip date: 2023-07-22T21:43:18+01:00 summary: gh-107028: tiny textual changes in logging docs and docstrings (GH-107029) files: M Doc/library/logging.handlers.rst M Lib/logging/handlers.py diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index 47bfe4ff7f90e..72e5ffb3a1218 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -917,8 +917,9 @@ should, then :meth:`flush` is expected to do the flushing. .. method:: flush() - You can override this to implement custom flushing behavior. This version - just zaps the buffer to empty. + For a :class:`BufferingHandler` instance, flushing means that it sets the + buffer to an empty list. This method can be overwritten to implement more useful + flushing behavior. .. method:: shouldFlush(record) @@ -950,9 +951,9 @@ should, then :meth:`flush` is expected to do the flushing. .. method:: flush() - For a :class:`MemoryHandler`, flushing means just sending the buffered + For a :class:`MemoryHandler` instance, flushing means just sending the buffered records to the target, if there is one. The buffer is also cleared when - this happens. Override if you want different behavior. + buffered records are sent to the target. Override if you want different behavior. .. method:: setTarget(target) diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py index 9847104446eaf..671cc9596b02d 100644 --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -1399,7 +1399,7 @@ def flush(self): records to the target, if there is one. Override if you want different behaviour. - The record buffer is also cleared by this operation. + The record buffer is only cleared if a target has been set. """ self.acquire() try: From webhook-mailer at python.org Sat Jul 22 16:58:03 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 20:58:03 -0000 Subject: [Python-checkins] gh-106320: Move _PyMethodWrapper_Type to internal C API (#107064) Message-ID: https://github.com/python/cpython/commit/ee15844db88fab3a282d14325662bcfd026272ac commit: ee15844db88fab3a282d14325662bcfd026272ac branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T20:57:59Z summary: gh-106320: Move _PyMethodWrapper_Type to internal C API (#107064) files: M Include/cpython/descrobject.h M Include/internal/pycore_descrobject.h M Objects/descrobject.c M Objects/object.c M Python/specialize.c diff --git a/Include/cpython/descrobject.h b/Include/cpython/descrobject.h index e2ea1b9a2d305..bbad8b59c225a 100644 --- a/Include/cpython/descrobject.h +++ b/Include/cpython/descrobject.h @@ -57,8 +57,6 @@ typedef struct { void *d_wrapped; /* This can be any function pointer */ } PyWrapperDescrObject; -PyAPI_DATA(PyTypeObject) _PyMethodWrapper_Type; - PyAPI_FUNC(PyObject *) PyDescr_NewWrapper(PyTypeObject *, struct wrapperbase *, void *); PyAPI_FUNC(int) PyDescr_IsData(PyObject *); diff --git a/Include/internal/pycore_descrobject.h b/Include/internal/pycore_descrobject.h index 76378569df90e..3cec59a68a3d2 100644 --- a/Include/internal/pycore_descrobject.h +++ b/Include/internal/pycore_descrobject.h @@ -20,6 +20,8 @@ typedef struct { typedef propertyobject _PyPropertyObject; +extern PyTypeObject _PyMethodWrapper_Type; + #ifdef __cplusplus } #endif diff --git a/Objects/descrobject.c b/Objects/descrobject.c index 810bd196e8f7e..74aa70bed7de9 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -4,11 +4,11 @@ #include "pycore_abstract.h" // _PyObject_RealIsSubclass() #include "pycore_call.h" // _PyStack_AsDict() #include "pycore_ceval.h" // _Py_EnterRecursiveCallTstate() +#include "pycore_descrobject.h" // _PyMethodWrapper_Type #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_tuple.h" // _PyTuple_ITEMS() #include "structmember.h" // PyMemberDef -#include "pycore_descrobject.h" /*[clinic input] class mappingproxy "mappingproxyobject *" "&PyDictProxy_Type" diff --git a/Objects/object.c b/Objects/object.c index 93396bf5491d3..740d4a22c103b 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -5,6 +5,7 @@ #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_ceval.h" // _Py_EnterRecursiveCallTstate() #include "pycore_context.h" // _PyContextTokenMissing_Type +#include "pycore_descrobject.h" // _PyMethodWrapper_Type #include "pycore_dict.h" // _PyObject_MakeDictFromInstanceAttributes() #include "pycore_floatobject.h" // _PyFloat_DebugMallocStats() #include "pycore_initconfig.h" // _PyStatus_EXCEPTION() diff --git a/Python/specialize.c b/Python/specialize.c index dcf4be712db20..5892f5441c98e 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -1,5 +1,6 @@ #include "Python.h" #include "pycore_code.h" +#include "pycore_descrobject.h" // _PyMethodWrapper_Type #include "pycore_dict.h" #include "pycore_function.h" // _PyFunction_GetVersionForCurrentState() #include "pycore_global_strings.h" // _Py_ID() @@ -7,9 +8,8 @@ #include "pycore_moduleobject.h" #include "pycore_object.h" #include "pycore_opcode.h" // _PyOpcode_Caches -#include "structmember.h" // struct PyMemberDef, T_OFFSET_EX -#include "pycore_descrobject.h" #include "pycore_pylifecycle.h" // _PyOS_URandomNonblock() +#include "structmember.h" // struct PyMemberDef, T_OFFSET_EX #include // rand() From webhook-mailer at python.org Sat Jul 22 17:07:44 2023 From: webhook-mailer at python.org (methane) Date: Sat, 22 Jul 2023 21:07:44 -0000 Subject: [Python-checkins] gh-106527: asyncio: optimize to add/remove readers and writers (#106528) Message-ID: https://github.com/python/cpython/commit/b7dc795dfd175c0d25a479cfaf94a13c368a5a7b commit: b7dc795dfd175c0d25a479cfaf94a13c368a5a7b branch: main author: J. Nick Koston committer: methane date: 2023-07-22T21:07:40Z summary: gh-106527: asyncio: optimize to add/remove readers and writers (#106528) files: A Misc/NEWS.d/next/Library/2023-07-07-18-22-07.gh-issue-106527.spHQ0W.rst M Lib/asyncio/selector_events.py M Lib/test/test_asyncio/test_selector_events.py diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index f895750e3cf95..d521b4e2e255a 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -274,9 +274,8 @@ def _ensure_fd_no_transport(self, fd): def _add_reader(self, fd, callback, *args): self._check_closed() handle = events.Handle(callback, args, self, None) - try: - key = self._selector.get_key(fd) - except KeyError: + key = self._selector.get_map().get(fd) + if key is None: self._selector.register(fd, selectors.EVENT_READ, (handle, None)) else: @@ -290,30 +289,27 @@ def _add_reader(self, fd, callback, *args): def _remove_reader(self, fd): if self.is_closed(): return False - try: - key = self._selector.get_key(fd) - except KeyError: + key = self._selector.get_map().get(fd) + if key is None: return False + mask, (reader, writer) = key.events, key.data + mask &= ~selectors.EVENT_READ + if not mask: + self._selector.unregister(fd) else: - mask, (reader, writer) = key.events, key.data - mask &= ~selectors.EVENT_READ - if not mask: - self._selector.unregister(fd) - else: - self._selector.modify(fd, mask, (None, writer)) + self._selector.modify(fd, mask, (None, writer)) - if reader is not None: - reader.cancel() - return True - else: - return False + if reader is not None: + reader.cancel() + return True + else: + return False def _add_writer(self, fd, callback, *args): self._check_closed() handle = events.Handle(callback, args, self, None) - try: - key = self._selector.get_key(fd) - except KeyError: + key = self._selector.get_map().get(fd) + if key is None: self._selector.register(fd, selectors.EVENT_WRITE, (None, handle)) else: @@ -328,24 +324,22 @@ def _remove_writer(self, fd): """Remove a writer callback.""" if self.is_closed(): return False - try: - key = self._selector.get_key(fd) - except KeyError: + key = self._selector.get_map().get(fd) + if key is None: return False + mask, (reader, writer) = key.events, key.data + # Remove both writer and connector. + mask &= ~selectors.EVENT_WRITE + if not mask: + self._selector.unregister(fd) else: - mask, (reader, writer) = key.events, key.data - # Remove both writer and connector. - mask &= ~selectors.EVENT_WRITE - if not mask: - self._selector.unregister(fd) - else: - self._selector.modify(fd, mask, (reader, None)) + self._selector.modify(fd, mask, (reader, None)) - if writer is not None: - writer.cancel() - return True - else: - return False + if writer is not None: + writer.cancel() + return True + else: + return False def add_reader(self, fd, callback, *args): """Add a reader callback.""" diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py index 47693ea4d3ce2..c22b780b5edcb 100644 --- a/Lib/test/test_asyncio/test_selector_events.py +++ b/Lib/test/test_asyncio/test_selector_events.py @@ -178,7 +178,7 @@ def test_sock_connect_resolve_using_socket_params(self, m_gai): sock.connect.assert_called_with(('127.0.0.1', 0)) def test_add_reader(self): - self.loop._selector.get_key.side_effect = KeyError + self.loop._selector.get_map.return_value = {} cb = lambda: True self.loop.add_reader(1, cb) @@ -192,8 +192,8 @@ def test_add_reader(self): def test_add_reader_existing(self): reader = mock.Mock() writer = mock.Mock() - self.loop._selector.get_key.return_value = selectors.SelectorKey( - 1, 1, selectors.EVENT_WRITE, (reader, writer)) + self.loop._selector.get_map.return_value = {1: selectors.SelectorKey( + 1, 1, selectors.EVENT_WRITE, (reader, writer))} cb = lambda: True self.loop.add_reader(1, cb) @@ -208,8 +208,8 @@ def test_add_reader_existing(self): def test_add_reader_existing_writer(self): writer = mock.Mock() - self.loop._selector.get_key.return_value = selectors.SelectorKey( - 1, 1, selectors.EVENT_WRITE, (None, writer)) + self.loop._selector.get_map.return_value = {1: selectors.SelectorKey( + 1, 1, selectors.EVENT_WRITE, (None, writer))} cb = lambda: True self.loop.add_reader(1, cb) @@ -222,8 +222,8 @@ def test_add_reader_existing_writer(self): self.assertEqual(writer, w) def test_remove_reader(self): - self.loop._selector.get_key.return_value = selectors.SelectorKey( - 1, 1, selectors.EVENT_READ, (None, None)) + self.loop._selector.get_map.return_value = {1: selectors.SelectorKey( + 1, 1, selectors.EVENT_READ, (None, None))} self.assertFalse(self.loop.remove_reader(1)) self.assertTrue(self.loop._selector.unregister.called) @@ -231,9 +231,9 @@ def test_remove_reader(self): def test_remove_reader_read_write(self): reader = mock.Mock() writer = mock.Mock() - self.loop._selector.get_key.return_value = selectors.SelectorKey( + self.loop._selector.get_map.return_value = {1: selectors.SelectorKey( 1, 1, selectors.EVENT_READ | selectors.EVENT_WRITE, - (reader, writer)) + (reader, writer))} self.assertTrue( self.loop.remove_reader(1)) @@ -243,12 +243,12 @@ def test_remove_reader_read_write(self): self.loop._selector.modify.call_args[0]) def test_remove_reader_unknown(self): - self.loop._selector.get_key.side_effect = KeyError + self.loop._selector.get_map.return_value = {} self.assertFalse( self.loop.remove_reader(1)) def test_add_writer(self): - self.loop._selector.get_key.side_effect = KeyError + self.loop._selector.get_map.return_value = {} cb = lambda: True self.loop.add_writer(1, cb) @@ -262,8 +262,8 @@ def test_add_writer(self): def test_add_writer_existing(self): reader = mock.Mock() writer = mock.Mock() - self.loop._selector.get_key.return_value = selectors.SelectorKey( - 1, 1, selectors.EVENT_READ, (reader, writer)) + self.loop._selector.get_map.return_value = {1: selectors.SelectorKey( + 1, 1, selectors.EVENT_READ, (reader, writer))} cb = lambda: True self.loop.add_writer(1, cb) @@ -277,8 +277,8 @@ def test_add_writer_existing(self): self.assertEqual(cb, w._callback) def test_remove_writer(self): - self.loop._selector.get_key.return_value = selectors.SelectorKey( - 1, 1, selectors.EVENT_WRITE, (None, None)) + self.loop._selector.get_map.return_value = {1: selectors.SelectorKey( + 1, 1, selectors.EVENT_WRITE, (None, None))} self.assertFalse(self.loop.remove_writer(1)) self.assertTrue(self.loop._selector.unregister.called) @@ -286,9 +286,9 @@ def test_remove_writer(self): def test_remove_writer_read_write(self): reader = mock.Mock() writer = mock.Mock() - self.loop._selector.get_key.return_value = selectors.SelectorKey( + self.loop._selector.get_map.return_value = {1: selectors.SelectorKey( 1, 1, selectors.EVENT_READ | selectors.EVENT_WRITE, - (reader, writer)) + (reader, writer))} self.assertTrue( self.loop.remove_writer(1)) @@ -298,7 +298,7 @@ def test_remove_writer_read_write(self): self.loop._selector.modify.call_args[0]) def test_remove_writer_unknown(self): - self.loop._selector.get_key.side_effect = KeyError + self.loop._selector.get_map.return_value = {} self.assertFalse( self.loop.remove_writer(1)) diff --git a/Misc/NEWS.d/next/Library/2023-07-07-18-22-07.gh-issue-106527.spHQ0W.rst b/Misc/NEWS.d/next/Library/2023-07-07-18-22-07.gh-issue-106527.spHQ0W.rst new file mode 100644 index 0000000000000..204bda1c73eb3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-07-18-22-07.gh-issue-106527.spHQ0W.rst @@ -0,0 +1 @@ +Reduce overhead to add and remove :mod:`asyncio` readers and writers. From webhook-mailer at python.org Sat Jul 22 17:11:33 2023 From: webhook-mailer at python.org (vsajip) Date: Sat, 22 Jul 2023 21:11:33 -0000 Subject: [Python-checkins] [3.12] gh-107028: tiny textual changes in logging docs and docstrings (GH-107029) (GH-107065) Message-ID: https://github.com/python/cpython/commit/dd431d791f2f6bd8b5746edec01ec402fe04345d commit: dd431d791f2f6bd8b5746edec01ec402fe04345d branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vsajip date: 2023-07-22T22:11:29+01:00 summary: [3.12] gh-107028: tiny textual changes in logging docs and docstrings (GH-107029) (GH-107065) (cherry picked from commit 5e5a34ac3a827e040cd89426b1774fec2123336a) files: M Doc/library/logging.handlers.rst M Lib/logging/handlers.py diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index 47bfe4ff7f90e..72e5ffb3a1218 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -917,8 +917,9 @@ should, then :meth:`flush` is expected to do the flushing. .. method:: flush() - You can override this to implement custom flushing behavior. This version - just zaps the buffer to empty. + For a :class:`BufferingHandler` instance, flushing means that it sets the + buffer to an empty list. This method can be overwritten to implement more useful + flushing behavior. .. method:: shouldFlush(record) @@ -950,9 +951,9 @@ should, then :meth:`flush` is expected to do the flushing. .. method:: flush() - For a :class:`MemoryHandler`, flushing means just sending the buffered + For a :class:`MemoryHandler` instance, flushing means just sending the buffered records to the target, if there is one. The buffer is also cleared when - this happens. Override if you want different behavior. + buffered records are sent to the target. Override if you want different behavior. .. method:: setTarget(target) diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py index 9847104446eaf..671cc9596b02d 100644 --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -1399,7 +1399,7 @@ def flush(self): records to the target, if there is one. Override if you want different behaviour. - The record buffer is also cleared by this operation. + The record buffer is only cleared if a target has been set. """ self.acquire() try: From webhook-mailer at python.org Sat Jul 22 17:12:09 2023 From: webhook-mailer at python.org (vsajip) Date: Sat, 22 Jul 2023 21:12:09 -0000 Subject: [Python-checkins] [3.11] gh-107028: tiny textual changes in logging docs and docstrings (GH-107029) (GH-107066) Message-ID: https://github.com/python/cpython/commit/9875b177a9ef6d062eee65a501621743e443f82e commit: 9875b177a9ef6d062eee65a501621743e443f82e branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vsajip date: 2023-07-22T22:12:06+01:00 summary: [3.11] gh-107028: tiny textual changes in logging docs and docstrings (GH-107029) (GH-107066) (cherry picked from commit 5e5a34ac3a827e040cd89426b1774fec2123336a) files: M Doc/library/logging.handlers.rst M Lib/logging/handlers.py diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index 7f638251602c7..d1566957e7ed1 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -917,8 +917,9 @@ should, then :meth:`flush` is expected to do the flushing. .. method:: flush() - You can override this to implement custom flushing behavior. This version - just zaps the buffer to empty. + For a :class:`BufferingHandler` instance, flushing means that it sets the + buffer to an empty list. This method can be overwritten to implement more useful + flushing behavior. .. method:: shouldFlush(record) @@ -950,9 +951,9 @@ should, then :meth:`flush` is expected to do the flushing. .. method:: flush() - For a :class:`MemoryHandler`, flushing means just sending the buffered + For a :class:`MemoryHandler` instance, flushing means just sending the buffered records to the target, if there is one. The buffer is also cleared when - this happens. Override if you want different behavior. + buffered records are sent to the target. Override if you want different behavior. .. method:: setTarget(target) diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py index f5a9760fd496b..eadd1b6340bcb 100644 --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -1399,7 +1399,7 @@ def flush(self): records to the target, if there is one. Override if you want different behaviour. - The record buffer is also cleared by this operation. + The record buffer is only cleared if a target has been set. """ self.acquire() try: From webhook-mailer at python.org Sat Jul 22 17:15:23 2023 From: webhook-mailer at python.org (methane) Date: Sat, 22 Jul 2023 21:15:23 -0000 Subject: [Python-checkins] gh-106847: Add -X warn_default_encoding in sys.flags Doc (GH-106854) Message-ID: https://github.com/python/cpython/commit/bd907dcc50860d4730798ef48dcc30c165aa76c8 commit: bd907dcc50860d4730798ef48dcc30c165aa76c8 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: methane date: 2023-07-23T06:15:19+09:00 summary: gh-106847: Add -X warn_default_encoding in sys.flags Doc (GH-106854) gh-106847: Add -X warn_default_encoding in sys.flags Doc (GH-106854) (cherry picked from commit fd84ac0ee0a8d5e34e0a106eed7e50539b61c5f8) Co-authored-by: qqwqqw689 <114795525+qqwqqw689 at users.noreply.github.com> Co-authored-by: Nikita Sobolev files: M Doc/library/sys.rst diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index c01d2d4527152..f6b67dc57c187 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -515,27 +515,28 @@ always available. The :term:`named tuple` *flags* exposes the status of command line flags. The attributes are read only. - ============================= ============================================================================================================== - attribute flag - ============================= ============================================================================================================== - :const:`debug` :option:`-d` - :const:`inspect` :option:`-i` - :const:`interactive` :option:`-i` - :const:`isolated` :option:`-I` - :const:`optimize` :option:`-O` or :option:`-OO` - :const:`dont_write_bytecode` :option:`-B` - :const:`no_user_site` :option:`-s` - :const:`no_site` :option:`-S` - :const:`ignore_environment` :option:`-E` - :const:`verbose` :option:`-v` - :const:`bytes_warning` :option:`-b` - :const:`quiet` :option:`-q` - :const:`hash_randomization` :option:`-R` - :const:`dev_mode` :option:`-X dev <-X>` (:ref:`Python Development Mode `) - :const:`utf8_mode` :option:`-X utf8 <-X>` - :const:`safe_path` :option:`-P` - :const:`int_max_str_digits` :option:`-X int_max_str_digits <-X>` (:ref:`integer string conversion length limitation `) - ============================= ============================================================================================================== + ============================== ============================================================================================================== + attribute flag + ============================== ============================================================================================================== + :const:`debug` :option:`-d` + :const:`inspect` :option:`-i` + :const:`interactive` :option:`-i` + :const:`isolated` :option:`-I` + :const:`optimize` :option:`-O` or :option:`-OO` + :const:`dont_write_bytecode` :option:`-B` + :const:`no_user_site` :option:`-s` + :const:`no_site` :option:`-S` + :const:`ignore_environment` :option:`-E` + :const:`verbose` :option:`-v` + :const:`bytes_warning` :option:`-b` + :const:`quiet` :option:`-q` + :const:`hash_randomization` :option:`-R` + :const:`dev_mode` :option:`-X dev <-X>` (:ref:`Python Development Mode `) + :const:`utf8_mode` :option:`-X utf8 <-X>` + :const:`safe_path` :option:`-P` + :const:`int_max_str_digits` :option:`-X int_max_str_digits <-X>` (:ref:`integer string conversion length limitation `) + :const:`warn_default_encoding` :option:`-X warn_default_encoding <-X>` + ============================== ============================================================================================================== .. versionchanged:: 3.2 Added ``quiet`` attribute for the new :option:`-q` flag. @@ -554,6 +555,9 @@ always available. Mode ` and the ``utf8_mode`` attribute for the new :option:`-X` ``utf8`` flag. + .. versionchanged:: 3.10 + Added ``warn_default_encoding`` attribute for :option:`-X` ``warn_default_encoding`` flag. + .. versionchanged:: 3.11 Added the ``safe_path`` attribute for :option:`-P` option. From webhook-mailer at python.org Sat Jul 22 17:35:31 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 21:35:31 -0000 Subject: [Python-checkins] GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069) Message-ID: https://github.com/python/cpython/commit/0927a2b25c059988e237108605ed8ab0c5459c53 commit: 0927a2b25c059988e237108605ed8ab0c5459c53 branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T21:35:27Z summary: GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069) Rename private C API constants: * Rename PY_MONITORING_UNGROUPED_EVENTS to _PY_MONITORING_UNGROUPED_EVENTS * Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS files: M Include/cpython/code.h M Include/internal/pycore_interp.h M Python/ceval.c M Python/instrumentation.c M Python/pystate.c diff --git a/Include/cpython/code.h b/Include/cpython/code.h index 1b65b0d01d89f..b8953e26ecddc 100644 --- a/Include/cpython/code.h +++ b/Include/cpython/code.h @@ -10,13 +10,13 @@ extern "C" { /* Count of all "real" monitoring events (not derived from other events) */ -#define PY_MONITORING_UNGROUPED_EVENTS 14 +#define _PY_MONITORING_UNGROUPED_EVENTS 14 /* Count of all monitoring events */ -#define PY_MONITORING_EVENTS 16 +#define _PY_MONITORING_EVENTS 16 /* Table of which tools are active for each monitored event. */ typedef struct _Py_Monitors { - uint8_t tools[PY_MONITORING_UNGROUPED_EVENTS]; + uint8_t tools[_PY_MONITORING_UNGROUPED_EVENTS]; } _Py_Monitors; /* Each instruction in a code object is a fixed-width value, diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index bb37cafe6286a..8d1b8ac6671f2 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -25,7 +25,7 @@ extern "C" { #include "pycore_gc.h" // struct _gc_runtime_state #include "pycore_global_objects.h" // struct _Py_interp_static_objects #include "pycore_import.h" // struct _import_state -#include "pycore_instruments.h" // PY_MONITORING_EVENTS +#include "pycore_instruments.h" // _PY_MONITORING_EVENTS #include "pycore_list.h" // struct _Py_list_state #include "pycore_object_state.h" // struct _py_object_state #include "pycore_obmalloc.h" // struct obmalloc_state @@ -190,7 +190,7 @@ struct _is { bool sys_trace_initialized; Py_ssize_t sys_profiling_threads; /* Count of threads with c_profilefunc set */ Py_ssize_t sys_tracing_threads; /* Count of threads with c_tracefunc set */ - PyObject *monitoring_callables[PY_MONITORING_TOOL_IDS][PY_MONITORING_EVENTS]; + PyObject *monitoring_callables[PY_MONITORING_TOOL_IDS][_PY_MONITORING_EVENTS]; PyObject *monitoring_tool_names[PY_MONITORING_TOOL_IDS]; struct _Py_interp_cached_objects cached_objects; diff --git a/Python/ceval.c b/Python/ceval.c index e9f082f85bd1b..a8987efc21a98 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1914,7 +1914,7 @@ static int do_monitor_exc(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, int event) { - assert(event < PY_MONITORING_UNGROUPED_EVENTS); + assert(event < _PY_MONITORING_UNGROUPED_EVENTS); PyObject *exc = PyErr_GetRaisedException(); assert(exc != NULL); int err = _Py_call_instrumentation_arg(tstate, event, frame, instr, exc); diff --git a/Python/instrumentation.c b/Python/instrumentation.c index e29748f0ad987..e1b62bb32ec8b 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -137,7 +137,7 @@ is_instrumented(int opcode) static inline bool monitors_equals(_Py_Monitors a, _Py_Monitors b) { - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { if (a.tools[i] != b.tools[i]) { return false; } @@ -150,7 +150,7 @@ static inline _Py_Monitors monitors_sub(_Py_Monitors a, _Py_Monitors b) { _Py_Monitors res; - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { res.tools[i] = a.tools[i] & ~b.tools[i]; } return res; @@ -161,7 +161,7 @@ static inline _Py_Monitors monitors_and(_Py_Monitors a, _Py_Monitors b) { _Py_Monitors res; - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { res.tools[i] = a.tools[i] & b.tools[i]; } return res; @@ -172,7 +172,7 @@ static inline _Py_Monitors monitors_or(_Py_Monitors a, _Py_Monitors b) { _Py_Monitors res; - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { res.tools[i] = a.tools[i] | b.tools[i]; } return res; @@ -181,7 +181,7 @@ monitors_or(_Py_Monitors a, _Py_Monitors b) static inline bool monitors_are_empty(_Py_Monitors m) { - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { if (m.tools[i]) { return false; } @@ -192,7 +192,7 @@ monitors_are_empty(_Py_Monitors m) static inline bool multiple_tools(_Py_Monitors *m) { - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { if (_Py_popcount32(m->tools[i]) > 1) { return true; } @@ -204,7 +204,7 @@ static inline _PyMonitoringEventSet get_events(_Py_Monitors *m, int tool_id) { _PyMonitoringEventSet result = 0; - for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_UNGROUPED_EVENTS; e++) { if ((m->tools[e] >> tool_id) & 1) { result |= (1 << e); } @@ -339,7 +339,7 @@ static void dump_monitors(const char *prefix, _Py_Monitors monitors, FILE*out) { fprintf(out, "%s monitors:\n", prefix); - for (int event = 0; event < PY_MONITORING_UNGROUPED_EVENTS; event++) { + for (int event = 0; event < _PY_MONITORING_UNGROUPED_EVENTS; event++) { fprintf(out, " Event %d: Tools %x\n", event, monitors.tools[event]); } } @@ -907,7 +907,7 @@ get_tools_for_instruction(PyCodeObject *code, PyInterpreterState *interp, int i, uint8_t tools; assert(event != PY_MONITORING_EVENT_LINE); assert(event != PY_MONITORING_EVENT_INSTRUCTION); - if (event >= PY_MONITORING_UNGROUPED_EVENTS) { + if (event >= _PY_MONITORING_UNGROUPED_EVENTS) { assert(event == PY_MONITORING_EVENT_C_RAISE || event == PY_MONITORING_EVENT_C_RETURN); event = PY_MONITORING_EVENT_CALL; @@ -1220,7 +1220,7 @@ _PyMonitoring_RegisterCallback(int tool_id, int event_id, PyObject *obj) { PyInterpreterState *is = _PyInterpreterState_GET(); assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); - assert(0 <= event_id && event_id < PY_MONITORING_EVENTS); + assert(0 <= event_id && event_id < _PY_MONITORING_EVENTS); PyObject *callback = is->monitoring_callables[tool_id][event_id]; is->monitoring_callables[tool_id][event_id] = Py_XNewRef(obj); return callback; @@ -1653,7 +1653,7 @@ static void set_events(_Py_Monitors *m, int tool_id, _PyMonitoringEventSet events) { assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); - for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_UNGROUPED_EVENTS; e++) { uint8_t *tools = &m->tools[e]; int val = (events >> e) & 1; *tools &= ~(1 << tool_id); @@ -1678,7 +1678,7 @@ _PyMonitoring_SetEvents(int tool_id, _PyMonitoringEventSet events) { assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); PyInterpreterState *interp = _PyInterpreterState_GET(); - assert(events < (1 << PY_MONITORING_UNGROUPED_EVENTS)); + assert(events < (1 << _PY_MONITORING_UNGROUPED_EVENTS)); if (check_tool(interp, tool_id)) { return -1; } @@ -1696,7 +1696,7 @@ _PyMonitoring_SetLocalEvents(PyCodeObject *code, int tool_id, _PyMonitoringEvent { assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); PyInterpreterState *interp = _PyInterpreterState_GET(); - assert(events < (1 << PY_MONITORING_UNGROUPED_EVENTS)); + assert(events < (1 << _PY_MONITORING_UNGROUPED_EVENTS)); if (check_tool(interp, tool_id)) { return -1; } @@ -1835,7 +1835,7 @@ monitoring_register_callback_impl(PyObject *module, int tool_id, int event, return NULL; } int event_id = _Py_bit_length(event)-1; - if (event_id < 0 || event_id >= PY_MONITORING_EVENTS) { + if (event_id < 0 || event_id >= _PY_MONITORING_EVENTS) { PyErr_Format(PyExc_ValueError, "invalid event %d", event); return NULL; } @@ -1885,7 +1885,7 @@ monitoring_set_events_impl(PyObject *module, int tool_id, int event_set) if (check_valid_tool(tool_id)) { return NULL; } - if (event_set < 0 || event_set >= (1 << PY_MONITORING_EVENTS)) { + if (event_set < 0 || event_set >= (1 << _PY_MONITORING_EVENTS)) { PyErr_Format(PyExc_ValueError, "invalid event set 0x%x", event_set); return NULL; } @@ -1927,7 +1927,7 @@ monitoring_get_local_events_impl(PyObject *module, int tool_id, _PyMonitoringEventSet event_set = 0; _PyCoMonitoringData *data = ((PyCodeObject *)code)->_co_monitoring; if (data != NULL) { - for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_UNGROUPED_EVENTS; e++) { if ((data->local_monitors.tools[e] >> tool_id) & 1) { event_set |= (1 << e); } @@ -1961,7 +1961,7 @@ monitoring_set_local_events_impl(PyObject *module, int tool_id, if (check_valid_tool(tool_id)) { return NULL; } - if (event_set < 0 || event_set >= (1 << PY_MONITORING_EVENTS)) { + if (event_set < 0 || event_set >= (1 << _PY_MONITORING_EVENTS)) { PyErr_Format(PyExc_ValueError, "invalid event set 0x%x", event_set); return NULL; } @@ -2042,7 +2042,7 @@ monitoring__all_events_impl(PyObject *module) if (res == NULL) { return NULL; } - for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_UNGROUPED_EVENTS; e++) { uint8_t tools = interp->monitors.tools[e]; if (tools == 0) { continue; @@ -2101,7 +2101,7 @@ PyObject *_Py_CreateMonitoringObject(void) if (err) { goto error; } - for (int i = 0; i < PY_MONITORING_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_EVENTS; i++) { if (add_power2_constant(events, event_names[i], i)) { goto error; } diff --git a/Python/pystate.c b/Python/pystate.c index a9b404bd5c93e..cdd975f3672b9 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -679,11 +679,11 @@ init_interpreter(PyInterpreterState *interp, _PyGC_InitState(&interp->gc); PyConfig_InitPythonConfig(&interp->config); _PyType_InitCache(interp); - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { interp->monitors.tools[i] = 0; } for (int t = 0; t < PY_MONITORING_TOOL_IDS; t++) { - for (int e = 0; e < PY_MONITORING_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_EVENTS; e++) { interp->monitoring_callables[t][e] = NULL; } @@ -841,11 +841,11 @@ interpreter_clear(PyInterpreterState *interp, PyThreadState *tstate) Py_CLEAR(interp->audit_hooks); - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { interp->monitors.tools[i] = 0; } for (int t = 0; t < PY_MONITORING_TOOL_IDS; t++) { - for (int e = 0; e < PY_MONITORING_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_EVENTS; e++) { Py_CLEAR(interp->monitoring_callables[t][e]); } } From webhook-mailer at python.org Sat Jul 22 17:41:14 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 21:41:14 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyModule API (#107070) Message-ID: https://github.com/python/cpython/commit/c1331ad50891b1727896afe62258bda73a459d40 commit: c1331ad50891b1727896afe62258bda73a459d40 branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T21:41:11Z summary: gh-106320: Remove private _PyModule API (#107070) Move private _PyModule API to the internal C API (pycore_moduleobject.h): * _PyModule_Clear() * _PyModule_ClearDict() * _PyModuleSpec_IsInitializing() * _PyModule_IsExtension() No longer export these functions. files: M Include/internal/pycore_moduleobject.h M Include/moduleobject.h diff --git a/Include/internal/pycore_moduleobject.h b/Include/internal/pycore_moduleobject.h index 31a31e724d0b2..5644bbe5e0552 100644 --- a/Include/internal/pycore_moduleobject.h +++ b/Include/internal/pycore_moduleobject.h @@ -8,6 +8,12 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +extern void _PyModule_Clear(PyObject *); +extern void _PyModule_ClearDict(PyObject *); +extern int _PyModuleSpec_IsInitializing(PyObject *); + +extern int _PyModule_IsExtension(PyObject *obj); + typedef struct { PyObject_HEAD PyObject *md_dict; diff --git a/Include/moduleobject.h b/Include/moduleobject.h index b8bdfe29d8040..ea08145381cee 100644 --- a/Include/moduleobject.h +++ b/Include/moduleobject.h @@ -27,11 +27,6 @@ PyAPI_FUNC(PyObject *) PyModule_GetNameObject(PyObject *); PyAPI_FUNC(const char *) PyModule_GetName(PyObject *); Py_DEPRECATED(3.2) PyAPI_FUNC(const char *) PyModule_GetFilename(PyObject *); PyAPI_FUNC(PyObject *) PyModule_GetFilenameObject(PyObject *); -#ifndef Py_LIMITED_API -PyAPI_FUNC(void) _PyModule_Clear(PyObject *); -PyAPI_FUNC(void) _PyModule_ClearDict(PyObject *); -PyAPI_FUNC(int) _PyModuleSpec_IsInitializing(PyObject *); -#endif PyAPI_FUNC(PyModuleDef*) PyModule_GetDef(PyObject*); PyAPI_FUNC(void*) PyModule_GetState(PyObject*); @@ -103,12 +98,6 @@ struct PyModuleDef { freefunc m_free; }; - -// Internal C API -#ifdef Py_BUILD_CORE -extern int _PyModule_IsExtension(PyObject *obj); -#endif - #ifdef __cplusplus } #endif From webhook-mailer at python.org Sat Jul 22 17:44:37 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 21:44:37 -0000 Subject: [Python-checkins] gh-106320: Remove _PyFunction_Vectorcall() API (#107071) Message-ID: https://github.com/python/cpython/commit/889851ecc355cad3d79f5692a6dbd3ae3127647b commit: 889851ecc355cad3d79f5692a6dbd3ae3127647b branch: main author: Victor Stinner committer: vstinner date: 2023-07-22T21:44:33Z summary: gh-106320: Remove _PyFunction_Vectorcall() API (#107071) Move _PyFunction_Vectorcall() API to the internal C API. No longer export the function. files: M Include/cpython/funcobject.h M Include/internal/pycore_function.h M Objects/call.c diff --git a/Include/cpython/funcobject.h b/Include/cpython/funcobject.h index 6f78f5868d016..de2013323d2c7 100644 --- a/Include/cpython/funcobject.h +++ b/Include/cpython/funcobject.h @@ -79,12 +79,6 @@ PyAPI_FUNC(int) PyFunction_SetClosure(PyObject *, PyObject *); PyAPI_FUNC(PyObject *) PyFunction_GetAnnotations(PyObject *); PyAPI_FUNC(int) PyFunction_SetAnnotations(PyObject *, PyObject *); -PyAPI_FUNC(PyObject *) _PyFunction_Vectorcall( - PyObject *func, - PyObject *const *stack, - size_t nargsf, - PyObject *kwnames); - #define _PyFunction_CAST(func) \ (assert(PyFunction_Check(func)), _Py_CAST(PyFunctionObject*, func)) diff --git a/Include/internal/pycore_function.h b/Include/internal/pycore_function.h index ecbb7001e7d84..e844d323ec792 100644 --- a/Include/internal/pycore_function.h +++ b/Include/internal/pycore_function.h @@ -8,6 +8,12 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +extern PyObject* _PyFunction_Vectorcall( + PyObject *func, + PyObject *const *stack, + size_t nargsf, + PyObject *kwnames); + #define FUNC_MAX_WATCHERS 8 struct _py_func_state { diff --git a/Objects/call.c b/Objects/call.c index 396552d85cfca..b1610dababd46 100644 --- a/Objects/call.c +++ b/Objects/call.c @@ -2,6 +2,7 @@ #include "pycore_call.h" // _PyObject_CallNoArgsTstate() #include "pycore_ceval.h" // _Py_EnterRecursiveCallTstate() #include "pycore_dict.h" // _PyDict_FromItems() +#include "pycore_function.h" // _PyFunction_Vectorcall() definition #include "pycore_modsupport.h" // _Py_VaBuildStack() #include "pycore_object.h" // _PyCFunctionWithKeywords_TrampolineCall() #include "pycore_pyerrors.h" // _PyErr_Occurred() From webhook-mailer at python.org Sat Jul 22 18:20:42 2023 From: webhook-mailer at python.org (vstinner) Date: Sat, 22 Jul 2023 22:20:42 -0000 Subject: [Python-checkins] [3.12] GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069) (#107075) Message-ID: https://github.com/python/cpython/commit/0d4a76654f755d091c14cfa1e57bcfa6333ae05a commit: 0d4a76654f755d091c14cfa1e57bcfa6333ae05a branch: 3.12 author: Victor Stinner committer: vstinner date: 2023-07-22T22:20:38Z summary: [3.12] GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069) (#107075) GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069) Rename private C API constants: * Rename PY_MONITORING_UNGROUPED_EVENTS to _PY_MONITORING_UNGROUPED_EVENTS * Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (cherry picked from commit 0927a2b25c059988e237108605ed8ab0c5459c53) files: M Include/cpython/code.h M Include/internal/pycore_interp.h M Python/ceval.c M Python/instrumentation.c M Python/pystate.c diff --git a/Include/cpython/code.h b/Include/cpython/code.h index 6bead361c7924..7449a98e4326e 100644 --- a/Include/cpython/code.h +++ b/Include/cpython/code.h @@ -10,13 +10,13 @@ extern "C" { /* Count of all "real" monitoring events (not derived from other events) */ -#define PY_MONITORING_UNGROUPED_EVENTS 14 +#define _PY_MONITORING_UNGROUPED_EVENTS 14 /* Count of all monitoring events */ -#define PY_MONITORING_EVENTS 16 +#define _PY_MONITORING_EVENTS 16 /* Table of which tools are active for each monitored event. */ typedef struct _Py_Monitors { - uint8_t tools[PY_MONITORING_UNGROUPED_EVENTS]; + uint8_t tools[_PY_MONITORING_UNGROUPED_EVENTS]; } _Py_Monitors; /* Each instruction in a code object is a fixed-width value, diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index 619225c939664..6109e93409898 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -25,7 +25,7 @@ extern "C" { #include "pycore_gc.h" // struct _gc_runtime_state #include "pycore_global_objects.h" // struct _Py_interp_static_objects #include "pycore_import.h" // struct _import_state -#include "pycore_instruments.h" // PY_MONITORING_EVENTS +#include "pycore_instruments.h" // _PY_MONITORING_EVENTS #include "pycore_list.h" // struct _Py_list_state #include "pycore_object_state.h" // struct _py_object_state #include "pycore_obmalloc.h" // struct obmalloc_state @@ -188,7 +188,7 @@ struct _is { bool sys_trace_initialized; Py_ssize_t sys_profiling_threads; /* Count of threads with c_profilefunc set */ Py_ssize_t sys_tracing_threads; /* Count of threads with c_tracefunc set */ - PyObject *monitoring_callables[PY_MONITORING_TOOL_IDS][PY_MONITORING_EVENTS]; + PyObject *monitoring_callables[PY_MONITORING_TOOL_IDS][_PY_MONITORING_EVENTS]; PyObject *monitoring_tool_names[PY_MONITORING_TOOL_IDS]; struct _Py_interp_cached_objects cached_objects; diff --git a/Python/ceval.c b/Python/ceval.c index c883a903e964f..72d679a4e3f53 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1997,7 +1997,7 @@ static int do_monitor_exc(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, int event) { - assert(event < PY_MONITORING_UNGROUPED_EVENTS); + assert(event < _PY_MONITORING_UNGROUPED_EVENTS); PyObject *exc = PyErr_GetRaisedException(); assert(exc != NULL); int err = _Py_call_instrumentation_arg(tstate, event, frame, instr, exc); diff --git a/Python/instrumentation.c b/Python/instrumentation.c index c70668e8295ae..2fd3344b555a9 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -138,7 +138,7 @@ is_instrumented(int opcode) static inline bool monitors_equals(_Py_Monitors a, _Py_Monitors b) { - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { if (a.tools[i] != b.tools[i]) { return false; } @@ -151,7 +151,7 @@ static inline _Py_Monitors monitors_sub(_Py_Monitors a, _Py_Monitors b) { _Py_Monitors res; - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { res.tools[i] = a.tools[i] & ~b.tools[i]; } return res; @@ -162,7 +162,7 @@ static inline _Py_Monitors monitors_and(_Py_Monitors a, _Py_Monitors b) { _Py_Monitors res; - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { res.tools[i] = a.tools[i] & b.tools[i]; } return res; @@ -173,7 +173,7 @@ static inline _Py_Monitors monitors_or(_Py_Monitors a, _Py_Monitors b) { _Py_Monitors res; - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { res.tools[i] = a.tools[i] | b.tools[i]; } return res; @@ -182,7 +182,7 @@ monitors_or(_Py_Monitors a, _Py_Monitors b) static inline bool monitors_are_empty(_Py_Monitors m) { - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { if (m.tools[i]) { return false; } @@ -193,7 +193,7 @@ monitors_are_empty(_Py_Monitors m) static inline bool multiple_tools(_Py_Monitors *m) { - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { if (_Py_popcount32(m->tools[i]) > 1) { return true; } @@ -205,7 +205,7 @@ static inline _PyMonitoringEventSet get_events(_Py_Monitors *m, int tool_id) { _PyMonitoringEventSet result = 0; - for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_UNGROUPED_EVENTS; e++) { if ((m->tools[e] >> tool_id) & 1) { result |= (1 << e); } @@ -340,7 +340,7 @@ static void dump_monitors(const char *prefix, _Py_Monitors monitors, FILE*out) { fprintf(out, "%s monitors:\n", prefix); - for (int event = 0; event < PY_MONITORING_UNGROUPED_EVENTS; event++) { + for (int event = 0; event < _PY_MONITORING_UNGROUPED_EVENTS; event++) { fprintf(out, " Event %d: Tools %x\n", event, monitors.tools[event]); } } @@ -910,7 +910,7 @@ get_tools_for_instruction(PyCodeObject * code, int i, int event) assert(event != PY_MONITORING_EVENT_INSTRUCTION); assert(instrumentation_cross_checks(PyThreadState_GET()->interp, code)); _PyCoMonitoringData *monitoring = code->_co_monitoring; - if (event >= PY_MONITORING_UNGROUPED_EVENTS) { + if (event >= _PY_MONITORING_UNGROUPED_EVENTS) { assert(event == PY_MONITORING_EVENT_C_RAISE || event == PY_MONITORING_EVENT_C_RETURN); event = PY_MONITORING_EVENT_CALL; @@ -1216,7 +1216,7 @@ _PyMonitoring_RegisterCallback(int tool_id, int event_id, PyObject *obj) { PyInterpreterState *is = _PyInterpreterState_Get(); assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); - assert(0 <= event_id && event_id < PY_MONITORING_EVENTS); + assert(0 <= event_id && event_id < _PY_MONITORING_EVENTS); PyObject *callback = is->monitoring_callables[tool_id][event_id]; is->monitoring_callables[tool_id][event_id] = Py_XNewRef(obj); return callback; @@ -1667,7 +1667,7 @@ static void set_events(_Py_Monitors *m, int tool_id, _PyMonitoringEventSet events) { assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); - for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_UNGROUPED_EVENTS; e++) { uint8_t *tools = &m->tools[e]; int val = (events >> e) & 1; *tools &= ~(1 << tool_id); @@ -1692,7 +1692,7 @@ _PyMonitoring_SetEvents(int tool_id, _PyMonitoringEventSet events) { assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); PyInterpreterState *interp = _PyInterpreterState_Get(); - assert(events < (1 << PY_MONITORING_UNGROUPED_EVENTS)); + assert(events < (1 << _PY_MONITORING_UNGROUPED_EVENTS)); if (check_tool(interp, tool_id)) { return -1; } @@ -1710,7 +1710,7 @@ _PyMonitoring_SetLocalEvents(PyCodeObject *code, int tool_id, _PyMonitoringEvent { assert(0 <= tool_id && tool_id < PY_MONITORING_TOOL_IDS); PyInterpreterState *interp = _PyInterpreterState_Get(); - assert(events < (1 << PY_MONITORING_UNGROUPED_EVENTS)); + assert(events < (1 << _PY_MONITORING_UNGROUPED_EVENTS)); if (check_tool(interp, tool_id)) { return -1; } @@ -1849,7 +1849,7 @@ monitoring_register_callback_impl(PyObject *module, int tool_id, int event, return NULL; } int event_id = _Py_bit_length(event)-1; - if (event_id < 0 || event_id >= PY_MONITORING_EVENTS) { + if (event_id < 0 || event_id >= _PY_MONITORING_EVENTS) { PyErr_Format(PyExc_ValueError, "invalid event %d", event); return NULL; } @@ -1899,7 +1899,7 @@ monitoring_set_events_impl(PyObject *module, int tool_id, int event_set) if (check_valid_tool(tool_id)) { return NULL; } - if (event_set < 0 || event_set >= (1 << PY_MONITORING_EVENTS)) { + if (event_set < 0 || event_set >= (1 << _PY_MONITORING_EVENTS)) { PyErr_Format(PyExc_ValueError, "invalid event set 0x%x", event_set); return NULL; } @@ -1941,7 +1941,7 @@ monitoring_get_local_events_impl(PyObject *module, int tool_id, _PyMonitoringEventSet event_set = 0; _PyCoMonitoringData *data = ((PyCodeObject *)code)->_co_monitoring; if (data != NULL) { - for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_UNGROUPED_EVENTS; e++) { if ((data->local_monitors.tools[e] >> tool_id) & 1) { event_set |= (1 << e); } @@ -1975,7 +1975,7 @@ monitoring_set_local_events_impl(PyObject *module, int tool_id, if (check_valid_tool(tool_id)) { return NULL; } - if (event_set < 0 || event_set >= (1 << PY_MONITORING_EVENTS)) { + if (event_set < 0 || event_set >= (1 << _PY_MONITORING_EVENTS)) { PyErr_Format(PyExc_ValueError, "invalid event set 0x%x", event_set); return NULL; } @@ -2056,7 +2056,7 @@ monitoring__all_events_impl(PyObject *module) if (res == NULL) { return NULL; } - for (int e = 0; e < PY_MONITORING_UNGROUPED_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_UNGROUPED_EVENTS; e++) { uint8_t tools = interp->monitors.tools[e]; if (tools == 0) { continue; @@ -2115,7 +2115,7 @@ PyObject *_Py_CreateMonitoringObject(void) if (err) { goto error; } - for (int i = 0; i < PY_MONITORING_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_EVENTS; i++) { if (add_power2_constant(events, event_names[i], i)) { goto error; } diff --git a/Python/pystate.c b/Python/pystate.c index da0e1c42d7275..e1261bf2acf95 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -678,11 +678,11 @@ init_interpreter(PyInterpreterState *interp, _PyGC_InitState(&interp->gc); PyConfig_InitPythonConfig(&interp->config); _PyType_InitCache(interp); - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { interp->monitors.tools[i] = 0; } for (int t = 0; t < PY_MONITORING_TOOL_IDS; t++) { - for (int e = 0; e < PY_MONITORING_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_EVENTS; e++) { interp->monitoring_callables[t][e] = NULL; } @@ -832,11 +832,11 @@ interpreter_clear(PyInterpreterState *interp, PyThreadState *tstate) Py_CLEAR(interp->audit_hooks); - for (int i = 0; i < PY_MONITORING_UNGROUPED_EVENTS; i++) { + for (int i = 0; i < _PY_MONITORING_UNGROUPED_EVENTS; i++) { interp->monitors.tools[i] = 0; } for (int t = 0; t < PY_MONITORING_TOOL_IDS; t++) { - for (int e = 0; e < PY_MONITORING_EVENTS; e++) { + for (int e = 0; e < _PY_MONITORING_EVENTS; e++) { Py_CLEAR(interp->monitoring_callables[t][e]); } } From webhook-mailer at python.org Sat Jul 22 18:30:46 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 22 Jul 2023 22:30:46 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Increase typing coverage (#107074) Message-ID: https://github.com/python/cpython/commit/3aeffc0d8f28655186f99d013ee9653c65b92f84 commit: 3aeffc0d8f28655186f99d013ee9653c65b92f84 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-22T22:30:42Z summary: gh-104050: Argument Clinic: Increase typing coverage (#107074) Co-authored-by: Alex Waygood files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 7a4c9c4cacf55..eecb81dcad588 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2950,12 +2950,18 @@ def parse_argument(self, args: list[str]) -> None: # All the functions after here are intended as extension points. # - def simple_declaration(self, by_reference=False, *, in_parser=False): + def simple_declaration( + self, + by_reference: bool = False, + *, + in_parser: bool = False + ) -> str: """ Computes the basic declaration of the variable. Used in computing the prototype declaration and the variable declaration. """ + assert isinstance(self.type, str) prototype = [self.type] if by_reference or not self.type.endswith('*'): prototype.append(" ") @@ -2970,7 +2976,7 @@ def simple_declaration(self, by_reference=False, *, in_parser=False): prototype.append(name) return "".join(prototype) - def declaration(self, *, in_parser=False) -> str: + def declaration(self, *, in_parser: bool = False) -> str: """ The C statement to declare this variable. """ @@ -3579,9 +3585,9 @@ class object_converter(CConverter): def converter_init( self, *, - converter=None, - type=None, - subclass_of=None + converter: str | None = None, + type: str | None = None, + subclass_of: str | None = None ) -> None: if converter: if subclass_of: @@ -3973,7 +3979,7 @@ class self_converter(CConverter): type = None format_unit = '' - def converter_init(self, *, type=None) -> None: + def converter_init(self, *, type: str | None = None) -> None: self.specified_type = type def pre_render(self): @@ -4047,7 +4053,7 @@ def render(self, parameter, data): assert data.impl_arguments[0] == self.name data.impl_arguments[0] = '(' + self.type + ")" + data.impl_arguments[0] - def set_template_dict(self, template_dict): + def set_template_dict(self, template_dict: TemplateDict) -> None: template_dict['self_name'] = self.name template_dict['self_type'] = self.parser_type kind = self.function.kind @@ -4066,7 +4072,7 @@ def set_template_dict(self, template_dict): line = f'{type_check} &&\n ' template_dict['self_type_check'] = line - type_object = self.function.cls.type_object + type_object = cls.type_object type_ptr = f'PyTypeObject *base_tp = {type_object};' template_dict['base_type_ptr'] = type_ptr @@ -4276,11 +4282,11 @@ def eval_ast_expr( class IndentStack: - def __init__(self): - self.indents = [] - self.margin = None + def __init__(self) -> None: + self.indents: list[int] = [] + self.margin: str | None = None - def _ensure(self): + def _ensure(self) -> None: if not self.indents: fail('IndentStack expected indents, but none are defined.') @@ -4341,6 +4347,7 @@ def indent(self, line: str) -> str: """ Indents a line by the currently defined margin. """ + assert self.margin is not None, "Cannot call .indent() before calling .infer()" return self.margin + line def dedent(self, line: str) -> str: @@ -4348,6 +4355,7 @@ def dedent(self, line: str) -> str: Dedents a line by the currently defined margin. (The inverse of 'indent'.) """ + assert self.margin is not None, "Cannot call .indent() before calling .infer()" margin = self.margin indent = self.indents[-1] if not line.startswith(margin): @@ -5232,6 +5240,7 @@ def parse_slash(self, function: Function) -> None: p.kind = inspect.Parameter.POSITIONAL_ONLY def state_parameter_docstring_start(self, line: str | None) -> None: + assert self.indent.margin is not None, "self.margin.infer() has not yet been called to set the margin" self.parameter_docstring_indent = len(self.indent.margin) assert self.indent.depth == 3 return self.next(self.state_parameter_docstring, line) @@ -5534,7 +5543,7 @@ def add_parameter(text): return docstring - def state_terminal(self, line): + def state_terminal(self, line: str | None) -> None: """ Called when processing the block is done. """ From webhook-mailer at python.org Sat Jul 22 19:20:07 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sat, 22 Jul 2023 23:20:07 -0000 Subject: [Python-checkins] gh-106962: Detect mpicc in configure.ac (#106961) Message-ID: https://github.com/python/cpython/commit/9a6b278769b9f24e0650283f6c347db8ae52b7b3 commit: 9a6b278769b9f24e0650283f6c347db8ae52b7b3 branch: main author: Lukas van de Wiel <30800501+LukasvdWiel at users.noreply.github.com> committer: erlend-aasland date: 2023-07-22T23:20:03Z summary: gh-106962: Detect mpicc in configure.ac (#106961) Don't let autoconf mistake MPI compilers for Intel compilers; filter out the MPI case to prevent Intel specific options from being applied. files: A Misc/NEWS.d/next/Build/2023-07-23-00-38-51.gh-issue-106962.VVYrWB.rst M configure M configure.ac diff --git a/Misc/NEWS.d/next/Build/2023-07-23-00-38-51.gh-issue-106962.VVYrWB.rst b/Misc/NEWS.d/next/Build/2023-07-23-00-38-51.gh-issue-106962.VVYrWB.rst new file mode 100644 index 0000000000000..32e196fe26d3b --- /dev/null +++ b/Misc/NEWS.d/next/Build/2023-07-23-00-38-51.gh-issue-106962.VVYrWB.rst @@ -0,0 +1 @@ +Detect MPI compilers in :file:`configure`. diff --git a/configure b/configure index e6fb5e3c2b0c2..18d404c8dc060 100755 --- a/configure +++ b/configure @@ -10154,6 +10154,9 @@ rm -f core conftest.err conftest.$ac_objext conftest.beam \ esac case "$CC" in +*mpicc*) + CFLAGS_NODIST="$CFLAGS_NODIST" + ;; *icc*) # ICC needs -fp-model strict or floats behave badly CFLAGS_NODIST="$CFLAGS_NODIST -fp-model strict" diff --git a/configure.ac b/configure.ac index a1ee78047692f..cdb88a3071976 100644 --- a/configure.ac +++ b/configure.ac @@ -2656,6 +2656,9 @@ yes) esac case "$CC" in +*mpicc*) + CFLAGS_NODIST="$CFLAGS_NODIST" + ;; *icc*) # ICC needs -fp-model strict or floats behave badly CFLAGS_NODIST="$CFLAGS_NODIST -fp-model strict" From webhook-mailer at python.org Sat Jul 22 20:01:48 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sun, 23 Jul 2023 00:01:48 -0000 Subject: [Python-checkins] [3.12] gh-106962: Detect mpicc in configure.ac (GH-106961) (#107081) Message-ID: https://github.com/python/cpython/commit/d87d67b9e65c38dd6d1d378a5c013eccf50756f1 commit: d87d67b9e65c38dd6d1d378a5c013eccf50756f1 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-23T00:01:44Z summary: [3.12] gh-106962: Detect mpicc in configure.ac (GH-106961) (#107081) Don't let autoconf mistake MPI compilers for Intel compilers; filter out the MPI case to prevent Intel specific options from being applied. (cherry picked from commit 9a6b278769b9f24e0650283f6c347db8ae52b7b3) Co-authored-by: Lukas van de Wiel <30800501+LukasvdWiel at users.noreply.github.com> files: A Misc/NEWS.d/next/Build/2023-07-23-00-38-51.gh-issue-106962.VVYrWB.rst M configure M configure.ac diff --git a/Misc/NEWS.d/next/Build/2023-07-23-00-38-51.gh-issue-106962.VVYrWB.rst b/Misc/NEWS.d/next/Build/2023-07-23-00-38-51.gh-issue-106962.VVYrWB.rst new file mode 100644 index 0000000000000..32e196fe26d3b --- /dev/null +++ b/Misc/NEWS.d/next/Build/2023-07-23-00-38-51.gh-issue-106962.VVYrWB.rst @@ -0,0 +1 @@ +Detect MPI compilers in :file:`configure`. diff --git a/configure b/configure index 91bf1d2c02370..3806da7723fda 100755 --- a/configure +++ b/configure @@ -10179,6 +10179,9 @@ rm -f core conftest.err conftest.$ac_objext conftest.beam \ esac case "$CC" in +*mpicc*) + CFLAGS_NODIST="$CFLAGS_NODIST" + ;; *icc*) # ICC needs -fp-model strict or floats behave badly CFLAGS_NODIST="$CFLAGS_NODIST -fp-model strict" diff --git a/configure.ac b/configure.ac index 0770f68a23f61..630db4bfbd53c 100644 --- a/configure.ac +++ b/configure.ac @@ -2656,6 +2656,9 @@ yes) esac case "$CC" in +*mpicc*) + CFLAGS_NODIST="$CFLAGS_NODIST" + ;; *icc*) # ICC needs -fp-model strict or floats behave badly CFLAGS_NODIST="$CFLAGS_NODIST -fp-model strict" From webhook-mailer at python.org Sat Jul 22 21:22:37 2023 From: webhook-mailer at python.org (gvanrossum) Date: Sun, 23 Jul 2023 01:22:37 -0000 Subject: [Python-checkins] gh-107082: Remove debug hack from get_first_executor in test_capi.test_misc (#107085) Message-ID: https://github.com/python/cpython/commit/7fc9be350af055538e70ece8d7de78414bad431e commit: 7fc9be350af055538e70ece8d7de78414bad431e branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-22T18:22:33-07:00 summary: gh-107082: Remove debug hack from get_first_executor in test_capi.test_misc (#107085) (Even though it doesn't look like it fixes gh-107082 -- see discussion there -- it still removes debug code that should never have been committed.) files: M Lib/test/test_capi/test_misc.py diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index dc155cc99961c..0d3f93941e820 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -2436,7 +2436,7 @@ def get_first_executor(func): co_code = code.co_code JUMP_BACKWARD = opcode.opmap["JUMP_BACKWARD"] for i in range(0, len(co_code), 2): - if co_code[i] == JUMP_BACKWARD or 1: + if co_code[i] == JUMP_BACKWARD: try: return _testinternalcapi.get_executor(code, i) except ValueError: From webhook-mailer at python.org Sun Jul 23 00:07:18 2023 From: webhook-mailer at python.org (gvanrossum) Date: Sun, 23 Jul 2023 04:07:18 -0000 Subject: [Python-checkins] gh-82500: Fix asyncio sendfile overflow on 32bit (#107056) Message-ID: https://github.com/python/cpython/commit/9eeb4b485f4f9e13e5cb638e43a0baecb3ff4e86 commit: 9eeb4b485f4f9e13e5cb638e43a0baecb3ff4e86 branch: main author: J. Nick Koston committer: gvanrossum date: 2023-07-22T21:07:14-07:00 summary: gh-82500: Fix asyncio sendfile overflow on 32bit (#107056) files: A Misc/NEWS.d/next/Library/2023-07-22-16-44-58.gh-issue-82500.cQYoPj.rst M Lib/asyncio/unix_events.py diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py index 17fb4d5f7646c..a2680865ed968 100644 --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -394,6 +394,9 @@ def _sock_sendfile_native_impl(self, fut, registered_fd, sock, fileno, fut.set_result(total_sent) return + # On 32-bit architectures truncate to 1GiB to avoid OverflowError + blocksize = min(blocksize, sys.maxsize//2 + 1) + try: sent = os.sendfile(fd, fileno, offset, blocksize) except (BlockingIOError, InterruptedError): diff --git a/Misc/NEWS.d/next/Library/2023-07-22-16-44-58.gh-issue-82500.cQYoPj.rst b/Misc/NEWS.d/next/Library/2023-07-22-16-44-58.gh-issue-82500.cQYoPj.rst new file mode 100644 index 0000000000000..065394fd6ee71 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-16-44-58.gh-issue-82500.cQYoPj.rst @@ -0,0 +1 @@ +Fix overflow on 32-bit systems with :mod:`asyncio` :func:`os.sendfile` implemention. From webhook-mailer at python.org Sun Jul 23 04:50:41 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 08:50:41 -0000 Subject: [Python-checkins] gh-102111: Add link to string escape sequences in re module (#106995) Message-ID: https://github.com/python/cpython/commit/0af247da0932692592ed85ba8b4a1520627ab4ac commit: 0af247da0932692592ed85ba8b4a1520627ab4ac branch: main author: wulmer committer: hugovk date: 2023-07-23T02:50:38-06:00 summary: gh-102111: Add link to string escape sequences in re module (#106995) Co-authored-by: Alex Waygood files: M Doc/library/re.rst M Doc/reference/lexical_analysis.rst diff --git a/Doc/library/re.rst b/Doc/library/re.rst index 629ee472cca68..3f03f0341d816 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -634,8 +634,8 @@ character ``'$'``. single: \x; in regular expressions single: \\; in regular expressions -Most of the standard escapes supported by Python string literals are also -accepted by the regular expression parser:: +Most of the :ref:`escape sequences ` supported by Python +string literals are also accepted by the regular expression parser:: \a \b \f \n \N \r \t \u diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 47062f86810e9..dde7ba1d941dc 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -549,6 +549,10 @@ retained), except that three unescaped quotes in a row terminate the literal. ( .. _escape-sequences: + +Escape sequences +^^^^^^^^^^^^^^^^ + Unless an ``'r'`` or ``'R'`` prefix is present, escape sequences in string and bytes literals are interpreted according to rules similar to those used by Standard C. The recognized escape sequences are: From webhook-mailer at python.org Sun Jul 23 04:57:56 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 08:57:56 -0000 Subject: [Python-checkins] gh-107017: removed mention that C does it the same way (#107020) Message-ID: https://github.com/python/cpython/commit/9629d4442e24681e79c1ba93cb92ed2e3b967224 commit: 9629d4442e24681e79c1ba93cb92ed2e3b967224 branch: main author: Jakub ?ervinka committer: hugovk date: 2023-07-23T02:57:52-06:00 summary: gh-107017: removed mention that C does it the same way (#107020) files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index e140f51f1dda7..138d87f892e89 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -4,8 +4,8 @@ More Control Flow Tools *********************** -Besides the :keyword:`while` statement just introduced, Python uses the usual -flow control statements known from other languages, with some twists. +As well as the :keyword:`while` statement just introduced, Python uses a few more +that we will encounter in this chapter. .. _tut-if: @@ -163,14 +163,21 @@ arguments. In chapter :ref:`tut-structures`, we will discuss in more detail abo :keyword:`!break` and :keyword:`!continue` Statements, and :keyword:`!else` Clauses on Loops ============================================================================================ -The :keyword:`break` statement, like in C, breaks out of the innermost enclosing +The :keyword:`break` statement breaks out of the innermost enclosing :keyword:`for` or :keyword:`while` loop. -Loop statements may have an :keyword:`!else` clause; it is executed when the loop -terminates through exhaustion of the iterable (with :keyword:`for`) or when the -condition becomes false (with :keyword:`while`), but not when the loop is -terminated by a :keyword:`break` statement. This is exemplified by the -following loop, which searches for prime numbers:: +A :keyword:`!for` or :keyword:`!while` loop can include an :keyword:`!else` clause. + +In a :keyword:`for` loop, the :keyword:`!else` clause is executed +after the loop reaches its final iteration. + +In a :keyword:`while` loop, it's executed after the loop's condition becomes false. + +In either kind of loop, the :keyword:`!else` clause is **not** executed +if the loop was terminated by a :keyword:`break`. + +This is exemplified in the following :keyword:`!for` loop, +which searches for prime numbers:: >>> for n in range(2, 10): ... for x in range(2, n): From webhook-mailer at python.org Sun Jul 23 04:59:20 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 08:59:20 -0000 Subject: [Python-checkins] [3.12] gh-102111: Add link to string escape sequences in re module (GH-106995) (#107096) Message-ID: https://github.com/python/cpython/commit/52804b38016d304754e3fc6fcfbb69d761176f00 commit: 52804b38016d304754e3fc6fcfbb69d761176f00 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-23T02:59:16-06:00 summary: [3.12] gh-102111: Add link to string escape sequences in re module (GH-106995) (#107096) Co-authored-by: wulmer Co-authored-by: Alex Waygood files: M Doc/library/re.rst M Doc/reference/lexical_analysis.rst diff --git a/Doc/library/re.rst b/Doc/library/re.rst index 629ee472cca68..3f03f0341d816 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -634,8 +634,8 @@ character ``'$'``. single: \x; in regular expressions single: \\; in regular expressions -Most of the standard escapes supported by Python string literals are also -accepted by the regular expression parser:: +Most of the :ref:`escape sequences ` supported by Python +string literals are also accepted by the regular expression parser:: \a \b \f \n \N \r \t \u diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 47062f86810e9..dde7ba1d941dc 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -549,6 +549,10 @@ retained), except that three unescaped quotes in a row terminate the literal. ( .. _escape-sequences: + +Escape sequences +^^^^^^^^^^^^^^^^ + Unless an ``'r'`` or ``'R'`` prefix is present, escape sequences in string and bytes literals are interpreted according to rules similar to those used by Standard C. The recognized escape sequences are: From webhook-mailer at python.org Sun Jul 23 05:00:45 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 09:00:45 -0000 Subject: [Python-checkins] gh-71261: Add paragraph on shadowing submodules with star imports (#107004) Message-ID: https://github.com/python/cpython/commit/680f3e1591e406deb04ba44adbef9a5a02395f80 commit: 680f3e1591e406deb04ba44adbef9a5a02395f80 branch: main author: wulmer committer: hugovk date: 2023-07-23T03:00:42-06:00 summary: gh-71261: Add paragraph on shadowing submodules with star imports (#107004) files: M Doc/tutorial/modules.rst diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst index 3bd034bcc9703..734dd1cfe6871 100644 --- a/Doc/tutorial/modules.rst +++ b/Doc/tutorial/modules.rst @@ -512,6 +512,22 @@ code:: This would mean that ``from sound.effects import *`` would import the three named submodules of the :mod:`sound.effects` package. +Be aware that submodules might become shadowed by locally defined names. For +example, if you added a ``reverse`` function to the +:file:`sound/effects/__init__.py` file, the ``from sound.effects import *`` +would only import the two submodules ``echo`` and ``surround``, but *not* the +``reverse`` submodule, because it is shadowed by the locally defined +``reverse`` function:: + + __all__ = [ + "echo", # refers to the 'echo.py' file + "surround", # refers to the 'surround.py' file + "reverse", # !!! refers to the 'reverse' function now !!! + ] + + def reverse(msg: str): # <-- this name shadows the 'reverse.py' submodule + return msg[::-1] # in the case of a 'from sound.effects import *' + If ``__all__`` is not defined, the statement ``from sound.effects import *`` does *not* import all submodules from the package :mod:`sound.effects` into the current namespace; it only ensures that the package :mod:`sound.effects` has From webhook-mailer at python.org Sun Jul 23 05:01:28 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 09:01:28 -0000 Subject: [Python-checkins] [3.12] gh-71261: Add paragraph on shadowing submodules with star imports (GH-107004) (#107100) Message-ID: https://github.com/python/cpython/commit/a80721b83d6dceffdea9520832f68f0e1bcb82ea commit: a80721b83d6dceffdea9520832f68f0e1bcb82ea branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-23T09:01:24Z summary: [3.12] gh-71261: Add paragraph on shadowing submodules with star imports (GH-107004) (#107100) Co-authored-by: wulmer files: M Doc/tutorial/modules.rst diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst index 3bd034bcc9703..734dd1cfe6871 100644 --- a/Doc/tutorial/modules.rst +++ b/Doc/tutorial/modules.rst @@ -512,6 +512,22 @@ code:: This would mean that ``from sound.effects import *`` would import the three named submodules of the :mod:`sound.effects` package. +Be aware that submodules might become shadowed by locally defined names. For +example, if you added a ``reverse`` function to the +:file:`sound/effects/__init__.py` file, the ``from sound.effects import *`` +would only import the two submodules ``echo`` and ``surround``, but *not* the +``reverse`` submodule, because it is shadowed by the locally defined +``reverse`` function:: + + __all__ = [ + "echo", # refers to the 'echo.py' file + "surround", # refers to the 'surround.py' file + "reverse", # !!! refers to the 'reverse' function now !!! + ] + + def reverse(msg: str): # <-- this name shadows the 'reverse.py' submodule + return msg[::-1] # in the case of a 'from sound.effects import *' + If ``__all__`` is not defined, the statement ``from sound.effects import *`` does *not* import all submodules from the package :mod:`sound.effects` into the current namespace; it only ensures that the package :mod:`sound.effects` has From webhook-mailer at python.org Sun Jul 23 05:10:42 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:10:42 -0000 Subject: [Python-checkins] gh-54738: Add argparse i18n howto (#104562) Message-ID: https://github.com/python/cpython/commit/dcd7acb04a719d8d30c8d03b80d3d48b6c035e14 commit: dcd7acb04a719d8d30c8d03b80d3d48b6c035e14 branch: main author: Tomas R committer: ambv date: 2023-07-23T11:10:38+02:00 summary: gh-54738: Add argparse i18n howto (#104562) files: A Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst M Doc/howto/argparse.rst M Doc/library/gettext.rst diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst index 52e98fa962019..ae5bab90bf813 100644 --- a/Doc/howto/argparse.rst +++ b/Doc/howto/argparse.rst @@ -788,6 +788,59 @@ but not both at the same time: -q, --quiet +How to translate the argparse output +==================================== + +The output of the :mod:`argparse` module such as its help text and error +messages are all made translatable using the :mod:`gettext` module. This +allows applications to easily localize messages produced by +:mod:`argparse`. See also :ref:`i18n-howto`. + +For instance, in this :mod:`argparse` output: + +.. code-block:: shell-session + + $ python prog.py --help + usage: prog.py [-h] [-v | -q] x y + + calculate X to the power of Y + + positional arguments: + x the base + y the exponent + + options: + -h, --help show this help message and exit + -v, --verbose + -q, --quiet + +The strings ``usage:``, ``positional arguments:``, ``options:`` and +``show this help message and exit`` are all translatable. + +In order to translate these strings, they must first be extracted +into a ``.po`` file. For example, using `Babel `__, +run this command: + +.. code-block:: shell-session + + $ pybabel extract -o messages.po /usr/lib/python3.12/argparse.py + +This command will extract all translatable strings from the :mod:`argparse` +module and output them into a file named ``messages.po``. This command assumes +that your Python installation is in ``/usr/lib``. + +You can find out the location of the :mod:`argparse` module on your system +using this script:: + + import argparse + print(argparse.__file__) + +Once the messages in the ``.po`` file are translated and the translations are +installed using :mod:`gettext`, :mod:`argparse` will be able to display the +translated messages. + +To translate your own strings in the :mod:`argparse` output, use :mod:`gettext`. + Conclusion ========== diff --git a/Doc/library/gettext.rst b/Doc/library/gettext.rst index 747f8703b750e..88a65b980d310 100644 --- a/Doc/library/gettext.rst +++ b/Doc/library/gettext.rst @@ -411,6 +411,7 @@ One difference between this module and Henstridge's: his catalog objects supported access through a mapping API, but this appears to be unused and so is not currently supported. +.. _i18n-howto: Internationalizing your programs and modules -------------------------------------------- diff --git a/Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst b/Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst new file mode 100644 index 0000000000000..4da58fc982b6d --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst @@ -0,0 +1 @@ +Add documentation on how to localize the :mod:`argparse` module. From webhook-mailer at python.org Sun Jul 23 05:11:29 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 09:11:29 -0000 Subject: [Python-checkins] [3.12] gh-107017: removed mention that C does it the same way (GH-107020) (#107097) Message-ID: https://github.com/python/cpython/commit/98626c3c71f88707d7297d18e01801f27192ab5c commit: 98626c3c71f88707d7297d18e01801f27192ab5c branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-23T03:11:26-06:00 summary: [3.12] gh-107017: removed mention that C does it the same way (GH-107020) (#107097) Co-authored-by: Jakub ?ervinka files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index e140f51f1dda7..138d87f892e89 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -4,8 +4,8 @@ More Control Flow Tools *********************** -Besides the :keyword:`while` statement just introduced, Python uses the usual -flow control statements known from other languages, with some twists. +As well as the :keyword:`while` statement just introduced, Python uses a few more +that we will encounter in this chapter. .. _tut-if: @@ -163,14 +163,21 @@ arguments. In chapter :ref:`tut-structures`, we will discuss in more detail abo :keyword:`!break` and :keyword:`!continue` Statements, and :keyword:`!else` Clauses on Loops ============================================================================================ -The :keyword:`break` statement, like in C, breaks out of the innermost enclosing +The :keyword:`break` statement breaks out of the innermost enclosing :keyword:`for` or :keyword:`while` loop. -Loop statements may have an :keyword:`!else` clause; it is executed when the loop -terminates through exhaustion of the iterable (with :keyword:`for`) or when the -condition becomes false (with :keyword:`while`), but not when the loop is -terminated by a :keyword:`break` statement. This is exemplified by the -following loop, which searches for prime numbers:: +A :keyword:`!for` or :keyword:`!while` loop can include an :keyword:`!else` clause. + +In a :keyword:`for` loop, the :keyword:`!else` clause is executed +after the loop reaches its final iteration. + +In a :keyword:`while` loop, it's executed after the loop's condition becomes false. + +In either kind of loop, the :keyword:`!else` clause is **not** executed +if the loop was terminated by a :keyword:`break`. + +This is exemplified in the following :keyword:`!for` loop, +which searches for prime numbers:: >>> for n in range(2, 10): ... for x in range(2, n): From webhook-mailer at python.org Sun Jul 23 05:11:38 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 09:11:38 -0000 Subject: [Python-checkins] [3.11] gh-107017: removed mention that C does it the same way (GH-107020) (#107098) Message-ID: https://github.com/python/cpython/commit/7d414873b3a020cec318d53cc1bb62ccdac061bd commit: 7d414873b3a020cec318d53cc1bb62ccdac061bd branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-23T03:11:35-06:00 summary: [3.11] gh-107017: removed mention that C does it the same way (GH-107020) (#107098) Co-authored-by: Jakub ?ervinka files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index e140f51f1dda7..138d87f892e89 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -4,8 +4,8 @@ More Control Flow Tools *********************** -Besides the :keyword:`while` statement just introduced, Python uses the usual -flow control statements known from other languages, with some twists. +As well as the :keyword:`while` statement just introduced, Python uses a few more +that we will encounter in this chapter. .. _tut-if: @@ -163,14 +163,21 @@ arguments. In chapter :ref:`tut-structures`, we will discuss in more detail abo :keyword:`!break` and :keyword:`!continue` Statements, and :keyword:`!else` Clauses on Loops ============================================================================================ -The :keyword:`break` statement, like in C, breaks out of the innermost enclosing +The :keyword:`break` statement breaks out of the innermost enclosing :keyword:`for` or :keyword:`while` loop. -Loop statements may have an :keyword:`!else` clause; it is executed when the loop -terminates through exhaustion of the iterable (with :keyword:`for`) or when the -condition becomes false (with :keyword:`while`), but not when the loop is -terminated by a :keyword:`break` statement. This is exemplified by the -following loop, which searches for prime numbers:: +A :keyword:`!for` or :keyword:`!while` loop can include an :keyword:`!else` clause. + +In a :keyword:`for` loop, the :keyword:`!else` clause is executed +after the loop reaches its final iteration. + +In a :keyword:`while` loop, it's executed after the loop's condition becomes false. + +In either kind of loop, the :keyword:`!else` clause is **not** executed +if the loop was terminated by a :keyword:`break`. + +This is exemplified in the following :keyword:`!for` loop, +which searches for prime numbers:: >>> for n in range(2, 10): ... for x in range(2, n): From webhook-mailer at python.org Sun Jul 23 05:12:56 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 09:12:56 -0000 Subject: [Python-checkins] [3.11] gh-71261: Add paragraph on shadowing submodules with star imports (GH-107004) (#107099) Message-ID: https://github.com/python/cpython/commit/5575b1204b9870301f2e5797a8535576487b5800 commit: 5575b1204b9870301f2e5797a8535576487b5800 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-23T03:12:52-06:00 summary: [3.11] gh-71261: Add paragraph on shadowing submodules with star imports (GH-107004) (#107099) Co-authored-by: wulmer files: M Doc/tutorial/modules.rst diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst index 3bd034bcc9703..734dd1cfe6871 100644 --- a/Doc/tutorial/modules.rst +++ b/Doc/tutorial/modules.rst @@ -512,6 +512,22 @@ code:: This would mean that ``from sound.effects import *`` would import the three named submodules of the :mod:`sound.effects` package. +Be aware that submodules might become shadowed by locally defined names. For +example, if you added a ``reverse`` function to the +:file:`sound/effects/__init__.py` file, the ``from sound.effects import *`` +would only import the two submodules ``echo`` and ``surround``, but *not* the +``reverse`` submodule, because it is shadowed by the locally defined +``reverse`` function:: + + __all__ = [ + "echo", # refers to the 'echo.py' file + "surround", # refers to the 'surround.py' file + "reverse", # !!! refers to the 'reverse' function now !!! + ] + + def reverse(msg: str): # <-- this name shadows the 'reverse.py' submodule + return msg[::-1] # in the case of a 'from sound.effects import *' + If ``__all__`` is not defined, the statement ``from sound.effects import *`` does *not* import all submodules from the package :mod:`sound.effects` into the current namespace; it only ensures that the package :mod:`sound.effects` has From webhook-mailer at python.org Sun Jul 23 05:23:48 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 09:23:48 -0000 Subject: [Python-checkins] gh-101100: Fix some broken sphinx references (#107095) Message-ID: https://github.com/python/cpython/commit/f5147c0cfbd7943ff10917225448c36a53f9828d commit: f5147c0cfbd7943ff10917225448c36a53f9828d branch: main author: wulmer committer: hugovk date: 2023-07-23T03:23:44-06:00 summary: gh-101100: Fix some broken sphinx references (#107095) files: M Doc/c-api/iterator.rst M Doc/c-api/mapping.rst M Doc/c-api/refcounting.rst M Doc/c-api/sequence.rst M Doc/howto/functional.rst M Doc/howto/regex.rst M Doc/howto/sorting.rst M Doc/howto/unicode.rst M Doc/library/_thread.rst M Doc/library/codeop.rst M Doc/library/constants.rst M Doc/tools/.nitignore diff --git a/Doc/c-api/iterator.rst b/Doc/c-api/iterator.rst index 3fcf099134d4d..95952237ca746 100644 --- a/Doc/c-api/iterator.rst +++ b/Doc/c-api/iterator.rst @@ -6,7 +6,7 @@ Iterator Objects ---------------- Python provides two general-purpose iterator objects. The first, a sequence -iterator, works with an arbitrary sequence supporting the :meth:`__getitem__` +iterator, works with an arbitrary sequence supporting the :meth:`~object.__getitem__` method. The second works with a callable object and a sentinel value, calling the callable for each item in the sequence, and ending the iteration when the sentinel value is returned. diff --git a/Doc/c-api/mapping.rst b/Doc/c-api/mapping.rst index 9176a4652cbf2..82011d9d36a52 100644 --- a/Doc/c-api/mapping.rst +++ b/Doc/c-api/mapping.rst @@ -13,7 +13,7 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and Return ``1`` if the object provides the mapping protocol or supports slicing, and ``0`` otherwise. Note that it returns ``1`` for Python classes with - a :meth:`__getitem__` method, since in general it is impossible to + a :meth:`~object.__getitem__` method, since in general it is impossible to determine what type of keys the class supports. This function always succeeds. @@ -90,7 +90,7 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and This is equivalent to the Python expression ``key in o``. This function always succeeds. - Note that exceptions which occur while calling the :meth:`__getitem__` + Note that exceptions which occur while calling the :meth:`~object.__getitem__` method will get suppressed. To get error reporting use :c:func:`PyObject_GetItem()` instead. @@ -101,7 +101,7 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and This is equivalent to the Python expression ``key in o``. This function always succeeds. - Note that exceptions which occur while calling the :meth:`__getitem__` + Note that exceptions which occur while calling the :meth:`~object.__getitem__` method and creating a temporary string object will get suppressed. To get error reporting use :c:func:`PyMapping_GetItemString()` instead. diff --git a/Doc/c-api/refcounting.rst b/Doc/c-api/refcounting.rst index d8e9c2da6f3ff..640c5c610899f 100644 --- a/Doc/c-api/refcounting.rst +++ b/Doc/c-api/refcounting.rst @@ -101,7 +101,7 @@ of Python objects. .. warning:: The deallocation function can cause arbitrary Python code to be invoked (e.g. - when a class instance with a :meth:`__del__` method is deallocated). While + when a class instance with a :meth:`~object.__del__` method is deallocated). While exceptions in such code are not propagated, the executed code has free access to all Python global variables. This means that any object that is reachable from a global variable should be in a consistent state before :c:func:`Py_DECREF` is diff --git a/Doc/c-api/sequence.rst b/Doc/c-api/sequence.rst index 402a3e5e09ff5..ce28839f5ba73 100644 --- a/Doc/c-api/sequence.rst +++ b/Doc/c-api/sequence.rst @@ -9,7 +9,7 @@ Sequence Protocol .. c:function:: int PySequence_Check(PyObject *o) Return ``1`` if the object provides the sequence protocol, and ``0`` otherwise. - Note that it returns ``1`` for Python classes with a :meth:`__getitem__` + Note that it returns ``1`` for Python classes with a :meth:`~object.__getitem__` method, unless they are :class:`dict` subclasses, since in general it is impossible to determine what type of keys the class supports. This function always succeeds. diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst index 5cf12cc52bde4..b0f9d22d74f0e 100644 --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -1072,8 +1072,8 @@ write the obvious :keyword:`for` loop:: A related function is :func:`itertools.accumulate(iterable, func=operator.add) `. It performs the same calculation, but instead of -returning only the final result, :func:`accumulate` returns an iterator that -also yields each partial result:: +returning only the final result, :func:`~itertools.accumulate` returns an iterator +that also yields each partial result:: itertools.accumulate([1, 2, 3, 4, 5]) => 1, 3, 6, 10, 15 diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst index 655df59e27b64..c19c48301f584 100644 --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -518,6 +518,8 @@ cache. Compilation Flags ----------------- +.. currentmodule:: re + Compilation flags let you modify some aspects of how regular expressions work. Flags are available in the :mod:`re` module under two names, a long name such as :const:`IGNORECASE` and a short, one-letter form such as :const:`I`. (If you're diff --git a/Doc/howto/sorting.rst b/Doc/howto/sorting.rst index decce12bf3faf..38dd09f0a721d 100644 --- a/Doc/howto/sorting.rst +++ b/Doc/howto/sorting.rst @@ -273,7 +273,7 @@ Odds and Ends * The sort routines use ``<`` when making comparisons between two objects. So, it is easy to add a standard sort order to a class by - defining an :meth:`__lt__` method: + defining an :meth:`~object.__lt__` method: .. doctest:: @@ -281,8 +281,8 @@ Odds and Ends >>> sorted(student_objects) [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] - However, note that ``<`` can fall back to using :meth:`__gt__` if - :meth:`__lt__` is not implemented (see :func:`object.__lt__`). + However, note that ``<`` can fall back to using :meth:`~object.__gt__` if + :meth:`~object.__lt__` is not implemented (see :func:`object.__lt__`). * Key functions need not depend directly on the objects being sorted. A key function can also access external resources. For instance, if the student grades diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst index b0faa68d24089..254fe72935535 100644 --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -424,8 +424,8 @@ lowercase letters 'ss'. A second tool is the :mod:`unicodedata` module's :func:`~unicodedata.normalize` function that converts strings to one -of several normal forms, where letters followed by a combining -character are replaced with single characters. :func:`normalize` can +of several normal forms, where letters followed by a combining character are +replaced with single characters. :func:`~unicodedata.normalize` can be used to perform string comparisons that won't falsely report inequality if two strings use combining characters differently: @@ -474,8 +474,8 @@ The Unicode Standard also specifies how to do caseless comparisons:: print(compare_caseless(single_char, multiple_chars)) -This will print ``True``. (Why is :func:`NFD` invoked twice? Because -there are a few characters that make :meth:`casefold` return a +This will print ``True``. (Why is :func:`!NFD` invoked twice? Because +there are a few characters that make :meth:`~str.casefold` return a non-normalized string, so the result needs to be normalized again. See section 3.13 of the Unicode Standard for a discussion and an example.) diff --git a/Doc/library/_thread.rst b/Doc/library/_thread.rst index 3ff6b48b083b5..0442c298c137b 100644 --- a/Doc/library/_thread.rst +++ b/Doc/library/_thread.rst @@ -150,8 +150,8 @@ This module defines the following constants and functions: .. data:: TIMEOUT_MAX The maximum value allowed for the *timeout* parameter of - :meth:`Lock.acquire`. Specifying a timeout greater than this value will - raise an :exc:`OverflowError`. + :meth:`Lock.acquire `. Specifying a timeout greater + than this value will raise an :exc:`OverflowError`. .. versionadded:: 3.2 @@ -217,8 +217,9 @@ In addition to these methods, lock objects can also be used via the * Calling :func:`sys.exit` or raising the :exc:`SystemExit` exception is equivalent to calling :func:`_thread.exit`. -* It is not possible to interrupt the :meth:`acquire` method on a lock --- the - :exc:`KeyboardInterrupt` exception will happen after the lock has been acquired. +* It is not possible to interrupt the :meth:`~threading.Lock.acquire` method on + a lock --- the :exc:`KeyboardInterrupt` exception will happen after the lock + has been acquired. * When the main thread exits, it is system defined whether the other threads survive. On most systems, they are killed without executing diff --git a/Doc/library/codeop.rst b/Doc/library/codeop.rst index 90df499f8207b..55606e1c5f09a 100644 --- a/Doc/library/codeop.rst +++ b/Doc/library/codeop.rst @@ -58,7 +58,7 @@ To do just the former: .. class:: Compile() - Instances of this class have :meth:`__call__` methods identical in signature to + Instances of this class have :meth:`~object.__call__` methods identical in signature to the built-in function :func:`compile`, but with the difference that if the instance compiles program text containing a :mod:`__future__` statement, the instance 'remembers' and compiles all subsequent program texts with the @@ -67,7 +67,7 @@ To do just the former: .. class:: CommandCompiler() - Instances of this class have :meth:`__call__` methods identical in signature to + Instances of this class have :meth:`~object.__call__` methods identical in signature to :func:`compile_command`; the difference is that if the instance compiles program text containing a :mod:`__future__` statement, the instance 'remembers' and compiles all subsequent program texts with the statement in force. diff --git a/Doc/library/constants.rst b/Doc/library/constants.rst index 38dd552a0363a..401dc9a320c5e 100644 --- a/Doc/library/constants.rst +++ b/Doc/library/constants.rst @@ -22,16 +22,16 @@ A small number of constants live in the built-in namespace. They are: An object frequently used to represent the absence of a value, as when default arguments are not passed to a function. Assignments to ``None`` are illegal and raise a :exc:`SyntaxError`. - ``None`` is the sole instance of the :data:`NoneType` type. + ``None`` is the sole instance of the :data:`~types.NoneType` type. .. data:: NotImplemented A special value which should be returned by the binary special methods - (e.g. :meth:`__eq__`, :meth:`__lt__`, :meth:`__add__`, :meth:`__rsub__`, + (e.g. :meth:`~object.__eq__`, :meth:`~object.__lt__`, :meth:`~object.__add__`, :meth:`~object.__rsub__`, etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods - (e.g. :meth:`__imul__`, :meth:`__iand__`, etc.) for the same purpose. + (e.g. :meth:`~object.__imul__`, :meth:`~object.__iand__`, etc.) for the same purpose. It should not be evaluated in a boolean context. ``NotImplemented`` is the sole instance of the :data:`types.NotImplementedType` type. diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 14d9b2e121d3b..818276a81111e 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -27,15 +27,12 @@ Doc/c-api/init_config.rst Doc/c-api/intro.rst Doc/c-api/iterator.rst Doc/c-api/long.rst -Doc/c-api/mapping.rst Doc/c-api/marshal.rst Doc/c-api/memory.rst Doc/c-api/memoryview.rst Doc/c-api/module.rst Doc/c-api/none.rst Doc/c-api/object.rst -Doc/c-api/refcounting.rst -Doc/c-api/sequence.rst Doc/c-api/set.rst Doc/c-api/stable.rst Doc/c-api/structures.rst @@ -59,18 +56,13 @@ Doc/glossary.rst Doc/howto/curses.rst Doc/howto/descriptor.rst Doc/howto/enum.rst -Doc/howto/functional.rst Doc/howto/instrumentation.rst Doc/howto/isolating-extensions.rst Doc/howto/logging-cookbook.rst Doc/howto/logging.rst -Doc/howto/regex.rst -Doc/howto/sorting.rst -Doc/howto/unicode.rst Doc/howto/urllib2.rst Doc/install/index.rst Doc/library/__future__.rst -Doc/library/_thread.rst Doc/library/abc.rst Doc/library/ast.rst Doc/library/asyncio-dev.rst @@ -88,13 +80,11 @@ Doc/library/calendar.rst Doc/library/cmd.rst Doc/library/code.rst Doc/library/codecs.rst -Doc/library/codeop.rst Doc/library/collections.abc.rst Doc/library/collections.rst Doc/library/concurrent.futures.rst Doc/library/concurrent.rst Doc/library/configparser.rst -Doc/library/constants.rst Doc/library/contextlib.rst Doc/library/copy.rst Doc/library/csv.rst From webhook-mailer at python.org Sun Jul 23 05:23:54 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:23:54 -0000 Subject: [Python-checkins] [3.11] gh-54738: Add argparse i18n howto (GH-104562) (#107101) Message-ID: https://github.com/python/cpython/commit/65e40aaed0567830d8ec88769c27321f3504cd80 commit: 65e40aaed0567830d8ec88769c27321f3504cd80 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-23T11:23:51+02:00 summary: [3.11] gh-54738: Add argparse i18n howto (GH-104562) (#107101) (cherry picked from commit dcd7acb04a719d8d30c8d03b80d3d48b6c035e14) Co-authored-by: Tomas R files: A Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst M Doc/howto/argparse.rst M Doc/library/gettext.rst diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst index 9ea140c41a09c..06a301a120c1c 100644 --- a/Doc/howto/argparse.rst +++ b/Doc/howto/argparse.rst @@ -759,6 +759,59 @@ but not both at the same time: -q, --quiet +How to translate the argparse output +==================================== + +The output of the :mod:`argparse` module such as its help text and error +messages are all made translatable using the :mod:`gettext` module. This +allows applications to easily localize messages produced by +:mod:`argparse`. See also :ref:`i18n-howto`. + +For instance, in this :mod:`argparse` output: + +.. code-block:: shell-session + + $ python prog.py --help + usage: prog.py [-h] [-v | -q] x y + + calculate X to the power of Y + + positional arguments: + x the base + y the exponent + + options: + -h, --help show this help message and exit + -v, --verbose + -q, --quiet + +The strings ``usage:``, ``positional arguments:``, ``options:`` and +``show this help message and exit`` are all translatable. + +In order to translate these strings, they must first be extracted +into a ``.po`` file. For example, using `Babel `__, +run this command: + +.. code-block:: shell-session + + $ pybabel extract -o messages.po /usr/lib/python3.12/argparse.py + +This command will extract all translatable strings from the :mod:`argparse` +module and output them into a file named ``messages.po``. This command assumes +that your Python installation is in ``/usr/lib``. + +You can find out the location of the :mod:`argparse` module on your system +using this script:: + + import argparse + print(argparse.__file__) + +Once the messages in the ``.po`` file are translated and the translations are +installed using :mod:`gettext`, :mod:`argparse` will be able to display the +translated messages. + +To translate your own strings in the :mod:`argparse` output, use :mod:`gettext`. + Conclusion ========== diff --git a/Doc/library/gettext.rst b/Doc/library/gettext.rst index 747f8703b750e..88a65b980d310 100644 --- a/Doc/library/gettext.rst +++ b/Doc/library/gettext.rst @@ -411,6 +411,7 @@ One difference between this module and Henstridge's: his catalog objects supported access through a mapping API, but this appears to be unused and so is not currently supported. +.. _i18n-howto: Internationalizing your programs and modules -------------------------------------------- diff --git a/Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst b/Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst new file mode 100644 index 0000000000000..4da58fc982b6d --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst @@ -0,0 +1 @@ +Add documentation on how to localize the :mod:`argparse` module. From webhook-mailer at python.org Sun Jul 23 05:24:41 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:24:41 -0000 Subject: [Python-checkins] [3.11] gh-106948: Add standard external names to nitpick_ignore (GH-106949) (#107061) Message-ID: https://github.com/python/cpython/commit/ec8718dd7c318202723ceeee9479f3824f3c019c commit: ec8718dd7c318202723ceeee9479f3824f3c019c branch: 3.11 author: Serhiy Storchaka committer: ambv date: 2023-07-23T11:24:37+02:00 summary: [3.11] gh-106948: Add standard external names to nitpick_ignore (GH-106949) (#107061) It includes standard C types, macros and variables like "size_t", "LONG_MAX" and "errno", and standard environment variables like "PATH".. (cherry picked from commit f8b7fe2f2647813ae8249675a80e59c117d30fe1) files: A Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst M Doc/c-api/arg.rst M Doc/c-api/memory.rst M Doc/c-api/unicode.rst M Doc/c-api/veryhigh.rst M Doc/conf.py M Doc/library/array.rst M Doc/library/ctypes.rst M Doc/library/os.rst M Doc/library/struct.rst M Doc/library/venv.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a5.rst diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index ee130ab28510b..9f7fd2fa8629e 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -584,7 +584,7 @@ Building values Same as ``s#``. ``u`` (:class:`str`) [const wchar_t \*] - Convert a null-terminated :c:expr:`wchar_t` buffer of Unicode (UTF-16 or UCS-4) + Convert a null-terminated :c:type:`wchar_t` buffer of Unicode (UTF-16 or UCS-4) data to a Python Unicode object. If the Unicode buffer pointer is ``NULL``, ``None`` is returned. diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 35c356f25c6c7..4ca3b8804427f 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -581,7 +581,7 @@ that the treatment of negative indices differs from a Python slice): default). A serial number, incremented by 1 on each call to a malloc-like or - realloc-like function. Big-endian ``size_t``. If "bad memory" is detected + realloc-like function. Big-endian :c:type:`size_t`. If "bad memory" is detected later, the serial number gives an excellent way to set a breakpoint on the next run, to capture the instant at which this block was passed out. The static function bumpserialno() in obmalloc.c is the only place the serial diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index b2241ea8bd2e1..a96e0738f716e 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -58,7 +58,7 @@ Python: .. c:type:: Py_UNICODE - This is a typedef of :c:expr:`wchar_t`, which is a 16-bit type or 32-bit type + This is a typedef of :c:type:`wchar_t`, which is a 16-bit type or 32-bit type depending on the platform. .. versionchanged:: 3.3 @@ -935,11 +935,11 @@ conversion function: wchar_t Support """"""""""""""" -:c:expr:`wchar_t` support for platforms which support it: +:c:type:`wchar_t` support for platforms which support it: .. c:function:: PyObject* PyUnicode_FromWideChar(const wchar_t *w, Py_ssize_t size) - Create a Unicode object from the :c:expr:`wchar_t` buffer *w* of the given *size*. + Create a Unicode object from the :c:type:`wchar_t` buffer *w* of the given *size*. Passing ``-1`` as the *size* indicates that the function must itself compute the length, using wcslen. Return ``NULL`` on failure. @@ -947,9 +947,9 @@ wchar_t Support .. c:function:: Py_ssize_t PyUnicode_AsWideChar(PyObject *unicode, wchar_t *w, Py_ssize_t size) - Copy the Unicode object contents into the :c:expr:`wchar_t` buffer *w*. At most - *size* :c:expr:`wchar_t` characters are copied (excluding a possibly trailing - null termination character). Return the number of :c:expr:`wchar_t` characters + Copy the Unicode object contents into the :c:type:`wchar_t` buffer *w*. At most + *size* :c:type:`wchar_t` characters are copied (excluding a possibly trailing + null termination character). Return the number of :c:type:`wchar_t` characters copied or ``-1`` in case of an error. Note that the resulting :c:expr:`wchar_t*` string may or may not be null-terminated. It is the responsibility of the caller to make sure that the :c:expr:`wchar_t*` string is null-terminated in case this is @@ -963,7 +963,7 @@ wchar_t Support Convert the Unicode object to a wide character string. The output string always ends with a null character. If *size* is not ``NULL``, write the number of wide characters (excluding the trailing null termination character) into - *\*size*. Note that the resulting :c:expr:`wchar_t` string might contain + *\*size*. Note that the resulting :c:type:`wchar_t` string might contain null characters, which would cause the string to be truncated when used with most C functions. If *size* is ``NULL`` and the :c:expr:`wchar_t*` string contains null characters a :exc:`ValueError` is raised. diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index cfbb44f4eb3a6..79515c9df07ed 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -17,7 +17,7 @@ parameter. The available start symbols are :c:data:`Py_eval_input`, following the functions which accept them as parameters. Note also that several of these functions take :c:expr:`FILE*` parameters. One -particular issue which needs to be handled carefully is that the :c:expr:`FILE` +particular issue which needs to be handled carefully is that the :c:type:`FILE` structure for different C libraries can be different and incompatible. Under Windows (at least), it is possible for dynamically linked extensions to actually use different libraries, so care should be taken that :c:expr:`FILE*` parameters diff --git a/Doc/conf.py b/Doc/conf.py index 3bd828f8ce9ed..c082717efda72 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -72,6 +72,58 @@ exclude_patterns.append(venvdir + '/*') nitpick_ignore = [ + # Standard C types + ('c:type', 'FILE'), + ('c:type', '__int'), + ('c:type', 'intmax_t'), + ('c:type', 'off_t'), + ('c:type', 'ptrdiff_t'), + ('c:type', 'siginfo_t'), + ('c:type', 'size_t'), + ('c:type', 'ssize_t'), + ('c:type', 'time_t'), + ('c:type', 'uintmax_t'), + ('c:type', 'va_list'), + ('c:type', 'wchar_t'), + # Standard C macros + ('c:macro', 'LLONG_MAX'), + ('c:macro', 'LLONG_MIN'), + ('c:macro', 'LONG_MAX'), + ('c:macro', 'LONG_MIN'), + # Standard C variables + ('c:data', 'errno'), + # Standard environment variables + ('envvar', 'BROWSER'), + ('envvar', 'COLUMNS'), + ('envvar', 'COMSPEC'), + ('envvar', 'DISPLAY'), + ('envvar', 'HOME'), + ('envvar', 'HOMEDRIVE'), + ('envvar', 'HOMEPATH'), + ('envvar', 'IDLESTARTUP'), + ('envvar', 'LANG'), + ('envvar', 'LANGUAGE'), + ('envvar', 'LC_ALL'), + ('envvar', 'LC_CTYPE'), + ('envvar', 'LC_COLLATE'), + ('envvar', 'LC_MESSAGES'), + ('envvar', 'LC_MONETARY'), + ('envvar', 'LC_NUMERIC'), + ('envvar', 'LC_TIME'), + ('envvar', 'LINES'), + ('envvar', 'LOGNAME'), + ('envvar', 'PAGER'), + ('envvar', 'PATH'), + ('envvar', 'PATHEXT'), + ('envvar', 'SOURCE_DATE_EPOCH'), + ('envvar', 'TEMP'), + ('envvar', 'TERM'), + ('envvar', 'TMP'), + ('envvar', 'TMPDIR'), + ('envvar', 'TZ'), + ('envvar', 'USER'), + ('envvar', 'USERNAME'), + ('envvar', 'USERPROFILE'), # Do not error nit-picky mode builds when _SubParsersAction.add_parser cannot # be resolved, as the method is currently undocumented. For context, see # https://github.com/python/cpython/pull/103289. diff --git a/Doc/library/array.rst b/Doc/library/array.rst index 75c49e0f6d1eb..d6cb8c623adbf 100644 --- a/Doc/library/array.rst +++ b/Doc/library/array.rst @@ -51,9 +51,9 @@ Notes: It can be 16 bits or 32 bits depending on the platform. .. versionchanged:: 3.9 - ``array('u')`` now uses ``wchar_t`` as C type instead of deprecated + ``array('u')`` now uses :c:type:`wchar_t` as C type instead of deprecated ``Py_UNICODE``. This change doesn't affect its behavior because - ``Py_UNICODE`` is alias of ``wchar_t`` since Python 3.3. + ``Py_UNICODE`` is alias of :c:type:`wchar_t` since Python 3.3. .. deprecated-removed:: 3.3 4.0 diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst index 2be2473f9b9a4..b55ec32ef5af5 100644 --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -221,7 +221,7 @@ Fundamental data types +----------------------+------------------------------------------+----------------------------+ | :class:`c_char` | :c:expr:`char` | 1-character bytes object | +----------------------+------------------------------------------+----------------------------+ -| :class:`c_wchar` | :c:expr:`wchar_t` | 1-character string | +| :class:`c_wchar` | :c:type:`wchar_t` | 1-character string | +----------------------+------------------------------------------+----------------------------+ | :class:`c_byte` | :c:expr:`char` | int | +----------------------+------------------------------------------+----------------------------+ @@ -244,9 +244,9 @@ Fundamental data types | :class:`c_ulonglong` | :c:expr:`unsigned __int64` or | int | | | :c:expr:`unsigned long long` | | +----------------------+------------------------------------------+----------------------------+ -| :class:`c_size_t` | :c:expr:`size_t` | int | +| :class:`c_size_t` | :c:type:`size_t` | int | +----------------------+------------------------------------------+----------------------------+ -| :class:`c_ssize_t` | :c:expr:`ssize_t` or | int | +| :class:`c_ssize_t` | :c:type:`ssize_t` or | int | | | :c:expr:`Py_ssize_t` | | +----------------------+------------------------------------------+----------------------------+ | :class:`c_float` | :c:expr:`float` | float | @@ -334,7 +334,7 @@ property:: The :func:`create_string_buffer` function replaces the old :func:`c_buffer` function (which is still available as an alias). To create a mutable memory -block containing unicode characters of the C type :c:expr:`wchar_t`, use the +block containing unicode characters of the C type :c:type:`wchar_t`, use the :func:`create_unicode_buffer` function. @@ -2361,7 +2361,7 @@ These are the fundamental ctypes data types: .. class:: c_wchar - Represents the C :c:expr:`wchar_t` datatype, and interprets the value as a + Represents the C :c:type:`wchar_t` datatype, and interprets the value as a single character unicode string. The constructor accepts an optional string initializer, the length of the string must be exactly one character. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 283dc5ad1541a..9d6d02652db90 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -4410,7 +4410,7 @@ written in Python, such as a mail server's external command delivery program. :data:`WNOHANG` and :data:`WNOWAIT` are additional optional flags. The return value is an object representing the data contained in the - :c:type:`!siginfo_t` structure with the following attributes: + :c:type:`siginfo_t` structure with the following attributes: * :attr:`!si_pid` (process ID) * :attr:`!si_uid` (real user ID of the child) diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index 26c8d650382ce..416b01db615fa 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -231,9 +231,9 @@ platform-dependent. | ``Q`` | :c:expr:`unsigned long | integer | 8 | \(2) | | | long` | | | | +--------+--------------------------+--------------------+----------------+------------+ -| ``n`` | :c:expr:`ssize_t` | integer | | \(3) | +| ``n`` | :c:type:`ssize_t` | integer | | \(3) | +--------+--------------------------+--------------------+----------------+------------+ -| ``N`` | :c:expr:`size_t` | integer | | \(3) | +| ``N`` | :c:type:`size_t` | integer | | \(3) | +--------+--------------------------+--------------------+----------------+------------+ | ``e`` | \(6) | float | 2 | \(4) | +--------+--------------------------+--------------------+----------------+------------+ diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst index 07a6f5f8920f9..0b9787d095d9a 100644 --- a/Doc/library/venv.rst +++ b/Doc/library/venv.rst @@ -60,7 +60,7 @@ running from a virtual environment. A virtual environment may be "activated" using a script in its binary directory (``bin`` on POSIX; ``Scripts`` on Windows). -This will prepend that directory to your :envvar:`!PATH`, so that running +This will prepend that directory to your :envvar:`PATH`, so that running :program:`python` will invoke the environment's Python interpreter and you can run installed scripts without having to use their full path. The invocation of the activation script is platform-specific @@ -100,10 +100,10 @@ In order to achieve this, scripts installed into virtual environments have a "shebang" line which points to the environment's Python interpreter, i.e. :samp:`#!/{}/bin/python`. This means that the script will run with that interpreter regardless of the -value of :envvar:`!PATH`. On Windows, "shebang" line processing is supported if +value of :envvar:`PATH`. On Windows, "shebang" line processing is supported if you have the :ref:`launcher` installed. Thus, double-clicking an installed script in a Windows Explorer window should run it with the correct interpreter -without the environment needing to be activated or on the :envvar:`!PATH`. +without the environment needing to be activated or on the :envvar:`PATH`. When a virtual environment has been activated, the :envvar:`!VIRTUAL_ENV` environment variable is set to the path of the environment. diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 782048c397f97..e3e49640f6c91 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -1983,7 +1983,7 @@ the form '-rwxrwxrwx'. struct ------ -The :mod:`struct` module now supports ``ssize_t`` and ``size_t`` via the +The :mod:`struct` module now supports :c:type:`ssize_t` and :c:type:`size_t` via the new codes ``n`` and ``N``, respectively. (Contributed by Antoine Pitrou in :issue:`3163`.) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 69d9147524cae..315de6b59fe5f 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2192,7 +2192,7 @@ encode error with ``\N{...}`` escapes. (Contributed by Serhiy Storchaka in :issue:`19676`.) A new :c:func:`PyErr_FormatV` function similar to :c:func:`PyErr_Format`, -but accepts a ``va_list`` argument. +but accepts a :c:type:`va_list` argument. (Contributed by Antoine Pitrou in :issue:`18711`.) A new :c:data:`PyExc_RecursionError` exception. diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 1f0fbbc219666..23db4022dc06a 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1115,9 +1115,9 @@ Changes in the Python API ``PyCF_ALLOW_TOP_LEVEL_AWAIT`` was clashing with ``CO_FUTURE_DIVISION``. (Contributed by Batuhan Taskaya in :issue:`39562`) -* ``array('u')`` now uses ``wchar_t`` as C type instead of ``Py_UNICODE``. +* ``array('u')`` now uses :c:type:`wchar_t` as C type instead of ``Py_UNICODE``. This change doesn't affect to its behavior because ``Py_UNICODE`` is alias - of ``wchar_t`` since Python 3.3. + of :c:type:`wchar_t` since Python 3.3. (Contributed by Inada Naoki in :issue:`34538`.) * The :func:`logging.getLogger` API now returns the root logger when passed diff --git a/Misc/NEWS.d/3.10.0a5.rst b/Misc/NEWS.d/3.10.0a5.rst index 1c7c7447cae06..2854afcc8f84f 100644 --- a/Misc/NEWS.d/3.10.0a5.rst +++ b/Misc/NEWS.d/3.10.0a5.rst @@ -667,4 +667,4 @@ exception (if an exception is set). Patch by Victor Stinner. .. section: C API Fixed a compiler warning in :c:func:`Py_UNICODE_ISSPACE()` on platforms with -signed ``wchar_t``. +signed :c:type:`wchar_t`. diff --git a/Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst b/Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst new file mode 100644 index 0000000000000..42b6348153b56 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst @@ -0,0 +1 @@ +Add a number of standard external names to ``nitpick_ignore``. From webhook-mailer at python.org Sun Jul 23 05:24:55 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:24:55 -0000 Subject: [Python-checkins] [3.12] gh-106948: Add standard external names to nitpick_ignore (GH-106949) (#107060) Message-ID: https://github.com/python/cpython/commit/456cf8b0972bedf035c648fcd5e384ead937f9db commit: 456cf8b0972bedf035c648fcd5e384ead937f9db branch: 3.12 author: Serhiy Storchaka committer: ambv date: 2023-07-23T11:24:51+02:00 summary: [3.12] gh-106948: Add standard external names to nitpick_ignore (GH-106949) (#107060) * [3.12] gh-106948: Add standard external names to nitpick_ignore (GH-106949) It includes standard C types, macros and variables like "size_t", "LONG_MAX" and "errno", and standard environment variables like "PATH". (cherry picked from commit f8b7fe2f2647813ae8249675a80e59c117d30fe1) Co-authored-by: Serhiy Storchaka * Delete 2023-05-31-18-37-57.gh-issue-105156.R4El5V.rst files: A Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst M Doc/c-api/arg.rst M Doc/c-api/memory.rst M Doc/c-api/unicode.rst M Doc/c-api/veryhigh.rst M Doc/conf.py M Doc/library/array.rst M Doc/library/ctypes.rst M Doc/library/os.rst M Doc/library/struct.rst M Doc/library/venv.rst M Doc/tools/.nitignore M Doc/whatsnew/3.12.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a5.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0b1.rst diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index eba8f2be70b45..ea176bd7d6a51 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -547,7 +547,7 @@ Building values Same as ``s#``. ``u`` (:class:`str`) [const wchar_t \*] - Convert a null-terminated :c:expr:`wchar_t` buffer of Unicode (UTF-16 or UCS-4) + Convert a null-terminated :c:type:`wchar_t` buffer of Unicode (UTF-16 or UCS-4) data to a Python Unicode object. If the Unicode buffer pointer is ``NULL``, ``None`` is returned. diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 35c356f25c6c7..4ca3b8804427f 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -581,7 +581,7 @@ that the treatment of negative indices differs from a Python slice): default). A serial number, incremented by 1 on each call to a malloc-like or - realloc-like function. Big-endian ``size_t``. If "bad memory" is detected + realloc-like function. Big-endian :c:type:`size_t`. If "bad memory" is detected later, the serial number gives an excellent way to set a breakpoint on the next run, to capture the instant at which this block was passed out. The static function bumpserialno() in obmalloc.c is the only place the serial diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 8a989c2afe871..957934c78595e 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -44,7 +44,7 @@ Python: .. c:type:: Py_UNICODE - This is a typedef of :c:expr:`wchar_t`, which is a 16-bit type or 32-bit type + This is a typedef of :c:type:`wchar_t`, which is a 16-bit type or 32-bit type depending on the platform. .. versionchanged:: 3.3 @@ -444,11 +444,11 @@ APIs: +----------+-----------------------------------------------------+ | ``ll`` | :c:expr:`long long` or :c:expr:`unsigned long long` | +----------+-----------------------------------------------------+ - | ``j`` | :c:expr:`intmax_t` or :c:expr:`uintmax_t` | + | ``j`` | :c:type:`intmax_t` or :c:type:`uintmax_t` | +----------+-----------------------------------------------------+ - | ``z`` | :c:expr:`size_t` or :c:expr:`ssize_t` | + | ``z`` | :c:type:`size_t` or :c:type:`ssize_t` | +----------+-----------------------------------------------------+ - | ``t`` | :c:expr:`ptrdiff_t` | + | ``t`` | :c:type:`ptrdiff_t` | +----------+-----------------------------------------------------+ The length modifier ``l`` for following conversions ``s`` or ``V`` specify @@ -527,7 +527,7 @@ APIs: .. note:: The width formatter unit is number of characters rather than bytes. - The precision formatter unit is number of bytes or :c:expr:`wchar_t` + The precision formatter unit is number of bytes or :c:type:`wchar_t` items (if the length modifier ``l`` is used) for ``"%s"`` and ``"%V"`` (if the ``PyObject*`` argument is ``NULL``), and a number of characters for ``"%A"``, ``"%U"``, ``"%S"``, ``"%R"`` and ``"%V"`` @@ -846,11 +846,11 @@ conversion function: wchar_t Support """"""""""""""" -:c:expr:`wchar_t` support for platforms which support it: +:c:type:`wchar_t` support for platforms which support it: .. c:function:: PyObject* PyUnicode_FromWideChar(const wchar_t *w, Py_ssize_t size) - Create a Unicode object from the :c:expr:`wchar_t` buffer *w* of the given *size*. + Create a Unicode object from the :c:type:`wchar_t` buffer *w* of the given *size*. Passing ``-1`` as the *size* indicates that the function must itself compute the length, using wcslen. Return ``NULL`` on failure. @@ -858,9 +858,9 @@ wchar_t Support .. c:function:: Py_ssize_t PyUnicode_AsWideChar(PyObject *unicode, wchar_t *w, Py_ssize_t size) - Copy the Unicode object contents into the :c:expr:`wchar_t` buffer *w*. At most - *size* :c:expr:`wchar_t` characters are copied (excluding a possibly trailing - null termination character). Return the number of :c:expr:`wchar_t` characters + Copy the Unicode object contents into the :c:type:`wchar_t` buffer *w*. At most + *size* :c:type:`wchar_t` characters are copied (excluding a possibly trailing + null termination character). Return the number of :c:type:`wchar_t` characters copied or ``-1`` in case of an error. Note that the resulting :c:expr:`wchar_t*` string may or may not be null-terminated. It is the responsibility of the caller to make sure that the :c:expr:`wchar_t*` string is null-terminated in case this is @@ -874,7 +874,7 @@ wchar_t Support Convert the Unicode object to a wide character string. The output string always ends with a null character. If *size* is not ``NULL``, write the number of wide characters (excluding the trailing null termination character) into - *\*size*. Note that the resulting :c:expr:`wchar_t` string might contain + *\*size*. Note that the resulting :c:type:`wchar_t` string might contain null characters, which would cause the string to be truncated when used with most C functions. If *size* is ``NULL`` and the :c:expr:`wchar_t*` string contains null characters a :exc:`ValueError` is raised. diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 1e8a945512d78..56fa2d6abd913 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -17,7 +17,7 @@ parameter. The available start symbols are :c:data:`Py_eval_input`, following the functions which accept them as parameters. Note also that several of these functions take :c:expr:`FILE*` parameters. One -particular issue which needs to be handled carefully is that the :c:expr:`FILE` +particular issue which needs to be handled carefully is that the :c:type:`FILE` structure for different C libraries can be different and incompatible. Under Windows (at least), it is possible for dynamically linked extensions to actually use different libraries, so care should be taken that :c:expr:`FILE*` parameters diff --git a/Doc/conf.py b/Doc/conf.py index 485c0bdf84df2..48d43736bfbc2 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -77,6 +77,58 @@ exclude_patterns.append(venvdir + '/*') nitpick_ignore = [ + # Standard C types + ('c:type', 'FILE'), + ('c:type', '__int'), + ('c:type', 'intmax_t'), + ('c:type', 'off_t'), + ('c:type', 'ptrdiff_t'), + ('c:type', 'siginfo_t'), + ('c:type', 'size_t'), + ('c:type', 'ssize_t'), + ('c:type', 'time_t'), + ('c:type', 'uintmax_t'), + ('c:type', 'va_list'), + ('c:type', 'wchar_t'), + # Standard C macros + ('c:macro', 'LLONG_MAX'), + ('c:macro', 'LLONG_MIN'), + ('c:macro', 'LONG_MAX'), + ('c:macro', 'LONG_MIN'), + # Standard C variables + ('c:data', 'errno'), + # Standard environment variables + ('envvar', 'BROWSER'), + ('envvar', 'COLUMNS'), + ('envvar', 'COMSPEC'), + ('envvar', 'DISPLAY'), + ('envvar', 'HOME'), + ('envvar', 'HOMEDRIVE'), + ('envvar', 'HOMEPATH'), + ('envvar', 'IDLESTARTUP'), + ('envvar', 'LANG'), + ('envvar', 'LANGUAGE'), + ('envvar', 'LC_ALL'), + ('envvar', 'LC_CTYPE'), + ('envvar', 'LC_COLLATE'), + ('envvar', 'LC_MESSAGES'), + ('envvar', 'LC_MONETARY'), + ('envvar', 'LC_NUMERIC'), + ('envvar', 'LC_TIME'), + ('envvar', 'LINES'), + ('envvar', 'LOGNAME'), + ('envvar', 'PAGER'), + ('envvar', 'PATH'), + ('envvar', 'PATHEXT'), + ('envvar', 'SOURCE_DATE_EPOCH'), + ('envvar', 'TEMP'), + ('envvar', 'TERM'), + ('envvar', 'TMP'), + ('envvar', 'TMPDIR'), + ('envvar', 'TZ'), + ('envvar', 'USER'), + ('envvar', 'USERNAME'), + ('envvar', 'USERPROFILE'), # Do not error nit-picky mode builds when _SubParsersAction.add_parser cannot # be resolved, as the method is currently undocumented. For context, see # https://github.com/python/cpython/pull/103289. diff --git a/Doc/library/array.rst b/Doc/library/array.rst index 75c49e0f6d1eb..d6cb8c623adbf 100644 --- a/Doc/library/array.rst +++ b/Doc/library/array.rst @@ -51,9 +51,9 @@ Notes: It can be 16 bits or 32 bits depending on the platform. .. versionchanged:: 3.9 - ``array('u')`` now uses ``wchar_t`` as C type instead of deprecated + ``array('u')`` now uses :c:type:`wchar_t` as C type instead of deprecated ``Py_UNICODE``. This change doesn't affect its behavior because - ``Py_UNICODE`` is alias of ``wchar_t`` since Python 3.3. + ``Py_UNICODE`` is alias of :c:type:`wchar_t` since Python 3.3. .. deprecated-removed:: 3.3 4.0 diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst index 81509c0920bb6..c253a45e1a8b5 100644 --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -220,7 +220,7 @@ Fundamental data types +----------------------+------------------------------------------+----------------------------+ | :class:`c_char` | :c:expr:`char` | 1-character bytes object | +----------------------+------------------------------------------+----------------------------+ -| :class:`c_wchar` | :c:expr:`wchar_t` | 1-character string | +| :class:`c_wchar` | :c:type:`wchar_t` | 1-character string | +----------------------+------------------------------------------+----------------------------+ | :class:`c_byte` | :c:expr:`char` | int | +----------------------+------------------------------------------+----------------------------+ @@ -243,9 +243,9 @@ Fundamental data types | :class:`c_ulonglong` | :c:expr:`unsigned __int64` or | int | | | :c:expr:`unsigned long long` | | +----------------------+------------------------------------------+----------------------------+ -| :class:`c_size_t` | :c:expr:`size_t` | int | +| :class:`c_size_t` | :c:type:`size_t` | int | +----------------------+------------------------------------------+----------------------------+ -| :class:`c_ssize_t` | :c:expr:`ssize_t` or | int | +| :class:`c_ssize_t` | :c:type:`ssize_t` or | int | | | :c:expr:`Py_ssize_t` | | +----------------------+------------------------------------------+----------------------------+ | :class:`c_time_t` | :c:type:`time_t` | int | @@ -335,7 +335,7 @@ property:: The :func:`create_string_buffer` function replaces the old :func:`c_buffer` function (which is still available as an alias). To create a mutable memory -block containing unicode characters of the C type :c:expr:`wchar_t`, use the +block containing unicode characters of the C type :c:type:`wchar_t`, use the :func:`create_unicode_buffer` function. @@ -478,7 +478,7 @@ By default functions are assumed to return the C :c:expr:`int` type. Other return types can be specified by setting the :attr:`restype` attribute of the function object. -The C prototype of ``time()`` is ``time_t time(time_t *)``. Because ``time_t`` +The C prototype of ``time()`` is ``time_t time(time_t *)``. Because :c:type:`time_t` might be of a different type than the default return type ``int``, you should specify the ``restype``:: @@ -2407,7 +2407,7 @@ These are the fundamental ctypes data types: .. class:: c_wchar - Represents the C :c:expr:`wchar_t` datatype, and interprets the value as a + Represents the C :c:type:`wchar_t` datatype, and interprets the value as a single character unicode string. The constructor accepts an optional string initializer, the length of the string must be exactly one character. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 127d1616388b3..57405916ed7b3 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -4650,7 +4650,7 @@ written in Python, such as a mail server's external command delivery program. :data:`WNOHANG` and :data:`WNOWAIT` are additional optional flags. The return value is an object representing the data contained in the - :c:type:`!siginfo_t` structure with the following attributes: + :c:type:`siginfo_t` structure with the following attributes: * :attr:`!si_pid` (process ID) * :attr:`!si_uid` (real user ID of the child) diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index 6d2739b4557fb..c94dfde4d5576 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -231,9 +231,9 @@ platform-dependent. | ``Q`` | :c:expr:`unsigned long | integer | 8 | \(2) | | | long` | | | | +--------+--------------------------+--------------------+----------------+------------+ -| ``n`` | :c:expr:`ssize_t` | integer | | \(3) | +| ``n`` | :c:type:`ssize_t` | integer | | \(3) | +--------+--------------------------+--------------------+----------------+------------+ -| ``N`` | :c:expr:`size_t` | integer | | \(3) | +| ``N`` | :c:type:`size_t` | integer | | \(3) | +--------+--------------------------+--------------------+----------------+------------+ | ``e`` | \(6) | float | 2 | \(4) | +--------+--------------------------+--------------------+----------------+------------+ diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst index 9e5672545dea3..2482441d64979 100644 --- a/Doc/library/venv.rst +++ b/Doc/library/venv.rst @@ -60,7 +60,7 @@ running from a virtual environment. A virtual environment may be "activated" using a script in its binary directory (``bin`` on POSIX; ``Scripts`` on Windows). -This will prepend that directory to your :envvar:`!PATH`, so that running +This will prepend that directory to your :envvar:`PATH`, so that running :program:`python` will invoke the environment's Python interpreter and you can run installed scripts without having to use their full path. The invocation of the activation script is platform-specific @@ -100,10 +100,10 @@ In order to achieve this, scripts installed into virtual environments have a "shebang" line which points to the environment's Python interpreter, i.e. :samp:`#!/{}/bin/python`. This means that the script will run with that interpreter regardless of the -value of :envvar:`!PATH`. On Windows, "shebang" line processing is supported if +value of :envvar:`PATH`. On Windows, "shebang" line processing is supported if you have the :ref:`launcher` installed. Thus, double-clicking an installed script in a Windows Explorer window should run it with the correct interpreter -without the environment needing to be activated or on the :envvar:`!PATH`. +without the environment needing to be activated or on the :envvar:`PATH`. When a virtual environment has been activated, the :envvar:`!VIRTUAL_ENV` environment variable is set to the path of the environment. diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 66b68305c5c69..0403cb34751ca 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -100,7 +100,6 @@ Doc/library/codecs.rst Doc/library/codeop.rst Doc/library/collections.abc.rst Doc/library/collections.rst -Doc/library/compileall.rst Doc/library/concurrent.futures.rst Doc/library/concurrent.rst Doc/library/configparser.rst @@ -109,8 +108,6 @@ Doc/library/contextlib.rst Doc/library/copy.rst Doc/library/csv.rst Doc/library/ctypes.rst -Doc/library/curses.ascii.rst -Doc/library/curses.rst Doc/library/datetime.rst Doc/library/dbm.rst Doc/library/decimal.rst @@ -148,7 +145,6 @@ Doc/library/http.client.rst Doc/library/http.cookiejar.rst Doc/library/http.cookies.rst Doc/library/http.server.rst -Doc/library/idle.rst Doc/library/importlib.resources.abc.rst Doc/library/importlib.resources.rst Doc/library/importlib.rst @@ -179,11 +175,9 @@ Doc/library/pickletools.rst Doc/library/platform.rst Doc/library/plistlib.rst Doc/library/poplib.rst -Doc/library/posix.rst Doc/library/pprint.rst Doc/library/profile.rst Doc/library/pty.rst -Doc/library/py_compile.rst Doc/library/pyclbr.rst Doc/library/pydoc.rst Doc/library/pyexpat.rst @@ -206,7 +200,6 @@ Doc/library/ssl.rst Doc/library/stat.rst Doc/library/stdtypes.rst Doc/library/string.rst -Doc/library/struct.rst Doc/library/subprocess.rst Doc/library/sunau.rst Doc/library/sys.rst @@ -273,7 +266,6 @@ Doc/tutorial/modules.rst Doc/tutorial/stdlib2.rst Doc/using/cmdline.rst Doc/using/configure.rst -Doc/using/unix.rst Doc/using/windows.rst Doc/whatsnew/2.0.rst Doc/whatsnew/2.1.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index a119f77bdceb4..6d5fae1d6fcc6 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1768,7 +1768,7 @@ Porting to Python 3.12 for example). * Add support of more formatting options (left aligning, octals, uppercase - hexadecimals, ``intmax_t``, ``ptrdiff_t``, ``wchar_t`` C + hexadecimals, :c:type:`intmax_t`, :c:type:`ptrdiff_t`, :c:type:`wchar_t` C strings, variable width and precision) in :c:func:`PyUnicode_FromFormat` and :c:func:`PyUnicode_FromFormatV`. (Contributed by Serhiy Storchaka in :gh:`98836`.) diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 765ba60d28c3a..aaa4e873fddd8 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -1984,7 +1984,7 @@ the form '-rwxrwxrwx'. struct ------ -The :mod:`struct` module now supports ``ssize_t`` and ``size_t`` via the +The :mod:`struct` module now supports :c:type:`ssize_t` and :c:type:`size_t` via the new codes ``n`` and ``N``, respectively. (Contributed by Antoine Pitrou in :issue:`3163`.) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 83ce591c30ee6..d23644b5093cb 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2192,7 +2192,7 @@ encode error with ``\N{...}`` escapes. (Contributed by Serhiy Storchaka in :issue:`19676`.) A new :c:func:`PyErr_FormatV` function similar to :c:func:`PyErr_Format`, -but accepts a ``va_list`` argument. +but accepts a :c:type:`va_list` argument. (Contributed by Antoine Pitrou in :issue:`18711`.) A new :c:data:`PyExc_RecursionError` exception. diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 1f0fbbc219666..23db4022dc06a 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1115,9 +1115,9 @@ Changes in the Python API ``PyCF_ALLOW_TOP_LEVEL_AWAIT`` was clashing with ``CO_FUTURE_DIVISION``. (Contributed by Batuhan Taskaya in :issue:`39562`) -* ``array('u')`` now uses ``wchar_t`` as C type instead of ``Py_UNICODE``. +* ``array('u')`` now uses :c:type:`wchar_t` as C type instead of ``Py_UNICODE``. This change doesn't affect to its behavior because ``Py_UNICODE`` is alias - of ``wchar_t`` since Python 3.3. + of :c:type:`wchar_t` since Python 3.3. (Contributed by Inada Naoki in :issue:`34538`.) * The :func:`logging.getLogger` API now returns the root logger when passed diff --git a/Misc/NEWS.d/3.10.0a5.rst b/Misc/NEWS.d/3.10.0a5.rst index 497e384917183..dc95e8ce072fd 100644 --- a/Misc/NEWS.d/3.10.0a5.rst +++ b/Misc/NEWS.d/3.10.0a5.rst @@ -667,4 +667,4 @@ exception (if an exception is set). Patch by Victor Stinner. .. section: C API Fixed a compiler warning in :c:func:`Py_UNICODE_ISSPACE()` on platforms with -signed ``wchar_t``. +signed :c:type:`wchar_t`. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index d37ce84eadbcb..ddd4b73275ec4 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -5308,7 +5308,7 @@ parameter. Patch by Kumar Aditya. .. section: Build Python now always use the ``%zu`` and ``%zd`` printf formats to format a -``size_t`` or ``Py_ssize_t`` number. Building Python 3.12 requires a C11 +:c:type:`size_t` or ``Py_ssize_t`` number. Building Python 3.12 requires a C11 compiler, so these printf formats are now always supported. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/3.12.0b1.rst b/Misc/NEWS.d/3.12.0b1.rst index 8cd88e9b0f55e..acb68db2db912 100644 --- a/Misc/NEWS.d/3.12.0b1.rst +++ b/Misc/NEWS.d/3.12.0b1.rst @@ -2382,7 +2382,7 @@ Patch by Dong-hee Na. .. section: C API Add support of more formatting options (left aligning, octals, uppercase -hexadecimals, :c:expr:`intmax_t`, :c:expr:`ptrdiff_t`, :c:expr:`wchar_t` C +hexadecimals, :c:type:`intmax_t`, :c:type:`ptrdiff_t`, :c:type:`wchar_t` C strings, variable width and precision) in :c:func:`PyUnicode_FromFormat` and :c:func:`PyUnicode_FromFormatV`. diff --git a/Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst b/Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst new file mode 100644 index 0000000000000..42b6348153b56 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-07-21-11-51-57.gh-issue-106948.K_JQ7j.rst @@ -0,0 +1 @@ +Add a number of standard external names to ``nitpick_ignore``. From webhook-mailer at python.org Sun Jul 23 05:29:16 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:29:16 -0000 Subject: [Python-checkins] [3.11] gh-75371: reformat Makefile.pre.in to accommodate for empty FRAMEWORKALTINSTALLLAST (GH-107035) (#107050) Message-ID: https://github.com/python/cpython/commit/5a7456623fc63a6a4af58eaba5c6ab91cf9109e0 commit: 5a7456623fc63a6a4af58eaba5c6ab91cf9109e0 branch: 3.11 author: ?ukasz Langa committer: ambv date: 2023-07-23T11:29:13+02:00 summary: [3.11] gh-75371: reformat Makefile.pre.in to accommodate for empty FRAMEWORKALTINSTALLLAST (GH-107035) (#107050) in the case of an empty FRAMEWORKALTINSTALLLAST, this patch prevents leaving an astray linebreak and two tabs in the resulting Makefile. Before change: ``` .PHONY: commoninstall commoninstall: check-clean-src \ altbininstall libinstall inclinstall libainstall \ sharedinstall altmaninstall \ ``` After change (with empty FRAMEWORKALTINSTALLLAST): ``` .PHONY: commoninstall commoninstall: check-clean-src \ altbininstall libinstall inclinstall libainstall \ sharedinstall altmaninstall ``` (cherry picked from commit 9c38206925246ab919cf558ac069ae9458720ba7) Co-authored-by: Moritz Neeb files: M Makefile.pre.in diff --git a/Makefile.pre.in b/Makefile.pre.in index 3948184ad8ea3..2a407a7df697e 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1764,8 +1764,7 @@ altinstall: commoninstall commoninstall: check-clean-src @FRAMEWORKALTINSTALLFIRST@ \ altbininstall libinstall inclinstall libainstall \ - sharedinstall oldsharedinstall altmaninstall \ - @FRAMEWORKALTINSTALLLAST@ + sharedinstall oldsharedinstall altmaninstall @FRAMEWORKALTINSTALLLAST@ # Install shared libraries enabled by Setup DESTDIRS= $(exec_prefix) $(LIBDIR) $(BINLIBDEST) $(DESTSHARED) From webhook-mailer at python.org Sun Jul 23 05:34:32 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:34:32 -0000 Subject: [Python-checkins] [3.12] gh-75371: reformat Makefile.pre.in to accommodate for empty FRAMEWORKALTINSTALLLAST (GH-107035) (#107049) Message-ID: https://github.com/python/cpython/commit/ac6b0fbdb870c002633bffc21d04e621abc962fb commit: ac6b0fbdb870c002633bffc21d04e621abc962fb branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-23T11:34:28+02:00 summary: [3.12] gh-75371: reformat Makefile.pre.in to accommodate for empty FRAMEWORKALTINSTALLLAST (GH-107035) (#107049) gh-75371: reformat Makefile.pre.in to accommodate for empty FRAMEWORKALTINSTALLLAST (GH-107035) in the case of an empty FRAMEWORKALTINSTALLLAST, this patch prevents leaving an astray linebreak and two tabs in the resulting Makefile. Before change: ``` .PHONY: commoninstall commoninstall: check-clean-src \ altbininstall libinstall inclinstall libainstall \ sharedinstall altmaninstall \ ``` After change (with empty FRAMEWORKALTINSTALLLAST): ``` .PHONY: commoninstall commoninstall: check-clean-src \ altbininstall libinstall inclinstall libainstall \ sharedinstall altmaninstall ``` (cherry picked from commit 9c38206925246ab919cf558ac069ae9458720ba7) Co-authored-by: Moritz Neeb files: M Makefile.pre.in diff --git a/Makefile.pre.in b/Makefile.pre.in index 8dacb570ccafa..393eec7b8d324 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1918,8 +1918,7 @@ altinstall: commoninstall .PHONY: commoninstall commoninstall: check-clean-src @FRAMEWORKALTINSTALLFIRST@ \ altbininstall libinstall inclinstall libainstall \ - sharedinstall altmaninstall \ - @FRAMEWORKALTINSTALLLAST@ + sharedinstall altmaninstall @FRAMEWORKALTINSTALLLAST@ # Install shared libraries enabled by Setup DESTDIRS= $(exec_prefix) $(LIBDIR) $(BINLIBDEST) $(DESTSHARED) From webhook-mailer at python.org Sun Jul 23 05:35:00 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:35:00 -0000 Subject: [Python-checkins] [3.11] Convert `doc.yml` workflow to be reusable (GH-103914 + GH-105151) (#107043) Message-ID: https://github.com/python/cpython/commit/561029aeb060941ea3dabece753928e6b4674653 commit: 561029aeb060941ea3dabece753928e6b4674653 branch: 3.11 author: Sviatoslav Sydorenko committer: ambv date: 2023-07-23T11:34:56+02:00 summary: [3.11] Convert `doc.yml` workflow to be reusable (GH-103914 + GH-105151) (#107043) Co-authored-by: Sviatoslav Sydorenko Co-authored-by: Adam Turner <9087854+AA-Turner at users.noreply.github.com> Co-authored-by: Hugo van Kemenade . (cherry picked from commit 88d14da76f579fe014cbd7c15e42be4234135fe9) (cherry picked from commit eaa670228066220f08c8d73f80365c50058d40b8) files: A .github/workflows/reusable-docs.yml D .github/workflows/doc.yml M .github/workflows/build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 6848ed6f54bf0..5dc63ff64e9ca 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -26,7 +26,7 @@ permissions: contents: read concurrency: - group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} + group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}-reusable cancel-in-progress: true jobs: @@ -35,6 +35,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 outputs: + run-docs: ${{ steps.docs-changes.outputs.run-docs || false }} run_tests: ${{ steps.check.outputs.run_tests }} run_ssl_tests: ${{ steps.check.outputs.run_ssl_tests }} config_hash: ${{ steps.config_hash.outputs.hash }} @@ -68,6 +69,29 @@ jobs: id: config_hash run: | echo "hash=${{ hashFiles('configure', 'configure.ac', '.github/workflows/build.yml') }}" >> $GITHUB_OUTPUT + - name: Get a list of the changed documentation-related files + if: github.event_name == 'pull_request' + id: changed-docs-files + uses: Ana06/get-changed-files at v2.2.0 + with: + filter: | + Doc/** + Misc/** + .github/workflows/reusable-docs.yml + format: csv # works for paths with spaces + - name: Check for docs changes + if: >- + github.event_name == 'pull_request' + && steps.changed-docs-files.outputs.added_modified_renamed != '' + id: docs-changes + run: | + echo "run-docs=true" >> "${GITHUB_OUTPUT}" + + check-docs: + name: Docs + needs: check_source + if: fromJSON(needs.check_source.outputs.run-docs) + uses: ./.github/workflows/reusable-docs.yml check_abi: name: 'Check if the ABI has changed' diff --git a/.github/workflows/doc.yml b/.github/workflows/reusable-docs.yml similarity index 89% rename from .github/workflows/doc.yml rename to .github/workflows/reusable-docs.yml index e32078c67fba6..eade14bc8d619 100644 --- a/.github/workflows/doc.yml +++ b/.github/workflows/reusable-docs.yml @@ -1,29 +1,8 @@ name: Docs on: + workflow_call: workflow_dispatch: - #push: - # branches: - # - 'main' - # - '3.11' - # - '3.10' - # - '3.9' - # - '3.8' - # - '3.7' - # paths: - # - 'Doc/**' - pull_request: - branches: - - 'main' - - '3.11' - - '3.10' - - '3.9' - - '3.8' - - '3.7' - paths: - - 'Doc/**' - - 'Misc/**' - - '.github/workflows/doc.yml' permissions: contents: read From webhook-mailer at python.org Sun Jul 23 05:35:21 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:35:21 -0000 Subject: [Python-checkins] [3.12] Convert `doc.yml` workflow to be reusable (GH-103914 + GH-105151) (#107042) Message-ID: https://github.com/python/cpython/commit/1703262c0a7ee9307f7104900f024cfc5235d01d commit: 1703262c0a7ee9307f7104900f024cfc5235d01d branch: 3.12 author: Sviatoslav Sydorenko committer: ambv date: 2023-07-23T11:35:17+02:00 summary: [3.12] Convert `doc.yml` workflow to be reusable (GH-103914 + GH-105151) (#107042) Co-authored-by: Sviatoslav Sydorenko Co-authored-by: Adam Turner <9087854+AA-Turner at users.noreply.github.com> Co-authored-by: Hugo van Kemenade (cherry picked from commit 88d14da76f579fe014cbd7c15e42be4234135fe9) (cherry picked from commit eaa670228066220f08c8d73f80365c50058d40b8) files: A .github/workflows/reusable-docs.yml D .github/workflows/doc.yml M .github/workflows/build.yml M Doc/tools/touch-clean-files.py diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 197953b513f06..d6504e6ca8cf3 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -28,7 +28,7 @@ permissions: contents: read concurrency: - group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} + group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}-reusable cancel-in-progress: true jobs: @@ -37,6 +37,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 outputs: + run-docs: ${{ steps.docs-changes.outputs.run-docs || false }} run_tests: ${{ steps.check.outputs.run_tests }} run_hypothesis: ${{ steps.check.outputs.run_hypothesis }} config_hash: ${{ steps.config_hash.outputs.hash }} @@ -79,6 +80,29 @@ jobs: id: config_hash run: | echo "hash=${{ hashFiles('configure', 'configure.ac', '.github/workflows/build.yml') }}" >> $GITHUB_OUTPUT + - name: Get a list of the changed documentation-related files + if: github.event_name == 'pull_request' + id: changed-docs-files + uses: Ana06/get-changed-files at v2.2.0 + with: + filter: | + Doc/** + Misc/** + .github/workflows/reusable-docs.yml + format: csv # works for paths with spaces + - name: Check for docs changes + if: >- + github.event_name == 'pull_request' + && steps.changed-docs-files.outputs.added_modified_renamed != '' + id: docs-changes + run: | + echo "run-docs=true" >> "${GITHUB_OUTPUT}" + + check-docs: + name: Docs + needs: check_source + if: fromJSON(needs.check_source.outputs.run-docs) + uses: ./.github/workflows/reusable-docs.yml check_abi: name: 'Check if the ABI has changed' diff --git a/.github/workflows/doc.yml b/.github/workflows/reusable-docs.yml similarity index 89% rename from .github/workflows/doc.yml rename to .github/workflows/reusable-docs.yml index 3211b526efc7a..b39d8cea6421e 100644 --- a/.github/workflows/doc.yml +++ b/.github/workflows/reusable-docs.yml @@ -1,31 +1,8 @@ name: Docs on: + workflow_call: workflow_dispatch: - #push: - # branches: - # - 'main' - # - '3.12' - # - '3.11' - # - '3.10' - # - '3.9' - # - '3.8' - # - '3.7' - # paths: - # - 'Doc/**' - pull_request: - branches: - - 'main' - - '3.12' - - '3.11' - - '3.10' - - '3.9' - - '3.8' - - '3.7' - paths: - - 'Doc/**' - - 'Misc/**' - - '.github/workflows/doc.yml' permissions: contents: read @@ -61,12 +38,14 @@ jobs: uses: Ana06/get-changed-files at v2.2.0 with: filter: "Doc/**" + format: csv # works for paths with spaces - name: 'Build changed files in nit-picky mode' if: github.event_name == 'pull_request' continue-on-error: true run: | + set -Eeuo pipefail # Mark files the pull request modified - touch ${{ steps.changed_files.outputs.added_modified }} + python Doc/tools/touch-clean-files.py --clean '${{ steps.changed_files.outputs.added_modified }}' # Build docs with the '-n' (nit-picky) option; convert warnings to annotations make -C Doc/ PYTHON=../python SPHINXOPTS="-q -n --keep-going" html 2>&1 | python Doc/tools/warnings-to-gh-actions.py diff --git a/Doc/tools/touch-clean-files.py b/Doc/tools/touch-clean-files.py index 19bc1be31deb2..2b045bd68a0cf 100644 --- a/Doc/tools/touch-clean-files.py +++ b/Doc/tools/touch-clean-files.py @@ -3,7 +3,9 @@ Touch files that must pass Sphinx nit-picky mode so they are rebuilt and we can catch regressions. """ - +import argparse +import csv +import sys from pathlib import Path wrong_directory_msg = "Must run this script from the repo root" @@ -28,14 +30,33 @@ rst for rst in Path("Doc/").rglob("*.rst") if rst.parts[1] not in EXCLUDE_SUBDIRS } -with Path("Doc/tools/.nitignore").open() as clean_files: - DIRTY = { + +parser = argparse.ArgumentParser( + description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter +) +parser.add_argument("-c", "--clean", help="Comma-separated list of clean files") +args = parser.parse_args() + +if args.clean: + clean_files = next(csv.reader([args.clean])) + CLEAN = { Path(filename.strip()) for filename in clean_files - if filename.strip() and not filename.startswith("#") + if Path(filename.strip()).is_file() } - -CLEAN = ALL_RST - DIRTY - EXCLUDE_FILES +elif args.clean is not None: + print( + "Not touching any files: an empty string `--clean` arg value passed.", + ) + sys.exit(0) +else: + with Path("Doc/tools/.nitignore").open() as ignored_files: + IGNORED = { + Path(filename.strip()) + for filename in ignored_files + if filename.strip() and not filename.startswith("#") + } + CLEAN = ALL_RST - IGNORED - EXCLUDE_FILES print("Touching:") for filename in sorted(CLEAN): From webhook-mailer at python.org Sun Jul 23 05:36:13 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:36:13 -0000 Subject: [Python-checkins] [3.12] gh-54738: Add argparse i18n howto (GH-104562) (#107102) Message-ID: https://github.com/python/cpython/commit/63ae7edd3518ecc6364e51d1f2a5748d8fa36b85 commit: 63ae7edd3518ecc6364e51d1f2a5748d8fa36b85 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-23T11:36:10+02:00 summary: [3.12] gh-54738: Add argparse i18n howto (GH-104562) (#107102) (cherry picked from commit dcd7acb04a719d8d30c8d03b80d3d48b6c035e14) Co-authored-by: Tomas R files: A Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst M Doc/howto/argparse.rst M Doc/library/gettext.rst diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst index 52e98fa962019..ae5bab90bf813 100644 --- a/Doc/howto/argparse.rst +++ b/Doc/howto/argparse.rst @@ -788,6 +788,59 @@ but not both at the same time: -q, --quiet +How to translate the argparse output +==================================== + +The output of the :mod:`argparse` module such as its help text and error +messages are all made translatable using the :mod:`gettext` module. This +allows applications to easily localize messages produced by +:mod:`argparse`. See also :ref:`i18n-howto`. + +For instance, in this :mod:`argparse` output: + +.. code-block:: shell-session + + $ python prog.py --help + usage: prog.py [-h] [-v | -q] x y + + calculate X to the power of Y + + positional arguments: + x the base + y the exponent + + options: + -h, --help show this help message and exit + -v, --verbose + -q, --quiet + +The strings ``usage:``, ``positional arguments:``, ``options:`` and +``show this help message and exit`` are all translatable. + +In order to translate these strings, they must first be extracted +into a ``.po`` file. For example, using `Babel `__, +run this command: + +.. code-block:: shell-session + + $ pybabel extract -o messages.po /usr/lib/python3.12/argparse.py + +This command will extract all translatable strings from the :mod:`argparse` +module and output them into a file named ``messages.po``. This command assumes +that your Python installation is in ``/usr/lib``. + +You can find out the location of the :mod:`argparse` module on your system +using this script:: + + import argparse + print(argparse.__file__) + +Once the messages in the ``.po`` file are translated and the translations are +installed using :mod:`gettext`, :mod:`argparse` will be able to display the +translated messages. + +To translate your own strings in the :mod:`argparse` output, use :mod:`gettext`. + Conclusion ========== diff --git a/Doc/library/gettext.rst b/Doc/library/gettext.rst index 747f8703b750e..88a65b980d310 100644 --- a/Doc/library/gettext.rst +++ b/Doc/library/gettext.rst @@ -411,6 +411,7 @@ One difference between this module and Henstridge's: his catalog objects supported access through a mapping API, but this appears to be unused and so is not currently supported. +.. _i18n-howto: Internationalizing your programs and modules -------------------------------------------- diff --git a/Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst b/Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst new file mode 100644 index 0000000000000..4da58fc982b6d --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-05-16-22-08-24.gh-issue-54738.mJvCnj.rst @@ -0,0 +1 @@ +Add documentation on how to localize the :mod:`argparse` module. From webhook-mailer at python.org Sun Jul 23 05:41:38 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 09:41:38 -0000 Subject: [Python-checkins] gh-106976: alphabetise bullets by module name task2-3 (#107005) Message-ID: https://github.com/python/cpython/commit/102a773716981c59ee7469e65e4528f7ab5e9c47 commit: 102a773716981c59ee7469e65e4528f7ab5e9c47 branch: main author: littlebutt's workshop committer: hugovk date: 2023-07-23T09:41:34Z summary: gh-106976: alphabetise bullets by module name task2-3 (#107005) Co-authored-by: Hugo van Kemenade files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index ea9806987f687..f043ec764ac55 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -966,67 +966,76 @@ Demos and Tools Deprecated ========== -* :class:`typing.Hashable` and :class:`typing.Sized` aliases for :class:`collections.abc.Hashable` - and :class:`collections.abc.Sized`. (:gh:`94309`.) - -* The :mod:`sqlite3` :ref:`default adapters and converters - ` are now deprecated. - Instead, use the :ref:`sqlite3-adapter-converter-recipes` - and tailor them to your needs. - (Contributed by Erlend E. Aasland in :gh:`90016`.) - -* In :meth:`~sqlite3.Cursor.execute`, :exc:`DeprecationWarning` is now emitted - when :ref:`named placeholders ` are used together with - parameters supplied as a :term:`sequence` instead of as a :class:`dict`. - Starting from Python 3.14, using named placeholders with parameters supplied - as a sequence will raise a :exc:`~sqlite3.ProgrammingError`. - (Contributed by Erlend E. Aasland in :gh:`101698`.) - -* The 3-arg signatures (type, value, traceback) of :meth:`~coroutine.throw`, - :meth:`~generator.throw` and :meth:`~agen.athrow` are deprecated and - may be removed in a future version of Python. Use the single-arg versions - of these functions instead. (Contributed by Ofey Chan in :gh:`89874`.) - -* :exc:`DeprecationWarning` is now raised when ``__package__`` on a - module differs from ``__spec__.parent`` (previously it was - :exc:`ImportWarning`). - (Contributed by Brett Cannon in :gh:`65961`.) - -* The :meth:`~asyncio.get_event_loop` method of the +* :mod:`asyncio`: The :meth:`~asyncio.get_event_loop` method of the default event loop policy now emits a :exc:`DeprecationWarning` if there is no current event loop set and it decides to create one. (Contributed by Serhiy Storchaka and Guido van Rossum in :gh:`100160`.) -* The :mod:`xml.etree.ElementTree` module now emits :exc:`DeprecationWarning` - when testing the truth value of an :class:`xml.etree.ElementTree.Element`. - Before, the Python implementation emitted :exc:`FutureWarning`, and the C - implementation emitted nothing. +* :mod:`calendar`: ``calendar.January`` and ``calendar.February`` constants are deprecated and + replaced by :data:`calendar.Month.JANUARY` and :data:`calendar.Month.FEBRUARY`. + (Contributed by Prince Roshan in :gh:`103636`.) -* In accordance with :pep:`699`, the ``ma_version_tag`` field in :c:type:`PyDictObject` - is deprecated for extension modules. Accessing this field will generate a compiler - warning at compile time. This field will be removed in Python 3.14. - (Contributed by Ramvikrams and Kumar Aditya in :gh:`101193`. PEP by Ken Jin.) +* :mod:`datetime`: :class:`datetime.datetime`'s :meth:`~datetime.datetime.utcnow` and + :meth:`~datetime.datetime.utcfromtimestamp` are deprecated and will be + removed in a future version. Instead, use timezone-aware objects to represent + datetimes in UTC: respectively, call :meth:`~datetime.datetime.now` and + :meth:`~datetime.datetime.fromtimestamp` with the *tz* parameter set to + :const:`datetime.UTC`. + (Contributed by Paul Ganssle in :gh:`103857`.) -* The ``st_ctime`` fields return by :func:`os.stat` and :func:`os.lstat` on +* :mod:`os`: The ``st_ctime`` fields return by :func:`os.stat` and :func:`os.lstat` on Windows are deprecated. In a future release, they will contain the last metadata change time, consistent with other platforms. For now, they still contain the creation time, which is also available in the new ``st_birthtime`` field. (Contributed by Steve Dower in :gh:`99726`.) -* The :data:`sys.last_type`, :data:`sys.last_value` and :data:`sys.last_traceback` +* :mod:`shutil`: The *onerror* argument of :func:`shutil.rmtree` is deprecated as will be removed + in Python 3.14. Use *onexc* instead. (Contributed by Irit Katriel in :gh:`102828`.) + +* :mod:`sqlite3`: + * :ref:`default adapters and converters + ` are now deprecated. + Instead, use the :ref:`sqlite3-adapter-converter-recipes` + and tailor them to your needs. + (Contributed by Erlend E. Aasland in :gh:`90016`.) + + * In :meth:`~sqlite3.Cursor.execute`, :exc:`DeprecationWarning` is now emitted + when :ref:`named placeholders ` are used together with + parameters supplied as a :term:`sequence` instead of as a :class:`dict`. + Starting from Python 3.14, using named placeholders with parameters supplied + as a sequence will raise a :exc:`~sqlite3.ProgrammingError`. + (Contributed by Erlend E. Aasland in :gh:`101698`.) + +* :mod:`sys`: The :data:`sys.last_type`, :data:`sys.last_value` and :data:`sys.last_traceback` fields are deprecated. Use :data:`sys.last_exc` instead. (Contributed by Irit Katriel in :gh:`102778`.) -* The *onerror* argument of :func:`shutil.rmtree` is deprecated as will be removed - in Python 3.14. Use *onexc* instead. (Contributed by Irit Katriel in :gh:`102828`.) - -* Extracting tar archives without specifying *filter* is deprecated until +* :mod:`tarfile`: Extracting tar archives without specifying *filter* is deprecated until Python 3.14, when ``'data'`` filter will become the default. See :ref:`tarfile-extraction-filter` for details. -* ``calendar.January`` and ``calendar.February`` constants are deprecated and - replaced by :data:`calendar.Month.JANUARY` and :data:`calendar.Month.FEBRUARY`. - (Contributed by Prince Roshan in :gh:`103636`.) +* :mod:`typing`: :class:`typing.Hashable` and :class:`typing.Sized` aliases for :class:`collections.abc.Hashable` + and :class:`collections.abc.Sized`. (:gh:`94309`.) + +* :mod:`xml.etree.ElementTree`: The module now emits :exc:`DeprecationWarning` + when testing the truth value of an :class:`xml.etree.ElementTree.Element`. + Before, the Python implementation emitted :exc:`FutureWarning`, and the C + implementation emitted nothing. + +* The 3-arg signatures (type, value, traceback) of :meth:`~coroutine.throw`, + :meth:`~generator.throw` and :meth:`~agen.athrow` are deprecated and + may be removed in a future version of Python. Use the single-arg versions + of these functions instead. (Contributed by Ofey Chan in :gh:`89874`.) + +* :exc:`DeprecationWarning` is now raised when ``__package__`` on a + module differs from ``__spec__.parent`` (previously it was + :exc:`ImportWarning`). + (Contributed by Brett Cannon in :gh:`65961`.) + +* In accordance with :pep:`699`, the ``ma_version_tag`` field in :c:type:`PyDictObject` + is deprecated for extension modules. Accessing this field will generate a compiler + warning at compile time. This field will be removed in Python 3.14. + (Contributed by Ramvikrams and Kumar Aditya in :gh:`101193`. PEP by Ken Jin.) * The bitwise inversion operator (``~``) on bool is deprecated. It will throw an error in Python 3.14. Use ``not`` for logical negation of bools instead. @@ -1034,16 +1043,6 @@ Deprecated ``int``, convert to int explicitly with ``~int(x)``. (Contributed by Tim Hoffmann in :gh:`103487`.) -* :class:`datetime.datetime`'s - :meth:`~datetime.datetime.utcnow` and - :meth:`~datetime.datetime.utcfromtimestamp` are deprecated and will be - removed in a future version. Instead, use timezone-aware objects to represent - datetimes in UTC: respectively, call - :meth:`~datetime.datetime.now` and - :meth:`~datetime.datetime.fromtimestamp` with the *tz* parameter set to - :const:`datetime.UTC`. - (Contributed by Paul Ganssle in :gh:`103857`.) - Pending Removal in Python 3.13 ------------------------------ @@ -1088,7 +1087,33 @@ APIs: Pending Removal in Python 3.14 ------------------------------ -* Deprecated the following :mod:`importlib.abc` classes, scheduled for removal in +* :mod:`argparse`: The *type*, *choices*, and *metavar* parameters + of :class:`!argparse.BooleanOptionalAction` are deprecated + and will be removed in 3.14. + (Contributed by Nikita Sobolev in :gh:`92248`.) + +* :mod:`ast`: The following :mod:`ast` features have been deprecated in documentation since + Python 3.8, now cause a :exc:`DeprecationWarning` to be emitted at runtime + when they are accessed or used, and will be removed in Python 3.14: + + * :class:`!ast.Num` + * :class:`!ast.Str` + * :class:`!ast.Bytes` + * :class:`!ast.NameConstant` + * :class:`!ast.Ellipsis` + + Use :class:`ast.Constant` instead. + (Contributed by Serhiy Storchaka in :gh:`90953`.) + +* :mod:`collections.abc`: Deprecated :class:`collections.abc.ByteString`. + Prefer :class:`Sequence` or :class:`collections.abc.Buffer`. + For use in typing, prefer a union, like ``bytes | bytearray``, or :class:`collections.abc.Buffer`. + (Contributed by Shantanu Jain in :gh:`91896`.) + +* :mod:`email`: Deprecated the *isdst* parameter in :func:`email.utils.localtime`. + (Contributed by Alan Williams in :gh:`72346`.) + +* :mod:`importlib.abc`: Deprecated the following classes, scheduled for removal in Python 3.14: * :class:`!importlib.abc.ResourceReader` @@ -1102,27 +1127,13 @@ Pending Removal in Python 3.14 (Contributed by Jason R. Coombs and Hugo van Kemenade in :gh:`93963`.) -* Deprecated :class:`collections.abc.ByteString`. - Prefer :class:`Sequence` or :class:`collections.abc.Buffer`. - For use in typing, prefer a union, like ``bytes | bytearray``, or :class:`collections.abc.Buffer`. - (Contributed by Shantanu Jain in :gh:`91896`.) - -* :class:`typing.ByteString`, deprecated since Python 3.9, now causes a - :exc:`DeprecationWarning` to be emitted when it is used. - -* Creating immutable types (:c:macro:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable - bases using the C API. - -* Deprecated the *isdst* parameter in :func:`email.utils.localtime`. - (Contributed by Alan Williams in :gh:`72346`.) - -* ``__package__`` and ``__cached__`` will cease to be set or taken - into consideration by the import system (:gh:`97879`). - -* Testing the truth value of an :class:`xml.etree.ElementTree.Element` - is deprecated and will raise an exception in Python 3.14. +* :mod:`itertools`: The module had undocumented, inefficient, historically buggy, + and inconsistent support for copy, deepcopy, and pickle operations. + This will be removed in 3.14 for a significant reduction in code + volume and maintenance burden. + (Contributed by Raymond Hettinger in :gh:`101588`.) -* The default :mod:`multiprocessing` start method will change to a safer one on +* :mod:`multiprocessing`: The default :mod:`multiprocessing` start method will change to a safer one on Linux, BSDs, and other non-macOS POSIX platforms where ``'fork'`` is currently the default (:gh:`84559`). Adding a runtime warning about this was deemed too disruptive as the majority of code is not expected to care. Use the @@ -1130,47 +1141,35 @@ Pending Removal in Python 3.14 :func:`~multiprocessing.set_start_method` APIs to explicitly specify when your code *requires* ``'fork'``. See :ref:`multiprocessing-start-methods`. -* :mod:`pty` has two undocumented ``master_open()`` and ``slave_open()`` +* :mod:`pkgutil`: :func:`pkgutil.find_loader` and :func:`pkgutil.get_loader` + now raise :exc:`DeprecationWarning`; + use :func:`importlib.util.find_spec` instead. + (Contributed by Nikita Sobolev in :gh:`97850`.) + +* :mod:`pty`: The module has two undocumented ``master_open()`` and ``slave_open()`` functions that have been deprecated since Python 2 but only gained a proper :exc:`DeprecationWarning` in 3.12. Remove them in 3.14. -* :mod:`itertools` had undocumented, inefficient, historically buggy, - and inconsistent support for copy, deepcopy, and pickle operations. - This will be removed in 3.14 for a significant reduction in code - volume and maintenance burden. - (Contributed by Raymond Hettinger in :gh:`101588`.) - -* Accessing ``co_lnotab`` was deprecated in :pep:`626` since 3.10 - and was planned to be removed in 3.12 - but it only got a proper :exc:`DeprecationWarning` in 3.12. - May be removed in 3.14. - (Contributed by Nikita Sobolev in :gh:`101866`.) - -* The *onerror* argument of :func:`shutil.rmtree` is deprecated in 3.12, +* :mod:`shutil`: The *onerror* argument of :func:`shutil.rmtree` is deprecated in 3.12, and will be removed in 3.14. -* The *type*, *choices*, and *metavar* parameters - of :class:`!argparse.BooleanOptionalAction` are deprecated - and will be removed in 3.14. - (Contributed by Nikita Sobolev in :gh:`92248`.) +* :mod:`typing`: :class:`typing.ByteString`, deprecated since Python 3.9, now causes a + :exc:`DeprecationWarning` to be emitted when it is used. -* :func:`pkgutil.find_loader` and :func:`pkgutil.get_loader` - now raise :exc:`DeprecationWarning`; - use :func:`importlib.util.find_spec` instead. - (Contributed by Nikita Sobolev in :gh:`97850`.) +* :mod:`xml.etree.ElementTree`: Testing the truth value of an :class:`xml.etree.ElementTree.Element` + is deprecated and will raise an exception in Python 3.14. -* The following :mod:`ast` features have been deprecated in documentation since - Python 3.8, now cause a :exc:`DeprecationWarning` to be emitted at runtime - when they are accessed or used, and will be removed in Python 3.14: +* Creating immutable types (:c:macro:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable + bases using the C API. - * :class:`!ast.Num` - * :class:`!ast.Str` - * :class:`!ast.Bytes` - * :class:`!ast.NameConstant` - * :class:`!ast.Ellipsis` +* ``__package__`` and ``__cached__`` will cease to be set or taken + into consideration by the import system (:gh:`97879`). - Use :class:`ast.Constant` instead. - (Contributed by Serhiy Storchaka in :gh:`90953`.) +* Accessing ``co_lnotab`` was deprecated in :pep:`626` since 3.10 + and was planned to be removed in 3.12 + but it only got a proper :exc:`DeprecationWarning` in 3.12. + May be removed in 3.14. + (Contributed by Nikita Sobolev in :gh:`101866`.) Pending Removal in Future Versions ---------------------------------- @@ -1193,13 +1192,29 @@ although there is currently no date scheduled for their removal. Removed ======= -* Remove the ``distutils`` package. It was deprecated in Python 3.10 by +* ``asynchat`` and ``asyncore``: These two modules have been removed + according to the schedule in :pep:`594`, + having been deprecated in Python 3.6. + Use :mod:`asyncio` instead. + (Contributed by Nikita Sobolev in :gh:`96580`.) + +* :mod:`configparser`: Several names deprecated in the :mod:`configparser` way back in 3.2 have + been removed per :gh:`89336`: + + * :class:`configparser.ParsingError` no longer has a ``filename`` attribute + or argument. Use the ``source`` attribute and argument instead. + * :mod:`configparser` no longer has a ``SafeConfigParser`` class. Use the + shorter :class:`~configparser.ConfigParser` name instead. + * :class:`configparser.ConfigParser` no longer has a ``readfp`` method. + Use :meth:`~configparser.ConfigParser.read_file` instead. + +* ``distutils``: Remove the ``distutils`` package. It was deprecated in Python 3.10 by :pep:`632` "Deprecate distutils module". For projects still using ``distutils`` and cannot be updated to something else, the ``setuptools`` project can be installed: it still provides ``distutils``. (Contributed by Victor Stinner in :gh:`92584`.) -* Remove the bundled setuptools wheel from :mod:`ensurepip`, +* :mod:`ensurepip`: Remove the bundled setuptools wheel from :mod:`ensurepip`, and stop installing setuptools in environments created by :mod:`venv`. ``pip (>= 22.1)`` does not require setuptools to be installed in the @@ -1217,94 +1232,9 @@ Removed (Contributed by Pradyun Gedam in :gh:`95299`.) -* Removed many old deprecated :mod:`unittest` features: - - - A number of :class:`~unittest.TestCase` method aliases: - - ============================ =============================== =============== - Deprecated alias Method Name Deprecated in - ============================ =============================== =============== - ``failUnless`` :meth:`.assertTrue` 3.1 - ``failIf`` :meth:`.assertFalse` 3.1 - ``failUnlessEqual`` :meth:`.assertEqual` 3.1 - ``failIfEqual`` :meth:`.assertNotEqual` 3.1 - ``failUnlessAlmostEqual`` :meth:`.assertAlmostEqual` 3.1 - ``failIfAlmostEqual`` :meth:`.assertNotAlmostEqual` 3.1 - ``failUnlessRaises`` :meth:`.assertRaises` 3.1 - ``assert_`` :meth:`.assertTrue` 3.2 - ``assertEquals`` :meth:`.assertEqual` 3.2 - ``assertNotEquals`` :meth:`.assertNotEqual` 3.2 - ``assertAlmostEquals`` :meth:`.assertAlmostEqual` 3.2 - ``assertNotAlmostEquals`` :meth:`.assertNotAlmostEqual` 3.2 - ``assertRegexpMatches`` :meth:`.assertRegex` 3.2 - ``assertRaisesRegexp`` :meth:`.assertRaisesRegex` 3.2 - ``assertNotRegexpMatches`` :meth:`.assertNotRegex` 3.5 - ============================ =============================== =============== - - You can use https://github.com/isidentical/teyit to automatically modernise - your unit tests. - - - Undocumented and broken :class:`~unittest.TestCase` method - ``assertDictContainsSubset`` (deprecated in Python 3.2). - - - Undocumented :meth:`TestLoader.loadTestsFromModule - ` parameter *use_load_tests* - (deprecated and ignored since Python 3.2). - - - An alias of the :class:`~unittest.TextTestResult` class: - ``_TextTestResult`` (deprecated in Python 3.2). - - (Contributed by Serhiy Storchaka in :issue:`45162`.) - -* Several names deprecated in the :mod:`configparser` way back in 3.2 have - been removed per :gh:`89336`: - - * :class:`configparser.ParsingError` no longer has a ``filename`` attribute - or argument. Use the ``source`` attribute and argument instead. - * :mod:`configparser` no longer has a ``SafeConfigParser`` class. Use the - shorter :class:`~configparser.ConfigParser` name instead. - * :class:`configparser.ConfigParser` no longer has a ``readfp`` method. - Use :meth:`~configparser.ConfigParser.read_file` instead. - -* The following undocumented :mod:`sqlite3` features, deprecated in Python - 3.10, are now removed: - - * ``sqlite3.enable_shared_cache()`` - * ``sqlite3.OptimizedUnicode`` - - If a shared cache must be used, open the database in URI mode using the - ``cache=shared`` query parameter. - - The ``sqlite3.OptimizedUnicode`` text factory has been an alias for - :class:`str` since Python 3.3. Code that previously set the text factory to - ``OptimizedUnicode`` can either use ``str`` explicitly, or rely on the - default value which is also ``str``. - - (Contributed by Erlend E. Aasland in :gh:`92548`.) - -* ``smtpd`` has been removed according to the schedule in :pep:`594`, - having been deprecated in Python 3.4.7 and 3.5.4. - Use aiosmtpd_ PyPI module or any other - :mod:`asyncio`-based server instead. - (Contributed by Oleg Iarygin in :gh:`93243`.) - -.. _aiosmtpd: https://pypi.org/project/aiosmtpd/ - -* ``asynchat`` and ``asyncore`` have been removed - according to the schedule in :pep:`594`, - having been deprecated in Python 3.6. - Use :mod:`asyncio` instead. - (Contributed by Nikita Sobolev in :gh:`96580`.) - -* Remove ``io.OpenWrapper`` and ``_pyio.OpenWrapper``, deprecated in Python - 3.10: just use :func:`open` instead. The :func:`open` (:func:`io.open`) - function is a built-in function. Since Python 3.10, :func:`!_pyio.open` is - also a static method. - (Contributed by Victor Stinner in :gh:`94169`.) - -* Remove the :func:`!ssl.RAND_pseudo_bytes` function, deprecated in Python 3.6: - use :func:`os.urandom` or :func:`ssl.RAND_bytes` instead. - (Contributed by Victor Stinner in :gh:`94199`.) +* :mod:`ftplib`: Remove the ``FTP_TLS.ssl_version`` class attribute: use the + *context* parameter instead. + (Contributed by Victor Stinner in :gh:`94172`.) * :mod:`gzip`: Remove the ``filename`` attribute of :class:`gzip.GzipFile`, deprecated since Python 2.6, use the :attr:`~gzip.GzipFile.name` attribute @@ -1312,43 +1242,13 @@ Removed extension if it was not present. (Contributed by Victor Stinner in :gh:`94196`.) -* Remove the :func:`!ssl.match_hostname` function. - It was deprecated in Python 3.7. OpenSSL performs - hostname matching since Python 3.7, Python no longer uses the - :func:`!ssl.match_hostname` function. - (Contributed by Victor Stinner in :gh:`94199`.) - -* Remove the :func:`!locale.format` function, deprecated in Python 3.7: - use :func:`locale.format_string` instead. - (Contributed by Victor Stinner in :gh:`94226`.) - * :mod:`hashlib`: Remove the pure Python implementation of :func:`hashlib.pbkdf2_hmac()`, deprecated in Python 3.10. Python 3.10 and newer requires OpenSSL 1.1.1 (:pep:`644`): this OpenSSL version provides a C implementation of :func:`~hashlib.pbkdf2_hmac()` which is faster. (Contributed by Victor Stinner in :gh:`94199`.) -* :mod:`xml.etree.ElementTree`: Remove the ``ElementTree.Element.copy()`` method of the - pure Python implementation, deprecated in Python 3.10, use the - :func:`copy.copy` function instead. The C implementation of :mod:`xml.etree.ElementTree` - has no ``copy()`` method, only a ``__copy__()`` method. - (Contributed by Victor Stinner in :gh:`94383`.) - -* :mod:`zipimport`: Remove ``find_loader()`` and ``find_module()`` methods, - deprecated in Python 3.10: use the ``find_spec()`` method instead. See - :pep:`451` for the rationale. - (Contributed by Victor Stinner in :gh:`94379`.) - -* Remove the :func:`!ssl.wrap_socket` function, deprecated in Python 3.7: - instead, create a :class:`ssl.SSLContext` object and call its - :class:`ssl.SSLContext.wrap_socket` method. Any package that still uses - :func:`!ssl.wrap_socket` is broken and insecure. The function neither sends a - SNI TLS extension nor validates server hostname. Code is subject to `CWE-295 - `_: Improper Certificate - Validation. - (Contributed by Victor Stinner in :gh:`94199`.) - -* Many previously deprecated cleanups in :mod:`importlib` have now been +* :mod:`importlib`: Many previously deprecated cleanups in :mod:`importlib` have now been completed: * References to, and support for :meth:`!module_repr()` has been removed. @@ -1413,6 +1313,115 @@ Removed ``PY_COMPILED``, ``C_EXTENSION``, ``PY_RESOURCE``, ``PKG_DIRECTORY``, ``C_BUILTIN``, ``PY_FROZEN``, ``PY_CODERESOURCE``, ``IMP_HOOK``. +* :mod:`io`: Remove ``io.OpenWrapper`` and ``_pyio.OpenWrapper``, deprecated in Python + 3.10: just use :func:`open` instead. The :func:`open` (:func:`io.open`) + function is a built-in function. Since Python 3.10, :func:`!_pyio.open` is + also a static method. + (Contributed by Victor Stinner in :gh:`94169`.) + +* :mod:`locale`: Remove the :func:`!locale.format` function, deprecated in Python 3.7: + use :func:`locale.format_string` instead. + (Contributed by Victor Stinner in :gh:`94226`.) + +* ``smtpd``: The module has been removed according to the schedule in :pep:`594`, + having been deprecated in Python 3.4.7 and 3.5.4. + Use aiosmtpd_ PyPI module or any other + :mod:`asyncio`-based server instead. + (Contributed by Oleg Iarygin in :gh:`93243`.) + +.. _aiosmtpd: https://pypi.org/project/aiosmtpd/ + +* :mod:`sqlite3`: The following undocumented :mod:`sqlite3` features, deprecated in Python + 3.10, are now removed: + + * ``sqlite3.enable_shared_cache()`` + * ``sqlite3.OptimizedUnicode`` + + If a shared cache must be used, open the database in URI mode using the + ``cache=shared`` query parameter. + + The ``sqlite3.OptimizedUnicode`` text factory has been an alias for + :class:`str` since Python 3.3. Code that previously set the text factory to + ``OptimizedUnicode`` can either use ``str`` explicitly, or rely on the + default value which is also ``str``. + + (Contributed by Erlend E. Aasland in :gh:`92548`.) + +* :mod:`ssl`: + + * Remove the :func:`!ssl.RAND_pseudo_bytes` function, deprecated in Python 3.6: + use :func:`os.urandom` or :func:`ssl.RAND_bytes` instead. + (Contributed by Victor Stinner in :gh:`94199`.) + + * Remove the :func:`!ssl.match_hostname` function. + It was deprecated in Python 3.7. OpenSSL performs + hostname matching since Python 3.7, Python no longer uses the + :func:`!ssl.match_hostname` function. + (Contributed by Victor Stinner in :gh:`94199`.) + + * Remove the :func:`!ssl.wrap_socket` function, deprecated in Python 3.7: + instead, create a :class:`ssl.SSLContext` object and call its + :class:`ssl.SSLContext.wrap_socket` method. Any package that still uses + :func:`!ssl.wrap_socket` is broken and insecure. The function neither sends a + SNI TLS extension nor validates server hostname. Code is subject to `CWE-295 + `_: Improper Certificate + Validation. + (Contributed by Victor Stinner in :gh:`94199`.) + +* :mod:`unittest`: Removed many old deprecated :mod:`unittest` features: + + - A number of :class:`~unittest.TestCase` method aliases: + + ============================ =============================== =============== + Deprecated alias Method Name Deprecated in + ============================ =============================== =============== + ``failUnless`` :meth:`.assertTrue` 3.1 + ``failIf`` :meth:`.assertFalse` 3.1 + ``failUnlessEqual`` :meth:`.assertEqual` 3.1 + ``failIfEqual`` :meth:`.assertNotEqual` 3.1 + ``failUnlessAlmostEqual`` :meth:`.assertAlmostEqual` 3.1 + ``failIfAlmostEqual`` :meth:`.assertNotAlmostEqual` 3.1 + ``failUnlessRaises`` :meth:`.assertRaises` 3.1 + ``assert_`` :meth:`.assertTrue` 3.2 + ``assertEquals`` :meth:`.assertEqual` 3.2 + ``assertNotEquals`` :meth:`.assertNotEqual` 3.2 + ``assertAlmostEquals`` :meth:`.assertAlmostEqual` 3.2 + ``assertNotAlmostEquals`` :meth:`.assertNotAlmostEqual` 3.2 + ``assertRegexpMatches`` :meth:`.assertRegex` 3.2 + ``assertRaisesRegexp`` :meth:`.assertRaisesRegex` 3.2 + ``assertNotRegexpMatches`` :meth:`.assertNotRegex` 3.5 + ============================ =============================== =============== + + You can use https://github.com/isidentical/teyit to automatically modernise + your unit tests. + + - Undocumented and broken :class:`~unittest.TestCase` method + ``assertDictContainsSubset`` (deprecated in Python 3.2). + + - Undocumented :meth:`TestLoader.loadTestsFromModule + ` parameter *use_load_tests* + (deprecated and ignored since Python 3.2). + + - An alias of the :class:`~unittest.TextTestResult` class: + ``_TextTestResult`` (deprecated in Python 3.2). + + (Contributed by Serhiy Storchaka in :issue:`45162`.) + +* :mod:`webbrowser`: Remove support for obsolete browsers from :mod:`webbrowser`. + Removed browsers include: Grail, Mosaic, Netscape, Galeon, Skipstone, + Iceape, Firebird, and Firefox versions 35 and below (:gh:`102871`). + +* :mod:`xml.etree.ElementTree`: Remove the ``ElementTree.Element.copy()`` method of the + pure Python implementation, deprecated in Python 3.10, use the + :func:`copy.copy` function instead. The C implementation of :mod:`xml.etree.ElementTree` + has no ``copy()`` method, only a ``__copy__()`` method. + (Contributed by Victor Stinner in :gh:`94383`.) + +* :mod:`zipimport`: Remove ``find_loader()`` and ``find_module()`` methods, + deprecated in Python 3.10: use the ``find_spec()`` method instead. See + :pep:`451` for the rationale. + (Contributed by Victor Stinner in :gh:`94379`.) + * Removed the ``suspicious`` rule from the documentation Makefile, and removed ``Doc/tools/rstlint.py``, both in favor of `sphinx-lint `_. @@ -1426,15 +1435,6 @@ Removed (*ssl_context* in :mod:`imaplib`) instead. (Contributed by Victor Stinner in :gh:`94172`.) -* :mod:`ftplib`: Remove the ``FTP_TLS.ssl_version`` class attribute: use the - *context* parameter instead. - (Contributed by Victor Stinner in :gh:`94172`.) - -* Remove support for obsolete browsers from :mod:`webbrowser`. - Removed browsers include: Grail, Mosaic, Netscape, Galeon, Skipstone, - Iceape, Firebird, and Firefox versions 35 and below (:gh:`102871`). - - .. _whatsnew312-porting-to-python312: Porting to Python 3.12 From webhook-mailer at python.org Sun Jul 23 05:43:31 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 09:43:31 -0000 Subject: [Python-checkins] gh-106996: Add the basics of a turtle graphics tutorial (#107072) Message-ID: https://github.com/python/cpython/commit/a2a0e51400685d8d44324bc1135f63b281a5a71d commit: a2a0e51400685d8d44324bc1135f63b281a5a71d branch: main author: Daniele Procida committer: hugovk date: 2023-07-23T03:43:27-06:00 summary: gh-106996: Add the basics of a turtle graphics tutorial (#107072) Co-authored-by: Hugo van Kemenade files: D Doc/includes/turtle-star.py M Doc/library/turtle.rst diff --git a/Doc/includes/turtle-star.py b/Doc/includes/turtle-star.py deleted file mode 100644 index 1a5db761b3238..0000000000000 --- a/Doc/includes/turtle-star.py +++ /dev/null @@ -1,10 +0,0 @@ -from turtle import * -color('red', 'yellow') -begin_fill() -while True: - forward(200) - left(170) - if abs(pos()) < 1: - break -end_fill() -done() diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index f14a677b7dd88..58ac546e75860 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -24,6 +24,23 @@ introduced in Logo `_, developed by Wally Feurzeig, Seymour Papert and Cynthia Solomon in 1967. + +Get started +=========== + +Imagine a robotic turtle starting at (0, 0) in the x-y plane. After an ``import turtle``, give it the +command ``turtle.forward(15)``, and it moves (on-screen!) 15 pixels in the +direction it is facing, drawing a line as it moves. Give it the command +``turtle.right(25)``, and it rotates in-place 25 degrees clockwise. + +.. sidebar:: Turtle star + + Turtle can draw intricate shapes using programs that repeat simple + moves. + + .. image:: turtle-star.* + :align: center + In Python, turtle graphics provides a representation of a physical "turtle" (a little robot with a pen) that draws on a sheet of paper on the floor. @@ -38,26 +55,127 @@ graphical output it can be a way to do that without the overhead of introducing more complex or external libraries into their work. -Get started -=========== +Tutorial +======== -Imagine a robotic turtle starting at (0, 0) in the x-y plane. After an ``import turtle``, give it the -command ``turtle.forward(15)``, and it moves (on-screen!) 15 pixels in the -direction it is facing, drawing a line as it moves. Give it the command -``turtle.right(25)``, and it rotates in-place 25 degrees clockwise. +New users should start here. In this tutorial we'll explore some of the +basics of turtle drawing. -.. sidebar:: Turtle star - Turtle can draw intricate shapes using programs that repeat simple - moves. +Starting a turtle environment +----------------------------- - .. image:: turtle-star.* - :align: center +In a Python shell, import all the objects of the ``turtle`` module:: + + from turtle import * + +If you run into a ``No module named '_tkinter'`` error, you'll have to +install the :mod:`Tk interface package ` on your system. + + +Basic drawing +------------- + +Send the turtle forward 100 steps:: + + forward(100) + +You should see (most likely, in a new window on your display) a line +drawn by the turtle, heading East. Change the direction of the turtle, +so that it turns 120 degrees left (anti-clockwise):: - .. literalinclude:: ../includes/turtle-star.py + left(120) -By combining together these and similar commands, intricate shapes and pictures -can easily be drawn. +Let's continue by drawing a triangle:: + + forward(100) + left(120) + forward(100) + +Notice how the turtle, represented by an arrow, points in different +directions as you steer it. + +Experiment with those commands, and also with ``backward()`` and +``right()``. + + +Pen control +~~~~~~~~~~~ + +Try changing the color - for example, ``color('blue')`` - and +width of the line - for example, ``width(3)`` - and then drawing again. + +You can also move the turtle around without drawing, by lifting up the pen: +``up()`` before moving. To start drawing again, use ``down()``. + + +The turtle's position +~~~~~~~~~~~~~~~~~~~~~ + +Send your turtle back to its starting-point (useful if it has disappeared +off-screen):: + + home() + +The home position is at the center of the turtle's screen. If you ever need to +know them, get the turtle's x-y co-ordinates with:: + + pos() + +Home is at ``(0, 0)``. + +And after a while, it will probably help to clear the window so we can start +anew:: + + clearscreen() + + +Making algorithmic patterns +--------------------------- + +Using loops, it's possible to build up geometric patterns:: + + for steps in range(100): + for c in ('blue', 'red', 'green'): + color(c) + forward(steps) + right(30) + + +\ - which of course, are limited only by the imagination! + +Let's draw the star shape at the top of this page. We want red lines, +filled in with yellow:: + + color('red') + fillcolor('yellow') + +Just as ``up()`` and ``down()`` determine whether lines will be drawn, +filling can be turned on and off:: + + begin_fill() + +Next we'll create a loop:: + + while True: + forward(200) + left(170) + if abs(pos()) < 1: + break + +``abs(pos()) < 1`` is a good way to know when the turtle is back at its +home position. + +Finally, complete the filling:: + + end_fill() + +(Note that filling only actually takes place when you give the +``end_fill()`` command.) + + +Explanation +=========== The :mod:`turtle` module is an extended reimplementation of the same-named module from the Python standard distribution up to version Python 2.5. @@ -112,8 +230,8 @@ To use multiple turtles on a screen one has to use the object-oriented interface omitted here. -Overview of available Turtle and Screen methods -================================================= +Turtle graphics reference +========================= Turtle methods -------------- From webhook-mailer at python.org Sun Jul 23 05:44:43 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 09:44:43 -0000 Subject: [Python-checkins] [3.12] gh-106976: alphabetise bullets by module name task2-3 (GH-107005) (#107106) Message-ID: https://github.com/python/cpython/commit/a73d5c5e2e0529642adc896f9ae4d3987ec5374a commit: a73d5c5e2e0529642adc896f9ae4d3987ec5374a branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-23T09:44:39Z summary: [3.12] gh-106976: alphabetise bullets by module name task2-3 (GH-107005) (#107106) Co-authored-by: littlebutt's workshop Co-authored-by: Hugo van Kemenade files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 6d5fae1d6fcc6..d1b10dbd1999d 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -967,67 +967,76 @@ Demos and Tools Deprecated ========== -* :class:`typing.Hashable` and :class:`typing.Sized` aliases for :class:`collections.abc.Hashable` - and :class:`collections.abc.Sized`. (:gh:`94309`.) - -* The :mod:`sqlite3` :ref:`default adapters and converters - ` are now deprecated. - Instead, use the :ref:`sqlite3-adapter-converter-recipes` - and tailor them to your needs. - (Contributed by Erlend E. Aasland in :gh:`90016`.) - -* In :meth:`~sqlite3.Cursor.execute`, :exc:`DeprecationWarning` is now emitted - when :ref:`named placeholders ` are used together with - parameters supplied as a :term:`sequence` instead of as a :class:`dict`. - Starting from Python 3.14, using named placeholders with parameters supplied - as a sequence will raise a :exc:`~sqlite3.ProgrammingError`. - (Contributed by Erlend E. Aasland in :gh:`101698`.) - -* The 3-arg signatures (type, value, traceback) of :meth:`~coroutine.throw`, - :meth:`~generator.throw` and :meth:`~agen.athrow` are deprecated and - may be removed in a future version of Python. Use the single-arg versions - of these functions instead. (Contributed by Ofey Chan in :gh:`89874`.) - -* :exc:`DeprecationWarning` is now raised when ``__package__`` on a - module differs from ``__spec__.parent`` (previously it was - :exc:`ImportWarning`). - (Contributed by Brett Cannon in :gh:`65961`.) - -* The :meth:`~asyncio.get_event_loop` method of the +* :mod:`asyncio`: The :meth:`~asyncio.get_event_loop` method of the default event loop policy now emits a :exc:`DeprecationWarning` if there is no current event loop set and it decides to create one. (Contributed by Serhiy Storchaka and Guido van Rossum in :gh:`100160`.) -* The :mod:`xml.etree.ElementTree` module now emits :exc:`DeprecationWarning` - when testing the truth value of an :class:`xml.etree.ElementTree.Element`. - Before, the Python implementation emitted :exc:`FutureWarning`, and the C - implementation emitted nothing. +* :mod:`calendar`: ``calendar.January`` and ``calendar.February`` constants are deprecated and + replaced by :data:`calendar.Month.JANUARY` and :data:`calendar.Month.FEBRUARY`. + (Contributed by Prince Roshan in :gh:`103636`.) -* In accordance with :pep:`699`, the ``ma_version_tag`` field in :c:type:`PyDictObject` - is deprecated for extension modules. Accessing this field will generate a compiler - warning at compile time. This field will be removed in Python 3.14. - (Contributed by Ramvikrams and Kumar Aditya in :gh:`101193`. PEP by Ken Jin.) +* :mod:`datetime`: :class:`datetime.datetime`'s :meth:`~datetime.datetime.utcnow` and + :meth:`~datetime.datetime.utcfromtimestamp` are deprecated and will be + removed in a future version. Instead, use timezone-aware objects to represent + datetimes in UTC: respectively, call :meth:`~datetime.datetime.now` and + :meth:`~datetime.datetime.fromtimestamp` with the *tz* parameter set to + :const:`datetime.UTC`. + (Contributed by Paul Ganssle in :gh:`103857`.) -* The ``st_ctime`` fields return by :func:`os.stat` and :func:`os.lstat` on +* :mod:`os`: The ``st_ctime`` fields return by :func:`os.stat` and :func:`os.lstat` on Windows are deprecated. In a future release, they will contain the last metadata change time, consistent with other platforms. For now, they still contain the creation time, which is also available in the new ``st_birthtime`` field. (Contributed by Steve Dower in :gh:`99726`.) -* The :data:`sys.last_type`, :data:`sys.last_value` and :data:`sys.last_traceback` +* :mod:`shutil`: The *onerror* argument of :func:`shutil.rmtree` is deprecated as will be removed + in Python 3.14. Use *onexc* instead. (Contributed by Irit Katriel in :gh:`102828`.) + +* :mod:`sqlite3`: + * :ref:`default adapters and converters + ` are now deprecated. + Instead, use the :ref:`sqlite3-adapter-converter-recipes` + and tailor them to your needs. + (Contributed by Erlend E. Aasland in :gh:`90016`.) + + * In :meth:`~sqlite3.Cursor.execute`, :exc:`DeprecationWarning` is now emitted + when :ref:`named placeholders ` are used together with + parameters supplied as a :term:`sequence` instead of as a :class:`dict`. + Starting from Python 3.14, using named placeholders with parameters supplied + as a sequence will raise a :exc:`~sqlite3.ProgrammingError`. + (Contributed by Erlend E. Aasland in :gh:`101698`.) + +* :mod:`sys`: The :data:`sys.last_type`, :data:`sys.last_value` and :data:`sys.last_traceback` fields are deprecated. Use :data:`sys.last_exc` instead. (Contributed by Irit Katriel in :gh:`102778`.) -* The *onerror* argument of :func:`shutil.rmtree` is deprecated as will be removed - in Python 3.14. Use *onexc* instead. (Contributed by Irit Katriel in :gh:`102828`.) - -* Extracting tar archives without specifying *filter* is deprecated until +* :mod:`tarfile`: Extracting tar archives without specifying *filter* is deprecated until Python 3.14, when ``'data'`` filter will become the default. See :ref:`tarfile-extraction-filter` for details. -* ``calendar.January`` and ``calendar.February`` constants are deprecated and - replaced by :data:`calendar.Month.JANUARY` and :data:`calendar.Month.FEBRUARY`. - (Contributed by Prince Roshan in :gh:`103636`.) +* :mod:`typing`: :class:`typing.Hashable` and :class:`typing.Sized` aliases for :class:`collections.abc.Hashable` + and :class:`collections.abc.Sized`. (:gh:`94309`.) + +* :mod:`xml.etree.ElementTree`: The module now emits :exc:`DeprecationWarning` + when testing the truth value of an :class:`xml.etree.ElementTree.Element`. + Before, the Python implementation emitted :exc:`FutureWarning`, and the C + implementation emitted nothing. + +* The 3-arg signatures (type, value, traceback) of :meth:`~coroutine.throw`, + :meth:`~generator.throw` and :meth:`~agen.athrow` are deprecated and + may be removed in a future version of Python. Use the single-arg versions + of these functions instead. (Contributed by Ofey Chan in :gh:`89874`.) + +* :exc:`DeprecationWarning` is now raised when ``__package__`` on a + module differs from ``__spec__.parent`` (previously it was + :exc:`ImportWarning`). + (Contributed by Brett Cannon in :gh:`65961`.) + +* In accordance with :pep:`699`, the ``ma_version_tag`` field in :c:type:`PyDictObject` + is deprecated for extension modules. Accessing this field will generate a compiler + warning at compile time. This field will be removed in Python 3.14. + (Contributed by Ramvikrams and Kumar Aditya in :gh:`101193`. PEP by Ken Jin.) * The bitwise inversion operator (``~``) on bool is deprecated. It will throw an error in Python 3.14. Use ``not`` for logical negation of bools instead. @@ -1035,16 +1044,6 @@ Deprecated ``int``, convert to int explicitly with ``~int(x)``. (Contributed by Tim Hoffmann in :gh:`103487`.) -* :class:`datetime.datetime`'s - :meth:`~datetime.datetime.utcnow` and - :meth:`~datetime.datetime.utcfromtimestamp` are deprecated and will be - removed in a future version. Instead, use timezone-aware objects to represent - datetimes in UTC: respectively, call - :meth:`~datetime.datetime.now` and - :meth:`~datetime.datetime.fromtimestamp` with the *tz* parameter set to - :const:`datetime.UTC`. - (Contributed by Paul Ganssle in :gh:`103857`.) - Pending Removal in Python 3.13 ------------------------------ @@ -1088,7 +1087,33 @@ APIs: Pending Removal in Python 3.14 ------------------------------ -* Deprecated the following :mod:`importlib.abc` classes, scheduled for removal in +* :mod:`argparse`: The *type*, *choices*, and *metavar* parameters + of :class:`!argparse.BooleanOptionalAction` are deprecated + and will be removed in 3.14. + (Contributed by Nikita Sobolev in :gh:`92248`.) + +* :mod:`ast`: The following :mod:`ast` features have been deprecated in documentation since + Python 3.8, now cause a :exc:`DeprecationWarning` to be emitted at runtime + when they are accessed or used, and will be removed in Python 3.14: + + * :class:`!ast.Num` + * :class:`!ast.Str` + * :class:`!ast.Bytes` + * :class:`!ast.NameConstant` + * :class:`!ast.Ellipsis` + + Use :class:`ast.Constant` instead. + (Contributed by Serhiy Storchaka in :gh:`90953`.) + +* :mod:`collections.abc`: Deprecated :class:`collections.abc.ByteString`. + Prefer :class:`Sequence` or :class:`collections.abc.Buffer`. + For use in typing, prefer a union, like ``bytes | bytearray``, or :class:`collections.abc.Buffer`. + (Contributed by Shantanu Jain in :gh:`91896`.) + +* :mod:`email`: Deprecated the *isdst* parameter in :func:`email.utils.localtime`. + (Contributed by Alan Williams in :gh:`72346`.) + +* :mod:`importlib.abc`: Deprecated the following classes, scheduled for removal in Python 3.14: * :class:`!importlib.abc.ResourceReader` @@ -1102,27 +1127,13 @@ Pending Removal in Python 3.14 (Contributed by Jason R. Coombs and Hugo van Kemenade in :gh:`93963`.) -* Deprecated :class:`collections.abc.ByteString`. - Prefer :class:`Sequence` or :class:`collections.abc.Buffer`. - For use in typing, prefer a union, like ``bytes | bytearray``, or :class:`collections.abc.Buffer`. - (Contributed by Shantanu Jain in :gh:`91896`.) - -* :class:`typing.ByteString`, deprecated since Python 3.9, now causes a - :exc:`DeprecationWarning` to be emitted when it is used. - -* Creating immutable types (:c:macro:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable - bases using the C API. - -* Deprecated the *isdst* parameter in :func:`email.utils.localtime`. - (Contributed by Alan Williams in :gh:`72346`.) - -* ``__package__`` and ``__cached__`` will cease to be set or taken - into consideration by the import system (:gh:`97879`). - -* Testing the truth value of an :class:`xml.etree.ElementTree.Element` - is deprecated and will raise an exception in Python 3.14. +* :mod:`itertools`: The module had undocumented, inefficient, historically buggy, + and inconsistent support for copy, deepcopy, and pickle operations. + This will be removed in 3.14 for a significant reduction in code + volume and maintenance burden. + (Contributed by Raymond Hettinger in :gh:`101588`.) -* The default :mod:`multiprocessing` start method will change to a safer one on +* :mod:`multiprocessing`: The default :mod:`multiprocessing` start method will change to a safer one on Linux, BSDs, and other non-macOS POSIX platforms where ``'fork'`` is currently the default (:gh:`84559`). Adding a runtime warning about this was deemed too disruptive as the majority of code is not expected to care. Use the @@ -1130,47 +1141,35 @@ Pending Removal in Python 3.14 :func:`~multiprocessing.set_start_method` APIs to explicitly specify when your code *requires* ``'fork'``. See :ref:`multiprocessing-start-methods`. -* :mod:`pty` has two undocumented ``master_open()`` and ``slave_open()`` +* :mod:`pkgutil`: :func:`pkgutil.find_loader` and :func:`pkgutil.get_loader` + now raise :exc:`DeprecationWarning`; + use :func:`importlib.util.find_spec` instead. + (Contributed by Nikita Sobolev in :gh:`97850`.) + +* :mod:`pty`: The module has two undocumented ``master_open()`` and ``slave_open()`` functions that have been deprecated since Python 2 but only gained a proper :exc:`DeprecationWarning` in 3.12. Remove them in 3.14. -* :mod:`itertools` had undocumented, inefficient, historically buggy, - and inconsistent support for copy, deepcopy, and pickle operations. - This will be removed in 3.14 for a significant reduction in code - volume and maintenance burden. - (Contributed by Raymond Hettinger in :gh:`101588`.) - -* Accessing ``co_lnotab`` was deprecated in :pep:`626` since 3.10 - and was planned to be removed in 3.12 - but it only got a proper :exc:`DeprecationWarning` in 3.12. - May be removed in 3.14. - (Contributed by Nikita Sobolev in :gh:`101866`.) - -* The *onerror* argument of :func:`shutil.rmtree` is deprecated in 3.12, +* :mod:`shutil`: The *onerror* argument of :func:`shutil.rmtree` is deprecated in 3.12, and will be removed in 3.14. -* The *type*, *choices*, and *metavar* parameters - of :class:`!argparse.BooleanOptionalAction` are deprecated - and will be removed in 3.14. - (Contributed by Nikita Sobolev in :gh:`92248`.) +* :mod:`typing`: :class:`typing.ByteString`, deprecated since Python 3.9, now causes a + :exc:`DeprecationWarning` to be emitted when it is used. -* :func:`pkgutil.find_loader` and :func:`pkgutil.get_loader` - now raise :exc:`DeprecationWarning`; - use :func:`importlib.util.find_spec` instead. - (Contributed by Nikita Sobolev in :gh:`97850`.) +* :mod:`xml.etree.ElementTree`: Testing the truth value of an :class:`xml.etree.ElementTree.Element` + is deprecated and will raise an exception in Python 3.14. -* The following :mod:`ast` features have been deprecated in documentation since - Python 3.8, now cause a :exc:`DeprecationWarning` to be emitted at runtime - when they are accessed or used, and will be removed in Python 3.14: +* Creating immutable types (:c:macro:`Py_TPFLAGS_IMMUTABLETYPE`) with mutable + bases using the C API. - * :class:`!ast.Num` - * :class:`!ast.Str` - * :class:`!ast.Bytes` - * :class:`!ast.NameConstant` - * :class:`!ast.Ellipsis` +* ``__package__`` and ``__cached__`` will cease to be set or taken + into consideration by the import system (:gh:`97879`). - Use :class:`ast.Constant` instead. - (Contributed by Serhiy Storchaka in :gh:`90953`.) +* Accessing ``co_lnotab`` was deprecated in :pep:`626` since 3.10 + and was planned to be removed in 3.12 + but it only got a proper :exc:`DeprecationWarning` in 3.12. + May be removed in 3.14. + (Contributed by Nikita Sobolev in :gh:`101866`.) Pending Removal in Future Versions ---------------------------------- @@ -1193,13 +1192,29 @@ although there is currently no date scheduled for their removal. Removed ======= -* Remove the ``distutils`` package. It was deprecated in Python 3.10 by +* ``asynchat`` and ``asyncore``: These two modules have been removed + according to the schedule in :pep:`594`, + having been deprecated in Python 3.6. + Use :mod:`asyncio` instead. + (Contributed by Nikita Sobolev in :gh:`96580`.) + +* :mod:`configparser`: Several names deprecated in the :mod:`configparser` way back in 3.2 have + been removed per :gh:`89336`: + + * :class:`configparser.ParsingError` no longer has a ``filename`` attribute + or argument. Use the ``source`` attribute and argument instead. + * :mod:`configparser` no longer has a ``SafeConfigParser`` class. Use the + shorter :class:`~configparser.ConfigParser` name instead. + * :class:`configparser.ConfigParser` no longer has a ``readfp`` method. + Use :meth:`~configparser.ConfigParser.read_file` instead. + +* ``distutils``: Remove the ``distutils`` package. It was deprecated in Python 3.10 by :pep:`632` "Deprecate distutils module". For projects still using ``distutils`` and cannot be updated to something else, the ``setuptools`` project can be installed: it still provides ``distutils``. (Contributed by Victor Stinner in :gh:`92584`.) -* Remove the bundled setuptools wheel from :mod:`ensurepip`, +* :mod:`ensurepip`: Remove the bundled setuptools wheel from :mod:`ensurepip`, and stop installing setuptools in environments created by :mod:`venv`. ``pip (>= 22.1)`` does not require setuptools to be installed in the @@ -1217,94 +1232,9 @@ Removed (Contributed by Pradyun Gedam in :gh:`95299`.) -* Removed many old deprecated :mod:`unittest` features: - - - A number of :class:`~unittest.TestCase` method aliases: - - ============================ =============================== =============== - Deprecated alias Method Name Deprecated in - ============================ =============================== =============== - ``failUnless`` :meth:`.assertTrue` 3.1 - ``failIf`` :meth:`.assertFalse` 3.1 - ``failUnlessEqual`` :meth:`.assertEqual` 3.1 - ``failIfEqual`` :meth:`.assertNotEqual` 3.1 - ``failUnlessAlmostEqual`` :meth:`.assertAlmostEqual` 3.1 - ``failIfAlmostEqual`` :meth:`.assertNotAlmostEqual` 3.1 - ``failUnlessRaises`` :meth:`.assertRaises` 3.1 - ``assert_`` :meth:`.assertTrue` 3.2 - ``assertEquals`` :meth:`.assertEqual` 3.2 - ``assertNotEquals`` :meth:`.assertNotEqual` 3.2 - ``assertAlmostEquals`` :meth:`.assertAlmostEqual` 3.2 - ``assertNotAlmostEquals`` :meth:`.assertNotAlmostEqual` 3.2 - ``assertRegexpMatches`` :meth:`.assertRegex` 3.2 - ``assertRaisesRegexp`` :meth:`.assertRaisesRegex` 3.2 - ``assertNotRegexpMatches`` :meth:`.assertNotRegex` 3.5 - ============================ =============================== =============== - - You can use https://github.com/isidentical/teyit to automatically modernise - your unit tests. - - - Undocumented and broken :class:`~unittest.TestCase` method - ``assertDictContainsSubset`` (deprecated in Python 3.2). - - - Undocumented :meth:`TestLoader.loadTestsFromModule - ` parameter *use_load_tests* - (deprecated and ignored since Python 3.2). - - - An alias of the :class:`~unittest.TextTestResult` class: - ``_TextTestResult`` (deprecated in Python 3.2). - - (Contributed by Serhiy Storchaka in :issue:`45162`.) - -* Several names deprecated in the :mod:`configparser` way back in 3.2 have - been removed per :gh:`89336`: - - * :class:`configparser.ParsingError` no longer has a ``filename`` attribute - or argument. Use the ``source`` attribute and argument instead. - * :mod:`configparser` no longer has a ``SafeConfigParser`` class. Use the - shorter :class:`~configparser.ConfigParser` name instead. - * :class:`configparser.ConfigParser` no longer has a ``readfp`` method. - Use :meth:`~configparser.ConfigParser.read_file` instead. - -* The following undocumented :mod:`sqlite3` features, deprecated in Python - 3.10, are now removed: - - * ``sqlite3.enable_shared_cache()`` - * ``sqlite3.OptimizedUnicode`` - - If a shared cache must be used, open the database in URI mode using the - ``cache=shared`` query parameter. - - The ``sqlite3.OptimizedUnicode`` text factory has been an alias for - :class:`str` since Python 3.3. Code that previously set the text factory to - ``OptimizedUnicode`` can either use ``str`` explicitly, or rely on the - default value which is also ``str``. - - (Contributed by Erlend E. Aasland in :gh:`92548`.) - -* ``smtpd`` has been removed according to the schedule in :pep:`594`, - having been deprecated in Python 3.4.7 and 3.5.4. - Use aiosmtpd_ PyPI module or any other - :mod:`asyncio`-based server instead. - (Contributed by Oleg Iarygin in :gh:`93243`.) - -.. _aiosmtpd: https://pypi.org/project/aiosmtpd/ - -* ``asynchat`` and ``asyncore`` have been removed - according to the schedule in :pep:`594`, - having been deprecated in Python 3.6. - Use :mod:`asyncio` instead. - (Contributed by Nikita Sobolev in :gh:`96580`.) - -* Remove ``io.OpenWrapper`` and ``_pyio.OpenWrapper``, deprecated in Python - 3.10: just use :func:`open` instead. The :func:`open` (:func:`io.open`) - function is a built-in function. Since Python 3.10, :func:`!_pyio.open` is - also a static method. - (Contributed by Victor Stinner in :gh:`94169`.) - -* Remove the :func:`!ssl.RAND_pseudo_bytes` function, deprecated in Python 3.6: - use :func:`os.urandom` or :func:`ssl.RAND_bytes` instead. - (Contributed by Victor Stinner in :gh:`94199`.) +* :mod:`ftplib`: Remove the ``FTP_TLS.ssl_version`` class attribute: use the + *context* parameter instead. + (Contributed by Victor Stinner in :gh:`94172`.) * :mod:`gzip`: Remove the ``filename`` attribute of :class:`gzip.GzipFile`, deprecated since Python 2.6, use the :attr:`~gzip.GzipFile.name` attribute @@ -1312,43 +1242,13 @@ Removed extension if it was not present. (Contributed by Victor Stinner in :gh:`94196`.) -* Remove the :func:`!ssl.match_hostname` function. - It was deprecated in Python 3.7. OpenSSL performs - hostname matching since Python 3.7, Python no longer uses the - :func:`!ssl.match_hostname` function. - (Contributed by Victor Stinner in :gh:`94199`.) - -* Remove the :func:`!locale.format` function, deprecated in Python 3.7: - use :func:`locale.format_string` instead. - (Contributed by Victor Stinner in :gh:`94226`.) - * :mod:`hashlib`: Remove the pure Python implementation of :func:`hashlib.pbkdf2_hmac()`, deprecated in Python 3.10. Python 3.10 and newer requires OpenSSL 1.1.1 (:pep:`644`): this OpenSSL version provides a C implementation of :func:`~hashlib.pbkdf2_hmac()` which is faster. (Contributed by Victor Stinner in :gh:`94199`.) -* :mod:`xml.etree.ElementTree`: Remove the ``ElementTree.Element.copy()`` method of the - pure Python implementation, deprecated in Python 3.10, use the - :func:`copy.copy` function instead. The C implementation of :mod:`xml.etree.ElementTree` - has no ``copy()`` method, only a ``__copy__()`` method. - (Contributed by Victor Stinner in :gh:`94383`.) - -* :mod:`zipimport`: Remove ``find_loader()`` and ``find_module()`` methods, - deprecated in Python 3.10: use the ``find_spec()`` method instead. See - :pep:`451` for the rationale. - (Contributed by Victor Stinner in :gh:`94379`.) - -* Remove the :func:`!ssl.wrap_socket` function, deprecated in Python 3.7: - instead, create a :class:`ssl.SSLContext` object and call its - :class:`ssl.SSLContext.wrap_socket` method. Any package that still uses - :func:`!ssl.wrap_socket` is broken and insecure. The function neither sends a - SNI TLS extension nor validates server hostname. Code is subject to `CWE-295 - `_: Improper Certificate - Validation. - (Contributed by Victor Stinner in :gh:`94199`.) - -* Many previously deprecated cleanups in :mod:`importlib` have now been +* :mod:`importlib`: Many previously deprecated cleanups in :mod:`importlib` have now been completed: * References to, and support for :meth:`!module_repr()` has been removed. @@ -1413,6 +1313,115 @@ Removed ``PY_COMPILED``, ``C_EXTENSION``, ``PY_RESOURCE``, ``PKG_DIRECTORY``, ``C_BUILTIN``, ``PY_FROZEN``, ``PY_CODERESOURCE``, ``IMP_HOOK``. +* :mod:`io`: Remove ``io.OpenWrapper`` and ``_pyio.OpenWrapper``, deprecated in Python + 3.10: just use :func:`open` instead. The :func:`open` (:func:`io.open`) + function is a built-in function. Since Python 3.10, :func:`!_pyio.open` is + also a static method. + (Contributed by Victor Stinner in :gh:`94169`.) + +* :mod:`locale`: Remove the :func:`!locale.format` function, deprecated in Python 3.7: + use :func:`locale.format_string` instead. + (Contributed by Victor Stinner in :gh:`94226`.) + +* ``smtpd``: The module has been removed according to the schedule in :pep:`594`, + having been deprecated in Python 3.4.7 and 3.5.4. + Use aiosmtpd_ PyPI module or any other + :mod:`asyncio`-based server instead. + (Contributed by Oleg Iarygin in :gh:`93243`.) + +.. _aiosmtpd: https://pypi.org/project/aiosmtpd/ + +* :mod:`sqlite3`: The following undocumented :mod:`sqlite3` features, deprecated in Python + 3.10, are now removed: + + * ``sqlite3.enable_shared_cache()`` + * ``sqlite3.OptimizedUnicode`` + + If a shared cache must be used, open the database in URI mode using the + ``cache=shared`` query parameter. + + The ``sqlite3.OptimizedUnicode`` text factory has been an alias for + :class:`str` since Python 3.3. Code that previously set the text factory to + ``OptimizedUnicode`` can either use ``str`` explicitly, or rely on the + default value which is also ``str``. + + (Contributed by Erlend E. Aasland in :gh:`92548`.) + +* :mod:`ssl`: + + * Remove the :func:`!ssl.RAND_pseudo_bytes` function, deprecated in Python 3.6: + use :func:`os.urandom` or :func:`ssl.RAND_bytes` instead. + (Contributed by Victor Stinner in :gh:`94199`.) + + * Remove the :func:`!ssl.match_hostname` function. + It was deprecated in Python 3.7. OpenSSL performs + hostname matching since Python 3.7, Python no longer uses the + :func:`!ssl.match_hostname` function. + (Contributed by Victor Stinner in :gh:`94199`.) + + * Remove the :func:`!ssl.wrap_socket` function, deprecated in Python 3.7: + instead, create a :class:`ssl.SSLContext` object and call its + :class:`ssl.SSLContext.wrap_socket` method. Any package that still uses + :func:`!ssl.wrap_socket` is broken and insecure. The function neither sends a + SNI TLS extension nor validates server hostname. Code is subject to `CWE-295 + `_: Improper Certificate + Validation. + (Contributed by Victor Stinner in :gh:`94199`.) + +* :mod:`unittest`: Removed many old deprecated :mod:`unittest` features: + + - A number of :class:`~unittest.TestCase` method aliases: + + ============================ =============================== =============== + Deprecated alias Method Name Deprecated in + ============================ =============================== =============== + ``failUnless`` :meth:`.assertTrue` 3.1 + ``failIf`` :meth:`.assertFalse` 3.1 + ``failUnlessEqual`` :meth:`.assertEqual` 3.1 + ``failIfEqual`` :meth:`.assertNotEqual` 3.1 + ``failUnlessAlmostEqual`` :meth:`.assertAlmostEqual` 3.1 + ``failIfAlmostEqual`` :meth:`.assertNotAlmostEqual` 3.1 + ``failUnlessRaises`` :meth:`.assertRaises` 3.1 + ``assert_`` :meth:`.assertTrue` 3.2 + ``assertEquals`` :meth:`.assertEqual` 3.2 + ``assertNotEquals`` :meth:`.assertNotEqual` 3.2 + ``assertAlmostEquals`` :meth:`.assertAlmostEqual` 3.2 + ``assertNotAlmostEquals`` :meth:`.assertNotAlmostEqual` 3.2 + ``assertRegexpMatches`` :meth:`.assertRegex` 3.2 + ``assertRaisesRegexp`` :meth:`.assertRaisesRegex` 3.2 + ``assertNotRegexpMatches`` :meth:`.assertNotRegex` 3.5 + ============================ =============================== =============== + + You can use https://github.com/isidentical/teyit to automatically modernise + your unit tests. + + - Undocumented and broken :class:`~unittest.TestCase` method + ``assertDictContainsSubset`` (deprecated in Python 3.2). + + - Undocumented :meth:`TestLoader.loadTestsFromModule + ` parameter *use_load_tests* + (deprecated and ignored since Python 3.2). + + - An alias of the :class:`~unittest.TextTestResult` class: + ``_TextTestResult`` (deprecated in Python 3.2). + + (Contributed by Serhiy Storchaka in :issue:`45162`.) + +* :mod:`webbrowser`: Remove support for obsolete browsers from :mod:`webbrowser`. + Removed browsers include: Grail, Mosaic, Netscape, Galeon, Skipstone, + Iceape, Firebird, and Firefox versions 35 and below (:gh:`102871`). + +* :mod:`xml.etree.ElementTree`: Remove the ``ElementTree.Element.copy()`` method of the + pure Python implementation, deprecated in Python 3.10, use the + :func:`copy.copy` function instead. The C implementation of :mod:`xml.etree.ElementTree` + has no ``copy()`` method, only a ``__copy__()`` method. + (Contributed by Victor Stinner in :gh:`94383`.) + +* :mod:`zipimport`: Remove ``find_loader()`` and ``find_module()`` methods, + deprecated in Python 3.10: use the ``find_spec()`` method instead. See + :pep:`451` for the rationale. + (Contributed by Victor Stinner in :gh:`94379`.) + * Removed the ``suspicious`` rule from the documentation Makefile, and removed ``Doc/tools/rstlint.py``, both in favor of `sphinx-lint `_. @@ -1426,15 +1435,6 @@ Removed (*ssl_context* in :mod:`imaplib`) instead. (Contributed by Victor Stinner in :gh:`94172`.) -* :mod:`ftplib`: Remove the ``FTP_TLS.ssl_version`` class attribute: use the - *context* parameter instead. - (Contributed by Victor Stinner in :gh:`94172`.) - -* Remove support for obsolete browsers from :mod:`webbrowser`. - Removed browsers include: Grail, Mosaic, Netscape, Galeon, Skipstone, - Iceape, Firebird, and Firefox versions 35 and below (:gh:`102871`). - - .. _whatsnew312-porting-to-python312: Porting to Python 3.12 From webhook-mailer at python.org Sun Jul 23 05:59:23 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 09:59:23 -0000 Subject: [Python-checkins] bpo-18319: gettext() can retrieve a message even if a plural form exists (#19869) Message-ID: https://github.com/python/cpython/commit/54632528eeba841e4a8cc95ecbd84c9aca8eef57 commit: 54632528eeba841e4a8cc95ecbd84c9aca8eef57 branch: main author: Gilles Bassi?re committer: ambv date: 2023-07-23T11:59:19+02:00 summary: bpo-18319: gettext() can retrieve a message even if a plural form exists (#19869) files: A Misc/NEWS.d/next/Library/2020-05-03-00-33-15.bpo-18319.faPTlx.rst M Lib/gettext.py M Lib/test/test_gettext.py diff --git a/Lib/gettext.py b/Lib/gettext.py index 6c5ec4e517f63..cc938e40028c2 100644 --- a/Lib/gettext.py +++ b/Lib/gettext.py @@ -422,10 +422,12 @@ def gettext(self, message): missing = object() tmsg = self._catalog.get(message, missing) if tmsg is missing: - if self._fallback: - return self._fallback.gettext(message) - return message - return tmsg + tmsg = self._catalog.get((message, self.plural(1)), missing) + if tmsg is not missing: + return tmsg + if self._fallback: + return self._fallback.gettext(message) + return message def ngettext(self, msgid1, msgid2, n): try: diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py index 1608d1b18e98f..aa3520d2c142e 100644 --- a/Lib/test/test_gettext.py +++ b/Lib/test/test_gettext.py @@ -320,6 +320,8 @@ def test_plural_forms1(self): eq(x, 'Hay %s fichero') x = gettext.ngettext('There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros') + x = gettext.gettext('There is %s file') + eq(x, 'Hay %s fichero') def test_plural_context_forms1(self): eq = self.assertEqual @@ -338,6 +340,8 @@ def test_plural_forms2(self): eq(x, 'Hay %s fichero') x = t.ngettext('There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros') + x = t.gettext('There is %s file') + eq(x, 'Hay %s fichero') def test_plural_context_forms2(self): eq = self.assertEqual diff --git a/Misc/NEWS.d/next/Library/2020-05-03-00-33-15.bpo-18319.faPTlx.rst b/Misc/NEWS.d/next/Library/2020-05-03-00-33-15.bpo-18319.faPTlx.rst new file mode 100644 index 0000000000000..a1a4cf6d63725 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-05-03-00-33-15.bpo-18319.faPTlx.rst @@ -0,0 +1,2 @@ +Ensure `gettext(msg)` retrieve translations even if a plural form exists. In +other words: `gettext(msg) == ngettext(msg, '', 1)`. From webhook-mailer at python.org Sun Jul 23 06:00:35 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 10:00:35 -0000 Subject: [Python-checkins] [3.12] gh-101100: Fix some broken sphinx references (GH-107095) (#107103) Message-ID: https://github.com/python/cpython/commit/332db37835c403a0939aa46527736a9ea898c819 commit: 332db37835c403a0939aa46527736a9ea898c819 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-23T12:00:31+02:00 summary: [3.12] gh-101100: Fix some broken sphinx references (GH-107095) (#107103) (cherry picked from commit f5147c0cfbd7943ff10917225448c36a53f9828d) Co-authored-by: wulmer files: M Doc/c-api/iterator.rst M Doc/c-api/mapping.rst M Doc/c-api/refcounting.rst M Doc/c-api/sequence.rst M Doc/howto/functional.rst M Doc/howto/regex.rst M Doc/howto/sorting.rst M Doc/howto/unicode.rst M Doc/library/_thread.rst M Doc/library/codeop.rst M Doc/library/constants.rst M Doc/tools/.nitignore diff --git a/Doc/c-api/iterator.rst b/Doc/c-api/iterator.rst index 3fcf099134d4d..95952237ca746 100644 --- a/Doc/c-api/iterator.rst +++ b/Doc/c-api/iterator.rst @@ -6,7 +6,7 @@ Iterator Objects ---------------- Python provides two general-purpose iterator objects. The first, a sequence -iterator, works with an arbitrary sequence supporting the :meth:`__getitem__` +iterator, works with an arbitrary sequence supporting the :meth:`~object.__getitem__` method. The second works with a callable object and a sentinel value, calling the callable for each item in the sequence, and ending the iteration when the sentinel value is returned. diff --git a/Doc/c-api/mapping.rst b/Doc/c-api/mapping.rst index cffb0ed50fb77..d94a9dc45b5eb 100644 --- a/Doc/c-api/mapping.rst +++ b/Doc/c-api/mapping.rst @@ -13,7 +13,7 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and Return ``1`` if the object provides the mapping protocol or supports slicing, and ``0`` otherwise. Note that it returns ``1`` for Python classes with - a :meth:`__getitem__` method, since in general it is impossible to + a :meth:`~object.__getitem__` method, since in general it is impossible to determine what type of keys the class supports. This function always succeeds. @@ -60,7 +60,7 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and This is equivalent to the Python expression ``key in o``. This function always succeeds. - Note that exceptions which occur while calling the :meth:`__getitem__` + Note that exceptions which occur while calling the :meth:`~object.__getitem__` method will get suppressed. To get error reporting use :c:func:`PyObject_GetItem()` instead. @@ -71,7 +71,7 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and This is equivalent to the Python expression ``key in o``. This function always succeeds. - Note that exceptions which occur while calling the :meth:`__getitem__` + Note that exceptions which occur while calling the :meth:`~object.__getitem__` method and creating a temporary string object will get suppressed. To get error reporting use :c:func:`PyMapping_GetItemString()` instead. diff --git a/Doc/c-api/refcounting.rst b/Doc/c-api/refcounting.rst index d8e9c2da6f3ff..640c5c610899f 100644 --- a/Doc/c-api/refcounting.rst +++ b/Doc/c-api/refcounting.rst @@ -101,7 +101,7 @@ of Python objects. .. warning:: The deallocation function can cause arbitrary Python code to be invoked (e.g. - when a class instance with a :meth:`__del__` method is deallocated). While + when a class instance with a :meth:`~object.__del__` method is deallocated). While exceptions in such code are not propagated, the executed code has free access to all Python global variables. This means that any object that is reachable from a global variable should be in a consistent state before :c:func:`Py_DECREF` is diff --git a/Doc/c-api/sequence.rst b/Doc/c-api/sequence.rst index 402a3e5e09ff5..ce28839f5ba73 100644 --- a/Doc/c-api/sequence.rst +++ b/Doc/c-api/sequence.rst @@ -9,7 +9,7 @@ Sequence Protocol .. c:function:: int PySequence_Check(PyObject *o) Return ``1`` if the object provides the sequence protocol, and ``0`` otherwise. - Note that it returns ``1`` for Python classes with a :meth:`__getitem__` + Note that it returns ``1`` for Python classes with a :meth:`~object.__getitem__` method, unless they are :class:`dict` subclasses, since in general it is impossible to determine what type of keys the class supports. This function always succeeds. diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst index 5cf12cc52bde4..b0f9d22d74f0e 100644 --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -1072,8 +1072,8 @@ write the obvious :keyword:`for` loop:: A related function is :func:`itertools.accumulate(iterable, func=operator.add) `. It performs the same calculation, but instead of -returning only the final result, :func:`accumulate` returns an iterator that -also yields each partial result:: +returning only the final result, :func:`~itertools.accumulate` returns an iterator +that also yields each partial result:: itertools.accumulate([1, 2, 3, 4, 5]) => 1, 3, 6, 10, 15 diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst index 655df59e27b64..c19c48301f584 100644 --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -518,6 +518,8 @@ cache. Compilation Flags ----------------- +.. currentmodule:: re + Compilation flags let you modify some aspects of how regular expressions work. Flags are available in the :mod:`re` module under two names, a long name such as :const:`IGNORECASE` and a short, one-letter form such as :const:`I`. (If you're diff --git a/Doc/howto/sorting.rst b/Doc/howto/sorting.rst index decce12bf3faf..38dd09f0a721d 100644 --- a/Doc/howto/sorting.rst +++ b/Doc/howto/sorting.rst @@ -273,7 +273,7 @@ Odds and Ends * The sort routines use ``<`` when making comparisons between two objects. So, it is easy to add a standard sort order to a class by - defining an :meth:`__lt__` method: + defining an :meth:`~object.__lt__` method: .. doctest:: @@ -281,8 +281,8 @@ Odds and Ends >>> sorted(student_objects) [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] - However, note that ``<`` can fall back to using :meth:`__gt__` if - :meth:`__lt__` is not implemented (see :func:`object.__lt__`). + However, note that ``<`` can fall back to using :meth:`~object.__gt__` if + :meth:`~object.__lt__` is not implemented (see :func:`object.__lt__`). * Key functions need not depend directly on the objects being sorted. A key function can also access external resources. For instance, if the student grades diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst index b0faa68d24089..254fe72935535 100644 --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -424,8 +424,8 @@ lowercase letters 'ss'. A second tool is the :mod:`unicodedata` module's :func:`~unicodedata.normalize` function that converts strings to one -of several normal forms, where letters followed by a combining -character are replaced with single characters. :func:`normalize` can +of several normal forms, where letters followed by a combining character are +replaced with single characters. :func:`~unicodedata.normalize` can be used to perform string comparisons that won't falsely report inequality if two strings use combining characters differently: @@ -474,8 +474,8 @@ The Unicode Standard also specifies how to do caseless comparisons:: print(compare_caseless(single_char, multiple_chars)) -This will print ``True``. (Why is :func:`NFD` invoked twice? Because -there are a few characters that make :meth:`casefold` return a +This will print ``True``. (Why is :func:`!NFD` invoked twice? Because +there are a few characters that make :meth:`~str.casefold` return a non-normalized string, so the result needs to be normalized again. See section 3.13 of the Unicode Standard for a discussion and an example.) diff --git a/Doc/library/_thread.rst b/Doc/library/_thread.rst index 3ff6b48b083b5..0442c298c137b 100644 --- a/Doc/library/_thread.rst +++ b/Doc/library/_thread.rst @@ -150,8 +150,8 @@ This module defines the following constants and functions: .. data:: TIMEOUT_MAX The maximum value allowed for the *timeout* parameter of - :meth:`Lock.acquire`. Specifying a timeout greater than this value will - raise an :exc:`OverflowError`. + :meth:`Lock.acquire `. Specifying a timeout greater + than this value will raise an :exc:`OverflowError`. .. versionadded:: 3.2 @@ -217,8 +217,9 @@ In addition to these methods, lock objects can also be used via the * Calling :func:`sys.exit` or raising the :exc:`SystemExit` exception is equivalent to calling :func:`_thread.exit`. -* It is not possible to interrupt the :meth:`acquire` method on a lock --- the - :exc:`KeyboardInterrupt` exception will happen after the lock has been acquired. +* It is not possible to interrupt the :meth:`~threading.Lock.acquire` method on + a lock --- the :exc:`KeyboardInterrupt` exception will happen after the lock + has been acquired. * When the main thread exits, it is system defined whether the other threads survive. On most systems, they are killed without executing diff --git a/Doc/library/codeop.rst b/Doc/library/codeop.rst index 90df499f8207b..55606e1c5f09a 100644 --- a/Doc/library/codeop.rst +++ b/Doc/library/codeop.rst @@ -58,7 +58,7 @@ To do just the former: .. class:: Compile() - Instances of this class have :meth:`__call__` methods identical in signature to + Instances of this class have :meth:`~object.__call__` methods identical in signature to the built-in function :func:`compile`, but with the difference that if the instance compiles program text containing a :mod:`__future__` statement, the instance 'remembers' and compiles all subsequent program texts with the @@ -67,7 +67,7 @@ To do just the former: .. class:: CommandCompiler() - Instances of this class have :meth:`__call__` methods identical in signature to + Instances of this class have :meth:`~object.__call__` methods identical in signature to :func:`compile_command`; the difference is that if the instance compiles program text containing a :mod:`__future__` statement, the instance 'remembers' and compiles all subsequent program texts with the statement in force. diff --git a/Doc/library/constants.rst b/Doc/library/constants.rst index 38dd552a0363a..401dc9a320c5e 100644 --- a/Doc/library/constants.rst +++ b/Doc/library/constants.rst @@ -22,16 +22,16 @@ A small number of constants live in the built-in namespace. They are: An object frequently used to represent the absence of a value, as when default arguments are not passed to a function. Assignments to ``None`` are illegal and raise a :exc:`SyntaxError`. - ``None`` is the sole instance of the :data:`NoneType` type. + ``None`` is the sole instance of the :data:`~types.NoneType` type. .. data:: NotImplemented A special value which should be returned by the binary special methods - (e.g. :meth:`__eq__`, :meth:`__lt__`, :meth:`__add__`, :meth:`__rsub__`, + (e.g. :meth:`~object.__eq__`, :meth:`~object.__lt__`, :meth:`~object.__add__`, :meth:`~object.__rsub__`, etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods - (e.g. :meth:`__imul__`, :meth:`__iand__`, etc.) for the same purpose. + (e.g. :meth:`~object.__imul__`, :meth:`~object.__iand__`, etc.) for the same purpose. It should not be evaluated in a boolean context. ``NotImplemented`` is the sole instance of the :data:`types.NotImplementedType` type. diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 0403cb34751ca..f333495017e2a 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -29,15 +29,12 @@ Doc/c-api/init_config.rst Doc/c-api/intro.rst Doc/c-api/iterator.rst Doc/c-api/long.rst -Doc/c-api/mapping.rst Doc/c-api/marshal.rst Doc/c-api/memory.rst Doc/c-api/memoryview.rst Doc/c-api/module.rst Doc/c-api/none.rst Doc/c-api/object.rst -Doc/c-api/refcounting.rst -Doc/c-api/sequence.rst Doc/c-api/set.rst Doc/c-api/stable.rst Doc/c-api/structures.rst @@ -62,19 +59,14 @@ Doc/glossary.rst Doc/howto/curses.rst Doc/howto/descriptor.rst Doc/howto/enum.rst -Doc/howto/functional.rst Doc/howto/instrumentation.rst Doc/howto/isolating-extensions.rst Doc/howto/logging-cookbook.rst Doc/howto/logging.rst -Doc/howto/regex.rst -Doc/howto/sorting.rst -Doc/howto/unicode.rst Doc/howto/urllib2.rst Doc/install/index.rst Doc/library/2to3.rst Doc/library/__future__.rst -Doc/library/_thread.rst Doc/library/abc.rst Doc/library/aifc.rst Doc/library/ast.rst @@ -97,13 +89,11 @@ Doc/library/cmath.rst Doc/library/cmd.rst Doc/library/code.rst Doc/library/codecs.rst -Doc/library/codeop.rst Doc/library/collections.abc.rst Doc/library/collections.rst Doc/library/concurrent.futures.rst Doc/library/concurrent.rst Doc/library/configparser.rst -Doc/library/constants.rst Doc/library/contextlib.rst Doc/library/copy.rst Doc/library/csv.rst From webhook-mailer at python.org Sun Jul 23 06:22:08 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 10:22:08 -0000 Subject: [Python-checkins] [3.12] gh-106996: Add the basics of a turtle graphics tutorial (GH-107072) (#107109) Message-ID: https://github.com/python/cpython/commit/f7e13002ff62ca31b24acae5d057874de7a967d5 commit: f7e13002ff62ca31b24acae5d057874de7a967d5 branch: 3.12 author: Daniele Procida committer: hugovk date: 2023-07-23T04:22:05-06:00 summary: [3.12] gh-106996: Add the basics of a turtle graphics tutorial (GH-107072) (#107109) files: D Doc/includes/turtle-star.py M Doc/library/turtle.rst diff --git a/Doc/includes/turtle-star.py b/Doc/includes/turtle-star.py deleted file mode 100644 index 1a5db761b3238..0000000000000 --- a/Doc/includes/turtle-star.py +++ /dev/null @@ -1,10 +0,0 @@ -from turtle import * -color('red', 'yellow') -begin_fill() -while True: - forward(200) - left(170) - if abs(pos()) < 1: - break -end_fill() -done() diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index c656f6d9cfdaa..a9c0fa96207ae 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -19,14 +19,10 @@ Introduction ============ -Turtle graphics is a popular way for introducing programming to kids. It was -part of the original Logo programming language developed by Wally Feurzeig, -Seymour Papert and Cynthia Solomon in 1967. - -Imagine a robotic turtle starting at (0, 0) in the x-y plane. After an ``import turtle``, give it the -command ``turtle.forward(15)``, and it moves (on-screen!) 15 pixels in the -direction it is facing, drawing a line as it moves. Give it the command -``turtle.right(25)``, and it rotates in-place 25 degrees clockwise. +Turtle graphics is an implementation of `the popular geometric drawing tools +introduced in Logo `_, developed by Wally Feurzeig, Seymour Papert and Cynthia Solomon +in 1967. .. sidebar:: Turtle star @@ -36,10 +32,141 @@ direction it is facing, drawing a line as it moves. Give it the command .. image:: turtle-star.* :align: center - .. literalinclude:: ../includes/turtle-star.py +In Python, turtle graphics provides a representation of a physical "turtle" +(a little robot with a pen) that draws on a sheet of paper on the floor. + +It's an effective and well-proven way for learners to encounter +programming concepts and interaction with software, as it provides instant, +visible feedback. It also provides convenient access to graphical output +in general. + +Turtle drawing was originally created as an educational tool, to be used by +teachers in the classroom. For the programmer who needs to produce some +graphical output it can be a way to do that without the overhead of +introducing more complex or external libraries into their work. + + +Tutorial +======== + +New users should start here. In this tutorial we'll explore some of the +basics of turtle drawing. + + +Starting a turtle environment +----------------------------- + +In a Python shell, import all the objects of the ``turtle`` module:: + + from turtle import * + +If you run into a ``No module named '_tkinter'`` error, you'll have to +install the :mod:`Tk interface package ` on your system. + + +Basic drawing +------------- + +Send the turtle forward 100 steps:: + + forward(100) + +You should see (most likely, in a new window on your display) a line +drawn by the turtle, heading East. Change the direction of the turtle, +so that it turns 120 degrees left (anti-clockwise):: + + left(120) + +Let's continue by drawing a triangle:: + + forward(100) + left(120) + forward(100) + +Notice how the turtle, represented by an arrow, points in different +directions as you steer it. + +Experiment with those commands, and also with ``backward()`` and +``right()``. + + +Pen control +~~~~~~~~~~~ + +Try changing the color - for example, ``color('blue')`` - and +width of the line - for example, ``width(3)`` - and then drawing again. + +You can also move the turtle around without drawing, by lifting up the pen: +``up()`` before moving. To start drawing again, use ``down()``. + + +The turtle's position +~~~~~~~~~~~~~~~~~~~~~ + +Send your turtle back to its starting-point (useful if it has disappeared +off-screen):: + + home() + +The home position is at the center of the turtle's screen. If you ever need to +know them, get the turtle's x-y co-ordinates with:: + + pos() + +Home is at ``(0, 0)``. + +And after a while, it will probably help to clear the window so we can start +anew:: + + clearscreen() + + +Making algorithmic patterns +--------------------------- + +Using loops, it's possible to build up geometric patterns:: + + for steps in range(100): + for c in ('blue', 'red', 'green'): + color(c) + forward(steps) + right(30) + + +\ - which of course, are limited only by the imagination! + +Let's draw the star shape at the top of this page. We want red lines, +filled in with yellow:: + + color('red') + fillcolor('yellow') + +Just as ``up()`` and ``down()`` determine whether lines will be drawn, +filling can be turned on and off:: + + begin_fill() + +Next we'll create a loop:: + + while True: + forward(200) + left(170) + if abs(pos()) < 1: + break + +``abs(pos()) < 1`` is a good way to know when the turtle is back at its +home position. + +Finally, complete the filling:: + + end_fill() + +(Note that filling only actually takes place when you give the +``end_fill()`` command.) + -By combining together these and similar commands, intricate shapes and pictures -can easily be drawn. +Explanation +=========== The :mod:`turtle` module is an extended reimplementation of the same-named module from the Python standard distribution up to version Python 2.5. @@ -94,8 +221,8 @@ To use multiple turtles on a screen one has to use the object-oriented interface omitted here. -Overview of available Turtle and Screen methods -================================================= +Turtle graphics reference +========================= Turtle methods -------------- From webhook-mailer at python.org Sun Jul 23 06:25:22 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 10:25:22 -0000 Subject: [Python-checkins] gh-106186: Don't report MultipartInvariantViolationDefect for valid multipart emails when parsing header only (#107016) Message-ID: https://github.com/python/cpython/commit/c65592c4d6d7552fb6284442906a96a6874cb266 commit: c65592c4d6d7552fb6284442906a96a6874cb266 branch: main author: htsedebenham <31847376+htsedebenham at users.noreply.github.com> committer: ambv date: 2023-07-23T12:25:18+02:00 summary: gh-106186: Don't report MultipartInvariantViolationDefect for valid multipart emails when parsing header only (#107016) files: A Lib/test/test_email/data/msg_47.txt A Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst M Lib/email/feedparser.py M Lib/test/test_email/test_email.py diff --git a/Lib/email/feedparser.py b/Lib/email/feedparser.py index 53d71f5022515..06d6b4a3afcd0 100644 --- a/Lib/email/feedparser.py +++ b/Lib/email/feedparser.py @@ -189,7 +189,7 @@ def close(self): assert not self._msgstack # Look for final set of defects if root.get_content_maintype() == 'multipart' \ - and not root.is_multipart(): + and not root.is_multipart() and not self._headersonly: defect = errors.MultipartInvariantViolationDefect() self.policy.handle_defect(root, defect) return root diff --git a/Lib/test/test_email/data/msg_47.txt b/Lib/test/test_email/data/msg_47.txt new file mode 100644 index 0000000000000..bb48b47d96baf --- /dev/null +++ b/Lib/test/test_email/data/msg_47.txt @@ -0,0 +1,14 @@ +Date: 01 Jan 2001 00:01+0000 +From: arthur at example.example +MIME-Version: 1.0 +Content-Type: multipart/mixed; boundary=foo + +--foo +Content-Type: text/plain +bar + +--foo +Content-Type: text/html +

baz

+ +--foo-- \ No newline at end of file diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index b4f3a2481976e..cdb6ef1275e52 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3712,6 +3712,16 @@ def test_bytes_header_parser(self): self.assertIsInstance(msg.get_payload(), str) self.assertIsInstance(msg.get_payload(decode=True), bytes) + def test_header_parser_multipart_is_valid(self): + # Don't flag valid multipart emails as having defects + with openfile('msg_47.txt', encoding="utf-8") as fp: + msgdata = fp.read() + + parser = email.parser.Parser(policy=email.policy.default) + parsed_msg = parser.parsestr(msgdata, headersonly=True) + + self.assertEqual(parsed_msg.defects, []) + def test_bytes_parser_does_not_close_file(self): with openfile('msg_02.txt', 'rb') as fp: email.parser.BytesParser().parse(fp) diff --git a/Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst b/Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst new file mode 100644 index 0000000000000..07fdcc96fa38a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst @@ -0,0 +1,3 @@ +Do not report ``MultipartInvariantViolationDefect`` defect +when the :class:`email.parser.Parser` class is used +to parse emails with ``headersonly=True``. From webhook-mailer at python.org Sun Jul 23 06:25:27 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 10:25:27 -0000 Subject: [Python-checkins] [3.11] gh-106996: Add the basics of a turtle graphics tutorial (GH-107072) (#107110) Message-ID: https://github.com/python/cpython/commit/6e3a2fb0fe07712156e1086d171ff2392343c32a commit: 6e3a2fb0fe07712156e1086d171ff2392343c32a branch: 3.11 author: Daniele Procida committer: hugovk date: 2023-07-23T04:25:23-06:00 summary: [3.11] gh-106996: Add the basics of a turtle graphics tutorial (GH-107072) (#107110) files: D Doc/includes/turtle-star.py M Doc/library/turtle.rst diff --git a/Doc/includes/turtle-star.py b/Doc/includes/turtle-star.py deleted file mode 100644 index 1a5db761b3238..0000000000000 --- a/Doc/includes/turtle-star.py +++ /dev/null @@ -1,10 +0,0 @@ -from turtle import * -color('red', 'yellow') -begin_fill() -while True: - forward(200) - left(170) - if abs(pos()) < 1: - break -end_fill() -done() diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index 2298c6c545451..03b299ba7279c 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -19,14 +19,10 @@ Introduction ============ -Turtle graphics is a popular way for introducing programming to kids. It was -part of the original Logo programming language developed by Wally Feurzeig, -Seymour Papert and Cynthia Solomon in 1967. - -Imagine a robotic turtle starting at (0, 0) in the x-y plane. After an ``import turtle``, give it the -command ``turtle.forward(15)``, and it moves (on-screen!) 15 pixels in the -direction it is facing, drawing a line as it moves. Give it the command -``turtle.right(25)``, and it rotates in-place 25 degrees clockwise. +Turtle graphics is an implementation of `the popular geometric drawing tools +introduced in Logo `_, developed by Wally Feurzeig, Seymour Papert and Cynthia Solomon +in 1967. .. sidebar:: Turtle star @@ -36,10 +32,141 @@ direction it is facing, drawing a line as it moves. Give it the command .. image:: turtle-star.* :align: center - .. literalinclude:: ../includes/turtle-star.py +In Python, turtle graphics provides a representation of a physical "turtle" +(a little robot with a pen) that draws on a sheet of paper on the floor. + +It's an effective and well-proven way for learners to encounter +programming concepts and interaction with software, as it provides instant, +visible feedback. It also provides convenient access to graphical output +in general. + +Turtle drawing was originally created as an educational tool, to be used by +teachers in the classroom. For the programmer who needs to produce some +graphical output it can be a way to do that without the overhead of +introducing more complex or external libraries into their work. + + +Tutorial +======== + +New users should start here. In this tutorial we'll explore some of the +basics of turtle drawing. + + +Starting a turtle environment +----------------------------- + +In a Python shell, import all the objects of the ``turtle`` module:: + + from turtle import * + +If you run into a ``No module named '_tkinter'`` error, you'll have to +install the :mod:`Tk interface package ` on your system. + + +Basic drawing +------------- + +Send the turtle forward 100 steps:: + + forward(100) + +You should see (most likely, in a new window on your display) a line +drawn by the turtle, heading East. Change the direction of the turtle, +so that it turns 120 degrees left (anti-clockwise):: + + left(120) + +Let's continue by drawing a triangle:: + + forward(100) + left(120) + forward(100) + +Notice how the turtle, represented by an arrow, points in different +directions as you steer it. + +Experiment with those commands, and also with ``backward()`` and +``right()``. + + +Pen control +~~~~~~~~~~~ + +Try changing the color - for example, ``color('blue')`` - and +width of the line - for example, ``width(3)`` - and then drawing again. + +You can also move the turtle around without drawing, by lifting up the pen: +``up()`` before moving. To start drawing again, use ``down()``. + + +The turtle's position +~~~~~~~~~~~~~~~~~~~~~ + +Send your turtle back to its starting-point (useful if it has disappeared +off-screen):: + + home() + +The home position is at the center of the turtle's screen. If you ever need to +know them, get the turtle's x-y co-ordinates with:: + + pos() + +Home is at ``(0, 0)``. + +And after a while, it will probably help to clear the window so we can start +anew:: + + clearscreen() + + +Making algorithmic patterns +--------------------------- + +Using loops, it's possible to build up geometric patterns:: + + for steps in range(100): + for c in ('blue', 'red', 'green'): + color(c) + forward(steps) + right(30) + + +\ - which of course, are limited only by the imagination! + +Let's draw the star shape at the top of this page. We want red lines, +filled in with yellow:: + + color('red') + fillcolor('yellow') + +Just as ``up()`` and ``down()`` determine whether lines will be drawn, +filling can be turned on and off:: + + begin_fill() + +Next we'll create a loop:: + + while True: + forward(200) + left(170) + if abs(pos()) < 1: + break + +``abs(pos()) < 1`` is a good way to know when the turtle is back at its +home position. + +Finally, complete the filling:: + + end_fill() + +(Note that filling only actually takes place when you give the +``end_fill()`` command.) + -By combining together these and similar commands, intricate shapes and pictures -can easily be drawn. +Explanation +=========== The :mod:`turtle` module is an extended reimplementation of the same-named module from the Python standard distribution up to version Python 2.5. @@ -94,8 +221,8 @@ To use multiple turtles on a screen one has to use the object-oriented interface omitted here. -Overview of available Turtle and Screen methods -================================================= +Turtle graphics reference +========================= Turtle methods -------------- From webhook-mailer at python.org Sun Jul 23 06:26:09 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 10:26:09 -0000 Subject: [Python-checkins] [3.12] gh-106969: Indicate no modules were added in 3.10 & 3.12 (GH-106988) (#107094) Message-ID: https://github.com/python/cpython/commit/074fcf15fa13f09c85e27f53951ef84d02d95454 commit: 074fcf15fa13f09c85e27f53951ef84d02d95454 branch: 3.12 author: Moritz Neeb committer: ambv date: 2023-07-23T12:26:05+02:00 summary: [3.12] gh-106969: Indicate no modules were added in 3.10 & 3.12 (GH-106988) (#107094) The "New Modules" section was left in place to ensure that the anchor link for new modules will still exist: /whatsnew/3.12.htmlGH-new-modules /whatsnew/3.10.htmlGH-new-modules This means that existing links to this section don't break. (cherry picked from commit 6dbffaed17d59079d6a2788d686009f762a3278f) Co-authored-by: Sebastiaan Zeeff <33516116+SebastiaanZ at users.noreply.github.com> files: M Doc/whatsnew/3.10.rst M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index bb271d0a45568..92d23462acc57 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -887,7 +887,7 @@ Other Language Changes New Modules =========== -* None yet. +* None. Improved Modules diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index d1b10dbd1999d..a7db645657db1 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -486,7 +486,7 @@ Other Language Changes New Modules =========== -* None yet. +* None. Improved Modules From webhook-mailer at python.org Sun Jul 23 06:26:15 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 10:26:15 -0000 Subject: [Python-checkins] [3.11] gh-106969: Indicate no modules were added in 3.10 (GH-106988) (#107093) Message-ID: https://github.com/python/cpython/commit/0574873e60e54bd4873025b4380116992e4923b9 commit: 0574873e60e54bd4873025b4380116992e4923b9 branch: 3.11 author: Moritz Neeb committer: ambv date: 2023-07-23T12:26:12+02:00 summary: [3.11] gh-106969: Indicate no modules were added in 3.10 (GH-106988) (#107093) [3.11] gh-106969: Indicate no modules were added in 3.10 & 3.12 (GH-106988) The "New Modules" section was left in place to ensure that the anchor link for new modules will still exist: /whatsnew/3.12.htmlGH-new-modules /whatsnew/3.10.htmlGH-new-modules This means that existing links to this section don't break.. (cherry picked from commit 6dbffaed17d59079d6a2788d686009f762a3278f) Co-authored-by: Sebastiaan Zeeff <33516116+SebastiaanZ at users.noreply.github.com> files: M Doc/whatsnew/3.10.rst diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 56b8ff36bee77..50b85174ffdef 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -886,7 +886,7 @@ Other Language Changes New Modules =========== -* None yet. +* None. Improved Modules From webhook-mailer at python.org Sun Jul 23 06:27:08 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 10:27:08 -0000 Subject: [Python-checkins] gh-107091: Fix the use of some C domain roles (#107092) Message-ID: https://github.com/python/cpython/commit/08a228da05a7aec937b65eea21f4091fa3c6b5cf commit: 08a228da05a7aec937b65eea21f4091fa3c6b5cf branch: main author: Serhiy Storchaka committer: ambv date: 2023-07-23T12:27:05+02:00 summary: gh-107091: Fix the use of some C domain roles (#107092) files: M Doc/c-api/buffer.rst M Doc/c-api/method.rst M Doc/c-api/module.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Doc/extending/extending.rst M Doc/howto/isolating-extensions.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a2.rst M Misc/NEWS.d/3.12.0a1.rst diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst index 91d1edd9b2ec4..6e5443f0d6cdc 100644 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -225,7 +225,7 @@ object via :c:func:`PyObject_GetBuffer`. Since the complexity of the logical structure of the memory can vary drastically, the consumer uses the *flags* argument to specify the exact buffer type it can handle. -All :c:data:`Py_buffer` fields are unambiguously defined by the request +All :c:type:`Py_buffer` fields are unambiguously defined by the request type. request-independent fields @@ -464,7 +464,7 @@ Buffer-related functions .. c:function:: Py_ssize_t PyBuffer_SizeFromFormat(const char *format) - Return the implied :c:data:`~Py_buffer.itemsize` from :c:data:`~Py_buffer.format`. + Return the implied :c:member:`~Py_buffer.itemsize` from :c:member:`~Py_buffer.format`. On error, raise an exception and return -1. .. versionadded:: 3.9 diff --git a/Doc/c-api/method.rst b/Doc/c-api/method.rst index 93ad30cd4f7a8..0d75ab8e1af11 100644 --- a/Doc/c-api/method.rst +++ b/Doc/c-api/method.rst @@ -7,8 +7,8 @@ Instance Method Objects .. index:: pair: object; instancemethod -An instance method is a wrapper for a :c:data:`PyCFunction` and the new way -to bind a :c:data:`PyCFunction` to a class object. It replaces the former call +An instance method is a wrapper for a :c:type:`PyCFunction` and the new way +to bind a :c:type:`PyCFunction` to a class object. It replaces the former call ``PyMethod_New(func, NULL, class)``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index a4f79f8989e96..e85baa03a2247 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -145,7 +145,7 @@ or request "multi-phase initialization" by returning the definition struct itsel .. c:member:: PyModuleDef_Base m_base - Always initialize this member to :c:data:`PyModuleDef_HEAD_INIT`. + Always initialize this member to :c:macro:`PyModuleDef_HEAD_INIT`. .. c:member:: const char *m_name diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index d9dcd22a24839..553d86aa3d997 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -215,7 +215,7 @@ Type Objects ``Py_TYPE(self)`` may be a *subclass* of the intended class, and subclasses are not necessarily defined in the same module as their superclass. See :c:type:`PyCMethod` to get the class that defines the method. - See :c:func:`PyType_GetModuleByDef` for cases when ``PyCMethod`` cannot + See :c:func:`PyType_GetModuleByDef` for cases when :c:type:`!PyCMethod` cannot be used. .. versionadded:: 3.9 diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 33591ed14eaf1..9aa076baaa977 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -2255,8 +2255,8 @@ Number Object Structures .. note:: - The :c:data:`nb_reserved` field should always be ``NULL``. It - was previously called :c:data:`nb_long`, and was renamed in + The :c:member:`~PyNumberMethods.nb_reserved` field should always be ``NULL``. It + was previously called :c:member:`!nb_long`, and was renamed in Python 3.0.1. .. c:member:: binaryfunc PyNumberMethods.nb_add diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 0163ba1a5b313..0c5da49fe6134 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -237,10 +237,10 @@ Note that the Python name for the exception object is :exc:`spam.error`. The being :exc:`Exception` (unless another class is passed in instead of ``NULL``), described in :ref:`bltin-exceptions`. -Note also that the :c:data:`SpamError` variable retains a reference to the newly +Note also that the :c:data:`!SpamError` variable retains a reference to the newly created exception class; this is intentional! Since the exception could be removed from the module by external code, an owned reference to the class is -needed to ensure that it will not be discarded, causing :c:data:`SpamError` to +needed to ensure that it will not be discarded, causing :c:data:`!SpamError` to become a dangling pointer. Should it become a dangling pointer, C code which raises the exception could cause a core dump or other unintended side effects. @@ -281,9 +281,9 @@ statement:: It returns ``NULL`` (the error indicator for functions returning object pointers) if an error is detected in the argument list, relying on the exception set by :c:func:`PyArg_ParseTuple`. Otherwise the string value of the argument has been -copied to the local variable :c:data:`command`. This is a pointer assignment and +copied to the local variable :c:data:`!command`. This is a pointer assignment and you are not supposed to modify the string to which it points (so in Standard C, -the variable :c:data:`command` should properly be declared as ``const char +the variable :c:data:`!command` should properly be declared as ``const char *command``). The next statement is a call to the Unix function :c:func:`system`, passing it @@ -291,7 +291,7 @@ the string we just got from :c:func:`PyArg_ParseTuple`:: sts = system(command); -Our :func:`spam.system` function must return the value of :c:data:`sts` as a +Our :func:`!spam.system` function must return the value of :c:data:`!sts` as a Python object. This is done using the function :c:func:`PyLong_FromLong`. :: return PyLong_FromLong(sts); diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index f01801ea47212..cc4a908febfb7 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -483,14 +483,14 @@ to get the state:: return NULL; } -``PyType_GetModuleByDef`` works by searching the +:c:func:`!PyType_GetModuleByDef` works by searching the :term:`method resolution order` (i.e. all superclasses) for the first superclass that has a corresponding module. .. note:: In very exotic cases (inheritance chains spanning multiple modules - created from the same definition), ``PyType_GetModuleByDef`` might not + created from the same definition), :c:func:`!PyType_GetModuleByDef` might not return the module of the true defining class. However, it will always return a module with the same definition, ensuring a compatible C memory layout. diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index bae78c1d9c205..ec5263ec35c52 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -2227,7 +2227,7 @@ New Features (Contributed by Christian Heimes in :issue:`45459`.) -* Added the :c:data:`PyType_GetModuleByDef` function, used to get the module +* Added the :c:func:`PyType_GetModuleByDef` function, used to get the module in which a method was defined, in cases where this information is not available directly (via :c:type:`PyCMethod`). (Contributed by Petr Viktorin in :issue:`46613`.) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index bf889b7ffbadf..e48cc13eff617 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1276,7 +1276,7 @@ New Features * :pep:`573`: Added :c:func:`PyType_FromModuleAndSpec` to associate a module with a class; :c:func:`PyType_GetModule` and :c:func:`PyType_GetModuleState` to retrieve the module and its state; and - :c:data:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to + :c:type:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to access the class it was defined in. (Contributed by Marcel Plch and Petr Viktorin in :issue:`38787`.) diff --git a/Misc/NEWS.d/3.10.0a2.rst b/Misc/NEWS.d/3.10.0a2.rst index eeb179a980bb8..78b25779802d6 100644 --- a/Misc/NEWS.d/3.10.0a2.rst +++ b/Misc/NEWS.d/3.10.0a2.rst @@ -847,8 +847,8 @@ Victor Stinner. .. section: C API Fix potential crash in deallocating method objects when dynamically -allocated `PyMethodDef`'s lifetime is managed through the ``self`` argument -of a `PyCFunction`. +allocated :c:type:`PyMethodDef`'s lifetime is managed through the ``self`` argument +of a :c:type:`PyCFunction`. .. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index 209d079feb514..8ae78ebb1b629 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -5934,7 +5934,7 @@ not override :c:member:`~PyTypeObject.tp_call` now inherit the .. nonce: aiRSgr .. section: C API -Creating :c:data:`immutable types ` with mutable +Creating :c:macro:`immutable types ` with mutable bases is deprecated and is planned to be disabled in Python 3.14. .. From webhook-mailer at python.org Sun Jul 23 07:50:24 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 23 Jul 2023 11:50:24 -0000 Subject: [Python-checkins] [3.12] gh-107091: Fix the use of some C domain roles (GH-107092) (GH-107113) Message-ID: https://github.com/python/cpython/commit/95a82dcbe74d1ed226a3df0763546f27bd5a6f61 commit: 95a82dcbe74d1ed226a3df0763546f27bd5a6f61 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-23T14:50:21+03:00 summary: [3.12] gh-107091: Fix the use of some C domain roles (GH-107092) (GH-107113) (cherry picked from commit 08a228da05a7aec937b65eea21f4091fa3c6b5cf) Co-authored-by: Serhiy Storchaka files: M Doc/c-api/buffer.rst M Doc/c-api/method.rst M Doc/c-api/module.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Doc/extending/extending.rst M Doc/howto/isolating-extensions.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a2.rst M Misc/NEWS.d/3.12.0a1.rst diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst index 91d1edd9b2ec4..6e5443f0d6cdc 100644 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -225,7 +225,7 @@ object via :c:func:`PyObject_GetBuffer`. Since the complexity of the logical structure of the memory can vary drastically, the consumer uses the *flags* argument to specify the exact buffer type it can handle. -All :c:data:`Py_buffer` fields are unambiguously defined by the request +All :c:type:`Py_buffer` fields are unambiguously defined by the request type. request-independent fields @@ -464,7 +464,7 @@ Buffer-related functions .. c:function:: Py_ssize_t PyBuffer_SizeFromFormat(const char *format) - Return the implied :c:data:`~Py_buffer.itemsize` from :c:data:`~Py_buffer.format`. + Return the implied :c:member:`~Py_buffer.itemsize` from :c:member:`~Py_buffer.format`. On error, raise an exception and return -1. .. versionadded:: 3.9 diff --git a/Doc/c-api/method.rst b/Doc/c-api/method.rst index 93ad30cd4f7a8..0d75ab8e1af11 100644 --- a/Doc/c-api/method.rst +++ b/Doc/c-api/method.rst @@ -7,8 +7,8 @@ Instance Method Objects .. index:: pair: object; instancemethod -An instance method is a wrapper for a :c:data:`PyCFunction` and the new way -to bind a :c:data:`PyCFunction` to a class object. It replaces the former call +An instance method is a wrapper for a :c:type:`PyCFunction` and the new way +to bind a :c:type:`PyCFunction` to a class object. It replaces the former call ``PyMethod_New(func, NULL, class)``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index bc8e3b23b9957..e358c5da14a69 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -145,7 +145,7 @@ or request "multi-phase initialization" by returning the definition struct itsel .. c:member:: PyModuleDef_Base m_base - Always initialize this member to :c:data:`PyModuleDef_HEAD_INIT`. + Always initialize this member to :c:macro:`PyModuleDef_HEAD_INIT`. .. c:member:: const char *m_name diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index d9dcd22a24839..553d86aa3d997 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -215,7 +215,7 @@ Type Objects ``Py_TYPE(self)`` may be a *subclass* of the intended class, and subclasses are not necessarily defined in the same module as their superclass. See :c:type:`PyCMethod` to get the class that defines the method. - See :c:func:`PyType_GetModuleByDef` for cases when ``PyCMethod`` cannot + See :c:func:`PyType_GetModuleByDef` for cases when :c:type:`!PyCMethod` cannot be used. .. versionadded:: 3.9 diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 33591ed14eaf1..9aa076baaa977 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -2255,8 +2255,8 @@ Number Object Structures .. note:: - The :c:data:`nb_reserved` field should always be ``NULL``. It - was previously called :c:data:`nb_long`, and was renamed in + The :c:member:`~PyNumberMethods.nb_reserved` field should always be ``NULL``. It + was previously called :c:member:`!nb_long`, and was renamed in Python 3.0.1. .. c:member:: binaryfunc PyNumberMethods.nb_add diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 76e0490d0d22d..097d86e30269c 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -235,10 +235,10 @@ Note that the Python name for the exception object is :exc:`spam.error`. The being :exc:`Exception` (unless another class is passed in instead of ``NULL``), described in :ref:`bltin-exceptions`. -Note also that the :c:data:`SpamError` variable retains a reference to the newly +Note also that the :c:data:`!SpamError` variable retains a reference to the newly created exception class; this is intentional! Since the exception could be removed from the module by external code, an owned reference to the class is -needed to ensure that it will not be discarded, causing :c:data:`SpamError` to +needed to ensure that it will not be discarded, causing :c:data:`!SpamError` to become a dangling pointer. Should it become a dangling pointer, C code which raises the exception could cause a core dump or other unintended side effects. @@ -279,9 +279,9 @@ statement:: It returns ``NULL`` (the error indicator for functions returning object pointers) if an error is detected in the argument list, relying on the exception set by :c:func:`PyArg_ParseTuple`. Otherwise the string value of the argument has been -copied to the local variable :c:data:`command`. This is a pointer assignment and +copied to the local variable :c:data:`!command`. This is a pointer assignment and you are not supposed to modify the string to which it points (so in Standard C, -the variable :c:data:`command` should properly be declared as ``const char +the variable :c:data:`!command` should properly be declared as ``const char *command``). The next statement is a call to the Unix function :c:func:`system`, passing it @@ -289,7 +289,7 @@ the string we just got from :c:func:`PyArg_ParseTuple`:: sts = system(command); -Our :func:`spam.system` function must return the value of :c:data:`sts` as a +Our :func:`!spam.system` function must return the value of :c:data:`!sts` as a Python object. This is done using the function :c:func:`PyLong_FromLong`. :: return PyLong_FromLong(sts); diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index f01801ea47212..cc4a908febfb7 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -483,14 +483,14 @@ to get the state:: return NULL; } -``PyType_GetModuleByDef`` works by searching the +:c:func:`!PyType_GetModuleByDef` works by searching the :term:`method resolution order` (i.e. all superclasses) for the first superclass that has a corresponding module. .. note:: In very exotic cases (inheritance chains spanning multiple modules - created from the same definition), ``PyType_GetModuleByDef`` might not + created from the same definition), :c:func:`!PyType_GetModuleByDef` might not return the module of the true defining class. However, it will always return a module with the same definition, ensuring a compatible C memory layout. diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 4d9aed0d0f9cd..712af6217b6a4 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -2227,7 +2227,7 @@ New Features (Contributed by Christian Heimes in :issue:`45459`.) -* Added the :c:data:`PyType_GetModuleByDef` function, used to get the module +* Added the :c:func:`PyType_GetModuleByDef` function, used to get the module in which a method was defined, in cases where this information is not available directly (via :c:type:`PyCMethod`). (Contributed by Petr Viktorin in :issue:`46613`.) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 23db4022dc06a..6de432d6c036b 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1276,7 +1276,7 @@ New Features * :pep:`573`: Added :c:func:`PyType_FromModuleAndSpec` to associate a module with a class; :c:func:`PyType_GetModule` and :c:func:`PyType_GetModuleState` to retrieve the module and its state; and - :c:data:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to + :c:type:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to access the class it was defined in. (Contributed by Marcel Plch and Petr Viktorin in :issue:`38787`.) diff --git a/Misc/NEWS.d/3.10.0a2.rst b/Misc/NEWS.d/3.10.0a2.rst index eeb179a980bb8..78b25779802d6 100644 --- a/Misc/NEWS.d/3.10.0a2.rst +++ b/Misc/NEWS.d/3.10.0a2.rst @@ -847,8 +847,8 @@ Victor Stinner. .. section: C API Fix potential crash in deallocating method objects when dynamically -allocated `PyMethodDef`'s lifetime is managed through the ``self`` argument -of a `PyCFunction`. +allocated :c:type:`PyMethodDef`'s lifetime is managed through the ``self`` argument +of a :c:type:`PyCFunction`. .. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index ddd4b73275ec4..51b5c6e35a4f2 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -5934,7 +5934,7 @@ not override :c:member:`~PyTypeObject.tp_call` now inherit the .. nonce: aiRSgr .. section: C API -Creating :c:data:`immutable types ` with mutable +Creating :c:macro:`immutable types ` with mutable bases is deprecated and is planned to be disabled in Python 3.14. .. From webhook-mailer at python.org Sun Jul 23 07:56:13 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 11:56:13 -0000 Subject: [Python-checkins] [3.11] gh-101100: Fix some broken sphinx references (GH-107095). (#107120) Message-ID: https://github.com/python/cpython/commit/c51f766ebb5974024d5fc138f71350a6fb30c9d9 commit: c51f766ebb5974024d5fc138f71350a6fb30c9d9 branch: 3.11 author: wulmer committer: hugovk date: 2023-07-23T05:56:09-06:00 summary: [3.11] gh-101100: Fix some broken sphinx references (GH-107095). (#107120) files: M Doc/c-api/iterator.rst M Doc/c-api/mapping.rst M Doc/c-api/refcounting.rst M Doc/c-api/sequence.rst M Doc/howto/functional.rst M Doc/howto/regex.rst M Doc/howto/sorting.rst M Doc/howto/unicode.rst M Doc/library/_thread.rst M Doc/library/codeop.rst M Doc/library/constants.rst diff --git a/Doc/c-api/iterator.rst b/Doc/c-api/iterator.rst index 3fcf099134d4d..95952237ca746 100644 --- a/Doc/c-api/iterator.rst +++ b/Doc/c-api/iterator.rst @@ -6,7 +6,7 @@ Iterator Objects ---------------- Python provides two general-purpose iterator objects. The first, a sequence -iterator, works with an arbitrary sequence supporting the :meth:`__getitem__` +iterator, works with an arbitrary sequence supporting the :meth:`~object.__getitem__` method. The second works with a callable object and a sentinel value, calling the callable for each item in the sequence, and ending the iteration when the sentinel value is returned. diff --git a/Doc/c-api/mapping.rst b/Doc/c-api/mapping.rst index cffb0ed50fb77..d94a9dc45b5eb 100644 --- a/Doc/c-api/mapping.rst +++ b/Doc/c-api/mapping.rst @@ -13,7 +13,7 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and Return ``1`` if the object provides the mapping protocol or supports slicing, and ``0`` otherwise. Note that it returns ``1`` for Python classes with - a :meth:`__getitem__` method, since in general it is impossible to + a :meth:`~object.__getitem__` method, since in general it is impossible to determine what type of keys the class supports. This function always succeeds. @@ -60,7 +60,7 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and This is equivalent to the Python expression ``key in o``. This function always succeeds. - Note that exceptions which occur while calling the :meth:`__getitem__` + Note that exceptions which occur while calling the :meth:`~object.__getitem__` method will get suppressed. To get error reporting use :c:func:`PyObject_GetItem()` instead. @@ -71,7 +71,7 @@ See also :c:func:`PyObject_GetItem`, :c:func:`PyObject_SetItem` and This is equivalent to the Python expression ``key in o``. This function always succeeds. - Note that exceptions which occur while calling the :meth:`__getitem__` + Note that exceptions which occur while calling the :meth:`~object.__getitem__` method and creating a temporary string object will get suppressed. To get error reporting use :c:func:`PyMapping_GetItemString()` instead. diff --git a/Doc/c-api/refcounting.rst b/Doc/c-api/refcounting.rst index 738bd77e9ce42..af25e8bb91368 100644 --- a/Doc/c-api/refcounting.rst +++ b/Doc/c-api/refcounting.rst @@ -81,7 +81,7 @@ objects. .. warning:: The deallocation function can cause arbitrary Python code to be invoked (e.g. - when a class instance with a :meth:`__del__` method is deallocated). While + when a class instance with a :meth:`~object.__del__` method is deallocated). While exceptions in such code are not propagated, the executed code has free access to all Python global variables. This means that any object that is reachable from a global variable should be in a consistent state before :c:func:`Py_DECREF` is diff --git a/Doc/c-api/sequence.rst b/Doc/c-api/sequence.rst index 402a3e5e09ff5..ce28839f5ba73 100644 --- a/Doc/c-api/sequence.rst +++ b/Doc/c-api/sequence.rst @@ -9,7 +9,7 @@ Sequence Protocol .. c:function:: int PySequence_Check(PyObject *o) Return ``1`` if the object provides the sequence protocol, and ``0`` otherwise. - Note that it returns ``1`` for Python classes with a :meth:`__getitem__` + Note that it returns ``1`` for Python classes with a :meth:`~object.__getitem__` method, unless they are :class:`dict` subclasses, since in general it is impossible to determine what type of keys the class supports. This function always succeeds. diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst index 64f2293b21e2e..efc25dac695da 100644 --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -1072,8 +1072,8 @@ write the obvious :keyword:`for` loop:: A related function is :func:`itertools.accumulate(iterable, func=operator.add) `. It performs the same calculation, but instead of -returning only the final result, :func:`accumulate` returns an iterator that -also yields each partial result:: +returning only the final result, :func:`~itertools.accumulate` returns an iterator +that also yields each partial result:: itertools.accumulate([1, 2, 3, 4, 5]) => 1, 3, 6, 10, 15 diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst index 5cd6140f19ca2..8d95d86ba398b 100644 --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -522,6 +522,8 @@ cache. Compilation Flags ----------------- +.. currentmodule:: re + Compilation flags let you modify some aspects of how regular expressions work. Flags are available in the :mod:`re` module under two names, a long name such as :const:`IGNORECASE` and a short, one-letter form such as :const:`I`. (If you're diff --git a/Doc/howto/sorting.rst b/Doc/howto/sorting.rst index decce12bf3faf..38dd09f0a721d 100644 --- a/Doc/howto/sorting.rst +++ b/Doc/howto/sorting.rst @@ -273,7 +273,7 @@ Odds and Ends * The sort routines use ``<`` when making comparisons between two objects. So, it is easy to add a standard sort order to a class by - defining an :meth:`__lt__` method: + defining an :meth:`~object.__lt__` method: .. doctest:: @@ -281,8 +281,8 @@ Odds and Ends >>> sorted(student_objects) [('dave', 'B', 10), ('jane', 'B', 12), ('john', 'A', 15)] - However, note that ``<`` can fall back to using :meth:`__gt__` if - :meth:`__lt__` is not implemented (see :func:`object.__lt__`). + However, note that ``<`` can fall back to using :meth:`~object.__gt__` if + :meth:`~object.__lt__` is not implemented (see :func:`object.__lt__`). * Key functions need not depend directly on the objects being sorted. A key function can also access external resources. For instance, if the student grades diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst index ca09aee72bf87..a5e0998ce8dcf 100644 --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -424,8 +424,8 @@ lowercase letters 'ss'. A second tool is the :mod:`unicodedata` module's :func:`~unicodedata.normalize` function that converts strings to one -of several normal forms, where letters followed by a combining -character are replaced with single characters. :func:`normalize` can +of several normal forms, where letters followed by a combining character are +replaced with single characters. :func:`~unicodedata.normalize` can be used to perform string comparisons that won't falsely report inequality if two strings use combining characters differently: @@ -474,8 +474,8 @@ The Unicode Standard also specifies how to do caseless comparisons:: print(compare_caseless(single_char, multiple_chars)) -This will print ``True``. (Why is :func:`NFD` invoked twice? Because -there are a few characters that make :meth:`casefold` return a +This will print ``True``. (Why is :func:`!NFD` invoked twice? Because +there are a few characters that make :meth:`~str.casefold` return a non-normalized string, so the result needs to be normalized again. See section 3.13 of the Unicode Standard for a discussion and an example.) diff --git a/Doc/library/_thread.rst b/Doc/library/_thread.rst index 21f7847d679af..4227a4fe1e33a 100644 --- a/Doc/library/_thread.rst +++ b/Doc/library/_thread.rst @@ -148,8 +148,8 @@ This module defines the following constants and functions: .. data:: TIMEOUT_MAX The maximum value allowed for the *timeout* parameter of - :meth:`Lock.acquire`. Specifying a timeout greater than this value will - raise an :exc:`OverflowError`. + :meth:`Lock.acquire `. Specifying a timeout greater + than this value will raise an :exc:`OverflowError`. .. versionadded:: 3.2 @@ -215,8 +215,9 @@ In addition to these methods, lock objects can also be used via the * Calling :func:`sys.exit` or raising the :exc:`SystemExit` exception is equivalent to calling :func:`_thread.exit`. -* It is not possible to interrupt the :meth:`acquire` method on a lock --- the - :exc:`KeyboardInterrupt` exception will happen after the lock has been acquired. +* It is not possible to interrupt the :meth:`~threading.Lock.acquire` method on + a lock --- the :exc:`KeyboardInterrupt` exception will happen after the lock + has been acquired. * When the main thread exits, it is system defined whether the other threads survive. On most systems, they are killed without executing diff --git a/Doc/library/codeop.rst b/Doc/library/codeop.rst index 90df499f8207b..55606e1c5f09a 100644 --- a/Doc/library/codeop.rst +++ b/Doc/library/codeop.rst @@ -58,7 +58,7 @@ To do just the former: .. class:: Compile() - Instances of this class have :meth:`__call__` methods identical in signature to + Instances of this class have :meth:`~object.__call__` methods identical in signature to the built-in function :func:`compile`, but with the difference that if the instance compiles program text containing a :mod:`__future__` statement, the instance 'remembers' and compiles all subsequent program texts with the @@ -67,7 +67,7 @@ To do just the former: .. class:: CommandCompiler() - Instances of this class have :meth:`__call__` methods identical in signature to + Instances of this class have :meth:`~object.__call__` methods identical in signature to :func:`compile_command`; the difference is that if the instance compiles program text containing a :mod:`__future__` statement, the instance 'remembers' and compiles all subsequent program texts with the statement in force. diff --git a/Doc/library/constants.rst b/Doc/library/constants.rst index 38dd552a0363a..401dc9a320c5e 100644 --- a/Doc/library/constants.rst +++ b/Doc/library/constants.rst @@ -22,16 +22,16 @@ A small number of constants live in the built-in namespace. They are: An object frequently used to represent the absence of a value, as when default arguments are not passed to a function. Assignments to ``None`` are illegal and raise a :exc:`SyntaxError`. - ``None`` is the sole instance of the :data:`NoneType` type. + ``None`` is the sole instance of the :data:`~types.NoneType` type. .. data:: NotImplemented A special value which should be returned by the binary special methods - (e.g. :meth:`__eq__`, :meth:`__lt__`, :meth:`__add__`, :meth:`__rsub__`, + (e.g. :meth:`~object.__eq__`, :meth:`~object.__lt__`, :meth:`~object.__add__`, :meth:`~object.__rsub__`, etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods - (e.g. :meth:`__imul__`, :meth:`__iand__`, etc.) for the same purpose. + (e.g. :meth:`~object.__imul__`, :meth:`~object.__iand__`, etc.) for the same purpose. It should not be evaluated in a boolean context. ``NotImplemented`` is the sole instance of the :data:`types.NotImplementedType` type. From webhook-mailer at python.org Sun Jul 23 08:06:36 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 23 Jul 2023 12:06:36 -0000 Subject: [Python-checkins] [3.11] gh-107091: Fix the use of some C domain roles (GH-107092) (GH-107121) Message-ID: https://github.com/python/cpython/commit/d2cdf0888bdab86d21cd70749b2d9a0eb426fa76 commit: d2cdf0888bdab86d21cd70749b2d9a0eb426fa76 branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-23T15:06:32+03:00 summary: [3.11] gh-107091: Fix the use of some C domain roles (GH-107092) (GH-107121) (cherry picked from commit 08a228da05a7aec937b65eea21f4091fa3c6b5cf) files: M Doc/c-api/buffer.rst M Doc/c-api/method.rst M Doc/c-api/module.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Doc/extending/extending.rst M Doc/howto/isolating-extensions.rst M Doc/whatsnew/3.11.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.10.0a2.rst diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst index 91d1edd9b2ec4..6e5443f0d6cdc 100644 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -225,7 +225,7 @@ object via :c:func:`PyObject_GetBuffer`. Since the complexity of the logical structure of the memory can vary drastically, the consumer uses the *flags* argument to specify the exact buffer type it can handle. -All :c:data:`Py_buffer` fields are unambiguously defined by the request +All :c:type:`Py_buffer` fields are unambiguously defined by the request type. request-independent fields @@ -464,7 +464,7 @@ Buffer-related functions .. c:function:: Py_ssize_t PyBuffer_SizeFromFormat(const char *format) - Return the implied :c:data:`~Py_buffer.itemsize` from :c:data:`~Py_buffer.format`. + Return the implied :c:member:`~Py_buffer.itemsize` from :c:member:`~Py_buffer.format`. On error, raise an exception and return -1. .. versionadded:: 3.9 diff --git a/Doc/c-api/method.rst b/Doc/c-api/method.rst index 93ad30cd4f7a8..0d75ab8e1af11 100644 --- a/Doc/c-api/method.rst +++ b/Doc/c-api/method.rst @@ -7,8 +7,8 @@ Instance Method Objects .. index:: pair: object; instancemethod -An instance method is a wrapper for a :c:data:`PyCFunction` and the new way -to bind a :c:data:`PyCFunction` to a class object. It replaces the former call +An instance method is a wrapper for a :c:type:`PyCFunction` and the new way +to bind a :c:type:`PyCFunction` to a class object. It replaces the former call ``PyMethod_New(func, NULL, class)``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index bc8e3b23b9957..e358c5da14a69 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -145,7 +145,7 @@ or request "multi-phase initialization" by returning the definition struct itsel .. c:member:: PyModuleDef_Base m_base - Always initialize this member to :c:data:`PyModuleDef_HEAD_INIT`. + Always initialize this member to :c:macro:`PyModuleDef_HEAD_INIT`. .. c:member:: const char *m_name diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index a4ca724ad671a..d6f32ed88b673 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -149,7 +149,7 @@ Type Objects ``Py_TYPE(self)`` may be a *subclass* of the intended class, and subclasses are not necessarily defined in the same module as their superclass. See :c:type:`PyCMethod` to get the class that defines the method. - See :c:func:`PyType_GetModuleByDef` for cases when ``PyCMethod`` cannot + See :c:func:`PyType_GetModuleByDef` for cases when :c:type:`!PyCMethod` cannot be used. .. versionadded:: 3.9 diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 32989841c5b1a..006e11b2e7235 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -2169,8 +2169,8 @@ Number Object Structures .. note:: - The :c:data:`nb_reserved` field should always be ``NULL``. It - was previously called :c:data:`nb_long`, and was renamed in + The :c:member:`~PyNumberMethods.nb_reserved` field should always be ``NULL``. It + was previously called :c:member:`!nb_long`, and was renamed in Python 3.0.1. .. c:member:: binaryfunc PyNumberMethods.nb_add diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 76e0490d0d22d..097d86e30269c 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -235,10 +235,10 @@ Note that the Python name for the exception object is :exc:`spam.error`. The being :exc:`Exception` (unless another class is passed in instead of ``NULL``), described in :ref:`bltin-exceptions`. -Note also that the :c:data:`SpamError` variable retains a reference to the newly +Note also that the :c:data:`!SpamError` variable retains a reference to the newly created exception class; this is intentional! Since the exception could be removed from the module by external code, an owned reference to the class is -needed to ensure that it will not be discarded, causing :c:data:`SpamError` to +needed to ensure that it will not be discarded, causing :c:data:`!SpamError` to become a dangling pointer. Should it become a dangling pointer, C code which raises the exception could cause a core dump or other unintended side effects. @@ -279,9 +279,9 @@ statement:: It returns ``NULL`` (the error indicator for functions returning object pointers) if an error is detected in the argument list, relying on the exception set by :c:func:`PyArg_ParseTuple`. Otherwise the string value of the argument has been -copied to the local variable :c:data:`command`. This is a pointer assignment and +copied to the local variable :c:data:`!command`. This is a pointer assignment and you are not supposed to modify the string to which it points (so in Standard C, -the variable :c:data:`command` should properly be declared as ``const char +the variable :c:data:`!command` should properly be declared as ``const char *command``). The next statement is a call to the Unix function :c:func:`system`, passing it @@ -289,7 +289,7 @@ the string we just got from :c:func:`PyArg_ParseTuple`:: sts = system(command); -Our :func:`spam.system` function must return the value of :c:data:`sts` as a +Our :func:`!spam.system` function must return the value of :c:data:`!sts` as a Python object. This is done using the function :c:func:`PyLong_FromLong`. :: return PyLong_FromLong(sts); diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index 522e0853b496a..c88c9edc0ccad 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -483,14 +483,14 @@ to get the state:: return NULL; } -``PyType_GetModuleByDef`` works by searching the +:c:func:`!PyType_GetModuleByDef` works by searching the :term:`method resolution order` (i.e. all superclasses) for the first superclass that has a corresponding module. .. note:: In very exotic cases (inheritance chains spanning multiple modules - created from the same definition), ``PyType_GetModuleByDef`` might not + created from the same definition), :c:func:`!PyType_GetModuleByDef` might not return the module of the true defining class. However, it will always return a module with the same definition, ensuring a compatible C memory layout. diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index f31b439ed2fd4..91f54f38428a7 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -2227,7 +2227,7 @@ New Features (Contributed by Christian Heimes in :issue:`45459`.) -* Added the :c:data:`PyType_GetModuleByDef` function, used to get the module +* Added the :c:func:`PyType_GetModuleByDef` function, used to get the module in which a method was defined, in cases where this information is not available directly (via :c:type:`PyCMethod`). (Contributed by Petr Viktorin in :issue:`46613`.) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 23db4022dc06a..6de432d6c036b 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1276,7 +1276,7 @@ New Features * :pep:`573`: Added :c:func:`PyType_FromModuleAndSpec` to associate a module with a class; :c:func:`PyType_GetModule` and :c:func:`PyType_GetModuleState` to retrieve the module and its state; and - :c:data:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to + :c:type:`PyCMethod` and :c:macro:`METH_METHOD` to allow a method to access the class it was defined in. (Contributed by Marcel Plch and Petr Viktorin in :issue:`38787`.) diff --git a/Misc/NEWS.d/3.10.0a2.rst b/Misc/NEWS.d/3.10.0a2.rst index cafd4804a1ee6..9f85616c9f28d 100644 --- a/Misc/NEWS.d/3.10.0a2.rst +++ b/Misc/NEWS.d/3.10.0a2.rst @@ -847,8 +847,8 @@ Victor Stinner. .. section: C API Fix potential crash in deallocating method objects when dynamically -allocated `PyMethodDef`'s lifetime is managed through the ``self`` argument -of a `PyCFunction`. +allocated :c:type:`PyMethodDef`'s lifetime is managed through the ``self`` argument +of a :c:type:`PyCFunction`. .. From webhook-mailer at python.org Sun Jul 23 08:07:27 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 12:07:27 -0000 Subject: [Python-checkins] gh-107017: Analolgy to Pascal and C replaced. (#107025) Message-ID: https://github.com/python/cpython/commit/e59da0c4f283b966ccb175fb94460f58211a9704 commit: e59da0c4f283b966ccb175fb94460f58211a9704 branch: main author: TommyUnreal <45427816+TommyUnreal at users.noreply.github.com> committer: hugovk date: 2023-07-23T06:07:24-06:00 summary: gh-107017: Analolgy to Pascal and C replaced. (#107025) Co-authored-by: Shantanu <12621235+hauntsaninja at users.noreply.github.com> files: M Doc/tutorial/introduction.rst diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index ebc2e9187534b..4b886f604aca2 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -52,8 +52,8 @@ Numbers The interpreter acts as a simple calculator: you can type an expression at it and it will write the value. Expression syntax is straightforward: the -operators ``+``, ``-``, ``*`` and ``/`` work just like in most other languages -(for example, Pascal or C); parentheses (``()``) can be used for grouping. +operators ``+``, ``-``, ``*`` and ``/`` can be used to perform +arithmetic; parentheses (``()``) can be used for grouping. For example:: >>> 2 + 2 From webhook-mailer at python.org Sun Jul 23 08:28:31 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 12:28:31 -0000 Subject: [Python-checkins] [3.12] gh-106186: Don't report MultipartInvariantViolationDefect for valid multipart emails when parsing header only (GH-107016) (#107111) Message-ID: https://github.com/python/cpython/commit/af95a1da465da82740541bf011a324fefaf53713 commit: af95a1da465da82740541bf011a324fefaf53713 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-23T14:28:28+02:00 summary: [3.12] gh-106186: Don't report MultipartInvariantViolationDefect for valid multipart emails when parsing header only (GH-107016) (#107111) (cherry picked from commit c65592c4d6d7552fb6284442906a96a6874cb266) Co-authored-by: htsedebenham <31847376+htsedebenham at users.noreply.github.com> files: A Lib/test/test_email/data/msg_47.txt A Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst M Lib/email/feedparser.py M Lib/test/test_email/test_email.py diff --git a/Lib/email/feedparser.py b/Lib/email/feedparser.py index 885097c7dda06..c2881d9bc52ba 100644 --- a/Lib/email/feedparser.py +++ b/Lib/email/feedparser.py @@ -187,7 +187,7 @@ def close(self): assert not self._msgstack # Look for final set of defects if root.get_content_maintype() == 'multipart' \ - and not root.is_multipart(): + and not root.is_multipart() and not self._headersonly: defect = errors.MultipartInvariantViolationDefect() self.policy.handle_defect(root, defect) return root diff --git a/Lib/test/test_email/data/msg_47.txt b/Lib/test/test_email/data/msg_47.txt new file mode 100644 index 0000000000000..bb48b47d96baf --- /dev/null +++ b/Lib/test/test_email/data/msg_47.txt @@ -0,0 +1,14 @@ +Date: 01 Jan 2001 00:01+0000 +From: arthur at example.example +MIME-Version: 1.0 +Content-Type: multipart/mixed; boundary=foo + +--foo +Content-Type: text/plain +bar + +--foo +Content-Type: text/html +

baz

+ +--foo-- \ No newline at end of file diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index b4f3a2481976e..cdb6ef1275e52 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3712,6 +3712,16 @@ def test_bytes_header_parser(self): self.assertIsInstance(msg.get_payload(), str) self.assertIsInstance(msg.get_payload(decode=True), bytes) + def test_header_parser_multipart_is_valid(self): + # Don't flag valid multipart emails as having defects + with openfile('msg_47.txt', encoding="utf-8") as fp: + msgdata = fp.read() + + parser = email.parser.Parser(policy=email.policy.default) + parsed_msg = parser.parsestr(msgdata, headersonly=True) + + self.assertEqual(parsed_msg.defects, []) + def test_bytes_parser_does_not_close_file(self): with openfile('msg_02.txt', 'rb') as fp: email.parser.BytesParser().parse(fp) diff --git a/Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst b/Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst new file mode 100644 index 0000000000000..07fdcc96fa38a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst @@ -0,0 +1,3 @@ +Do not report ``MultipartInvariantViolationDefect`` defect +when the :class:`email.parser.Parser` class is used +to parse emails with ``headersonly=True``. From webhook-mailer at python.org Sun Jul 23 08:28:39 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 12:28:39 -0000 Subject: [Python-checkins] [3.11] gh-106186: Don't report MultipartInvariantViolationDefect for valid multipart emails when parsing header only (GH-107016) (#107112) Message-ID: https://github.com/python/cpython/commit/afa24d52b821c2020e0966f96a6205bca7db85e9 commit: afa24d52b821c2020e0966f96a6205bca7db85e9 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-23T14:28:35+02:00 summary: [3.11] gh-106186: Don't report MultipartInvariantViolationDefect for valid multipart emails when parsing header only (GH-107016) (#107112) (cherry picked from commit c65592c4d6d7552fb6284442906a96a6874cb266) Co-authored-by: htsedebenham <31847376+htsedebenham at users.noreply.github.com> files: A Lib/test/test_email/data/msg_47.txt A Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst M Lib/email/feedparser.py M Lib/test/test_email/test_email.py diff --git a/Lib/email/feedparser.py b/Lib/email/feedparser.py index 97d3f5144d606..e400dc7fb89b0 100644 --- a/Lib/email/feedparser.py +++ b/Lib/email/feedparser.py @@ -189,7 +189,7 @@ def close(self): assert not self._msgstack # Look for final set of defects if root.get_content_maintype() == 'multipart' \ - and not root.is_multipart(): + and not root.is_multipart() and not self._headersonly: defect = errors.MultipartInvariantViolationDefect() self.policy.handle_defect(root, defect) return root diff --git a/Lib/test/test_email/data/msg_47.txt b/Lib/test/test_email/data/msg_47.txt new file mode 100644 index 0000000000000..bb48b47d96baf --- /dev/null +++ b/Lib/test/test_email/data/msg_47.txt @@ -0,0 +1,14 @@ +Date: 01 Jan 2001 00:01+0000 +From: arthur at example.example +MIME-Version: 1.0 +Content-Type: multipart/mixed; boundary=foo + +--foo +Content-Type: text/plain +bar + +--foo +Content-Type: text/html +

baz

+ +--foo-- \ No newline at end of file diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index ca8212825ffca..2bb651609f572 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3696,6 +3696,16 @@ def test_bytes_header_parser(self): self.assertIsInstance(msg.get_payload(), str) self.assertIsInstance(msg.get_payload(decode=True), bytes) + def test_header_parser_multipart_is_valid(self): + # Don't flag valid multipart emails as having defects + with openfile('msg_47.txt', encoding="utf-8") as fp: + msgdata = fp.read() + + parser = email.parser.Parser(policy=email.policy.default) + parsed_msg = parser.parsestr(msgdata, headersonly=True) + + self.assertEqual(parsed_msg.defects, []) + def test_bytes_parser_does_not_close_file(self): with openfile('msg_02.txt', 'rb') as fp: email.parser.BytesParser().parse(fp) diff --git a/Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst b/Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst new file mode 100644 index 0000000000000..07fdcc96fa38a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-13-09-28.gh-issue-106186.EIsUNG.rst @@ -0,0 +1,3 @@ +Do not report ``MultipartInvariantViolationDefect`` defect +when the :class:`email.parser.Parser` class is used +to parse emails with ``headersonly=True``. From webhook-mailer at python.org Sun Jul 23 08:29:11 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 12:29:11 -0000 Subject: [Python-checkins] [3.12] bpo-18319: gettext() can retrieve a message even if a plural form exists (GH-19869) (#107108) Message-ID: https://github.com/python/cpython/commit/bd72fb19ef1a01fde067109288ecacc62121f0ae commit: bd72fb19ef1a01fde067109288ecacc62121f0ae branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-23T14:29:08+02:00 summary: [3.12] bpo-18319: gettext() can retrieve a message even if a plural form exists (GH-19869) (#107108) (cherry picked from commit 54632528eeba841e4a8cc95ecbd84c9aca8eef57) Co-authored-by: Gilles Bassi?re files: A Misc/NEWS.d/next/Library/2020-05-03-00-33-15.bpo-18319.faPTlx.rst M Lib/gettext.py M Lib/test/test_gettext.py diff --git a/Lib/gettext.py b/Lib/gettext.py index 6c5ec4e517f63..cc938e40028c2 100644 --- a/Lib/gettext.py +++ b/Lib/gettext.py @@ -422,10 +422,12 @@ def gettext(self, message): missing = object() tmsg = self._catalog.get(message, missing) if tmsg is missing: - if self._fallback: - return self._fallback.gettext(message) - return message - return tmsg + tmsg = self._catalog.get((message, self.plural(1)), missing) + if tmsg is not missing: + return tmsg + if self._fallback: + return self._fallback.gettext(message) + return message def ngettext(self, msgid1, msgid2, n): try: diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py index 1608d1b18e98f..aa3520d2c142e 100644 --- a/Lib/test/test_gettext.py +++ b/Lib/test/test_gettext.py @@ -320,6 +320,8 @@ def test_plural_forms1(self): eq(x, 'Hay %s fichero') x = gettext.ngettext('There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros') + x = gettext.gettext('There is %s file') + eq(x, 'Hay %s fichero') def test_plural_context_forms1(self): eq = self.assertEqual @@ -338,6 +340,8 @@ def test_plural_forms2(self): eq(x, 'Hay %s fichero') x = t.ngettext('There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros') + x = t.gettext('There is %s file') + eq(x, 'Hay %s fichero') def test_plural_context_forms2(self): eq = self.assertEqual diff --git a/Misc/NEWS.d/next/Library/2020-05-03-00-33-15.bpo-18319.faPTlx.rst b/Misc/NEWS.d/next/Library/2020-05-03-00-33-15.bpo-18319.faPTlx.rst new file mode 100644 index 0000000000000..a1a4cf6d63725 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-05-03-00-33-15.bpo-18319.faPTlx.rst @@ -0,0 +1,2 @@ +Ensure `gettext(msg)` retrieve translations even if a plural form exists. In +other words: `gettext(msg) == ngettext(msg, '', 1)`. From webhook-mailer at python.org Sun Jul 23 08:58:24 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 12:58:24 -0000 Subject: [Python-checkins] [3.12] Introduce a gate/check GHA job (GH-97533) (#107114) Message-ID: https://github.com/python/cpython/commit/18c68ec81663bee058d28f9a223f5b28b0a8d4e4 commit: 18c68ec81663bee058d28f9a223f5b28b0a8d4e4 branch: 3.12 author: Sviatoslav Sydorenko committer: ambv date: 2023-07-23T14:58:20+02:00 summary: [3.12] Introduce a gate/check GHA job (GH-97533) (#107114) (cherry picked from commit e7cd557) files: M .github/workflows/build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index d6504e6ca8cf3..20208dbdd2f5f 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -562,3 +562,60 @@ jobs: run: make pythoninfo - name: Tests run: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu" + + all-required-green: # This job does nothing and is only used for the branch protection + name: All required checks pass + if: always() + + needs: + - check_source # Transitive dependency, needed to access `run_tests` value + - check-docs + - check_generated_files + - build_win32 + - build_win_amd64 + - build_macos + - build_ubuntu + - build_ubuntu_ssltests + - test_hypothesis + - build_asan + + runs-on: ubuntu-latest + + steps: + - name: Check whether the needed jobs succeeded or failed + uses: re-actors/alls-green at 05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe + with: + allowed-failures: >- + build_macos, + build_ubuntu_ssltests, + build_win32, + test_hypothesis, + allowed-skips: >- + ${{ + !fromJSON(needs.check_source.outputs.run-docs) + && ' + check-docs, + ' + || '' + }} + ${{ + needs.check_source.outputs.run_tests != 'true' + && ' + check_generated_files, + build_win32, + build_win_amd64, + build_macos, + build_ubuntu, + build_ubuntu_ssltests, + build_asan, + ' + || '' + }} + ${{ + !fromJSON(needs.check_source.outputs.run_hypothesis) + && ' + test_hypothesis, + ' + || '' + }} + jobs: ${{ toJSON(needs) }} From webhook-mailer at python.org Sun Jul 23 08:58:45 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 12:58:45 -0000 Subject: [Python-checkins] [3.11] Introduce a gate/check GHA job (GH-97533) (#107115) Message-ID: https://github.com/python/cpython/commit/0cdc3a575d14d710045084a615ef7f2536423727 commit: 0cdc3a575d14d710045084a615ef7f2536423727 branch: 3.11 author: Sviatoslav Sydorenko committer: ambv date: 2023-07-23T14:58:41+02:00 summary: [3.11] Introduce a gate/check GHA job (GH-97533) (#107115) (cherry picked from commit e7cd557) files: M .github/workflows/build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 5dc63ff64e9ca..450a445f2bc44 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -439,3 +439,60 @@ jobs: run: make pythoninfo - name: Tests run: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu" + + all-required-green: # This job does nothing and is only used for the branch protection + name: All required checks pass + if: always() + + needs: + - check_source # Transitive dependency, needed to access `run_tests` value + - check-docs + - check_generated_files + - build_win32 + - build_win_amd64 + - build_macos + - build_ubuntu + - build_ubuntu_ssltests + - test_hypothesis + - build_asan + + runs-on: ubuntu-latest + + steps: + - name: Check whether the needed jobs succeeded or failed + uses: re-actors/alls-green at 05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe + with: + allowed-failures: >- + build_macos, + build_ubuntu_ssltests, + build_win32, + test_hypothesis, + allowed-skips: >- + ${{ + !fromJSON(needs.check_source.outputs.run-docs) + && ' + check-docs, + ' + || '' + }} + ${{ + needs.check_source.outputs.run_tests != 'true' + && ' + check_generated_files, + build_win32, + build_win_amd64, + build_macos, + build_ubuntu, + build_ubuntu_ssltests, + build_asan, + ' + || '' + }} + ${{ + !fromJSON(needs.check_source.outputs.run_hypothesis) + && ' + test_hypothesis, + ' + || '' + }} + jobs: ${{ toJSON(needs) }} From webhook-mailer at python.org Sun Jul 23 09:33:23 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 13:33:23 -0000 Subject: [Python-checkins] =?utf-8?b?WzMuMTFdIPCflKUgRHJvcCBoeXBvdGhlc2lz?= =?utf-8?q?_job_dep_=40_GHA_=28=23107128=29?= Message-ID: https://github.com/python/cpython/commit/07acf5756f146b798483b1ce0411c0a3f26cdfc8 commit: 07acf5756f146b798483b1ce0411c0a3f26cdfc8 branch: 3.11 author: Sviatoslav Sydorenko committer: ambv date: 2023-07-23T13:33:19Z summary: [3.11] ? Drop hypothesis job dep @ GHA (#107128) This fixes an incorrect conflict resolution problem that happened in 0cdc3a575d14d710045084a615ef7f2536423727 while backporting PR #97533 as PR #107115 (merged prematurely). This problem caused GitHub Actions CI/CD to crash while attempting to load the workflow file definition, preventing the jobs that are defined in `.github/workflows/build.yml` from actually starting. files: M .github/workflows/build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 450a445f2bc44..e7d3488ef8d9e 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -453,7 +453,6 @@ jobs: - build_macos - build_ubuntu - build_ubuntu_ssltests - - test_hypothesis - build_asan runs-on: ubuntu-latest @@ -466,7 +465,6 @@ jobs: build_macos, build_ubuntu_ssltests, build_win32, - test_hypothesis, allowed-skips: >- ${{ !fromJSON(needs.check_source.outputs.run-docs) @@ -488,11 +486,4 @@ jobs: ' || '' }} - ${{ - !fromJSON(needs.check_source.outputs.run_hypothesis) - && ' - test_hypothesis, - ' - || '' - }} jobs: ${{ toJSON(needs) }} From webhook-mailer at python.org Sun Jul 23 09:51:16 2023 From: webhook-mailer at python.org (corona10) Date: Sun, 23 Jul 2023 13:51:16 -0000 Subject: [Python-checkins] gh-107122: Add clear method to dbm.gdbm.module (gh-107127) Message-ID: https://github.com/python/cpython/commit/b273837fea129df8477da557ad3cb23a0ed12c20 commit: b273837fea129df8477da557ad3cb23a0ed12c20 branch: main author: Dong-hee Na committer: corona10 date: 2023-07-23T13:51:12Z summary: gh-107122: Add clear method to dbm.gdbm.module (gh-107127) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-23-13-07-34.gh-issue-107122.9HFUyb.rst M Doc/library/dbm.rst M Lib/test/test_dbm_gnu.py M Modules/_gdbmmodule.c M Modules/clinic/_gdbmmodule.c.h diff --git a/Doc/library/dbm.rst b/Doc/library/dbm.rst index 2be499337a2a1..d226fa9f962cb 100644 --- a/Doc/library/dbm.rst +++ b/Doc/library/dbm.rst @@ -245,6 +245,13 @@ supported. Close the ``gdbm`` database. + .. method:: gdbm.clear() + + Remove all items from the ``gdbm`` database. + + .. versionadded:: 3.13 + + :mod:`dbm.ndbm` --- Interface based on ndbm ------------------------------------------- diff --git a/Lib/test/test_dbm_gnu.py b/Lib/test/test_dbm_gnu.py index 73602cab5180f..e20addf1f04f1 100644 --- a/Lib/test/test_dbm_gnu.py +++ b/Lib/test/test_dbm_gnu.py @@ -192,6 +192,20 @@ def test_open_with_bytes_path(self): def test_open_with_pathlib_bytes_path(self): gdbm.open(FakePath(os.fsencode(filename)), "c").close() + def test_clear(self): + kvs = [('foo', 'bar'), ('1234', '5678')] + with gdbm.open(filename, 'c') as db: + for k, v in kvs: + db[k] = v + self.assertIn(k, db) + self.assertEqual(len(db), len(kvs)) + + db.clear() + for k, v in kvs: + self.assertNotIn(k, db) + self.assertEqual(len(db), 0) + + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-23-13-07-34.gh-issue-107122.9HFUyb.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-23-13-07-34.gh-issue-107122.9HFUyb.rst new file mode 100644 index 0000000000000..c2e1c58c1b42c --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-23-13-07-34.gh-issue-107122.9HFUyb.rst @@ -0,0 +1 @@ +Add :meth:`dbm.gdbm.clear` to :mod:`dbm.gdbm`. Patch By Dong-hee Na. diff --git a/Modules/_gdbmmodule.c b/Modules/_gdbmmodule.c index bedbdc081425c..47f2da8e0e3d0 100644 --- a/Modules/_gdbmmodule.c +++ b/Modules/_gdbmmodule.c @@ -561,6 +561,37 @@ _gdbm_gdbm_sync_impl(gdbmobject *self, PyTypeObject *cls) Py_RETURN_NONE; } +/*[clinic input] +_gdbm.gdbm.clear + cls: defining_class + / +Remove all items from the database. + +[clinic start generated code]*/ + +static PyObject * +_gdbm_gdbm_clear_impl(gdbmobject *self, PyTypeObject *cls) +/*[clinic end generated code: output=673577c573318661 input=34136d52fcdd4210]*/ +{ + _gdbm_state *state = PyType_GetModuleState(cls); + assert(state != NULL); + check_gdbmobject_open(self, state->gdbm_error); + datum key; + // Invalidate cache + self->di_size = -1; + while (1) { + key = gdbm_firstkey(self->di_dbm); + if (key.dptr == NULL) { + break; + } + if (gdbm_delete(self->di_dbm, key) < 0) { + PyErr_SetString(state->gdbm_error, "cannot delete item from database"); + return NULL; + } + } + Py_RETURN_NONE; +} + static PyObject * gdbm__enter__(PyObject *self, PyObject *args) { @@ -582,6 +613,7 @@ static PyMethodDef gdbm_methods[] = { _GDBM_GDBM_SYNC_METHODDEF _GDBM_GDBM_GET_METHODDEF _GDBM_GDBM_SETDEFAULT_METHODDEF + _GDBM_GDBM_CLEAR_METHODDEF {"__enter__", gdbm__enter__, METH_NOARGS, NULL}, {"__exit__", gdbm__exit__, METH_VARARGS, NULL}, {NULL, NULL} /* sentinel */ diff --git a/Modules/clinic/_gdbmmodule.c.h b/Modules/clinic/_gdbmmodule.c.h index 5c6aeeee7789f..76f6db318f8cc 100644 --- a/Modules/clinic/_gdbmmodule.c.h +++ b/Modules/clinic/_gdbmmodule.c.h @@ -247,6 +247,28 @@ _gdbm_gdbm_sync(gdbmobject *self, PyTypeObject *cls, PyObject *const *args, Py_s return _gdbm_gdbm_sync_impl(self, cls); } +PyDoc_STRVAR(_gdbm_gdbm_clear__doc__, +"clear($self, /)\n" +"--\n" +"\n" +"Remove all items from the database."); + +#define _GDBM_GDBM_CLEAR_METHODDEF \ + {"clear", _PyCFunction_CAST(_gdbm_gdbm_clear), METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _gdbm_gdbm_clear__doc__}, + +static PyObject * +_gdbm_gdbm_clear_impl(gdbmobject *self, PyTypeObject *cls); + +static PyObject * +_gdbm_gdbm_clear(gdbmobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + if (nargs) { + PyErr_SetString(PyExc_TypeError, "clear() takes no arguments"); + return NULL; + } + return _gdbm_gdbm_clear_impl(self, cls); +} + PyDoc_STRVAR(dbmopen__doc__, "open($module, filename, flags=\'r\', mode=0o666, /)\n" "--\n" @@ -322,4 +344,4 @@ dbmopen(PyObject *module, PyObject *const *args, Py_ssize_t nargs) exit: return return_value; } -/*[clinic end generated code: output=c6e721d82335adb3 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=8c613cbd88e57480 input=a9049054013a1b77]*/ From webhook-mailer at python.org Sun Jul 23 09:59:09 2023 From: webhook-mailer at python.org (hugovk) Date: Sun, 23 Jul 2023 13:59:09 -0000 Subject: [Python-checkins] [3.11] gh-107017: Analolgy to Pascal and C replaced. (GH-107025) (#107123) Message-ID: https://github.com/python/cpython/commit/6138ecdeb80d3a62d5cef27b08669495bccbe19b commit: 6138ecdeb80d3a62d5cef27b08669495bccbe19b branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-23T13:59:05Z summary: [3.11] gh-107017: Analolgy to Pascal and C replaced. (GH-107025) (#107123) Co-authored-by: TommyUnreal <45427816+TommyUnreal at users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja at users.noreply.github.com> Co-authored-by: Hugo van Kemenade files: M Doc/tutorial/introduction.rst diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index b7a89905e4f38..e8b582dfe85d2 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -52,8 +52,8 @@ Numbers The interpreter acts as a simple calculator: you can type an expression at it and it will write the value. Expression syntax is straightforward: the -operators ``+``, ``-``, ``*`` and ``/`` work just like in most other languages -(for example, Pascal or C); parentheses (``()``) can be used for grouping. +operators ``+``, ``-``, ``*`` and ``/`` can be used to perform +arithmetic; parentheses (``()``) can be used for grouping. For example:: >>> 2 + 2 From webhook-mailer at python.org Sun Jul 23 09:59:54 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 13:59:54 -0000 Subject: [Python-checkins] [3.12] gh-107017: Analolgy to Pascal and C replaced. (GH-107025) (#107124) Message-ID: https://github.com/python/cpython/commit/e02ddb33ce4b576e83170dd773e003a2936c7821 commit: e02ddb33ce4b576e83170dd773e003a2936c7821 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-23T13:59:50Z summary: [3.12] gh-107017: Analolgy to Pascal and C replaced. (GH-107025) (#107124) (cherry picked from commit e59da0c4f283b966ccb175fb94460f58211a9704) Co-authored-by: TommyUnreal <45427816+TommyUnreal at users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja at users.noreply.github.com> files: M Doc/tutorial/introduction.rst diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index ebc2e9187534b..4b886f604aca2 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -52,8 +52,8 @@ Numbers The interpreter acts as a simple calculator: you can type an expression at it and it will write the value. Expression syntax is straightforward: the -operators ``+``, ``-``, ``*`` and ``/`` work just like in most other languages -(for example, Pascal or C); parentheses (``()``) can be used for grouping. +operators ``+``, ``-``, ``*`` and ``/`` can be used to perform +arithmetic; parentheses (``()``) can be used for grouping. For example:: >>> 2 + 2 From webhook-mailer at python.org Sun Jul 23 10:08:32 2023 From: webhook-mailer at python.org (ambv) Date: Sun, 23 Jul 2023 14:08:32 -0000 Subject: [Python-checkins] gh-62519: Make pgettext search plurals when translation is not found (#107118) Message-ID: https://github.com/python/cpython/commit/b3c34e55c053846beb35f5e4253ef237b3494bd0 commit: b3c34e55c053846beb35f5e4253ef237b3494bd0 branch: main author: Tomas R committer: ambv date: 2023-07-23T16:08:28+02:00 summary: gh-62519: Make pgettext search plurals when translation is not found (#107118) Co-authored-by: ?ukasz Langa files: A Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst M Lib/gettext.py M Lib/test/test_gettext.py diff --git a/Lib/gettext.py b/Lib/gettext.py index cc938e40028c2..b72b15f82d435 100644 --- a/Lib/gettext.py +++ b/Lib/gettext.py @@ -446,10 +446,12 @@ def pgettext(self, context, message): missing = object() tmsg = self._catalog.get(ctxt_msg_id, missing) if tmsg is missing: - if self._fallback: - return self._fallback.pgettext(context, message) - return message - return tmsg + tmsg = self._catalog.get((ctxt_msg_id, self.plural(1)), missing) + if tmsg is not missing: + return tmsg + if self._fallback: + return self._fallback.pgettext(context, message) + return message def npgettext(self, context, msgid1, msgid2, n): ctxt_msg_id = self.CONTEXT % (context, msgid1) diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py index aa3520d2c142e..8430fc234d00e 100644 --- a/Lib/test/test_gettext.py +++ b/Lib/test/test_gettext.py @@ -331,6 +331,8 @@ def test_plural_context_forms1(self): x = gettext.npgettext('With context', 'There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros (context)') + x = gettext.pgettext('With context', 'There is %s file') + eq(x, 'Hay %s fichero (context)') def test_plural_forms2(self): eq = self.assertEqual @@ -353,6 +355,8 @@ def test_plural_context_forms2(self): x = t.npgettext('With context', 'There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros (context)') + x = gettext.pgettext('With context', 'There is %s file') + eq(x, 'Hay %s fichero (context)') # Examples from http://www.gnu.org/software/gettext/manual/gettext.html diff --git a/Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst b/Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst new file mode 100644 index 0000000000000..96e2a3dcc24fb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst @@ -0,0 +1,2 @@ +Make :func:`gettext.pgettext` search plural definitions when +translation is not found. From webhook-mailer at python.org Sun Jul 23 10:26:27 2023 From: webhook-mailer at python.org (corona10) Date: Sun, 23 Jul 2023 14:26:27 -0000 Subject: [Python-checkins] gh-107122: Add clear method to dbm.ndbm module (gh-107126) Message-ID: https://github.com/python/cpython/commit/0ae4870d09de82ed5063b6998c172cc63628437e commit: 0ae4870d09de82ed5063b6998c172cc63628437e branch: main author: Dong-hee Na committer: corona10 date: 2023-07-23T14:26:23Z summary: gh-107122: Add clear method to dbm.ndbm module (gh-107126) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-23-21-16-54.gh-issue-107122.VNuNcq.rst M Doc/library/dbm.rst M Lib/test/test_dbm.py M Lib/test/test_dbm_ndbm.py M Modules/_dbmmodule.c M Modules/clinic/_dbmmodule.c.h diff --git a/Doc/library/dbm.rst b/Doc/library/dbm.rst index d226fa9f962cb..766847b971b64 100644 --- a/Doc/library/dbm.rst +++ b/Doc/library/dbm.rst @@ -320,6 +320,12 @@ to locate the appropriate header file to simplify building this module. Close the ``ndbm`` database. + .. method:: ndbm.clear() + + Remove all items from the ``ndbm`` database. + + .. versionadded:: 3.13 + :mod:`dbm.dumb` --- Portable DBM implementation ----------------------------------------------- diff --git a/Lib/test/test_dbm.py b/Lib/test/test_dbm.py index f21eebc6530c7..e3924d8ec8b5c 100644 --- a/Lib/test/test_dbm.py +++ b/Lib/test/test_dbm.py @@ -155,6 +155,21 @@ def test_keys(self): self.assertNotIn(b'xxx', d) self.assertRaises(KeyError, lambda: d[b'xxx']) + def test_clear(self): + with dbm.open(_fname, 'c') as d: + self.assertEqual(d.keys(), []) + a = [(b'a', b'b'), (b'12345678910', b'019237410982340912840198242')] + for k, v in a: + d[k] = v + for k, _ in a: + self.assertIn(k, d) + self.assertEqual(len(d), len(a)) + + d.clear() + self.assertEqual(len(d), 0) + for k, _ in a: + self.assertNotIn(k, d) + def setUp(self): self.addCleanup(setattr, dbm, '_defaultmod', dbm._defaultmod) dbm._defaultmod = self.module diff --git a/Lib/test/test_dbm_ndbm.py b/Lib/test/test_dbm_ndbm.py index 8f37e3cc624e2..e0f31c9a9a337 100644 --- a/Lib/test/test_dbm_ndbm.py +++ b/Lib/test/test_dbm_ndbm.py @@ -147,6 +147,19 @@ def test_bool_on_closed_db_raises(self): db['a'] = 'b' self.assertRaises(dbm.ndbm.error, bool, db) + def test_clear(self): + kvs = [('foo', 'bar'), ('1234', '5678')] + with dbm.ndbm.open(self.filename, 'c') as db: + for k, v in kvs: + db[k] = v + self.assertIn(k, db) + self.assertEqual(len(db), len(kvs)) + + db.clear() + for k, v in kvs: + self.assertNotIn(k, db) + self.assertEqual(len(db), 0) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-23-21-16-54.gh-issue-107122.VNuNcq.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-23-21-16-54.gh-issue-107122.VNuNcq.rst new file mode 100644 index 0000000000000..d63aeb54ae145 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-23-21-16-54.gh-issue-107122.VNuNcq.rst @@ -0,0 +1 @@ +Add :meth:`dbm.ndbm.clear` to :mod:`dbm.ndbm`. Patch By Dong-hee Na. diff --git a/Modules/_dbmmodule.c b/Modules/_dbmmodule.c index 5be444d53e8da..bd807698927e8 100644 --- a/Modules/_dbmmodule.c +++ b/Modules/_dbmmodule.c @@ -414,6 +414,38 @@ _dbm_dbm_setdefault_impl(dbmobject *self, PyTypeObject *cls, const char *key, return default_value; } +/*[clinic input] +_dbm.dbm.clear + cls: defining_class + / +Remove all items from the database. + +[clinic start generated code]*/ + +static PyObject * +_dbm_dbm_clear_impl(dbmobject *self, PyTypeObject *cls) +/*[clinic end generated code: output=8d126b9e1d01a434 input=43aa6ca1acb7f5f5]*/ +{ + _dbm_state *state = PyType_GetModuleState(cls); + assert(state != NULL); + check_dbmobject_open(self, state->dbm_error); + datum key; + // Invalidate cache + self->di_size = -1; + while (1) { + key = dbm_firstkey(self->di_dbm); + if (key.dptr == NULL) { + break; + } + if (dbm_delete(self->di_dbm, key) < 0) { + dbm_clearerr(self->di_dbm); + PyErr_SetString(state->dbm_error, "cannot delete item from database"); + return NULL; + } + } + Py_RETURN_NONE; +} + static PyObject * dbm__enter__(PyObject *self, PyObject *args) { @@ -431,6 +463,7 @@ static PyMethodDef dbm_methods[] = { _DBM_DBM_KEYS_METHODDEF _DBM_DBM_GET_METHODDEF _DBM_DBM_SETDEFAULT_METHODDEF + _DBM_DBM_CLEAR_METHODDEF {"__enter__", dbm__enter__, METH_NOARGS, NULL}, {"__exit__", dbm__exit__, METH_VARARGS, NULL}, {NULL, NULL} /* sentinel */ diff --git a/Modules/clinic/_dbmmodule.c.h b/Modules/clinic/_dbmmodule.c.h index 172dc4b9d5793..98aac07423c8a 100644 --- a/Modules/clinic/_dbmmodule.c.h +++ b/Modules/clinic/_dbmmodule.c.h @@ -138,6 +138,28 @@ _dbm_dbm_setdefault(dbmobject *self, PyTypeObject *cls, PyObject *const *args, P return return_value; } +PyDoc_STRVAR(_dbm_dbm_clear__doc__, +"clear($self, /)\n" +"--\n" +"\n" +"Remove all items from the database."); + +#define _DBM_DBM_CLEAR_METHODDEF \ + {"clear", _PyCFunction_CAST(_dbm_dbm_clear), METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _dbm_dbm_clear__doc__}, + +static PyObject * +_dbm_dbm_clear_impl(dbmobject *self, PyTypeObject *cls); + +static PyObject * +_dbm_dbm_clear(dbmobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + if (nargs) { + PyErr_SetString(PyExc_TypeError, "clear() takes no arguments"); + return NULL; + } + return _dbm_dbm_clear_impl(self, cls); +} + PyDoc_STRVAR(dbmopen__doc__, "open($module, filename, flags=\'r\', mode=0o666, /)\n" "--\n" @@ -200,4 +222,4 @@ dbmopen(PyObject *module, PyObject *const *args, Py_ssize_t nargs) exit: return return_value; } -/*[clinic end generated code: output=28dcf736654137c2 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=b3053c67ecfcc29c input=a9049054013a1b77]*/ From webhook-mailer at python.org Sun Jul 23 11:08:53 2023 From: webhook-mailer at python.org (corona10) Date: Sun, 23 Jul 2023 15:08:53 -0000 Subject: [Python-checkins] gh-107122: Update what's news for dbm.*dbm.clear() method (gh-107135) Message-ID: https://github.com/python/cpython/commit/d1b839b4538fbfc5a3f28d9b0b0422ec026e334c commit: d1b839b4538fbfc5a3f28d9b0b0422ec026e334c branch: main author: Dong-hee Na committer: corona10 date: 2023-07-24T00:08:50+09:00 summary: gh-107122: Update what's news for dbm.*dbm.clear() method (gh-107135) files: M Doc/whatsnew/3.13.rst M Misc/NEWS.d/next/Core and Builtins/2023-07-23-13-07-34.gh-issue-107122.9HFUyb.rst M Misc/NEWS.d/next/Core and Builtins/2023-07-23-21-16-54.gh-issue-107122.VNuNcq.rst diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 1d34d8a0fa463..8ca0abf552576 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -101,6 +101,13 @@ array It can be used instead of ``'u'`` type code, which is deprecated. (Contributed by Inada Naoki in :gh:`80480`.) +dbm +--- + +* Add :meth:`dbm.gnu.gdbm.clear` and :meth:`dbm.ndbm.ndbm.clear` methods that remove all items + from the database. + (Contributed by Dong-hee Na in :gh:`107122`.) + io -- diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-23-13-07-34.gh-issue-107122.9HFUyb.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-23-13-07-34.gh-issue-107122.9HFUyb.rst index c2e1c58c1b42c..64ac8ac6df09b 100644 --- a/Misc/NEWS.d/next/Core and Builtins/2023-07-23-13-07-34.gh-issue-107122.9HFUyb.rst +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-23-13-07-34.gh-issue-107122.9HFUyb.rst @@ -1 +1 @@ -Add :meth:`dbm.gdbm.clear` to :mod:`dbm.gdbm`. Patch By Dong-hee Na. +Add :meth:`dbm.gnu.gdbm.clear` to :mod:`dbm.gnu`. Patch By Dong-hee Na. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-23-21-16-54.gh-issue-107122.VNuNcq.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-23-21-16-54.gh-issue-107122.VNuNcq.rst index d63aeb54ae145..5b7cc98ddc641 100644 --- a/Misc/NEWS.d/next/Core and Builtins/2023-07-23-21-16-54.gh-issue-107122.VNuNcq.rst +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-23-21-16-54.gh-issue-107122.VNuNcq.rst @@ -1 +1 @@ -Add :meth:`dbm.ndbm.clear` to :mod:`dbm.ndbm`. Patch By Dong-hee Na. +Add :meth:`dbm.ndbm.ndbm.clear` to :mod:`dbm.ndbm`. Patch By Dong-hee Na. From webhook-mailer at python.org Sun Jul 23 11:18:54 2023 From: webhook-mailer at python.org (JulienPalard) Date: Sun, 23 Jul 2023 15:18:54 -0000 Subject: [Python-checkins] Remove superflous whitespaces in `layout.html`. (GH-107067) Message-ID: https://github.com/python/cpython/commit/956b3de816f4b18f6b947a11c8eec6c39f60ea88 commit: 956b3de816f4b18f6b947a11c8eec6c39f60ea88 branch: main author: Ezio Melotti committer: JulienPalard date: 2023-07-23T17:18:50+02:00 summary: Remove superflous whitespaces in `layout.html`. (GH-107067) files: M Doc/tools/templates/layout.html diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index 18a49271df5f2..9832feba14167 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -4,16 +4,16 @@ {%- if outdated %}
{% trans %}This document is for an old version of Python that is no longer supported. - You should upgrade, and read the {% endtrans %} - {% trans %} Python documentation for the current stable release{% endtrans %}. + You should upgrade, and read the{% endtrans %} + {% trans %}Python documentation for the current stable release{% endtrans %}.
{%- endif %} {%- if is_deployment_preview %}
{% trans %}This is a deploy preview created from a pull request. - For authoritative documentation, see {% endtrans %} - {% trans %} the current stable release{% endtrans %}. + For authoritative documentation, see{% endtrans %} + {% trans %}the current stable release{% endtrans %}.
{%- endif %} {% endblock %} From webhook-mailer at python.org Sun Jul 23 15:07:16 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 23 Jul 2023 19:07:16 -0000 Subject: [Python-checkins] gh-106320: Remove _PyIsSelectable_fd() C API (#107142) Message-ID: https://github.com/python/cpython/commit/adb27ea2d5e2f74f4be8f15b199889161c33e801 commit: adb27ea2d5e2f74f4be8f15b199889161c33e801 branch: main author: Victor Stinner committer: vstinner date: 2023-07-23T19:07:12Z summary: gh-106320: Remove _PyIsSelectable_fd() C API (#107142) Move _PyIsSelectable_fd() macro to the internal C API (pycore_fileutils.h). files: M Include/fileobject.h M Include/internal/pycore_fileutils.h M Modules/_ssl.c diff --git a/Include/fileobject.h b/Include/fileobject.h index 2deef544d667a..6a6d11409497f 100644 --- a/Include/fileobject.h +++ b/Include/fileobject.h @@ -29,14 +29,6 @@ Py_DEPRECATED(3.12) PyAPI_DATA(int) Py_HasFileSystemDefaultEncoding; Py_DEPRECATED(3.12) PyAPI_DATA(int) Py_UTF8Mode; #endif -/* A routine to check if a file descriptor can be select()-ed. */ -#ifdef _MSC_VER - /* On Windows, any socket fd can be select()-ed, no matter how high */ - #define _PyIsSelectable_fd(FD) (1) -#else - #define _PyIsSelectable_fd(FD) ((unsigned int)(FD) < (unsigned int)FD_SETSIZE) -#endif - #ifndef Py_LIMITED_API # define Py_CPYTHON_FILEOBJECT_H # include "cpython/fileobject.h" diff --git a/Include/internal/pycore_fileutils.h b/Include/internal/pycore_fileutils.h index ef6642d00f1b5..7ba9b3ee58be4 100644 --- a/Include/internal/pycore_fileutils.h +++ b/Include/internal/pycore_fileutils.h @@ -10,6 +10,13 @@ extern "C" { #include /* struct lconv */ +/* A routine to check if a file descriptor can be select()-ed. */ +#ifdef _MSC_VER + /* On Windows, any socket fd can be select()-ed, no matter how high */ + #define _PyIsSelectable_fd(FD) (1) +#else + #define _PyIsSelectable_fd(FD) ((unsigned int)(FD) < (unsigned int)FD_SETSIZE) +#endif struct _fileutils_state { int force_ascii; diff --git a/Modules/_ssl.c b/Modules/_ssl.c index ed720b4295f8e..699431c248a07 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -26,6 +26,7 @@ #define OPENSSL_NO_DEPRECATED 1 #include "Python.h" +#include "pycore_fileutils.h" // _PyIsSelectable_fd() #include "pycore_weakref.h" // _PyWeakref_GET_REF() /* Include symbols from _socket module */ From webhook-mailer at python.org Sun Jul 23 15:16:25 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 23 Jul 2023 19:16:25 -0000 Subject: [Python-checkins] gh-106320: Remove _PyTuple_MaybeUntrack() C API (#107143) Message-ID: https://github.com/python/cpython/commit/0810b0c435415c09c1907c6f418585bed558a2c1 commit: 0810b0c435415c09c1907c6f418585bed558a2c1 branch: main author: Victor Stinner committer: vstinner date: 2023-07-23T19:16:21Z summary: gh-106320: Remove _PyTuple_MaybeUntrack() C API (#107143) Move _PyTuple_MaybeUntrack() and _PyTuple_DebugMallocStats() functions to the internal C API (pycore_tuple.h). No longer export these functions. files: M Include/cpython/tupleobject.h M Include/internal/pycore_tuple.h diff --git a/Include/cpython/tupleobject.h b/Include/cpython/tupleobject.h index 370da1612a61e..e530c8beda44a 100644 --- a/Include/cpython/tupleobject.h +++ b/Include/cpython/tupleobject.h @@ -11,7 +11,6 @@ typedef struct { } PyTupleObject; PyAPI_FUNC(int) _PyTuple_Resize(PyObject **, Py_ssize_t); -PyAPI_FUNC(void) _PyTuple_MaybeUntrack(PyObject *); /* Cast argument to PyTupleObject* type. */ #define _PyTuple_CAST(op) \ @@ -37,5 +36,3 @@ PyTuple_SET_ITEM(PyObject *op, Py_ssize_t index, PyObject *value) { } #define PyTuple_SET_ITEM(op, index, value) \ PyTuple_SET_ITEM(_PyObject_CAST(op), (index), _PyObject_CAST(value)) - -PyAPI_FUNC(void) _PyTuple_DebugMallocStats(FILE *out); diff --git a/Include/internal/pycore_tuple.h b/Include/internal/pycore_tuple.h index 335edad89792c..4fa7a12206bcb 100644 --- a/Include/internal/pycore_tuple.h +++ b/Include/internal/pycore_tuple.h @@ -8,8 +8,8 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -#include "tupleobject.h" /* _PyTuple_CAST() */ - +extern void _PyTuple_MaybeUntrack(PyObject *); +extern void _PyTuple_DebugMallocStats(FILE *out); /* runtime lifecycle */ From webhook-mailer at python.org Sun Jul 23 16:09:12 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 23 Jul 2023 20:09:12 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyObject C API (#107147) Message-ID: https://github.com/python/cpython/commit/0d6dfd68d22762c115d202515fe3310bfb42e327 commit: 0d6dfd68d22762c115d202515fe3310bfb42e327 branch: main author: Victor Stinner committer: vstinner date: 2023-07-23T20:09:08Z summary: gh-106320: Remove private _PyObject C API (#107147) Move private debug _PyObject functions to the internal C API (pycore_object.h): * _PyDebugAllocatorStats() * _PyObject_CheckConsistency() * _PyObject_DebugTypeStats() * _PyObject_IsFreed() No longer export most of these functions, except of _PyObject_IsFreed(). Move test functions using _PyObject_IsFreed() from _testcapi to _testinternalcapi. check_pyobject_is_freed() test no longer catch _testcapi.error: the tested function cannot raise _testcapi.error. files: M Include/cpython/object.h M Include/internal/pycore_object.h M Lib/test/test_capi/test_mem.py M Modules/_testcapi/mem.c M Modules/_testclinic.c M Modules/_testinternalcapi.c M Objects/dictobject.c M Objects/floatobject.c M Objects/listobject.c M Objects/obmalloc.c M Objects/tupleobject.c M Python/sysmodule.c diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 49101c1875686..90e45f65e298e 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -289,7 +289,6 @@ PyAPI_FUNC(PyObject *) PyType_GetDict(PyTypeObject *); PyAPI_FUNC(int) PyObject_Print(PyObject *, FILE *, int); PyAPI_FUNC(void) _Py_BreakPoint(void); PyAPI_FUNC(void) _PyObject_Dump(PyObject *); -PyAPI_FUNC(int) _PyObject_IsFreed(PyObject *); PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *); PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, _Py_Identifier *); @@ -377,12 +376,6 @@ PyAPI_FUNC(PyObject *) _PyObject_FunctionStr(PyObject *); #endif -PyAPI_FUNC(void) -_PyDebugAllocatorStats(FILE *out, const char *block_name, int num_blocks, - size_t sizeof_block); -PyAPI_FUNC(void) -_PyObject_DebugTypeStats(FILE *out); - /* Define a pair of assertion macros: _PyObject_ASSERT_FROM(), _PyObject_ASSERT_WITH_MSG() and _PyObject_ASSERT(). @@ -431,21 +424,6 @@ PyAPI_FUNC(void) _Py_NO_RETURN _PyObject_AssertFailed( int line, const char *function); -/* Check if an object is consistent. For example, ensure that the reference - counter is greater than or equal to 1, and ensure that ob_type is not NULL. - - Call _PyObject_AssertFailed() if the object is inconsistent. - - If check_content is zero, only check header fields: reduce the overhead. - - The function always return 1. The return value is just here to be able to - write: - - assert(_PyObject_CheckConsistency(obj, 1)); */ -PyAPI_FUNC(int) _PyObject_CheckConsistency( - PyObject *op, - int check_content); - /* Trashcan mechanism, thanks to Christian Tismer. diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h index a9c996557430a..661820c894dc0 100644 --- a/Include/internal/pycore_object.h +++ b/Include/internal/pycore_object.h @@ -14,6 +14,27 @@ extern "C" { #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_runtime.h" // _PyRuntime +/* Check if an object is consistent. For example, ensure that the reference + counter is greater than or equal to 1, and ensure that ob_type is not NULL. + + Call _PyObject_AssertFailed() if the object is inconsistent. + + If check_content is zero, only check header fields: reduce the overhead. + + The function always return 1. The return value is just here to be able to + write: + + assert(_PyObject_CheckConsistency(obj, 1)); */ +extern int _PyObject_CheckConsistency(PyObject *op, int check_content); + +extern void _PyDebugAllocatorStats(FILE *out, const char *block_name, + int num_blocks, size_t sizeof_block); + +extern void _PyObject_DebugTypeStats(FILE *out); + +// Export for shared _testinternalcapi extension +PyAPI_DATA(int) _PyObject_IsFreed(PyObject *); + /* We need to maintain an internal copy of Py{Var}Object_HEAD_INIT to avoid designated initializer conflicts in C++20. If we use the deinition in object.h, we will be mixing designated and non-designated initializers in diff --git a/Lib/test/test_capi/test_mem.py b/Lib/test/test_capi/test_mem.py index a9ff410cb93ab..527000875b724 100644 --- a/Lib/test/test_capi/test_mem.py +++ b/Lib/test/test_capi/test_mem.py @@ -8,8 +8,10 @@ from test.support.script_helper import assert_python_failure, assert_python_ok -# Skip this test if the _testcapi module isn't available. +# Skip this test if the _testcapi and _testinternalcapi extensions are not +# available. _testcapi = import_helper.import_module('_testcapi') +_testinternalcapi = import_helper.import_module('_testinternalcapi') @requires_subprocess() class PyMemDebugTests(unittest.TestCase): @@ -84,16 +86,13 @@ def test_pyobject_malloc_without_gil(self): def check_pyobject_is_freed(self, func_name): code = textwrap.dedent(f''' - import gc, os, sys, _testcapi + import gc, os, sys, _testinternalcapi # Disable the GC to avoid crash on GC collection gc.disable() - try: - _testcapi.{func_name}() - # Exit immediately to avoid a crash while deallocating - # the invalid object - os._exit(0) - except _testcapi.error: - os._exit(1) + _testinternalcapi.{func_name}() + # Exit immediately to avoid a crash while deallocating + # the invalid object + os._exit(0) ''') assert_python_ok( '-c', code, diff --git a/Modules/_testcapi/mem.c b/Modules/_testcapi/mem.c index 979b3a4b2b1af..0b2b1cd31fb0a 100644 --- a/Modules/_testcapi/mem.c +++ b/Modules/_testcapi/mem.c @@ -526,75 +526,6 @@ pymem_malloc_without_gil(PyObject *self, PyObject *args) Py_RETURN_NONE; } -static PyObject * -test_pyobject_is_freed(const char *test_name, PyObject *op) -{ - if (!_PyObject_IsFreed(op)) { - PyErr_SetString(PyExc_AssertionError, - "object is not seen as freed"); - return NULL; - } - Py_RETURN_NONE; -} - -static PyObject * -check_pyobject_null_is_freed(PyObject *self, PyObject *Py_UNUSED(args)) -{ - PyObject *op = NULL; - return test_pyobject_is_freed("check_pyobject_null_is_freed", op); -} - - -static PyObject * -check_pyobject_uninitialized_is_freed(PyObject *self, - PyObject *Py_UNUSED(args)) -{ - PyObject *op = (PyObject *)PyObject_Malloc(sizeof(PyObject)); - if (op == NULL) { - return NULL; - } - /* Initialize reference count to avoid early crash in ceval or GC */ - Py_SET_REFCNT(op, 1); - /* object fields like ob_type are uninitialized! */ - return test_pyobject_is_freed("check_pyobject_uninitialized_is_freed", op); -} - - -static PyObject * -check_pyobject_forbidden_bytes_is_freed(PyObject *self, - PyObject *Py_UNUSED(args)) -{ - /* Allocate an incomplete PyObject structure: truncate 'ob_type' field */ - PyObject *op = (PyObject *)PyObject_Malloc(offsetof(PyObject, ob_type)); - if (op == NULL) { - return NULL; - } - /* Initialize reference count to avoid early crash in ceval or GC */ - Py_SET_REFCNT(op, 1); - /* ob_type field is after the memory block: part of "forbidden bytes" - when using debug hooks on memory allocators! */ - return test_pyobject_is_freed("check_pyobject_forbidden_bytes_is_freed", op); -} - - -static PyObject * -check_pyobject_freed_is_freed(PyObject *self, PyObject *Py_UNUSED(args)) -{ - /* This test would fail if run with the address sanitizer */ -#ifdef _Py_ADDRESS_SANITIZER - Py_RETURN_NONE; -#else - PyObject *op = PyObject_CallNoArgs((PyObject *)&PyBaseObject_Type); - if (op == NULL) { - return NULL; - } - Py_TYPE(op)->tp_dealloc(op); - /* Reset reference count to avoid early crash in ceval or GC */ - Py_SET_REFCNT(op, 1); - /* object memory is freed! */ - return test_pyobject_is_freed("check_pyobject_freed_is_freed", op); -#endif -} // Tracemalloc tests static PyObject * @@ -656,12 +587,6 @@ tracemalloc_untrack(PyObject *self, PyObject *args) } static PyMethodDef test_methods[] = { - {"check_pyobject_forbidden_bytes_is_freed", - check_pyobject_forbidden_bytes_is_freed, METH_NOARGS}, - {"check_pyobject_freed_is_freed", check_pyobject_freed_is_freed, METH_NOARGS}, - {"check_pyobject_null_is_freed", check_pyobject_null_is_freed, METH_NOARGS}, - {"check_pyobject_uninitialized_is_freed", - check_pyobject_uninitialized_is_freed, METH_NOARGS}, {"pymem_api_misuse", pymem_api_misuse, METH_NOARGS}, {"pymem_buffer_overflow", pymem_buffer_overflow, METH_NOARGS}, {"pymem_getallocatorsname", test_pymem_getallocatorsname, METH_NOARGS}, diff --git a/Modules/_testclinic.c b/Modules/_testclinic.c index 26cdb4371ca24..8ba8f8ecadec7 100644 --- a/Modules/_testclinic.c +++ b/Modules/_testclinic.c @@ -6,6 +6,7 @@ #undef NDEBUG #include "Python.h" +#include "pycore_object.h" // _PyObject_IsFreed() // Used for clone_with_conv_f1 and clone_with_conv_v2 diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index ecc2721a4988f..e2be9a130217d 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -10,7 +10,6 @@ #undef NDEBUG #include "Python.h" -#include "frameobject.h" #include "pycore_atomic_funcs.h" // _Py_atomic_int_get() #include "pycore_bitutils.h" // _Py_bswap32() #include "pycore_bytesobject.h" // _PyBytes_Find() @@ -23,9 +22,12 @@ #include "pycore_initconfig.h" // _Py_GetConfigsAsDict() #include "pycore_interp.h" // _PyInterpreterState_GetConfigCopy() #include "pycore_interp_id.h" // _PyInterpreterID_LookUp() +#include "pycore_object.h" // _PyObject_IsFreed() #include "pycore_pathconfig.h" // _PyPathConfig_ClearGlobal() #include "pycore_pyerrors.h" // _Py_UTF8_Edit_Cost() #include "pycore_pystate.h" // _PyThreadState_GET() + +#include "frameobject.h" #include "osdefs.h" // MAXPATHLEN #include "clinic/_testinternalcapi.c.h" @@ -1446,6 +1448,77 @@ test_atexit(PyObject *self, PyObject *Py_UNUSED(args)) } +static PyObject * +test_pyobject_is_freed(const char *test_name, PyObject *op) +{ + if (!_PyObject_IsFreed(op)) { + PyErr_SetString(PyExc_AssertionError, + "object is not seen as freed"); + return NULL; + } + Py_RETURN_NONE; +} + +static PyObject * +check_pyobject_null_is_freed(PyObject *self, PyObject *Py_UNUSED(args)) +{ + PyObject *op = NULL; + return test_pyobject_is_freed("check_pyobject_null_is_freed", op); +} + + +static PyObject * +check_pyobject_uninitialized_is_freed(PyObject *self, + PyObject *Py_UNUSED(args)) +{ + PyObject *op = (PyObject *)PyObject_Malloc(sizeof(PyObject)); + if (op == NULL) { + return NULL; + } + /* Initialize reference count to avoid early crash in ceval or GC */ + Py_SET_REFCNT(op, 1); + /* object fields like ob_type are uninitialized! */ + return test_pyobject_is_freed("check_pyobject_uninitialized_is_freed", op); +} + + +static PyObject * +check_pyobject_forbidden_bytes_is_freed(PyObject *self, + PyObject *Py_UNUSED(args)) +{ + /* Allocate an incomplete PyObject structure: truncate 'ob_type' field */ + PyObject *op = (PyObject *)PyObject_Malloc(offsetof(PyObject, ob_type)); + if (op == NULL) { + return NULL; + } + /* Initialize reference count to avoid early crash in ceval or GC */ + Py_SET_REFCNT(op, 1); + /* ob_type field is after the memory block: part of "forbidden bytes" + when using debug hooks on memory allocators! */ + return test_pyobject_is_freed("check_pyobject_forbidden_bytes_is_freed", op); +} + + +static PyObject * +check_pyobject_freed_is_freed(PyObject *self, PyObject *Py_UNUSED(args)) +{ + /* This test would fail if run with the address sanitizer */ +#ifdef _Py_ADDRESS_SANITIZER + Py_RETURN_NONE; +#else + PyObject *op = PyObject_CallNoArgs((PyObject *)&PyBaseObject_Type); + if (op == NULL) { + return NULL; + } + Py_TYPE(op)->tp_dealloc(op); + /* Reset reference count to avoid early crash in ceval or GC */ + Py_SET_REFCNT(op, 1); + /* object memory is freed! */ + return test_pyobject_is_freed("check_pyobject_freed_is_freed", op); +#endif +} + + static PyMethodDef module_functions[] = { {"get_configs", get_configs, METH_NOARGS}, {"get_recursion_depth", get_recursion_depth, METH_NOARGS}, @@ -1502,6 +1575,12 @@ static PyMethodDef module_functions[] = { {"test_tstate_capi", test_tstate_capi, METH_NOARGS, NULL}, {"_PyUnicode_TransformDecimalAndSpaceToASCII", unicode_transformdecimalandspacetoascii, METH_O}, {"test_atexit", test_atexit, METH_NOARGS}, + {"check_pyobject_forbidden_bytes_is_freed", + check_pyobject_forbidden_bytes_is_freed, METH_NOARGS}, + {"check_pyobject_freed_is_freed", check_pyobject_freed_is_freed, METH_NOARGS}, + {"check_pyobject_null_is_freed", check_pyobject_null_is_freed, METH_NOARGS}, + {"check_pyobject_uninitialized_is_freed", + check_pyobject_uninitialized_is_freed, METH_NOARGS}, {NULL, NULL} /* sentinel */ }; diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 26e2dc31e3664..41ae1fca90b46 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -118,7 +118,7 @@ As a consequence of this, split keys have a maximum size of 16. #include "pycore_code.h" // stats #include "pycore_dict.h" // PyDictKeysObject #include "pycore_gc.h" // _PyObject_GC_IS_TRACKED() -#include "pycore_object.h" // _PyObject_GC_TRACK() +#include "pycore_object.h" // _PyObject_GC_TRACK(), _PyDebugAllocatorStats() #include "pycore_pyerrors.h" // _PyErr_GetRaisedException() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_setobject.h" // _PySet_NextEntry() diff --git a/Objects/floatobject.c b/Objects/floatobject.c index fa55481f09dec..6a0c2e033e3e9 100644 --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -10,7 +10,7 @@ #include "pycore_interp.h" // _PyInterpreterState.float_state #include "pycore_long.h" // _PyLong_GetOne() #include "pycore_modsupport.h" // _PyArg_NoKwnames() -#include "pycore_object.h" // _PyObject_Init() +#include "pycore_object.h" // _PyObject_Init(), _PyDebugAllocatorStats() #include "pycore_pymath.h" // _PY_SHORT_FLOAT_REPR #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_structseq.h" // _PyStructSequence_FiniBuiltin() diff --git a/Objects/listobject.c b/Objects/listobject.c index 144ede6351e03..c0da9dd916851 100644 --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -6,7 +6,7 @@ #include "pycore_list.h" // struct _Py_list_state, _PyListIterObject #include "pycore_long.h" // _PyLong_DigitCount #include "pycore_modsupport.h" // _PyArg_NoKwnames() -#include "pycore_object.h" // _PyObject_GC_TRACK() +#include "pycore_object.h" // _PyObject_GC_TRACK(), _PyDebugAllocatorStats() #include "pycore_tuple.h" // _PyTuple_FromArray() #include diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index eb68d7c030d29..7d552ff57c8f1 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -2,6 +2,7 @@ #include "Python.h" #include "pycore_code.h" // stats +#include "pycore_object.h" // _PyDebugAllocatorStats() definition #include "pycore_obmalloc.h" #include "pycore_pyerrors.h" // _Py_FatalErrorFormat() #include "pycore_pymem.h" diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c index e85af2b75e473..c3ff40fdb14c6 100644 --- a/Objects/tupleobject.c +++ b/Objects/tupleobject.c @@ -6,7 +6,7 @@ #include "pycore_gc.h" // _PyObject_GC_IS_TRACKED() #include "pycore_initconfig.h" // _PyStatus_OK() #include "pycore_modsupport.h" // _PyArg_NoKwnames() -#include "pycore_object.h" // _PyObject_GC_TRACK(), _Py_FatalRefcountError() +#include "pycore_object.h" // _PyObject_GC_TRACK(), _Py_FatalRefcountError(), _PyDebugAllocatorStats() /*[clinic input] class tuple "PyTupleObject *" "&PyTuple_Type" diff --git a/Python/sysmodule.c b/Python/sysmodule.c index fea3f61ee0176..fb033a6b5634a 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -22,7 +22,7 @@ Data members: #include "pycore_long.h" // _PY_LONG_MAX_STR_DIGITS_THRESHOLD #include "pycore_modsupport.h" // _PyModule_CreateInitialized() #include "pycore_namespace.h" // _PyNamespace_New() -#include "pycore_object.h" // _PyObject_IS_GC() +#include "pycore_object.h" // _PyObject_IS_GC(), _PyObject_DebugTypeStats() #include "pycore_pathconfig.h" // _PyPathConfig_ComputeSysPath0() #include "pycore_pyerrors.h" // _PyErr_GetRaisedException() #include "pycore_pylifecycle.h" // _PyErr_WriteUnraisableDefaultHook() From webhook-mailer at python.org Sun Jul 23 16:10:15 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 23 Jul 2023 20:10:15 -0000 Subject: [Python-checkins] gh-106320: Remove _PyBytes_Join() C API (#107144) Message-ID: https://github.com/python/cpython/commit/7d41ead9191a18bc473534f6f8bd1095f2232cc2 commit: 7d41ead9191a18bc473534f6f8bd1095f2232cc2 branch: main author: Victor Stinner committer: vstinner date: 2023-07-23T20:10:12Z summary: gh-106320: Remove _PyBytes_Join() C API (#107144) Move private _PyBytes functions to the internal C API (pycore_bytesobject.h): * _PyBytes_DecodeEscape() * _PyBytes_FormatEx() * _PyBytes_FromHex() * _PyBytes_Join() No longer export these functions. files: M Include/cpython/bytesobject.h M Include/internal/pycore_bytesobject.h M Modules/_io/bufferedio.c M Parser/string_parser.c diff --git a/Include/cpython/bytesobject.h b/Include/cpython/bytesobject.h index 0af4c83b1e5bc..816823716e9a6 100644 --- a/Include/cpython/bytesobject.h +++ b/Include/cpython/bytesobject.h @@ -15,18 +15,6 @@ typedef struct { } PyBytesObject; PyAPI_FUNC(int) _PyBytes_Resize(PyObject **, Py_ssize_t); -PyAPI_FUNC(PyObject*) _PyBytes_FormatEx( - const char *format, - Py_ssize_t format_len, - PyObject *args, - int use_bytearray); -PyAPI_FUNC(PyObject*) _PyBytes_FromHex( - PyObject *string, - int use_bytearray); - -/* Helper for PyBytes_DecodeEscape that detects invalid escape chars. */ -PyAPI_FUNC(PyObject *) _PyBytes_DecodeEscape(const char *, Py_ssize_t, - const char *, const char **); /* Macros and static inline functions, trading safety for speed */ #define _PyBytes_CAST(op) \ @@ -43,7 +31,3 @@ static inline Py_ssize_t PyBytes_GET_SIZE(PyObject *op) { return Py_SIZE(self); } #define PyBytes_GET_SIZE(self) PyBytes_GET_SIZE(_PyObject_CAST(self)) - -/* _PyBytes_Join(sep, x) is like sep.join(x). sep must be PyBytesObject*, - x must be an iterable object. */ -PyAPI_FUNC(PyObject *) _PyBytes_Join(PyObject *sep, PyObject *x); diff --git a/Include/internal/pycore_bytesobject.h b/Include/internal/pycore_bytesobject.h index 115c0c52c8f9a..980065a65f399 100644 --- a/Include/internal/pycore_bytesobject.h +++ b/Include/internal/pycore_bytesobject.h @@ -8,6 +8,25 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +extern PyObject* _PyBytes_FormatEx( + const char *format, + Py_ssize_t format_len, + PyObject *args, + int use_bytearray); + +extern PyObject* _PyBytes_FromHex( + PyObject *string, + int use_bytearray); + +// Helper for PyBytes_DecodeEscape that detects invalid escape chars. +// Export for test_peg_generator. +PyAPI_FUNC(PyObject*) _PyBytes_DecodeEscape(const char *, Py_ssize_t, + const char *, const char **); + +/* _PyBytes_Join(sep, x) is like sep.join(x). sep must be PyBytesObject*, + x must be an iterable object. */ +extern PyObject* _PyBytes_Join(PyObject *sep, PyObject *x); + /* Substring Search. diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c index bfc3d2558c9e3..efc8cb93d8851 100644 --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -8,8 +8,9 @@ */ #include "Python.h" +#include "pycore_bytesobject.h" // _PyBytes_Join() #include "pycore_call.h" // _PyObject_CallNoArgs() -#include "pycore_object.h" +#include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "pycore_pyerrors.h" // _Py_FatalErrorFormat() #include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing() #include "structmember.h" // PyMemberDef diff --git a/Parser/string_parser.c b/Parser/string_parser.c index bc1f99d607ae4..72898c38b79bd 100644 --- a/Parser/string_parser.c +++ b/Parser/string_parser.c @@ -1,6 +1,7 @@ #include #include +#include "pycore_bytesobject.h" // _PyBytes_DecodeEscape() #include "pycore_unicodeobject.h" // _PyUnicode_DecodeUnicodeEscapeInternal() #include "tokenizer.h" From webhook-mailer at python.org Sun Jul 23 16:57:00 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sun, 23 Jul 2023 20:57:00 -0000 Subject: [Python-checkins] gh-106948: Docs: Disable links for C standard library functions, OS utility functions and system calls (#107062) Message-ID: https://github.com/python/cpython/commit/b447e19e720e6781025432a40eb72b1cc93ac944 commit: b447e19e720e6781025432a40eb72b1cc93ac944 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-23T22:56:56+02:00 summary: gh-106948: Docs: Disable links for C standard library functions, OS utility functions and system calls (#107062) Co-authored-by: Serhiy Storchaka files: M Doc/c-api/exceptions.rst M Doc/c-api/sys.rst M Doc/conf.py M Doc/howto/instrumentation.rst M Doc/library/mailbox.rst M Doc/library/os.rst M Doc/library/select.rst M Doc/library/signal.rst M Doc/tools/.nitignore M Doc/whatsnew/2.6.rst M Doc/whatsnew/2.7.rst M Misc/NEWS.d/3.10.0a1.rst diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 3d4ceb360f5b1..4d81b273638b1 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -163,7 +163,7 @@ For convenience, some of these functions will always return a This is a convenience function to raise an exception when a C library function has returned an error and set the C variable :c:data:`errno`. It constructs a tuple object whose first item is the integer :c:data:`errno` value and whose - second item is the corresponding error message (gotten from :c:func:`strerror`), + second item is the corresponding error message (gotten from :c:func:`!strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the :c:data:`errno` value is :c:macro:`EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 433d69e40f947..6a717122c8a2f 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -106,7 +106,7 @@ Operating System Utilities .. c:function:: PyOS_sighandler_t PyOS_getsig(int i) Return the current signal handler for signal *i*. This is a thin wrapper around - either :c:func:`sigaction` or :c:func:`signal`. Do not call those functions + either :c:func:`!sigaction` or :c:func:`!signal`. Do not call those functions directly! :c:type:`PyOS_sighandler_t` is a typedef alias for :c:expr:`void (\*)(int)`. @@ -114,7 +114,7 @@ Operating System Utilities .. c:function:: PyOS_sighandler_t PyOS_setsig(int i, PyOS_sighandler_t h) Set the signal handler for signal *i* to be *h*; return the old signal handler. - This is a thin wrapper around either :c:func:`sigaction` or :c:func:`signal`. Do + This is a thin wrapper around either :c:func:`!sigaction` or :c:func:`!signal`. Do not call those functions directly! :c:type:`PyOS_sighandler_t` is a typedef alias for :c:expr:`void (\*)(int)`. diff --git a/Doc/conf.py b/Doc/conf.py index bd01850589b27..453f4fc63883a 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -77,6 +77,24 @@ exclude_patterns.append(venvdir + '/*') nitpick_ignore = [ + # Standard C functions + ('c:func', 'calloc'), + ('c:func', 'dlopen'), + ('c:func', 'exec'), + ('c:func', 'fcntl'), + ('c:func', 'fork'), + ('c:func', 'free'), + ('c:func', 'gmtime'), + ('c:func', 'localtime'), + ('c:func', 'main'), + ('c:func', 'malloc'), + ('c:func', 'printf'), + ('c:func', 'realloc'), + ('c:func', 'snprintf'), + ('c:func', 'sprintf'), + ('c:func', 'stat'), + ('c:func', 'system'), + ('c:func', 'vsnprintf'), # Standard C types ('c:type', 'FILE'), ('c:type', '__int'), diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst index 4ce15c69dac90..875f846aad005 100644 --- a/Doc/howto/instrumentation.rst +++ b/Doc/howto/instrumentation.rst @@ -292,11 +292,11 @@ Available static markers .. object:: function__return(str filename, str funcname, int lineno) - This marker is the converse of :c:func:`function__entry`, and indicates that + This marker is the converse of :c:func:`!function__entry`, and indicates that execution of a Python function has ended (either via ``return``, or via an exception). It is only triggered for pure-Python (bytecode) functions. - The arguments are the same as for :c:func:`function__entry` + The arguments are the same as for :c:func:`!function__entry` .. object:: line(str filename, str funcname, int lineno) @@ -304,7 +304,7 @@ Available static markers the equivalent of line-by-line tracing with a Python profiler. It is not triggered within C functions. - The arguments are the same as for :c:func:`function__entry`. + The arguments are the same as for :c:func:`!function__entry`. .. object:: gc__start(int generation) diff --git a/Doc/library/mailbox.rst b/Doc/library/mailbox.rst index 56908dedea1b4..91df07d914cae 100644 --- a/Doc/library/mailbox.rst +++ b/Doc/library/mailbox.rst @@ -477,7 +477,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. + :c:func:`!flock` and :c:func:`!lockf` system calls. .. seealso:: @@ -588,7 +588,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. For MH mailboxes, locking + :c:func:`!flock` and :c:func:`!lockf` system calls. For MH mailboxes, locking the mailbox means locking the :file:`.mh_sequences` file and, only for the duration of any operations that affect them, locking individual message files. @@ -686,7 +686,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. + :c:func:`!flock` and :c:func:`!lockf` system calls. .. seealso:: @@ -737,7 +737,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. + :c:func:`!flock` and :c:func:`!lockf` system calls. .. seealso:: diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 57405916ed7b3..bbf227aab649e 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -714,14 +714,14 @@ process and user. .. function:: getsid(pid, /) - Call the system call :c:func:`getsid`. See the Unix manual for the semantics. + Call the system call :c:func:`!getsid`. See the Unix manual for the semantics. .. availability:: Unix, not Emscripten, not WASI. .. function:: setsid() - Call the system call :c:func:`setsid`. See the Unix manual for the semantics. + Call the system call :c:func:`!setsid`. See the Unix manual for the semantics. .. availability:: Unix, not Emscripten, not WASI. @@ -739,7 +739,7 @@ process and user. .. function:: strerror(code, /) Return the error message corresponding to the error code in *code*. - On platforms where :c:func:`strerror` returns ``NULL`` when given an unknown + On platforms where :c:func:`!strerror` returns ``NULL`` when given an unknown error number, :exc:`ValueError` is raised. diff --git a/Doc/library/select.rst b/Doc/library/select.rst index b0891b0c8f584..c2941e628d9d7 100644 --- a/Doc/library/select.rst +++ b/Doc/library/select.rst @@ -6,10 +6,10 @@ -------------- -This module provides access to the :c:func:`select` and :c:func:`poll` functions -available in most operating systems, :c:func:`devpoll` available on -Solaris and derivatives, :c:func:`epoll` available on Linux 2.5+ and -:c:func:`kqueue` available on most BSD. +This module provides access to the :c:func:`!select` and :c:func:`!poll` functions +available in most operating systems, :c:func:`!devpoll` available on +Solaris and derivatives, :c:func:`!epoll` available on Linux 2.5+ and +:c:func:`!kqueue` available on most BSD. Note that on Windows, it only works for sockets; on other operating systems, it also works for other file types (in particular, on Unix, it works on pipes). It cannot be used on regular files to determine whether a file has grown since @@ -41,10 +41,10 @@ The module defines the following: polling object; see section :ref:`devpoll-objects` below for the methods supported by devpoll objects. - :c:func:`devpoll` objects are linked to the number of file + :c:func:`!devpoll` objects are linked to the number of file descriptors allowed at the time of instantiation. If your program - reduces this value, :c:func:`devpoll` will fail. If your program - increases this value, :c:func:`devpoll` may return an + reduces this value, :c:func:`!devpoll` will fail. If your program + increases this value, :c:func:`!devpoll` may return an incomplete list of active file descriptors. The new file descriptor is :ref:`non-inheritable `. @@ -62,7 +62,7 @@ The module defines the following: *sizehint* informs epoll about the expected number of events to be registered. It must be positive, or ``-1`` to use the default. It is only - used on older systems where :c:func:`epoll_create1` is not available; + used on older systems where :c:func:`!epoll_create1` is not available; otherwise it has no effect (though its value is still checked). *flags* is deprecated and completely ignored. However, when supplied, its @@ -117,7 +117,7 @@ The module defines the following: .. function:: select(rlist, wlist, xlist[, timeout]) - This is a straightforward interface to the Unix :c:func:`select` system call. + This is a straightforward interface to the Unix :c:func:`!select` system call. The first three arguments are iterables of 'waitable objects': either integers representing file descriptors or objects with a parameterless method named :meth:`~io.IOBase.fileno` returning such an integer: @@ -154,7 +154,7 @@ The module defines the following: .. index:: single: WinSock File objects on Windows are not acceptable, but sockets are. On Windows, - the underlying :c:func:`select` function is provided by the WinSock + the underlying :c:func:`!select` function is provided by the WinSock library, and does not handle file descriptors that don't originate from WinSock. @@ -169,7 +169,7 @@ The module defines the following: The minimum number of bytes which can be written without blocking to a pipe when the pipe has been reported as ready for writing by :func:`~select.select`, - :func:`poll` or another interface in this module. This doesn't apply + :func:`!poll` or another interface in this module. This doesn't apply to other kind of file-like objects such as sockets. This value is guaranteed by POSIX to be at least 512. @@ -184,11 +184,11 @@ The module defines the following: ``/dev/poll`` Polling Objects ----------------------------- -Solaris and derivatives have ``/dev/poll``. While :c:func:`select` is -O(highest file descriptor) and :c:func:`poll` is O(number of file +Solaris and derivatives have ``/dev/poll``. While :c:func:`!select` is +O(highest file descriptor) and :c:func:`!poll` is O(number of file descriptors), ``/dev/poll`` is O(active file descriptors). -``/dev/poll`` behaviour is very close to the standard :c:func:`poll` +``/dev/poll`` behaviour is very close to the standard :c:func:`!poll` object. @@ -222,7 +222,7 @@ object. implement :meth:`!fileno`, so they can also be used as the argument. *eventmask* is an optional bitmask describing the type of events you want to - check for. The constants are the same that with :c:func:`poll` + check for. The constants are the same that with :c:func:`!poll` object. The default value is a combination of the constants :const:`POLLIN`, :const:`POLLPRI`, and :const:`POLLOUT`. @@ -231,7 +231,7 @@ object. Registering a file descriptor that's already registered is not an error, but the result is undefined. The appropriate action is to unregister or modify it first. This is an important difference - compared with :c:func:`poll`. + compared with :c:func:`!poll`. .. method:: devpoll.modify(fd[, eventmask]) @@ -376,13 +376,13 @@ Edge and Level Trigger Polling (epoll) Objects Polling Objects --------------- -The :c:func:`poll` system call, supported on most Unix systems, provides better +The :c:func:`!poll` system call, supported on most Unix systems, provides better scalability for network servers that service many, many clients at the same -time. :c:func:`poll` scales better because the system call only requires listing -the file descriptors of interest, while :c:func:`select` builds a bitmap, turns +time. :c:func:`!poll` scales better because the system call only requires listing +the file descriptors of interest, while :c:func:`!select` builds a bitmap, turns on bits for the fds of interest, and then afterward the whole bitmap has to be -linearly scanned again. :c:func:`select` is O(highest file descriptor), while -:c:func:`poll` is O(number of file descriptors). +linearly scanned again. :c:func:`!select` is O(highest file descriptor), while +:c:func:`!poll` is O(number of file descriptors). .. method:: poll.register(fd[, eventmask]) diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index 523d1ac500136..e53315bee3ea3 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -562,7 +562,7 @@ The :mod:`signal` module defines the following functions: Note that installing a signal handler with :func:`signal` will reset the restart behaviour to interruptible by implicitly calling - :c:func:`siginterrupt` with a true *flag* value for the given signal. + :c:func:`!siginterrupt` with a true *flag* value for the given signal. .. function:: signal(signalnum, handler) diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 818276a81111e..fc6de10eb1c2c 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -56,7 +56,6 @@ Doc/glossary.rst Doc/howto/curses.rst Doc/howto/descriptor.rst Doc/howto/enum.rst -Doc/howto/instrumentation.rst Doc/howto/isolating-extensions.rst Doc/howto/logging-cookbook.rst Doc/howto/logging.rst diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index fb56690ce2511..ad899d53886c5 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -2289,7 +2289,7 @@ changes, or look through the Subversion logs for all the details. (Contributed by Raymond Hettinger; :issue:`1861`.) * The :mod:`select` module now has wrapper functions - for the Linux :c:func:`epoll` and BSD :c:func:`kqueue` system calls. + for the Linux :c:func:`!epoll` and BSD :c:func:`!kqueue` system calls. :meth:`modify` method was added to the existing :class:`poll` objects; ``pollobj.modify(fd, eventmask)`` takes a file descriptor or file object and an event mask, modifying the recorded event mask @@ -2328,7 +2328,7 @@ changes, or look through the Subversion logs for all the details. one for reading and one for writing. The writable descriptor will be passed to :func:`set_wakeup_fd`, and the readable descriptor will be added to the list of descriptors monitored by the event loop via - :c:func:`select` or :c:func:`poll`. + :c:func:`!select` or :c:func:`!poll`. On receiving a signal, a byte will be written and the main event loop will be woken up, avoiding the need to poll. @@ -2982,7 +2982,7 @@ Changes to Python's build process and to the C API include: * Python now must be compiled with C89 compilers (after 19 years!). This means that the Python source tree has dropped its - own implementations of :c:func:`memmove` and :c:func:`strerror`, which + own implementations of :c:func:`!memmove` and :c:func:`!strerror`, which are in the C89 standard library. * Python 2.6 can be built with Microsoft Visual Studio 2008 (version diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index 7bc68b0cc4e66..4b5a31f8a8481 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -355,7 +355,7 @@ added as a more powerful replacement for the This means Python now supports three different modules for parsing command-line arguments: :mod:`getopt`, :mod:`optparse`, and :mod:`argparse`. The :mod:`getopt` module closely resembles the C -library's :c:func:`getopt` function, so it remains useful if you're writing a +library's :c:func:`!getopt` function, so it remains useful if you're writing a Python prototype that will eventually be rewritten in C. :mod:`optparse` becomes redundant, but there are no plans to remove it because there are many scripts still using it, and there's no diff --git a/Misc/NEWS.d/3.10.0a1.rst b/Misc/NEWS.d/3.10.0a1.rst index 572841db7d89d..79d85a40df8bb 100644 --- a/Misc/NEWS.d/3.10.0a1.rst +++ b/Misc/NEWS.d/3.10.0a1.rst @@ -228,8 +228,8 @@ format string in f-string and :meth:`str.format`. .. section: Core and Builtins The implementation of :func:`signal.siginterrupt` now uses -:c:func:`sigaction` (if it is available in the system) instead of the -deprecated :c:func:`siginterrupt`. Patch by Pablo Galindo. +:c:func:`!sigaction` (if it is available in the system) instead of the +deprecated :c:func:`!siginterrupt`. Patch by Pablo Galindo. .. From webhook-mailer at python.org Sun Jul 23 17:13:27 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sun, 23 Jul 2023 21:13:27 -0000 Subject: [Python-checkins] [3.12] gh-106948: Docs: Disable links for C standard library functions, OS utility functions and system calls (GH-107062) (#107154) Message-ID: https://github.com/python/cpython/commit/40a337fbaa820feb3198d55d8b4f389360e0b2b8 commit: 40a337fbaa820feb3198d55d8b4f389360e0b2b8 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-23T23:13:23+02:00 summary: [3.12] gh-106948: Docs: Disable links for C standard library functions, OS utility functions and system calls (GH-107062) (#107154) (cherry picked from commit b447e19e720e6781025432a40eb72b1cc93ac944) Co-authored-by: Erlend E. Aasland Co-authored-by: Serhiy Storchaka files: M Doc/c-api/exceptions.rst M Doc/c-api/sys.rst M Doc/conf.py M Doc/howto/instrumentation.rst M Doc/library/mailbox.rst M Doc/library/os.rst M Doc/library/select.rst M Doc/library/signal.rst M Doc/tools/.nitignore M Doc/whatsnew/2.6.rst M Doc/whatsnew/2.7.rst M Misc/NEWS.d/3.10.0a1.rst diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 3d4ceb360f5b1..4d81b273638b1 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -163,7 +163,7 @@ For convenience, some of these functions will always return a This is a convenience function to raise an exception when a C library function has returned an error and set the C variable :c:data:`errno`. It constructs a tuple object whose first item is the integer :c:data:`errno` value and whose - second item is the corresponding error message (gotten from :c:func:`strerror`), + second item is the corresponding error message (gotten from :c:func:`!strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the :c:data:`errno` value is :c:macro:`EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 3550193c1af82..7e0e982a04ec3 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -106,7 +106,7 @@ Operating System Utilities .. c:function:: PyOS_sighandler_t PyOS_getsig(int i) Return the current signal handler for signal *i*. This is a thin wrapper around - either :c:func:`sigaction` or :c:func:`signal`. Do not call those functions + either :c:func:`!sigaction` or :c:func:`!signal`. Do not call those functions directly! :c:type:`PyOS_sighandler_t` is a typedef alias for :c:expr:`void (\*)(int)`. @@ -114,7 +114,7 @@ Operating System Utilities .. c:function:: PyOS_sighandler_t PyOS_setsig(int i, PyOS_sighandler_t h) Set the signal handler for signal *i* to be *h*; return the old signal handler. - This is a thin wrapper around either :c:func:`sigaction` or :c:func:`signal`. Do + This is a thin wrapper around either :c:func:`!sigaction` or :c:func:`!signal`. Do not call those functions directly! :c:type:`PyOS_sighandler_t` is a typedef alias for :c:expr:`void (\*)(int)`. diff --git a/Doc/conf.py b/Doc/conf.py index 48d43736bfbc2..067aa1d9e2291 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -77,6 +77,24 @@ exclude_patterns.append(venvdir + '/*') nitpick_ignore = [ + # Standard C functions + ('c:func', 'calloc'), + ('c:func', 'dlopen'), + ('c:func', 'exec'), + ('c:func', 'fcntl'), + ('c:func', 'fork'), + ('c:func', 'free'), + ('c:func', 'gmtime'), + ('c:func', 'localtime'), + ('c:func', 'main'), + ('c:func', 'malloc'), + ('c:func', 'printf'), + ('c:func', 'realloc'), + ('c:func', 'snprintf'), + ('c:func', 'sprintf'), + ('c:func', 'stat'), + ('c:func', 'system'), + ('c:func', 'vsnprintf'), # Standard C types ('c:type', 'FILE'), ('c:type', '__int'), diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst index 4ce15c69dac90..875f846aad005 100644 --- a/Doc/howto/instrumentation.rst +++ b/Doc/howto/instrumentation.rst @@ -292,11 +292,11 @@ Available static markers .. object:: function__return(str filename, str funcname, int lineno) - This marker is the converse of :c:func:`function__entry`, and indicates that + This marker is the converse of :c:func:`!function__entry`, and indicates that execution of a Python function has ended (either via ``return``, or via an exception). It is only triggered for pure-Python (bytecode) functions. - The arguments are the same as for :c:func:`function__entry` + The arguments are the same as for :c:func:`!function__entry` .. object:: line(str filename, str funcname, int lineno) @@ -304,7 +304,7 @@ Available static markers the equivalent of line-by-line tracing with a Python profiler. It is not triggered within C functions. - The arguments are the same as for :c:func:`function__entry`. + The arguments are the same as for :c:func:`!function__entry`. .. object:: gc__start(int generation) diff --git a/Doc/library/mailbox.rst b/Doc/library/mailbox.rst index 56908dedea1b4..91df07d914cae 100644 --- a/Doc/library/mailbox.rst +++ b/Doc/library/mailbox.rst @@ -477,7 +477,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. + :c:func:`!flock` and :c:func:`!lockf` system calls. .. seealso:: @@ -588,7 +588,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. For MH mailboxes, locking + :c:func:`!flock` and :c:func:`!lockf` system calls. For MH mailboxes, locking the mailbox means locking the :file:`.mh_sequences` file and, only for the duration of any operations that affect them, locking individual message files. @@ -686,7 +686,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. + :c:func:`!flock` and :c:func:`!lockf` system calls. .. seealso:: @@ -737,7 +737,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. + :c:func:`!flock` and :c:func:`!lockf` system calls. .. seealso:: diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 57405916ed7b3..bbf227aab649e 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -714,14 +714,14 @@ process and user. .. function:: getsid(pid, /) - Call the system call :c:func:`getsid`. See the Unix manual for the semantics. + Call the system call :c:func:`!getsid`. See the Unix manual for the semantics. .. availability:: Unix, not Emscripten, not WASI. .. function:: setsid() - Call the system call :c:func:`setsid`. See the Unix manual for the semantics. + Call the system call :c:func:`!setsid`. See the Unix manual for the semantics. .. availability:: Unix, not Emscripten, not WASI. @@ -739,7 +739,7 @@ process and user. .. function:: strerror(code, /) Return the error message corresponding to the error code in *code*. - On platforms where :c:func:`strerror` returns ``NULL`` when given an unknown + On platforms where :c:func:`!strerror` returns ``NULL`` when given an unknown error number, :exc:`ValueError` is raised. diff --git a/Doc/library/select.rst b/Doc/library/select.rst index b0891b0c8f584..c2941e628d9d7 100644 --- a/Doc/library/select.rst +++ b/Doc/library/select.rst @@ -6,10 +6,10 @@ -------------- -This module provides access to the :c:func:`select` and :c:func:`poll` functions -available in most operating systems, :c:func:`devpoll` available on -Solaris and derivatives, :c:func:`epoll` available on Linux 2.5+ and -:c:func:`kqueue` available on most BSD. +This module provides access to the :c:func:`!select` and :c:func:`!poll` functions +available in most operating systems, :c:func:`!devpoll` available on +Solaris and derivatives, :c:func:`!epoll` available on Linux 2.5+ and +:c:func:`!kqueue` available on most BSD. Note that on Windows, it only works for sockets; on other operating systems, it also works for other file types (in particular, on Unix, it works on pipes). It cannot be used on regular files to determine whether a file has grown since @@ -41,10 +41,10 @@ The module defines the following: polling object; see section :ref:`devpoll-objects` below for the methods supported by devpoll objects. - :c:func:`devpoll` objects are linked to the number of file + :c:func:`!devpoll` objects are linked to the number of file descriptors allowed at the time of instantiation. If your program - reduces this value, :c:func:`devpoll` will fail. If your program - increases this value, :c:func:`devpoll` may return an + reduces this value, :c:func:`!devpoll` will fail. If your program + increases this value, :c:func:`!devpoll` may return an incomplete list of active file descriptors. The new file descriptor is :ref:`non-inheritable `. @@ -62,7 +62,7 @@ The module defines the following: *sizehint* informs epoll about the expected number of events to be registered. It must be positive, or ``-1`` to use the default. It is only - used on older systems where :c:func:`epoll_create1` is not available; + used on older systems where :c:func:`!epoll_create1` is not available; otherwise it has no effect (though its value is still checked). *flags* is deprecated and completely ignored. However, when supplied, its @@ -117,7 +117,7 @@ The module defines the following: .. function:: select(rlist, wlist, xlist[, timeout]) - This is a straightforward interface to the Unix :c:func:`select` system call. + This is a straightforward interface to the Unix :c:func:`!select` system call. The first three arguments are iterables of 'waitable objects': either integers representing file descriptors or objects with a parameterless method named :meth:`~io.IOBase.fileno` returning such an integer: @@ -154,7 +154,7 @@ The module defines the following: .. index:: single: WinSock File objects on Windows are not acceptable, but sockets are. On Windows, - the underlying :c:func:`select` function is provided by the WinSock + the underlying :c:func:`!select` function is provided by the WinSock library, and does not handle file descriptors that don't originate from WinSock. @@ -169,7 +169,7 @@ The module defines the following: The minimum number of bytes which can be written without blocking to a pipe when the pipe has been reported as ready for writing by :func:`~select.select`, - :func:`poll` or another interface in this module. This doesn't apply + :func:`!poll` or another interface in this module. This doesn't apply to other kind of file-like objects such as sockets. This value is guaranteed by POSIX to be at least 512. @@ -184,11 +184,11 @@ The module defines the following: ``/dev/poll`` Polling Objects ----------------------------- -Solaris and derivatives have ``/dev/poll``. While :c:func:`select` is -O(highest file descriptor) and :c:func:`poll` is O(number of file +Solaris and derivatives have ``/dev/poll``. While :c:func:`!select` is +O(highest file descriptor) and :c:func:`!poll` is O(number of file descriptors), ``/dev/poll`` is O(active file descriptors). -``/dev/poll`` behaviour is very close to the standard :c:func:`poll` +``/dev/poll`` behaviour is very close to the standard :c:func:`!poll` object. @@ -222,7 +222,7 @@ object. implement :meth:`!fileno`, so they can also be used as the argument. *eventmask* is an optional bitmask describing the type of events you want to - check for. The constants are the same that with :c:func:`poll` + check for. The constants are the same that with :c:func:`!poll` object. The default value is a combination of the constants :const:`POLLIN`, :const:`POLLPRI`, and :const:`POLLOUT`. @@ -231,7 +231,7 @@ object. Registering a file descriptor that's already registered is not an error, but the result is undefined. The appropriate action is to unregister or modify it first. This is an important difference - compared with :c:func:`poll`. + compared with :c:func:`!poll`. .. method:: devpoll.modify(fd[, eventmask]) @@ -376,13 +376,13 @@ Edge and Level Trigger Polling (epoll) Objects Polling Objects --------------- -The :c:func:`poll` system call, supported on most Unix systems, provides better +The :c:func:`!poll` system call, supported on most Unix systems, provides better scalability for network servers that service many, many clients at the same -time. :c:func:`poll` scales better because the system call only requires listing -the file descriptors of interest, while :c:func:`select` builds a bitmap, turns +time. :c:func:`!poll` scales better because the system call only requires listing +the file descriptors of interest, while :c:func:`!select` builds a bitmap, turns on bits for the fds of interest, and then afterward the whole bitmap has to be -linearly scanned again. :c:func:`select` is O(highest file descriptor), while -:c:func:`poll` is O(number of file descriptors). +linearly scanned again. :c:func:`!select` is O(highest file descriptor), while +:c:func:`!poll` is O(number of file descriptors). .. method:: poll.register(fd[, eventmask]) diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index 523d1ac500136..e53315bee3ea3 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -562,7 +562,7 @@ The :mod:`signal` module defines the following functions: Note that installing a signal handler with :func:`signal` will reset the restart behaviour to interruptible by implicitly calling - :c:func:`siginterrupt` with a true *flag* value for the given signal. + :c:func:`!siginterrupt` with a true *flag* value for the given signal. .. function:: signal(signalnum, handler) diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index f333495017e2a..b6adbcb34663e 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -59,7 +59,6 @@ Doc/glossary.rst Doc/howto/curses.rst Doc/howto/descriptor.rst Doc/howto/enum.rst -Doc/howto/instrumentation.rst Doc/howto/isolating-extensions.rst Doc/howto/logging-cookbook.rst Doc/howto/logging.rst diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index 72a273fdd8d12..2b8fa1546a588 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -2289,7 +2289,7 @@ changes, or look through the Subversion logs for all the details. (Contributed by Raymond Hettinger; :issue:`1861`.) * The :mod:`select` module now has wrapper functions - for the Linux :c:func:`epoll` and BSD :c:func:`kqueue` system calls. + for the Linux :c:func:`!epoll` and BSD :c:func:`!kqueue` system calls. :meth:`modify` method was added to the existing :class:`poll` objects; ``pollobj.modify(fd, eventmask)`` takes a file descriptor or file object and an event mask, modifying the recorded event mask @@ -2328,7 +2328,7 @@ changes, or look through the Subversion logs for all the details. one for reading and one for writing. The writable descriptor will be passed to :func:`set_wakeup_fd`, and the readable descriptor will be added to the list of descriptors monitored by the event loop via - :c:func:`select` or :c:func:`poll`. + :c:func:`!select` or :c:func:`!poll`. On receiving a signal, a byte will be written and the main event loop will be woken up, avoiding the need to poll. @@ -2982,7 +2982,7 @@ Changes to Python's build process and to the C API include: * Python now must be compiled with C89 compilers (after 19 years!). This means that the Python source tree has dropped its - own implementations of :c:func:`memmove` and :c:func:`strerror`, which + own implementations of :c:func:`!memmove` and :c:func:`!strerror`, which are in the C89 standard library. * Python 2.6 can be built with Microsoft Visual Studio 2008 (version diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index a1d522b7ce6da..cf8b00fd30979 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -355,7 +355,7 @@ added as a more powerful replacement for the This means Python now supports three different modules for parsing command-line arguments: :mod:`getopt`, :mod:`optparse`, and :mod:`argparse`. The :mod:`getopt` module closely resembles the C -library's :c:func:`getopt` function, so it remains useful if you're writing a +library's :c:func:`!getopt` function, so it remains useful if you're writing a Python prototype that will eventually be rewritten in C. :mod:`optparse` becomes redundant, but there are no plans to remove it because there are many scripts still using it, and there's no diff --git a/Misc/NEWS.d/3.10.0a1.rst b/Misc/NEWS.d/3.10.0a1.rst index f7eb8036bc5cf..c1437172a38be 100644 --- a/Misc/NEWS.d/3.10.0a1.rst +++ b/Misc/NEWS.d/3.10.0a1.rst @@ -228,8 +228,8 @@ format string in f-string and :meth:`str.format`. .. section: Core and Builtins The implementation of :func:`signal.siginterrupt` now uses -:c:func:`sigaction` (if it is available in the system) instead of the -deprecated :c:func:`siginterrupt`. Patch by Pablo Galindo. +:c:func:`!sigaction` (if it is available in the system) instead of the +deprecated :c:func:`!siginterrupt`. Patch by Pablo Galindo. .. From webhook-mailer at python.org Sun Jul 23 17:49:51 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Sun, 23 Jul 2023 21:49:51 -0000 Subject: [Python-checkins] [3.11] gh-106948: Docs: Disable links for C standard library functions, OS utility functions and system calls (#107062) (#107157) Message-ID: https://github.com/python/cpython/commit/b3e600a632d018ce868bec0ef26116b8637fe8e0 commit: b3e600a632d018ce868bec0ef26116b8637fe8e0 branch: 3.11 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-23T23:49:48+02:00 summary: [3.11] gh-106948: Docs: Disable links for C standard library functions, OS utility functions and system calls (#107062) (#107157) (cherry picked from commit b447e19e720e6781025432a40eb72b1cc93ac944) Co-authored-by: Serhiy Storchaka files: M Doc/c-api/exceptions.rst M Doc/c-api/sys.rst M Doc/conf.py M Doc/howto/instrumentation.rst M Doc/library/mailbox.rst M Doc/library/os.rst M Doc/library/select.rst M Doc/library/signal.rst M Doc/whatsnew/2.6.rst M Doc/whatsnew/2.7.rst M Misc/NEWS.d/3.10.0a1.rst diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 5dad596f6197f..6ca53bd29f9e8 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -152,7 +152,7 @@ For convenience, some of these functions will always return a This is a convenience function to raise an exception when a C library function has returned an error and set the C variable :c:data:`errno`. It constructs a tuple object whose first item is the integer :c:data:`errno` value and whose - second item is the corresponding error message (gotten from :c:func:`strerror`), + second item is the corresponding error message (gotten from :c:func:`!strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the :c:data:`errno` value is :c:macro:`EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 305a315549d15..57a36c038b592 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -104,7 +104,7 @@ Operating System Utilities .. c:function:: PyOS_sighandler_t PyOS_getsig(int i) Return the current signal handler for signal *i*. This is a thin wrapper around - either :c:func:`sigaction` or :c:func:`signal`. Do not call those functions + either :c:func:`!sigaction` or :c:func:`!signal`. Do not call those functions directly! :c:type:`PyOS_sighandler_t` is a typedef alias for :c:expr:`void (\*)(int)`. @@ -112,7 +112,7 @@ Operating System Utilities .. c:function:: PyOS_sighandler_t PyOS_setsig(int i, PyOS_sighandler_t h) Set the signal handler for signal *i* to be *h*; return the old signal handler. - This is a thin wrapper around either :c:func:`sigaction` or :c:func:`signal`. Do + This is a thin wrapper around either :c:func:`!sigaction` or :c:func:`!signal`. Do not call those functions directly! :c:type:`PyOS_sighandler_t` is a typedef alias for :c:expr:`void (\*)(int)`. diff --git a/Doc/conf.py b/Doc/conf.py index c082717efda72..28c35f2b0a234 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -72,6 +72,24 @@ exclude_patterns.append(venvdir + '/*') nitpick_ignore = [ + # Standard C functions + ('c:func', 'calloc'), + ('c:func', 'dlopen'), + ('c:func', 'exec'), + ('c:func', 'fcntl'), + ('c:func', 'fork'), + ('c:func', 'free'), + ('c:func', 'gmtime'), + ('c:func', 'localtime'), + ('c:func', 'main'), + ('c:func', 'malloc'), + ('c:func', 'printf'), + ('c:func', 'realloc'), + ('c:func', 'snprintf'), + ('c:func', 'sprintf'), + ('c:func', 'stat'), + ('c:func', 'system'), + ('c:func', 'vsnprintf'), # Standard C types ('c:type', 'FILE'), ('c:type', '__int'), diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst index 4ce15c69dac90..875f846aad005 100644 --- a/Doc/howto/instrumentation.rst +++ b/Doc/howto/instrumentation.rst @@ -292,11 +292,11 @@ Available static markers .. object:: function__return(str filename, str funcname, int lineno) - This marker is the converse of :c:func:`function__entry`, and indicates that + This marker is the converse of :c:func:`!function__entry`, and indicates that execution of a Python function has ended (either via ``return``, or via an exception). It is only triggered for pure-Python (bytecode) functions. - The arguments are the same as for :c:func:`function__entry` + The arguments are the same as for :c:func:`!function__entry` .. object:: line(str filename, str funcname, int lineno) @@ -304,7 +304,7 @@ Available static markers the equivalent of line-by-line tracing with a Python profiler. It is not triggered within C functions. - The arguments are the same as for :c:func:`function__entry`. + The arguments are the same as for :c:func:`!function__entry`. .. object:: gc__start(int generation) diff --git a/Doc/library/mailbox.rst b/Doc/library/mailbox.rst index 56908dedea1b4..91df07d914cae 100644 --- a/Doc/library/mailbox.rst +++ b/Doc/library/mailbox.rst @@ -477,7 +477,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. + :c:func:`!flock` and :c:func:`!lockf` system calls. .. seealso:: @@ -588,7 +588,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. For MH mailboxes, locking + :c:func:`!flock` and :c:func:`!lockf` system calls. For MH mailboxes, locking the mailbox means locking the :file:`.mh_sequences` file and, only for the duration of any operations that affect them, locking individual message files. @@ -686,7 +686,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. + :c:func:`!flock` and :c:func:`!lockf` system calls. .. seealso:: @@ -737,7 +737,7 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF. unlock() Three locking mechanisms are used---dot locking and, if available, the - :c:func:`flock` and :c:func:`lockf` system calls. + :c:func:`!flock` and :c:func:`!lockf` system calls. .. seealso:: diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 9d6d02652db90..dcb4e7d3ab259 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -665,14 +665,14 @@ process and user. .. function:: getsid(pid, /) - Call the system call :c:func:`getsid`. See the Unix manual for the semantics. + Call the system call :c:func:`!getsid`. See the Unix manual for the semantics. .. availability:: Unix, not Emscripten, not WASI. .. function:: setsid() - Call the system call :c:func:`setsid`. See the Unix manual for the semantics. + Call the system call :c:func:`!setsid`. See the Unix manual for the semantics. .. availability:: Unix, not Emscripten, not WASI. @@ -690,7 +690,7 @@ process and user. .. function:: strerror(code, /) Return the error message corresponding to the error code in *code*. - On platforms where :c:func:`strerror` returns ``NULL`` when given an unknown + On platforms where :c:func:`!strerror` returns ``NULL`` when given an unknown error number, :exc:`ValueError` is raised. diff --git a/Doc/library/select.rst b/Doc/library/select.rst index b0891b0c8f584..c2941e628d9d7 100644 --- a/Doc/library/select.rst +++ b/Doc/library/select.rst @@ -6,10 +6,10 @@ -------------- -This module provides access to the :c:func:`select` and :c:func:`poll` functions -available in most operating systems, :c:func:`devpoll` available on -Solaris and derivatives, :c:func:`epoll` available on Linux 2.5+ and -:c:func:`kqueue` available on most BSD. +This module provides access to the :c:func:`!select` and :c:func:`!poll` functions +available in most operating systems, :c:func:`!devpoll` available on +Solaris and derivatives, :c:func:`!epoll` available on Linux 2.5+ and +:c:func:`!kqueue` available on most BSD. Note that on Windows, it only works for sockets; on other operating systems, it also works for other file types (in particular, on Unix, it works on pipes). It cannot be used on regular files to determine whether a file has grown since @@ -41,10 +41,10 @@ The module defines the following: polling object; see section :ref:`devpoll-objects` below for the methods supported by devpoll objects. - :c:func:`devpoll` objects are linked to the number of file + :c:func:`!devpoll` objects are linked to the number of file descriptors allowed at the time of instantiation. If your program - reduces this value, :c:func:`devpoll` will fail. If your program - increases this value, :c:func:`devpoll` may return an + reduces this value, :c:func:`!devpoll` will fail. If your program + increases this value, :c:func:`!devpoll` may return an incomplete list of active file descriptors. The new file descriptor is :ref:`non-inheritable `. @@ -62,7 +62,7 @@ The module defines the following: *sizehint* informs epoll about the expected number of events to be registered. It must be positive, or ``-1`` to use the default. It is only - used on older systems where :c:func:`epoll_create1` is not available; + used on older systems where :c:func:`!epoll_create1` is not available; otherwise it has no effect (though its value is still checked). *flags* is deprecated and completely ignored. However, when supplied, its @@ -117,7 +117,7 @@ The module defines the following: .. function:: select(rlist, wlist, xlist[, timeout]) - This is a straightforward interface to the Unix :c:func:`select` system call. + This is a straightforward interface to the Unix :c:func:`!select` system call. The first three arguments are iterables of 'waitable objects': either integers representing file descriptors or objects with a parameterless method named :meth:`~io.IOBase.fileno` returning such an integer: @@ -154,7 +154,7 @@ The module defines the following: .. index:: single: WinSock File objects on Windows are not acceptable, but sockets are. On Windows, - the underlying :c:func:`select` function is provided by the WinSock + the underlying :c:func:`!select` function is provided by the WinSock library, and does not handle file descriptors that don't originate from WinSock. @@ -169,7 +169,7 @@ The module defines the following: The minimum number of bytes which can be written without blocking to a pipe when the pipe has been reported as ready for writing by :func:`~select.select`, - :func:`poll` or another interface in this module. This doesn't apply + :func:`!poll` or another interface in this module. This doesn't apply to other kind of file-like objects such as sockets. This value is guaranteed by POSIX to be at least 512. @@ -184,11 +184,11 @@ The module defines the following: ``/dev/poll`` Polling Objects ----------------------------- -Solaris and derivatives have ``/dev/poll``. While :c:func:`select` is -O(highest file descriptor) and :c:func:`poll` is O(number of file +Solaris and derivatives have ``/dev/poll``. While :c:func:`!select` is +O(highest file descriptor) and :c:func:`!poll` is O(number of file descriptors), ``/dev/poll`` is O(active file descriptors). -``/dev/poll`` behaviour is very close to the standard :c:func:`poll` +``/dev/poll`` behaviour is very close to the standard :c:func:`!poll` object. @@ -222,7 +222,7 @@ object. implement :meth:`!fileno`, so they can also be used as the argument. *eventmask* is an optional bitmask describing the type of events you want to - check for. The constants are the same that with :c:func:`poll` + check for. The constants are the same that with :c:func:`!poll` object. The default value is a combination of the constants :const:`POLLIN`, :const:`POLLPRI`, and :const:`POLLOUT`. @@ -231,7 +231,7 @@ object. Registering a file descriptor that's already registered is not an error, but the result is undefined. The appropriate action is to unregister or modify it first. This is an important difference - compared with :c:func:`poll`. + compared with :c:func:`!poll`. .. method:: devpoll.modify(fd[, eventmask]) @@ -376,13 +376,13 @@ Edge and Level Trigger Polling (epoll) Objects Polling Objects --------------- -The :c:func:`poll` system call, supported on most Unix systems, provides better +The :c:func:`!poll` system call, supported on most Unix systems, provides better scalability for network servers that service many, many clients at the same -time. :c:func:`poll` scales better because the system call only requires listing -the file descriptors of interest, while :c:func:`select` builds a bitmap, turns +time. :c:func:`!poll` scales better because the system call only requires listing +the file descriptors of interest, while :c:func:`!select` builds a bitmap, turns on bits for the fds of interest, and then afterward the whole bitmap has to be -linearly scanned again. :c:func:`select` is O(highest file descriptor), while -:c:func:`poll` is O(number of file descriptors). +linearly scanned again. :c:func:`!select` is O(highest file descriptor), while +:c:func:`!poll` is O(number of file descriptors). .. method:: poll.register(fd[, eventmask]) diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index 523d1ac500136..e53315bee3ea3 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -562,7 +562,7 @@ The :mod:`signal` module defines the following functions: Note that installing a signal handler with :func:`signal` will reset the restart behaviour to interruptible by implicitly calling - :c:func:`siginterrupt` with a true *flag* value for the given signal. + :c:func:`!siginterrupt` with a true *flag* value for the given signal. .. function:: signal(signalnum, handler) diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index 72a273fdd8d12..2b8fa1546a588 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -2289,7 +2289,7 @@ changes, or look through the Subversion logs for all the details. (Contributed by Raymond Hettinger; :issue:`1861`.) * The :mod:`select` module now has wrapper functions - for the Linux :c:func:`epoll` and BSD :c:func:`kqueue` system calls. + for the Linux :c:func:`!epoll` and BSD :c:func:`!kqueue` system calls. :meth:`modify` method was added to the existing :class:`poll` objects; ``pollobj.modify(fd, eventmask)`` takes a file descriptor or file object and an event mask, modifying the recorded event mask @@ -2328,7 +2328,7 @@ changes, or look through the Subversion logs for all the details. one for reading and one for writing. The writable descriptor will be passed to :func:`set_wakeup_fd`, and the readable descriptor will be added to the list of descriptors monitored by the event loop via - :c:func:`select` or :c:func:`poll`. + :c:func:`!select` or :c:func:`!poll`. On receiving a signal, a byte will be written and the main event loop will be woken up, avoiding the need to poll. @@ -2982,7 +2982,7 @@ Changes to Python's build process and to the C API include: * Python now must be compiled with C89 compilers (after 19 years!). This means that the Python source tree has dropped its - own implementations of :c:func:`memmove` and :c:func:`strerror`, which + own implementations of :c:func:`!memmove` and :c:func:`!strerror`, which are in the C89 standard library. * Python 2.6 can be built with Microsoft Visual Studio 2008 (version diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst index ba6aebafd59d9..09798ca82d261 100644 --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -355,7 +355,7 @@ added as a more powerful replacement for the This means Python now supports three different modules for parsing command-line arguments: :mod:`getopt`, :mod:`optparse`, and :mod:`argparse`. The :mod:`getopt` module closely resembles the C -library's :c:func:`getopt` function, so it remains useful if you're writing a +library's :c:func:`!getopt` function, so it remains useful if you're writing a Python prototype that will eventually be rewritten in C. :mod:`optparse` becomes redundant, but there are no plans to remove it because there are many scripts still using it, and there's no diff --git a/Misc/NEWS.d/3.10.0a1.rst b/Misc/NEWS.d/3.10.0a1.rst index 58e57a951ae60..2ef511c6e227e 100644 --- a/Misc/NEWS.d/3.10.0a1.rst +++ b/Misc/NEWS.d/3.10.0a1.rst @@ -228,8 +228,8 @@ format string in f-string and :meth:`str.format`. .. section: Core and Builtins The implementation of :func:`signal.siginterrupt` now uses -:c:func:`sigaction` (if it is available in the system) instead of the -deprecated :c:func:`siginterrupt`. Patch by Pablo Galindo. +:c:func:`!sigaction` (if it is available in the system) instead of the +deprecated :c:func:`!siginterrupt`. Patch by Pablo Galindo. .. From webhook-mailer at python.org Sun Jul 23 17:50:04 2023 From: webhook-mailer at python.org (hauntsaninja) Date: Sun, 23 Jul 2023 21:50:04 -0000 Subject: [Python-checkins] gh-105291: Add link to migration guide for distutils (#107130) Message-ID: https://github.com/python/cpython/commit/7ca2d8e053e5fdbac4a9cec74701a1676c17a167 commit: 7ca2d8e053e5fdbac4a9cec74701a1676c17a167 branch: main author: cLupus committer: hauntsaninja <12621235+hauntsaninja at users.noreply.github.com> date: 2023-07-23T14:50:01-07:00 summary: gh-105291: Add link to migration guide for distutils (#107130) Co-authored-by: Shantanu <12621235+hauntsaninja at users.noreply.github.com> files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index f043ec764ac55..68e5b0691f00e 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -83,7 +83,9 @@ Important deprecations, removals or restrictions: * :pep:`623`: Remove wstr from Unicode -* :pep:`632`: Remove the ``distutils`` package +* :pep:`632`: Remove the ``distutils`` package. See + `the migration guide `_ + for advice on its replacement. Improved Error Messages ======================= From webhook-mailer at python.org Sun Jul 23 17:53:28 2023 From: webhook-mailer at python.org (hauntsaninja) Date: Sun, 23 Jul 2023 21:53:28 -0000 Subject: [Python-checkins] Docs: fix typo in os.pwrite docstring (#107087) Message-ID: https://github.com/python/cpython/commit/8de8a817ee81e352603380ddd1ae91accc99d4a9 commit: 8de8a817ee81e352603380ddd1ae91accc99d4a9 branch: main author: J?r?me Carretero committer: hauntsaninja <12621235+hauntsaninja at users.noreply.github.com> date: 2023-07-23T14:53:25-07:00 summary: Docs: fix typo in os.pwrite docstring (#107087) files: M Modules/clinic/posixmodule.c.h M Modules/posixmodule.c diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h index a8f6ce026a331..81e3162e679d1 100644 --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -7366,7 +7366,7 @@ PyDoc_STRVAR(os_pwrite__doc__, "Write bytes to a file descriptor starting at a particular offset.\n" "\n" "Write buffer to fd, starting at offset bytes from the beginning of\n" -"the file. Returns the number of bytes writte. Does not change the\n" +"the file. Returns the number of bytes written. Does not change the\n" "current file offset."); #define OS_PWRITE_METHODDEF \ @@ -11981,4 +11981,4 @@ os_waitstatus_to_exitcode(PyObject *module, PyObject *const *args, Py_ssize_t na #ifndef OS_WAITSTATUS_TO_EXITCODE_METHODDEF #define OS_WAITSTATUS_TO_EXITCODE_METHODDEF #endif /* !defined(OS_WAITSTATUS_TO_EXITCODE_METHODDEF) */ -/*[clinic end generated code: output=a7e8c3df2db09717 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=a85a386b212b0631 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 23bf978d0cdbf..7e04d12a9e4e7 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -11283,13 +11283,13 @@ os.pwrite -> Py_ssize_t Write bytes to a file descriptor starting at a particular offset. Write buffer to fd, starting at offset bytes from the beginning of -the file. Returns the number of bytes writte. Does not change the +the file. Returns the number of bytes written. Does not change the current file offset. [clinic start generated code]*/ static Py_ssize_t os_pwrite_impl(PyObject *module, int fd, Py_buffer *buffer, Py_off_t offset) -/*[clinic end generated code: output=c74da630758ee925 input=19903f1b3dd26377]*/ +/*[clinic end generated code: output=c74da630758ee925 input=614acbc7e5a0339a]*/ { Py_ssize_t size; int async_err = 0; From webhook-mailer at python.org Sun Jul 23 18:00:55 2023 From: webhook-mailer at python.org (hauntsaninja) Date: Sun, 23 Jul 2023 22:00:55 -0000 Subject: [Python-checkins] [3.12] gh-105291: Add link to migration guide for distutils (GH-107130) (#107160) Message-ID: https://github.com/python/cpython/commit/84c5676e91468642f64f137de85301a5b766b758 commit: 84c5676e91468642f64f137de85301a5b766b758 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hauntsaninja <12621235+hauntsaninja at users.noreply.github.com> date: 2023-07-23T15:00:51-07:00 summary: [3.12] gh-105291: Add link to migration guide for distutils (GH-107130) (#107160) Co-authored-by: cLupus Co-authored-by: Shantanu <12621235+hauntsaninja at users.noreply.github.com> files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index a7db645657db1..71b9358323e3d 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -84,7 +84,9 @@ Important deprecations, removals or restrictions: * :pep:`623`: Remove wstr from Unicode -* :pep:`632`: Remove the ``distutils`` package +* :pep:`632`: Remove the ``distutils`` package. See + `the migration guide `_ + for advice on its replacement. Improved Error Messages ======================= From webhook-mailer at python.org Sun Jul 23 18:25:52 2023 From: webhook-mailer at python.org (vstinner) Date: Sun, 23 Jul 2023 22:25:52 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyObject C API (#107159) Message-ID: https://github.com/python/cpython/commit/1e501122876f7e6abf979fdb756d7edd0d924ba1 commit: 1e501122876f7e6abf979fdb756d7edd0d924ba1 branch: main author: Victor Stinner committer: vstinner date: 2023-07-23T22:25:48Z summary: gh-106320: Remove private _PyObject C API (#107159) Move private _PyObject and private _PyType functions to the internal C API (pycore_object.h): * _PyObject_GetMethod() * _PyObject_IsAbstract() * _PyObject_NextNotImplemented() * _PyType_CalculateMetaclass() * _PyType_GetDocFromInternalDoc() * _PyType_GetTextSignatureFromInternalDoc() No longer export these functions. files: M Include/cpython/object.h M Include/internal/pycore_object.h diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 90e45f65e298e..fd45fa57e6282 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -275,14 +275,6 @@ PyAPI_FUNC(const char *) _PyType_Name(PyTypeObject *); PyAPI_FUNC(PyObject *) _PyType_Lookup(PyTypeObject *, PyObject *); PyAPI_FUNC(PyObject *) _PyType_LookupId(PyTypeObject *, _Py_Identifier *); PyAPI_FUNC(PyObject *) _PyObject_LookupSpecialId(PyObject *, _Py_Identifier *); -#ifndef Py_BUILD_CORE -// Backward compatibility for 3rd-party extensions -// that may be using the old name. -#define _PyObject_LookupSpecial _PyObject_LookupSpecialId -#endif -PyAPI_FUNC(PyTypeObject *) _PyType_CalculateMetaclass(PyTypeObject *, PyObject *); -PyAPI_FUNC(PyObject *) _PyType_GetDocFromInternalDoc(const char *, const char *); -PyAPI_FUNC(PyObject *) _PyType_GetTextSignatureFromInternalDoc(const char *, const char *); PyAPI_FUNC(PyObject *) PyType_GetModuleByDef(PyTypeObject *, PyModuleDef *); PyAPI_FUNC(PyObject *) PyType_GetDict(PyTypeObject *); @@ -290,14 +282,10 @@ PyAPI_FUNC(int) PyObject_Print(PyObject *, FILE *, int); PyAPI_FUNC(void) _Py_BreakPoint(void); PyAPI_FUNC(void) _PyObject_Dump(PyObject *); -PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *); PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, _Py_Identifier *); PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, _Py_Identifier *, PyObject *); -PyAPI_FUNC(int) _PyObject_GetMethod(PyObject *obj, PyObject *name, PyObject **method); - PyAPI_FUNC(PyObject **) _PyObject_GetDictPtr(PyObject *); -PyAPI_FUNC(PyObject *) _PyObject_NextNotImplemented(PyObject *); PyAPI_FUNC(void) PyObject_CallFinalizer(PyObject *); PyAPI_FUNC(int) PyObject_CallFinalizerFromDealloc(PyObject *); diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h index 661820c894dc0..96aea477d064a 100644 --- a/Include/internal/pycore_object.h +++ b/Include/internal/pycore_object.h @@ -376,7 +376,11 @@ static inline int _PyType_SUPPORTS_WEAKREFS(PyTypeObject *type) { } extern PyObject* _PyType_AllocNoTrack(PyTypeObject *type, Py_ssize_t nitems); -PyObject *_PyType_NewManagedObject(PyTypeObject *type); +extern PyObject *_PyType_NewManagedObject(PyTypeObject *type); + +extern PyTypeObject* _PyType_CalculateMetaclass(PyTypeObject *, PyObject *); +extern PyObject* _PyType_GetDocFromInternalDoc(const char *, const char *); +extern PyObject* _PyType_GetTextSignatureFromInternalDoc(const char *, const char *); extern int _PyObject_InitializeDict(PyObject *obj); int _PyObject_InitInlineValues(PyObject *obj, PyTypeObject *tp); @@ -432,6 +436,11 @@ extern int _PyObject_IsInstanceDictEmpty(PyObject *); PyAPI_FUNC(PyObject *) _PyObject_LookupSpecial(PyObject *, PyObject *); +extern int _PyObject_IsAbstract(PyObject *); + +extern int _PyObject_GetMethod(PyObject *obj, PyObject *name, PyObject **method); +extern PyObject* _PyObject_NextNotImplemented(PyObject *); + /* C function call trampolines to mitigate bad function pointer casts. * * Typical native ABIs ignore additional arguments or fill in missing From webhook-mailer at python.org Mon Jul 24 01:56:01 2023 From: webhook-mailer at python.org (hugovk) Date: Mon, 24 Jul 2023 05:56:01 -0000 Subject: [Python-checkins] gh-107017: Change Chapter Strings to Texts in the Introduction chapter. (#107104) Message-ID: https://github.com/python/cpython/commit/2cf99026d6320f38937257da1ab014fc873a11a6 commit: 2cf99026d6320f38937257da1ab014fc873a11a6 branch: main author: TommyUnreal <45427816+TommyUnreal at users.noreply.github.com> committer: hugovk date: 2023-07-24T05:55:57Z summary: gh-107017: Change Chapter Strings to Texts in the Introduction chapter. (#107104) Co-authored-by: Hugo van Kemenade files: M Doc/tutorial/introduction.rst diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index 4b886f604aca2..0fc75c7d7532e 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -138,16 +138,25 @@ and uses the ``j`` or ``J`` suffix to indicate the imaginary part .. _tut-strings: -Strings -------- +Text +---- -Besides numbers, Python can also manipulate strings, which can be expressed -in several ways. They can be enclosed in single quotes (``'...'``) or -double quotes (``"..."``) with the same result [#]_. ``\`` can be used -to escape quotes:: +Python can manipulate text (represented by type :class:`str`, so-called +"strings") as well as numbers. This includes characters "``!``", words +"``rabbit``", names "``Paris``", sentences "``Got your back.``", etc. +"``Yay! :)``". They can be enclosed in single quotes (``'...'``) or double +quotes (``"..."``) with the same result [#]_. >>> 'spam eggs' # single quotes 'spam eggs' + >>> "Paris rabbit got your back :)! Yay!" # double quotes + 'Paris rabbit got your back :)! Yay!' + >>> '1975' # digits and numerals enclosed in quotes are also strings + '1975' + +To quote a quote, we need to "escape" it, by preceding it with ``\``. +Alternatively, we can use the other type of quotation marks:: + >>> 'doesn\'t' # use \' to escape the single quote... "doesn't" >>> "doesn't" # ...or use double quotes instead @@ -159,23 +168,14 @@ to escape quotes:: >>> '"Isn\'t," they said.' '"Isn\'t," they said.' -In the interactive interpreter, the output string is enclosed in quotes and -special characters are escaped with backslashes. While this might sometimes -look different from the input (the enclosing quotes could change), the two -strings are equivalent. The string is enclosed in double quotes if -the string contains a single quote and no double quotes, otherwise it is -enclosed in single quotes. The :func:`print` function produces a more -readable output, by omitting the enclosing quotes and by printing escaped -and special characters:: +In the Python shell, the string definition and output string can look +different. The :func:`print` function produces a more readable output, by +omitting the enclosing quotes and by printing escaped and special characters:: - >>> '"Isn\'t," they said.' - '"Isn\'t," they said.' - >>> print('"Isn\'t," they said.') - "Isn't," they said. >>> s = 'First line.\nSecond line.' # \n means newline - >>> s # without print(), \n is included in the output + >>> s # without print(), special characters are included in the string 'First line.\nSecond line.' - >>> print(s) # with print(), \n produces a new line + >>> print(s) # with print(), special characters are interpreted, so \n produces new line First line. Second line. From webhook-mailer at python.org Mon Jul 24 07:54:43 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 24 Jul 2023 11:54:43 -0000 Subject: [Python-checkins] Docs: Remove duplicate word in Argument Clinic howto heading (#107169) Message-ID: https://github.com/python/cpython/commit/ebe44a5155e9abc70c4b8914ad26b27c2b84f72b commit: ebe44a5155e9abc70c4b8914ad26b27c2b84f72b branch: main author: Hakan Celik committer: erlend-aasland date: 2023-07-24T13:54:39+02:00 summary: Docs: Remove duplicate word in Argument Clinic howto heading (#107169) files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index f6bf1d2234f88..933fecab9ddd5 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -582,8 +582,8 @@ How-to guides ============= -How to to rename C functions and variables generated by Argument Clinic ------------------------------------------------------------------------ +How to rename C functions and variables generated by Argument Clinic +-------------------------------------------------------------------- Argument Clinic automatically names the functions it generates for you. Occasionally this may cause a problem, if the generated name collides with From webhook-mailer at python.org Mon Jul 24 08:03:05 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 24 Jul 2023 12:03:05 -0000 Subject: [Python-checkins] [3.12] Docs: Remove duplicate word in Argument Clinic howto heading (GH-107169) (#107171) Message-ID: https://github.com/python/cpython/commit/03271069afc494061c92efa8b8f792e3f2c0b6e4 commit: 03271069afc494061c92efa8b8f792e3f2c0b6e4 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-24T12:03:01Z summary: [3.12] Docs: Remove duplicate word in Argument Clinic howto heading (GH-107169) (#107171) (cherry picked from commit ebe44a5155e9abc70c4b8914ad26b27c2b84f72b) Co-authored-by: Hakan Celik files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index f6bf1d2234f88..933fecab9ddd5 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -582,8 +582,8 @@ How-to guides ============= -How to to rename C functions and variables generated by Argument Clinic ------------------------------------------------------------------------ +How to rename C functions and variables generated by Argument Clinic +-------------------------------------------------------------------- Argument Clinic automatically names the functions it generates for you. Occasionally this may cause a problem, if the generated name collides with From webhook-mailer at python.org Mon Jul 24 08:06:10 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 24 Jul 2023 12:06:10 -0000 Subject: [Python-checkins] [3.11] Docs: Remove duplicate word in Argument Clinic howto heading (GH-107169) (#107172) Message-ID: https://github.com/python/cpython/commit/798d83bec89cb2e4565f7e16654fc3908a79a746 commit: 798d83bec89cb2e4565f7e16654fc3908a79a746 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-24T12:06:06Z summary: [3.11] Docs: Remove duplicate word in Argument Clinic howto heading (GH-107169) (#107172) (cherry picked from commit ebe44a5155e9abc70c4b8914ad26b27c2b84f72b) Co-authored-by: Hakan Celik files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 50703d9b4723d..4a83f6dde3a51 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -574,8 +574,8 @@ How-to guides ============= -How to to rename C functions and variables generated by Argument Clinic ------------------------------------------------------------------------ +How to rename C functions and variables generated by Argument Clinic +-------------------------------------------------------------------- Argument Clinic automatically names the functions it generates for you. Occasionally this may cause a problem, if the generated name collides with From webhook-mailer at python.org Mon Jul 24 08:54:42 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 12:54:42 -0000 Subject: [Python-checkins] Fix PyVectorcall_Function doc versionadded (#107140) Message-ID: https://github.com/python/cpython/commit/0a9b339363a59be1249189c767ed6f46fd71e1c7 commit: 0a9b339363a59be1249189c767ed6f46fd71e1c7 branch: main author: da-woods committer: vstinner date: 2023-07-24T14:54:38+02:00 summary: Fix PyVectorcall_Function doc versionadded (#107140) The documentation implies that PyVectorcall_Function() was available in Python 3.8. This is half-true - it was available under a different name. I think it's clearer to set the "version added" to 3.9. files: M Doc/c-api/call.rst diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index 8be4b5ab80c2d..aed4ae44c76ee 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -152,7 +152,7 @@ Vectorcall Support API This is mostly useful to check whether or not *op* supports vectorcall, which can be done by checking ``PyVectorcall_Function(op) != NULL``. - .. versionadded:: 3.8 + .. versionadded:: 3.9 .. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) From webhook-mailer at python.org Mon Jul 24 09:06:21 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 13:06:21 -0000 Subject: [Python-checkins] [3.12] Fix PyVectorcall_Function doc versionadded (GH-107140) (#107173) Message-ID: https://github.com/python/cpython/commit/4fd576478c99b999feac68b71f317fd9d320662f commit: 4fd576478c99b999feac68b71f317fd9d320662f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-24T13:06:16Z summary: [3.12] Fix PyVectorcall_Function doc versionadded (GH-107140) (#107173) Fix PyVectorcall_Function doc versionadded (GH-107140) The documentation implies that PyVectorcall_Function() was available in Python 3.8. This is half-true - it was available under a different name. I think it's clearer to set the "version added" to 3.9. (cherry picked from commit 0a9b339363a59be1249189c767ed6f46fd71e1c7) Co-authored-by: da-woods files: M Doc/c-api/call.rst diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index 143fce3cdf477..f4e401442c3d2 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -165,7 +165,7 @@ Vectorcall Support API This is mostly useful to check whether or not *op* supports vectorcall, which can be done by checking ``PyVectorcall_Function(op) != NULL``. - .. versionadded:: 3.8 + .. versionadded:: 3.9 .. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) From webhook-mailer at python.org Mon Jul 24 09:06:50 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 13:06:50 -0000 Subject: [Python-checkins] [3.11] Fix PyVectorcall_Function doc versionadded (GH-107140) (#107174) Message-ID: https://github.com/python/cpython/commit/579868e8f1f7995570b2c101b8805e015341f62c commit: 579868e8f1f7995570b2c101b8805e015341f62c branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-24T13:06:46Z summary: [3.11] Fix PyVectorcall_Function doc versionadded (GH-107140) (#107174) Fix PyVectorcall_Function doc versionadded (GH-107140) The documentation implies that PyVectorcall_Function() was available in Python 3.8. This is half-true - it was available under a different name. I think it's clearer to set the "version added" to 3.9. (cherry picked from commit 0a9b339363a59be1249189c767ed6f46fd71e1c7) Co-authored-by: da-woods files: M Doc/c-api/call.rst diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index a574965bb1260..838af604dca02 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -156,7 +156,7 @@ Vectorcall Support API This is mostly useful to check whether or not *op* supports vectorcall, which can be done by checking ``PyVectorcall_Function(op) != NULL``. - .. versionadded:: 3.8 + .. versionadded:: 3.9 .. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) From webhook-mailer at python.org Mon Jul 24 10:02:11 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 14:02:11 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyDict C API (#107145) Message-ID: https://github.com/python/cpython/commit/fd66baf34a73871c651326a41886240bf6f986d7 commit: fd66baf34a73871c651326a41886240bf6f986d7 branch: main author: Victor Stinner committer: vstinner date: 2023-07-24T14:02:03Z summary: gh-106320: Remove private _PyDict C API (#107145) Move private _PyDict functions to the internal C API (pycore_dict.h): * _PyDict_Contains_KnownHash() * _PyDict_DebugMallocStats() * _PyDict_DelItemIf() * _PyDict_GetItemWithError() * _PyDict_HasOnlyStringKeys() * _PyDict_MaybeUntrack() * _PyDict_MergeEx() No longer export these functions. files: M Include/cpython/dictobject.h M Include/internal/pycore_dict.h M Modules/_weakref.c M Modules/gcmodule.c M Objects/setobject.c M Python/_warnings.c M Python/bltinmodule.c M Python/getargs.c M Python/sysmodule.c diff --git a/Include/cpython/dictobject.h b/Include/cpython/dictobject.h index ddada922020aa..2a42794fdf0c8 100644 --- a/Include/cpython/dictobject.h +++ b/Include/cpython/dictobject.h @@ -34,7 +34,6 @@ typedef struct { PyAPI_FUNC(PyObject *) _PyDict_GetItem_KnownHash(PyObject *mp, PyObject *key, Py_hash_t hash); -PyAPI_FUNC(PyObject *) _PyDict_GetItemWithError(PyObject *dp, PyObject *key); PyAPI_FUNC(PyObject *) _PyDict_GetItemIdWithError(PyObject *dp, _Py_Identifier *key); PyAPI_FUNC(PyObject *) _PyDict_GetItemStringWithError(PyObject *, const char *); @@ -44,8 +43,7 @@ PyAPI_FUNC(int) _PyDict_SetItem_KnownHash(PyObject *mp, PyObject *key, PyObject *item, Py_hash_t hash); PyAPI_FUNC(int) _PyDict_DelItem_KnownHash(PyObject *mp, PyObject *key, Py_hash_t hash); -PyAPI_FUNC(int) _PyDict_DelItemIf(PyObject *mp, PyObject *key, - int (*predicate)(PyObject *value)); + PyAPI_FUNC(int) _PyDict_Next( PyObject *mp, Py_ssize_t *pos, PyObject **key, PyObject **value, Py_hash_t *hash); @@ -58,25 +56,16 @@ static inline Py_ssize_t PyDict_GET_SIZE(PyObject *op) { } #define PyDict_GET_SIZE(op) PyDict_GET_SIZE(_PyObject_CAST(op)) -PyAPI_FUNC(int) _PyDict_Contains_KnownHash(PyObject *, PyObject *, Py_hash_t); PyAPI_FUNC(int) _PyDict_ContainsId(PyObject *, _Py_Identifier *); + PyAPI_FUNC(PyObject *) _PyDict_NewPresized(Py_ssize_t minused); -PyAPI_FUNC(void) _PyDict_MaybeUntrack(PyObject *mp); -PyAPI_FUNC(int) _PyDict_HasOnlyStringKeys(PyObject *mp); PyAPI_FUNC(Py_ssize_t) _PyDict_SizeOf(PyDictObject *); PyAPI_FUNC(PyObject *) _PyDict_Pop(PyObject *, PyObject *, PyObject *); #define _PyDict_HasSplitTable(d) ((d)->ma_values != NULL) -/* Like PyDict_Merge, but override can be 0, 1 or 2. If override is 0, - the first occurrence of a key wins, if override is 1, the last occurrence - of a key wins, if override is 2, a KeyError with conflicting key as - argument is raised. -*/ -PyAPI_FUNC(int) _PyDict_MergeEx(PyObject *mp, PyObject *other, int override); PyAPI_FUNC(int) _PyDict_SetItemId(PyObject *dp, _Py_Identifier *key, PyObject *item); PyAPI_FUNC(int) _PyDict_DelItemId(PyObject *mp, _Py_Identifier *key); -PyAPI_FUNC(void) _PyDict_DebugMallocStats(FILE *out); /* _PyDictView */ diff --git a/Include/internal/pycore_dict.h b/Include/internal/pycore_dict.h index 6253e0841ad34..26a913435ef9c 100644 --- a/Include/internal/pycore_dict.h +++ b/Include/internal/pycore_dict.h @@ -12,6 +12,26 @@ extern "C" { #include "pycore_dict_state.h" #include "pycore_runtime.h" // _PyRuntime +// Unsafe flavor of PyDict_GetItemWithError(): no error checking +extern PyObject* _PyDict_GetItemWithError(PyObject *dp, PyObject *key); + +extern int _PyDict_Contains_KnownHash(PyObject *, PyObject *, Py_hash_t); + +extern int _PyDict_DelItemIf(PyObject *mp, PyObject *key, + int (*predicate)(PyObject *value)); + +extern int _PyDict_HasOnlyStringKeys(PyObject *mp); + +extern void _PyDict_MaybeUntrack(PyObject *mp); + +/* Like PyDict_Merge, but override can be 0, 1 or 2. If override is 0, + the first occurrence of a key wins, if override is 1, the last occurrence + of a key wins, if override is 2, a KeyError with conflicting key as + argument is raised. +*/ +extern int _PyDict_MergeEx(PyObject *mp, PyObject *other, int override); + +extern void _PyDict_DebugMallocStats(FILE *out); /* runtime lifecycle */ diff --git a/Modules/_weakref.c b/Modules/_weakref.c index b5d80cbd731a2..dcb2984e3e817 100644 --- a/Modules/_weakref.c +++ b/Modules/_weakref.c @@ -1,4 +1,5 @@ #include "Python.h" +#include "pycore_dict.h" // _PyDict_DelItemIf() #include "pycore_object.h" // _PyObject_GET_WEAKREFS_LISTPTR #include "pycore_weakref.h" // _PyWeakref_IS_DEAD() diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c index 97644a7cee774..246c0a9e160aa 100644 --- a/Modules/gcmodule.c +++ b/Modules/gcmodule.c @@ -25,12 +25,13 @@ #include "Python.h" #include "pycore_context.h" +#include "pycore_dict.h" // _PyDict_MaybeUntrack() #include "pycore_initconfig.h" -#include "pycore_interp.h" // PyInterpreterState.gc +#include "pycore_interp.h" // PyInterpreterState.gc #include "pycore_object.h" #include "pycore_pyerrors.h" -#include "pycore_pystate.h" // _PyThreadState_GET() -#include "pycore_weakref.h" // _PyWeakref_ClearRef() +#include "pycore_pystate.h" // _PyThreadState_GET() +#include "pycore_weakref.h" // _PyWeakref_ClearRef() #include "pydtrace.h" typedef struct _gc_runtime_state GCState; diff --git a/Objects/setobject.c b/Objects/setobject.c index 2f12320254ec9..c96b62e38ec27 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -32,6 +32,7 @@ */ #include "Python.h" +#include "pycore_dict.h" // _PyDict_Contains_KnownHash() #include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "pycore_setobject.h" // _PySet_NextEntry() definition diff --git a/Python/_warnings.c b/Python/_warnings.c index 82e621243a0c1..40ec5f613d5bf 100644 --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -1,4 +1,5 @@ #include "Python.h" +#include "pycore_dict.h" // _PyDict_GetItemWithError() #include "pycore_frame.h" #include "pycore_initconfig.h" #include "pycore_interp.h" // PyInterpreterState.warnings diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c index 7d77fd5c0c328..9baf233614879 100644 --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -6,6 +6,7 @@ #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_ceval.h" // _PyEval_Vector() #include "pycore_compile.h" // _PyAST_Compile() +#include "pycore_dict.h" // _PyDict_GetItemWithError() #include "pycore_long.h" // _PyLong_CompactValue #include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _Py_AddToAllObjects() diff --git a/Python/getargs.c b/Python/getargs.c index 45befab4f8bc3..efcf2e333af8d 100644 --- a/Python/getargs.c +++ b/Python/getargs.c @@ -2,8 +2,9 @@ /* New getargs implementation */ #include "Python.h" -#include "pycore_tuple.h" // _PyTuple_ITEMS() +#include "pycore_dict.h" // _PyDict_HasOnlyStringKeys() #include "pycore_pylifecycle.h" // _PyArg_Fini +#include "pycore_tuple.h" // _PyTuple_ITEMS() #include #include diff --git a/Python/sysmodule.c b/Python/sysmodule.c index fb033a6b5634a..be026d95ba7e7 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -17,6 +17,7 @@ Data members: #include "Python.h" #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_ceval.h" // _PyEval_SetAsyncGenFinalizer() +#include "pycore_dict.h" // _PyDict_GetItemWithError() #include "pycore_frame.h" // _PyInterpreterFrame #include "pycore_initconfig.h" // _PyStatus_EXCEPTION() #include "pycore_long.h" // _PY_LONG_MAX_STR_DIGITS_THRESHOLD From webhook-mailer at python.org Mon Jul 24 10:15:06 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 14:15:06 -0000 Subject: [Python-checkins] gh-107162: Document errcode.h usage in its comment (#107177) Message-ID: https://github.com/python/cpython/commit/751695327562462705959e3b1cd4a37dba522178 commit: 751695327562462705959e3b1cd4a37dba522178 branch: main author: Victor Stinner committer: vstinner date: 2023-07-24T14:14:58Z summary: gh-107162: Document errcode.h usage in its comment (#107177) Public and documented PyRun_InteractiveOneFlags() C API uses it. files: M Include/errcode.h diff --git a/Include/errcode.h b/Include/errcode.h index 54ae929bf2587..8d44e9ae55919 100644 --- a/Include/errcode.h +++ b/Include/errcode.h @@ -1,18 +1,24 @@ +// Error codes passed around between file input, tokenizer, parser and +// interpreter. This is necessary so we can turn them into Python +// exceptions at a higher level. Note that some errors have a +// slightly different meaning when passed from the tokenizer to the +// parser than when passed from the parser to the interpreter; e.g. +// the parser only returns E_EOF when it hits EOF immediately, and it +// never returns E_OK. +// +// The public PyRun_InteractiveOneObjectEx() function can return E_EOF, +// same as its variants: +// +// * PyRun_InteractiveOneObject() +// * PyRun_InteractiveOneFlags() +// * PyRun_InteractiveOne() + #ifndef Py_ERRCODE_H #define Py_ERRCODE_H #ifdef __cplusplus extern "C" { #endif - -/* Error codes passed around between file input, tokenizer, parser and - interpreter. This is necessary so we can turn them into Python - exceptions at a higher level. Note that some errors have a - slightly different meaning when passed from the tokenizer to the - parser than when passed from the parser to the interpreter; e.g. - the parser only returns E_EOF when it hits EOF immediately, and it - never returns E_OK. */ - #define E_OK 10 /* No error */ #define E_EOF 11 /* End Of File */ #define E_INTR 12 /* Interrupted */ From webhook-mailer at python.org Mon Jul 24 11:22:22 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 24 Jul 2023 15:22:22 -0000 Subject: [Python-checkins] Docs: Add missing markup to Argument Clinic docs (#106876) Message-ID: https://github.com/python/cpython/commit/ff5f94b72c8aad8e45c397c263dbe7f19221735f commit: ff5f94b72c8aad8e45c397c263dbe7f19221735f branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-24T15:22:18Z summary: Docs: Add missing markup to Argument Clinic docs (#106876) Co-authored-by: Ezio Melotti Co-authored-by: Serhiy Storchaka files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 933fecab9ddd5..98d3632ff0232 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -96,7 +96,8 @@ things with all the information you give it. Basic concepts and usage ------------------------ -Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``. +Argument Clinic ships with CPython; you'll find it in +:source:`Tools/clinic/clinic.py`. If you run that script, specifying a C file as an argument: .. code-block:: shell-session @@ -178,9 +179,10 @@ Let's dive in! 1. Find a Python builtin that calls either :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords`, and hasn't been converted to work with Argument Clinic yet. - For my example I'm using ``_pickle.Pickler.dump()``. + For my example I'm using + :py:meth:`_pickle.Pickler.dump `. -2. If the call to the ``PyArg_Parse`` function uses any of the +2. If the call to the :c:func:`!PyArg_Parse*` function uses any of the following format units: .. code-block:: none @@ -197,10 +199,10 @@ Let's dive in! support all of these scenarios. But these are advanced topics?let's do something simpler for your first function. - Also, if the function has multiple calls to :c:func:`PyArg_ParseTuple` + Also, if the function has multiple calls to :c:func:`!PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different types for the same argument, or if the function uses something besides - PyArg_Parse functions to parse its arguments, it probably + :c:func:`!PyArg_Parse*` functions to parse its arguments, it probably isn't suitable for conversion to Argument Clinic. Argument Clinic doesn't support generic functions or polymorphic parameters. @@ -217,7 +219,7 @@ Let's dive in! If the old docstring had a first line that looked like a function signature, throw that line away. (The docstring doesn't need it - anymore?when you use ``help()`` on your builtin in the future, + anymore?when you use :py:func:`help` on your builtin in the future, the first line will be built automatically based on the function's signature.) @@ -264,7 +266,7 @@ Let's dive in! When you declare a class, you must also specify two aspects of its type in C: the type declaration you'd use for a pointer to an instance of - this class, and a pointer to the :c:type:`PyTypeObject` for this class. + this class, and a pointer to the :c:type:`!PyTypeObject` for this class. Sample:: @@ -313,10 +315,10 @@ Let's dive in! Clinic easier. For each parameter, copy the "format unit" for that - parameter from the ``PyArg_Parse()`` format argument and + parameter from the :c:func:`PyArg_Parse` format argument and specify *that* as its converter, as a quoted string. ("format unit" is the formal name for the one-to-three - character substring of the ``format`` parameter that tells + character substring of the *format* parameter that tells the argument parsing function what the type of the variable is and how to convert it. For more on format units please see :ref:`arg-parsing`.) @@ -349,7 +351,7 @@ Let's dive in! itself before the first keyword-only argument, indented the same as the parameter lines. - (``_pickle.Pickler.dump`` has neither, so our sample is unchanged.) + (:py:meth:`!_pickle.Pickler.dump` has neither, so our sample is unchanged.) 10. If the existing C function calls :c:func:`PyArg_ParseTuple` @@ -410,7 +412,7 @@ Let's dive in! 12. Save and close the file, then run ``Tools/clinic/clinic.py`` on it. With luck everything worked---your block now has output, and - a ``.c.h`` file has been generated! Reopen the file in your + a :file:`.c.h` file has been generated! Reopen the file in your text editor to see:: /*[clinic input] @@ -431,8 +433,8 @@ Let's dive in! it found an error in your input. Keep fixing your errors and retrying until Argument Clinic processes your file without complaint. - For readability, most of the glue code has been generated to a ``.c.h`` - file. You'll need to include that in your original ``.c`` file, + For readability, most of the glue code has been generated to a :file:`.c.h` + file. You'll need to include that in your original :file:`.c` file, typically right after the clinic module block:: #include "clinic/_pickle.c.h" @@ -446,8 +448,8 @@ Let's dive in! ensure that the code generated by Argument Clinic calls the *exact* same function. - Second, the format string passed in to :c:func:`PyArg_ParseTuple` or - :c:func:`PyArg_ParseTupleAndKeywords` should be *exactly* the same + Second, the format string passed in to :c:func:`!PyArg_ParseTuple` or + :c:func:`!PyArg_ParseTupleAndKeywords` should be *exactly* the same as the hand-written one in the existing function, up to the colon or semi-colon. @@ -469,7 +471,7 @@ Let's dive in! {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, This static structure should be *exactly* the same as the existing static - :c:type:`PyMethodDef` structure for this builtin. + :c:type:`!PyMethodDef` structure for this builtin. If any of these items differ in *any way*, adjust your Argument Clinic function specification and rerun @@ -539,14 +541,14 @@ Let's dive in! ... 15. Remember the macro with the :c:type:`PyMethodDef` structure for this - function? Find the existing :c:type:`PyMethodDef` structure for this + function? Find the existing :c:type:`!PyMethodDef` structure for this function and replace it with a reference to the macro. (If the builtin is at module scope, this will probably be very near the end of the file; if the builtin is a class method, this will probably be below but relatively near to the implementation.) Note that the body of the macro contains a trailing comma. So when you - replace the existing static :c:type:`PyMethodDef` structure with the macro, + replace the existing static :c:type:`!PyMethodDef` structure with the macro, *don't* add a comma to the end. Sample:: @@ -562,7 +564,7 @@ Let's dive in! &_Py_ID(new_unique_py_id) - If it does, you'll have to run ``Tools/scripts/generate_global_objects.py`` + If it does, you'll have to run ``make regen-global-objects`` to regenerate the list of precompiled identifiers at this point. @@ -570,7 +572,7 @@ Let's dive in! This change should not introduce any new compile-time warnings or errors, and there should be no externally visible change to Python's behavior. - Well, except for one difference: ``inspect.signature()`` run on your function + Well, except for one difference: :py:func:`inspect.signature` run on your function should now provide a valid signature! Congratulations, you've ported your first function to work with Argument Clinic! @@ -594,15 +596,15 @@ Argument Clinic will use that function name for the base (generated) function, then add ``"_impl"`` to the end and use that for the name of the impl function. For example, if we wanted to rename the C function names generated for -``pickle.Pickler.dump``, it'd look like this:: +:py:meth:`pickle.Pickler.dump`, it'd look like this:: /*[clinic input] pickle.Pickler.dump as pickler_dumper ... -The base function would now be named ``pickler_dumper()``, -and the impl function would now be named ``pickler_dumper_impl()``. +The base function would now be named :c:func:`!pickler_dumper`, +and the impl function would now be named :c:func:`!pickler_dumper_impl`. Similarly, you may have a problem where you want to give a parameter @@ -620,9 +622,9 @@ using the same ``"as"`` syntax:: fix_imports: bool = True Here, the name used in Python (in the signature and the ``keywords`` -array) would be ``file``, but the C variable would be named ``file_obj``. +array) would be *file*, but the C variable would be named ``file_obj``. -You can use this to rename the ``self`` parameter too! +You can use this to rename the *self* parameter too! How to convert functions using ``PyArg_UnpackTuple`` @@ -630,7 +632,7 @@ How to convert functions using ``PyArg_UnpackTuple`` To convert a function parsing its arguments with :c:func:`PyArg_UnpackTuple`, simply write out all the arguments, specifying each as an ``object``. You -may specify the ``type`` argument to cast the type as appropriate. All +may specify the *type* argument to cast the type as appropriate. All arguments should be marked positional-only (add a ``/`` on a line by itself after the last argument). @@ -649,16 +651,16 @@ keyword-only arguments.) This approach was used to simulate optional arguments back before :c:func:`PyArg_ParseTupleAndKeywords` was created. While functions using this approach can often be converted to -use :c:func:`PyArg_ParseTupleAndKeywords`, optional arguments, and default values, +use :c:func:`!PyArg_ParseTupleAndKeywords`, optional arguments, and default values, it's not always possible. Some of these legacy functions have -behaviors :c:func:`PyArg_ParseTupleAndKeywords` doesn't directly support. -The most obvious example is the builtin function ``range()``, which has +behaviors :c:func:`!PyArg_ParseTupleAndKeywords` doesn't directly support. +The most obvious example is the builtin function :py:func:`range`, which has an optional argument on the *left* side of its required argument! -Another example is ``curses.window.addch()``, which has a group of two +Another example is :py:meth:`curses.window.addch`, which has a group of two arguments that must always be specified together. (The arguments are -called ``x`` and ``y``; if you call the function passing in ``x``, -you must also pass in ``y``?and if you don't pass in ``x`` you may not -pass in ``y`` either.) +called *x* and *y*; if you call the function passing in *x*, +you must also pass in *y* ? and if you don't pass in *x* you may not +pass in *y* either.) In any case, the goal of Argument Clinic is to support argument parsing for all existing CPython builtins without changing their semantics. @@ -679,7 +681,7 @@ can *only* be used with positional-only parameters. To specify an optional group, add a ``[`` on a line by itself before the parameters you wish to group together, and a ``]`` on a line by itself -after these parameters. As an example, here's how ``curses.window.addch`` +after these parameters. As an example, here's how :py:meth:`curses.window.addch` uses optional groups to make the first two parameters and the last parameter optional:: @@ -765,25 +767,25 @@ the same converters. All arguments to Argument Clinic converters are keyword-only. All Argument Clinic converters accept the following arguments: - ``c_default`` + *c_default* The default value for this parameter when defined in C. Specifically, this will be the initializer for the variable declared in the "parse function". See :ref:`the section on default values ` for how to use this. Specified as a string. - ``annotation`` + *annotation* The annotation value for this parameter. Not currently supported, because :pep:`8` mandates that the Python library may not use annotations. - ``unused`` + *unused* Wrap the argument with :c:macro:`Py_UNUSED` in the impl function signature. In addition, some converters accept additional arguments. Here is a list of these arguments, along with their meanings: - ``accept`` + *accept* A set of Python types (and possibly pseudo-types); this restricts the allowable Python argument to values of these types. (This is not a general-purpose facility; as a rule it only supports @@ -791,38 +793,38 @@ of these arguments, along with their meanings: To accept ``None``, add ``NoneType`` to this set. - ``bitwise`` + *bitwise* Only supported for unsigned integers. The native integer value of this Python argument will be written to the parameter without any range checking, even for negative values. - ``converter`` + *converter* Only supported by the ``object`` converter. Specifies the name of a :ref:`C "converter function" ` to use to convert this object to a native type. - ``encoding`` + *encoding* Only supported for strings. Specifies the encoding to use when converting this string from a Python str (Unicode) value into a C ``char *`` value. - ``subclass_of`` + *subclass_of* Only supported for the ``object`` converter. Requires that the Python value be a subclass of a Python type, as expressed in C. - ``type`` + *type* Only supported for the ``object`` and ``self`` converters. Specifies the C type that will be used to declare the variable. Default value is ``"PyObject *"``. - ``zeroes`` + *zeroes* Only supported for strings. If true, embedded NUL bytes (``'\\0'``) are permitted inside the value. The length of the string will be passed in to the impl function, just after the string parameter, as a parameter named ``_length``. Please note, not every possible combination of arguments will work. -Usually these arguments are implemented by specific ``PyArg_ParseTuple`` +Usually these arguments are implemented by specific :c:func:`PyArg_ParseTuple` *format units*, with specific behavior. For example, currently you cannot call ``unsigned_short`` without also specifying ``bitwise=True``. Although it's perfectly reasonable to think this would work, these semantics don't @@ -922,19 +924,19 @@ conversion functions, or types, or strings specifying an encoding. (But "legacy converters" don't support arguments. That's why we skipped them for your first function.) The argument you specified to the format unit is now an argument to the converter; this -argument is either ``converter`` (for ``O&``), ``subclass_of`` (for ``O!``), -or ``encoding`` (for all the format units that start with ``e``). +argument is either *converter* (for ``O&``), *subclass_of* (for ``O!``), +or *encoding* (for all the format units that start with ``e``). -When using ``subclass_of``, you may also want to use the other -custom argument for ``object()``: ``type``, which lets you set the type +When using *subclass_of*, you may also want to use the other +custom argument for ``object()``: *type*, which lets you set the type actually used for the parameter. For example, if you want to ensure -that the object is a subclass of ``PyUnicode_Type``, you probably want +that the object is a subclass of :c:var:`PyUnicode_Type`, you probably want to use the converter ``object(type='PyUnicodeObject *', subclass_of='&PyUnicode_Type')``. One possible problem with using Argument Clinic: it takes away some possible flexibility for the format units starting with ``e``. When writing a -``PyArg_Parse`` call by hand, you could theoretically decide at runtime what -encoding string to pass in to :c:func:`PyArg_ParseTuple`. But now this string must +:c:func:`!PyArg_Parse*` call by hand, you could theoretically decide at runtime what +encoding string to pass to that call. But now this string must be hard-coded at Argument-Clinic-preprocessing-time. This limitation is deliberate; it made supporting this format unit much easier, and may allow for future optimizations. This restriction doesn't seem unreasonable; CPython itself always passes in static @@ -987,7 +989,7 @@ expression. Currently the following are explicitly supported: * Numeric constants (integer and float) * String constants * ``True``, ``False``, and ``None`` -* Simple symbolic constants like ``sys.maxsize``, which must +* Simple symbolic constants like :py:data:`sys.maxsize`, which must start with the name of the module (In the future, this may need to get even more elaborate, @@ -1008,28 +1010,28 @@ Consider the following example: foo: Py_ssize_t = sys.maxsize - 1 -``sys.maxsize`` can have different values on different platforms. Therefore +:py:data:`sys.maxsize` can have different values on different platforms. Therefore Argument Clinic can't simply evaluate that expression locally and hard-code it in C. So it stores the default in such a way that it will get evaluated at runtime, when the user asks for the function's signature. What namespace is available when the expression is evaluated? It's evaluated in the context of the module the builtin came from. So, if your module has an -attribute called "``max_widgets``", you may simply use it: +attribute called :py:attr:`!max_widgets`, you may simply use it: .. code-block:: none foo: Py_ssize_t = max_widgets If the symbol isn't found in the current module, it fails over to looking in -``sys.modules``. That's how it can find ``sys.maxsize`` for example. (Since you -don't know in advance what modules the user will load into their interpreter, +:py:data:`sys.modules`. That's how it can find :py:data:`sys.maxsize` for example. +(Since you don't know in advance what modules the user will load into their interpreter, it's best to restrict yourself to modules that are preloaded by Python itself.) Evaluating default values only at runtime means Argument Clinic can't compute the correct equivalent C default value. So you need to tell it explicitly. When you use an expression, you must also specify the equivalent expression -in C, using the ``c_default`` parameter to the converter: +in C, using the *c_default* parameter to the converter: .. code-block:: none @@ -1095,7 +1097,7 @@ indicate an error has occurred? Normally, a function returns a valid (non-``NUL pointer for success, and ``NULL`` for failure. But if you use an integer return converter, all integers are valid. How can Argument Clinic detect an error? Its solution: each return converter implicitly looks for a special value that indicates an error. If you return -that value, and an error has been set (``PyErr_Occurred()`` returns a true +that value, and an error has been set (c:func:`PyErr_Occurred` returns a true value), then the generated code will propagate the error. Otherwise it will encode the value you return like normal. @@ -1201,9 +1203,9 @@ using a default converter. It automatically sets the ``type`` of this parameter to the "pointer to an instance" you specified when you declared the type. However, you can override Argument Clinic's converter and specify one yourself. -Just add your own ``self`` parameter as the first parameter in a +Just add your own *self* parameter as the first parameter in a block, and ensure that its converter is an instance of -``self_converter`` or a subclass thereof. +:class:`!self_converter` or a subclass thereof. What's the point? This lets you override the type of ``self``, or give it a different default name. @@ -1211,7 +1213,7 @@ or give it a different default name. How do you specify the custom type you want to cast ``self`` to? If you only have one or two functions with the same type for ``self``, you can directly use Argument Clinic's existing ``self`` converter, -passing in the type you want to use as the ``type`` parameter:: +passing in the type you want to use as the *type* parameter:: /*[clinic input] @@ -1226,7 +1228,7 @@ passing in the type you want to use as the ``type`` parameter:: On the other hand, if you have a lot of functions that will use the same type for ``self``, it's best to create your own converter, subclassing -``self_converter`` but overwriting the ``type`` member:: +:class:`!self_converter` but overwriting the :py:attr:`!type` member:: /*[python input] class PicklerObject_converter(self_converter): @@ -1254,8 +1256,8 @@ module level state. Use :c:func:`PyType_FromModuleAndSpec` to associate a new heap type with a module. You can now use :c:func:`PyType_GetModuleState` on the defining class to fetch the module state, for example from a module method. -Example from ``Modules/zlibmodule.c``. First, ``defining_class`` is added to -the clinic input:: +Example from :source:`Modules/zlibmodule.c`. +First, ``defining_class`` is added to the clinic input:: /*[clinic input] zlib.Compress.compress @@ -1285,16 +1287,17 @@ module state:: Each method may only have one argument using this converter, and it must appear after ``self``, or, if ``self`` is not used, as the first argument. The argument will be of type ``PyTypeObject *``. The argument will not appear in the -``__text_signature__``. +:py:attr:`!__text_signature__`. -The ``defining_class`` converter is not compatible with ``__init__`` and ``__new__`` -methods, which cannot use the ``METH_METHOD`` convention. +The ``defining_class`` converter is not compatible with :py:meth:`!__init__` +and :py:meth:`!__new__` methods, which cannot use the :c:macro:`METH_METHOD` +convention. It is not possible to use ``defining_class`` with slot methods. In order to fetch the module state from such methods, use :c:func:`PyType_GetModuleByDef` to look up the module and then :c:func:`PyModule_GetState` to fetch the module state. Example from the ``setattro`` slot method in -``Modules/_threadmodule.c``:: +:source:`Modules/_threadmodule.c`:: static int local_setattro(localobject *self, PyObject *name, PyObject *v) @@ -1312,7 +1315,7 @@ How to write a custom converter ------------------------------- As we hinted at in the previous section... you can write your own converters! -A converter is simply a Python class that inherits from ``CConverter``. +A converter is simply a Python class that inherits from :py:class:`!CConverter`. The main purpose of a custom converter is if you have a parameter using the ``O&`` format unit?parsing this parameter means calling a :c:func:`PyArg_ParseTuple` "converter function". @@ -1323,61 +1326,74 @@ will be automatically registered with Argument Clinic; its name will be the name of your class with the ``_converter`` suffix stripped off. (This is accomplished with a metaclass.) -You shouldn't subclass ``CConverter.__init__``. Instead, you should -write a ``converter_init()`` function. ``converter_init()`` -always accepts a ``self`` parameter; after that, all additional +You shouldn't subclass :py:meth:`!CConverter.__init__`. Instead, you should +write a :py:meth:`!converter_init` function. :py:meth:`!converter_init` +always accepts a *self* parameter; after that, all additional parameters *must* be keyword-only. Any arguments passed in to the converter in Argument Clinic will be passed along to your -``converter_init()``. +:py:meth:`!converter_init`. -There are some additional members of ``CConverter`` you may wish +There are some additional members of :py:class:`!CConverter` you may wish to specify in your subclass. Here's the current list: -``type`` - The C type to use for this variable. - ``type`` should be a Python string specifying the type, e.g. ``int``. - If this is a pointer type, the type string should end with ``' *'``. +.. module:: clinic -``default`` - The Python default value for this parameter, as a Python value. - Or the magic value ``unspecified`` if there is no default. +.. class:: CConverter -``py_default`` - ``default`` as it should appear in Python code, - as a string. - Or ``None`` if there is no default. + .. attribute:: type -``c_default`` - ``default`` as it should appear in C code, - as a string. - Or ``None`` if there is no default. + The C type to use for this variable. + :attr:`!type` should be a Python string specifying the type, + e.g. ``'int'``. + If this is a pointer type, the type string should end with ``' *'``. -``c_ignored_default`` - The default value used to initialize the C variable when - there is no default, but not specifying a default may - result in an "uninitialized variable" warning. This can - easily happen when using option groups?although - properly written code will never actually use this value, - the variable does get passed in to the impl, and the - C compiler will complain about the "use" of the - uninitialized value. This value should always be a - non-empty string. + .. attribute:: default -``converter`` - The name of the C converter function, as a string. + The Python default value for this parameter, as a Python value. + Or the magic value ``unspecified`` if there is no default. -``impl_by_reference`` - A boolean value. If true, - Argument Clinic will add a ``&`` in front of the name of - the variable when passing it into the impl function. + .. attribute:: py_default -``parse_by_reference`` - A boolean value. If true, - Argument Clinic will add a ``&`` in front of the name of - the variable when passing it into :c:func:`PyArg_ParseTuple`. + :attr:`!default` as it should appear in Python code, + as a string. + Or ``None`` if there is no default. + .. attribute:: c_default -Here's the simplest example of a custom converter, from ``Modules/zlibmodule.c``:: + :attr:`!default` as it should appear in C code, + as a string. + Or ``None`` if there is no default. + + .. attribute:: c_ignored_default + + The default value used to initialize the C variable when + there is no default, but not specifying a default may + result in an "uninitialized variable" warning. This can + easily happen when using option groups?although + properly written code will never actually use this value, + the variable does get passed in to the impl, and the + C compiler will complain about the "use" of the + uninitialized value. This value should always be a + non-empty string. + + .. attribute:: converter + + The name of the C converter function, as a string. + + .. attribute:: impl_by_reference + + A boolean value. If true, + Argument Clinic will add a ``&`` in front of the name of + the variable when passing it into the impl function. + + .. attribute:: parse_by_reference + + A boolean value. If true, + Argument Clinic will add a ``&`` in front of the name of + the variable when passing it into :c:func:`PyArg_ParseTuple`. + + +Here's the simplest example of a custom converter, from :source:`Modules/zlibmodule.c`:: /*[python input] @@ -1397,7 +1413,7 @@ automatically support default values. More sophisticated custom converters can insert custom C code to handle initialization and cleanup. You can see more examples of custom converters in the CPython -source tree; grep the C files for the string ``CConverter``. +source tree; grep the C files for the string :py:class:`!CConverter`. How to write a custom return converter @@ -1407,18 +1423,18 @@ Writing a custom return converter is much like writing a custom converter. Except it's somewhat simpler, because return converters are themselves much simpler. -Return converters must subclass ``CReturnConverter``. +Return converters must subclass :py:class:`!CReturnConverter`. There are no examples yet of custom return converters, because they are not widely used yet. If you wish to -write your own return converter, please read ``Tools/clinic/clinic.py``, -specifically the implementation of ``CReturnConverter`` and +write your own return converter, please read :source:`Tools/clinic/clinic.py`, +specifically the implementation of :py:class:`!CReturnConverter` and all its subclasses. How to convert ``METH_O`` and ``METH_NOARGS`` functions ------------------------------------------------------- -To convert a function using ``METH_O``, make sure the function's +To convert a function using :c:macro:`METH_O`, make sure the function's single argument is using the ``object`` converter, and mark the arguments as positional-only:: @@ -1430,24 +1446,25 @@ arguments as positional-only:: [clinic start generated code]*/ -To convert a function using ``METH_NOARGS``, just don't specify +To convert a function using :c:macro:`METH_NOARGS`, just don't specify any arguments. You can still use a self converter, a return converter, and specify -a ``type`` argument to the object converter for ``METH_O``. +a *type* argument to the object converter for :c:macro:`METH_O`. How to convert ``tp_new`` and ``tp_init`` functions --------------------------------------------------- -You can convert ``tp_new`` and ``tp_init`` functions. Just name -them ``__new__`` or ``__init__`` as appropriate. Notes: +You can convert :c:member:`~PyTypeObject.tp_new` and +:c:member:`~PyTypeObject.tp_init` functions. +Just name them ``__new__`` or ``__init__`` as appropriate. Notes: * The function name generated for ``__new__`` doesn't end in ``__new__`` like it would by default. It's just the name of the class, converted into a valid C identifier. -* No ``PyMethodDef`` ``#define`` is generated for these functions. +* No :c:type:`PyMethodDef` ``#define`` is generated for these functions. * ``__init__`` functions return ``int``, not ``PyObject *``. @@ -1482,7 +1499,7 @@ Let's start with defining some terminology: *field* A field, in this context, is a subsection of Clinic's output. - For example, the ``#define`` for the ``PyMethodDef`` structure + For example, the ``#define`` for the :c:type:`PyMethodDef` structure is a field, called ``methoddef_define``. Clinic has seven different fields it can output per function definition: @@ -1526,8 +1543,8 @@ Let's start with defining some terminology: The filename chosen for the file is ``{basename}.clinic{extension}``, where ``basename`` and ``extension`` were assigned the output from ``os.path.splitext()`` run on the current file. (Example: - the ``file`` destination for ``_pickle.c`` would be written to - ``_pickle.clinic.c``.) + the ``file`` destination for :file:`_pickle.c` would be written to + :file:`_pickle.clinic.c`.) **Important: When using a** ``file`` **destination, you** *must check in* **the generated file!** @@ -1780,7 +1797,7 @@ like so:: } #endif /* HAVE_FUNCTIONNAME */ -Then, remove those three lines from the ``PyMethodDef`` structure, +Then, remove those three lines from the :c:type:`PyMethodDef` structure, replacing them with the macro Argument Clinic generated: .. code-block:: none @@ -1821,7 +1838,7 @@ This may mean that you get a complaint from Argument Clinic: When this happens, just open your file, find the ``dump buffer`` block that Argument Clinic added to your file (it'll be at the very bottom), then -move it above the ``PyMethodDef`` structure where that macro is used. +move it above the :c:type:`PyMethodDef` structure where that macro is used. How to use Argument Clinic in Python files From webhook-mailer at python.org Mon Jul 24 11:32:56 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 24 Jul 2023 15:32:56 -0000 Subject: [Python-checkins] [3.12] Docs: Add missing markup to Argument Clinic docs (GH-106876) (#107181) Message-ID: https://github.com/python/cpython/commit/5ff4dfa0cb6faf28368691c6291b02bd8a38ecb8 commit: 5ff4dfa0cb6faf28368691c6291b02bd8a38ecb8 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-24T15:32:51Z summary: [3.12] Docs: Add missing markup to Argument Clinic docs (GH-106876) (#107181) (cherry picked from commit ff5f94b72c8aad8e45c397c263dbe7f19221735f) Co-authored-by: Erlend E. Aasland Co-authored-by: Ezio Melotti Co-authored-by: Serhiy Storchaka files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 933fecab9ddd5..98d3632ff0232 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -96,7 +96,8 @@ things with all the information you give it. Basic concepts and usage ------------------------ -Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``. +Argument Clinic ships with CPython; you'll find it in +:source:`Tools/clinic/clinic.py`. If you run that script, specifying a C file as an argument: .. code-block:: shell-session @@ -178,9 +179,10 @@ Let's dive in! 1. Find a Python builtin that calls either :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords`, and hasn't been converted to work with Argument Clinic yet. - For my example I'm using ``_pickle.Pickler.dump()``. + For my example I'm using + :py:meth:`_pickle.Pickler.dump `. -2. If the call to the ``PyArg_Parse`` function uses any of the +2. If the call to the :c:func:`!PyArg_Parse*` function uses any of the following format units: .. code-block:: none @@ -197,10 +199,10 @@ Let's dive in! support all of these scenarios. But these are advanced topics?let's do something simpler for your first function. - Also, if the function has multiple calls to :c:func:`PyArg_ParseTuple` + Also, if the function has multiple calls to :c:func:`!PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different types for the same argument, or if the function uses something besides - PyArg_Parse functions to parse its arguments, it probably + :c:func:`!PyArg_Parse*` functions to parse its arguments, it probably isn't suitable for conversion to Argument Clinic. Argument Clinic doesn't support generic functions or polymorphic parameters. @@ -217,7 +219,7 @@ Let's dive in! If the old docstring had a first line that looked like a function signature, throw that line away. (The docstring doesn't need it - anymore?when you use ``help()`` on your builtin in the future, + anymore?when you use :py:func:`help` on your builtin in the future, the first line will be built automatically based on the function's signature.) @@ -264,7 +266,7 @@ Let's dive in! When you declare a class, you must also specify two aspects of its type in C: the type declaration you'd use for a pointer to an instance of - this class, and a pointer to the :c:type:`PyTypeObject` for this class. + this class, and a pointer to the :c:type:`!PyTypeObject` for this class. Sample:: @@ -313,10 +315,10 @@ Let's dive in! Clinic easier. For each parameter, copy the "format unit" for that - parameter from the ``PyArg_Parse()`` format argument and + parameter from the :c:func:`PyArg_Parse` format argument and specify *that* as its converter, as a quoted string. ("format unit" is the formal name for the one-to-three - character substring of the ``format`` parameter that tells + character substring of the *format* parameter that tells the argument parsing function what the type of the variable is and how to convert it. For more on format units please see :ref:`arg-parsing`.) @@ -349,7 +351,7 @@ Let's dive in! itself before the first keyword-only argument, indented the same as the parameter lines. - (``_pickle.Pickler.dump`` has neither, so our sample is unchanged.) + (:py:meth:`!_pickle.Pickler.dump` has neither, so our sample is unchanged.) 10. If the existing C function calls :c:func:`PyArg_ParseTuple` @@ -410,7 +412,7 @@ Let's dive in! 12. Save and close the file, then run ``Tools/clinic/clinic.py`` on it. With luck everything worked---your block now has output, and - a ``.c.h`` file has been generated! Reopen the file in your + a :file:`.c.h` file has been generated! Reopen the file in your text editor to see:: /*[clinic input] @@ -431,8 +433,8 @@ Let's dive in! it found an error in your input. Keep fixing your errors and retrying until Argument Clinic processes your file without complaint. - For readability, most of the glue code has been generated to a ``.c.h`` - file. You'll need to include that in your original ``.c`` file, + For readability, most of the glue code has been generated to a :file:`.c.h` + file. You'll need to include that in your original :file:`.c` file, typically right after the clinic module block:: #include "clinic/_pickle.c.h" @@ -446,8 +448,8 @@ Let's dive in! ensure that the code generated by Argument Clinic calls the *exact* same function. - Second, the format string passed in to :c:func:`PyArg_ParseTuple` or - :c:func:`PyArg_ParseTupleAndKeywords` should be *exactly* the same + Second, the format string passed in to :c:func:`!PyArg_ParseTuple` or + :c:func:`!PyArg_ParseTupleAndKeywords` should be *exactly* the same as the hand-written one in the existing function, up to the colon or semi-colon. @@ -469,7 +471,7 @@ Let's dive in! {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, This static structure should be *exactly* the same as the existing static - :c:type:`PyMethodDef` structure for this builtin. + :c:type:`!PyMethodDef` structure for this builtin. If any of these items differ in *any way*, adjust your Argument Clinic function specification and rerun @@ -539,14 +541,14 @@ Let's dive in! ... 15. Remember the macro with the :c:type:`PyMethodDef` structure for this - function? Find the existing :c:type:`PyMethodDef` structure for this + function? Find the existing :c:type:`!PyMethodDef` structure for this function and replace it with a reference to the macro. (If the builtin is at module scope, this will probably be very near the end of the file; if the builtin is a class method, this will probably be below but relatively near to the implementation.) Note that the body of the macro contains a trailing comma. So when you - replace the existing static :c:type:`PyMethodDef` structure with the macro, + replace the existing static :c:type:`!PyMethodDef` structure with the macro, *don't* add a comma to the end. Sample:: @@ -562,7 +564,7 @@ Let's dive in! &_Py_ID(new_unique_py_id) - If it does, you'll have to run ``Tools/scripts/generate_global_objects.py`` + If it does, you'll have to run ``make regen-global-objects`` to regenerate the list of precompiled identifiers at this point. @@ -570,7 +572,7 @@ Let's dive in! This change should not introduce any new compile-time warnings or errors, and there should be no externally visible change to Python's behavior. - Well, except for one difference: ``inspect.signature()`` run on your function + Well, except for one difference: :py:func:`inspect.signature` run on your function should now provide a valid signature! Congratulations, you've ported your first function to work with Argument Clinic! @@ -594,15 +596,15 @@ Argument Clinic will use that function name for the base (generated) function, then add ``"_impl"`` to the end and use that for the name of the impl function. For example, if we wanted to rename the C function names generated for -``pickle.Pickler.dump``, it'd look like this:: +:py:meth:`pickle.Pickler.dump`, it'd look like this:: /*[clinic input] pickle.Pickler.dump as pickler_dumper ... -The base function would now be named ``pickler_dumper()``, -and the impl function would now be named ``pickler_dumper_impl()``. +The base function would now be named :c:func:`!pickler_dumper`, +and the impl function would now be named :c:func:`!pickler_dumper_impl`. Similarly, you may have a problem where you want to give a parameter @@ -620,9 +622,9 @@ using the same ``"as"`` syntax:: fix_imports: bool = True Here, the name used in Python (in the signature and the ``keywords`` -array) would be ``file``, but the C variable would be named ``file_obj``. +array) would be *file*, but the C variable would be named ``file_obj``. -You can use this to rename the ``self`` parameter too! +You can use this to rename the *self* parameter too! How to convert functions using ``PyArg_UnpackTuple`` @@ -630,7 +632,7 @@ How to convert functions using ``PyArg_UnpackTuple`` To convert a function parsing its arguments with :c:func:`PyArg_UnpackTuple`, simply write out all the arguments, specifying each as an ``object``. You -may specify the ``type`` argument to cast the type as appropriate. All +may specify the *type* argument to cast the type as appropriate. All arguments should be marked positional-only (add a ``/`` on a line by itself after the last argument). @@ -649,16 +651,16 @@ keyword-only arguments.) This approach was used to simulate optional arguments back before :c:func:`PyArg_ParseTupleAndKeywords` was created. While functions using this approach can often be converted to -use :c:func:`PyArg_ParseTupleAndKeywords`, optional arguments, and default values, +use :c:func:`!PyArg_ParseTupleAndKeywords`, optional arguments, and default values, it's not always possible. Some of these legacy functions have -behaviors :c:func:`PyArg_ParseTupleAndKeywords` doesn't directly support. -The most obvious example is the builtin function ``range()``, which has +behaviors :c:func:`!PyArg_ParseTupleAndKeywords` doesn't directly support. +The most obvious example is the builtin function :py:func:`range`, which has an optional argument on the *left* side of its required argument! -Another example is ``curses.window.addch()``, which has a group of two +Another example is :py:meth:`curses.window.addch`, which has a group of two arguments that must always be specified together. (The arguments are -called ``x`` and ``y``; if you call the function passing in ``x``, -you must also pass in ``y``?and if you don't pass in ``x`` you may not -pass in ``y`` either.) +called *x* and *y*; if you call the function passing in *x*, +you must also pass in *y* ? and if you don't pass in *x* you may not +pass in *y* either.) In any case, the goal of Argument Clinic is to support argument parsing for all existing CPython builtins without changing their semantics. @@ -679,7 +681,7 @@ can *only* be used with positional-only parameters. To specify an optional group, add a ``[`` on a line by itself before the parameters you wish to group together, and a ``]`` on a line by itself -after these parameters. As an example, here's how ``curses.window.addch`` +after these parameters. As an example, here's how :py:meth:`curses.window.addch` uses optional groups to make the first two parameters and the last parameter optional:: @@ -765,25 +767,25 @@ the same converters. All arguments to Argument Clinic converters are keyword-only. All Argument Clinic converters accept the following arguments: - ``c_default`` + *c_default* The default value for this parameter when defined in C. Specifically, this will be the initializer for the variable declared in the "parse function". See :ref:`the section on default values ` for how to use this. Specified as a string. - ``annotation`` + *annotation* The annotation value for this parameter. Not currently supported, because :pep:`8` mandates that the Python library may not use annotations. - ``unused`` + *unused* Wrap the argument with :c:macro:`Py_UNUSED` in the impl function signature. In addition, some converters accept additional arguments. Here is a list of these arguments, along with their meanings: - ``accept`` + *accept* A set of Python types (and possibly pseudo-types); this restricts the allowable Python argument to values of these types. (This is not a general-purpose facility; as a rule it only supports @@ -791,38 +793,38 @@ of these arguments, along with their meanings: To accept ``None``, add ``NoneType`` to this set. - ``bitwise`` + *bitwise* Only supported for unsigned integers. The native integer value of this Python argument will be written to the parameter without any range checking, even for negative values. - ``converter`` + *converter* Only supported by the ``object`` converter. Specifies the name of a :ref:`C "converter function" ` to use to convert this object to a native type. - ``encoding`` + *encoding* Only supported for strings. Specifies the encoding to use when converting this string from a Python str (Unicode) value into a C ``char *`` value. - ``subclass_of`` + *subclass_of* Only supported for the ``object`` converter. Requires that the Python value be a subclass of a Python type, as expressed in C. - ``type`` + *type* Only supported for the ``object`` and ``self`` converters. Specifies the C type that will be used to declare the variable. Default value is ``"PyObject *"``. - ``zeroes`` + *zeroes* Only supported for strings. If true, embedded NUL bytes (``'\\0'``) are permitted inside the value. The length of the string will be passed in to the impl function, just after the string parameter, as a parameter named ``_length``. Please note, not every possible combination of arguments will work. -Usually these arguments are implemented by specific ``PyArg_ParseTuple`` +Usually these arguments are implemented by specific :c:func:`PyArg_ParseTuple` *format units*, with specific behavior. For example, currently you cannot call ``unsigned_short`` without also specifying ``bitwise=True``. Although it's perfectly reasonable to think this would work, these semantics don't @@ -922,19 +924,19 @@ conversion functions, or types, or strings specifying an encoding. (But "legacy converters" don't support arguments. That's why we skipped them for your first function.) The argument you specified to the format unit is now an argument to the converter; this -argument is either ``converter`` (for ``O&``), ``subclass_of`` (for ``O!``), -or ``encoding`` (for all the format units that start with ``e``). +argument is either *converter* (for ``O&``), *subclass_of* (for ``O!``), +or *encoding* (for all the format units that start with ``e``). -When using ``subclass_of``, you may also want to use the other -custom argument for ``object()``: ``type``, which lets you set the type +When using *subclass_of*, you may also want to use the other +custom argument for ``object()``: *type*, which lets you set the type actually used for the parameter. For example, if you want to ensure -that the object is a subclass of ``PyUnicode_Type``, you probably want +that the object is a subclass of :c:var:`PyUnicode_Type`, you probably want to use the converter ``object(type='PyUnicodeObject *', subclass_of='&PyUnicode_Type')``. One possible problem with using Argument Clinic: it takes away some possible flexibility for the format units starting with ``e``. When writing a -``PyArg_Parse`` call by hand, you could theoretically decide at runtime what -encoding string to pass in to :c:func:`PyArg_ParseTuple`. But now this string must +:c:func:`!PyArg_Parse*` call by hand, you could theoretically decide at runtime what +encoding string to pass to that call. But now this string must be hard-coded at Argument-Clinic-preprocessing-time. This limitation is deliberate; it made supporting this format unit much easier, and may allow for future optimizations. This restriction doesn't seem unreasonable; CPython itself always passes in static @@ -987,7 +989,7 @@ expression. Currently the following are explicitly supported: * Numeric constants (integer and float) * String constants * ``True``, ``False``, and ``None`` -* Simple symbolic constants like ``sys.maxsize``, which must +* Simple symbolic constants like :py:data:`sys.maxsize`, which must start with the name of the module (In the future, this may need to get even more elaborate, @@ -1008,28 +1010,28 @@ Consider the following example: foo: Py_ssize_t = sys.maxsize - 1 -``sys.maxsize`` can have different values on different platforms. Therefore +:py:data:`sys.maxsize` can have different values on different platforms. Therefore Argument Clinic can't simply evaluate that expression locally and hard-code it in C. So it stores the default in such a way that it will get evaluated at runtime, when the user asks for the function's signature. What namespace is available when the expression is evaluated? It's evaluated in the context of the module the builtin came from. So, if your module has an -attribute called "``max_widgets``", you may simply use it: +attribute called :py:attr:`!max_widgets`, you may simply use it: .. code-block:: none foo: Py_ssize_t = max_widgets If the symbol isn't found in the current module, it fails over to looking in -``sys.modules``. That's how it can find ``sys.maxsize`` for example. (Since you -don't know in advance what modules the user will load into their interpreter, +:py:data:`sys.modules`. That's how it can find :py:data:`sys.maxsize` for example. +(Since you don't know in advance what modules the user will load into their interpreter, it's best to restrict yourself to modules that are preloaded by Python itself.) Evaluating default values only at runtime means Argument Clinic can't compute the correct equivalent C default value. So you need to tell it explicitly. When you use an expression, you must also specify the equivalent expression -in C, using the ``c_default`` parameter to the converter: +in C, using the *c_default* parameter to the converter: .. code-block:: none @@ -1095,7 +1097,7 @@ indicate an error has occurred? Normally, a function returns a valid (non-``NUL pointer for success, and ``NULL`` for failure. But if you use an integer return converter, all integers are valid. How can Argument Clinic detect an error? Its solution: each return converter implicitly looks for a special value that indicates an error. If you return -that value, and an error has been set (``PyErr_Occurred()`` returns a true +that value, and an error has been set (c:func:`PyErr_Occurred` returns a true value), then the generated code will propagate the error. Otherwise it will encode the value you return like normal. @@ -1201,9 +1203,9 @@ using a default converter. It automatically sets the ``type`` of this parameter to the "pointer to an instance" you specified when you declared the type. However, you can override Argument Clinic's converter and specify one yourself. -Just add your own ``self`` parameter as the first parameter in a +Just add your own *self* parameter as the first parameter in a block, and ensure that its converter is an instance of -``self_converter`` or a subclass thereof. +:class:`!self_converter` or a subclass thereof. What's the point? This lets you override the type of ``self``, or give it a different default name. @@ -1211,7 +1213,7 @@ or give it a different default name. How do you specify the custom type you want to cast ``self`` to? If you only have one or two functions with the same type for ``self``, you can directly use Argument Clinic's existing ``self`` converter, -passing in the type you want to use as the ``type`` parameter:: +passing in the type you want to use as the *type* parameter:: /*[clinic input] @@ -1226,7 +1228,7 @@ passing in the type you want to use as the ``type`` parameter:: On the other hand, if you have a lot of functions that will use the same type for ``self``, it's best to create your own converter, subclassing -``self_converter`` but overwriting the ``type`` member:: +:class:`!self_converter` but overwriting the :py:attr:`!type` member:: /*[python input] class PicklerObject_converter(self_converter): @@ -1254,8 +1256,8 @@ module level state. Use :c:func:`PyType_FromModuleAndSpec` to associate a new heap type with a module. You can now use :c:func:`PyType_GetModuleState` on the defining class to fetch the module state, for example from a module method. -Example from ``Modules/zlibmodule.c``. First, ``defining_class`` is added to -the clinic input:: +Example from :source:`Modules/zlibmodule.c`. +First, ``defining_class`` is added to the clinic input:: /*[clinic input] zlib.Compress.compress @@ -1285,16 +1287,17 @@ module state:: Each method may only have one argument using this converter, and it must appear after ``self``, or, if ``self`` is not used, as the first argument. The argument will be of type ``PyTypeObject *``. The argument will not appear in the -``__text_signature__``. +:py:attr:`!__text_signature__`. -The ``defining_class`` converter is not compatible with ``__init__`` and ``__new__`` -methods, which cannot use the ``METH_METHOD`` convention. +The ``defining_class`` converter is not compatible with :py:meth:`!__init__` +and :py:meth:`!__new__` methods, which cannot use the :c:macro:`METH_METHOD` +convention. It is not possible to use ``defining_class`` with slot methods. In order to fetch the module state from such methods, use :c:func:`PyType_GetModuleByDef` to look up the module and then :c:func:`PyModule_GetState` to fetch the module state. Example from the ``setattro`` slot method in -``Modules/_threadmodule.c``:: +:source:`Modules/_threadmodule.c`:: static int local_setattro(localobject *self, PyObject *name, PyObject *v) @@ -1312,7 +1315,7 @@ How to write a custom converter ------------------------------- As we hinted at in the previous section... you can write your own converters! -A converter is simply a Python class that inherits from ``CConverter``. +A converter is simply a Python class that inherits from :py:class:`!CConverter`. The main purpose of a custom converter is if you have a parameter using the ``O&`` format unit?parsing this parameter means calling a :c:func:`PyArg_ParseTuple` "converter function". @@ -1323,61 +1326,74 @@ will be automatically registered with Argument Clinic; its name will be the name of your class with the ``_converter`` suffix stripped off. (This is accomplished with a metaclass.) -You shouldn't subclass ``CConverter.__init__``. Instead, you should -write a ``converter_init()`` function. ``converter_init()`` -always accepts a ``self`` parameter; after that, all additional +You shouldn't subclass :py:meth:`!CConverter.__init__`. Instead, you should +write a :py:meth:`!converter_init` function. :py:meth:`!converter_init` +always accepts a *self* parameter; after that, all additional parameters *must* be keyword-only. Any arguments passed in to the converter in Argument Clinic will be passed along to your -``converter_init()``. +:py:meth:`!converter_init`. -There are some additional members of ``CConverter`` you may wish +There are some additional members of :py:class:`!CConverter` you may wish to specify in your subclass. Here's the current list: -``type`` - The C type to use for this variable. - ``type`` should be a Python string specifying the type, e.g. ``int``. - If this is a pointer type, the type string should end with ``' *'``. +.. module:: clinic -``default`` - The Python default value for this parameter, as a Python value. - Or the magic value ``unspecified`` if there is no default. +.. class:: CConverter -``py_default`` - ``default`` as it should appear in Python code, - as a string. - Or ``None`` if there is no default. + .. attribute:: type -``c_default`` - ``default`` as it should appear in C code, - as a string. - Or ``None`` if there is no default. + The C type to use for this variable. + :attr:`!type` should be a Python string specifying the type, + e.g. ``'int'``. + If this is a pointer type, the type string should end with ``' *'``. -``c_ignored_default`` - The default value used to initialize the C variable when - there is no default, but not specifying a default may - result in an "uninitialized variable" warning. This can - easily happen when using option groups?although - properly written code will never actually use this value, - the variable does get passed in to the impl, and the - C compiler will complain about the "use" of the - uninitialized value. This value should always be a - non-empty string. + .. attribute:: default -``converter`` - The name of the C converter function, as a string. + The Python default value for this parameter, as a Python value. + Or the magic value ``unspecified`` if there is no default. -``impl_by_reference`` - A boolean value. If true, - Argument Clinic will add a ``&`` in front of the name of - the variable when passing it into the impl function. + .. attribute:: py_default -``parse_by_reference`` - A boolean value. If true, - Argument Clinic will add a ``&`` in front of the name of - the variable when passing it into :c:func:`PyArg_ParseTuple`. + :attr:`!default` as it should appear in Python code, + as a string. + Or ``None`` if there is no default. + .. attribute:: c_default -Here's the simplest example of a custom converter, from ``Modules/zlibmodule.c``:: + :attr:`!default` as it should appear in C code, + as a string. + Or ``None`` if there is no default. + + .. attribute:: c_ignored_default + + The default value used to initialize the C variable when + there is no default, but not specifying a default may + result in an "uninitialized variable" warning. This can + easily happen when using option groups?although + properly written code will never actually use this value, + the variable does get passed in to the impl, and the + C compiler will complain about the "use" of the + uninitialized value. This value should always be a + non-empty string. + + .. attribute:: converter + + The name of the C converter function, as a string. + + .. attribute:: impl_by_reference + + A boolean value. If true, + Argument Clinic will add a ``&`` in front of the name of + the variable when passing it into the impl function. + + .. attribute:: parse_by_reference + + A boolean value. If true, + Argument Clinic will add a ``&`` in front of the name of + the variable when passing it into :c:func:`PyArg_ParseTuple`. + + +Here's the simplest example of a custom converter, from :source:`Modules/zlibmodule.c`:: /*[python input] @@ -1397,7 +1413,7 @@ automatically support default values. More sophisticated custom converters can insert custom C code to handle initialization and cleanup. You can see more examples of custom converters in the CPython -source tree; grep the C files for the string ``CConverter``. +source tree; grep the C files for the string :py:class:`!CConverter`. How to write a custom return converter @@ -1407,18 +1423,18 @@ Writing a custom return converter is much like writing a custom converter. Except it's somewhat simpler, because return converters are themselves much simpler. -Return converters must subclass ``CReturnConverter``. +Return converters must subclass :py:class:`!CReturnConverter`. There are no examples yet of custom return converters, because they are not widely used yet. If you wish to -write your own return converter, please read ``Tools/clinic/clinic.py``, -specifically the implementation of ``CReturnConverter`` and +write your own return converter, please read :source:`Tools/clinic/clinic.py`, +specifically the implementation of :py:class:`!CReturnConverter` and all its subclasses. How to convert ``METH_O`` and ``METH_NOARGS`` functions ------------------------------------------------------- -To convert a function using ``METH_O``, make sure the function's +To convert a function using :c:macro:`METH_O`, make sure the function's single argument is using the ``object`` converter, and mark the arguments as positional-only:: @@ -1430,24 +1446,25 @@ arguments as positional-only:: [clinic start generated code]*/ -To convert a function using ``METH_NOARGS``, just don't specify +To convert a function using :c:macro:`METH_NOARGS`, just don't specify any arguments. You can still use a self converter, a return converter, and specify -a ``type`` argument to the object converter for ``METH_O``. +a *type* argument to the object converter for :c:macro:`METH_O`. How to convert ``tp_new`` and ``tp_init`` functions --------------------------------------------------- -You can convert ``tp_new`` and ``tp_init`` functions. Just name -them ``__new__`` or ``__init__`` as appropriate. Notes: +You can convert :c:member:`~PyTypeObject.tp_new` and +:c:member:`~PyTypeObject.tp_init` functions. +Just name them ``__new__`` or ``__init__`` as appropriate. Notes: * The function name generated for ``__new__`` doesn't end in ``__new__`` like it would by default. It's just the name of the class, converted into a valid C identifier. -* No ``PyMethodDef`` ``#define`` is generated for these functions. +* No :c:type:`PyMethodDef` ``#define`` is generated for these functions. * ``__init__`` functions return ``int``, not ``PyObject *``. @@ -1482,7 +1499,7 @@ Let's start with defining some terminology: *field* A field, in this context, is a subsection of Clinic's output. - For example, the ``#define`` for the ``PyMethodDef`` structure + For example, the ``#define`` for the :c:type:`PyMethodDef` structure is a field, called ``methoddef_define``. Clinic has seven different fields it can output per function definition: @@ -1526,8 +1543,8 @@ Let's start with defining some terminology: The filename chosen for the file is ``{basename}.clinic{extension}``, where ``basename`` and ``extension`` were assigned the output from ``os.path.splitext()`` run on the current file. (Example: - the ``file`` destination for ``_pickle.c`` would be written to - ``_pickle.clinic.c``.) + the ``file`` destination for :file:`_pickle.c` would be written to + :file:`_pickle.clinic.c`.) **Important: When using a** ``file`` **destination, you** *must check in* **the generated file!** @@ -1780,7 +1797,7 @@ like so:: } #endif /* HAVE_FUNCTIONNAME */ -Then, remove those three lines from the ``PyMethodDef`` structure, +Then, remove those three lines from the :c:type:`PyMethodDef` structure, replacing them with the macro Argument Clinic generated: .. code-block:: none @@ -1821,7 +1838,7 @@ This may mean that you get a complaint from Argument Clinic: When this happens, just open your file, find the ``dump buffer`` block that Argument Clinic added to your file (it'll be at the very bottom), then -move it above the ``PyMethodDef`` structure where that macro is used. +move it above the :c:type:`PyMethodDef` structure where that macro is used. How to use Argument Clinic in Python files From webhook-mailer at python.org Mon Jul 24 11:45:17 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 24 Jul 2023 15:45:17 -0000 Subject: [Python-checkins] [3.11] Docs: Add missing markup to Argument Clinic docs (#106876) (#107182) Message-ID: https://github.com/python/cpython/commit/604eb040598ea4b05e15ad98e0a1781babcad775 commit: 604eb040598ea4b05e15ad98e0a1781babcad775 branch: 3.11 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-24T15:45:13Z summary: [3.11] Docs: Add missing markup to Argument Clinic docs (#106876) (#107182) (cherry picked from commit ff5f94b72c8aad8e45c397c263dbe7f19221735f) Co-authored-by: Ezio Melotti Co-authored-by: Serhiy Storchaka files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 4a83f6dde3a51..2d332f03d91a2 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -96,7 +96,8 @@ things with all the information you give it. Basic concepts and usage ------------------------ -Argument Clinic ships with CPython; you'll find it in ``Tools/clinic/clinic.py``. +Argument Clinic ships with CPython; you'll find it in +:source:`Tools/clinic/clinic.py`. If you run that script, specifying a C file as an argument: .. code-block:: shell-session @@ -178,9 +179,10 @@ Let's dive in! 1. Find a Python builtin that calls either :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords`, and hasn't been converted to work with Argument Clinic yet. - For my example I'm using ``_pickle.Pickler.dump()``. + For my example I'm using + :py:meth:`_pickle.Pickler.dump `. -2. If the call to the ``PyArg_Parse`` function uses any of the +2. If the call to the :c:func:`!PyArg_Parse*` function uses any of the following format units: .. code-block:: none @@ -197,10 +199,10 @@ Let's dive in! support all of these scenarios. But these are advanced topics?let's do something simpler for your first function. - Also, if the function has multiple calls to :c:func:`PyArg_ParseTuple` + Also, if the function has multiple calls to :c:func:`!PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different types for the same argument, or if the function uses something besides - PyArg_Parse functions to parse its arguments, it probably + :c:func:`!PyArg_Parse*` functions to parse its arguments, it probably isn't suitable for conversion to Argument Clinic. Argument Clinic doesn't support generic functions or polymorphic parameters. @@ -217,7 +219,7 @@ Let's dive in! If the old docstring had a first line that looked like a function signature, throw that line away. (The docstring doesn't need it - anymore?when you use ``help()`` on your builtin in the future, + anymore?when you use :py:func:`help` on your builtin in the future, the first line will be built automatically based on the function's signature.) @@ -264,7 +266,7 @@ Let's dive in! When you declare a class, you must also specify two aspects of its type in C: the type declaration you'd use for a pointer to an instance of - this class, and a pointer to the :c:type:`PyTypeObject` for this class. + this class, and a pointer to the :c:type:`!PyTypeObject` for this class. Sample:: @@ -313,10 +315,10 @@ Let's dive in! Clinic easier. For each parameter, copy the "format unit" for that - parameter from the ``PyArg_Parse()`` format argument and + parameter from the :c:func:`PyArg_Parse` format argument and specify *that* as its converter, as a quoted string. ("format unit" is the formal name for the one-to-three - character substring of the ``format`` parameter that tells + character substring of the *format* parameter that tells the argument parsing function what the type of the variable is and how to convert it. For more on format units please see :ref:`arg-parsing`.) @@ -349,7 +351,7 @@ Let's dive in! itself before the first keyword-only argument, indented the same as the parameter lines. - (``_pickle.Pickler.dump`` has neither, so our sample is unchanged.) + (:py:meth:`!_pickle.Pickler.dump` has neither, so our sample is unchanged.) 10. If the existing C function calls :c:func:`PyArg_ParseTuple` @@ -410,7 +412,7 @@ Let's dive in! 12. Save and close the file, then run ``Tools/clinic/clinic.py`` on it. With luck everything worked---your block now has output, and - a ``.c.h`` file has been generated! Reopen the file in your + a :file:`.c.h` file has been generated! Reopen the file in your text editor to see:: /*[clinic input] @@ -431,8 +433,8 @@ Let's dive in! it found an error in your input. Keep fixing your errors and retrying until Argument Clinic processes your file without complaint. - For readability, most of the glue code has been generated to a ``.c.h`` - file. You'll need to include that in your original ``.c`` file, + For readability, most of the glue code has been generated to a :file:`.c.h` + file. You'll need to include that in your original :file:`.c` file, typically right after the clinic module block:: #include "clinic/_pickle.c.h" @@ -446,8 +448,8 @@ Let's dive in! ensure that the code generated by Argument Clinic calls the *exact* same function. - Second, the format string passed in to :c:func:`PyArg_ParseTuple` or - :c:func:`PyArg_ParseTupleAndKeywords` should be *exactly* the same + Second, the format string passed in to :c:func:`!PyArg_ParseTuple` or + :c:func:`!PyArg_ParseTupleAndKeywords` should be *exactly* the same as the hand-written one in the existing function, up to the colon or semi-colon. @@ -469,7 +471,7 @@ Let's dive in! {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, This static structure should be *exactly* the same as the existing static - :c:type:`PyMethodDef` structure for this builtin. + :c:type:`!PyMethodDef` structure for this builtin. If any of these items differ in *any way*, adjust your Argument Clinic function specification and rerun @@ -539,14 +541,14 @@ Let's dive in! ... 15. Remember the macro with the :c:type:`PyMethodDef` structure for this - function? Find the existing :c:type:`PyMethodDef` structure for this + function? Find the existing :c:type:`!PyMethodDef` structure for this function and replace it with a reference to the macro. (If the builtin is at module scope, this will probably be very near the end of the file; if the builtin is a class method, this will probably be below but relatively near to the implementation.) Note that the body of the macro contains a trailing comma. So when you - replace the existing static :c:type:`PyMethodDef` structure with the macro, + replace the existing static :c:type:`!PyMethodDef` structure with the macro, *don't* add a comma to the end. Sample:: @@ -562,7 +564,7 @@ Let's dive in! This change should not introduce any new compile-time warnings or errors, and there should be no externally visible change to Python's behavior. - Well, except for one difference: ``inspect.signature()`` run on your function + Well, except for one difference: :py:func:`inspect.signature` run on your function should now provide a valid signature! Congratulations, you've ported your first function to work with Argument Clinic! @@ -586,15 +588,15 @@ Argument Clinic will use that function name for the base (generated) function, then add ``"_impl"`` to the end and use that for the name of the impl function. For example, if we wanted to rename the C function names generated for -``pickle.Pickler.dump``, it'd look like this:: +:py:meth:`pickle.Pickler.dump`, it'd look like this:: /*[clinic input] pickle.Pickler.dump as pickler_dumper ... -The base function would now be named ``pickler_dumper()``, -and the impl function would now be named ``pickler_dumper_impl()``. +The base function would now be named :c:func:`!pickler_dumper`, +and the impl function would now be named :c:func:`!pickler_dumper_impl`. Similarly, you may have a problem where you want to give a parameter @@ -612,9 +614,9 @@ using the same ``"as"`` syntax:: fix_imports: bool = True Here, the name used in Python (in the signature and the ``keywords`` -array) would be ``file``, but the C variable would be named ``file_obj``. +array) would be *file*, but the C variable would be named ``file_obj``. -You can use this to rename the ``self`` parameter too! +You can use this to rename the *self* parameter too! How to convert functions using ``PyArg_UnpackTuple`` @@ -622,7 +624,7 @@ How to convert functions using ``PyArg_UnpackTuple`` To convert a function parsing its arguments with :c:func:`PyArg_UnpackTuple`, simply write out all the arguments, specifying each as an ``object``. You -may specify the ``type`` argument to cast the type as appropriate. All +may specify the *type* argument to cast the type as appropriate. All arguments should be marked positional-only (add a ``/`` on a line by itself after the last argument). @@ -641,16 +643,16 @@ keyword-only arguments.) This approach was used to simulate optional arguments back before :c:func:`PyArg_ParseTupleAndKeywords` was created. While functions using this approach can often be converted to -use :c:func:`PyArg_ParseTupleAndKeywords`, optional arguments, and default values, +use :c:func:`!PyArg_ParseTupleAndKeywords`, optional arguments, and default values, it's not always possible. Some of these legacy functions have -behaviors :c:func:`PyArg_ParseTupleAndKeywords` doesn't directly support. -The most obvious example is the builtin function ``range()``, which has +behaviors :c:func:`!PyArg_ParseTupleAndKeywords` doesn't directly support. +The most obvious example is the builtin function :py:func:`range`, which has an optional argument on the *left* side of its required argument! -Another example is ``curses.window.addch()``, which has a group of two +Another example is :py:meth:`curses.window.addch`, which has a group of two arguments that must always be specified together. (The arguments are -called ``x`` and ``y``; if you call the function passing in ``x``, -you must also pass in ``y``?and if you don't pass in ``x`` you may not -pass in ``y`` either.) +called *x* and *y*; if you call the function passing in *x*, +you must also pass in *y* ? and if you don't pass in *x* you may not +pass in *y* either.) In any case, the goal of Argument Clinic is to support argument parsing for all existing CPython builtins without changing their semantics. @@ -671,7 +673,7 @@ can *only* be used with positional-only parameters. To specify an optional group, add a ``[`` on a line by itself before the parameters you wish to group together, and a ``]`` on a line by itself -after these parameters. As an example, here's how ``curses.window.addch`` +after these parameters. As an example, here's how :py:meth:`curses.window.addch` uses optional groups to make the first two parameters and the last parameter optional:: @@ -757,14 +759,14 @@ the same converters. All arguments to Argument Clinic converters are keyword-only. All Argument Clinic converters accept the following arguments: - ``c_default`` + *c_default* The default value for this parameter when defined in C. Specifically, this will be the initializer for the variable declared in the "parse function". See :ref:`the section on default values ` for how to use this. Specified as a string. - ``annotation`` + *annotation* The annotation value for this parameter. Not currently supported, because :pep:`8` mandates that the Python library may not use annotations. @@ -772,7 +774,7 @@ All Argument Clinic converters accept the following arguments: In addition, some converters accept additional arguments. Here is a list of these arguments, along with their meanings: - ``accept`` + *accept* A set of Python types (and possibly pseudo-types); this restricts the allowable Python argument to values of these types. (This is not a general-purpose facility; as a rule it only supports @@ -780,38 +782,38 @@ of these arguments, along with their meanings: To accept ``None``, add ``NoneType`` to this set. - ``bitwise`` + *bitwise* Only supported for unsigned integers. The native integer value of this Python argument will be written to the parameter without any range checking, even for negative values. - ``converter`` + *converter* Only supported by the ``object`` converter. Specifies the name of a :ref:`C "converter function" ` to use to convert this object to a native type. - ``encoding`` + *encoding* Only supported for strings. Specifies the encoding to use when converting this string from a Python str (Unicode) value into a C ``char *`` value. - ``subclass_of`` + *subclass_of* Only supported for the ``object`` converter. Requires that the Python value be a subclass of a Python type, as expressed in C. - ``type`` + *type* Only supported for the ``object`` and ``self`` converters. Specifies the C type that will be used to declare the variable. Default value is ``"PyObject *"``. - ``zeroes`` + *zeroes* Only supported for strings. If true, embedded NUL bytes (``'\\0'``) are permitted inside the value. The length of the string will be passed in to the impl function, just after the string parameter, as a parameter named ``_length``. Please note, not every possible combination of arguments will work. -Usually these arguments are implemented by specific ``PyArg_ParseTuple`` +Usually these arguments are implemented by specific :c:func:`PyArg_ParseTuple` *format units*, with specific behavior. For example, currently you cannot call ``unsigned_short`` without also specifying ``bitwise=True``. Although it's perfectly reasonable to think this would work, these semantics don't @@ -911,19 +913,19 @@ conversion functions, or types, or strings specifying an encoding. (But "legacy converters" don't support arguments. That's why we skipped them for your first function.) The argument you specified to the format unit is now an argument to the converter; this -argument is either ``converter`` (for ``O&``), ``subclass_of`` (for ``O!``), -or ``encoding`` (for all the format units that start with ``e``). +argument is either *converter* (for ``O&``), *subclass_of* (for ``O!``), +or *encoding* (for all the format units that start with ``e``). -When using ``subclass_of``, you may also want to use the other -custom argument for ``object()``: ``type``, which lets you set the type +When using *subclass_of*, you may also want to use the other +custom argument for ``object()``: *type*, which lets you set the type actually used for the parameter. For example, if you want to ensure -that the object is a subclass of ``PyUnicode_Type``, you probably want +that the object is a subclass of :c:var:`PyUnicode_Type`, you probably want to use the converter ``object(type='PyUnicodeObject *', subclass_of='&PyUnicode_Type')``. One possible problem with using Argument Clinic: it takes away some possible flexibility for the format units starting with ``e``. When writing a -``PyArg_Parse`` call by hand, you could theoretically decide at runtime what -encoding string to pass in to :c:func:`PyArg_ParseTuple`. But now this string must +:c:func:`!PyArg_Parse*` call by hand, you could theoretically decide at runtime what +encoding string to pass to that call. But now this string must be hard-coded at Argument-Clinic-preprocessing-time. This limitation is deliberate; it made supporting this format unit much easier, and may allow for future optimizations. This restriction doesn't seem unreasonable; CPython itself always passes in static @@ -976,7 +978,7 @@ expression. Currently the following are explicitly supported: * Numeric constants (integer and float) * String constants * ``True``, ``False``, and ``None`` -* Simple symbolic constants like ``sys.maxsize``, which must +* Simple symbolic constants like :py:data:`sys.maxsize`, which must start with the name of the module (In the future, this may need to get even more elaborate, @@ -997,28 +999,28 @@ Consider the following example: foo: Py_ssize_t = sys.maxsize - 1 -``sys.maxsize`` can have different values on different platforms. Therefore +:py:data:`sys.maxsize` can have different values on different platforms. Therefore Argument Clinic can't simply evaluate that expression locally and hard-code it in C. So it stores the default in such a way that it will get evaluated at runtime, when the user asks for the function's signature. What namespace is available when the expression is evaluated? It's evaluated in the context of the module the builtin came from. So, if your module has an -attribute called "``max_widgets``", you may simply use it: +attribute called :py:attr:`!max_widgets`, you may simply use it: .. code-block:: none foo: Py_ssize_t = max_widgets If the symbol isn't found in the current module, it fails over to looking in -``sys.modules``. That's how it can find ``sys.maxsize`` for example. (Since you -don't know in advance what modules the user will load into their interpreter, +:py:data:`sys.modules`. That's how it can find :py:data:`sys.maxsize` for example. +(Since you don't know in advance what modules the user will load into their interpreter, it's best to restrict yourself to modules that are preloaded by Python itself.) Evaluating default values only at runtime means Argument Clinic can't compute the correct equivalent C default value. So you need to tell it explicitly. When you use an expression, you must also specify the equivalent expression -in C, using the ``c_default`` parameter to the converter: +in C, using the *c_default* parameter to the converter: .. code-block:: none @@ -1084,7 +1086,7 @@ indicate an error has occurred? Normally, a function returns a valid (non-``NUL pointer for success, and ``NULL`` for failure. But if you use an integer return converter, all integers are valid. How can Argument Clinic detect an error? Its solution: each return converter implicitly looks for a special value that indicates an error. If you return -that value, and an error has been set (``PyErr_Occurred()`` returns a true +that value, and an error has been set (c:func:`PyErr_Occurred` returns a true value), then the generated code will propagate the error. Otherwise it will encode the value you return like normal. @@ -1195,9 +1197,9 @@ using a default converter. It automatically sets the ``type`` of this parameter to the "pointer to an instance" you specified when you declared the type. However, you can override Argument Clinic's converter and specify one yourself. -Just add your own ``self`` parameter as the first parameter in a +Just add your own *self* parameter as the first parameter in a block, and ensure that its converter is an instance of -``self_converter`` or a subclass thereof. +:class:`!self_converter` or a subclass thereof. What's the point? This lets you override the type of ``self``, or give it a different default name. @@ -1205,7 +1207,7 @@ or give it a different default name. How do you specify the custom type you want to cast ``self`` to? If you only have one or two functions with the same type for ``self``, you can directly use Argument Clinic's existing ``self`` converter, -passing in the type you want to use as the ``type`` parameter:: +passing in the type you want to use as the *type* parameter:: /*[clinic input] @@ -1220,7 +1222,7 @@ passing in the type you want to use as the ``type`` parameter:: On the other hand, if you have a lot of functions that will use the same type for ``self``, it's best to create your own converter, subclassing -``self_converter`` but overwriting the ``type`` member:: +:class:`!self_converter` but overwriting the :py:attr:`!type` member:: /*[python input] class PicklerObject_converter(self_converter): @@ -1248,8 +1250,8 @@ module level state. Use :c:func:`PyType_FromModuleAndSpec` to associate a new heap type with a module. You can now use :c:func:`PyType_GetModuleState` on the defining class to fetch the module state, for example from a module method. -Example from ``Modules/zlibmodule.c``. First, ``defining_class`` is added to -the clinic input:: +Example from :source:`Modules/zlibmodule.c`. +First, ``defining_class`` is added to the clinic input:: /*[clinic input] zlib.Compress.compress @@ -1279,16 +1281,17 @@ module state:: Each method may only have one argument using this converter, and it must appear after ``self``, or, if ``self`` is not used, as the first argument. The argument will be of type ``PyTypeObject *``. The argument will not appear in the -``__text_signature__``. +:py:attr:`!__text_signature__`. -The ``defining_class`` converter is not compatible with ``__init__`` and ``__new__`` -methods, which cannot use the ``METH_METHOD`` convention. +The ``defining_class`` converter is not compatible with :py:meth:`!__init__` +and :py:meth:`!__new__` methods, which cannot use the :c:macro:`METH_METHOD` +convention. It is not possible to use ``defining_class`` with slot methods. In order to fetch the module state from such methods, use :c:func:`PyType_GetModuleByDef` to look up the module and then :c:func:`PyModule_GetState` to fetch the module state. Example from the ``setattro`` slot method in -``Modules/_threadmodule.c``:: +:source:`Modules/_threadmodule.c`:: static int local_setattro(localobject *self, PyObject *name, PyObject *v) @@ -1306,7 +1309,7 @@ How to write a custom converter ------------------------------- As we hinted at in the previous section... you can write your own converters! -A converter is simply a Python class that inherits from ``CConverter``. +A converter is simply a Python class that inherits from :py:class:`!CConverter`. The main purpose of a custom converter is if you have a parameter using the ``O&`` format unit?parsing this parameter means calling a :c:func:`PyArg_ParseTuple` "converter function". @@ -1317,61 +1320,74 @@ will be automatically registered with Argument Clinic; its name will be the name of your class with the ``_converter`` suffix stripped off. (This is accomplished with a metaclass.) -You shouldn't subclass ``CConverter.__init__``. Instead, you should -write a ``converter_init()`` function. ``converter_init()`` -always accepts a ``self`` parameter; after that, all additional +You shouldn't subclass :py:meth:`!CConverter.__init__`. Instead, you should +write a :py:meth:`!converter_init` function. :py:meth:`!converter_init` +always accepts a *self* parameter; after that, all additional parameters *must* be keyword-only. Any arguments passed in to the converter in Argument Clinic will be passed along to your -``converter_init()``. +:py:meth:`!converter_init`. -There are some additional members of ``CConverter`` you may wish +There are some additional members of :py:class:`!CConverter` you may wish to specify in your subclass. Here's the current list: -``type`` - The C type to use for this variable. - ``type`` should be a Python string specifying the type, e.g. ``int``. - If this is a pointer type, the type string should end with ``' *'``. +.. module:: clinic -``default`` - The Python default value for this parameter, as a Python value. - Or the magic value ``unspecified`` if there is no default. +.. class:: CConverter -``py_default`` - ``default`` as it should appear in Python code, - as a string. - Or ``None`` if there is no default. + .. attribute:: type -``c_default`` - ``default`` as it should appear in C code, - as a string. - Or ``None`` if there is no default. + The C type to use for this variable. + :attr:`!type` should be a Python string specifying the type, + e.g. ``'int'``. + If this is a pointer type, the type string should end with ``' *'``. -``c_ignored_default`` - The default value used to initialize the C variable when - there is no default, but not specifying a default may - result in an "uninitialized variable" warning. This can - easily happen when using option groups?although - properly written code will never actually use this value, - the variable does get passed in to the impl, and the - C compiler will complain about the "use" of the - uninitialized value. This value should always be a - non-empty string. + .. attribute:: default -``converter`` - The name of the C converter function, as a string. + The Python default value for this parameter, as a Python value. + Or the magic value ``unspecified`` if there is no default. -``impl_by_reference`` - A boolean value. If true, - Argument Clinic will add a ``&`` in front of the name of - the variable when passing it into the impl function. + .. attribute:: py_default -``parse_by_reference`` - A boolean value. If true, - Argument Clinic will add a ``&`` in front of the name of - the variable when passing it into :c:func:`PyArg_ParseTuple`. + :attr:`!default` as it should appear in Python code, + as a string. + Or ``None`` if there is no default. + .. attribute:: c_default -Here's the simplest example of a custom converter, from ``Modules/zlibmodule.c``:: + :attr:`!default` as it should appear in C code, + as a string. + Or ``None`` if there is no default. + + .. attribute:: c_ignored_default + + The default value used to initialize the C variable when + there is no default, but not specifying a default may + result in an "uninitialized variable" warning. This can + easily happen when using option groups?although + properly written code will never actually use this value, + the variable does get passed in to the impl, and the + C compiler will complain about the "use" of the + uninitialized value. This value should always be a + non-empty string. + + .. attribute:: converter + + The name of the C converter function, as a string. + + .. attribute:: impl_by_reference + + A boolean value. If true, + Argument Clinic will add a ``&`` in front of the name of + the variable when passing it into the impl function. + + .. attribute:: parse_by_reference + + A boolean value. If true, + Argument Clinic will add a ``&`` in front of the name of + the variable when passing it into :c:func:`PyArg_ParseTuple`. + + +Here's the simplest example of a custom converter, from :source:`Modules/zlibmodule.c`:: /*[python input] @@ -1391,7 +1407,7 @@ automatically support default values. More sophisticated custom converters can insert custom C code to handle initialization and cleanup. You can see more examples of custom converters in the CPython -source tree; grep the C files for the string ``CConverter``. +source tree; grep the C files for the string :py:class:`!CConverter`. How to write a custom return converter @@ -1401,18 +1417,18 @@ Writing a custom return converter is much like writing a custom converter. Except it's somewhat simpler, because return converters are themselves much simpler. -Return converters must subclass ``CReturnConverter``. +Return converters must subclass :py:class:`!CReturnConverter`. There are no examples yet of custom return converters, because they are not widely used yet. If you wish to -write your own return converter, please read ``Tools/clinic/clinic.py``, -specifically the implementation of ``CReturnConverter`` and +write your own return converter, please read :source:`Tools/clinic/clinic.py`, +specifically the implementation of :py:class:`!CReturnConverter` and all its subclasses. How to convert ``METH_O`` and ``METH_NOARGS`` functions ------------------------------------------------------- -To convert a function using ``METH_O``, make sure the function's +To convert a function using :c:macro:`METH_O`, make sure the function's single argument is using the ``object`` converter, and mark the arguments as positional-only:: @@ -1424,24 +1440,25 @@ arguments as positional-only:: [clinic start generated code]*/ -To convert a function using ``METH_NOARGS``, just don't specify +To convert a function using :c:macro:`METH_NOARGS`, just don't specify any arguments. You can still use a self converter, a return converter, and specify -a ``type`` argument to the object converter for ``METH_O``. +a *type* argument to the object converter for :c:macro:`METH_O`. How to convert ``tp_new`` and ``tp_init`` functions --------------------------------------------------- -You can convert ``tp_new`` and ``tp_init`` functions. Just name -them ``__new__`` or ``__init__`` as appropriate. Notes: +You can convert :c:member:`~PyTypeObject.tp_new` and +:c:member:`~PyTypeObject.tp_init` functions. +Just name them ``__new__`` or ``__init__`` as appropriate. Notes: * The function name generated for ``__new__`` doesn't end in ``__new__`` like it would by default. It's just the name of the class, converted into a valid C identifier. -* No ``PyMethodDef`` ``#define`` is generated for these functions. +* No :c:type:`PyMethodDef` ``#define`` is generated for these functions. * ``__init__`` functions return ``int``, not ``PyObject *``. @@ -1476,7 +1493,7 @@ Let's start with defining some terminology: *field* A field, in this context, is a subsection of Clinic's output. - For example, the ``#define`` for the ``PyMethodDef`` structure + For example, the ``#define`` for the :c:type:`PyMethodDef` structure is a field, called ``methoddef_define``. Clinic has seven different fields it can output per function definition: @@ -1520,8 +1537,8 @@ Let's start with defining some terminology: The filename chosen for the file is ``{basename}.clinic{extension}``, where ``basename`` and ``extension`` were assigned the output from ``os.path.splitext()`` run on the current file. (Example: - the ``file`` destination for ``_pickle.c`` would be written to - ``_pickle.clinic.c``.) + the ``file`` destination for :file:`_pickle.c` would be written to + :file:`_pickle.clinic.c`.) **Important: When using a** ``file`` **destination, you** *must check in* **the generated file!** @@ -1774,7 +1791,7 @@ like so:: } #endif /* HAVE_FUNCTIONNAME */ -Then, remove those three lines from the ``PyMethodDef`` structure, +Then, remove those three lines from the :c:type:`PyMethodDef` structure, replacing them with the macro Argument Clinic generated: .. code-block:: none @@ -1815,7 +1832,7 @@ This may mean that you get a complaint from Argument Clinic: When this happens, just open your file, find the ``dump buffer`` block that Argument Clinic added to your file (it'll be at the very bottom), then -move it above the ``PyMethodDef`` structure where that macro is used. +move it above the :c:type:`PyMethodDef` structure where that macro is used. How to use Argument Clinic in Python files From webhook-mailer at python.org Mon Jul 24 12:30:27 2023 From: webhook-mailer at python.org (hugovk) Date: Mon, 24 Jul 2023 16:30:27 -0000 Subject: [Python-checkins] [3.12] gh-107017: Change Chapter Strings to Texts in the Introduction chapter. (GH-107104) (#107167) Message-ID: https://github.com/python/cpython/commit/beb5514826e5e1fb2ec2cd1ad137b84091399f7a commit: beb5514826e5e1fb2ec2cd1ad137b84091399f7a branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-24T16:30:22Z summary: [3.12] gh-107017: Change Chapter Strings to Texts in the Introduction chapter. (GH-107104) (#107167) Co-authored-by: TommyUnreal <45427816+TommyUnreal at users.noreply.github.com> Co-authored-by: Hugo van Kemenade files: M Doc/tutorial/introduction.rst diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index 4b886f604aca2..0fc75c7d7532e 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -138,16 +138,25 @@ and uses the ``j`` or ``J`` suffix to indicate the imaginary part .. _tut-strings: -Strings -------- +Text +---- -Besides numbers, Python can also manipulate strings, which can be expressed -in several ways. They can be enclosed in single quotes (``'...'``) or -double quotes (``"..."``) with the same result [#]_. ``\`` can be used -to escape quotes:: +Python can manipulate text (represented by type :class:`str`, so-called +"strings") as well as numbers. This includes characters "``!``", words +"``rabbit``", names "``Paris``", sentences "``Got your back.``", etc. +"``Yay! :)``". They can be enclosed in single quotes (``'...'``) or double +quotes (``"..."``) with the same result [#]_. >>> 'spam eggs' # single quotes 'spam eggs' + >>> "Paris rabbit got your back :)! Yay!" # double quotes + 'Paris rabbit got your back :)! Yay!' + >>> '1975' # digits and numerals enclosed in quotes are also strings + '1975' + +To quote a quote, we need to "escape" it, by preceding it with ``\``. +Alternatively, we can use the other type of quotation marks:: + >>> 'doesn\'t' # use \' to escape the single quote... "doesn't" >>> "doesn't" # ...or use double quotes instead @@ -159,23 +168,14 @@ to escape quotes:: >>> '"Isn\'t," they said.' '"Isn\'t," they said.' -In the interactive interpreter, the output string is enclosed in quotes and -special characters are escaped with backslashes. While this might sometimes -look different from the input (the enclosing quotes could change), the two -strings are equivalent. The string is enclosed in double quotes if -the string contains a single quote and no double quotes, otherwise it is -enclosed in single quotes. The :func:`print` function produces a more -readable output, by omitting the enclosing quotes and by printing escaped -and special characters:: +In the Python shell, the string definition and output string can look +different. The :func:`print` function produces a more readable output, by +omitting the enclosing quotes and by printing escaped and special characters:: - >>> '"Isn\'t," they said.' - '"Isn\'t," they said.' - >>> print('"Isn\'t," they said.') - "Isn't," they said. >>> s = 'First line.\nSecond line.' # \n means newline - >>> s # without print(), \n is included in the output + >>> s # without print(), special characters are included in the string 'First line.\nSecond line.' - >>> print(s) # with print(), \n produces a new line + >>> print(s) # with print(), special characters are interpreted, so \n produces new line First line. Second line. From webhook-mailer at python.org Mon Jul 24 12:38:27 2023 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 24 Jul 2023 16:38:27 -0000 Subject: [Python-checkins] Thoroughly refactor the cases generator (#107151) Message-ID: https://github.com/python/cpython/commit/032f4809094bf03d92c54e46b305c499ef7e3165 commit: 032f4809094bf03d92c54e46b305c499ef7e3165 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-24T09:38:23-07:00 summary: Thoroughly refactor the cases generator (#107151) This mostly extracts a whole bunch of stuff out of generate_cases.py into separate files, but there are a few other things going on here. - analysis.py: `Analyzer` etc. - instructions.py: `Instruction` etc. - flags.py: `InstructionFlags`, `variable_used`, `variable_used_unspecialized` - formatting.py: `Formatter` etc. - Rename parser.py to parsing.py, to avoid conflict with stdlib parser.py - Blackify most things - Fix most mypy errors - Remove output filenames from Generator state, add them to `write_instructions()` etc. - Fix unit tests files: A Tools/cases_generator/analysis.py A Tools/cases_generator/flags.py A Tools/cases_generator/formatting.py A Tools/cases_generator/instructions.py A Tools/cases_generator/parsing.py D Tools/cases_generator/parser.py M Lib/test/test_generated_cases.py M Tools/cases_generator/generate_cases.py diff --git a/Lib/test/test_generated_cases.py b/Lib/test/test_generated_cases.py index ba0e5e8b0f695..9c23c2e82058c 100644 --- a/Lib/test/test_generated_cases.py +++ b/Lib/test/test_generated_cases.py @@ -6,8 +6,10 @@ test_tools.skip_if_missing('cases_generator') with test_tools.imports_under_tool('cases_generator'): + import analysis + import formatting import generate_cases - from parser import StackEffect + from parsing import StackEffect class TestEffects(unittest.TestCase): @@ -27,37 +29,37 @@ def test_effect_sizes(self): StackEffect("q", "", "", ""), StackEffect("r", "", "", ""), ] - self.assertEqual(generate_cases.effect_size(x), (1, "")) - self.assertEqual(generate_cases.effect_size(y), (0, "oparg")) - self.assertEqual(generate_cases.effect_size(z), (0, "oparg*2")) + self.assertEqual(formatting.effect_size(x), (1, "")) + self.assertEqual(formatting.effect_size(y), (0, "oparg")) + self.assertEqual(formatting.effect_size(z), (0, "oparg*2")) self.assertEqual( - generate_cases.list_effect_size(input_effects), + formatting.list_effect_size(input_effects), (1, "oparg + oparg*2"), ) self.assertEqual( - generate_cases.list_effect_size(output_effects), + formatting.list_effect_size(output_effects), (2, "oparg*4"), ) self.assertEqual( - generate_cases.list_effect_size(other_effects), + formatting.list_effect_size(other_effects), (2, "(oparg<<1)"), ) self.assertEqual( - generate_cases.string_effect_size( - generate_cases.list_effect_size(input_effects), + formatting.string_effect_size( + formatting.list_effect_size(input_effects), ), "1 + oparg + oparg*2", ) self.assertEqual( - generate_cases.string_effect_size( - generate_cases.list_effect_size(output_effects), + formatting.string_effect_size( + formatting.list_effect_size(output_effects), ), "2 + oparg*4", ) self.assertEqual( - generate_cases.string_effect_size( - generate_cases.list_effect_size(other_effects), + formatting.string_effect_size( + formatting.list_effect_size(other_effects), ), "2 + (oparg<<1)", ) @@ -90,23 +92,17 @@ def tearDown(self) -> None: def run_cases_test(self, input: str, expected: str): with open(self.temp_input_filename, "w+") as temp_input: - temp_input.write(generate_cases.BEGIN_MARKER) + temp_input.write(analysis.BEGIN_MARKER) temp_input.write(input) - temp_input.write(generate_cases.END_MARKER) + temp_input.write(analysis.END_MARKER) temp_input.flush() - a = generate_cases.Analyzer( - [self.temp_input_filename], - self.temp_output_filename, - self.temp_metadata_filename, - self.temp_pymetadata_filename, - self.temp_executor_filename, - ) + a = generate_cases.Generator([self.temp_input_filename]) a.parse() a.analyze() if a.errors: raise RuntimeError(f"Found {a.errors} errors") - a.write_instructions() + a.write_instructions(self.temp_output_filename, False) with open(self.temp_output_filename) as temp_output: lines = temp_output.readlines() diff --git a/Tools/cases_generator/analysis.py b/Tools/cases_generator/analysis.py new file mode 100644 index 0000000000000..53ae7360b8ae1 --- /dev/null +++ b/Tools/cases_generator/analysis.py @@ -0,0 +1,412 @@ +import re +import sys +import typing + +from flags import InstructionFlags, variable_used +from formatting import prettify_filename, UNUSED +from instructions import ( + ActiveCacheEffect, + Component, + Instruction, + InstructionOrCacheEffect, + MacroInstruction, + MacroParts, + OverriddenInstructionPlaceHolder, + PseudoInstruction, + StackEffectMapping, +) +import parsing +from parsing import StackEffect + +BEGIN_MARKER = "// BEGIN BYTECODES //" +END_MARKER = "// END BYTECODES //" + +RESERVED_WORDS = { + "co_consts": "Use FRAME_CO_CONSTS.", + "co_names": "Use FRAME_CO_NAMES.", +} + +RE_PREDICTED = r"^\s*(?:GO_TO_INSTRUCTION\(|DEOPT_IF\(.*?,\s*)(\w+)\);\s*(?://.*)?$" + + +class Analyzer: + """Parse input, analyze it, and write to output.""" + + input_filenames: list[str] + errors: int = 0 + + def __init__(self, input_filenames: list[str]): + self.input_filenames = input_filenames + + def error(self, msg: str, node: parsing.Node) -> None: + lineno = 0 + filename = "" + if context := node.context: + filename = context.owner.filename + # Use line number of first non-comment in the node + for token in context.owner.tokens[context.begin : context.end]: + lineno = token.line + if token.kind != "COMMENT": + break + print(f"{filename}:{lineno}: {msg}", file=sys.stderr) + self.errors += 1 + + everything: list[ + parsing.InstDef + | parsing.Macro + | parsing.Pseudo + | OverriddenInstructionPlaceHolder + ] + instrs: dict[str, Instruction] # Includes ops + macros: dict[str, parsing.Macro] + macro_instrs: dict[str, MacroInstruction] + families: dict[str, parsing.Family] + pseudos: dict[str, parsing.Pseudo] + pseudo_instrs: dict[str, PseudoInstruction] + + def parse(self) -> None: + """Parse the source text. + + We only want the parser to see the stuff between the + begin and end markers. + """ + + self.everything = [] + self.instrs = {} + self.macros = {} + self.families = {} + self.pseudos = {} + + instrs_idx: dict[str, int] = dict() + + for filename in self.input_filenames: + self.parse_file(filename, instrs_idx) + + files = " + ".join(self.input_filenames) + print( + f"Read {len(self.instrs)} instructions/ops, " + f"{len(self.macros)} macros, {len(self.pseudos)} pseudos, " + f"and {len(self.families)} families from {files}", + file=sys.stderr, + ) + + def parse_file(self, filename: str, instrs_idx: dict[str, int]) -> None: + with open(filename) as file: + src = file.read() + + psr = parsing.Parser(src, filename=prettify_filename(filename)) + + # Skip until begin marker + while tkn := psr.next(raw=True): + if tkn.text == BEGIN_MARKER: + break + else: + raise psr.make_syntax_error( + f"Couldn't find {BEGIN_MARKER!r} in {psr.filename}" + ) + start = psr.getpos() + + # Find end marker, then delete everything after it + while tkn := psr.next(raw=True): + if tkn.text == END_MARKER: + break + del psr.tokens[psr.getpos() - 1 :] + + # Parse from start + psr.setpos(start) + thing: parsing.Node | None + thing_first_token = psr.peek() + while thing := psr.definition(): + thing = typing.cast( + parsing.InstDef | parsing.Macro | parsing.Pseudo | parsing.Family, thing + ) + if ws := [w for w in RESERVED_WORDS if variable_used(thing, w)]: + self.error( + f"'{ws[0]}' is a reserved word. {RESERVED_WORDS[ws[0]]}", thing + ) + + match thing: + case parsing.InstDef(name=name): + if name in self.instrs: + if not thing.override: + raise psr.make_syntax_error( + f"Duplicate definition of '{name}' @ {thing.context} " + f"previous definition @ {self.instrs[name].inst.context}", + thing_first_token, + ) + self.everything[ + instrs_idx[name] + ] = OverriddenInstructionPlaceHolder(name=name) + if name not in self.instrs and thing.override: + raise psr.make_syntax_error( + f"Definition of '{name}' @ {thing.context} is supposed to be " + "an override but no previous definition exists.", + thing_first_token, + ) + self.instrs[name] = Instruction(thing) + instrs_idx[name] = len(self.everything) + self.everything.append(thing) + case parsing.Macro(name): + self.macros[name] = thing + self.everything.append(thing) + case parsing.Family(name): + self.families[name] = thing + case parsing.Pseudo(name): + self.pseudos[name] = thing + self.everything.append(thing) + case _: + typing.assert_never(thing) + if not psr.eof(): + raise psr.make_syntax_error(f"Extra stuff at the end of {filename}") + + def analyze(self) -> None: + """Analyze the inputs. + + Raises SystemExit if there is an error. + """ + self.analyze_macros_and_pseudos() + self.find_predictions() + self.map_families() + self.check_families() + + def find_predictions(self) -> None: + """Find the instructions that need PREDICTED() labels.""" + for instr in self.instrs.values(): + targets: set[str] = set() + for line in instr.block_text: + if m := re.match(RE_PREDICTED, line): + targets.add(m.group(1)) + for target in targets: + if target_instr := self.instrs.get(target): + target_instr.predicted = True + elif target_macro := self.macro_instrs.get(target): + target_macro.predicted = True + else: + self.error( + f"Unknown instruction {target!r} predicted in {instr.name!r}", + instr.inst, # TODO: Use better location + ) + + def map_families(self) -> None: + """Link instruction names back to their family, if they have one.""" + for family in self.families.values(): + for member in [family.name] + family.members: + if member_instr := self.instrs.get(member): + if ( + member_instr.family is not family + and member_instr.family is not None + ): + self.error( + f"Instruction {member} is a member of multiple families " + f"({member_instr.family.name}, {family.name}).", + family, + ) + else: + member_instr.family = family + elif not self.macro_instrs.get(member): + self.error( + f"Unknown instruction {member!r} referenced in family {family.name!r}", + family, + ) + + def check_families(self) -> None: + """Check each family: + + - Must have at least 2 members (including head) + - Head and all members must be known instructions + - Head and all members must have the same cache, input and output effects + """ + for family in self.families.values(): + if family.name not in self.macro_instrs and family.name not in self.instrs: + self.error( + f"Family {family.name!r} has unknown instruction {family.name!r}", + family, + ) + members = [ + member + for member in family.members + if member in self.instrs or member in self.macro_instrs + ] + if members != family.members: + unknown = set(family.members) - set(members) + self.error( + f"Family {family.name!r} has unknown members: {unknown}", family + ) + expected_effects = self.effect_counts(family.name) + for member in members: + member_effects = self.effect_counts(member) + if member_effects != expected_effects: + self.error( + f"Family {family.name!r} has inconsistent " + f"(cache, input, output) effects:\n" + f" {family.name} = {expected_effects}; " + f"{member} = {member_effects}", + family, + ) + + def effect_counts(self, name: str) -> tuple[int, int, int]: + if instr := self.instrs.get(name): + cache = instr.cache_offset + input = len(instr.input_effects) + output = len(instr.output_effects) + elif mac := self.macro_instrs.get(name): + cache = mac.cache_offset + input, output = 0, 0 + for part in mac.parts: + if isinstance(part, Component): + # A component may pop what the previous component pushed, + # so we offset the input/output counts by that. + delta_i = len(part.instr.input_effects) + delta_o = len(part.instr.output_effects) + offset = min(delta_i, output) + input += delta_i - offset + output += delta_o - offset + else: + assert False, f"Unknown instruction {name!r}" + return cache, input, output + + def analyze_macros_and_pseudos(self) -> None: + """Analyze each macro and pseudo instruction.""" + self.macro_instrs = {} + self.pseudo_instrs = {} + for name, macro in self.macros.items(): + self.macro_instrs[name] = self.analyze_macro(macro) + for name, pseudo in self.pseudos.items(): + self.pseudo_instrs[name] = self.analyze_pseudo(pseudo) + + def analyze_macro(self, macro: parsing.Macro) -> MacroInstruction: + components = self.check_macro_components(macro) + stack, initial_sp = self.stack_analysis(components) + sp = initial_sp + parts: MacroParts = [] + flags = InstructionFlags.newEmpty() + offset = 0 + for component in components: + match component: + case parsing.CacheEffect() as ceffect: + parts.append(ceffect) + offset += ceffect.size + case Instruction() as instr: + part, sp, offset = self.analyze_instruction( + instr, stack, sp, offset + ) + parts.append(part) + flags.add(instr.instr_flags) + case _: + typing.assert_never(component) + final_sp = sp + format = "IB" + if offset: + format += "C" + "0" * (offset - 1) + return MacroInstruction( + macro.name, stack, initial_sp, final_sp, format, flags, macro, parts, offset + ) + + def analyze_pseudo(self, pseudo: parsing.Pseudo) -> PseudoInstruction: + targets = [self.instrs[target] for target in pseudo.targets] + assert targets + # Make sure the targets have the same fmt + fmts = list(set([t.instr_fmt for t in targets])) + assert len(fmts) == 1 + assert len(list(set([t.instr_flags.bitmap() for t in targets]))) == 1 + return PseudoInstruction(pseudo.name, targets, fmts[0], targets[0].instr_flags) + + def analyze_instruction( + self, instr: Instruction, stack: list[StackEffect], sp: int, offset: int + ) -> tuple[Component, int, int]: + input_mapping: StackEffectMapping = [] + for ieffect in reversed(instr.input_effects): + sp -= 1 + input_mapping.append((stack[sp], ieffect)) + output_mapping: StackEffectMapping = [] + for oeffect in instr.output_effects: + output_mapping.append((stack[sp], oeffect)) + sp += 1 + active_effects: list[ActiveCacheEffect] = [] + for ceffect in instr.cache_effects: + if ceffect.name != UNUSED: + active_effects.append(ActiveCacheEffect(ceffect, offset)) + offset += ceffect.size + return ( + Component(instr, input_mapping, output_mapping, active_effects), + sp, + offset, + ) + + def check_macro_components( + self, macro: parsing.Macro + ) -> list[InstructionOrCacheEffect]: + components: list[InstructionOrCacheEffect] = [] + for uop in macro.uops: + match uop: + case parsing.OpName(name): + if name not in self.instrs: + self.error(f"Unknown instruction {name!r}", macro) + components.append(self.instrs[name]) + case parsing.CacheEffect(): + components.append(uop) + case _: + typing.assert_never(uop) + return components + + def stack_analysis( + self, components: typing.Iterable[InstructionOrCacheEffect] + ) -> tuple[list[StackEffect], int]: + """Analyze a macro. + + Ignore cache effects. + + Return the list of variables (as StackEffects) and the initial stack pointer. + """ + lowest = current = highest = 0 + conditions: dict[int, str] = {} # Indexed by 'current'. + last_instr: Instruction | None = None + for thing in components: + if isinstance(thing, Instruction): + last_instr = thing + for thing in components: + match thing: + case Instruction() as instr: + if any( + eff.size for eff in instr.input_effects + instr.output_effects + ): + # TODO: Eventually this will be needed, at least for macros. + self.error( + f"Instruction {instr.name!r} has variable-sized stack effect, " + "which are not supported in macro instructions", + instr.inst, # TODO: Pass name+location of macro + ) + if any(eff.cond for eff in instr.input_effects): + self.error( + f"Instruction {instr.name!r} has conditional input stack effect, " + "which are not supported in macro instructions", + instr.inst, # TODO: Pass name+location of macro + ) + if ( + any(eff.cond for eff in instr.output_effects) + and instr is not last_instr + ): + self.error( + f"Instruction {instr.name!r} has conditional output stack effect, " + "but is not the last instruction in a macro", + instr.inst, # TODO: Pass name+location of macro + ) + current -= len(instr.input_effects) + lowest = min(lowest, current) + for eff in instr.output_effects: + if eff.cond: + conditions[current] = eff.cond + current += 1 + highest = max(highest, current) + case parsing.CacheEffect(): + pass + case _: + typing.assert_never(thing) + # At this point, 'current' is the net stack effect, + # and 'lowest' and 'highest' are the extremes. + # Note that 'lowest' may be negative. + stack = [ + StackEffect(f"_tmp_{i}", "", conditions.get(highest - i, "")) + for i in reversed(range(1, highest - lowest + 1)) + ] + return stack, -lowest diff --git a/Tools/cases_generator/flags.py b/Tools/cases_generator/flags.py new file mode 100644 index 0000000000000..78acd93b73ac3 --- /dev/null +++ b/Tools/cases_generator/flags.py @@ -0,0 +1,102 @@ +import dataclasses + +from formatting import Formatter +import lexer as lx +import parsing + + + at dataclasses.dataclass +class InstructionFlags: + """Construct and manipulate instruction flags""" + + HAS_ARG_FLAG: bool + HAS_CONST_FLAG: bool + HAS_NAME_FLAG: bool + HAS_JUMP_FLAG: bool + HAS_FREE_FLAG: bool + HAS_LOCAL_FLAG: bool + + def __post_init__(self): + self.bitmask = {name: (1 << i) for i, name in enumerate(self.names())} + + @staticmethod + def fromInstruction(instr: parsing.Node): + + has_free = ( + variable_used(instr, "PyCell_New") + or variable_used(instr, "PyCell_GET") + or variable_used(instr, "PyCell_SET") + ) + + return InstructionFlags( + HAS_ARG_FLAG=variable_used(instr, "oparg"), + HAS_CONST_FLAG=variable_used(instr, "FRAME_CO_CONSTS"), + HAS_NAME_FLAG=variable_used(instr, "FRAME_CO_NAMES"), + HAS_JUMP_FLAG=variable_used(instr, "JUMPBY"), + HAS_FREE_FLAG=has_free, + HAS_LOCAL_FLAG=( + variable_used(instr, "GETLOCAL") or variable_used(instr, "SETLOCAL") + ) + and not has_free, + ) + + @staticmethod + def newEmpty(): + return InstructionFlags(False, False, False, False, False, False) + + def add(self, other: "InstructionFlags") -> None: + for name, value in dataclasses.asdict(other).items(): + if value: + setattr(self, name, value) + + def names(self, value=None): + if value is None: + return dataclasses.asdict(self).keys() + return [n for n, v in dataclasses.asdict(self).items() if v == value] + + def bitmap(self) -> int: + flags = 0 + for name in self.names(): + if getattr(self, name): + flags |= self.bitmask[name] + return flags + + @classmethod + def emit_macros(cls, out: Formatter): + flags = cls.newEmpty() + for name, value in flags.bitmask.items(): + out.emit(f"#define {name} ({value})") + + for name, value in flags.bitmask.items(): + out.emit( + f"#define OPCODE_{name[:-len('_FLAG')]}(OP) " + f"(_PyOpcode_opcode_metadata[OP].flags & ({name}))" + ) + + +def variable_used(node: parsing.Node, name: str) -> bool: + """Determine whether a variable with a given name is used in a node.""" + return any( + token.kind == "IDENTIFIER" and token.text == name for token in node.tokens + ) + + +def variable_used_unspecialized(node: parsing.Node, name: str) -> bool: + """Like variable_used(), but skips #if ENABLE_SPECIALIZATION blocks.""" + tokens: list[lx.Token] = [] + skipping = False + for i, token in enumerate(node.tokens): + if token.kind == "MACRO": + text = "".join(token.text.split()) + # TODO: Handle nested #if + if text == "#if": + if ( + i + 1 < len(node.tokens) + and node.tokens[i + 1].text == "ENABLE_SPECIALIZATION" + ): + skipping = True + elif text in ("#else", "#endif"): + skipping = False + if not skipping: + tokens.append(token) + return any(token.kind == "IDENTIFIER" and token.text == name for token in tokens) diff --git a/Tools/cases_generator/formatting.py b/Tools/cases_generator/formatting.py new file mode 100644 index 0000000000000..6b5a375af891b --- /dev/null +++ b/Tools/cases_generator/formatting.py @@ -0,0 +1,188 @@ +import contextlib +import re +import typing + +from parsing import StackEffect + +UNUSED = "unused" + + +class Formatter: + """Wraps an output stream with the ability to indent etc.""" + + stream: typing.TextIO + prefix: str + emit_line_directives: bool = False + lineno: int # Next line number, 1-based + filename: str # Slightly improved stream.filename + nominal_lineno: int + nominal_filename: str + + def __init__( + self, stream: typing.TextIO, indent: int, + emit_line_directives: bool = False, comment: str = "//", + ) -> None: + self.stream = stream + self.prefix = " " * indent + self.emit_line_directives = emit_line_directives + self.comment = comment + self.lineno = 1 + self.filename = prettify_filename(self.stream.name) + self.nominal_lineno = 1 + self.nominal_filename = self.filename + + def write_raw(self, s: str) -> None: + self.stream.write(s) + newlines = s.count("\n") + self.lineno += newlines + self.nominal_lineno += newlines + + def emit(self, arg: str) -> None: + if arg: + self.write_raw(f"{self.prefix}{arg}\n") + else: + self.write_raw("\n") + + def set_lineno(self, lineno: int, filename: str) -> None: + if self.emit_line_directives: + if lineno != self.nominal_lineno or filename != self.nominal_filename: + self.emit(f'#line {lineno} "{filename}"') + self.nominal_lineno = lineno + self.nominal_filename = filename + + def reset_lineno(self) -> None: + if self.lineno != self.nominal_lineno or self.filename != self.nominal_filename: + self.set_lineno(self.lineno + 1, self.filename) + + @contextlib.contextmanager + def indent(self): + self.prefix += " " + yield + self.prefix = self.prefix[:-4] + + @contextlib.contextmanager + def block(self, head: str, tail: str = ""): + if head: + self.emit(head + " {") + else: + self.emit("{") + with self.indent(): + yield + self.emit("}" + tail) + + def stack_adjust( + self, + input_effects: list[StackEffect], + output_effects: list[StackEffect], + ): + shrink, isym = list_effect_size(input_effects) + grow, osym = list_effect_size(output_effects) + diff = grow - shrink + if isym and isym != osym: + self.emit(f"STACK_SHRINK({isym});") + if diff < 0: + self.emit(f"STACK_SHRINK({-diff});") + if diff > 0: + self.emit(f"STACK_GROW({diff});") + if osym and osym != isym: + self.emit(f"STACK_GROW({osym});") + + def declare(self, dst: StackEffect, src: StackEffect | None): + if dst.name == UNUSED or dst.cond == "0": + return + typ = f"{dst.type}" if dst.type else "PyObject *" + if src: + cast = self.cast(dst, src) + init = f" = {cast}{src.name}" + elif dst.cond: + init = " = NULL" + else: + init = "" + sepa = "" if typ.endswith("*") else " " + self.emit(f"{typ}{sepa}{dst.name}{init};") + + def assign(self, dst: StackEffect, src: StackEffect): + if src.name == UNUSED: + return + if src.size: + # Don't write sized arrays -- it's up to the user code. + return + cast = self.cast(dst, src) + if re.match(r"^REG\(oparg(\d+)\)$", dst.name): + self.emit(f"Py_XSETREF({dst.name}, {cast}{src.name});") + else: + stmt = f"{dst.name} = {cast}{src.name};" + if src.cond and src.cond != "1": + if src.cond == "0": + # It will not be executed + return + stmt = f"if ({src.cond}) {{ {stmt} }}" + self.emit(stmt) + + def cast(self, dst: StackEffect, src: StackEffect) -> str: + return f"({dst.type or 'PyObject *'})" if src.type != dst.type else "" + + +def prettify_filename(filename: str) -> str: + # Make filename more user-friendly and less platform-specific, + # it is only used for error reporting at this point. + filename = filename.replace("\\", "/") + if filename.startswith("./"): + filename = filename[2:] + if filename.endswith(".new"): + filename = filename[:-4] + return filename + + +def list_effect_size(effects: list[StackEffect]) -> tuple[int, str]: + numeric = 0 + symbolic: list[str] = [] + for effect in effects: + diff, sym = effect_size(effect) + numeric += diff + if sym: + symbolic.append(maybe_parenthesize(sym)) + return numeric, " + ".join(symbolic) + + +def effect_size(effect: StackEffect) -> tuple[int, str]: + """Return the 'size' impact of a stack effect. + + Returns a tuple (numeric, symbolic) where: + + - numeric is an int giving the statically analyzable size of the effect + - symbolic is a string representing a variable effect (e.g. 'oparg*2') + + At most one of these will be non-zero / non-empty. + """ + if effect.size: + assert not effect.cond, "Array effects cannot have a condition" + return 0, effect.size + elif effect.cond: + if effect.cond in ("0", "1"): + return int(effect.cond), "" + return 0, f"{maybe_parenthesize(effect.cond)} ? 1 : 0" + else: + return 1, "" + + +def maybe_parenthesize(sym: str) -> str: + """Add parentheses around a string if it contains an operator. + + An exception is made for '*' which is common and harmless + in the context where the symbolic size is used. + """ + if re.match(r"^[\s\w*]+$", sym): + return sym + else: + return f"({sym})" + + +def string_effect_size(arg: tuple[int, str]) -> str: + numeric, symbolic = arg + if numeric and symbolic: + return f"{numeric} + {symbolic}" + elif symbolic: + return symbolic + else: + return str(numeric) diff --git a/Tools/cases_generator/generate_cases.py b/Tools/cases_generator/generate_cases.py index 3a679b2090f19..967e1e2f5b63b 100644 --- a/Tools/cases_generator/generate_cases.py +++ b/Tools/cases_generator/generate_cases.py @@ -1,21 +1,31 @@ """Generate the main interpreter switch. - Reads the instruction definitions from bytecodes.c. Writes the cases to generated_cases.c.h, which is #included in ceval.c. """ import argparse import contextlib -import dataclasses import os import posixpath -import re import sys import typing -import lexer as lx -import parser -from parser import StackEffect +from analysis import Analyzer +from formatting import Formatter, list_effect_size, maybe_parenthesize +from flags import InstructionFlags, variable_used +from instructions import ( + AnyInstruction, + Component, + Instruction, + MacroInstruction, + MacroParts, + PseudoInstruction, + StackEffect, + OverriddenInstructionPlaceHolder, + TIER_TWO, +) +import parsing +from parsing import StackEffect HERE = os.path.dirname(__file__) @@ -33,13 +43,6 @@ DEFAULT_EXECUTOR_OUTPUT = os.path.relpath( os.path.join(ROOT, "Python/executor_cases.c.h") ) -BEGIN_MARKER = "// BEGIN BYTECODES //" -END_MARKER = "// END BYTECODES //" -RE_PREDICTED = ( - r"^\s*(?:GO_TO_INSTRUCTION\(|DEOPT_IF\(.*?,\s*)(\w+)\);\s*(?://.*)?$" -) -UNUSED = "unused" -BITS_PER_CODE_UNIT = 16 # Constants used instead of size for macro expansions. # Note: 1, 2, 4 must match actual cache entry sizes. @@ -52,10 +55,7 @@ "OPARG_BOTTOM": 6, } -RESERVED_WORDS = { - "co_consts" : "Use FRAME_CO_CONSTS.", - "co_names": "Use FRAME_CO_NAMES.", -} +INSTR_FMT_PREFIX = "INSTR_FMT_" arg_parser = argparse.ArgumentParser( description="Generate the code for the interpreter switch.", @@ -65,10 +65,18 @@ "-o", "--output", type=str, help="Generated code", default=DEFAULT_OUTPUT ) arg_parser.add_argument( - "-m", "--metadata", type=str, help="Generated C metadata", default=DEFAULT_METADATA_OUTPUT + "-m", + "--metadata", + type=str, + help="Generated C metadata", + default=DEFAULT_METADATA_OUTPUT, ) arg_parser.add_argument( - "-p", "--pymetadata", type=str, help="Generated Python metadata", default=DEFAULT_PYMETADATA_OUTPUT + "-p", + "--pymetadata", + type=str, + help="Generated Python metadata", + default=DEFAULT_PYMETADATA_OUTPUT, ) arg_parser.add_argument( "-l", "--emit-line-directives", help="Emit #line directives", action="store_true" @@ -85,966 +93,9 @@ ) -def effect_size(effect: StackEffect) -> tuple[int, str]: - """Return the 'size' impact of a stack effect. - - Returns a tuple (numeric, symbolic) where: - - - numeric is an int giving the statically analyzable size of the effect - - symbolic is a string representing a variable effect (e.g. 'oparg*2') - - At most one of these will be non-zero / non-empty. - """ - if effect.size: - assert not effect.cond, "Array effects cannot have a condition" - return 0, effect.size - elif effect.cond: - if effect.cond in ("0", "1"): - return int(effect.cond), "" - return 0, f"{maybe_parenthesize(effect.cond)} ? 1 : 0" - else: - return 1, "" - - -def maybe_parenthesize(sym: str) -> str: - """Add parentheses around a string if it contains an operator. - - An exception is made for '*' which is common and harmless - in the context where the symbolic size is used. - """ - if re.match(r"^[\s\w*]+$", sym): - return sym - else: - return f"({sym})" - - -def list_effect_size(effects: list[StackEffect]) -> tuple[int, str]: - numeric = 0 - symbolic: list[str] = [] - for effect in effects: - diff, sym = effect_size(effect) - numeric += diff - if sym: - symbolic.append(maybe_parenthesize(sym)) - return numeric, " + ".join(symbolic) - - -def string_effect_size(arg: tuple[int, str]) -> str: - numeric, symbolic = arg - if numeric and symbolic: - return f"{numeric} + {symbolic}" - elif symbolic: - return symbolic - else: - return str(numeric) - - -class Formatter: - """Wraps an output stream with the ability to indent etc.""" - - stream: typing.TextIO - prefix: str - emit_line_directives: bool = False - lineno: int # Next line number, 1-based - filename: str # Slightly improved stream.filename - nominal_lineno: int - nominal_filename: str - - def __init__( - self, stream: typing.TextIO, indent: int, - emit_line_directives: bool = False, comment: str = "//", - ) -> None: - self.stream = stream - self.prefix = " " * indent - self.emit_line_directives = emit_line_directives - self.comment = comment - self.lineno = 1 - self.filename = prettify_filename(self.stream.name) - self.nominal_lineno = 1 - self.nominal_filename = self.filename - - def write_raw(self, s: str) -> None: - self.stream.write(s) - newlines = s.count("\n") - self.lineno += newlines - self.nominal_lineno += newlines - - def emit(self, arg: str) -> None: - if arg: - self.write_raw(f"{self.prefix}{arg}\n") - else: - self.write_raw("\n") - - def set_lineno(self, lineno: int, filename: str) -> None: - if self.emit_line_directives: - if lineno != self.nominal_lineno or filename != self.nominal_filename: - self.emit(f'#line {lineno} "{filename}"') - self.nominal_lineno = lineno - self.nominal_filename = filename - - def reset_lineno(self) -> None: - if self.lineno != self.nominal_lineno or self.filename != self.nominal_filename: - self.set_lineno(self.lineno + 1, self.filename) - - @contextlib.contextmanager - def indent(self): - self.prefix += " " - yield - self.prefix = self.prefix[:-4] - - @contextlib.contextmanager - def block(self, head: str, tail: str = ""): - if head: - self.emit(head + " {") - else: - self.emit("{") - with self.indent(): - yield - self.emit("}" + tail) - - def stack_adjust( - self, - input_effects: list[StackEffect], - output_effects: list[StackEffect], - ): - shrink, isym = list_effect_size(input_effects) - grow, osym = list_effect_size(output_effects) - diff = grow - shrink - if isym and isym != osym: - self.emit(f"STACK_SHRINK({isym});") - if diff < 0: - self.emit(f"STACK_SHRINK({-diff});") - if diff > 0: - self.emit(f"STACK_GROW({diff});") - if osym and osym != isym: - self.emit(f"STACK_GROW({osym});") - - def declare(self, dst: StackEffect, src: StackEffect | None): - if dst.name == UNUSED or dst.cond == "0": - return - typ = f"{dst.type}" if dst.type else "PyObject *" - if src: - cast = self.cast(dst, src) - init = f" = {cast}{src.name}" - elif dst.cond: - init = " = NULL" - else: - init = "" - sepa = "" if typ.endswith("*") else " " - self.emit(f"{typ}{sepa}{dst.name}{init};") - - def assign(self, dst: StackEffect, src: StackEffect): - if src.name == UNUSED: - return - if src.size: - # Don't write sized arrays -- it's up to the user code. - return - cast = self.cast(dst, src) - if re.match(r"^REG\(oparg(\d+)\)$", dst.name): - self.emit(f"Py_XSETREF({dst.name}, {cast}{src.name});") - else: - stmt = f"{dst.name} = {cast}{src.name};" - if src.cond and src.cond != "1": - if src.cond == "0": - # It will not be executed - return - stmt = f"if ({src.cond}) {{ {stmt} }}" - self.emit(stmt) - - def cast(self, dst: StackEffect, src: StackEffect) -> str: - return f"({dst.type or 'PyObject *'})" if src.type != dst.type else "" - - at dataclasses.dataclass -class InstructionFlags: - """Construct and manipulate instruction flags""" - - HAS_ARG_FLAG: bool - HAS_CONST_FLAG: bool - HAS_NAME_FLAG: bool - HAS_JUMP_FLAG: bool - HAS_FREE_FLAG: bool - HAS_LOCAL_FLAG: bool - - def __post_init__(self): - self.bitmask = { - name : (1 << i) for i, name in enumerate(self.names()) - } - - @staticmethod - def fromInstruction(instr: "AnyInstruction"): - - has_free = (variable_used(instr, "PyCell_New") or - variable_used(instr, "PyCell_GET") or - variable_used(instr, "PyCell_SET")) - - return InstructionFlags( - HAS_ARG_FLAG=variable_used(instr, "oparg"), - HAS_CONST_FLAG=variable_used(instr, "FRAME_CO_CONSTS"), - HAS_NAME_FLAG=variable_used(instr, "FRAME_CO_NAMES"), - HAS_JUMP_FLAG=variable_used(instr, "JUMPBY"), - HAS_FREE_FLAG=has_free, - HAS_LOCAL_FLAG=(variable_used(instr, "GETLOCAL") or - variable_used(instr, "SETLOCAL")) and - not has_free, - ) - - @staticmethod - def newEmpty(): - return InstructionFlags(False, False, False, False, False, False) - - def add(self, other: "InstructionFlags") -> None: - for name, value in dataclasses.asdict(other).items(): - if value: - setattr(self, name, value) - - def names(self, value=None): - if value is None: - return dataclasses.asdict(self).keys() - return [n for n, v in dataclasses.asdict(self).items() if v == value] - - def bitmap(self) -> int: - flags = 0 - for name in self.names(): - if getattr(self, name): - flags |= self.bitmask[name] - return flags - - @classmethod - def emit_macros(cls, out: Formatter): - flags = cls.newEmpty() - for name, value in flags.bitmask.items(): - out.emit(f"#define {name} ({value})"); - - for name, value in flags.bitmask.items(): - out.emit( - f"#define OPCODE_{name[:-len('_FLAG')]}(OP) " - f"(_PyOpcode_opcode_metadata[OP].flags & ({name}))") - - - at dataclasses.dataclass -class ActiveCacheEffect: - """Wraps a CacheEffect that is actually used, in context.""" - effect: parser.CacheEffect - offset: int - - -FORBIDDEN_NAMES_IN_UOPS = ( - "resume_with_error", - "kwnames", - "next_instr", - "oparg1", # Proxy for super-instructions like LOAD_FAST_LOAD_FAST - "JUMPBY", - "DISPATCH", - "INSTRUMENTED_JUMP", - "throwflag", - "exception_unwind", - "import_from", - "import_name", - "_PyObject_CallNoArgs", # Proxy for BEFORE_WITH -) - - -# Interpreter tiers -TIER_ONE = 1 # Specializing adaptive interpreter (PEP 659) -TIER_TWO = 2 # Experimental tracing interpreter -Tiers: typing.TypeAlias = typing.Literal[1, 2] - - - at dataclasses.dataclass -class Instruction: - """An instruction with additional data and code.""" - - # Parts of the underlying instruction definition - inst: parser.InstDef - kind: typing.Literal["inst", "op"] - name: str - block: parser.Block - block_text: list[str] # Block.text, less curlies, less PREDICT() calls - block_line: int # First line of block in original code - - # Computed by constructor - always_exits: bool - cache_offset: int - cache_effects: list[parser.CacheEffect] - input_effects: list[StackEffect] - output_effects: list[StackEffect] - unmoved_names: frozenset[str] - instr_fmt: str - instr_flags: InstructionFlags - active_caches: list[ActiveCacheEffect] - - # Set later - family: parser.Family | None = None - predicted: bool = False - - def __init__(self, inst: parser.InstDef): - self.inst = inst - self.kind = inst.kind - self.name = inst.name - self.block = inst.block - self.block_text, self.check_eval_breaker, self.block_line = \ - extract_block_text(self.block) - self.always_exits = always_exits(self.block_text) - self.cache_effects = [ - effect for effect in inst.inputs if isinstance(effect, parser.CacheEffect) - ] - self.cache_offset = sum(c.size for c in self.cache_effects) - self.input_effects = [ - effect for effect in inst.inputs if isinstance(effect, StackEffect) - ] - self.output_effects = inst.outputs # For consistency/completeness - unmoved_names: set[str] = set() - for ieffect, oeffect in zip(self.input_effects, self.output_effects): - if ieffect.name == oeffect.name: - unmoved_names.add(ieffect.name) - else: - break - self.unmoved_names = frozenset(unmoved_names) - - self.instr_flags = InstructionFlags.fromInstruction(inst) - - self.active_caches = [] - offset = 0 - for effect in self.cache_effects: - if effect.name != UNUSED: - self.active_caches.append(ActiveCacheEffect(effect, offset)) - offset += effect.size - - if self.instr_flags.HAS_ARG_FLAG: - fmt = "IB" - else: - fmt = "IX" - if offset: - fmt += "C" + "0"*(offset-1) - self.instr_fmt = fmt - - def is_viable_uop(self) -> bool: - """Whether this instruction is viable as a uop.""" - dprint: typing.Callable[..., None] = lambda *args, **kwargs: None - # if self.name.startswith("CALL"): - # dprint = print - - if self.name == "EXIT_TRACE": - return True # This has 'return frame' but it's okay - if self.always_exits: - dprint(f"Skipping {self.name} because it always exits") - return False - if len(self.active_caches) > 1: - # print(f"Skipping {self.name} because it has >1 cache entries") - return False - res = True - for forbidden in FORBIDDEN_NAMES_IN_UOPS: - # NOTE: To disallow unspecialized uops, use - # if variable_used(self.inst, forbidden): - if variable_used_unspecialized(self.inst, forbidden): - dprint(f"Skipping {self.name} because it uses {forbidden}") - res = False - return res - - def write(self, out: Formatter, tier: Tiers = TIER_ONE) -> None: - """Write one instruction, sans prologue and epilogue.""" - # Write a static assertion that a family's cache size is correct - if family := self.family: - if self.name == family.name: - if cache_size := family.size: - out.emit( - f"static_assert({cache_size} == " - f'{self.cache_offset}, "incorrect cache size");' - ) - - # Write input stack effect variable declarations and initializations - ieffects = list(reversed(self.input_effects)) - for i, ieffect in enumerate(ieffects): - isize = string_effect_size( - list_effect_size([ieff for ieff in ieffects[: i + 1]]) - ) - if ieffect.size: - src = StackEffect(f"(stack_pointer - {maybe_parenthesize(isize)})", "PyObject **") - elif ieffect.cond: - src = StackEffect(f"({ieffect.cond}) ? stack_pointer[-{maybe_parenthesize(isize)}] : NULL", "") - else: - src = StackEffect(f"stack_pointer[-{maybe_parenthesize(isize)}]", "") - out.declare(ieffect, src) - - # Write output stack effect variable declarations - isize = string_effect_size(list_effect_size(self.input_effects)) - input_names = {ieffect.name for ieffect in self.input_effects} - for i, oeffect in enumerate(self.output_effects): - if oeffect.name not in input_names: - if oeffect.size: - osize = string_effect_size( - list_effect_size([oeff for oeff in self.output_effects[:i]]) - ) - offset = "stack_pointer" - if isize != osize: - if isize != "0": - offset += f" - ({isize})" - if osize != "0": - offset += f" + {osize}" - src = StackEffect(offset, "PyObject **") - out.declare(oeffect, src) - else: - out.declare(oeffect, None) - - # out.emit(f"next_instr += OPSIZE({self.inst.name}) - 1;") - - self.write_body(out, 0, self.active_caches, tier=tier) - - # Skip the rest if the block always exits - if self.always_exits: - return - - # Write net stack growth/shrinkage - out.stack_adjust( - [ieff for ieff in self.input_effects], - [oeff for oeff in self.output_effects], - ) - - # Write output stack effect assignments - oeffects = list(reversed(self.output_effects)) - for i, oeffect in enumerate(oeffects): - if oeffect.name in self.unmoved_names: - continue - osize = string_effect_size( - list_effect_size([oeff for oeff in oeffects[: i + 1]]) - ) - if oeffect.size: - dst = StackEffect(f"stack_pointer - {maybe_parenthesize(osize)}", "PyObject **") - else: - dst = StackEffect(f"stack_pointer[-{maybe_parenthesize(osize)}]", "") - out.assign(dst, oeffect) - - # Write cache effect - if tier == TIER_ONE and self.cache_offset: - out.emit(f"next_instr += {self.cache_offset};") - - def write_body( - self, - out: Formatter, - dedent: int, - active_caches: list[ActiveCacheEffect], - tier: Tiers = TIER_ONE, - ) -> None: - """Write the instruction body.""" - # Write cache effect variable declarations and initializations - for active in active_caches: - ceffect = active.effect - bits = ceffect.size * BITS_PER_CODE_UNIT - if bits == 64: - # NOTE: We assume that 64-bit data in the cache - # is always an object pointer. - # If this becomes false, we need a way to specify - # syntactically what type the cache data is. - typ = "PyObject *" - func = "read_obj" - else: - typ = f"uint{bits}_t " - func = f"read_u{bits}" - if tier == TIER_ONE: - out.emit( - f"{typ}{ceffect.name} = {func}(&next_instr[{active.offset}].cache);" - ) - else: - out.emit(f"{typ}{ceffect.name} = ({typ.strip()})operand;") - - # Write the body, substituting a goto for ERROR_IF() and other stuff - assert dedent <= 0 - extra = " " * -dedent - names_to_skip = self.unmoved_names | frozenset({UNUSED, "null"}) - offset = 0 - context = self.block.context - assert context is not None and context.owner is not None - filename = context.owner.filename - for line in self.block_text: - out.set_lineno(self.block_line + offset, filename) - offset += 1 - if m := re.match(r"(\s*)ERROR_IF\((.+), (\w+)\);\s*(?://.*)?$", line): - space, cond, label = m.groups() - space = extra + space - # ERROR_IF() must pop the inputs from the stack. - # The code block is responsible for DECREF()ing them. - # NOTE: If the label doesn't exist, just add it to ceval.c. - - # Don't pop common input/output effects at the bottom! - # These aren't DECREF'ed so they can stay. - ieffs = list(self.input_effects) - oeffs = list(self.output_effects) - while ieffs and oeffs and ieffs[0] == oeffs[0]: - ieffs.pop(0) - oeffs.pop(0) - ninputs, symbolic = list_effect_size(ieffs) - if ninputs: - label = f"pop_{ninputs}_{label}" - if symbolic: - out.write_raw( - f"{space}if ({cond}) {{ STACK_SHRINK({symbolic}); goto {label}; }}\n" - ) - else: - out.write_raw(f"{space}if ({cond}) goto {label};\n") - elif m := re.match(r"(\s*)DECREF_INPUTS\(\);\s*(?://.*)?$", line): - out.reset_lineno() - space = extra + m.group(1) - for ieff in self.input_effects: - if ieff.name in names_to_skip: - continue - if ieff.size: - out.write_raw( - f"{space}for (int _i = {ieff.size}; --_i >= 0;) {{\n" - ) - out.write_raw(f"{space} Py_DECREF({ieff.name}[_i]);\n") - out.write_raw(f"{space}}}\n") - else: - decref = "XDECREF" if ieff.cond else "DECREF" - out.write_raw(f"{space}Py_{decref}({ieff.name});\n") - else: - out.write_raw(extra + line) - out.reset_lineno() - - -InstructionOrCacheEffect = Instruction | parser.CacheEffect -StackEffectMapping = list[tuple[StackEffect, StackEffect]] - - - at dataclasses.dataclass -class Component: - instr: Instruction - input_mapping: StackEffectMapping - output_mapping: StackEffectMapping - active_caches: list[ActiveCacheEffect] - - def write_body(self, out: Formatter) -> None: - with out.block(""): - input_names = {ieffect.name for _, ieffect in self.input_mapping} - for var, ieffect in self.input_mapping: - out.declare(ieffect, var) - for _, oeffect in self.output_mapping: - if oeffect.name not in input_names: - out.declare(oeffect, None) - - self.instr.write_body(out, -4, self.active_caches) - - for var, oeffect in self.output_mapping: - out.assign(var, oeffect) - - -MacroParts = list[Component | parser.CacheEffect] - - - at dataclasses.dataclass -class MacroInstruction: - """A macro instruction.""" - - name: str - stack: list[StackEffect] - initial_sp: int - final_sp: int - instr_fmt: str - instr_flags: InstructionFlags - macro: parser.Macro - parts: MacroParts - cache_offset: int - predicted: bool = False - - - at dataclasses.dataclass -class PseudoInstruction: - """A pseudo instruction.""" - - name: str - targets: list[Instruction] - instr_fmt: str - instr_flags: InstructionFlags - - - at dataclasses.dataclass -class OverriddenInstructionPlaceHolder: - name: str - - -AnyInstruction = Instruction | MacroInstruction | PseudoInstruction -INSTR_FMT_PREFIX = "INSTR_FMT_" - - -class Analyzer: - """Parse input, analyze it, and write to output.""" - - input_filenames: list[str] - output_filename: str - metadata_filename: str - pymetadata_filename: str - executor_filename: str - errors: int = 0 - emit_line_directives: bool = False - - def __init__( - self, - input_filenames: list[str], - output_filename: str, - metadata_filename: str, - pymetadata_filename: str, - executor_filename: str, - ): - """Read the input file.""" - self.input_filenames = input_filenames - self.output_filename = output_filename - self.metadata_filename = metadata_filename - self.pymetadata_filename = pymetadata_filename - self.executor_filename = executor_filename - - def error(self, msg: str, node: parser.Node) -> None: - lineno = 0 - filename = "" - if context := node.context: - filename = context.owner.filename - # Use line number of first non-comment in the node - for token in context.owner.tokens[context.begin : context.end]: - lineno = token.line - if token.kind != "COMMENT": - break - print(f"{filename}:{lineno}: {msg}", file=sys.stderr) - self.errors += 1 - - everything: list[ - parser.InstDef | parser.Macro | parser.Pseudo | OverriddenInstructionPlaceHolder - ] - instrs: dict[str, Instruction] # Includes ops - macros: dict[str, parser.Macro] - macro_instrs: dict[str, MacroInstruction] - families: dict[str, parser.Family] - pseudos: dict[str, parser.Pseudo] - pseudo_instrs: dict[str, PseudoInstruction] - - def parse(self) -> None: - """Parse the source text. - - We only want the parser to see the stuff between the - begin and end markers. - """ - - self.everything = [] - self.instrs = {} - self.macros = {} - self.families = {} - self.pseudos = {} - - instrs_idx: dict[str, int] = dict() - - for filename in self.input_filenames: - self.parse_file(filename, instrs_idx) - - files = " + ".join(self.input_filenames) - print( - f"Read {len(self.instrs)} instructions/ops, " - f"{len(self.macros)} macros, {len(self.pseudos)} pseudos, " - f"and {len(self.families)} families from {files}", - file=sys.stderr, - ) - - def parse_file(self, filename: str, instrs_idx: dict[str, int]) -> None: - with open(filename) as file: - src = file.read() - - - psr = parser.Parser(src, filename=prettify_filename(filename)) - - # Skip until begin marker - while tkn := psr.next(raw=True): - if tkn.text == BEGIN_MARKER: - break - else: - raise psr.make_syntax_error( - f"Couldn't find {BEGIN_MARKER!r} in {psr.filename}" - ) - start = psr.getpos() - - # Find end marker, then delete everything after it - while tkn := psr.next(raw=True): - if tkn.text == END_MARKER: - break - del psr.tokens[psr.getpos() - 1 :] - - # Parse from start - psr.setpos(start) - thing: parser.InstDef | parser.Macro | parser.Pseudo | parser.Family | None - thing_first_token = psr.peek() - while thing := psr.definition(): - if ws := [w for w in RESERVED_WORDS if variable_used(thing, w)]: - self.error(f"'{ws[0]}' is a reserved word. {RESERVED_WORDS[ws[0]]}", thing) - - match thing: - case parser.InstDef(name=name): - if name in self.instrs: - if not thing.override: - raise psr.make_syntax_error( - f"Duplicate definition of '{name}' @ {thing.context} " - f"previous definition @ {self.instrs[name].inst.context}", - thing_first_token, - ) - self.everything[instrs_idx[name]] = OverriddenInstructionPlaceHolder(name=name) - if name not in self.instrs and thing.override: - raise psr.make_syntax_error( - f"Definition of '{name}' @ {thing.context} is supposed to be " - "an override but no previous definition exists.", - thing_first_token, - ) - self.instrs[name] = Instruction(thing) - instrs_idx[name] = len(self.everything) - self.everything.append(thing) - case parser.Macro(name): - self.macros[name] = thing - self.everything.append(thing) - case parser.Family(name): - self.families[name] = thing - case parser.Pseudo(name): - self.pseudos[name] = thing - self.everything.append(thing) - case _: - typing.assert_never(thing) - if not psr.eof(): - raise psr.make_syntax_error(f"Extra stuff at the end of {filename}") - - def analyze(self) -> None: - """Analyze the inputs. - - Raises SystemExit if there is an error. - """ - self.analyze_macros_and_pseudos() - self.find_predictions() - self.map_families() - self.check_families() - - def find_predictions(self) -> None: - """Find the instructions that need PREDICTED() labels.""" - for instr in self.instrs.values(): - targets: set[str] = set() - for line in instr.block_text: - if m := re.match(RE_PREDICTED, line): - targets.add(m.group(1)) - for target in targets: - if target_instr := self.instrs.get(target): - target_instr.predicted = True - elif target_macro := self.macro_instrs.get(target): - target_macro.predicted = True - else: - self.error( - f"Unknown instruction {target!r} predicted in {instr.name!r}", - instr.inst, # TODO: Use better location - ) - - def map_families(self) -> None: - """Link instruction names back to their family, if they have one.""" - for family in self.families.values(): - for member in [family.name] + family.members: - if member_instr := self.instrs.get(member): - if member_instr.family not in (family, None): - self.error( - f"Instruction {member} is a member of multiple families " - f"({member_instr.family.name}, {family.name}).", - family, - ) - else: - member_instr.family = family - elif not self.macro_instrs.get(member): - self.error( - f"Unknown instruction {member!r} referenced in family {family.name!r}", - family, - ) - - def check_families(self) -> None: - """Check each family: - - - Must have at least 2 members (including head) - - Head and all members must be known instructions - - Head and all members must have the same cache, input and output effects - """ - for family in self.families.values(): - if family.name not in self.macro_instrs and family.name not in self.instrs: - self.error( - f"Family {family.name!r} has unknown instruction {family.name!r}", - family, - ) - members = [ - member - for member in family.members - if member in self.instrs or member in self.macro_instrs - ] - if members != family.members: - unknown = set(family.members) - set(members) - self.error( - f"Family {family.name!r} has unknown members: {unknown}", family - ) - expected_effects = self.effect_counts(family.name) - for member in members: - member_effects = self.effect_counts(member) - if member_effects != expected_effects: - self.error( - f"Family {family.name!r} has inconsistent " - f"(cache, input, output) effects:\n" - f" {family.name} = {expected_effects}; " - f"{member} = {member_effects}", - family, - ) - - def effect_counts(self, name: str) -> tuple[int, int, int]: - if instr := self.instrs.get(name): - cache = instr.cache_offset - input = len(instr.input_effects) - output = len(instr.output_effects) - elif mac := self.macro_instrs.get(name): - cache = mac.cache_offset - input, output = 0, 0 - for part in mac.parts: - if isinstance(part, Component): - # A component may pop what the previous component pushed, - # so we offset the input/output counts by that. - delta_i = len(part.instr.input_effects) - delta_o = len(part.instr.output_effects) - offset = min(delta_i, output) - input += delta_i - offset - output += delta_o - offset - else: - assert False, f"Unknown instruction {name!r}" - return cache, input, output - - def analyze_macros_and_pseudos(self) -> None: - """Analyze each macro and pseudo instruction.""" - self.macro_instrs = {} - self.pseudo_instrs = {} - for name, macro in self.macros.items(): - self.macro_instrs[name] = self.analyze_macro(macro) - for name, pseudo in self.pseudos.items(): - self.pseudo_instrs[name] = self.analyze_pseudo(pseudo) - - def analyze_macro(self, macro: parser.Macro) -> MacroInstruction: - components = self.check_macro_components(macro) - stack, initial_sp = self.stack_analysis(components) - sp = initial_sp - parts: MacroParts = [] - flags = InstructionFlags.newEmpty() - offset = 0 - for component in components: - match component: - case parser.CacheEffect() as ceffect: - parts.append(ceffect) - offset += ceffect.size - case Instruction() as instr: - part, sp, offset = self.analyze_instruction(instr, stack, sp, offset) - parts.append(part) - flags.add(instr.instr_flags) - case _: - typing.assert_never(component) - final_sp = sp - format = "IB" - if offset: - format += "C" + "0"*(offset-1) - return MacroInstruction( - macro.name, stack, initial_sp, final_sp, format, flags, macro, parts, offset - ) - - def analyze_pseudo(self, pseudo: parser.Pseudo) -> PseudoInstruction: - targets = [self.instrs[target] for target in pseudo.targets] - assert targets - # Make sure the targets have the same fmt - fmts = list(set([t.instr_fmt for t in targets])) - assert(len(fmts) == 1) - assert(len(list(set([t.instr_flags.bitmap() for t in targets]))) == 1) - return PseudoInstruction(pseudo.name, targets, fmts[0], targets[0].instr_flags) - - def analyze_instruction( - self, instr: Instruction, stack: list[StackEffect], sp: int, offset: int - ) -> tuple[Component, int, int]: - input_mapping: StackEffectMapping = [] - for ieffect in reversed(instr.input_effects): - sp -= 1 - input_mapping.append((stack[sp], ieffect)) - output_mapping: StackEffectMapping = [] - for oeffect in instr.output_effects: - output_mapping.append((stack[sp], oeffect)) - sp += 1 - active_effects: list[ActiveCacheEffect] = [] - for ceffect in instr.cache_effects: - if ceffect.name != UNUSED: - active_effects.append(ActiveCacheEffect(ceffect, offset)) - offset += ceffect.size - return Component(instr, input_mapping, output_mapping, active_effects), sp, offset - - def check_macro_components( - self, macro: parser.Macro - ) -> list[InstructionOrCacheEffect]: - components: list[InstructionOrCacheEffect] = [] - for uop in macro.uops: - match uop: - case parser.OpName(name): - if name not in self.instrs: - self.error(f"Unknown instruction {name!r}", macro) - components.append(self.instrs[name]) - case parser.CacheEffect(): - components.append(uop) - case _: - typing.assert_never(uop) - return components - - def stack_analysis( - self, components: typing.Iterable[InstructionOrCacheEffect] - ) -> tuple[list[StackEffect], int]: - """Analyze a macro. - - Ignore cache effects. - - Return the list of variables (as StackEffects) and the initial stack pointer. - """ - lowest = current = highest = 0 - conditions: dict[int, str] = {} # Indexed by 'current'. - last_instr: Instruction | None = None - for thing in components: - if isinstance(thing, Instruction): - last_instr = thing - for thing in components: - match thing: - case Instruction() as instr: - if any( - eff.size for eff in instr.input_effects + instr.output_effects - ): - # TODO: Eventually this will be needed, at least for macros. - self.error( - f"Instruction {instr.name!r} has variable-sized stack effect, " - "which are not supported in macro instructions", - instr.inst, # TODO: Pass name+location of macro - ) - if any(eff.cond for eff in instr.input_effects): - self.error( - f"Instruction {instr.name!r} has conditional input stack effect, " - "which are not supported in macro instructions", - instr.inst, # TODO: Pass name+location of macro - ) - if any(eff.cond for eff in instr.output_effects) and instr is not last_instr: - self.error( - f"Instruction {instr.name!r} has conditional output stack effect, " - "but is not the last instruction in a macro", - instr.inst, # TODO: Pass name+location of macro - ) - current -= len(instr.input_effects) - lowest = min(lowest, current) - for eff in instr.output_effects: - if eff.cond: - conditions[current] = eff.cond - current += 1 - highest = max(highest, current) - case parser.CacheEffect(): - pass - case _: - typing.assert_never(thing) - # At this point, 'current' is the net stack effect, - # and 'lowest' and 'highest' are the extremes. - # Note that 'lowest' may be negative. - stack = [ - StackEffect(f"_tmp_{i}", "", conditions.get(highest - i, "")) - for i in reversed(range(1, highest - lowest + 1)) - ] - return stack, -lowest - +class Generator(Analyzer): def get_stack_effect_info( - self, thing: parser.InstDef | parser.Macro | parser.Pseudo + self, thing: parsing.InstDef | parsing.Macro | parsing.Pseudo ) -> tuple[AnyInstruction | None, str | None, str | None]: def effect_str(effects: list[StackEffect]) -> str: n_effect, sym_effect = list_effect_size(effects) @@ -1053,8 +104,10 @@ def effect_str(effects: list[StackEffect]) -> str: return str(n_effect) instr: AnyInstruction | None + popped: str | None + pushed: str | None match thing: - case parser.InstDef(): + case parsing.InstDef(): if thing.kind != "op": instr = self.instrs[thing.name] popped = effect_str(instr.input_effects) @@ -1063,7 +116,7 @@ def effect_str(effects: list[StackEffect]) -> str: instr = None popped = "" pushed = "" - case parser.Macro(): + case parsing.Macro(): instr = self.macro_instrs[thing.name] parts = [comp for comp in instr.parts if isinstance(comp, Component)] # Note: stack_analysis() already verifies that macro components @@ -1084,7 +137,11 @@ def effect_str(effects: list[StackEffect]) -> str: if effect.cond in ("0", "1"): pushed_symbolic.append(effect.cond) else: - pushed_symbolic.append(maybe_parenthesize(f"{maybe_parenthesize(effect.cond)} ? 1 : 0")) + pushed_symbolic.append( + maybe_parenthesize( + f"{maybe_parenthesize(effect.cond)} ? 1 : 0" + ) + ) sp += 1 high = max(sp, high) if high != max(0, sp): @@ -1096,7 +153,7 @@ def effect_str(effects: list[StackEffect]) -> str: popped = str(-low) pushed_symbolic.append(str(sp - low - len(pushed_symbolic))) pushed = " + ".join(pushed_symbolic) - case parser.Pseudo(): + case parsing.Pseudo(): instr = self.pseudo_instrs[thing.name] popped = pushed = None # Calculate stack effect, and check that it's the the same @@ -1135,10 +192,14 @@ def write_function( ) -> None: self.out.emit("") self.out.emit("#ifndef NEED_OPCODE_METADATA") - self.out.emit(f"extern int _PyOpcode_num_{direction}(int opcode, int oparg, bool jump);") + self.out.emit( + f"extern int _PyOpcode_num_{direction}(int opcode, int oparg, bool jump);" + ) self.out.emit("#else") self.out.emit("int") - self.out.emit(f"_PyOpcode_num_{direction}(int opcode, int oparg, bool jump) {{") + self.out.emit( + f"_PyOpcode_num_{direction}(int opcode, int oparg, bool jump) {{" + ) self.out.emit(" switch(opcode) {") for instr, effect in data: self.out.emit(f" case {instr.name}:") @@ -1159,7 +220,7 @@ def from_source_files(self) -> str: try: filename = os.path.relpath(filename, ROOT) except ValueError: - # May happen on Windows if root and temp on different volumes + # May happen on Windows if root and temp on different volumes pass filenames.append(filename) paths = f"\n{self.out.comment} ".join(filenames) @@ -1170,20 +231,21 @@ def write_provenance_header(self): self.out.write_raw(self.from_source_files()) self.out.write_raw(f"{self.out.comment} Do not edit!\n") - def write_metadata(self) -> None: + def write_metadata(self, metadata_filename: str, pymetadata_filename: str) -> None: """Write instruction metadata to output file.""" # Compute the set of all instruction formats. all_formats: set[str] = set() for thing in self.everything: + format: str | None match thing: case OverriddenInstructionPlaceHolder(): continue - case parser.InstDef(): + case parsing.InstDef(): format = self.instrs[thing.name].instr_fmt - case parser.Macro(): + case parsing.Macro(): format = self.macro_instrs[thing.name].instr_fmt - case parser.Pseudo(): + case parsing.Pseudo(): format = None for target in self.pseudos[thing.name].targets: target_instr = self.instrs.get(target) @@ -1192,13 +254,14 @@ def write_metadata(self) -> None: format = target_instr.instr_fmt else: assert format == target_instr.instr_fmt + assert format is not None case _: typing.assert_never(thing) all_formats.add(format) # Turn it into a list of enum definitions. format_enums = [INSTR_FMT_PREFIX + format for format in sorted(all_formats)] - with open(self.metadata_filename, "w") as f: + with open(metadata_filename, "w") as f: # Create formatter self.out = Formatter(f, 0) @@ -1220,7 +283,8 @@ def write_metadata(self) -> None: self.out.emit( "#define IS_VALID_OPCODE(OP) \\\n" " (((OP) >= 0) && ((OP) < OPCODE_METADATA_SIZE) && \\\n" - " (_PyOpcode_opcode_metadata[(OP)].valid_entry))") + " (_PyOpcode_opcode_metadata[(OP)].valid_entry))" + ) self.out.emit("") InstructionFlags.emit_macros(self.out) @@ -1234,17 +298,23 @@ def write_metadata(self) -> None: with self.out.block("struct opcode_macro_expansion", ";"): self.out.emit("int nuops;") - self.out.emit("struct { int16_t uop; int8_t size; int8_t offset; } uops[8];") + self.out.emit( + "struct { int16_t uop; int8_t size; int8_t offset; } uops[8];" + ) self.out.emit("") for key, value in OPARG_SIZES.items(): self.out.emit(f"#define {key} {value}") self.out.emit("") - self.out.emit("#define OPCODE_METADATA_FMT(OP) " - "(_PyOpcode_opcode_metadata[(OP)].instr_format)") + self.out.emit( + "#define OPCODE_METADATA_FMT(OP) " + "(_PyOpcode_opcode_metadata[(OP)].instr_format)" + ) self.out.emit("#define SAME_OPCODE_METADATA(OP1, OP2) \\") - self.out.emit(" (OPCODE_METADATA_FMT(OP1) == OPCODE_METADATA_FMT(OP2))") + self.out.emit( + " (OPCODE_METADATA_FMT(OP1) == OPCODE_METADATA_FMT(OP2))" + ) self.out.emit("") # Write metadata array declaration @@ -1253,27 +323,35 @@ def write_metadata(self) -> None: self.out.emit("#define OPCODE_MACRO_EXPANSION_SIZE 256") self.out.emit("") self.out.emit("#ifndef NEED_OPCODE_METADATA") - self.out.emit("extern const struct opcode_metadata " - "_PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE];") - self.out.emit("extern const struct opcode_macro_expansion " - "_PyOpcode_macro_expansion[OPCODE_MACRO_EXPANSION_SIZE];") - self.out.emit("extern const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE];") + self.out.emit( + "extern const struct opcode_metadata " + "_PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE];" + ) + self.out.emit( + "extern const struct opcode_macro_expansion " + "_PyOpcode_macro_expansion[OPCODE_MACRO_EXPANSION_SIZE];" + ) + self.out.emit( + "extern const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE];" + ) self.out.emit("#else // if NEED_OPCODE_METADATA") - self.out.emit("const struct opcode_metadata " - "_PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE] = {") + self.out.emit( + "const struct opcode_metadata " + "_PyOpcode_opcode_metadata[OPCODE_METADATA_SIZE] = {" + ) # Write metadata for each instruction for thing in self.everything: match thing: case OverriddenInstructionPlaceHolder(): continue - case parser.InstDef(): + case parsing.InstDef(): if thing.kind != "op": self.write_metadata_for_inst(self.instrs[thing.name]) - case parser.Macro(): + case parsing.Macro(): self.write_metadata_for_macro(self.macro_instrs[thing.name]) - case parser.Pseudo(): + case parsing.Pseudo(): self.write_metadata_for_pseudo(self.pseudo_instrs[thing.name]) case _: typing.assert_never(thing) @@ -1291,32 +369,38 @@ def write_metadata(self) -> None: match thing: case OverriddenInstructionPlaceHolder(): pass - case parser.InstDef(name=name): + case parsing.InstDef(name=name): instr = self.instrs[name] # Since an 'op' is not a bytecode, it has no expansion; but 'inst' is if instr.kind == "inst" and instr.is_viable_uop(): # Construct a dummy Component -- input/output mappings are not used part = Component(instr, [], [], instr.active_caches) self.write_macro_expansions(instr.name, [part]) - elif instr.kind == "inst" and variable_used(instr.inst, "oparg1"): - assert variable_used(instr.inst, "oparg2"), "Half super-instr?" + elif instr.kind == "inst" and variable_used( + instr.inst, "oparg1" + ): + assert variable_used( + instr.inst, "oparg2" + ), "Half super-instr?" self.write_super_expansions(instr.name) - case parser.Macro(): + case parsing.Macro(): mac = self.macro_instrs[thing.name] self.write_macro_expansions(mac.name, mac.parts) - case parser.Pseudo(): + case parsing.Pseudo(): pass case _: typing.assert_never(thing) - with self.out.block("const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE] =", ";"): - self.write_uop_items(lambda name, counter: f"[{name}] = \"{name}\",") + with self.out.block( + "const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE] =", ";" + ): + self.write_uop_items(lambda name, counter: f'[{name}] = "{name}",') self.out.emit("#endif // NEED_OPCODE_METADATA") - with open(self.pymetadata_filename, "w") as f: + with open(pymetadata_filename, "w") as f: # Create formatter - self.out = Formatter(f, 0, comment = "#") + self.out = Formatter(f, 0, comment="#") self.write_provenance_header() @@ -1324,10 +408,10 @@ def write_metadata(self) -> None: self.out.emit("_specializations = {") for name, family in self.families.items(): with self.out.indent(): - self.out.emit(f"\"{family.name}\": [") + self.out.emit(f'"{family.name}": [') with self.out.indent(): for m in family.members: - self.out.emit(f"\"{m}\",") + self.out.emit(f'"{m}",') self.out.emit(f"],") self.out.emit("}") @@ -1335,15 +419,17 @@ def write_metadata(self) -> None: self.out.emit("") self.out.emit("# An irregular case:") self.out.emit( - "_specializations[\"BINARY_OP\"].append(" - "\"BINARY_OP_INPLACE_ADD_UNICODE\")") + '_specializations["BINARY_OP"].append(' + '"BINARY_OP_INPLACE_ADD_UNICODE")' + ) # Make list of specialized instructions self.out.emit("") self.out.emit( "_specialized_instructions = [" - "opcode for family in _specializations.values() for opcode in family" - "]") + "opcode for family in _specializations.values() for opcode in family" + "]" + ) def write_pseudo_instrs(self) -> None: """Write the IS_PSEUDO_INSTR macro""" @@ -1432,16 +518,18 @@ def write_super_expansions(self, name: str) -> None: ] self.write_expansions(name, expansions) - def write_expansions(self, name: str, expansions: list[tuple[str, int, int]]) -> None: - pieces = [f"{{ {name}, {size}, {offset} }}" for name, size, offset in expansions] + def write_expansions( + self, name: str, expansions: list[tuple[str, int, int]] + ) -> None: + pieces = [ + f"{{ {name}, {size}, {offset} }}" for name, size, offset in expansions + ] self.out.emit( f"[{name}] = " f"{{ .nuops = {len(pieces)}, .uops = {{ {', '.join(pieces)} }} }}," ) - def emit_metadata_entry( - self, name: str, fmt: str, flags: InstructionFlags - ) -> None: + def emit_metadata_entry(self, name: str, fmt: str, flags: InstructionFlags) -> None: flag_names = flags.names(value=True) if not flag_names: flag_names.append("0") @@ -1462,11 +550,13 @@ def write_metadata_for_pseudo(self, ps: PseudoInstruction) -> None: """Write metadata for a macro-instruction.""" self.emit_metadata_entry(ps.name, ps.instr_fmt, ps.instr_flags) - def write_instructions(self) -> None: + def write_instructions( + self, output_filename: str, emit_line_directives: bool + ) -> None: """Write instructions to output file.""" - with open(self.output_filename, "w") as f: + with open(output_filename, "w") as f: # Create formatter - self.out = Formatter(f, 8, self.emit_line_directives) + self.out = Formatter(f, 8, emit_line_directives) self.write_provenance_header() @@ -1478,35 +568,37 @@ def write_instructions(self) -> None: match thing: case OverriddenInstructionPlaceHolder(): self.write_overridden_instr_place_holder(thing) - case parser.InstDef(): + case parsing.InstDef(): if thing.kind != "op": n_instrs += 1 self.write_instr(self.instrs[thing.name]) - case parser.Macro(): + case parsing.Macro(): n_macros += 1 self.write_macro(self.macro_instrs[thing.name]) - case parser.Pseudo(): + case parsing.Pseudo(): n_pseudos += 1 case _: typing.assert_never(thing) print( f"Wrote {n_instrs} instructions, {n_macros} macros, " - f"and {n_pseudos} pseudos to {self.output_filename}", + f"and {n_pseudos} pseudos to {output_filename}", file=sys.stderr, ) - def write_executor_instructions(self) -> None: + def write_executor_instructions( + self, executor_filename: str, emit_line_directives: bool + ) -> None: """Generate cases for the Tier 2 interpreter.""" - with open(self.executor_filename, "w") as f: - self.out = Formatter(f, 8, self.emit_line_directives) + with open(executor_filename, "w") as f: + self.out = Formatter(f, 8, emit_line_directives) self.write_provenance_header() for thing in self.everything: match thing: case OverriddenInstructionPlaceHolder(): # TODO: Is this helpful? self.write_overridden_instr_place_holder(thing) - case parser.InstDef(): + case parsing.InstDef(): instr = self.instrs[thing.name] if instr.is_viable_uop(): self.out.emit("") @@ -1517,22 +609,24 @@ def write_executor_instructions(self) -> None: self.out.emit("break;") # elif instr.kind != "op": # print(f"NOTE: {thing.name} is not a viable uop") - case parser.Macro(): + case parsing.Macro(): pass - case parser.Pseudo(): + case parsing.Pseudo(): pass case _: typing.assert_never(thing) print( - f"Wrote some stuff to {self.executor_filename}", + f"Wrote some stuff to {executor_filename}", file=sys.stderr, ) - def write_overridden_instr_place_holder(self, - place_holder: OverriddenInstructionPlaceHolder) -> None: + def write_overridden_instr_place_holder( + self, place_holder: OverriddenInstructionPlaceHolder + ) -> None: self.out.emit("") self.out.emit( - f"{self.out.comment} TARGET({place_holder.name}) overridden by later definition") + f"{self.out.comment} TARGET({place_holder.name}) overridden by later definition" + ) def write_instr(self, instr: Instruction) -> None: name = instr.name @@ -1555,7 +649,7 @@ def write_macro(self, mac: MacroInstruction) -> None: cache_adjust = 0 for part in mac.parts: match part: - case parser.CacheEffect(size=size): + case parsing.CacheEffect(size=size): cache_adjust += size case Component() as comp: last_instr = comp.instr @@ -1603,7 +697,7 @@ def wrap_macro(self, mac: MacroInstruction): yield - self.out.stack_adjust(ieffects[:mac.initial_sp], mac.stack[:mac.final_sp]) + self.out.stack_adjust(ieffects[: mac.initial_sp], mac.stack[: mac.final_sp]) for i, var in enumerate(reversed(mac.stack[: mac.final_sp]), 1): dst = StackEffect(f"stack_pointer[-{i}]", "") @@ -1612,99 +706,6 @@ def wrap_macro(self, mac: MacroInstruction): self.out.emit(f"DISPATCH();") -def prettify_filename(filename: str) -> str: - # Make filename more user-friendly and less platform-specific, - # it is only used for error reporting at this point. - filename = filename.replace("\\", "/") - if filename.startswith("./"): - filename = filename[2:] - if filename.endswith(".new"): - filename = filename[:-4] - return filename - - -def extract_block_text(block: parser.Block) -> tuple[list[str], bool, int]: - # Get lines of text with proper dedent - blocklines = block.text.splitlines(True) - first_token: lx.Token = block.tokens[0] # IndexError means the context is broken - block_line = first_token.begin[0] - - # Remove blank lines from both ends - while blocklines and not blocklines[0].strip(): - blocklines.pop(0) - block_line += 1 - while blocklines and not blocklines[-1].strip(): - blocklines.pop() - - # Remove leading and trailing braces - assert blocklines and blocklines[0].strip() == "{" - assert blocklines and blocklines[-1].strip() == "}" - blocklines.pop() - blocklines.pop(0) - block_line += 1 - - # Remove trailing blank lines - while blocklines and not blocklines[-1].strip(): - blocklines.pop() - - # Separate CHECK_EVAL_BREAKER() macro from end - check_eval_breaker = \ - blocklines != [] and blocklines[-1].strip() == "CHECK_EVAL_BREAKER();" - if check_eval_breaker: - del blocklines[-1] - - return blocklines, check_eval_breaker, block_line - - -def always_exits(lines: list[str]) -> bool: - """Determine whether a block always ends in a return/goto/etc.""" - if not lines: - return False - line = lines[-1].rstrip() - # Indent must match exactly (TODO: Do something better) - if line[:12] != " " * 12: - return False - line = line[12:] - return line.startswith( - ( - "goto ", - "return ", - "DISPATCH", - "GO_TO_", - "Py_UNREACHABLE()", - "ERROR_IF(true, ", - ) - ) - - -def variable_used(node: parser.Node, name: str) -> bool: - """Determine whether a variable with a given name is used in a node.""" - return any( - token.kind == "IDENTIFIER" and token.text == name for token in node.tokens - ) - - -def variable_used_unspecialized(node: parser.Node, name: str) -> bool: - """Like variable_used(), but skips #if ENABLE_SPECIALIZATION blocks.""" - tokens: list[lx.Token] = [] - skipping = False - for i, token in enumerate(node.tokens): - if token.kind == "MACRO": - text = "".join(token.text.split()) - # TODO: Handle nested #if - if text == "#if": - if ( - i + 1 < len(node.tokens) - and node.tokens[i + 1].text == "ENABLE_SPECIALIZATION" - ): - skipping = True - elif text in ("#else", "#endif"): - skipping = False - if not skipping: - tokens.append(token) - return any(token.kind == "IDENTIFIER" and token.text == name for token in tokens) - - def main(): """Parse command line, parse input, analyze, write output.""" args = arg_parser.parse_args() # Prints message and sys.exit(2) on error @@ -1712,17 +713,17 @@ def main(): args.input.append(DEFAULT_INPUT) # Raises OSError if input unreadable - a = Analyzer(args.input, args.output, args.metadata, args.pymetadata, args.executor_cases) + a = Generator(args.input) - if args.emit_line_directives: - a.emit_line_directives = True a.parse() # Raises SyntaxError on failure a.analyze() # Prints messages and sets a.errors on failure if a.errors: sys.exit(f"Found {a.errors} errors") - a.write_instructions() # Raises OSError if output can't be written - a.write_metadata() - a.write_executor_instructions() + + # These raise OSError if output can't be written + a.write_instructions(args.output, args.emit_line_directives) + a.write_metadata(args.metadata, args.pymetadata) + a.write_executor_instructions(args.executor_cases, args.emit_line_directives) if __name__ == "__main__": diff --git a/Tools/cases_generator/instructions.py b/Tools/cases_generator/instructions.py new file mode 100644 index 0000000000000..6f42699d900b4 --- /dev/null +++ b/Tools/cases_generator/instructions.py @@ -0,0 +1,424 @@ +import dataclasses +import re +import typing + +from flags import InstructionFlags, variable_used_unspecialized +from formatting import ( + Formatter, + UNUSED, + string_effect_size, + list_effect_size, + maybe_parenthesize, +) +import lexer as lx +import parsing +from parsing import StackEffect + +BITS_PER_CODE_UNIT = 16 + + + at dataclasses.dataclass +class ActiveCacheEffect: + """Wraps a CacheEffect that is actually used, in context.""" + + effect: parsing.CacheEffect + offset: int + + +FORBIDDEN_NAMES_IN_UOPS = ( + "resume_with_error", + "kwnames", + "next_instr", + "oparg1", # Proxy for super-instructions like LOAD_FAST_LOAD_FAST + "JUMPBY", + "DISPATCH", + "INSTRUMENTED_JUMP", + "throwflag", + "exception_unwind", + "import_from", + "import_name", + "_PyObject_CallNoArgs", # Proxy for BEFORE_WITH +) + + +# Interpreter tiers +TIER_ONE: typing.Final = 1 # Specializing adaptive interpreter (PEP 659) +TIER_TWO: typing.Final = 2 # Experimental tracing interpreter +Tiers: typing.TypeAlias = typing.Literal[1, 2] + + + at dataclasses.dataclass +class Instruction: + """An instruction with additional data and code.""" + + # Parts of the underlying instruction definition + inst: parsing.InstDef + kind: typing.Literal["inst", "op"] + name: str + block: parsing.Block + block_text: list[str] # Block.text, less curlies, less PREDICT() calls + block_line: int # First line of block in original code + + # Computed by constructor + always_exits: bool + cache_offset: int + cache_effects: list[parsing.CacheEffect] + input_effects: list[StackEffect] + output_effects: list[StackEffect] + unmoved_names: frozenset[str] + instr_fmt: str + instr_flags: InstructionFlags + active_caches: list[ActiveCacheEffect] + + # Set later + family: parsing.Family | None = None + predicted: bool = False + + def __init__(self, inst: parsing.InstDef): + self.inst = inst + self.kind = inst.kind + self.name = inst.name + self.block = inst.block + self.block_text, self.check_eval_breaker, self.block_line = extract_block_text( + self.block + ) + self.always_exits = always_exits(self.block_text) + self.cache_effects = [ + effect for effect in inst.inputs if isinstance(effect, parsing.CacheEffect) + ] + self.cache_offset = sum(c.size for c in self.cache_effects) + self.input_effects = [ + effect for effect in inst.inputs if isinstance(effect, StackEffect) + ] + self.output_effects = inst.outputs # For consistency/completeness + unmoved_names: set[str] = set() + for ieffect, oeffect in zip(self.input_effects, self.output_effects): + if ieffect.name == oeffect.name: + unmoved_names.add(ieffect.name) + else: + break + self.unmoved_names = frozenset(unmoved_names) + + self.instr_flags = InstructionFlags.fromInstruction(inst) + + self.active_caches = [] + offset = 0 + for effect in self.cache_effects: + if effect.name != UNUSED: + self.active_caches.append(ActiveCacheEffect(effect, offset)) + offset += effect.size + + if self.instr_flags.HAS_ARG_FLAG: + fmt = "IB" + else: + fmt = "IX" + if offset: + fmt += "C" + "0" * (offset - 1) + self.instr_fmt = fmt + + def is_viable_uop(self) -> bool: + """Whether this instruction is viable as a uop.""" + dprint: typing.Callable[..., None] = lambda *args, **kwargs: None + # if self.name.startswith("CALL"): + # dprint = print + + if self.name == "EXIT_TRACE": + return True # This has 'return frame' but it's okay + if self.always_exits: + dprint(f"Skipping {self.name} because it always exits") + return False + if len(self.active_caches) > 1: + # print(f"Skipping {self.name} because it has >1 cache entries") + return False + res = True + for forbidden in FORBIDDEN_NAMES_IN_UOPS: + # NOTE: To disallow unspecialized uops, use + # if variable_used(self.inst, forbidden): + if variable_used_unspecialized(self.inst, forbidden): + dprint(f"Skipping {self.name} because it uses {forbidden}") + res = False + return res + + def write(self, out: Formatter, tier: Tiers = TIER_ONE) -> None: + """Write one instruction, sans prologue and epilogue.""" + # Write a static assertion that a family's cache size is correct + if family := self.family: + if self.name == family.name: + if cache_size := family.size: + out.emit( + f"static_assert({cache_size} == " + f'{self.cache_offset}, "incorrect cache size");' + ) + + # Write input stack effect variable declarations and initializations + ieffects = list(reversed(self.input_effects)) + for i, ieffect in enumerate(ieffects): + isize = string_effect_size( + list_effect_size([ieff for ieff in ieffects[: i + 1]]) + ) + if ieffect.size: + src = StackEffect( + f"(stack_pointer - {maybe_parenthesize(isize)})", "PyObject **" + ) + elif ieffect.cond: + src = StackEffect( + f"({ieffect.cond}) ? stack_pointer[-{maybe_parenthesize(isize)}] : NULL", + "", + ) + else: + src = StackEffect(f"stack_pointer[-{maybe_parenthesize(isize)}]", "") + out.declare(ieffect, src) + + # Write output stack effect variable declarations + isize = string_effect_size(list_effect_size(self.input_effects)) + input_names = {ieffect.name for ieffect in self.input_effects} + for i, oeffect in enumerate(self.output_effects): + if oeffect.name not in input_names: + if oeffect.size: + osize = string_effect_size( + list_effect_size([oeff for oeff in self.output_effects[:i]]) + ) + offset = "stack_pointer" + if isize != osize: + if isize != "0": + offset += f" - ({isize})" + if osize != "0": + offset += f" + {osize}" + src = StackEffect(offset, "PyObject **") + out.declare(oeffect, src) + else: + out.declare(oeffect, None) + + # out.emit(f"next_instr += OPSIZE({self.inst.name}) - 1;") + + self.write_body(out, 0, self.active_caches, tier=tier) + + # Skip the rest if the block always exits + if self.always_exits: + return + + # Write net stack growth/shrinkage + out.stack_adjust( + [ieff for ieff in self.input_effects], + [oeff for oeff in self.output_effects], + ) + + # Write output stack effect assignments + oeffects = list(reversed(self.output_effects)) + for i, oeffect in enumerate(oeffects): + if oeffect.name in self.unmoved_names: + continue + osize = string_effect_size( + list_effect_size([oeff for oeff in oeffects[: i + 1]]) + ) + if oeffect.size: + dst = StackEffect( + f"stack_pointer - {maybe_parenthesize(osize)}", "PyObject **" + ) + else: + dst = StackEffect(f"stack_pointer[-{maybe_parenthesize(osize)}]", "") + out.assign(dst, oeffect) + + # Write cache effect + if tier == TIER_ONE and self.cache_offset: + out.emit(f"next_instr += {self.cache_offset};") + + def write_body( + self, + out: Formatter, + dedent: int, + active_caches: list[ActiveCacheEffect], + tier: Tiers = TIER_ONE, + ) -> None: + """Write the instruction body.""" + # Write cache effect variable declarations and initializations + for active in active_caches: + ceffect = active.effect + bits = ceffect.size * BITS_PER_CODE_UNIT + if bits == 64: + # NOTE: We assume that 64-bit data in the cache + # is always an object pointer. + # If this becomes false, we need a way to specify + # syntactically what type the cache data is. + typ = "PyObject *" + func = "read_obj" + else: + typ = f"uint{bits}_t " + func = f"read_u{bits}" + if tier == TIER_ONE: + out.emit( + f"{typ}{ceffect.name} = {func}(&next_instr[{active.offset}].cache);" + ) + else: + out.emit(f"{typ}{ceffect.name} = ({typ.strip()})operand;") + + # Write the body, substituting a goto for ERROR_IF() and other stuff + assert dedent <= 0 + extra = " " * -dedent + names_to_skip = self.unmoved_names | frozenset({UNUSED, "null"}) + offset = 0 + context = self.block.context + assert context is not None and context.owner is not None + filename = context.owner.filename + for line in self.block_text: + out.set_lineno(self.block_line + offset, filename) + offset += 1 + if m := re.match(r"(\s*)ERROR_IF\((.+), (\w+)\);\s*(?://.*)?$", line): + space, cond, label = m.groups() + space = extra + space + # ERROR_IF() must pop the inputs from the stack. + # The code block is responsible for DECREF()ing them. + # NOTE: If the label doesn't exist, just add it to ceval.c. + + # Don't pop common input/output effects at the bottom! + # These aren't DECREF'ed so they can stay. + ieffs = list(self.input_effects) + oeffs = list(self.output_effects) + while ieffs and oeffs and ieffs[0] == oeffs[0]: + ieffs.pop(0) + oeffs.pop(0) + ninputs, symbolic = list_effect_size(ieffs) + if ninputs: + label = f"pop_{ninputs}_{label}" + if symbolic: + out.write_raw( + f"{space}if ({cond}) {{ STACK_SHRINK({symbolic}); goto {label}; }}\n" + ) + else: + out.write_raw(f"{space}if ({cond}) goto {label};\n") + elif m := re.match(r"(\s*)DECREF_INPUTS\(\);\s*(?://.*)?$", line): + out.reset_lineno() + space = extra + m.group(1) + for ieff in self.input_effects: + if ieff.name in names_to_skip: + continue + if ieff.size: + out.write_raw( + f"{space}for (int _i = {ieff.size}; --_i >= 0;) {{\n" + ) + out.write_raw(f"{space} Py_DECREF({ieff.name}[_i]);\n") + out.write_raw(f"{space}}}\n") + else: + decref = "XDECREF" if ieff.cond else "DECREF" + out.write_raw(f"{space}Py_{decref}({ieff.name});\n") + else: + out.write_raw(extra + line) + out.reset_lineno() + + +InstructionOrCacheEffect = Instruction | parsing.CacheEffect +StackEffectMapping = list[tuple[StackEffect, StackEffect]] + + + at dataclasses.dataclass +class Component: + instr: Instruction + input_mapping: StackEffectMapping + output_mapping: StackEffectMapping + active_caches: list[ActiveCacheEffect] + + def write_body(self, out: Formatter) -> None: + with out.block(""): + input_names = {ieffect.name for _, ieffect in self.input_mapping} + for var, ieffect in self.input_mapping: + out.declare(ieffect, var) + for _, oeffect in self.output_mapping: + if oeffect.name not in input_names: + out.declare(oeffect, None) + + self.instr.write_body(out, -4, self.active_caches) + + for var, oeffect in self.output_mapping: + out.assign(var, oeffect) + + +MacroParts = list[Component | parsing.CacheEffect] + + + at dataclasses.dataclass +class MacroInstruction: + """A macro instruction.""" + + name: str + stack: list[StackEffect] + initial_sp: int + final_sp: int + instr_fmt: str + instr_flags: InstructionFlags + macro: parsing.Macro + parts: MacroParts + cache_offset: int + predicted: bool = False + + + at dataclasses.dataclass +class PseudoInstruction: + """A pseudo instruction.""" + + name: str + targets: list[Instruction] + instr_fmt: str + instr_flags: InstructionFlags + + + at dataclasses.dataclass +class OverriddenInstructionPlaceHolder: + name: str + + +AnyInstruction = Instruction | MacroInstruction | PseudoInstruction + + +def extract_block_text(block: parsing.Block) -> tuple[list[str], bool, int]: + # Get lines of text with proper dedent + blocklines = block.text.splitlines(True) + first_token: lx.Token = block.tokens[0] # IndexError means the context is broken + block_line = first_token.begin[0] + + # Remove blank lines from both ends + while blocklines and not blocklines[0].strip(): + blocklines.pop(0) + block_line += 1 + while blocklines and not blocklines[-1].strip(): + blocklines.pop() + + # Remove leading and trailing braces + assert blocklines and blocklines[0].strip() == "{" + assert blocklines and blocklines[-1].strip() == "}" + blocklines.pop() + blocklines.pop(0) + block_line += 1 + + # Remove trailing blank lines + while blocklines and not blocklines[-1].strip(): + blocklines.pop() + + # Separate CHECK_EVAL_BREAKER() macro from end + check_eval_breaker = ( + blocklines != [] and blocklines[-1].strip() == "CHECK_EVAL_BREAKER();" + ) + if check_eval_breaker: + del blocklines[-1] + + return blocklines, check_eval_breaker, block_line + + +def always_exits(lines: list[str]) -> bool: + """Determine whether a block always ends in a return/goto/etc.""" + if not lines: + return False + line = lines[-1].rstrip() + # Indent must match exactly (TODO: Do something better) + if line[:12] != " " * 12: + return False + line = line[12:] + return line.startswith( + ( + "goto ", + "return ", + "DISPATCH", + "GO_TO_", + "Py_UNREACHABLE()", + "ERROR_IF(true, ", + ) + ) diff --git a/Tools/cases_generator/parser.py b/Tools/cases_generator/parsing.py similarity index 96% rename from Tools/cases_generator/parser.py rename to Tools/cases_generator/parsing.py index ac77e7eae81ad..290285dc03123 100644 --- a/Tools/cases_generator/parser.py +++ b/Tools/cases_generator/parsing.py @@ -1,7 +1,7 @@ """Parser for bytecodes.inst.""" from dataclasses import dataclass, field -from typing import NamedTuple, Callable, TypeVar, Literal +from typing import NamedTuple, Callable, TypeVar, Literal, cast import lexer as lx from plexer import PLexer @@ -19,7 +19,7 @@ def contextual_wrapper(self: P) -> N | None: res = func(self) if res is None: self.setpos(begin) - return + return None end = self.getpos() res.context = Context(begin, end, self) return res @@ -147,6 +147,7 @@ def definition(self) -> InstDef | Macro | Pseudo | Family | None: return family if pseudo := self.pseudo_def(): return pseudo + return None @contextual def inst_def(self) -> InstDef | None: @@ -166,7 +167,8 @@ def inst_header(self) -> InstHeader | None: # TODO: Make INST a keyword in the lexer. override = bool(self.expect(lx.OVERRIDE)) register = bool(self.expect(lx.REGISTER)) - if (tkn := self.expect(lx.IDENTIFIER)) and (kind := tkn.text) in ("inst", "op"): + if (tkn := self.expect(lx.IDENTIFIER)) and tkn.text in ("inst", "op"): + kind = cast(Literal["inst", "op"], tkn.text) if self.expect(lx.LPAREN) and (tkn := self.expect(lx.IDENTIFIER)): name = tkn.text if self.expect(lx.COMMA): @@ -190,6 +192,7 @@ def inputs(self) -> list[InputEffect] | None: # input (',' input)* here = self.getpos() if inp := self.input(): + inp = cast(InputEffect, inp) near = self.getpos() if self.expect(lx.COMMA): if rest := self.inputs(): @@ -232,6 +235,7 @@ def cache_effect(self) -> CacheEffect | None: raise self.make_syntax_error(f"Expected integer, got {num!r}") else: return CacheEffect(tkn.text, size) + return None @contextual def stack_effect(self) -> StackEffect | None: @@ -258,6 +262,7 @@ def stack_effect(self) -> StackEffect | None: type_text = "PyObject **" size_text = size.text.strip() return StackEffect(tkn.text, type_text, cond_text, size_text) + return None @contextual def expression(self) -> Expression | None: @@ -288,6 +293,7 @@ def expression(self) -> Expression | None: def op(self) -> OpName | None: if tkn := self.expect(lx.IDENTIFIER): return OpName(tkn.text) + return None @contextual def macro_def(self) -> Macro | None: @@ -300,16 +306,20 @@ def macro_def(self) -> Macro | None: self.require(lx.SEMI) res = Macro(tkn.text, uops) return res + return None def uops(self) -> list[UOp] | None: if uop := self.uop(): + uop = cast(UOp, uop) uops = [uop] while self.expect(lx.PLUS): if uop := self.uop(): + uop = cast(UOp, uop) uops.append(uop) else: raise self.make_syntax_error("Expected op name or cache effect") return uops + return None @contextual def uop(self) -> UOp | None: @@ -327,6 +337,7 @@ def uop(self) -> UOp | None: raise self.make_syntax_error("Expected integer") else: return OpName(tkn.text) + return None @contextual def family_def(self) -> Family | None: @@ -385,6 +396,7 @@ def members(self) -> list[str] | None: def block(self) -> Block | None: if self.c_blob(): return Block() + return None def c_blob(self) -> list[lx.Token]: tokens: list[lx.Token] = [] From webhook-mailer at python.org Mon Jul 24 12:39:47 2023 From: webhook-mailer at python.org (hugovk) Date: Mon, 24 Jul 2023 16:39:47 -0000 Subject: [Python-checkins] [3.11] gh-107017: Change Chapter Strings to Texts in the Introduction chapter. (GH-107104) (#107168) Message-ID: https://github.com/python/cpython/commit/8725d36fda2125fd089e6a714ba70089ba7498db commit: 8725d36fda2125fd089e6a714ba70089ba7498db branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-24T16:39:43Z summary: [3.11] gh-107017: Change Chapter Strings to Texts in the Introduction chapter. (GH-107104) (#107168) Co-authored-by: TommyUnreal <45427816+TommyUnreal at users.noreply.github.com> Co-authored-by: Hugo van Kemenade files: M Doc/tutorial/introduction.rst diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index e8b582dfe85d2..24ae720fe2bb6 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -138,16 +138,25 @@ and uses the ``j`` or ``J`` suffix to indicate the imaginary part .. _tut-strings: -Strings -------- +Text +---- -Besides numbers, Python can also manipulate strings, which can be expressed -in several ways. They can be enclosed in single quotes (``'...'``) or -double quotes (``"..."``) with the same result [#]_. ``\`` can be used -to escape quotes:: +Python can manipulate text (represented by type :class:`str`, so-called +"strings") as well as numbers. This includes characters "``!``", words +"``rabbit``", names "``Paris``", sentences "``Got your back.``", etc. +"``Yay! :)``". They can be enclosed in single quotes (``'...'``) or double +quotes (``"..."``) with the same result [#]_. >>> 'spam eggs' # single quotes 'spam eggs' + >>> "Paris rabbit got your back :)! Yay!" # double quotes + 'Paris rabbit got your back :)! Yay!' + >>> '1975' # digits and numerals enclosed in quotes are also strings + '1975' + +To quote a quote, we need to "escape" it, by preceding it with ``\``. +Alternatively, we can use the other type of quotation marks:: + >>> 'doesn\'t' # use \' to escape the single quote... "doesn't" >>> "doesn't" # ...or use double quotes instead @@ -159,23 +168,14 @@ to escape quotes:: >>> '"Isn\'t," they said.' '"Isn\'t," they said.' -In the interactive interpreter, the output string is enclosed in quotes and -special characters are escaped with backslashes. While this might sometimes -look different from the input (the enclosing quotes could change), the two -strings are equivalent. The string is enclosed in double quotes if -the string contains a single quote and no double quotes, otherwise it is -enclosed in single quotes. The :func:`print` function produces a more -readable output, by omitting the enclosing quotes and by printing escaped -and special characters:: +In the Python shell, the string definition and output string can look +different. The :func:`print` function produces a more readable output, by +omitting the enclosing quotes and by printing escaped and special characters:: - >>> '"Isn\'t," they said.' - '"Isn\'t," they said.' - >>> print('"Isn\'t," they said.') - "Isn't," they said. >>> s = 'First line.\nSecond line.' # \n means newline - >>> s # without print(), \n is included in the output + >>> s # without print(), special characters are included in the string 'First line.\nSecond line.' - >>> print(s) # with print(), \n produces a new line + >>> print(s) # with print(), special characters are interpreted, so \n produces new line First line. Second line. From webhook-mailer at python.org Mon Jul 24 14:26:33 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 18:26:33 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyUnicode C API (#107185) Message-ID: https://github.com/python/cpython/commit/d27eb1e406a8789d9eaba6dbfed5c2e5abe294fd commit: d27eb1e406a8789d9eaba6dbfed5c2e5abe294fd branch: main author: Victor Stinner committer: vstinner date: 2023-07-24T18:26:29Z summary: gh-106320: Remove private _PyUnicode C API (#107185) Move private _PyUnicode functions to the internal C API (pycore_unicodeobject.h): * _PyUnicode_IsCaseIgnorable() * _PyUnicode_IsCased() * _PyUnicode_IsXidContinue() * _PyUnicode_IsXidStart() * _PyUnicode_ToFoldedFull() * _PyUnicode_ToLowerFull() * _PyUnicode_ToTitleFull() * _PyUnicode_ToUpperFull() No longer export these functions. files: M Include/cpython/unicodeobject.h M Include/internal/pycore_unicodeobject.h diff --git a/Include/cpython/unicodeobject.h b/Include/cpython/unicodeobject.h index 51eb998db4ab6..fa00b350a799f 100644 --- a/Include/cpython/unicodeobject.h +++ b/Include/cpython/unicodeobject.h @@ -473,14 +473,6 @@ PyAPI_FUNC(int) _PyUnicode_IsTitlecase( Py_UCS4 ch /* Unicode character */ ); -PyAPI_FUNC(int) _PyUnicode_IsXidStart( - Py_UCS4 ch /* Unicode character */ - ); - -PyAPI_FUNC(int) _PyUnicode_IsXidContinue( - Py_UCS4 ch /* Unicode character */ - ); - PyAPI_FUNC(int) _PyUnicode_IsWhitespace( const Py_UCS4 ch /* Unicode character */ ); @@ -501,34 +493,6 @@ PyAPI_FUNC(Py_UCS4) _PyUnicode_ToTitlecase( Py_UCS4 ch /* Unicode character */ ); -PyAPI_FUNC(int) _PyUnicode_ToLowerFull( - Py_UCS4 ch, /* Unicode character */ - Py_UCS4 *res - ); - -PyAPI_FUNC(int) _PyUnicode_ToTitleFull( - Py_UCS4 ch, /* Unicode character */ - Py_UCS4 *res - ); - -PyAPI_FUNC(int) _PyUnicode_ToUpperFull( - Py_UCS4 ch, /* Unicode character */ - Py_UCS4 *res - ); - -PyAPI_FUNC(int) _PyUnicode_ToFoldedFull( - Py_UCS4 ch, /* Unicode character */ - Py_UCS4 *res - ); - -PyAPI_FUNC(int) _PyUnicode_IsCaseIgnorable( - Py_UCS4 ch /* Unicode character */ - ); - -PyAPI_FUNC(int) _PyUnicode_IsCased( - Py_UCS4 ch /* Unicode character */ - ); - PyAPI_FUNC(int) _PyUnicode_ToDecimalDigit( Py_UCS4 ch /* Unicode character */ ); diff --git a/Include/internal/pycore_unicodeobject.h b/Include/internal/pycore_unicodeobject.h index ad59c3e385f2d..d0381123fe280 100644 --- a/Include/internal/pycore_unicodeobject.h +++ b/Include/internal/pycore_unicodeobject.h @@ -11,6 +11,19 @@ extern "C" { #include "pycore_fileutils.h" // _Py_error_handler #include "pycore_ucnhash.h" // _PyUnicode_Name_CAPI +/* --- Characters Type APIs ----------------------------------------------- */ + +extern int _PyUnicode_IsXidStart(Py_UCS4 ch); +extern int _PyUnicode_IsXidContinue(Py_UCS4 ch); +extern int _PyUnicode_ToLowerFull(Py_UCS4 ch, Py_UCS4 *res); +extern int _PyUnicode_ToTitleFull(Py_UCS4 ch, Py_UCS4 *res); +extern int _PyUnicode_ToUpperFull(Py_UCS4 ch, Py_UCS4 *res); +extern int _PyUnicode_ToFoldedFull(Py_UCS4 ch, Py_UCS4 *res); +extern int _PyUnicode_IsCaseIgnorable(Py_UCS4 ch); +extern int _PyUnicode_IsCased(Py_UCS4 ch); + +/* --- Unicode API -------------------------------------------------------- */ + PyAPI_FUNC(int) _PyUnicode_CheckConsistency( PyObject *op, int check_content); From webhook-mailer at python.org Mon Jul 24 14:48:10 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 18:48:10 -0000 Subject: [Python-checkins] gh-106320: Remove private _PyMem API (#107187) Message-ID: https://github.com/python/cpython/commit/307186704d8327d6c67b0650e49b125758a25bbc commit: 307186704d8327d6c67b0650e49b125758a25bbc branch: main author: Victor Stinner committer: vstinner date: 2023-07-24T18:48:06Z summary: gh-106320: Remove private _PyMem API (#107187) Move private _PyMem functions to the internal C API (pycore_pymem.h): * _PyMem_GetCurrentAllocatorName() * _PyMem_RawStrdup() * _PyMem_RawWcsdup() * _PyMem_Strdup() No longer export these functions. Move pymem_getallocatorsname() function from _testcapi to _testinternalcapi, since the API moved to the internal C API. files: M Include/cpython/pymem.h M Include/internal/pycore_pymem.h M Lib/test/pythoninfo.py M Lib/test/test_cmd_line.py M Lib/test/test_sys.py M Modules/_testcapi/mem.c M Modules/_testinternalcapi.c M Modules/getpath.c diff --git a/Include/cpython/pymem.h b/Include/cpython/pymem.h index d1054d76520b9..b75f1c4d2425d 100644 --- a/Include/cpython/pymem.h +++ b/Include/cpython/pymem.h @@ -7,18 +7,6 @@ PyAPI_FUNC(void *) PyMem_RawCalloc(size_t nelem, size_t elsize); PyAPI_FUNC(void *) PyMem_RawRealloc(void *ptr, size_t new_size); PyAPI_FUNC(void) PyMem_RawFree(void *ptr); -/* Try to get the allocators name set by _PyMem_SetupAllocators(). */ -PyAPI_FUNC(const char*) _PyMem_GetCurrentAllocatorName(void); - -/* strdup() using PyMem_RawMalloc() */ -PyAPI_FUNC(char *) _PyMem_RawStrdup(const char *str); - -/* strdup() using PyMem_Malloc() */ -PyAPI_FUNC(char *) _PyMem_Strdup(const char *str); - -/* wcsdup() using PyMem_RawMalloc() */ -PyAPI_FUNC(wchar_t*) _PyMem_RawWcsdup(const wchar_t *str); - typedef enum { /* PyMem_RawMalloc(), PyMem_RawRealloc() and PyMem_RawFree() */ diff --git a/Include/internal/pycore_pymem.h b/Include/internal/pycore_pymem.h index c2f03254bb876..0e53e6650d181 100644 --- a/Include/internal/pycore_pymem.h +++ b/Include/internal/pycore_pymem.h @@ -8,8 +8,20 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -#include "pymem.h" // PyMemAllocatorName +// Try to get the allocators name set by _PyMem_SetupAllocators(). +// Return NULL if unknown. +// Export for shared _testinternalcapi extension. +PyAPI_FUNC(const char*) _PyMem_GetCurrentAllocatorName(void); +// strdup() using PyMem_RawMalloc() +extern char* _PyMem_RawStrdup(const char *str); + +// strdup() using PyMem_Malloc(). +// Export for shared _pickle extension. +PyAPI_FUNC(char*) _PyMem_Strdup(const char *str); + +// wcsdup() using PyMem_RawMalloc() +extern wchar_t* _PyMem_RawWcsdup(const wchar_t *str); typedef struct { /* We tag each block with an API ID in order to tag API violations */ diff --git a/Lib/test/pythoninfo.py b/Lib/test/pythoninfo.py index b84c14400d42f..e4e098dd84cfb 100644 --- a/Lib/test/pythoninfo.py +++ b/Lib/test/pythoninfo.py @@ -637,11 +637,11 @@ def collect_decimal(info_add): def collect_testcapi(info_add): try: - import _testcapi + import _testinternalcapi except ImportError: return - call_func(info_add, 'pymem.allocator', _testcapi, 'pymem_getallocatorsname') + call_func(info_add, 'pymem.allocator', _testinternalcapi, 'pymem_getallocatorsname') def collect_resource(info_add): diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index 9429800306359..e88b7c8572d9e 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -717,11 +717,11 @@ def test_xdev(self): # Memory allocator debug hooks try: - import _testcapi + import _testinternalcapi except ImportError: pass else: - code = "import _testcapi; print(_testcapi.pymem_getallocatorsname())" + code = "import _testinternalcapi; print(_testinternalcapi.pymem_getallocatorsname())" with support.SuppressCrashReport(): out = self.run_xdev("-c", code, check_exitcode=False) if support.with_pymalloc(): @@ -783,7 +783,7 @@ def test_warnings_filter_precedence(self): self.assertEqual(out, expected_filters) def check_pythonmalloc(self, env_var, name): - code = 'import _testcapi; print(_testcapi.pymem_getallocatorsname())' + code = 'import _testinternalcapi; print(_testinternalcapi.pymem_getallocatorsname())' env = dict(os.environ) env.pop('PYTHONDEVMODE', None) if env_var is not None: diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index 37f75ad54387a..7861fa26c8007 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -960,12 +960,12 @@ def test_debugmallocstats(self): "sys.getallocatedblocks unavailable on this build") def test_getallocatedblocks(self): try: - import _testcapi + import _testinternalcapi except ImportError: with_pymalloc = support.with_pymalloc() else: try: - alloc_name = _testcapi.pymem_getallocatorsname() + alloc_name = _testinternalcapi.pymem_getallocatorsname() except RuntimeError as exc: # "cannot get allocators name" (ex: tracemalloc is used) with_pymalloc = True diff --git a/Modules/_testcapi/mem.c b/Modules/_testcapi/mem.c index 0b2b1cd31fb0a..ef9234d7f8955 100644 --- a/Modules/_testcapi/mem.c +++ b/Modules/_testcapi/mem.c @@ -440,17 +440,6 @@ test_pymem_alloc0(PyObject *self, PyObject *Py_UNUSED(ignored)) Py_RETURN_NONE; } -static PyObject * -test_pymem_getallocatorsname(PyObject *self, PyObject *args) -{ - const char *name = _PyMem_GetCurrentAllocatorName(); - if (name == NULL) { - PyErr_SetString(PyExc_RuntimeError, "cannot get allocators name"); - return NULL; - } - return PyUnicode_FromString(name); -} - static PyObject * test_pymem_setrawallocators(PyObject *self, PyObject *Py_UNUSED(ignored)) { @@ -589,7 +578,6 @@ tracemalloc_untrack(PyObject *self, PyObject *args) static PyMethodDef test_methods[] = { {"pymem_api_misuse", pymem_api_misuse, METH_NOARGS}, {"pymem_buffer_overflow", pymem_buffer_overflow, METH_NOARGS}, - {"pymem_getallocatorsname", test_pymem_getallocatorsname, METH_NOARGS}, {"pymem_malloc_without_gil", pymem_malloc_without_gil, METH_NOARGS}, {"pyobject_malloc_without_gil", pyobject_malloc_without_gil, METH_NOARGS}, {"remove_mem_hooks", remove_mem_hooks, METH_NOARGS, diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index e2be9a130217d..98902d6134be8 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -1519,6 +1519,18 @@ check_pyobject_freed_is_freed(PyObject *self, PyObject *Py_UNUSED(args)) } +static PyObject * +test_pymem_getallocatorsname(PyObject *self, PyObject *args) +{ + const char *name = _PyMem_GetCurrentAllocatorName(); + if (name == NULL) { + PyErr_SetString(PyExc_RuntimeError, "cannot get allocators name"); + return NULL; + } + return PyUnicode_FromString(name); +} + + static PyMethodDef module_functions[] = { {"get_configs", get_configs, METH_NOARGS}, {"get_recursion_depth", get_recursion_depth, METH_NOARGS}, @@ -1581,6 +1593,7 @@ static PyMethodDef module_functions[] = { {"check_pyobject_null_is_freed", check_pyobject_null_is_freed, METH_NOARGS}, {"check_pyobject_uninitialized_is_freed", check_pyobject_uninitialized_is_freed, METH_NOARGS}, + {"pymem_getallocatorsname", test_pymem_getallocatorsname, METH_NOARGS}, {NULL, NULL} /* sentinel */ }; diff --git a/Modules/getpath.c b/Modules/getpath.c index abe7c3c3c30a9..76e3c7e65249f 100644 --- a/Modules/getpath.c +++ b/Modules/getpath.c @@ -1,11 +1,13 @@ /* Return the initial module search path. */ #include "Python.h" +#include "pycore_fileutils.h" // _Py_abspath() +#include "pycore_initconfig.h" // _PyStatus_EXCEPTION() +#include "pycore_pathconfig.h" // _PyPathConfig_ReadGlobal() +#include "pycore_pymem.h" // _PyMem_RawWcsdup() + #include "marshal.h" // PyMarshal_ReadObjectFromString #include "osdefs.h" // DELIM -#include "pycore_initconfig.h" -#include "pycore_fileutils.h" -#include "pycore_pathconfig.h" #include #ifdef MS_WINDOWS From webhook-mailer at python.org Mon Jul 24 14:49:20 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 24 Jul 2023 18:49:20 -0000 Subject: [Python-checkins] gh-106368: Increase Argument Clinic CLI test coverage (#107156) Message-ID: https://github.com/python/cpython/commit/83a2837b328c58b243f7d97bec12c64ec66681c5 commit: 83a2837b328c58b243f7d97bec12c64ec66681c5 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-24T20:49:16+02:00 summary: gh-106368: Increase Argument Clinic CLI test coverage (#107156) Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicExternalTest test suite. Co-authored-by: Nikita Sobolev files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index c24b8cd3899f2..f527120336226 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -4,11 +4,14 @@ from test import support, test_tools from test.support import os_helper +from test.support import SHORT_TIMEOUT, requires_subprocess +from test.support.os_helper import TESTFN, unlink from textwrap import dedent from unittest import TestCase import collections import inspect import os.path +import subprocess import sys import unittest @@ -1346,31 +1349,190 @@ def test_scaffolding(self): class ClinicExternalTest(TestCase): maxDiff = None + def _do_test(self, *args, expect_success=True): + clinic_py = os.path.join(test_tools.toolsdir, "clinic", "clinic.py") + with subprocess.Popen( + [sys.executable, "-Xutf8", clinic_py, *args], + encoding="utf-8", + bufsize=0, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) as proc: + proc.wait() + if expect_success == bool(proc.returncode): + self.fail("".join(proc.stderr)) + stdout = proc.stdout.read() + stderr = proc.stderr.read() + # Clinic never writes to stderr. + self.assertEqual(stderr, "") + return stdout + + def expect_success(self, *args): + return self._do_test(*args) + + def expect_failure(self, *args): + return self._do_test(*args, expect_success=False) + def test_external(self): CLINIC_TEST = 'clinic.test.c' - # bpo-42398: Test that the destination file is left unchanged if the - # content does not change. Moreover, check also that the file - # modification time does not change in this case. source = support.findfile(CLINIC_TEST) with open(source, 'r', encoding='utf-8') as f: orig_contents = f.read() - with os_helper.temp_dir() as tmp_dir: - testfile = os.path.join(tmp_dir, CLINIC_TEST) - with open(testfile, 'w', encoding='utf-8') as f: - f.write(orig_contents) - old_mtime_ns = os.stat(testfile).st_mtime_ns - - clinic.parse_file(testfile) + # Run clinic CLI and verify that it does not complain. + self.addCleanup(unlink, TESTFN) + out = self.expect_success("-f", "-o", TESTFN, source) + self.assertEqual(out, "") - with open(testfile, 'r', encoding='utf-8') as f: - new_contents = f.read() - new_mtime_ns = os.stat(testfile).st_mtime_ns + with open(TESTFN, 'r', encoding='utf-8') as f: + new_contents = f.read() self.assertEqual(new_contents, orig_contents) + + def test_no_change(self): + # bpo-42398: Test that the destination file is left unchanged if the + # content does not change. Moreover, check also that the file + # modification time does not change in this case. + code = dedent(""" + /*[clinic input] + [clinic start generated code]*/ + /*[clinic end generated code: output=da39a3ee5e6b4b0d input=da39a3ee5e6b4b0d]*/ + """) + with os_helper.temp_dir() as tmp_dir: + fn = os.path.join(tmp_dir, "test.c") + with open(fn, "w", encoding="utf-8") as f: + f.write(code) + pre_mtime = os.stat(fn).st_mtime_ns + self.expect_success(fn) + post_mtime = os.stat(fn).st_mtime_ns # Don't change the file modification time # if the content does not change - self.assertEqual(new_mtime_ns, old_mtime_ns) + self.assertEqual(pre_mtime, post_mtime) + + def test_cli_force(self): + invalid_input = dedent(""" + /*[clinic input] + output preset block + module test + test.fn + a: int + [clinic start generated code]*/ + + const char *hand_edited = "output block is overwritten"; + /*[clinic end generated code: output=bogus input=bogus]*/ + """) + fail_msg = dedent(""" + Checksum mismatch! + Expected: bogus + Computed: 2ed19 + Suggested fix: remove all generated code including the end marker, + or use the '-f' option. + """) + with os_helper.temp_dir() as tmp_dir: + fn = os.path.join(tmp_dir, "test.c") + with open(fn, "w", encoding="utf-8") as f: + f.write(invalid_input) + # First, run the CLI without -f and expect failure. + # Note, we cannot check the entire fail msg, because the path to + # the tmp file will change for every run. + out = self.expect_failure(fn) + self.assertTrue(out.endswith(fail_msg)) + # Then, force regeneration; success expected. + out = self.expect_success("-f", fn) + self.assertEqual(out, "") + # Verify by checking the checksum. + checksum = ( + "/*[clinic end generated code: " + "output=2124c291eb067d76 input=9543a8d2da235301]*/\n" + ) + with open(fn, 'r', encoding='utf-8') as f: + generated = f.read() + self.assertTrue(generated.endswith(checksum)) + + def test_cli_verbose(self): + with os_helper.temp_dir() as tmp_dir: + fn = os.path.join(tmp_dir, "test.c") + with open(fn, "w", encoding="utf-8") as f: + f.write("") + out = self.expect_success("-v", fn) + self.assertEqual(out.strip(), fn) + + def test_cli_help(self): + out = self.expect_success("-h") + self.assertIn("usage: clinic.py", out) + + def test_cli_converters(self): + prelude = dedent(""" + Legacy converters: + B C D L O S U Y Z Z# + b c d f h i l p s s# s* u u# w* y y# y* z z# z* + + Converters: + """) + expected_converters = ( + "bool", + "byte", + "char", + "defining_class", + "double", + "fildes", + "float", + "int", + "long", + "long_long", + "object", + "Py_buffer", + "Py_complex", + "Py_ssize_t", + "Py_UNICODE", + "PyByteArrayObject", + "PyBytesObject", + "self", + "short", + "size_t", + "slice_index", + "str", + "unicode", + "unsigned_char", + "unsigned_int", + "unsigned_long", + "unsigned_long_long", + "unsigned_short", + ) + finale = dedent(""" + Return converters: + bool() + double() + float() + init() + int() + long() + Py_ssize_t() + size_t() + unsigned_int() + unsigned_long() + + All converters also accept (c_default=None, py_default=None, annotation=None). + All return converters also accept (py_default=None). + """) + out = self.expect_success("--converters") + # We cannot simply compare the output, because the repr of the *accept* + # param may change (it's a set, thus unordered). So, let's compare the + # start and end of the expected output, and then assert that the + # converters appear lined up in alphabetical order. + self.assertTrue(out.startswith(prelude), out) + self.assertTrue(out.endswith(finale), out) + + out = out.removeprefix(prelude) + out = out.removesuffix(finale) + lines = out.split("\n") + for converter, line in zip(expected_converters, lines): + line = line.lstrip() + with self.subTest(converter=converter): + self.assertTrue( + line.startswith(converter), + f"expected converter {converter!r}, got {line!r}" + ) try: From webhook-mailer at python.org Mon Jul 24 15:19:40 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 24 Jul 2023 19:19:40 -0000 Subject: [Python-checkins] [3.12] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156) (#107189) Message-ID: https://github.com/python/cpython/commit/69d4e8c090efd4f3a958ba056580bc1ee9865d0e commit: 69d4e8c090efd4f3a958ba056580bc1ee9865d0e branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-24T19:19:36Z summary: [3.12] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156) (#107189) Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicExternalTest test suite. (cherry picked from commit 83a2837b328c58b243f7d97bec12c64ec66681c5) Co-authored-by: Erlend E. Aasland Co-authored-by: Nikita Sobolev files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index c24b8cd3899f2..f527120336226 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -4,11 +4,14 @@ from test import support, test_tools from test.support import os_helper +from test.support import SHORT_TIMEOUT, requires_subprocess +from test.support.os_helper import TESTFN, unlink from textwrap import dedent from unittest import TestCase import collections import inspect import os.path +import subprocess import sys import unittest @@ -1346,31 +1349,190 @@ def test_scaffolding(self): class ClinicExternalTest(TestCase): maxDiff = None + def _do_test(self, *args, expect_success=True): + clinic_py = os.path.join(test_tools.toolsdir, "clinic", "clinic.py") + with subprocess.Popen( + [sys.executable, "-Xutf8", clinic_py, *args], + encoding="utf-8", + bufsize=0, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) as proc: + proc.wait() + if expect_success == bool(proc.returncode): + self.fail("".join(proc.stderr)) + stdout = proc.stdout.read() + stderr = proc.stderr.read() + # Clinic never writes to stderr. + self.assertEqual(stderr, "") + return stdout + + def expect_success(self, *args): + return self._do_test(*args) + + def expect_failure(self, *args): + return self._do_test(*args, expect_success=False) + def test_external(self): CLINIC_TEST = 'clinic.test.c' - # bpo-42398: Test that the destination file is left unchanged if the - # content does not change. Moreover, check also that the file - # modification time does not change in this case. source = support.findfile(CLINIC_TEST) with open(source, 'r', encoding='utf-8') as f: orig_contents = f.read() - with os_helper.temp_dir() as tmp_dir: - testfile = os.path.join(tmp_dir, CLINIC_TEST) - with open(testfile, 'w', encoding='utf-8') as f: - f.write(orig_contents) - old_mtime_ns = os.stat(testfile).st_mtime_ns - - clinic.parse_file(testfile) + # Run clinic CLI and verify that it does not complain. + self.addCleanup(unlink, TESTFN) + out = self.expect_success("-f", "-o", TESTFN, source) + self.assertEqual(out, "") - with open(testfile, 'r', encoding='utf-8') as f: - new_contents = f.read() - new_mtime_ns = os.stat(testfile).st_mtime_ns + with open(TESTFN, 'r', encoding='utf-8') as f: + new_contents = f.read() self.assertEqual(new_contents, orig_contents) + + def test_no_change(self): + # bpo-42398: Test that the destination file is left unchanged if the + # content does not change. Moreover, check also that the file + # modification time does not change in this case. + code = dedent(""" + /*[clinic input] + [clinic start generated code]*/ + /*[clinic end generated code: output=da39a3ee5e6b4b0d input=da39a3ee5e6b4b0d]*/ + """) + with os_helper.temp_dir() as tmp_dir: + fn = os.path.join(tmp_dir, "test.c") + with open(fn, "w", encoding="utf-8") as f: + f.write(code) + pre_mtime = os.stat(fn).st_mtime_ns + self.expect_success(fn) + post_mtime = os.stat(fn).st_mtime_ns # Don't change the file modification time # if the content does not change - self.assertEqual(new_mtime_ns, old_mtime_ns) + self.assertEqual(pre_mtime, post_mtime) + + def test_cli_force(self): + invalid_input = dedent(""" + /*[clinic input] + output preset block + module test + test.fn + a: int + [clinic start generated code]*/ + + const char *hand_edited = "output block is overwritten"; + /*[clinic end generated code: output=bogus input=bogus]*/ + """) + fail_msg = dedent(""" + Checksum mismatch! + Expected: bogus + Computed: 2ed19 + Suggested fix: remove all generated code including the end marker, + or use the '-f' option. + """) + with os_helper.temp_dir() as tmp_dir: + fn = os.path.join(tmp_dir, "test.c") + with open(fn, "w", encoding="utf-8") as f: + f.write(invalid_input) + # First, run the CLI without -f and expect failure. + # Note, we cannot check the entire fail msg, because the path to + # the tmp file will change for every run. + out = self.expect_failure(fn) + self.assertTrue(out.endswith(fail_msg)) + # Then, force regeneration; success expected. + out = self.expect_success("-f", fn) + self.assertEqual(out, "") + # Verify by checking the checksum. + checksum = ( + "/*[clinic end generated code: " + "output=2124c291eb067d76 input=9543a8d2da235301]*/\n" + ) + with open(fn, 'r', encoding='utf-8') as f: + generated = f.read() + self.assertTrue(generated.endswith(checksum)) + + def test_cli_verbose(self): + with os_helper.temp_dir() as tmp_dir: + fn = os.path.join(tmp_dir, "test.c") + with open(fn, "w", encoding="utf-8") as f: + f.write("") + out = self.expect_success("-v", fn) + self.assertEqual(out.strip(), fn) + + def test_cli_help(self): + out = self.expect_success("-h") + self.assertIn("usage: clinic.py", out) + + def test_cli_converters(self): + prelude = dedent(""" + Legacy converters: + B C D L O S U Y Z Z# + b c d f h i l p s s# s* u u# w* y y# y* z z# z* + + Converters: + """) + expected_converters = ( + "bool", + "byte", + "char", + "defining_class", + "double", + "fildes", + "float", + "int", + "long", + "long_long", + "object", + "Py_buffer", + "Py_complex", + "Py_ssize_t", + "Py_UNICODE", + "PyByteArrayObject", + "PyBytesObject", + "self", + "short", + "size_t", + "slice_index", + "str", + "unicode", + "unsigned_char", + "unsigned_int", + "unsigned_long", + "unsigned_long_long", + "unsigned_short", + ) + finale = dedent(""" + Return converters: + bool() + double() + float() + init() + int() + long() + Py_ssize_t() + size_t() + unsigned_int() + unsigned_long() + + All converters also accept (c_default=None, py_default=None, annotation=None). + All return converters also accept (py_default=None). + """) + out = self.expect_success("--converters") + # We cannot simply compare the output, because the repr of the *accept* + # param may change (it's a set, thus unordered). So, let's compare the + # start and end of the expected output, and then assert that the + # converters appear lined up in alphabetical order. + self.assertTrue(out.startswith(prelude), out) + self.assertTrue(out.endswith(finale), out) + + out = out.removeprefix(prelude) + out = out.removesuffix(finale) + lines = out.split("\n") + for converter, line in zip(expected_converters, lines): + line = line.lstrip() + with self.subTest(converter=converter): + self.assertTrue( + line.startswith(converter), + f"expected converter {converter!r}, got {line!r}" + ) try: From webhook-mailer at python.org Mon Jul 24 15:20:48 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 19:20:48 -0000 Subject: [Python-checkins] GH-96803: Move PyUnstable_InterpreterFrame_GetCode() to Python.h (#107188) Message-ID: https://github.com/python/cpython/commit/837fa5c0cd92e70f625377c9ffb7ac31a23d7d63 commit: 837fa5c0cd92e70f625377c9ffb7ac31a23d7d63 branch: main author: Victor Stinner committer: vstinner date: 2023-07-24T21:20:44+02:00 summary: GH-96803: Move PyUnstable_InterpreterFrame_GetCode() to Python.h (#107188) Declare the following 3 PyUnstable functions in Include/cpython/pyframe.h rather than Include/cpython/frameobject.h, so they are now provided by the standard "#include ". files: M Include/cpython/frameobject.h M Include/cpython/pyframe.h diff --git a/Include/cpython/frameobject.h b/Include/cpython/frameobject.h index a3dc666178645..4e19535c656f2 100644 --- a/Include/cpython/frameobject.h +++ b/Include/cpython/frameobject.h @@ -4,8 +4,6 @@ # error "this header file must not be included directly" #endif -struct _PyInterpreterFrame; - /* Standard object interface */ PyAPI_FUNC(PyFrameObject *) PyFrame_New(PyThreadState *, PyCodeObject *, @@ -29,18 +27,3 @@ PyAPI_FUNC(int) _PyFrame_IsEntryFrame(PyFrameObject *frame); PyAPI_FUNC(int) PyFrame_FastToLocalsWithError(PyFrameObject *f); PyAPI_FUNC(void) PyFrame_FastToLocals(PyFrameObject *); - -/* The following functions are for use by debuggers and other tools - * implementing custom frame evaluators with PEP 523. */ - -/* Returns the code object of the frame (strong reference). - * Does not raise an exception. */ -PyAPI_FUNC(PyObject *) PyUnstable_InterpreterFrame_GetCode(struct _PyInterpreterFrame *frame); - -/* Returns a byte ofsset into the last executed instruction. - * Does not raise an exception. */ -PyAPI_FUNC(int) PyUnstable_InterpreterFrame_GetLasti(struct _PyInterpreterFrame *frame); - -/* Returns the currently executing line number, or -1 if there is no line number. - * Does not raise an exception. */ -PyAPI_FUNC(int) PyUnstable_InterpreterFrame_GetLine(struct _PyInterpreterFrame *frame); diff --git a/Include/cpython/pyframe.h b/Include/cpython/pyframe.h index 6ec292718aff1..0e2afff925e31 100644 --- a/Include/cpython/pyframe.h +++ b/Include/cpython/pyframe.h @@ -16,3 +16,20 @@ PyAPI_FUNC(PyObject *) PyFrame_GetGenerator(PyFrameObject *frame); PyAPI_FUNC(int) PyFrame_GetLasti(PyFrameObject *frame); PyAPI_FUNC(PyObject*) PyFrame_GetVar(PyFrameObject *frame, PyObject *name); PyAPI_FUNC(PyObject*) PyFrame_GetVarString(PyFrameObject *frame, const char *name); + +/* The following functions are for use by debuggers and other tools + * implementing custom frame evaluators with PEP 523. */ + +struct _PyInterpreterFrame; + +/* Returns the code object of the frame (strong reference). + * Does not raise an exception. */ +PyAPI_FUNC(PyObject *) PyUnstable_InterpreterFrame_GetCode(struct _PyInterpreterFrame *frame); + +/* Returns a byte ofsset into the last executed instruction. + * Does not raise an exception. */ +PyAPI_FUNC(int) PyUnstable_InterpreterFrame_GetLasti(struct _PyInterpreterFrame *frame); + +/* Returns the currently executing line number, or -1 if there is no line number. + * Does not raise an exception. */ +PyAPI_FUNC(int) PyUnstable_InterpreterFrame_GetLine(struct _PyInterpreterFrame *frame); From webhook-mailer at python.org Mon Jul 24 15:32:42 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 24 Jul 2023 19:32:42 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: annotate `main()` (#107192) Message-ID: https://github.com/python/cpython/commit/db17529f80e18f80a680bd588a71be32ef7ec9fb commit: db17529f80e18f80a680bd588a71be32ef7ec9fb branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-24T19:32:38Z summary: gh-104050: Argument Clinic: annotate `main()` (#107192) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index eecb81dcad588..c9c57f1fdbb3b 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -5591,7 +5591,7 @@ def state_terminal(self, line: str | None) -> None: clinic = None -def main(argv): +def main(argv: list[str]) -> None: import sys import argparse cmdline = argparse.ArgumentParser( @@ -5619,8 +5619,8 @@ def main(argv): print() cmdline.print_usage() sys.exit(-1) - converters = [] - return_converters = [] + converters: list[tuple[str, str]] = [] + return_converters: list[tuple[str, str]] = [] ignored = set(""" add_c_converter add_c_return_converter @@ -5716,4 +5716,5 @@ def main(argv): if __name__ == "__main__": - sys.exit(main(sys.argv[1:])) + main(sys.argv[1:]) + sys.exit(0) From webhook-mailer at python.org Mon Jul 24 15:42:08 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 19:42:08 -0000 Subject: [Python-checkins] gh-98608: Move PyInterpreterConfig to pylifecycle.h (#107191) Message-ID: https://github.com/python/cpython/commit/e717b47ed8ae7017f0bfb835fe673aa836e8fcca commit: e717b47ed8ae7017f0bfb835fe673aa836e8fcca branch: main author: Victor Stinner committer: vstinner date: 2023-07-24T19:42:04Z summary: gh-98608: Move PyInterpreterConfig to pylifecycle.h (#107191) Move PyInterpreterConfig structure and associated macros from initconfig.h to pylifecycle.h: it's not related to the Python Initialization Configuration. files: M Include/cpython/initconfig.h M Include/cpython/pylifecycle.h diff --git a/Include/cpython/initconfig.h b/Include/cpython/initconfig.h index c103c2026e40e..cbae97f12f537 100644 --- a/Include/cpython/initconfig.h +++ b/Include/cpython/initconfig.h @@ -242,45 +242,6 @@ PyAPI_FUNC(PyStatus) PyConfig_SetWideStringList(PyConfig *config, Py_ssize_t length, wchar_t **items); -/* --- PyInterpreterConfig ------------------------------------ */ - -#define PyInterpreterConfig_DEFAULT_GIL (0) -#define PyInterpreterConfig_SHARED_GIL (1) -#define PyInterpreterConfig_OWN_GIL (2) - -typedef struct { - // XXX "allow_object_sharing"? "own_objects"? - int use_main_obmalloc; - int allow_fork; - int allow_exec; - int allow_threads; - int allow_daemon_threads; - int check_multi_interp_extensions; - int gil; -} PyInterpreterConfig; - -#define _PyInterpreterConfig_INIT \ - { \ - .use_main_obmalloc = 0, \ - .allow_fork = 0, \ - .allow_exec = 0, \ - .allow_threads = 1, \ - .allow_daemon_threads = 0, \ - .check_multi_interp_extensions = 1, \ - .gil = PyInterpreterConfig_OWN_GIL, \ - } - -#define _PyInterpreterConfig_LEGACY_INIT \ - { \ - .use_main_obmalloc = 1, \ - .allow_fork = 1, \ - .allow_exec = 1, \ - .allow_threads = 1, \ - .allow_daemon_threads = 1, \ - .check_multi_interp_extensions = 0, \ - .gil = PyInterpreterConfig_SHARED_GIL, \ - } - /* --- Helper functions --------------------------------------- */ /* Get the original command line arguments, before Python modified them. diff --git a/Include/cpython/pylifecycle.h b/Include/cpython/pylifecycle.h index 8af34b0564251..a78cec9525d5f 100644 --- a/Include/cpython/pylifecycle.h +++ b/Include/cpython/pylifecycle.h @@ -35,6 +35,45 @@ PyAPI_FUNC(void) _Py_NO_RETURN Py_ExitStatusException(PyStatus err); PyAPI_FUNC(int) Py_FdIsInteractive(FILE *, const char *); +/* --- PyInterpreterConfig ------------------------------------ */ + +#define PyInterpreterConfig_DEFAULT_GIL (0) +#define PyInterpreterConfig_SHARED_GIL (1) +#define PyInterpreterConfig_OWN_GIL (2) + +typedef struct { + // XXX "allow_object_sharing"? "own_objects"? + int use_main_obmalloc; + int allow_fork; + int allow_exec; + int allow_threads; + int allow_daemon_threads; + int check_multi_interp_extensions; + int gil; +} PyInterpreterConfig; + +#define _PyInterpreterConfig_INIT \ + { \ + .use_main_obmalloc = 0, \ + .allow_fork = 0, \ + .allow_exec = 0, \ + .allow_threads = 1, \ + .allow_daemon_threads = 0, \ + .check_multi_interp_extensions = 1, \ + .gil = PyInterpreterConfig_OWN_GIL, \ + } + +#define _PyInterpreterConfig_LEGACY_INIT \ + { \ + .use_main_obmalloc = 1, \ + .allow_fork = 1, \ + .allow_exec = 1, \ + .allow_threads = 1, \ + .allow_daemon_threads = 1, \ + .check_multi_interp_extensions = 0, \ + .gil = PyInterpreterConfig_SHARED_GIL, \ + } + PyAPI_FUNC(PyStatus) Py_NewInterpreterFromConfig( PyThreadState **tstate_p, const PyInterpreterConfig *config); From webhook-mailer at python.org Mon Jul 24 15:47:54 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 19:47:54 -0000 Subject: [Python-checkins] gh-102304: Rename _Py_IncRefTotal_DO_NOT_USE_THIS() (#107193) Message-ID: https://github.com/python/cpython/commit/8ebc9fc321ba1eeb3282c2170f351c54956893e6 commit: 8ebc9fc321ba1eeb3282c2170f351c54956893e6 branch: main author: Victor Stinner committer: vstinner date: 2023-07-24T19:47:50Z summary: gh-102304: Rename _Py_IncRefTotal_DO_NOT_USE_THIS() (#107193) * Rename _Py_IncRefTotal_DO_NOT_USE_THIS() to _Py_INCREF_IncRefTotal() * Rename _Py_DecRefTotal_DO_NOT_USE_THIS() to _Py_DECREF_DecRefTotal() * Remove temporary _Py_INC_REFTOTAL() and _Py_DEC_REFTOTAL() macros files: M Include/object.h M Objects/object.c diff --git a/Include/object.h b/Include/object.h index 7f2e4e90615e7..7182eba3adfe1 100644 --- a/Include/object.h +++ b/Include/object.h @@ -594,10 +594,8 @@ you can count such references to the type object.) #if defined(Py_REF_DEBUG) && !defined(Py_LIMITED_API) PyAPI_FUNC(void) _Py_NegativeRefcount(const char *filename, int lineno, PyObject *op); -PyAPI_FUNC(void) _Py_IncRefTotal_DO_NOT_USE_THIS(void); -PyAPI_FUNC(void) _Py_DecRefTotal_DO_NOT_USE_THIS(void); -# define _Py_INC_REFTOTAL() _Py_IncRefTotal_DO_NOT_USE_THIS() -# define _Py_DEC_REFTOTAL() _Py_DecRefTotal_DO_NOT_USE_THIS() +PyAPI_FUNC(void) _Py_INCREF_IncRefTotal(void); +PyAPI_FUNC(void) _Py_DECREF_DecRefTotal(void); #endif // Py_REF_DEBUG && !Py_LIMITED_API PyAPI_FUNC(void) _Py_Dealloc(PyObject *); @@ -646,7 +644,7 @@ static inline Py_ALWAYS_INLINE void Py_INCREF(PyObject *op) #endif _Py_INCREF_STAT_INC(); #ifdef Py_REF_DEBUG - _Py_INC_REFTOTAL(); + _Py_INCREF_IncRefTotal(); #endif #endif } @@ -675,7 +673,7 @@ static inline void Py_DECREF(const char *filename, int lineno, PyObject *op) return; } _Py_DECREF_STAT_INC(); - _Py_DEC_REFTOTAL(); + _Py_DECREF_DecRefTotal(); if (--op->ob_refcnt != 0) { if (op->ob_refcnt < 0) { _Py_NegativeRefcount(filename, lineno, op); @@ -703,9 +701,6 @@ static inline Py_ALWAYS_INLINE void Py_DECREF(PyObject *op) #define Py_DECREF(op) Py_DECREF(_PyObject_CAST(op)) #endif -#undef _Py_INC_REFTOTAL -#undef _Py_DEC_REFTOTAL - /* Safely decref `op` and set `op` to NULL, especially useful in tp_clear * and tp_dealloc implementations. diff --git a/Objects/object.c b/Objects/object.c index 740d4a22c103b..b724eb5f8f079 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -207,14 +207,14 @@ _Py_NegativeRefcount(const char *filename, int lineno, PyObject *op) /* This is used strictly by Py_INCREF(). */ void -_Py_IncRefTotal_DO_NOT_USE_THIS(void) +_Py_INCREF_IncRefTotal(void) { reftotal_increment(_PyInterpreterState_GET()); } /* This is used strictly by Py_DECREF(). */ void -_Py_DecRefTotal_DO_NOT_USE_THIS(void) +_Py_DECREF_DecRefTotal(void) { reftotal_decrement(_PyInterpreterState_GET()); } From webhook-mailer at python.org Mon Jul 24 15:54:46 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 19:54:46 -0000 Subject: [Python-checkins] [3.12] GH-96803: Move PyUnstable_InterpreterFrame_GetCode() to Python.h (GH-107188) (#107195) Message-ID: https://github.com/python/cpython/commit/f573a6a281e49560a1022a13c39e44a9207ef6d2 commit: f573a6a281e49560a1022a13c39e44a9207ef6d2 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-24T19:54:42Z summary: [3.12] GH-96803: Move PyUnstable_InterpreterFrame_GetCode() to Python.h (GH-107188) (#107195) GH-96803: Move PyUnstable_InterpreterFrame_GetCode() to Python.h (GH-107188) Declare the following 3 PyUnstable functions in Include/cpython/pyframe.h rather than Include/cpython/frameobject.h, so they are now provided by the standard "GH-include ". (cherry picked from commit 837fa5c0cd92e70f625377c9ffb7ac31a23d7d63) Co-authored-by: Victor Stinner files: M Include/cpython/frameobject.h M Include/cpython/pyframe.h diff --git a/Include/cpython/frameobject.h b/Include/cpython/frameobject.h index a3dc666178645..4e19535c656f2 100644 --- a/Include/cpython/frameobject.h +++ b/Include/cpython/frameobject.h @@ -4,8 +4,6 @@ # error "this header file must not be included directly" #endif -struct _PyInterpreterFrame; - /* Standard object interface */ PyAPI_FUNC(PyFrameObject *) PyFrame_New(PyThreadState *, PyCodeObject *, @@ -29,18 +27,3 @@ PyAPI_FUNC(int) _PyFrame_IsEntryFrame(PyFrameObject *frame); PyAPI_FUNC(int) PyFrame_FastToLocalsWithError(PyFrameObject *f); PyAPI_FUNC(void) PyFrame_FastToLocals(PyFrameObject *); - -/* The following functions are for use by debuggers and other tools - * implementing custom frame evaluators with PEP 523. */ - -/* Returns the code object of the frame (strong reference). - * Does not raise an exception. */ -PyAPI_FUNC(PyObject *) PyUnstable_InterpreterFrame_GetCode(struct _PyInterpreterFrame *frame); - -/* Returns a byte ofsset into the last executed instruction. - * Does not raise an exception. */ -PyAPI_FUNC(int) PyUnstable_InterpreterFrame_GetLasti(struct _PyInterpreterFrame *frame); - -/* Returns the currently executing line number, or -1 if there is no line number. - * Does not raise an exception. */ -PyAPI_FUNC(int) PyUnstable_InterpreterFrame_GetLine(struct _PyInterpreterFrame *frame); diff --git a/Include/cpython/pyframe.h b/Include/cpython/pyframe.h index 6ec292718aff1..0e2afff925e31 100644 --- a/Include/cpython/pyframe.h +++ b/Include/cpython/pyframe.h @@ -16,3 +16,20 @@ PyAPI_FUNC(PyObject *) PyFrame_GetGenerator(PyFrameObject *frame); PyAPI_FUNC(int) PyFrame_GetLasti(PyFrameObject *frame); PyAPI_FUNC(PyObject*) PyFrame_GetVar(PyFrameObject *frame, PyObject *name); PyAPI_FUNC(PyObject*) PyFrame_GetVarString(PyFrameObject *frame, const char *name); + +/* The following functions are for use by debuggers and other tools + * implementing custom frame evaluators with PEP 523. */ + +struct _PyInterpreterFrame; + +/* Returns the code object of the frame (strong reference). + * Does not raise an exception. */ +PyAPI_FUNC(PyObject *) PyUnstable_InterpreterFrame_GetCode(struct _PyInterpreterFrame *frame); + +/* Returns a byte ofsset into the last executed instruction. + * Does not raise an exception. */ +PyAPI_FUNC(int) PyUnstable_InterpreterFrame_GetLasti(struct _PyInterpreterFrame *frame); + +/* Returns the currently executing line number, or -1 if there is no line number. + * Does not raise an exception. */ +PyAPI_FUNC(int) PyUnstable_InterpreterFrame_GetLine(struct _PyInterpreterFrame *frame); From webhook-mailer at python.org Mon Jul 24 16:12:49 2023 From: webhook-mailer at python.org (carljm) Date: Mon, 24 Jul 2023 20:12:49 -0000 Subject: [Python-checkins] gh-106727: Add `__module__` check for `inspect.getsource(cls)` (#106968) Message-ID: https://github.com/python/cpython/commit/b38370349139e06f3a44150943ee44e345567d77 commit: b38370349139e06f3a44150943ee44e345567d77 branch: main author: Tian Gao committer: carljm date: 2023-07-24T13:12:45-07:00 summary: gh-106727: Add `__module__` check for `inspect.getsource(cls)` (#106968) files: M Lib/inspect.py M Lib/test/test_inspect.py diff --git a/Lib/inspect.py b/Lib/inspect.py index 675714dc8b3f7..c8211833dd083 100644 --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -1078,7 +1078,8 @@ def get_lineno(self): # First, let's see if there are any method definitions for member in self.cls.__dict__.values(): - if isinstance(member, types.FunctionType): + if (isinstance(member, types.FunctionType) and + member.__module__ == self.cls.__module__): for lineno, end_lineno in self.lineno_found: if lineno <= member.__code__.co_firstlineno <= end_lineno: return lineno diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py index 33a593f3591d6..3fbfc07325553 100644 --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -15,6 +15,7 @@ import shutil import sys import types +import tempfile import textwrap import unicodedata import unittest @@ -963,6 +964,33 @@ def test_nested_class_definition_inside_function(self): self.assertSourceEqual(mod2.cls213, 218, 222) self.assertSourceEqual(mod2.cls213().func219(), 220, 221) + def test_class_with_method_from_other_module(self): + with tempfile.TemporaryDirectory() as tempdir: + with open(os.path.join(tempdir, 'inspect_actual%spy' % os.extsep), + 'w', encoding='utf-8') as f: + f.write(textwrap.dedent(""" + import inspect_other + class A: + def f(self): + pass + class A: + def f(self): + pass # correct one + A.f = inspect_other.A.f + """)) + + with open(os.path.join(tempdir, 'inspect_other%spy' % os.extsep), + 'w', encoding='utf-8') as f: + f.write(textwrap.dedent(""" + class A: + def f(self): + pass + """)) + + with DirsOnSysPath(tempdir): + import inspect_actual + self.assertIn("correct", inspect.getsource(inspect_actual.A)) + @unittest.skipIf( support.is_emscripten or support.is_wasi, "socket.accept is broken" From webhook-mailer at python.org Mon Jul 24 16:13:11 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 20:13:11 -0000 Subject: [Python-checkins] [3.12] gh-98608: Move PyInterpreterConfig to pylifecycle.h (GH-107191) (#107198) Message-ID: https://github.com/python/cpython/commit/5bf7165e591f2e7502672cc591cea480a5b91a8f commit: 5bf7165e591f2e7502672cc591cea480a5b91a8f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-24T20:13:07Z summary: [3.12] gh-98608: Move PyInterpreterConfig to pylifecycle.h (GH-107191) (#107198) gh-98608: Move PyInterpreterConfig to pylifecycle.h (GH-107191) Move PyInterpreterConfig structure and associated macros from initconfig.h to pylifecycle.h: it's not related to the Python Initialization Configuration. (cherry picked from commit e717b47ed8ae7017f0bfb835fe673aa836e8fcca) Co-authored-by: Victor Stinner files: M Include/cpython/initconfig.h M Include/cpython/pylifecycle.h diff --git a/Include/cpython/initconfig.h b/Include/cpython/initconfig.h index c103c2026e40e..cbae97f12f537 100644 --- a/Include/cpython/initconfig.h +++ b/Include/cpython/initconfig.h @@ -242,45 +242,6 @@ PyAPI_FUNC(PyStatus) PyConfig_SetWideStringList(PyConfig *config, Py_ssize_t length, wchar_t **items); -/* --- PyInterpreterConfig ------------------------------------ */ - -#define PyInterpreterConfig_DEFAULT_GIL (0) -#define PyInterpreterConfig_SHARED_GIL (1) -#define PyInterpreterConfig_OWN_GIL (2) - -typedef struct { - // XXX "allow_object_sharing"? "own_objects"? - int use_main_obmalloc; - int allow_fork; - int allow_exec; - int allow_threads; - int allow_daemon_threads; - int check_multi_interp_extensions; - int gil; -} PyInterpreterConfig; - -#define _PyInterpreterConfig_INIT \ - { \ - .use_main_obmalloc = 0, \ - .allow_fork = 0, \ - .allow_exec = 0, \ - .allow_threads = 1, \ - .allow_daemon_threads = 0, \ - .check_multi_interp_extensions = 1, \ - .gil = PyInterpreterConfig_OWN_GIL, \ - } - -#define _PyInterpreterConfig_LEGACY_INIT \ - { \ - .use_main_obmalloc = 1, \ - .allow_fork = 1, \ - .allow_exec = 1, \ - .allow_threads = 1, \ - .allow_daemon_threads = 1, \ - .check_multi_interp_extensions = 0, \ - .gil = PyInterpreterConfig_SHARED_GIL, \ - } - /* --- Helper functions --------------------------------------- */ /* Get the original command line arguments, before Python modified them. diff --git a/Include/cpython/pylifecycle.h b/Include/cpython/pylifecycle.h index 314a5cc5b942a..4daea33bf8011 100644 --- a/Include/cpython/pylifecycle.h +++ b/Include/cpython/pylifecycle.h @@ -63,6 +63,45 @@ PyAPI_FUNC(int) _Py_CoerceLegacyLocale(int warn); PyAPI_FUNC(int) _Py_LegacyLocaleDetected(int warn); PyAPI_FUNC(char *) _Py_SetLocaleFromEnv(int category); +/* --- PyInterpreterConfig ------------------------------------ */ + +#define PyInterpreterConfig_DEFAULT_GIL (0) +#define PyInterpreterConfig_SHARED_GIL (1) +#define PyInterpreterConfig_OWN_GIL (2) + +typedef struct { + // XXX "allow_object_sharing"? "own_objects"? + int use_main_obmalloc; + int allow_fork; + int allow_exec; + int allow_threads; + int allow_daemon_threads; + int check_multi_interp_extensions; + int gil; +} PyInterpreterConfig; + +#define _PyInterpreterConfig_INIT \ + { \ + .use_main_obmalloc = 0, \ + .allow_fork = 0, \ + .allow_exec = 0, \ + .allow_threads = 1, \ + .allow_daemon_threads = 0, \ + .check_multi_interp_extensions = 1, \ + .gil = PyInterpreterConfig_OWN_GIL, \ + } + +#define _PyInterpreterConfig_LEGACY_INIT \ + { \ + .use_main_obmalloc = 1, \ + .allow_fork = 1, \ + .allow_exec = 1, \ + .allow_threads = 1, \ + .allow_daemon_threads = 1, \ + .check_multi_interp_extensions = 0, \ + .gil = PyInterpreterConfig_SHARED_GIL, \ + } + PyAPI_FUNC(PyStatus) Py_NewInterpreterFromConfig( PyThreadState **tstate_p, const PyInterpreterConfig *config); From webhook-mailer at python.org Mon Jul 24 16:15:00 2023 From: webhook-mailer at python.org (carljm) Date: Mon, 24 Jul 2023 20:15:00 -0000 Subject: [Python-checkins] gh-106917: fix super classmethod calls to non-classmethods (#106977) Message-ID: https://github.com/python/cpython/commit/e5d5522612e03af3941db1d270bf6caebf330b8a commit: e5d5522612e03af3941db1d270bf6caebf330b8a branch: main author: Carl Meyer committer: carljm date: 2023-07-24T13:14:56-07:00 summary: gh-106917: fix super classmethod calls to non-classmethods (#106977) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-21-14-37-48.gh-issue-106917.1jWp_m.rst M Lib/test/test_super.py M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h diff --git a/Lib/test/test_super.py b/Lib/test/test_super.py index 664cf70b3cf0f..43162c540b55a 100644 --- a/Lib/test/test_super.py +++ b/Lib/test/test_super.py @@ -5,6 +5,9 @@ from test import shadowed_super +ADAPTIVE_WARMUP_DELAY = 2 + + class A: def f(self): return 'A' @@ -419,8 +422,47 @@ def test(name): super(MyType, type(mytype)).__setattr__(mytype, "bar", 1) self.assertEqual(mytype.bar, 1) - test("foo1") - test("foo2") + for _ in range(ADAPTIVE_WARMUP_DELAY): + test("foo1") + + def test_reassigned_new(self): + class A: + def __new__(cls): + pass + + def __init_subclass__(cls): + if "__new__" not in cls.__dict__: + cls.__new__ = cls.__new__ + + class B(A): + pass + + class C(B): + def __new__(cls): + return super().__new__(cls) + + for _ in range(ADAPTIVE_WARMUP_DELAY): + C() + + def test_mixed_staticmethod_hierarchy(self): + # This test is just a desugared version of `test_reassigned_new` + class A: + @staticmethod + def some(cls, *args, **kwargs): + self.assertFalse(args) + self.assertFalse(kwargs) + + class B(A): + def some(cls, *args, **kwargs): + return super().some(cls, *args, **kwargs) + + class C(B): + @staticmethod + def some(cls): + return super().some(cls) + + for _ in range(ADAPTIVE_WARMUP_DELAY): + C.some(C) if __name__ == "__main__": diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-21-14-37-48.gh-issue-106917.1jWp_m.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-21-14-37-48.gh-issue-106917.1jWp_m.rst new file mode 100644 index 0000000000000..82c74d5465458 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-21-14-37-48.gh-issue-106917.1jWp_m.rst @@ -0,0 +1,4 @@ +Fix classmethod-style :func:`super` method calls (i.e., where the second +argument to :func:`super`, or the implied second argument drawn from +``self/cls`` in the case of zero-arg super, is a type) when the target of +the call is not a classmethod. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 7ce8978e998b9..0705f5b0aac43 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1721,7 +1721,7 @@ dummy_func( PyTypeObject *cls = (PyTypeObject *)class; int method_found = 0; res2 = _PySuper_Lookup(cls, self, name, - cls->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); + Py_TYPE(self)->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); Py_DECREF(global_super); Py_DECREF(class); if (res2 == NULL) { diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 064965a1a1ae7..0f04b428ba805 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1406,7 +1406,7 @@ PyTypeObject *cls = (PyTypeObject *)class; int method_found = 0; res2 = _PySuper_Lookup(cls, self, name, - cls->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); + Py_TYPE(self)->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); Py_DECREF(global_super); Py_DECREF(class); if (res2 == NULL) { diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 9d25fc604314a..02ad69a6bc4da 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -2154,7 +2154,7 @@ PyTypeObject *cls = (PyTypeObject *)class; int method_found = 0; res2 = _PySuper_Lookup(cls, self, name, - cls->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); + Py_TYPE(self)->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); Py_DECREF(global_super); Py_DECREF(class); if (res2 == NULL) { From webhook-mailer at python.org Mon Jul 24 16:29:54 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 24 Jul 2023 20:29:54 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: annotate `format_docstring()` (#107200) Message-ID: https://github.com/python/cpython/commit/ac2d85a174d6fd0b1fd06aca6657fb002cd120f3 commit: ac2d85a174d6fd0b1fd06aca6657fb002cd120f3 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-24T20:29:50Z summary: gh-104050: Argument clinic: annotate `format_docstring()` (#107200) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index c9c57f1fdbb3b..59e0bf684d32e 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -5294,8 +5294,10 @@ def state_function_docstring(self, line): new_docstring += line self.function.docstring = new_docstring - def format_docstring(self): + def format_docstring(self) -> str: f = self.function + assert f is not None + assert f.full_name is not None new_or_init = f.kind.new_or_init if new_or_init and not f.docstring: @@ -5338,7 +5340,7 @@ def format_docstring(self): right_bracket_count = 0 - def fix_right_bracket_count(desired): + def fix_right_bracket_count(desired: int) -> str: nonlocal right_bracket_count s = '' while right_bracket_count < desired: @@ -5372,7 +5374,7 @@ def fix_right_bracket_count(desired): last_p = parameters[-1] line_length = len(''.join(text)) indent = " " * line_length - def add_parameter(text): + def add_parameter(text: str) -> None: nonlocal line_length nonlocal first_parameter if first_parameter: @@ -5488,9 +5490,9 @@ def add_parameter(text): add(p.name) add('\n') add(textwrap.indent(rstrip_lines(p.docstring.rstrip()), " ")) - parameters = output() - if parameters: - parameters += '\n' + parameters_output = output() + if parameters_output: + parameters_output += '\n' ## ## docstring body @@ -5538,7 +5540,7 @@ def add_parameter(text): add(docstring) docstring = output() - docstring = linear_format(docstring, parameters=parameters) + docstring = linear_format(docstring, parameters=parameters_output) docstring = docstring.rstrip() return docstring From webhook-mailer at python.org Mon Jul 24 16:59:55 2023 From: webhook-mailer at python.org (vstinner) Date: Mon, 24 Jul 2023 20:59:55 -0000 Subject: [Python-checkins] [3.12] gh-102304: Rename _Py_IncRefTotal_DO_NOT_USE_THIS() (GH-107193) (#107199) Message-ID: https://github.com/python/cpython/commit/3923639a77656915c3499b0283a45da727308f2a commit: 3923639a77656915c3499b0283a45da727308f2a branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-24T22:59:51+02:00 summary: [3.12] gh-102304: Rename _Py_IncRefTotal_DO_NOT_USE_THIS() (GH-107193) (#107199) gh-102304: Rename _Py_IncRefTotal_DO_NOT_USE_THIS() (GH-107193) * Rename _Py_IncRefTotal_DO_NOT_USE_THIS() to _Py_INCREF_IncRefTotal() * Rename _Py_DecRefTotal_DO_NOT_USE_THIS() to _Py_DECREF_DecRefTotal() * Remove temporary _Py_INC_REFTOTAL() and _Py_DEC_REFTOTAL() macros (cherry picked from commit 8ebc9fc321ba1eeb3282c2170f351c54956893e6) Co-authored-by: Victor Stinner files: M Include/object.h M Objects/object.c diff --git a/Include/object.h b/Include/object.h index 3ef64511399c6..7564b9623be79 100644 --- a/Include/object.h +++ b/Include/object.h @@ -588,10 +588,8 @@ you can count such references to the type object.) #if defined(Py_REF_DEBUG) && !defined(Py_LIMITED_API) PyAPI_FUNC(void) _Py_NegativeRefcount(const char *filename, int lineno, PyObject *op); -PyAPI_FUNC(void) _Py_IncRefTotal_DO_NOT_USE_THIS(void); -PyAPI_FUNC(void) _Py_DecRefTotal_DO_NOT_USE_THIS(void); -# define _Py_INC_REFTOTAL() _Py_IncRefTotal_DO_NOT_USE_THIS() -# define _Py_DEC_REFTOTAL() _Py_DecRefTotal_DO_NOT_USE_THIS() +PyAPI_FUNC(void) _Py_INCREF_IncRefTotal(void); +PyAPI_FUNC(void) _Py_DECREF_DecRefTotal(void); #endif // Py_REF_DEBUG && !Py_LIMITED_API PyAPI_FUNC(void) _Py_Dealloc(PyObject *); @@ -640,7 +638,7 @@ static inline Py_ALWAYS_INLINE void Py_INCREF(PyObject *op) #endif _Py_INCREF_STAT_INC(); #ifdef Py_REF_DEBUG - _Py_INC_REFTOTAL(); + _Py_INCREF_IncRefTotal(); #endif #endif } @@ -669,7 +667,7 @@ static inline void Py_DECREF(const char *filename, int lineno, PyObject *op) return; } _Py_DECREF_STAT_INC(); - _Py_DEC_REFTOTAL(); + _Py_DECREF_DecRefTotal(); if (--op->ob_refcnt != 0) { if (op->ob_refcnt < 0) { _Py_NegativeRefcount(filename, lineno, op); @@ -697,9 +695,6 @@ static inline Py_ALWAYS_INLINE void Py_DECREF(PyObject *op) #define Py_DECREF(op) Py_DECREF(_PyObject_CAST(op)) #endif -#undef _Py_INC_REFTOTAL -#undef _Py_DEC_REFTOTAL - /* Safely decref `op` and set `op` to NULL, especially useful in tp_clear * and tp_dealloc implementations. diff --git a/Objects/object.c b/Objects/object.c index b8bdf459201d9..6a73c3588da79 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -206,14 +206,14 @@ _Py_NegativeRefcount(const char *filename, int lineno, PyObject *op) /* This is used strictly by Py_INCREF(). */ void -_Py_IncRefTotal_DO_NOT_USE_THIS(void) +_Py_INCREF_IncRefTotal(void) { reftotal_increment(_PyInterpreterState_GET()); } /* This is used strictly by Py_DECREF(). */ void -_Py_DecRefTotal_DO_NOT_USE_THIS(void) +_Py_DECREF_DecRefTotal(void) { reftotal_decrement(_PyInterpreterState_GET()); } From webhook-mailer at python.org Mon Jul 24 17:09:03 2023 From: webhook-mailer at python.org (iritkatriel) Date: Mon, 24 Jul 2023 21:09:03 -0000 Subject: [Python-checkins] gh-106149: move _PyCfg_BasicblockLastInstr and make it local to flowgraph.c (#107180) Message-ID: https://github.com/python/cpython/commit/7608fa8fd50a8e4caff5f56e50e4e85848617d81 commit: 7608fa8fd50a8e4caff5f56e50e4e85848617d81 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-24T22:08:59+01:00 summary: gh-106149: move _PyCfg_BasicblockLastInstr and make it local to flowgraph.c (#107180) files: M Include/internal/pycore_flowgraph.h M Python/flowgraph.c diff --git a/Include/internal/pycore_flowgraph.h b/Include/internal/pycore_flowgraph.h index 4a01574809fff..83b60ed849a95 100644 --- a/Include/internal/pycore_flowgraph.h +++ b/Include/internal/pycore_flowgraph.h @@ -88,25 +88,12 @@ int _PyCfgBuilder_Addop(_PyCfgBuilder *g, int opcode, int oparg, _PyCompilerSrcL int _PyCfgBuilder_Init(_PyCfgBuilder *g); void _PyCfgBuilder_Fini(_PyCfgBuilder *g); -_PyCfgInstruction* _PyCfg_BasicblockLastInstr(const _PyCfgBasicblock *b); int _PyCfg_OptimizeCodeUnit(_PyCfgBuilder *g, PyObject *consts, PyObject *const_cache, int code_flags, int nlocals, int nparams, int firstlineno); int _PyCfg_Stackdepth(_PyCfgBasicblock *entryblock, int code_flags); void _PyCfg_ConvertPseudoOps(_PyCfgBasicblock *entryblock); int _PyCfg_ResolveJumps(_PyCfgBuilder *g); - -static inline int -basicblock_nofallthrough(const _PyCfgBasicblock *b) { - _PyCfgInstruction *last = _PyCfg_BasicblockLastInstr(b); - return (last && - (IS_SCOPE_EXIT_OPCODE(last->i_opcode) || - IS_UNCONDITIONAL_JUMP_OPCODE(last->i_opcode))); -} - -#define BB_NO_FALLTHROUGH(B) (basicblock_nofallthrough(B)) -#define BB_HAS_FALLTHROUGH(B) (!basicblock_nofallthrough(B)) - PyCodeObject * _PyAssemble_MakeCodeObject(_PyCompile_CodeUnitMetadata *u, PyObject *const_cache, PyObject *consts, int maxdepth, _PyCompile_InstructionSequence *instrs, diff --git a/Python/flowgraph.c b/Python/flowgraph.c index e485ed103147a..f8e30ef7515a2 100644 --- a/Python/flowgraph.c +++ b/Python/flowgraph.c @@ -137,6 +137,27 @@ basicblock_append_instructions(basicblock *target, basicblock *source) return SUCCESS; } +static cfg_instr * +basicblock_last_instr(const basicblock *b) { + assert(b->b_iused >= 0); + if (b->b_iused > 0) { + assert(b->b_instr != NULL); + return &b->b_instr[b->b_iused - 1]; + } + return NULL; +} + +static inline int +basicblock_nofallthrough(const _PyCfgBasicblock *b) { + _PyCfgInstruction *last = basicblock_last_instr(b); + return (last && + (IS_SCOPE_EXIT_OPCODE(last->i_opcode) || + IS_UNCONDITIONAL_JUMP_OPCODE(last->i_opcode))); +} + +#define BB_NO_FALLTHROUGH(B) (basicblock_nofallthrough(B)) +#define BB_HAS_FALLTHROUGH(B) (!basicblock_nofallthrough(B)) + static basicblock * copy_basicblock(cfg_builder *g, basicblock *block) { @@ -186,7 +207,7 @@ dump_instr(cfg_instr *i) static inline int basicblock_returns(const basicblock *b) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); return last && (last->i_opcode == RETURN_VALUE || last->i_opcode == RETURN_CONST); } @@ -228,26 +249,16 @@ cfg_builder_use_next_block(cfg_builder *g, basicblock *block) return block; } -cfg_instr * -_PyCfg_BasicblockLastInstr(const basicblock *b) { - assert(b->b_iused >= 0); - if (b->b_iused > 0) { - assert(b->b_instr != NULL); - return &b->b_instr[b->b_iused - 1]; - } - return NULL; -} - static inline int basicblock_exits_scope(const basicblock *b) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); return last && IS_SCOPE_EXIT_OPCODE(last->i_opcode); } static bool cfg_builder_current_block_is_terminated(cfg_builder *g) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(g->g_curblock); + cfg_instr *last = basicblock_last_instr(g->g_curblock); if (last && IS_TERMINATOR_OPCODE(last->i_opcode)) { return true; } @@ -371,7 +382,7 @@ no_empty_basic_blocks(cfg_builder *g) { static bool no_redundant_jumps(cfg_builder *g) { for (basicblock *b = g->g_entryblock; b != NULL; b = b->b_next) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); if (last != NULL) { if (IS_UNCONDITIONAL_JUMP_OPCODE(last->i_opcode)) { assert(last->i_target != b->b_next); @@ -390,7 +401,7 @@ no_redundant_jumps(cfg_builder *g) { static int normalize_jumps_in_block(cfg_builder *g, basicblock *b) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); if (last == NULL || !is_jump(last) || IS_UNCONDITIONAL_JUMP_OPCODE(last->i_opcode)) { return SUCCESS; @@ -953,7 +964,7 @@ remove_redundant_jumps(cfg_builder *g) { */ assert(no_empty_basic_blocks(g)); for (basicblock *b = g->g_entryblock; b != NULL; b = b->b_next) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); assert(last != NULL); assert(!IS_ASSEMBLER_OPCODE(last->i_opcode)); if (IS_UNCONDITIONAL_JUMP_OPCODE(last->i_opcode)) { @@ -979,7 +990,7 @@ remove_redundant_jumps(cfg_builder *g) { */ static int inline_small_exit_blocks(basicblock *bb) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(bb); + cfg_instr *last = basicblock_last_instr(bb); if (last == NULL) { return 0; } @@ -1715,7 +1726,7 @@ scan_block_for_locals(basicblock *b, basicblock ***sp) if (b->b_next && BB_HAS_FALLTHROUGH(b)) { maybe_push(b->b_next, unsafe_mask, sp); } - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); if (last && is_jump(last)) { assert(last->i_target != NULL); maybe_push(last->i_target, unsafe_mask, sp); @@ -2020,7 +2031,7 @@ push_cold_blocks_to_end(cfg_builder *g, int code_flags) { b->b_next = explicit_jump; /* set target */ - cfg_instr *last = _PyCfg_BasicblockLastInstr(explicit_jump); + cfg_instr *last = basicblock_last_instr(explicit_jump); last->i_target = explicit_jump->b_next; } } @@ -2126,7 +2137,7 @@ duplicate_exits_without_lineno(cfg_builder *g) */ basicblock *entryblock = g->g_entryblock; for (basicblock *b = entryblock; b != NULL; b = b->b_next) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); assert(last != NULL); if (is_jump(last)) { basicblock *target = last->i_target; @@ -2150,7 +2161,7 @@ duplicate_exits_without_lineno(cfg_builder *g) for (basicblock *b = entryblock; b != NULL; b = b->b_next) { if (BB_HAS_FALLTHROUGH(b) && b->b_next && b->b_iused > 0) { if (is_exit_without_lineno(b->b_next)) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); assert(last != NULL); b->b_next->b_instr[0].i_loc = last->i_loc; } @@ -2170,7 +2181,7 @@ duplicate_exits_without_lineno(cfg_builder *g) static void propagate_line_numbers(basicblock *entryblock) { for (basicblock *b = entryblock; b != NULL; b = b->b_next) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); if (last == NULL) { continue; } @@ -2210,7 +2221,7 @@ guarantee_lineno_for_exits(basicblock *entryblock, int firstlineno) { int lineno = firstlineno; assert(lineno > 0); for (basicblock *b = entryblock; b != NULL; b = b->b_next) { - cfg_instr *last = _PyCfg_BasicblockLastInstr(b); + cfg_instr *last = basicblock_last_instr(b); if (last == NULL) { continue; } From webhook-mailer at python.org Mon Jul 24 17:13:21 2023 From: webhook-mailer at python.org (carljm) Date: Mon, 24 Jul 2023 21:13:21 -0000 Subject: [Python-checkins] [3.12] gh-106917: fix super classmethod calls to non-classmethods (GH-106977). (#107204) Message-ID: https://github.com/python/cpython/commit/5fd028b677ae1dd20a60fc2f43d4380b809d70f0 commit: 5fd028b677ae1dd20a60fc2f43d4380b809d70f0 branch: 3.12 author: Carl Meyer committer: carljm date: 2023-07-24T21:13:17Z summary: [3.12] gh-106917: fix super classmethod calls to non-classmethods (GH-106977). (#107204) (cherry picked from commit e5d5522612e03af3941db1d270bf6caebf330b8a) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-21-14-37-48.gh-issue-106917.1jWp_m.rst M Lib/test/test_super.py M Python/bytecodes.c M Python/generated_cases.c.h diff --git a/Lib/test/test_super.py b/Lib/test/test_super.py index 664cf70b3cf0f..43162c540b55a 100644 --- a/Lib/test/test_super.py +++ b/Lib/test/test_super.py @@ -5,6 +5,9 @@ from test import shadowed_super +ADAPTIVE_WARMUP_DELAY = 2 + + class A: def f(self): return 'A' @@ -419,8 +422,47 @@ def test(name): super(MyType, type(mytype)).__setattr__(mytype, "bar", 1) self.assertEqual(mytype.bar, 1) - test("foo1") - test("foo2") + for _ in range(ADAPTIVE_WARMUP_DELAY): + test("foo1") + + def test_reassigned_new(self): + class A: + def __new__(cls): + pass + + def __init_subclass__(cls): + if "__new__" not in cls.__dict__: + cls.__new__ = cls.__new__ + + class B(A): + pass + + class C(B): + def __new__(cls): + return super().__new__(cls) + + for _ in range(ADAPTIVE_WARMUP_DELAY): + C() + + def test_mixed_staticmethod_hierarchy(self): + # This test is just a desugared version of `test_reassigned_new` + class A: + @staticmethod + def some(cls, *args, **kwargs): + self.assertFalse(args) + self.assertFalse(kwargs) + + class B(A): + def some(cls, *args, **kwargs): + return super().some(cls, *args, **kwargs) + + class C(B): + @staticmethod + def some(cls): + return super().some(cls) + + for _ in range(ADAPTIVE_WARMUP_DELAY): + C.some(C) if __name__ == "__main__": diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-21-14-37-48.gh-issue-106917.1jWp_m.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-21-14-37-48.gh-issue-106917.1jWp_m.rst new file mode 100644 index 0000000000000..82c74d5465458 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-21-14-37-48.gh-issue-106917.1jWp_m.rst @@ -0,0 +1,4 @@ +Fix classmethod-style :func:`super` method calls (i.e., where the second +argument to :func:`super`, or the implied second argument drawn from +``self/cls`` in the case of zero-arg super, is a type) when the target of +the call is not a classmethod. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 0baf2451ee4f8..b914afa07fba4 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1663,7 +1663,7 @@ dummy_func( PyTypeObject *cls = (PyTypeObject *)class; int method_found = 0; res2 = _PySuper_Lookup(cls, self, name, - cls->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); + Py_TYPE(self)->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); Py_DECREF(global_super); Py_DECREF(class); if (res2 == NULL) { diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 103373ec0db01..281c8b5e5245b 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -2411,7 +2411,7 @@ PyTypeObject *cls = (PyTypeObject *)class; int method_found = 0; res2 = _PySuper_Lookup(cls, self, name, - cls->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); + Py_TYPE(self)->tp_getattro == PyObject_GenericGetAttr ? &method_found : NULL); Py_DECREF(global_super); Py_DECREF(class); if (res2 == NULL) { From webhook-mailer at python.org Mon Jul 24 17:24:02 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 24 Jul 2023 21:24:02 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: misc improvements to type annotation coverage (#107206) Message-ID: https://github.com/python/cpython/commit/d93fe5afe2832a28b1e76e0a89d0a35c0d064bde commit: d93fe5afe2832a28b1e76e0a89d0a35c0d064bde branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-24T21:23:58Z summary: gh-104050: Argument clinic: misc improvements to type annotation coverage (#107206) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 59e0bf684d32e..c62407830a982 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2640,7 +2640,7 @@ class LandMine: # try to access any __message__: str - def __getattribute__(self, name: str): + def __getattribute__(self, name: str) -> Any: if name in ('__repr__', '__message__'): return super().__getattribute__(name) # raise RuntimeError(repr(name)) @@ -3896,7 +3896,7 @@ def converter_init(self, *, accept: TypeSet = {buffer}) -> None: self.format_unit = format_unit - def cleanup(self): + def cleanup(self) -> str: name = self.name return "".join(["if (", name, ".obj) {\n PyBuffer_Release(&", name, ");\n}\n"]) @@ -4115,7 +4115,7 @@ def __init__( self, *, py_default: str | None = None, - **kwargs + **kwargs: Any ) -> None: self.py_default = py_default try: @@ -4493,7 +4493,7 @@ def directive_destination( self, name: str, command: str, - *args + *args: str ) -> None: match command: case "new": @@ -4847,12 +4847,13 @@ def state_parameters_start(self, line: str | None) -> None: return self.next(self.state_parameter, line) - def to_required(self): + def to_required(self) -> None: """ Transition to the "required" parameter state. """ if self.parameter_state is not ParamState.REQUIRED: self.parameter_state = ParamState.REQUIRED + assert self.function is not None for p in self.function.parameters.values(): p.group = -p.group @@ -5000,7 +5001,7 @@ def parse_parameter(self, line: str) -> None: # of disallowed ast nodes. class DetectBadNodes(ast.NodeVisitor): bad = False - def bad_node(self, node): + def bad_node(self, node: ast.AST) -> None: self.bad = True # inline function call @@ -5248,7 +5249,9 @@ def state_parameter_docstring_start(self, line: str | None) -> None: # every line of the docstring must start with at least F spaces, # where F > P. # these F spaces will be stripped. - def state_parameter_docstring(self, line): + def state_parameter_docstring(self, line: str | None) -> None: + assert line is not None + stripped = line.strip() if stripped.startswith('#'): return @@ -5263,7 +5266,7 @@ def state_parameter_docstring(self, line): assert self.indent.depth == 1 return self.next(self.state_function_docstring, line) - assert self.function.parameters + assert self.function and self.function.parameters last_parameter = next(reversed(list(self.function.parameters.values()))) new_docstring = last_parameter.docstring @@ -5276,7 +5279,10 @@ def state_parameter_docstring(self, line): last_parameter.docstring = new_docstring # the final stanza of the DSL is the docstring. - def state_function_docstring(self, line): + def state_function_docstring(self, line: str | None) -> None: + assert self.function is not None + assert line is not None + if self.group: fail("Function " + self.function.name + " has a ] without a matching [.") From webhook-mailer at python.org Mon Jul 24 17:38:54 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 24 Jul 2023 21:38:54 -0000 Subject: [Python-checkins] gh-104050: Argument Clinic: Annotate `CRenderData.__init__` (#107207) Message-ID: https://github.com/python/cpython/commit/4bbf071635504ee1c6d44f99c98e3fad191a3b13 commit: 4bbf071635504ee1c6d44f99c98e3fad191a3b13 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-24T21:38:50Z summary: gh-104050: Argument Clinic: Annotate `CRenderData.__init__` (#107207) Argument Clinic: Annotate `CRenderData.__init__` files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index c62407830a982..ff2151505b973 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -372,36 +372,36 @@ def version_comparitor(version1: str, version2: str) -> Literal[-1, 0, 1]: class CRenderData: - def __init__(self): + def __init__(self) -> None: # The C statements to declare variables. # Should be full lines with \n eol characters. - self.declarations = [] + self.declarations: list[str] = [] # The C statements required to initialize the variables before the parse call. # Should be full lines with \n eol characters. - self.initializers = [] + self.initializers: list[str] = [] # The C statements needed to dynamically modify the values # parsed by the parse call, before calling the impl. - self.modifications = [] + self.modifications: list[str] = [] # The entries for the "keywords" array for PyArg_ParseTuple. # Should be individual strings representing the names. - self.keywords = [] + self.keywords: list[str] = [] # The "format units" for PyArg_ParseTuple. # Should be individual strings that will get - self.format_units = [] + self.format_units: list[str] = [] # The varargs arguments for PyArg_ParseTuple. - self.parse_arguments = [] + self.parse_arguments: list[str] = [] # The parameter declarations for the impl function. - self.impl_parameters = [] + self.impl_parameters: list[str] = [] # The arguments to the impl function at the time it's called. - self.impl_arguments = [] + self.impl_arguments: list[str] = [] # For return converters: the name of the variable that # should receive the value returned by the impl. @@ -411,17 +411,17 @@ def __init__(self): # value from the parse function. This is also where # you should check the _return_value for errors, and # "goto exit" if there are any. - self.return_conversion = [] + self.return_conversion: list[str] = [] self.converter_retval = "_return_value" # The C statements required to do some operations # after the end of parsing but before cleaning up. # These operations may be, for example, memory deallocations which # can only be done without any error happening during argument parsing. - self.post_parsing = [] + self.post_parsing: list[str] = [] # The C statements required to clean up after the impl call. - self.cleanup = [] + self.cleanup: list[str] = [] class FormatCounterFormatter(string.Formatter): From webhook-mailer at python.org Mon Jul 24 20:54:13 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 00:54:13 -0000 Subject: [Python-checkins] gh-107211: Rename PySymtable_Lookup() to _PySymtable_Lookup() (#107212) Message-ID: https://github.com/python/cpython/commit/2e0744955f1c213a55738de848a9d928b78ef289 commit: 2e0744955f1c213a55738de848a9d928b78ef289 branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T00:54:09Z summary: gh-107211: Rename PySymtable_Lookup() to _PySymtable_Lookup() (#107212) Rename the internal PySymtable_Lookup() function to _PySymtable_Lookup() and no longer export it. files: M Include/internal/pycore_symtable.h M Python/compile.c M Python/symtable.c diff --git a/Include/internal/pycore_symtable.h b/Include/internal/pycore_symtable.h index c8e0578a23175..1d782ca2c96e0 100644 --- a/Include/internal/pycore_symtable.h +++ b/Include/internal/pycore_symtable.h @@ -101,7 +101,7 @@ extern struct symtable* _PySymtable_Build( struct _mod *mod, PyObject *filename, PyFutureFeatures *future); -PyAPI_FUNC(PySTEntryObject *) PySymtable_Lookup(struct symtable *, void *); +extern PySTEntryObject* _PySymtable_Lookup(struct symtable *, void *); extern void _PySymtable_Free(struct symtable *); diff --git a/Python/compile.c b/Python/compile.c index 600118768e639..5915a4bc2f81b 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -1280,7 +1280,7 @@ compiler_enter_scope(struct compiler *c, identifier name, u->u_metadata.u_argcount = 0; u->u_metadata.u_posonlyargcount = 0; u->u_metadata.u_kwonlyargcount = 0; - u->u_ste = PySymtable_Lookup(c->c_st, key); + u->u_ste = _PySymtable_Lookup(c->c_st, key); if (!u->u_ste) { compiler_unit_free(u); return ERROR; @@ -5685,7 +5685,7 @@ compiler_comprehension(struct compiler *c, expr_ty e, int type, comprehension_ty outermost; int scope_type = c->u->u_scope_type; int is_top_level_await = IS_TOP_LEVEL_AWAIT(c); - PySTEntryObject *entry = PySymtable_Lookup(c->c_st, (void *)e); + PySTEntryObject *entry = _PySymtable_Lookup(c->c_st, (void *)e); if (entry == NULL) { goto error; } diff --git a/Python/symtable.c b/Python/symtable.c index e2c00d17480dd..2407367a58a9b 100644 --- a/Python/symtable.c +++ b/Python/symtable.c @@ -389,7 +389,7 @@ _PySymtable_Free(struct symtable *st) } PySTEntryObject * -PySymtable_Lookup(struct symtable *st, void *key) +_PySymtable_Lookup(struct symtable *st, void *key) { PyObject *k, *v; From webhook-mailer at python.org Mon Jul 24 21:44:15 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 01:44:15 -0000 Subject: [Python-checkins] gh-107211: No longer export internal functions (1) (#107213) Message-ID: https://github.com/python/cpython/commit/11306a91bd316bb8d9a42ff188626ac5f76b3684 commit: 11306a91bd316bb8d9a42ff188626ac5f76b3684 branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T03:44:11+02:00 summary: gh-107211: No longer export internal functions (1) (#107213) No longer export these 49 internal C API functions: * _PyArgv_AsWstrList() * _PyCode_New() * _PyCode_Validate() * _PyFloat_DebugMallocStats() * _PyFloat_FormatAdvancedWriter() * _PyImport_CheckSubinterpIncompatibleExtensionAllowed() * _PyImport_ClearExtension() * _PyImport_GetModuleId() * _PyImport_SetModuleString() * _PyInterpreterState_IDDecref() * _PyInterpreterState_IDIncref() * _PyInterpreterState_IDInitref() * _PyInterpreterState_LookUpID() * _PyWideStringList_AsList() * _PyWideStringList_CheckConsistency() * _PyWideStringList_Clear() * _PyWideStringList_Copy() * _PyWideStringList_Extend() * _Py_ClearArgcArgv() * _Py_DecodeUTF8Ex() * _Py_DecodeUTF8_surrogateescape() * _Py_EncodeLocaleRaw() * _Py_EncodeUTF8Ex() * _Py_GetEnv() * _Py_GetForceASCII() * _Py_GetLocaleEncoding() * _Py_GetLocaleEncodingObject() * _Py_GetLocaleconvNumeric() * _Py_ResetForceASCII() * _Py_device_encoding() * _Py_dg_dtoa() * _Py_dg_freedtoa() * _Py_dg_strtod() * _Py_get_blocking() * _Py_get_env_flag() * _Py_get_inheritable() * _Py_get_osfhandle_noraise() * _Py_get_xoption() * _Py_open() * _Py_open_osfhandle() * _Py_open_osfhandle_noraise() * _Py_read() * _Py_set_blocking() * _Py_str_to_int() * _Py_wfopen() * _Py_wgetcwd() * _Py_wreadlink() * _Py_wrealpath() * _Py_write() files: M Include/internal/pycore_call.h M Include/internal/pycore_code.h M Include/internal/pycore_compile.h M Include/internal/pycore_dtoa.h M Include/internal/pycore_fileutils.h M Include/internal/pycore_floatobject.h M Include/internal/pycore_genobject.h M Include/internal/pycore_import.h M Include/internal/pycore_initconfig.h M Include/internal/pycore_interp.h diff --git a/Include/internal/pycore_call.h b/Include/internal/pycore_call.h index 9c32035d474b3..c0d61785802f6 100644 --- a/Include/internal/pycore_call.h +++ b/Include/internal/pycore_call.h @@ -22,8 +22,8 @@ extern "C" { #define _PY_FASTCALL_SMALL_STACK 5 -// Export for shared stdlib extensions like the math extension, -// function used via inlined _PyObject_VectorcallTstate() function. +// Export for 'math' shared extension, function used +// via inlined _PyObject_VectorcallTstate() function. PyAPI_FUNC(PyObject*) _Py_CheckFunctionResult( PyThreadState *tstate, PyObject *callable, @@ -68,7 +68,7 @@ extern PyObject * _PyObject_CallMethodFormat( const char *format, ...); -// Export for shared stdlib extensions like the array extension +// Export for 'array' shared extension PyAPI_FUNC(PyObject*) _PyObject_CallMethod( PyObject *obj, PyObject *name, @@ -120,8 +120,8 @@ _PyObject_CallMethodIdOneArg(PyObject *self, _Py_Identifier *name, PyObject *arg // Call callable using tp_call. Arguments are like PyObject_Vectorcall(), // except that nargs is plainly the number of arguments without flags. // -// Export for shared stdlib extensions like the math extension, -// function used via inlined _PyObject_VectorcallTstate() function. +// Export for 'math' shared extension, function used +// via inlined _PyObject_VectorcallTstate() function. PyAPI_FUNC(PyObject*) _PyObject_MakeTpCall( PyThreadState *tstate, PyObject *callable, diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index b6b1aeca6e5c5..62a943b115f9e 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -203,8 +203,8 @@ struct _PyCodeConstructor { // back to a regular function signature. Regardless, this approach // wouldn't be appropriate if this weren't a strictly internal API. // (See the comments in https://github.com/python/cpython/pull/26258.) -PyAPI_FUNC(int) _PyCode_Validate(struct _PyCodeConstructor *); -PyAPI_FUNC(PyCodeObject *) _PyCode_New(struct _PyCodeConstructor *); +extern int _PyCode_Validate(struct _PyCodeConstructor *); +extern PyCodeObject* _PyCode_New(struct _PyCodeConstructor *); /* Private API */ @@ -274,7 +274,7 @@ extern int _PyStaticCode_Init(PyCodeObject *co); #define EVAL_CALL_STAT_INC_IF_FUNCTION(name, callable) \ do { if (_py_stats && PyFunction_Check(callable)) _py_stats->call_stats.eval_calls[name]++; } while (0) -// Used by the _opcode extension which is built as a shared library +// Export for stdlib '_opcode' shared extension PyAPI_FUNC(PyObject*) _Py_GetSpecializationStats(void); #else diff --git a/Include/internal/pycore_compile.h b/Include/internal/pycore_compile.h index beb37cced06db..fa2f64021e73c 100644 --- a/Include/internal/pycore_compile.h +++ b/Include/internal/pycore_compile.h @@ -11,7 +11,7 @@ extern "C" { struct _arena; // Type defined in pycore_pyarena.h struct _mod; // Type defined in pycore_ast.h -// Export the symbol for test_peg_generator (built as a library) +// Export for 'test_peg_generator' shared extension PyAPI_FUNC(PyCodeObject*) _PyAST_Compile( struct _mod *mod, PyObject *filename, @@ -91,6 +91,7 @@ int _PyCompile_ConstCacheMergeOne(PyObject *const_cache, PyObject **obj); /* Access compiler internals for unit testing */ +// Export for '_testinternalcapi' shared extension PyAPI_FUNC(PyObject*) _PyCompile_CleanDoc(PyObject *doc); PyAPI_FUNC(PyObject*) _PyCompile_CodeGen( diff --git a/Include/internal/pycore_dtoa.h b/Include/internal/pycore_dtoa.h index 4d9681d59a64f..ac62a4d300720 100644 --- a/Include/internal/pycore_dtoa.h +++ b/Include/internal/pycore_dtoa.h @@ -60,10 +60,10 @@ struct _dtoa_state { /* These functions are used by modules compiled as C extension like math: they must be exported. */ -PyAPI_FUNC(double) _Py_dg_strtod(const char *str, char **ptr); -PyAPI_FUNC(char *) _Py_dg_dtoa(double d, int mode, int ndigits, - int *decpt, int *sign, char **rve); -PyAPI_FUNC(void) _Py_dg_freedtoa(char *s); +extern double _Py_dg_strtod(const char *str, char **ptr); +extern char* _Py_dg_dtoa(double d, int mode, int ndigits, + int *decpt, int *sign, char **rve); +extern void _Py_dg_freedtoa(char *s); #endif // _PY_SHORT_FLOAT_REPR == 1 diff --git a/Include/internal/pycore_fileutils.h b/Include/internal/pycore_fileutils.h index 7ba9b3ee58be4..daa32c0dff609 100644 --- a/Include/internal/pycore_fileutils.h +++ b/Include/internal/pycore_fileutils.h @@ -52,11 +52,11 @@ PyAPI_FUNC(int) _Py_EncodeLocaleEx( int current_locale, _Py_error_handler errors); -PyAPI_FUNC(char*) _Py_EncodeLocaleRaw( +extern char* _Py_EncodeLocaleRaw( const wchar_t *text, size_t *error_pos); -PyAPI_FUNC(PyObject *) _Py_device_encoding(int); +extern PyObject* _Py_device_encoding(int); #if defined(MS_WINDOWS) || defined(__APPLE__) /* On Windows, the count parameter of read() is an int (bpo-9015, bpo-9611). @@ -109,7 +109,7 @@ PyAPI_FUNC(int) _Py_stat( PyObject *path, struct stat *status); -PyAPI_FUNC(int) _Py_open( +extern int _Py_open( const char *pathname, int flags); @@ -117,16 +117,16 @@ PyAPI_FUNC(int) _Py_open_noraise( const char *pathname, int flags); -PyAPI_FUNC(FILE *) _Py_wfopen( +extern FILE* _Py_wfopen( const wchar_t *path, const wchar_t *mode); -PyAPI_FUNC(Py_ssize_t) _Py_read( +extern Py_ssize_t _Py_read( int fd, void *buf, size_t count); -PyAPI_FUNC(Py_ssize_t) _Py_write( +extern Py_ssize_t _Py_write( int fd, const void *buf, size_t count); @@ -137,7 +137,7 @@ PyAPI_FUNC(Py_ssize_t) _Py_write_noraise( size_t count); #ifdef HAVE_READLINK -PyAPI_FUNC(int) _Py_wreadlink( +extern int _Py_wreadlink( const wchar_t *path, wchar_t *buf, /* Number of characters of 'buf' buffer @@ -146,7 +146,7 @@ PyAPI_FUNC(int) _Py_wreadlink( #endif #ifdef HAVE_REALPATH -PyAPI_FUNC(wchar_t*) _Py_wrealpath( +extern wchar_t* _Py_wrealpath( const wchar_t *path, wchar_t *resolved_path, /* Number of characters of 'resolved_path' buffer @@ -154,13 +154,13 @@ PyAPI_FUNC(wchar_t*) _Py_wrealpath( size_t resolved_path_len); #endif -PyAPI_FUNC(wchar_t*) _Py_wgetcwd( +extern wchar_t* _Py_wgetcwd( wchar_t *buf, /* Number of characters of 'buf' buffer including the trailing NUL character */ size_t buflen); -PyAPI_FUNC(int) _Py_get_inheritable(int fd); +extern int _Py_get_inheritable(int fd); PyAPI_FUNC(int) _Py_set_inheritable(int fd, int inheritable, int *atomic_flag_works); @@ -170,18 +170,18 @@ PyAPI_FUNC(int) _Py_set_inheritable_async_safe(int fd, int inheritable, PyAPI_FUNC(int) _Py_dup(int fd); -PyAPI_FUNC(int) _Py_get_blocking(int fd); +extern int _Py_get_blocking(int fd); -PyAPI_FUNC(int) _Py_set_blocking(int fd, int blocking); +extern int _Py_set_blocking(int fd, int blocking); #ifdef MS_WINDOWS -PyAPI_FUNC(void*) _Py_get_osfhandle_noraise(int fd); +extern void* _Py_get_osfhandle_noraise(int fd); PyAPI_FUNC(void*) _Py_get_osfhandle(int fd); -PyAPI_FUNC(int) _Py_open_osfhandle_noraise(void *handle, int flags); +extern int _Py_open_osfhandle_noraise(void *handle, int flags); -PyAPI_FUNC(int) _Py_open_osfhandle(void *handle, int flags); +extern int _Py_open_osfhandle(void *handle, int flags); #endif /* MS_WINDOWS */ // This is used after getting NULL back from Py_DecodeLocale(). @@ -190,9 +190,9 @@ PyAPI_FUNC(int) _Py_open_osfhandle(void *handle, int flags); ? _PyStatus_ERR("cannot decode " NAME) \ : _PyStatus_NO_MEMORY() -PyAPI_DATA(int) _Py_HasFileSystemDefaultEncodeErrors; +extern int _Py_HasFileSystemDefaultEncodeErrors; -PyAPI_FUNC(int) _Py_DecodeUTF8Ex( +extern int _Py_DecodeUTF8Ex( const char *arg, Py_ssize_t arglen, wchar_t **wstr, @@ -200,7 +200,7 @@ PyAPI_FUNC(int) _Py_DecodeUTF8Ex( const char **reason, _Py_error_handler errors); -PyAPI_FUNC(int) _Py_EncodeUTF8Ex( +extern int _Py_EncodeUTF8Ex( const wchar_t *text, char **str, size_t *error_pos, @@ -208,7 +208,7 @@ PyAPI_FUNC(int) _Py_EncodeUTF8Ex( int raw_malloc, _Py_error_handler errors); -PyAPI_FUNC(wchar_t*) _Py_DecodeUTF8_surrogateescape( +extern wchar_t* _Py_DecodeUTF8_surrogateescape( const char *arg, Py_ssize_t arglen, size_t *wlen); @@ -216,25 +216,25 @@ PyAPI_FUNC(wchar_t*) _Py_DecodeUTF8_surrogateescape( extern int _Py_wstat(const wchar_t *, struct stat *); -PyAPI_FUNC(int) _Py_GetForceASCII(void); +extern int _Py_GetForceASCII(void); /* Reset "force ASCII" mode (if it was initialized). This function should be called when Python changes the LC_CTYPE locale, so the "force ASCII" mode can be detected again on the new locale encoding. */ -PyAPI_FUNC(void) _Py_ResetForceASCII(void); +extern void _Py_ResetForceASCII(void); -PyAPI_FUNC(int) _Py_GetLocaleconvNumeric( +extern int _Py_GetLocaleconvNumeric( struct lconv *lc, PyObject **decimal_point, PyObject **thousands_sep); PyAPI_FUNC(void) _Py_closerange(int first, int last); -PyAPI_FUNC(wchar_t*) _Py_GetLocaleEncoding(void); -PyAPI_FUNC(PyObject*) _Py_GetLocaleEncodingObject(void); +extern wchar_t* _Py_GetLocaleEncoding(void); +extern PyObject* _Py_GetLocaleEncodingObject(void); #ifdef HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION extern int _Py_LocaleUsesNonUnicodeWchar(void); @@ -253,13 +253,13 @@ extern int _Py_abspath(const wchar_t *path, wchar_t **abspath_p); #ifdef MS_WINDOWS extern int _PyOS_getfullpathname(const wchar_t *path, wchar_t **abspath_p); #endif -extern wchar_t * _Py_join_relfile(const wchar_t *dirname, - const wchar_t *relfile); +extern wchar_t* _Py_join_relfile(const wchar_t *dirname, + const wchar_t *relfile); extern int _Py_add_relfile(wchar_t *dirname, const wchar_t *relfile, size_t bufsize); extern size_t _Py_find_basename(const wchar_t *filename); -PyAPI_FUNC(wchar_t *) _Py_normpath(wchar_t *path, Py_ssize_t size); +PyAPI_FUNC(wchar_t*) _Py_normpath(wchar_t *path, Py_ssize_t size); // The Windows Games API family does not provide these functions // so provide our own implementations. Remove them in case they get added diff --git a/Include/internal/pycore_floatobject.h b/Include/internal/pycore_floatobject.h index 6abba04033d28..6b9af03c25ec3 100644 --- a/Include/internal/pycore_floatobject.h +++ b/Include/internal/pycore_floatobject.h @@ -55,12 +55,12 @@ struct _Py_float_state { void _PyFloat_ExactDealloc(PyObject *op); -PyAPI_FUNC(void) _PyFloat_DebugMallocStats(FILE* out); +extern void _PyFloat_DebugMallocStats(FILE* out); /* Format the object based on the format_spec, as defined in PEP 3101 (Advanced String Formatting). */ -PyAPI_FUNC(int) _PyFloat_FormatAdvancedWriter( +extern int _PyFloat_FormatAdvancedWriter( _PyUnicodeWriter *writer, PyObject *obj, PyObject *format_spec, diff --git a/Include/internal/pycore_genobject.h b/Include/internal/pycore_genobject.h index fb63d9c3537a6..96a6ec43d7a08 100644 --- a/Include/internal/pycore_genobject.h +++ b/Include/internal/pycore_genobject.h @@ -11,9 +11,9 @@ extern "C" { extern PyObject *_PyGen_yf(PyGenObject *); extern void _PyGen_Finalize(PyObject *self); -// _asyncio shared extensions uses _PyGen_SetStopIterationValue() and -// _PyGen_FetchStopIterationValue() +// Export for '_asyncio' shared extension PyAPI_FUNC(int) _PyGen_SetStopIterationValue(PyObject *); +// Export for '_asyncio' shared extension PyAPI_FUNC(int) _PyGen_FetchStopIterationValue(PyObject **); extern PyObject *_PyCoro_GetAwaitableIter(PyObject *o); diff --git a/Include/internal/pycore_import.h b/Include/internal/pycore_import.h index c048ae88d9000..0b4ad0770d00a 100644 --- a/Include/internal/pycore_import.h +++ b/Include/internal/pycore_import.h @@ -9,9 +9,9 @@ extern "C" { extern int _PyImport_IsInitialized(PyInterpreterState *); -PyAPI_FUNC(PyObject *) _PyImport_GetModuleId(_Py_Identifier *name); +extern PyObject* _PyImport_GetModuleId(_Py_Identifier *name); PyAPI_FUNC(int) _PyImport_SetModule(PyObject *name, PyObject *module); -PyAPI_FUNC(int) _PyImport_SetModuleString(const char *name, PyObject* module); +extern int _PyImport_SetModuleString(const char *name, PyObject* module); extern void _PyImport_AcquireLock(PyInterpreterState *interp); extern int _PyImport_ReleaseLock(PyInterpreterState *interp); @@ -24,8 +24,8 @@ extern int _PyImport_FixupBuiltin( extern int _PyImport_FixupExtensionObject(PyObject*, PyObject *, PyObject *, PyObject *); -PyAPI_FUNC(PyObject *) _PyImport_GetModuleAttr(PyObject *, PyObject *); -PyAPI_FUNC(PyObject *) _PyImport_GetModuleAttrString(const char *, const char *); +PyAPI_FUNC(PyObject*) _PyImport_GetModuleAttr(PyObject *, PyObject *); +PyAPI_FUNC(PyObject*) _PyImport_GetModuleAttrString(const char *, const char *); struct _import_runtime_state { @@ -192,12 +192,12 @@ PyAPI_DATA(const struct _frozen *) _PyImport_FrozenStdlib; PyAPI_DATA(const struct _frozen *) _PyImport_FrozenTest; extern const struct _module_alias * _PyImport_FrozenAliases; -PyAPI_FUNC(int) _PyImport_CheckSubinterpIncompatibleExtensionAllowed( +extern int _PyImport_CheckSubinterpIncompatibleExtensionAllowed( const char *name); -// for testing -PyAPI_FUNC(int) _PyImport_ClearExtension(PyObject *name, PyObject *filename); +// Export for '_testinternalcapi' shared extension +PyAPI_DATA(int) _PyImport_ClearExtension(PyObject *name, PyObject *filename); #ifdef __cplusplus } diff --git a/Include/internal/pycore_initconfig.h b/Include/internal/pycore_initconfig.h index 4cbd14a61d454..67ca8cae3a73b 100644 --- a/Include/internal/pycore_initconfig.h +++ b/Include/internal/pycore_initconfig.h @@ -49,14 +49,14 @@ struct pyruntimestate; #define _PyWideStringList_INIT (PyWideStringList){.length = 0, .items = NULL} #ifndef NDEBUG -PyAPI_FUNC(int) _PyWideStringList_CheckConsistency(const PyWideStringList *list); +extern int _PyWideStringList_CheckConsistency(const PyWideStringList *list); #endif -PyAPI_FUNC(void) _PyWideStringList_Clear(PyWideStringList *list); -PyAPI_FUNC(int) _PyWideStringList_Copy(PyWideStringList *list, +extern void _PyWideStringList_Clear(PyWideStringList *list); +extern int _PyWideStringList_Copy(PyWideStringList *list, const PyWideStringList *list2); -PyAPI_FUNC(PyStatus) _PyWideStringList_Extend(PyWideStringList *list, +extern PyStatus _PyWideStringList_Extend(PyWideStringList *list, const PyWideStringList *list2); -PyAPI_FUNC(PyObject*) _PyWideStringList_AsList(const PyWideStringList *list); +extern PyObject* _PyWideStringList_AsList(const PyWideStringList *list); /* --- _PyArgv ---------------------------------------------------- */ @@ -68,28 +68,28 @@ typedef struct _PyArgv { wchar_t * const *wchar_argv; } _PyArgv; -PyAPI_FUNC(PyStatus) _PyArgv_AsWstrList(const _PyArgv *args, +extern PyStatus _PyArgv_AsWstrList(const _PyArgv *args, PyWideStringList *list); /* --- Helper functions ------------------------------------------- */ -PyAPI_FUNC(int) _Py_str_to_int( +extern int _Py_str_to_int( const char *str, int *result); -PyAPI_FUNC(const wchar_t*) _Py_get_xoption( +extern const wchar_t* _Py_get_xoption( const PyWideStringList *xoptions, const wchar_t *name); -PyAPI_FUNC(const char*) _Py_GetEnv( +extern const char* _Py_GetEnv( int use_environment, const char *name); -PyAPI_FUNC(void) _Py_get_env_flag( +extern void _Py_get_env_flag( int use_environment, int *flag, const char *name); /* Py_GetArgcArgv() helper */ -PyAPI_FUNC(void) _Py_ClearArgcArgv(void); +extern void _Py_ClearArgcArgv(void); /* --- _PyPreCmdline ------------------------------------------------- */ diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index 8d1b8ac6671f2..260614e937370 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -232,11 +232,11 @@ struct _xidregitem { crossinterpdatafunc getdata; }; -PyAPI_FUNC(PyInterpreterState*) _PyInterpreterState_LookUpID(int64_t); +extern PyInterpreterState* _PyInterpreterState_LookUpID(int64_t); -PyAPI_FUNC(int) _PyInterpreterState_IDInitref(PyInterpreterState *); -PyAPI_FUNC(int) _PyInterpreterState_IDIncref(PyInterpreterState *); -PyAPI_FUNC(void) _PyInterpreterState_IDDecref(PyInterpreterState *); +extern int _PyInterpreterState_IDInitref(PyInterpreterState *); +extern int _PyInterpreterState_IDIncref(PyInterpreterState *); +extern void _PyInterpreterState_IDDecref(PyInterpreterState *); PyAPI_FUNC(PyObject*) _PyInterpreterState_GetMainModule(PyInterpreterState *); From webhook-mailer at python.org Mon Jul 24 21:49:32 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 01:49:32 -0000 Subject: [Python-checkins] gh-107211: No longer export internal functions (2) (#107214) Message-ID: https://github.com/python/cpython/commit/5a61692c6c334c017248bf3d8bb6422ff7b958e8 commit: 5a61692c6c334c017248bf3d8bb6422ff7b958e8 branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T03:49:28+02:00 summary: gh-107211: No longer export internal functions (2) (#107214) No longer export these 43 internal C API functions: * _PyDict_CheckConsistency() * _PyErr_ChainStackItem() * _PyErr_CheckSignals() * _PyErr_CheckSignalsTstate() * _PyErr_Clear() * _PyErr_ExceptionMatches() * _PyErr_Fetch() * _PyErr_Format() * _PyErr_FormatFromCauseTstate() * _PyErr_GetExcInfo() * _PyErr_GetHandledException() * _PyErr_GetTopmostException() * _PyErr_NoMemory() * _PyErr_NormalizeException() * _PyErr_Restore() * _PyErr_SetHandledException() * _PyErr_SetNone() * _PyErr_SetObject() * _PyErr_SetString() * _PyErr_StackItemToExcInfoTuple() * _PyExc_CreateExceptionGroup() * _PyExc_PrepReraiseStar() * _PyException_AddNote() * _PyInterpreterState_Enable() * _PyLong_FormatAdvancedWriter() * _PyLong_FormatBytesWriter() * _PyLong_FormatWriter() * _PyMem_GetAllocatorName() * _PyMem_SetDefaultAllocator() * _PyMem_SetupAllocators() * _PyOS_InterruptOccurred() * _PyRuntimeState_Fini() * _PyRuntimeState_Init() * _PyRuntime_Finalize() * _PyRuntime_Initialize() * _PyThreadState_Bind() * _PyThreadState_DeleteExcept() * _PyThreadState_New() * _PyThreadState_Swap() * _PyType_CheckConsistency() * _PyUnicodeTranslateError_Create() * _Py_DumpExtensionModules() * _Py_FatalErrorFormat() files: M Include/internal/pycore_long.h M Include/internal/pycore_namespace.h M Include/internal/pycore_object.h M Include/internal/pycore_pyarena.h M Include/internal/pycore_pyerrors.h M Include/internal/pycore_pymem.h M Include/internal/pycore_pystate.h M Include/internal/pycore_runtime.h M Include/internal/pycore_setobject.h M Include/internal/pycore_signal.h diff --git a/Include/internal/pycore_long.h b/Include/internal/pycore_long.h index 3f01694e5f5ac..8c4d4a616d417 100644 --- a/Include/internal/pycore_long.h +++ b/Include/internal/pycore_long.h @@ -79,9 +79,9 @@ static inline PyObject* _PyLong_FromUnsignedChar(unsigned char i) return Py_NewRef((PyObject *)&_PyLong_SMALL_INTS[_PY_NSMALLNEGINTS+i]); } -PyObject *_PyLong_Add(PyLongObject *left, PyLongObject *right); -PyObject *_PyLong_Multiply(PyLongObject *left, PyLongObject *right); -PyObject *_PyLong_Subtract(PyLongObject *left, PyLongObject *right); +extern PyObject *_PyLong_Add(PyLongObject *left, PyLongObject *right); +extern PyObject *_PyLong_Multiply(PyLongObject *left, PyLongObject *right); +extern PyObject *_PyLong_Subtract(PyLongObject *left, PyLongObject *right); /* Used by Python/mystrtoul.c, _PyBytes_FromHex(), _PyBytes_DecodeEscape(), etc. */ @@ -89,20 +89,20 @@ PyAPI_DATA(unsigned char) _PyLong_DigitValue[256]; /* Format the object based on the format_spec, as defined in PEP 3101 (Advanced String Formatting). */ -PyAPI_FUNC(int) _PyLong_FormatAdvancedWriter( +extern int _PyLong_FormatAdvancedWriter( _PyUnicodeWriter *writer, PyObject *obj, PyObject *format_spec, Py_ssize_t start, Py_ssize_t end); -PyAPI_FUNC(int) _PyLong_FormatWriter( +extern int _PyLong_FormatWriter( _PyUnicodeWriter *writer, PyObject *obj, int base, int alternate); -PyAPI_FUNC(char*) _PyLong_FormatBytesWriter( +extern char* _PyLong_FormatBytesWriter( _PyBytesWriter *writer, char *str, PyObject *obj, diff --git a/Include/internal/pycore_namespace.h b/Include/internal/pycore_namespace.h index cb76f040693d1..94235471265e1 100644 --- a/Include/internal/pycore_namespace.h +++ b/Include/internal/pycore_namespace.h @@ -10,9 +10,9 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -PyAPI_DATA(PyTypeObject) _PyNamespace_Type; +extern PyTypeObject _PyNamespace_Type; -PyAPI_FUNC(PyObject *) _PyNamespace_New(PyObject *kwds); +PyAPI_FUNC(PyObject*) _PyNamespace_New(PyObject *kwds); #ifdef __cplusplus } diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h index 96aea477d064a..c65c21c3e77fb 100644 --- a/Include/internal/pycore_object.h +++ b/Include/internal/pycore_object.h @@ -156,8 +156,8 @@ _Py_DECREF_NO_DEALLOC(PyObject *op) #endif -PyAPI_FUNC(int) _PyType_CheckConsistency(PyTypeObject *type); -PyAPI_FUNC(int) _PyDict_CheckConsistency(PyObject *mp, int check_content); +extern int _PyType_CheckConsistency(PyTypeObject *type); +extern int _PyDict_CheckConsistency(PyObject *mp, int check_content); /* Update the Python traceback of an object. This function must be called when a memory block is reused from a free list. diff --git a/Include/internal/pycore_pyarena.h b/Include/internal/pycore_pyarena.h index d78972a88ca23..08262fba2daee 100644 --- a/Include/internal/pycore_pyarena.h +++ b/Include/internal/pycore_pyarena.h @@ -1,4 +1,6 @@ /* An arena-like memory interface for the compiler. + * + * Export symbols for test_peg_generator. */ #ifndef Py_INTERNAL_PYARENA_H diff --git a/Include/internal/pycore_pyerrors.h b/Include/internal/pycore_pyerrors.h index e3ba4b75e3cfc..45929f40a0549 100644 --- a/Include/internal/pycore_pyerrors.h +++ b/Include/internal/pycore_pyerrors.h @@ -11,46 +11,47 @@ extern "C" { /* Error handling definitions */ -PyAPI_FUNC(_PyErr_StackItem*) _PyErr_GetTopmostException(PyThreadState *tstate); -PyAPI_FUNC(PyObject*) _PyErr_GetHandledException(PyThreadState *); -PyAPI_FUNC(void) _PyErr_SetHandledException(PyThreadState *, PyObject *); -PyAPI_FUNC(void) _PyErr_GetExcInfo(PyThreadState *, PyObject **, PyObject **, PyObject **); - -/* Like PyErr_Format(), but saves current exception as __context__ and - __cause__. - */ -PyAPI_FUNC(PyObject *) _PyErr_FormatFromCause( +extern _PyErr_StackItem* _PyErr_GetTopmostException(PyThreadState *tstate); +extern PyObject* _PyErr_GetHandledException(PyThreadState *); +extern void _PyErr_SetHandledException(PyThreadState *, PyObject *); +extern void _PyErr_GetExcInfo(PyThreadState *, PyObject **, PyObject **, PyObject **); + +// Like PyErr_Format(), but saves current exception as __context__ and +// __cause__. +// Export for '_sqlite3' shared extension. +PyAPI_FUNC(PyObject*) _PyErr_FormatFromCause( PyObject *exception, const char *format, /* ASCII-encoded string */ ... ); -PyAPI_FUNC(int) _PyException_AddNote( +extern int _PyException_AddNote( PyObject *exc, PyObject *note); -PyAPI_FUNC(int) _PyErr_CheckSignals(void); +extern int _PyErr_CheckSignals(void); /* Support for adding program text to SyntaxErrors */ -PyAPI_FUNC(PyObject *) _PyErr_ProgramDecodedTextObject( +// Export for test_peg_generator +PyAPI_FUNC(PyObject*) _PyErr_ProgramDecodedTextObject( PyObject *filename, int lineno, const char* encoding); -PyAPI_FUNC(PyObject *) _PyUnicodeTranslateError_Create( +extern PyObject* _PyUnicodeTranslateError_Create( PyObject *object, Py_ssize_t start, Py_ssize_t end, const char *reason /* UTF-8 encoded string */ ); -PyAPI_FUNC(void) _Py_NO_RETURN _Py_FatalErrorFormat( +extern void _Py_NO_RETURN _Py_FatalErrorFormat( const char *func, const char *format, ...); -extern PyObject *_PyErr_SetImportErrorWithNameFrom( +extern PyObject* _PyErr_SetImportErrorWithNameFrom( PyObject *, PyObject *, PyObject *, @@ -79,80 +80,79 @@ static inline void _PyErr_ClearExcState(_PyErr_StackItem *exc_state) Py_CLEAR(exc_state->exc_value); } -PyAPI_FUNC(PyObject*) _PyErr_StackItemToExcInfoTuple( +extern PyObject* _PyErr_StackItemToExcInfoTuple( _PyErr_StackItem *err_info); -PyAPI_FUNC(void) _PyErr_Fetch( +extern void _PyErr_Fetch( PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **traceback); -extern PyObject * -_PyErr_GetRaisedException(PyThreadState *tstate); +extern PyObject* _PyErr_GetRaisedException(PyThreadState *tstate); -PyAPI_FUNC(int) _PyErr_ExceptionMatches( +extern int _PyErr_ExceptionMatches( PyThreadState *tstate, PyObject *exc); -void -_PyErr_SetRaisedException(PyThreadState *tstate, PyObject *exc); +extern void _PyErr_SetRaisedException(PyThreadState *tstate, PyObject *exc); -PyAPI_FUNC(void) _PyErr_Restore( +extern void _PyErr_Restore( PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *traceback); -PyAPI_FUNC(void) _PyErr_SetObject( +extern void _PyErr_SetObject( PyThreadState *tstate, PyObject *type, PyObject *value); -PyAPI_FUNC(void) _PyErr_ChainStackItem(void); +extern void _PyErr_ChainStackItem(void); -PyAPI_FUNC(void) _PyErr_Clear(PyThreadState *tstate); +extern void _PyErr_Clear(PyThreadState *tstate); -PyAPI_FUNC(void) _PyErr_SetNone(PyThreadState *tstate, PyObject *exception); +extern void _PyErr_SetNone(PyThreadState *tstate, PyObject *exception); -PyAPI_FUNC(PyObject *) _PyErr_NoMemory(PyThreadState *tstate); +extern PyObject* _PyErr_NoMemory(PyThreadState *tstate); -PyAPI_FUNC(void) _PyErr_SetString( +extern void _PyErr_SetString( PyThreadState *tstate, PyObject *exception, const char *string); -PyAPI_FUNC(PyObject *) _PyErr_Format( +extern PyObject* _PyErr_Format( PyThreadState *tstate, PyObject *exception, const char *format, ...); -PyAPI_FUNC(void) _PyErr_NormalizeException( +extern void _PyErr_NormalizeException( PyThreadState *tstate, PyObject **exc, PyObject **val, PyObject **tb); -PyAPI_FUNC(PyObject *) _PyErr_FormatFromCauseTstate( +extern PyObject* _PyErr_FormatFromCauseTstate( PyThreadState *tstate, PyObject *exception, const char *format, ...); -PyAPI_FUNC(PyObject *) _PyExc_CreateExceptionGroup( +extern PyObject* _PyExc_CreateExceptionGroup( const char *msg, PyObject *excs); -PyAPI_FUNC(PyObject *) _PyExc_PrepReraiseStar( +extern PyObject* _PyExc_PrepReraiseStar( PyObject *orig, PyObject *excs); -PyAPI_FUNC(int) _PyErr_CheckSignalsTstate(PyThreadState *tstate); +extern int _PyErr_CheckSignalsTstate(PyThreadState *tstate); -PyAPI_FUNC(void) _Py_DumpExtensionModules(int fd, PyInterpreterState *interp); +extern void _Py_DumpExtensionModules(int fd, PyInterpreterState *interp); extern PyObject* _Py_Offer_Suggestions(PyObject* exception); +// Export for '_testinternalcapi' shared extension PyAPI_FUNC(Py_ssize_t) _Py_UTF8_Edit_Cost(PyObject *str_a, PyObject *str_b, Py_ssize_t max_cost); diff --git a/Include/internal/pycore_pymem.h b/Include/internal/pycore_pymem.h index 0e53e6650d181..017a60abc6362 100644 --- a/Include/internal/pycore_pymem.h +++ b/Include/internal/pycore_pymem.h @@ -48,7 +48,7 @@ struct _pymem_allocators { /* Set the memory allocator of the specified domain to the default. Save the old allocator into *old_alloc if it's non-NULL. Return on success, or return -1 if the domain is unknown. */ -PyAPI_FUNC(int) _PyMem_SetDefaultAllocator( +extern int _PyMem_SetDefaultAllocator( PyMemAllocatorDomain domain, PyMemAllocatorEx *old_alloc); @@ -94,14 +94,14 @@ static inline int _PyMem_IsPtrFreed(const void *ptr) #endif } -PyAPI_FUNC(int) _PyMem_GetAllocatorName( +extern int _PyMem_GetAllocatorName( const char *name, PyMemAllocatorName *allocator); /* Configure the Python memory allocators. Pass PYMEM_ALLOCATOR_DEFAULT to use default allocators. PYMEM_ALLOCATOR_NOT_SET does nothing. */ -PyAPI_FUNC(int) _PyMem_SetupAllocators(PyMemAllocatorName allocator); +extern int _PyMem_SetupAllocators(PyMemAllocatorName allocator); #ifdef __cplusplus diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h index 0659084194d29..230096f5416b9 100644 --- a/Include/internal/pycore_pystate.h +++ b/Include/internal/pycore_pystate.h @@ -121,15 +121,16 @@ static inline PyInterpreterState* _PyInterpreterState_GET(void) { // PyThreadState functions -PyAPI_FUNC(PyThreadState *) _PyThreadState_New(PyInterpreterState *interp); -PyAPI_FUNC(void) _PyThreadState_Bind(PyThreadState *tstate); -PyAPI_FUNC(void) _PyThreadState_DeleteExcept(PyThreadState *tstate); +extern PyThreadState * _PyThreadState_New(PyInterpreterState *interp); +extern void _PyThreadState_Bind(PyThreadState *tstate); +extern void _PyThreadState_DeleteExcept(PyThreadState *tstate); extern void _PyThreadState_InitDetached(PyThreadState *, PyInterpreterState *); extern void _PyThreadState_ClearDetached(PyThreadState *); extern void _PyThreadState_BindDetached(PyThreadState *); extern void _PyThreadState_UnbindDetached(PyThreadState *); +// Export for '_testinternalcapi' shared extension PyAPI_FUNC(PyObject*) _PyThreadState_GetDict(PyThreadState *tstate); /* The implementation of sys._current_frames() Returns a dict mapping @@ -145,25 +146,25 @@ extern PyObject* _PyThread_CurrentExceptions(void); /* Other */ -PyAPI_FUNC(PyThreadState *) _PyThreadState_Swap( +extern PyThreadState * _PyThreadState_Swap( _PyRuntimeState *runtime, PyThreadState *newts); -PyAPI_FUNC(PyStatus) _PyInterpreterState_Enable(_PyRuntimeState *runtime); +extern PyStatus _PyInterpreterState_Enable(_PyRuntimeState *runtime); #ifdef HAVE_FORK extern PyStatus _PyInterpreterState_DeleteExceptMain(_PyRuntimeState *runtime); extern void _PySignal_AfterFork(void); #endif - +// Export for the stable ABI PyAPI_FUNC(int) _PyState_AddModule( PyThreadState *tstate, PyObject* module, PyModuleDef* def); -PyAPI_FUNC(int) _PyOS_InterruptOccurred(PyThreadState *tstate); +extern int _PyOS_InterruptOccurred(PyThreadState *tstate); #define HEAD_LOCK(runtime) \ PyThread_acquire_lock((runtime)->interpreters.mutex, WAIT_LOCK) @@ -172,6 +173,7 @@ PyAPI_FUNC(int) _PyOS_InterruptOccurred(PyThreadState *tstate); // Get the configuration of the current interpreter. // The caller must hold the GIL. +// Export for test_peg_generator. PyAPI_FUNC(const PyConfig*) _Py_GetConfig(void); diff --git a/Include/internal/pycore_runtime.h b/Include/internal/pycore_runtime.h index a16d4202b616d..eb6b66b335f4b 100644 --- a/Include/internal/pycore_runtime.h +++ b/Include/internal/pycore_runtime.h @@ -274,8 +274,8 @@ typedef struct pyruntimestate { PyAPI_DATA(_PyRuntimeState) _PyRuntime; -PyAPI_FUNC(PyStatus) _PyRuntimeState_Init(_PyRuntimeState *runtime); -PyAPI_FUNC(void) _PyRuntimeState_Fini(_PyRuntimeState *runtime); +extern PyStatus _PyRuntimeState_Init(_PyRuntimeState *runtime); +extern void _PyRuntimeState_Fini(_PyRuntimeState *runtime); #ifdef HAVE_FORK extern PyStatus _PyRuntimeState_ReInitThreads(_PyRuntimeState *runtime); @@ -283,9 +283,9 @@ extern PyStatus _PyRuntimeState_ReInitThreads(_PyRuntimeState *runtime); /* Initialize _PyRuntimeState. Return NULL on success, or return an error message on failure. */ -PyAPI_FUNC(PyStatus) _PyRuntime_Initialize(void); +extern PyStatus _PyRuntime_Initialize(void); -PyAPI_FUNC(void) _PyRuntime_Finalize(void); +extern void _PyRuntime_Finalize(void); static inline PyThreadState* diff --git a/Include/internal/pycore_setobject.h b/Include/internal/pycore_setobject.h index 96f9aea7be9fa..1b63479e77441 100644 --- a/Include/internal/pycore_setobject.h +++ b/Include/internal/pycore_setobject.h @@ -8,12 +8,14 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -// _pickle shared extension uses _PySet_NextEntry() and _PySet_Update() +// Export for 'pickle' shared extension PyAPI_FUNC(int) _PySet_NextEntry( PyObject *set, Py_ssize_t *pos, PyObject **key, Py_hash_t *hash); + +// Export for 'pickle' shared extension PyAPI_FUNC(int) _PySet_Update(PyObject *set, PyObject *iterable); // Export _PySet_Dummy for the gdb plugin's benefit diff --git a/Include/internal/pycore_signal.h b/Include/internal/pycore_signal.h index 1a454ba6f4e8f..46b57d556e5ef 100644 --- a/Include/internal/pycore_signal.h +++ b/Include/internal/pycore_signal.h @@ -14,7 +14,8 @@ extern "C" { #include // NSIG -/* Restore signals that the interpreter has called SIG_IGN on to SIG_DFL. */ +// Restore signals that the interpreter has called SIG_IGN on to SIG_DFL. +// Export for '_posixsubprocess' shared extension. PyAPI_FUNC(void) _Py_RestoreSignals(void); #ifdef _SIG_MAXSIG From webhook-mailer at python.org Mon Jul 24 22:25:49 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 02:25:49 -0000 Subject: [Python-checkins] gh-107211: No longer export internal functions (3) (#107215) Message-ID: https://github.com/python/cpython/commit/0d0520af834dc92035d54d302e67f50f86353d41 commit: 0d0520af834dc92035d54d302e67f50f86353d41 branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T02:25:45Z summary: gh-107211: No longer export internal functions (3) (#107215) No longer export these 14 internal C API functions: * _PySys_Audit() * _PySys_SetAttr() * _PyTraceBack_FromFrame() * _PyTraceBack_Print_Indented() * _PyUnicode_FormatAdvancedWriter() * _PyUnicode_ScanIdentifier() * _PyWarnings_Init() * _Py_DumpASCII() * _Py_DumpDecimal() * _Py_DumpHexadecimal() * _Py_DumpTraceback() * _Py_DumpTracebackThreads() * _Py_WriteIndent() * _Py_WriteIndentedMargin() files: M Include/internal/pycore_sysmodule.h M Include/internal/pycore_time.h M Include/internal/pycore_traceback.h M Include/internal/pycore_tracemalloc.h M Include/internal/pycore_typeobject.h M Include/internal/pycore_unicodeobject.h M Include/internal/pycore_warnings.h diff --git a/Include/internal/pycore_sysmodule.h b/Include/internal/pycore_sysmodule.h index b4b1febafa447..89a2f7628645b 100644 --- a/Include/internal/pycore_sysmodule.h +++ b/Include/internal/pycore_sysmodule.h @@ -8,7 +8,7 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -PyAPI_FUNC(int) _PySys_Audit( +extern int _PySys_Audit( PyThreadState *tstate, const char *event, const char *argFormat, @@ -18,7 +18,7 @@ PyAPI_FUNC(int) _PySys_Audit( PyAPI_FUNC() to not export the symbol. */ extern void _PySys_ClearAuditHooks(PyThreadState *tstate); -PyAPI_FUNC(int) _PySys_SetAttr(PyObject *, PyObject *); +extern int _PySys_SetAttr(PyObject *, PyObject *); extern int _PySys_ClearAttrString(PyInterpreterState *interp, const char *name, int verbose); diff --git a/Include/internal/pycore_time.h b/Include/internal/pycore_time.h index 3d394e8d36a13..318fe4b3a3223 100644 --- a/Include/internal/pycore_time.h +++ b/Include/internal/pycore_time.h @@ -324,10 +324,12 @@ extern int _PyTime_GetPerfCounterWithInfo( // Create a deadline. // Pseudo code: _PyTime_GetMonotonicClock() + timeout. +// Export for '_ssl' shared extension. PyAPI_FUNC(_PyTime_t) _PyDeadline_Init(_PyTime_t timeout); // Get remaining time from a deadline. // Pseudo code: deadline - _PyTime_GetMonotonicClock(). +// Export for '_ssl' shared extension. PyAPI_FUNC(_PyTime_t) _PyDeadline_Get(_PyTime_t deadline); diff --git a/Include/internal/pycore_traceback.h b/Include/internal/pycore_traceback.h index c393b2c136f2d..21fb4a25a0fac 100644 --- a/Include/internal/pycore_traceback.h +++ b/Include/internal/pycore_traceback.h @@ -25,7 +25,7 @@ extern "C" { This function is signal safe. */ -PyAPI_FUNC(void) _Py_DumpTraceback( +extern void _Py_DumpTraceback( int fd, PyThreadState *tstate); @@ -52,7 +52,7 @@ PyAPI_FUNC(void) _Py_DumpTraceback( This function is signal safe. */ -PyAPI_FUNC(const char*) _Py_DumpTracebackThreads( +extern const char* _Py_DumpTracebackThreads( int fd, PyInterpreterState *interp, PyThreadState *current_tstate); @@ -64,23 +64,23 @@ PyAPI_FUNC(const char*) _Py_DumpTracebackThreads( string which is not ready (PyUnicode_WCHAR_KIND). This function is signal safe. */ -PyAPI_FUNC(void) _Py_DumpASCII(int fd, PyObject *text); +extern void _Py_DumpASCII(int fd, PyObject *text); /* Format an integer as decimal into the file descriptor fd. This function is signal safe. */ -PyAPI_FUNC(void) _Py_DumpDecimal( +extern void _Py_DumpDecimal( int fd, size_t value); /* Format an integer as hexadecimal with width digits into fd file descriptor. The function is signal safe. */ -PyAPI_FUNC(void) _Py_DumpHexadecimal( +extern void _Py_DumpHexadecimal( int fd, uintptr_t value, Py_ssize_t width); -PyAPI_FUNC(PyObject*) _PyTraceBack_FromFrame( +extern PyObject* _PyTraceBack_FromFrame( PyObject *tb_next, PyFrameObject *frame); @@ -89,11 +89,11 @@ PyAPI_FUNC(PyObject*) _PyTraceBack_FromFrame( /* Write the traceback tb to file f. Prefix each line with indent spaces followed by the margin (if it is not NULL). */ -PyAPI_FUNC(int) _PyTraceBack_Print_Indented( +extern int _PyTraceBack_Print_Indented( PyObject *tb, int indent, const char* margin, const char *header_margin, const char *header, PyObject *f); -PyAPI_FUNC(int) _Py_WriteIndentedMargin(int, const char*, PyObject *); -PyAPI_FUNC(int) _Py_WriteIndent(int, PyObject *); +extern int _Py_WriteIndentedMargin(int, const char*, PyObject *); +extern int _Py_WriteIndent(int, PyObject *); #ifdef __cplusplus } diff --git a/Include/internal/pycore_tracemalloc.h b/Include/internal/pycore_tracemalloc.h index cfc4d1fe43999..7ddc5bac5d10a 100644 --- a/Include/internal/pycore_tracemalloc.h +++ b/Include/internal/pycore_tracemalloc.h @@ -117,14 +117,16 @@ struct _tracemalloc_runtime_state { } -/* Get the traceback where a memory block was allocated. - - Return a tuple of (filename: str, lineno: int) tuples. - - Return None if the tracemalloc module is disabled or if the memory block - is not tracked by tracemalloc. - - Raise an exception and return NULL on error. */ +// Get the traceback where a memory block was allocated. +// +// Return a tuple of (filename: str, lineno: int) tuples. +// +// Return None if the tracemalloc module is disabled or if the memory block +// is not tracked by tracemalloc. +// +// Raise an exception and return NULL on error. +// +// Export for '_testinternalcapi' shared extension. PyAPI_FUNC(PyObject*) _PyTraceMalloc_GetTraceback( unsigned int domain, uintptr_t ptr); diff --git a/Include/internal/pycore_typeobject.h b/Include/internal/pycore_typeobject.h index 8f3fbbcdb5ffc..bf32667f8e36e 100644 --- a/Include/internal/pycore_typeobject.h +++ b/Include/internal/pycore_typeobject.h @@ -114,6 +114,7 @@ extern static_builtin_state * _PyStaticType_GetState(PyInterpreterState *, PyTyp extern void _PyStaticType_ClearWeakRefs(PyInterpreterState *, PyTypeObject *type); extern void _PyStaticType_Dealloc(PyInterpreterState *, PyTypeObject *); +// Export for 'math' shared extension PyAPI_FUNC(PyObject *) _PyType_GetDict(PyTypeObject *); extern PyObject * _PyType_GetBases(PyTypeObject *type); extern PyObject * _PyType_GetMRO(PyTypeObject *type); diff --git a/Include/internal/pycore_unicodeobject.h b/Include/internal/pycore_unicodeobject.h index d0381123fe280..8ec80ddb83d10 100644 --- a/Include/internal/pycore_unicodeobject.h +++ b/Include/internal/pycore_unicodeobject.h @@ -187,7 +187,7 @@ _PyUnicodeWriter_Dealloc(_PyUnicodeWriter *writer); /* Format the object based on the format_spec, as defined in PEP 3101 (Advanced String Formatting). */ -PyAPI_FUNC(int) _PyUnicode_FormatAdvancedWriter( +extern int _PyUnicode_FormatAdvancedWriter( _PyUnicodeWriter *writer, PyObject *obj, PyObject *format_spec, @@ -245,8 +245,9 @@ extern PyObject* _PyUnicode_DecodeUnicodeEscapeStateful( const char *errors, /* error handling */ Py_ssize_t *consumed); /* bytes consumed */ -/* Helper for PyUnicode_DecodeUnicodeEscape that detects invalid escape - chars. */ +// Helper for PyUnicode_DecodeUnicodeEscape that detects invalid escape +// chars. +// Export for test_peg_generator. PyAPI_FUNC(PyObject*) _PyUnicode_DecodeUnicodeEscapeInternal( const char *string, /* Unicode-Escape encoded string */ Py_ssize_t length, /* size of string */ @@ -369,6 +370,7 @@ PyAPI_FUNC(int) _PyUnicode_Equal(PyObject *, PyObject *); extern int _PyUnicode_WideCharString_Converter(PyObject *, void *); extern int _PyUnicode_WideCharString_Opt_Converter(PyObject *, void *); +// Export for test_peg_generator PyAPI_FUNC(Py_ssize_t) _PyUnicode_ScanIdentifier(PyObject *); /* --- Runtime lifecycle -------------------------------------------------- */ diff --git a/Include/internal/pycore_warnings.h b/Include/internal/pycore_warnings.h index 452d6b96ce4f1..9785d7cc467de 100644 --- a/Include/internal/pycore_warnings.h +++ b/Include/internal/pycore_warnings.h @@ -19,7 +19,7 @@ struct _warnings_runtime_state { extern int _PyWarnings_InitState(PyInterpreterState *interp); -PyAPI_FUNC(PyObject*) _PyWarnings_Init(void); +extern PyObject* _PyWarnings_Init(void); extern void _PyErr_WarnUnawaitedCoroutine(PyObject *coro); extern void _PyErr_WarnUnawaitedAgenMethod(PyAsyncGenObject *agen, PyObject *method); From webhook-mailer at python.org Mon Jul 24 23:16:32 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 03:16:32 -0000 Subject: [Python-checkins] gh-107211: No longer export internal functions (4) (#107217) Message-ID: https://github.com/python/cpython/commit/c5b13d6f80630d29bf5dca9cde53dfe15cf94f8b commit: c5b13d6f80630d29bf5dca9cde53dfe15cf94f8b branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T03:16:28Z summary: gh-107211: No longer export internal functions (4) (#107217) No longer export these 2 internal C API functions: * _PyEval_SignalAsyncExc() * _PyEval_SignalReceived() Add also comments explaining why some internal functions have to be exported, and update existing comments. files: M Include/internal/pycore_atexit.h M Include/internal/pycore_ceval.h M Include/internal/pycore_code.h M Include/internal/pycore_initconfig.h M Include/internal/pycore_interp.h M Include/internal/pycore_interp_id.h M Include/internal/pycore_namespace.h M Include/internal/pycore_object.h M Include/internal/pycore_pathconfig.h M Include/internal/pycore_pylifecycle.h M Include/internal/pycore_pymem.h M Include/internal/pycore_structseq.h M Include/internal/pycore_typeobject.h diff --git a/Include/internal/pycore_atexit.h b/Include/internal/pycore_atexit.h index fc5cb6d882643..3966df70e2616 100644 --- a/Include/internal/pycore_atexit.h +++ b/Include/internal/pycore_atexit.h @@ -51,6 +51,7 @@ struct atexit_state { int callback_len; }; +// Export for '_xxinterpchannels' shared extension PyAPI_FUNC(int) _Py_AtExit( PyInterpreterState *interp, atexit_datacallbackfunc func, diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h index ea5b99ebba07b..05b7380597812 100644 --- a/Include/internal/pycore_ceval.h +++ b/Include/internal/pycore_ceval.h @@ -23,13 +23,14 @@ struct _ceval_runtime_state; extern void _Py_FinishPendingCalls(PyThreadState *tstate); extern void _PyEval_InitState(PyInterpreterState *, PyThread_type_lock); extern void _PyEval_FiniState(struct _ceval_state *ceval); -PyAPI_FUNC(void) _PyEval_SignalReceived(PyInterpreterState *interp); +extern void _PyEval_SignalReceived(PyInterpreterState *interp); +// Export for '_testinternalcapi' shared extension PyAPI_FUNC(int) _PyEval_AddPendingCall( PyInterpreterState *interp, int (*func)(void *), void *arg, int mainthreadonly); -PyAPI_FUNC(void) _PyEval_SignalAsyncExc(PyInterpreterState *interp); +extern void _PyEval_SignalAsyncExc(PyInterpreterState *interp); #ifdef HAVE_FORK extern PyStatus _PyEval_ReInitThreads(PyThreadState *tstate); #endif @@ -122,6 +123,7 @@ static inline int _Py_MakeRecCheck(PyThreadState *tstate) { } #endif +// Export for _Py_EnterRecursiveCall() PyAPI_FUNC(int) _Py_CheckRecursiveCall( PyThreadState *tstate, const char *where); diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index 62a943b115f9e..bcdf8645c8557 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -262,7 +262,6 @@ extern int _PyStaticCode_Init(PyCodeObject *co); #ifdef Py_STATS - #define STAT_INC(opname, name) do { if (_py_stats) _py_stats->opcode_stats[opname].specialization.name++; } while (0) #define STAT_DEC(opname, name) do { if (_py_stats) _py_stats->opcode_stats[opname].specialization.name--; } while (0) #define OPCODE_EXE_INC(opname) do { if (_py_stats) _py_stats->opcode_stats[opname].execution_count++; } while (0) @@ -274,7 +273,7 @@ extern int _PyStaticCode_Init(PyCodeObject *co); #define EVAL_CALL_STAT_INC_IF_FUNCTION(name, callable) \ do { if (_py_stats && PyFunction_Check(callable)) _py_stats->call_stats.eval_calls[name]++; } while (0) -// Export for stdlib '_opcode' shared extension +// Export for '_opcode' shared extension PyAPI_FUNC(PyObject*) _Py_GetSpecializationStats(void); #else diff --git a/Include/internal/pycore_initconfig.h b/Include/internal/pycore_initconfig.h index 67ca8cae3a73b..0945bb0936f03 100644 --- a/Include/internal/pycore_initconfig.h +++ b/Include/internal/pycore_initconfig.h @@ -122,6 +122,7 @@ extern PyStatus _PyPreCmdline_Read(_PyPreCmdline *cmdline, /* --- PyPreConfig ----------------------------------------------- */ +// Export for '_testembed' program PyAPI_FUNC(void) _PyPreConfig_InitCompatConfig(PyPreConfig *preconfig); extern void _PyPreConfig_InitFromConfig( PyPreConfig *preconfig, @@ -146,6 +147,7 @@ typedef enum { _PyConfig_INIT_ISOLATED = 3 } _PyConfigInitEnum; +// Export for '_testembed' program PyAPI_FUNC(void) _PyConfig_InitCompatConfig(PyConfig *config); extern PyStatus _PyConfig_Copy( PyConfig *config, diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index 260614e937370..522ce4a80ecc2 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -238,6 +238,7 @@ extern int _PyInterpreterState_IDInitref(PyInterpreterState *); extern int _PyInterpreterState_IDIncref(PyInterpreterState *); extern void _PyInterpreterState_IDDecref(PyInterpreterState *); +// Export for '_xxsubinterpreters' shared extension PyAPI_FUNC(PyObject*) _PyInterpreterState_GetMainModule(PyInterpreterState *); extern const PyConfig* _PyInterpreterState_GetConfig(PyInterpreterState *interp); @@ -253,7 +254,9 @@ extern const PyConfig* _PyInterpreterState_GetConfig(PyInterpreterState *interp) The caller must hold the GIL. Once done with the configuration, PyConfig_Clear() must be called to clear - it. */ + it. + + Export for '_testinternalcapi' shared extension. */ PyAPI_FUNC(int) _PyInterpreterState_GetConfigCopy( struct PyConfig *config); @@ -271,7 +274,9 @@ PyAPI_FUNC(int) _PyInterpreterState_GetConfigCopy( Return 0 on success. Raise an exception and return -1 on error. - The configuration should come from _PyInterpreterState_GetConfigCopy(). */ + The configuration should come from _PyInterpreterState_GetConfigCopy(). + + Export for '_testinternalcapi' shared extension. */ PyAPI_FUNC(int) _PyInterpreterState_SetConfig( const struct PyConfig *config); diff --git a/Include/internal/pycore_interp_id.h b/Include/internal/pycore_interp_id.h index 34c96df98ef73..8c6f8315cac39 100644 --- a/Include/internal/pycore_interp_id.h +++ b/Include/internal/pycore_interp_id.h @@ -10,16 +10,16 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -// Export for the _xxsubinterpreters shared extension +// Export for '_xxsubinterpreters' shared extension PyAPI_DATA(PyTypeObject) _PyInterpreterID_Type; -// Export for the _xxsubinterpreters shared extension +// Export for '_xxsubinterpreters' shared extension PyAPI_FUNC(PyObject*) _PyInterpreterID_New(int64_t); -// Export for the _xxinterpchannels shared extension +// Export for '_xxinterpchannels' shared extension PyAPI_FUNC(PyObject*) _PyInterpreterState_GetIDObject(PyInterpreterState *); -// Export for the _testinternalcapi shared extension +// Export for '_testinternalcapi' shared extension PyAPI_FUNC(PyInterpreterState*) _PyInterpreterID_LookUp(PyObject *); #ifdef __cplusplus diff --git a/Include/internal/pycore_namespace.h b/Include/internal/pycore_namespace.h index 94235471265e1..f165cf15319a5 100644 --- a/Include/internal/pycore_namespace.h +++ b/Include/internal/pycore_namespace.h @@ -12,6 +12,7 @@ extern "C" { extern PyTypeObject _PyNamespace_Type; +// Export for '_testmultiphase' shared extension PyAPI_FUNC(PyObject*) _PyNamespace_New(PyObject *kwds); #ifdef __cplusplus diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h index c65c21c3e77fb..6bf9dc245d5de 100644 --- a/Include/internal/pycore_object.h +++ b/Include/internal/pycore_object.h @@ -434,7 +434,8 @@ extern PyObject ** _PyObject_ComputedDictPointer(PyObject *); extern void _PyObject_FreeInstanceAttributes(PyObject *obj); extern int _PyObject_IsInstanceDictEmpty(PyObject *); -PyAPI_FUNC(PyObject *) _PyObject_LookupSpecial(PyObject *, PyObject *); +// Export for 'math' shared extension +PyAPI_FUNC(PyObject*) _PyObject_LookupSpecial(PyObject *, PyObject *); extern int _PyObject_IsAbstract(PyObject *); diff --git a/Include/internal/pycore_pathconfig.h b/Include/internal/pycore_pathconfig.h index b8deaa0c3eb06..729f3e09ce1ca 100644 --- a/Include/internal/pycore_pathconfig.h +++ b/Include/internal/pycore_pathconfig.h @@ -8,6 +8,7 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +// Export for '_testinternalcapi' shared extension PyAPI_FUNC(void) _PyPathConfig_ClearGlobal(void); extern PyStatus _PyPathConfig_ReadGlobal(PyConfig *config); extern PyStatus _PyPathConfig_UpdateGlobal(const PyConfig *config); diff --git a/Include/internal/pycore_pylifecycle.h b/Include/internal/pycore_pylifecycle.h index fb28652515909..b4d5b1f1239e1 100644 --- a/Include/internal/pycore_pylifecycle.h +++ b/Include/internal/pycore_pylifecycle.h @@ -103,6 +103,7 @@ PyAPI_FUNC(int) _Py_IsInterpreterFinalizing(PyInterpreterState *interp); /* Random */ extern int _PyOS_URandom(void *buffer, Py_ssize_t size); +// Export for '_random' shared extension PyAPI_FUNC(int) _PyOS_URandomNonblock(void *buffer, Py_ssize_t size); /* Legacy locale support */ diff --git a/Include/internal/pycore_pymem.h b/Include/internal/pycore_pymem.h index 017a60abc6362..6b5113714dbeb 100644 --- a/Include/internal/pycore_pymem.h +++ b/Include/internal/pycore_pymem.h @@ -10,14 +10,14 @@ extern "C" { // Try to get the allocators name set by _PyMem_SetupAllocators(). // Return NULL if unknown. -// Export for shared _testinternalcapi extension. +// Export for '_testinternalcapi' shared extension. PyAPI_FUNC(const char*) _PyMem_GetCurrentAllocatorName(void); // strdup() using PyMem_RawMalloc() extern char* _PyMem_RawStrdup(const char *str); // strdup() using PyMem_Malloc(). -// Export for shared _pickle extension. +// Export for '_pickle ' shared extension. PyAPI_FUNC(char*) _PyMem_Strdup(const char *str); // wcsdup() using PyMem_RawMalloc() diff --git a/Include/internal/pycore_structseq.h b/Include/internal/pycore_structseq.h index 6f5dfc12707cf..5cff165627502 100644 --- a/Include/internal/pycore_structseq.h +++ b/Include/internal/pycore_structseq.h @@ -11,7 +11,8 @@ extern "C" { /* other API */ -PyAPI_FUNC(PyTypeObject *) _PyStructSequence_NewType( +// Export for '_curses' shared extension +PyAPI_FUNC(PyTypeObject*) _PyStructSequence_NewType( PyStructSequence_Desc *desc, unsigned long tp_flags); diff --git a/Include/internal/pycore_typeobject.h b/Include/internal/pycore_typeobject.h index bf32667f8e36e..2ad2adad0aa41 100644 --- a/Include/internal/pycore_typeobject.h +++ b/Include/internal/pycore_typeobject.h @@ -114,7 +114,7 @@ extern static_builtin_state * _PyStaticType_GetState(PyInterpreterState *, PyTyp extern void _PyStaticType_ClearWeakRefs(PyInterpreterState *, PyTypeObject *type); extern void _PyStaticType_Dealloc(PyInterpreterState *, PyTypeObject *); -// Export for 'math' shared extension +// Export for 'math' shared extension via _PyType_IsReady() function PyAPI_FUNC(PyObject *) _PyType_GetDict(PyTypeObject *); extern PyObject * _PyType_GetBases(PyTypeObject *type); extern PyObject * _PyType_GetMRO(PyTypeObject *type); From webhook-mailer at python.org Mon Jul 24 23:48:08 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 03:48:08 -0000 Subject: [Python-checkins] gh-107211: No longer export internal variables (#107218) Message-ID: https://github.com/python/cpython/commit/3b309319cc2920587a8c74a6969fb47d69281126 commit: 3b309319cc2920587a8c74a6969fb47d69281126 branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T03:48:04Z summary: gh-107211: No longer export internal variables (#107218) No longer export these 5 internal C API variables: * _PyBufferWrapper_Type * _PyImport_FrozenBootstrap * _PyImport_FrozenStdlib * _PyImport_FrozenTest * _Py_SwappedOp Fix the definition of these internal functions, replace PyAPI_DATA() with PyAPI_FUNC(): * _PyImport_ClearExtension() * _PyObject_IsFreed() * _PyThreadState_GetCurrent() files: M Include/internal/pycore_import.h M Include/internal/pycore_long.h M Include/internal/pycore_object.h M Include/internal/pycore_pyhash.h M Include/internal/pycore_pystate.h M Include/internal/pycore_typeobject.h diff --git a/Include/internal/pycore_import.h b/Include/internal/pycore_import.h index 0b4ad0770d00a..c12ef7cebadc2 100644 --- a/Include/internal/pycore_import.h +++ b/Include/internal/pycore_import.h @@ -187,9 +187,13 @@ struct _module_alias { const char *orig; /* ASCII encoded string */ }; -PyAPI_DATA(const struct _frozen *) _PyImport_FrozenBootstrap; -PyAPI_DATA(const struct _frozen *) _PyImport_FrozenStdlib; -PyAPI_DATA(const struct _frozen *) _PyImport_FrozenTest; +// Export for test_ctypes +PyAPI_DATA(const struct _frozen*) _PyImport_FrozenBootstrap; +// Export for test_ctypes +PyAPI_DATA(const struct _frozen*) _PyImport_FrozenStdlib; +// Export for test_ctypes +PyAPI_DATA(const struct _frozen*) _PyImport_FrozenTest; + extern const struct _module_alias * _PyImport_FrozenAliases; extern int _PyImport_CheckSubinterpIncompatibleExtensionAllowed( @@ -197,7 +201,7 @@ extern int _PyImport_CheckSubinterpIncompatibleExtensionAllowed( // Export for '_testinternalcapi' shared extension -PyAPI_DATA(int) _PyImport_ClearExtension(PyObject *name, PyObject *filename); +PyAPI_FUNC(int) _PyImport_ClearExtension(PyObject *name, PyObject *filename); #ifdef __cplusplus } diff --git a/Include/internal/pycore_long.h b/Include/internal/pycore_long.h index 8c4d4a616d417..3f8d8adc83b20 100644 --- a/Include/internal/pycore_long.h +++ b/Include/internal/pycore_long.h @@ -83,8 +83,8 @@ extern PyObject *_PyLong_Add(PyLongObject *left, PyLongObject *right); extern PyObject *_PyLong_Multiply(PyLongObject *left, PyLongObject *right); extern PyObject *_PyLong_Subtract(PyLongObject *left, PyLongObject *right); -/* Used by Python/mystrtoul.c, _PyBytes_FromHex(), - _PyBytes_DecodeEscape(), etc. */ +// Used by _PyBytes_FromHex(), _PyBytes_DecodeEscape(), Python/mystrtoul.c. +// Export for 'binascii' shared extension. PyAPI_DATA(unsigned char) _PyLong_DigitValue[256]; /* Format the object based on the format_spec, as defined in PEP 3101 diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h index 6bf9dc245d5de..b730aba991283 100644 --- a/Include/internal/pycore_object.h +++ b/Include/internal/pycore_object.h @@ -33,7 +33,7 @@ extern void _PyDebugAllocatorStats(FILE *out, const char *block_name, extern void _PyObject_DebugTypeStats(FILE *out); // Export for shared _testinternalcapi extension -PyAPI_DATA(int) _PyObject_IsFreed(PyObject *); +PyAPI_FUNC(int) _PyObject_IsFreed(PyObject *); /* We need to maintain an internal copy of Py{Var}Object_HEAD_INIT to avoid designated initializer conflicts in C++20. If we use the deinition in @@ -469,11 +469,14 @@ extern PyObject* _PyCFunctionWithKeywords_TrampolineCall( (meth)((self), (args), (kw)) #endif // __EMSCRIPTEN__ && PY_CALL_TRAMPOLINE -// _pickle shared extension uses _PyNone_Type and _PyNotImplemented_Type +// Export for '_pickle' shared extension PyAPI_DATA(PyTypeObject) _PyNone_Type; +// Export for '_pickle' shared extension PyAPI_DATA(PyTypeObject) _PyNotImplemented_Type; -/* Maps Py_LT to Py_GT, ..., Py_GE to Py_LE. Defined in Objects/object.c. */ +// Maps Py_LT to Py_GT, ..., Py_GE to Py_LE. +// Defined in Objects/object.c. +// Export for the stable ABI. PyAPI_DATA(int) _Py_SwappedOp[]; #ifdef __cplusplus diff --git a/Include/internal/pycore_pyhash.h b/Include/internal/pycore_pyhash.h index 59e416ca18a72..da9abd28b499a 100644 --- a/Include/internal/pycore_pyhash.h +++ b/Include/internal/pycore_pyhash.h @@ -74,7 +74,7 @@ typedef union { } expat; } _Py_HashSecret_t; -// _elementtree shared extension uses _Py_HashSecret.expat +// Export for '_elementtree' shared extension PyAPI_DATA(_Py_HashSecret_t) _Py_HashSecret; #ifdef Py_DEBUG diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h index 230096f5416b9..f0db757c77c58 100644 --- a/Include/internal/pycore_pystate.h +++ b/Include/internal/pycore_pystate.h @@ -66,7 +66,7 @@ _Py_ThreadCanHandleSignals(PyInterpreterState *interp) #if defined(HAVE_THREAD_LOCAL) && !defined(Py_BUILD_CORE_MODULE) extern _Py_thread_local PyThreadState *_Py_tss_tstate; #endif -PyAPI_DATA(PyThreadState *) _PyThreadState_GetCurrent(void); +PyAPI_FUNC(PyThreadState *) _PyThreadState_GetCurrent(void); /* Get the current Python thread state. diff --git a/Include/internal/pycore_typeobject.h b/Include/internal/pycore_typeobject.h index 2ad2adad0aa41..aba672effe392 100644 --- a/Include/internal/pycore_typeobject.h +++ b/Include/internal/pycore_typeobject.h @@ -129,18 +129,17 @@ _PyType_IsReady(PyTypeObject *type) return _PyType_GetDict(type) != NULL; } -PyObject * -_Py_type_getattro_impl(PyTypeObject *type, PyObject *name, int *suppress_missing_attribute); -PyObject * -_Py_type_getattro(PyTypeObject *type, PyObject *name); +extern PyObject* _Py_type_getattro_impl(PyTypeObject *type, PyObject *name, + int *suppress_missing_attribute); +extern PyObject* _Py_type_getattro(PyTypeObject *type, PyObject *name); -PyObject *_Py_slot_tp_getattro(PyObject *self, PyObject *name); -PyObject *_Py_slot_tp_getattr_hook(PyObject *self, PyObject *name); +extern PyObject* _Py_slot_tp_getattro(PyObject *self, PyObject *name); +extern PyObject* _Py_slot_tp_getattr_hook(PyObject *self, PyObject *name); -PyAPI_DATA(PyTypeObject) _PyBufferWrapper_Type; +extern PyTypeObject _PyBufferWrapper_Type; -PyObject * -_PySuper_Lookup(PyTypeObject *su_type, PyObject *su_obj, PyObject *name, int *meth_found); +extern PyObject* _PySuper_Lookup(PyTypeObject *su_type, PyObject *su_obj, + PyObject *name, int *meth_found); #ifdef __cplusplus } From webhook-mailer at python.org Tue Jul 25 04:18:23 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Tue, 25 Jul 2023 08:18:23 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: improve typing around adding C converters (#107209) Message-ID: https://github.com/python/cpython/commit/dbfe73837d621bc531491828c1e7967671014a62 commit: dbfe73837d621bc531491828c1e7967671014a62 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-25T09:18:19+01:00 summary: gh-104050: Argument clinic: improve typing around adding C converters (#107209) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index ff2151505b973..9c18fec055f81 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -44,6 +44,7 @@ NoReturn, Protocol, TypeGuard, + TypeVar, overload, ) @@ -2647,10 +2648,12 @@ def __getattribute__(self, name: str) -> Any: fail("Stepped on a land mine, trying to access attribute " + repr(name) + ":\n" + self.__message__) +CConverterClassT = TypeVar("CConverterClassT", bound=type["CConverter"]) + def add_c_converter( - f: type[CConverter], + f: CConverterClassT, name: str | None = None -) -> type[CConverter]: +) -> CConverterClassT: if not name: name = f.__name__ if not name.endswith('_converter'): @@ -2659,7 +2662,7 @@ def add_c_converter( converters[name] = f return f -def add_default_legacy_c_converter(cls): +def add_default_legacy_c_converter(cls: CConverterClassT) -> CConverterClassT: # automatically add converter for default format unit # (but without stomping on the existing one if it's already # set, in case you subclass) @@ -2670,16 +2673,19 @@ def add_default_legacy_c_converter(cls): def add_legacy_c_converter( format_unit: str, - **kwargs -) -> Callable[[ConverterType], ConverterType]: + **kwargs: Any +) -> Callable[[CConverterClassT], CConverterClassT]: """ Adds a legacy converter. """ - def closure(f): + def closure(f: CConverterClassT) -> CConverterClassT: + added_f: Callable[..., CConverter] if not kwargs: added_f = f else: - added_f = functools.partial(f, **kwargs) + # mypy's special-casing for functools.partial + # can't quite grapple with this code here + added_f = functools.partial(f, **kwargs) # type: ignore[arg-type] if format_unit: legacy_converters[format_unit] = added_f return f From webhook-mailer at python.org Tue Jul 25 04:49:11 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Tue, 25 Jul 2023 08:49:11 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: more misc typing coverage improvements (#107210) Message-ID: https://github.com/python/cpython/commit/d9e34db993ebc4340459ac5cb3a66c1b83451a66 commit: d9e34db993ebc4340459ac5cb3a66c1b83451a66 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-25T08:49:07Z summary: gh-104050: Argument clinic: more misc typing coverage improvements (#107210) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 9c18fec055f81..8c959ed4d0044 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -437,7 +437,9 @@ class FormatCounterFormatter(string.Formatter): def __init__(self) -> None: self.counts = collections.Counter[str]() - def get_value(self, key: str, args, kwargs) -> str: # type: ignore[override] + def get_value( + self, key: str, args: object, kwargs: object # type: ignore[override] + ) -> Literal['']: self.counts[key] += 1 return '' @@ -2797,7 +2799,7 @@ class CConverter(metaclass=CConverterAutoRegister): # This lets the self_converter overrule the user-settable # name, *just* for the text signature. # Only set by self_converter. - signature_name = None + signature_name: str | None = None # keep in sync with self_converter.__init__! def __init__(self, @@ -2811,8 +2813,8 @@ def __init__(self, py_default: str | None = None, annotation: str | Literal[Sentinels.unspecified] = unspecified, unused: bool = False, - **kwargs - ): + **kwargs: Any + ) -> None: self.name = ensure_legal_c_identifier(name) self.py_name = py_name self.unused = unused @@ -2849,7 +2851,7 @@ def __init__(self, self.converter_init(**kwargs) self.function = function - def converter_init(self): + def converter_init(self) -> None: pass def is_optional(self) -> bool: @@ -3032,7 +3034,7 @@ def cleanup(self) -> str: """ return "" - def pre_render(self): + def pre_render(self) -> None: """ A second initialization function, like converter_init, called just before rendering. @@ -3169,7 +3171,7 @@ class defining_class_converter(CConverter): format_unit = '' show_in_signature = False - def converter_init(self, *, type=None) -> None: + def converter_init(self, *, type: str | None = None) -> None: self.specified_type = type def render(self, parameter, data) -> None: @@ -3321,7 +3323,9 @@ class int_converter(CConverter): format_unit = 'i' c_ignored_default = "0" - def converter_init(self, *, accept: TypeSet = {int}, type=None) -> None: + def converter_init( + self, *, accept: TypeSet = {int}, type: str | None = None + ) -> None: if accept == {str}: self.format_unit = 'C' elif accept != {int}: @@ -3982,14 +3986,15 @@ class self_converter(CConverter): A special-case converter: this is the default converter used for "self". """ - type = None + type: str | None = None format_unit = '' def converter_init(self, *, type: str | None = None) -> None: self.specified_type = type - def pre_render(self): + def pre_render(self) -> None: f = self.function + assert isinstance(f, Function) default_type, default_name = correct_name_for_self(f) self.signature_name = default_name self.type = self.specified_type or self.type or default_type @@ -4038,7 +4043,9 @@ def pre_render(self): # in the impl call. @property - def parser_type(self): + def parser_type(self) -> str: + assert self.type is not None + assert isinstance(self.function, Function) return required_type_for_self_for_parser(self.function) or self.type def render(self, parameter, data): From webhook-mailer at python.org Tue Jul 25 05:59:29 2023 From: webhook-mailer at python.org (pfmoore) Date: Tue, 25 Jul 2023 09:59:29 -0000 Subject: [Python-checkins] gh-106774: Update bundled pip version to 23.2.1 (#106775) Message-ID: https://github.com/python/cpython/commit/f443b54a2f14e386a91fe4b09f41a265445008b8 commit: f443b54a2f14e386a91fe4b09f41a265445008b8 branch: main author: Paul Moore committer: pfmoore date: 2023-07-25T10:59:25+01:00 summary: gh-106774: Update bundled pip version to 23.2.1 (#106775) * Update bundled pip version to 23.2.1 files: A Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl A Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst D Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl M Lib/ensurepip/__init__.py diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py index 5f4f1d75b43e6..1fb1d505cfd0c 100644 --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -10,7 +10,7 @@ __all__ = ["version", "bootstrap"] _PACKAGE_NAMES = ('pip',) -_PIP_VERSION = "23.1.2" +_PIP_VERSION = "23.2.1" _PROJECTS = [ ("pip", _PIP_VERSION, "py3"), ] diff --git a/Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl b/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl similarity index 74% rename from Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl rename to Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl index 6a2515615ccda..ba28ef02e265f 100644 Binary files a/Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl and b/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl differ diff --git a/Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst b/Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst new file mode 100644 index 0000000000000..ed467573b89e1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst @@ -0,0 +1 @@ +Update the bundled copy of pip to version 23.2.1. From webhook-mailer at python.org Tue Jul 25 06:32:50 2023 From: webhook-mailer at python.org (pfmoore) Date: Tue, 25 Jul 2023 10:32:50 -0000 Subject: [Python-checkins] [3.12] gh-106774: Update bundled pip version to 23.2.1 (GH-106775) (gh-107222) Message-ID: https://github.com/python/cpython/commit/2cdde10c5bb38f825c52af80b775b247fef24f07 commit: 2cdde10c5bb38f825c52af80b775b247fef24f07 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: pfmoore date: 2023-07-25T11:32:45+01:00 summary: [3.12] gh-106774: Update bundled pip version to 23.2.1 (GH-106775) (gh-107222) gh-106774: Update bundled pip version to 23.2.1 (GH-106775) files: A Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl A Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst D Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl M Lib/ensurepip/__init__.py diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py index 5f4f1d75b43e6..1fb1d505cfd0c 100644 --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -10,7 +10,7 @@ __all__ = ["version", "bootstrap"] _PACKAGE_NAMES = ('pip',) -_PIP_VERSION = "23.1.2" +_PIP_VERSION = "23.2.1" _PROJECTS = [ ("pip", _PIP_VERSION, "py3"), ] diff --git a/Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl b/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl similarity index 74% rename from Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl rename to Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl index 6a2515615ccda..ba28ef02e265f 100644 Binary files a/Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl and b/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl differ diff --git a/Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst b/Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst new file mode 100644 index 0000000000000..ed467573b89e1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst @@ -0,0 +1 @@ +Update the bundled copy of pip to version 23.2.1. From webhook-mailer at python.org Tue Jul 25 06:38:23 2023 From: webhook-mailer at python.org (pfmoore) Date: Tue, 25 Jul 2023 10:38:23 -0000 Subject: [Python-checkins] [3.11] gh-106774: Update bundled pip version to 23.2.1 (GH-106775) (gh-107223) Message-ID: https://github.com/python/cpython/commit/058741cc39dd2b63907931e2b626f7428ba13253 commit: 058741cc39dd2b63907931e2b626f7428ba13253 branch: 3.11 author: Paul Moore committer: pfmoore date: 2023-07-25T11:38:19+01:00 summary: [3.11] gh-106774: Update bundled pip version to 23.2.1 (GH-106775) (gh-107223) * Update bundled pip version to 23.2.1. (cherry picked from commit f443b54a2f14e386a91fe4b09f41a265445008b8) files: A Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl A Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst D Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl M Lib/ensurepip/__init__.py diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py index cac9d30b7a8fc..81db9fa922d04 100644 --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -11,7 +11,7 @@ __all__ = ["version", "bootstrap"] _PACKAGE_NAMES = ('setuptools', 'pip') _SETUPTOOLS_VERSION = "65.5.0" -_PIP_VERSION = "23.1.2" +_PIP_VERSION = "23.2.1" _PROJECTS = [ ("setuptools", _SETUPTOOLS_VERSION, "py3"), ("pip", _PIP_VERSION, "py3"), diff --git a/Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl b/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl similarity index 74% rename from Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl rename to Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl index 6a2515615ccda..ba28ef02e265f 100644 Binary files a/Lib/ensurepip/_bundled/pip-23.1.2-py3-none-any.whl and b/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl differ diff --git a/Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst b/Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst new file mode 100644 index 0000000000000..ed467573b89e1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-15-10-24-56.gh-issue-106774.FJcqCj.rst @@ -0,0 +1 @@ +Update the bundled copy of pip to version 23.2.1. From webhook-mailer at python.org Tue Jul 25 07:15:18 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 11:15:18 -0000 Subject: [Python-checkins] [3.11] gh-99612: Fix PyUnicode_DecodeUTF8Stateful() for ASCII-only data (GH-99613) (GH-107224) Message-ID: https://github.com/python/cpython/commit/b8b3e6afc0a48c3cbb7c36d2f73e332edcd6058c commit: b8b3e6afc0a48c3cbb7c36d2f73e332edcd6058c branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-25T14:15:14+03:00 summary: [3.11] gh-99612: Fix PyUnicode_DecodeUTF8Stateful() for ASCII-only data (GH-99613) (GH-107224) Previously *consumed was not set in this case. (cherry picked from commit f08e52ccb027f6f703302b8c1a82db9fd3934270) files: A Lib/test/test_capi/test_codecs.py A Misc/NEWS.d/next/C API/2022-11-20-09-52-50.gh-issue-99612.eBHksg.rst M Modules/_testcapimodule.c M Objects/unicodeobject.c diff --git a/Lib/test/test_capi/test_codecs.py b/Lib/test/test_capi/test_codecs.py new file mode 100644 index 0000000000000..e46726192aa05 --- /dev/null +++ b/Lib/test/test_capi/test_codecs.py @@ -0,0 +1,54 @@ +import unittest +from test.support import import_helper + +_testcapi = import_helper.import_module('_testcapi') + + +class CAPITest(unittest.TestCase): + + def test_decodeutf8(self): + """Test PyUnicode_DecodeUTF8()""" + decodeutf8 = _testcapi.unicode_decodeutf8 + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-8') + self.assertEqual(decodeutf8(b), s) + self.assertEqual(decodeutf8(b, 'strict'), s) + + self.assertRaises(UnicodeDecodeError, decodeutf8, b'\x80') + self.assertRaises(UnicodeDecodeError, decodeutf8, b'\xc0') + self.assertRaises(UnicodeDecodeError, decodeutf8, b'\xff') + self.assertRaises(UnicodeDecodeError, decodeutf8, b'a\xf0\x9f') + self.assertEqual(decodeutf8(b'a\xf0\x9f', 'replace'), 'a\ufffd') + self.assertEqual(decodeutf8(b'a\xf0\x9fb', 'replace'), 'a\ufffdb') + + self.assertRaises(LookupError, decodeutf8, b'a\x80', 'foo') + # TODO: Test PyUnicode_DecodeUTF8() with NULL as data and + # negative size. + + def test_decodeutf8stateful(self): + """Test PyUnicode_DecodeUTF8Stateful()""" + decodeutf8stateful = _testcapi.unicode_decodeutf8stateful + + for s in ['abc', '\xa1\xa2', '\u4f60\u597d', 'a\U0001f600']: + b = s.encode('utf-8') + self.assertEqual(decodeutf8stateful(b), (s, len(b))) + self.assertEqual(decodeutf8stateful(b, 'strict'), (s, len(b))) + + self.assertRaises(UnicodeDecodeError, decodeutf8stateful, b'\x80') + self.assertRaises(UnicodeDecodeError, decodeutf8stateful, b'\xc0') + self.assertRaises(UnicodeDecodeError, decodeutf8stateful, b'\xff') + self.assertEqual(decodeutf8stateful(b'a\xf0\x9f'), ('a', 1)) + self.assertEqual(decodeutf8stateful(b'a\xf0\x9f', 'replace'), ('a', 1)) + self.assertRaises(UnicodeDecodeError, decodeutf8stateful, b'a\xf0\x9fb') + self.assertEqual(decodeutf8stateful(b'a\xf0\x9fb', 'replace'), ('a\ufffdb', 4)) + + self.assertRaises(LookupError, decodeutf8stateful, b'a\x80', 'foo') + # TODO: Test PyUnicode_DecodeUTF8Stateful() with NULL as data and + # negative size. + # TODO: Test PyUnicode_DecodeUTF8Stateful() with NULL as the address of + # "consumed". + + +if __name__ == "__main__": + unittest.main() diff --git a/Misc/NEWS.d/next/C API/2022-11-20-09-52-50.gh-issue-99612.eBHksg.rst b/Misc/NEWS.d/next/C API/2022-11-20-09-52-50.gh-issue-99612.eBHksg.rst new file mode 100644 index 0000000000000..40e3c8db5403c --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-11-20-09-52-50.gh-issue-99612.eBHksg.rst @@ -0,0 +1,2 @@ +Fix :c:func:`PyUnicode_DecodeUTF8Stateful` for ASCII-only data: +``*consumed`` was not set. diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index b40368eb4b340..b29a919a8325b 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -2307,6 +2307,40 @@ unicode_asutf8andsize(PyObject *self, PyObject *args) return Py_BuildValue("(Nn)", result, utf8_len); } +/* Test PyUnicode_DecodeUTF8() */ +static PyObject * +unicode_decodeutf8(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + return PyUnicode_DecodeUTF8(data, size, errors); +} + +/* Test PyUnicode_DecodeUTF8Stateful() */ +static PyObject * +unicode_decodeutf8stateful(PyObject *self, PyObject *args) +{ + const char *data; + Py_ssize_t size; + const char *errors = NULL; + Py_ssize_t consumed = 123456789; + PyObject *result; + + if (!PyArg_ParseTuple(args, "y#|z", &data, &size, &errors)) + return NULL; + + result = PyUnicode_DecodeUTF8Stateful(data, size, errors, &consumed); + if (!result) { + return NULL; + } + return Py_BuildValue("(Nn)", result, consumed); +} + static PyObject * unicode_findchar(PyObject *self, PyObject *args) { @@ -6562,7 +6596,8 @@ static PyMethodDef TestMethods[] = { {"unicode_asucs4", unicode_asucs4, METH_VARARGS}, {"unicode_asutf8", unicode_asutf8, METH_VARARGS}, {"unicode_asutf8andsize", unicode_asutf8andsize, METH_VARARGS}, - {"unicode_findchar", unicode_findchar, METH_VARARGS}, + {"unicode_decodeutf8", unicode_decodeutf8, METH_VARARGS}, + {"unicode_decodeutf8stateful",unicode_decodeutf8stateful, METH_VARARGS}, {"unicode_findchar", unicode_findchar, METH_VARARGS}, {"unicode_copycharacters", unicode_copycharacters, METH_VARARGS}, #if USE_UNICODE_WCHAR_CACHE {"unicode_legacy_string", unicode_legacy_string, METH_VARARGS}, diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 50deefe0adc2c..ffa8b982bdc29 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -5120,6 +5120,9 @@ unicode_decode_utf8(const char *s, Py_ssize_t size, } s += ascii_decode(s, end, PyUnicode_1BYTE_DATA(u)); if (s == end) { + if (consumed) { + *consumed = size; + } return u; } From webhook-mailer at python.org Tue Jul 25 07:34:53 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 11:34:53 -0000 Subject: [Python-checkins] gh-86493: Modernize modules initialization code (GH-106858) Message-ID: https://github.com/python/cpython/commit/329e4a1a3f8c53d9d120d2eed93b95a04b826f2f commit: 329e4a1a3f8c53d9d120d2eed93b95a04b826f2f branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-25T14:34:49+03:00 summary: gh-86493: Modernize modules initialization code (GH-106858) Use PyModule_Add() or PyModule_AddObjectRef() instead of soft deprecated PyModule_AddObject(). files: M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/includes/custom.c M Doc/includes/sublist.c M Modules/_datetimemodule.c M Modules/_decimal/_decimal.c M Modules/_gdbmmodule.c M Modules/_hashopenssl.c M Modules/_heapqmodule.c M Modules/_localemodule.c M Modules/_lzmamodule.c M Modules/_multiprocessing/multiprocessing.c M Modules/_ssl.c M Modules/_testcapi/heaptype.c M Modules/_testmultiphase.c M Modules/_tkinter.c M Modules/_weakref.c M Modules/arraymodule.c M Modules/binascii.c M Modules/cjkcodecs/cjkcodecs.h M Modules/md5module.c M Modules/mmapmodule.c M Modules/overlapped.c M Modules/pyexpat.c M Modules/resource.c M Modules/sha1module.c M Modules/termios.c M Modules/unicodedata.c M Modules/zlibmodule.c diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 0c5da49fe6134..00a9edcf2e414 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -221,9 +221,7 @@ with an exception object:: return NULL; SpamError = PyErr_NewException("spam.error", NULL, NULL); - Py_XINCREF(SpamError); - if (PyModule_AddObject(m, "error", SpamError) < 0) { - Py_XDECREF(SpamError); + if (PyModule_AddObjectRef(m, "error", SpamError) < 0) { Py_CLEAR(SpamError); Py_DECREF(m); return NULL; @@ -1281,8 +1279,7 @@ function must take care of initializing the C API pointer array:: /* Create a Capsule containing the API pointer array's address */ c_api_object = PyCapsule_New((void *)PySpam_API, "spam._C_API", NULL); - if (PyModule_AddObject(m, "_C_API", c_api_object) < 0) { - Py_XDECREF(c_api_object); + if (PyModule_Add(m, "_C_API", c_api_object) < 0) { Py_DECREF(m); return NULL; } diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index ba09fb1b05c0b..4f9c8899ffee1 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -180,9 +180,7 @@ This initializes the :class:`Custom` type, filling in a number of members to the appropriate default values, including :attr:`ob_type` that we initially set to ``NULL``. :: - Py_INCREF(&CustomType); - if (PyModule_AddObject(m, "Custom", (PyObject *) &CustomType) < 0) { - Py_DECREF(&CustomType); + if (PyModule_AddObjectRef(m, "Custom", (PyObject *) &CustomType) < 0) { Py_DECREF(m); return NULL; } @@ -862,9 +860,7 @@ function:: if (m == NULL) return NULL; - Py_INCREF(&SubListType); - if (PyModule_AddObject(m, "SubList", (PyObject *) &SubListType) < 0) { - Py_DECREF(&SubListType); + if (PyModule_AddObjectRef(m, "SubList", (PyObject *) &SubListType) < 0) { Py_DECREF(m); return NULL; } diff --git a/Doc/includes/custom.c b/Doc/includes/custom.c index 9cfba50ace25d..5253f87936021 100644 --- a/Doc/includes/custom.c +++ b/Doc/includes/custom.c @@ -34,9 +34,7 @@ PyInit_custom(void) if (m == NULL) return NULL; - Py_INCREF(&CustomType); - if (PyModule_AddObject(m, "Custom", (PyObject *) &CustomType) < 0) { - Py_DECREF(&CustomType); + if (PyModule_AddObjectRef(m, "Custom", (PyObject *) &CustomType) < 0) { Py_DECREF(m); return NULL; } diff --git a/Doc/includes/sublist.c b/Doc/includes/sublist.c index b36dadf07eae8..d8aba463f30ba 100644 --- a/Doc/includes/sublist.c +++ b/Doc/includes/sublist.c @@ -58,9 +58,7 @@ PyInit_sublist(void) if (m == NULL) return NULL; - Py_INCREF(&SubListType); - if (PyModule_AddObject(m, "SubList", (PyObject *) &SubListType) < 0) { - Py_DECREF(&SubListType); + if (PyModule_AddObjectRef(m, "SubList", (PyObject *) &SubListType) < 0) { Py_DECREF(m); return NULL; } diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index b8cb0c012fd53..e0e34b78d5d84 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -6834,8 +6834,7 @@ _datetime_exec(PyObject *module) return -1; } - if (PyModule_AddObject(module, "datetime_CAPI", x) < 0) { - Py_DECREF(x); + if (PyModule_Add(module, "datetime_CAPI", x) < 0) { return -1; } diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index f531def7b3a33..671976a220a64 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -6053,9 +6053,8 @@ PyInit__decimal(void) /* Init mpd_ssize_t constants */ for (ssize_cm = ssize_constants; ssize_cm->name != NULL; ssize_cm++) { - ASSIGN_PTR(obj, PyLong_FromSsize_t(ssize_cm->val)); - CHECK_INT(PyModule_AddObject(m, ssize_cm->name, obj)); - obj = NULL; + CHECK_INT(PyModule_Add(m, ssize_cm->name, + PyLong_FromSsize_t(ssize_cm->val))); } /* Init int constants */ diff --git a/Modules/_gdbmmodule.c b/Modules/_gdbmmodule.c index 47f2da8e0e3d0..eff36fd7fb669 100644 --- a/Modules/_gdbmmodule.c +++ b/Modules/_gdbmmodule.c @@ -787,11 +787,7 @@ _gdbm_exec(PyObject *module) defined(GDBM_VERSION_PATCH) PyObject *obj = Py_BuildValue("iii", GDBM_VERSION_MAJOR, GDBM_VERSION_MINOR, GDBM_VERSION_PATCH); - if (obj == NULL) { - return -1; - } - if (PyModule_AddObject(module, "_GDBM_VERSION", obj) < 0) { - Py_DECREF(obj); + if (PyModule_Add(module, "_GDBM_VERSION", obj) < 0) { return -1; } #endif diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c index 6d60037cd2054..ee6fb8b4b0364 100644 --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -1889,12 +1889,7 @@ hashlib_md_meth_names(PyObject *module) return -1; } - if (PyModule_AddObject(module, "openssl_md_meth_names", state.set) < 0) { - Py_DECREF(state.set); - return -1; - } - - return 0; + return PyModule_Add(module, "openssl_md_meth_names", state.set); } /*[clinic input] diff --git a/Modules/_heapqmodule.c b/Modules/_heapqmodule.c index 00285ae01f857..9d4ec256ee9e3 100644 --- a/Modules/_heapqmodule.c +++ b/Modules/_heapqmodule.c @@ -672,9 +672,7 @@ From all times, sorting has always been a Great Art! :-)\n"); static int heapq_exec(PyObject *m) { - PyObject *about = PyUnicode_FromString(__about__); - if (PyModule_AddObject(m, "__about__", about) < 0) { - Py_DECREF(about); + if (PyModule_Add(m, "__about__", PyUnicode_FromString(__about__)) < 0) { return -1; } return 0; diff --git a/Modules/_localemodule.c b/Modules/_localemodule.c index 970530facd01b..34915de7515d6 100644 --- a/Modules/_localemodule.c +++ b/Modules/_localemodule.c @@ -844,12 +844,7 @@ _locale_exec(PyObject *module) _locale_state *state = get_locale_state(module); state->Error = PyErr_NewException("locale.Error", NULL, NULL); - if (state->Error == NULL) { - return -1; - } - Py_INCREF(get_locale_state(module)->Error); - if (PyModule_AddObject(module, "Error", get_locale_state(module)->Error) < 0) { - Py_DECREF(get_locale_state(module)->Error); + if (PyModule_AddObjectRef(module, "Error", state->Error) < 0) { return -1; } diff --git a/Modules/_lzmamodule.c b/Modules/_lzmamodule.c index 02a32ddccb3ed..edb2e3673f0b2 100644 --- a/Modules/_lzmamodule.c +++ b/Modules/_lzmamodule.c @@ -1498,15 +1498,7 @@ _lzma__decode_filter_properties_impl(PyObject *module, lzma_vli filter_id, static int module_add_int_constant(PyObject *m, const char *name, long long value) { - PyObject *o = PyLong_FromLongLong(value); - if (o == NULL) { - return -1; - } - if (PyModule_AddObject(m, name, o) == 0) { - return 0; - } - Py_DECREF(o); - return -1; + return PyModule_Add(m, name, PyLong_FromLongLong(value)); } static int diff --git a/Modules/_multiprocessing/multiprocessing.c b/Modules/_multiprocessing/multiprocessing.c index 8f9daa5c3de0c..16b5cb5dd9ec7 100644 --- a/Modules/_multiprocessing/multiprocessing.c +++ b/Modules/_multiprocessing/multiprocessing.c @@ -266,8 +266,7 @@ multiprocessing_exec(PyObject *module) ADD_FLAG(HAVE_BROKEN_SEM_UNLINK); #endif - if (PyModule_AddObject(module, "flags", flags) < 0) { - Py_DECREF(flags); + if (PyModule_Add(module, "flags", flags) < 0) { return -1; } diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 699431c248a07..b61d10d9f59fd 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -5986,7 +5986,7 @@ sslmodule_init_constants(PyObject *m) #define addbool(m, key, value) \ do { \ PyObject *bool_obj = (value) ? Py_True : Py_False; \ - PyModule_AddObject((m), (key), Py_NewRef(bool_obj)); \ + PyModule_AddObjectRef((m), (key), bool_obj); \ } while (0) addbool(m, "HAS_SNI", 1); diff --git a/Modules/_testcapi/heaptype.c b/Modules/_testcapi/heaptype.c index c124871e43343..2da06f447afaa 100644 --- a/Modules/_testcapi/heaptype.c +++ b/Modules/_testcapi/heaptype.c @@ -1122,94 +1122,62 @@ _PyTestCapi_Init_Heaptype(PyObject *m) { return -1; } +#define ADD(name, value) do { \ + if (PyModule_Add(m, name, value) < 0) { \ + return -1; \ + } \ + } while (0) + PyObject *HeapDocCType = PyType_FromSpec(&HeapDocCType_spec); - if (HeapDocCType == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapDocCType", HeapDocCType); + ADD("HeapDocCType", HeapDocCType); /* bpo-41832: Add a new type to test PyType_FromSpec() now can accept a NULL tp_doc slot. */ PyObject *NullTpDocType = PyType_FromSpec(&NullTpDocType_spec); - if (NullTpDocType == NULL) { - return -1; - } - PyModule_AddObject(m, "NullTpDocType", NullTpDocType); + ADD("NullTpDocType", NullTpDocType); PyObject *HeapGcCType = PyType_FromSpec(&HeapGcCType_spec); - if (HeapGcCType == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapGcCType", HeapGcCType); + ADD("HeapGcCType", HeapGcCType); PyObject *HeapCType = PyType_FromSpec(&HeapCType_spec); if (HeapCType == NULL) { return -1; } PyObject *subclass_bases = PyTuple_Pack(1, HeapCType); + Py_DECREF(HeapCType); if (subclass_bases == NULL) { return -1; } PyObject *HeapCTypeSubclass = PyType_FromSpecWithBases(&HeapCTypeSubclass_spec, subclass_bases); - if (HeapCTypeSubclass == NULL) { - return -1; - } Py_DECREF(subclass_bases); - PyModule_AddObject(m, "HeapCTypeSubclass", HeapCTypeSubclass); + ADD("HeapCTypeSubclass", HeapCTypeSubclass); PyObject *HeapCTypeWithDict = PyType_FromSpec(&HeapCTypeWithDict_spec); - if (HeapCTypeWithDict == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeWithDict", HeapCTypeWithDict); + ADD("HeapCTypeWithDict", HeapCTypeWithDict); PyObject *HeapCTypeWithDict2 = PyType_FromSpec(&HeapCTypeWithDict2_spec); - if (HeapCTypeWithDict2 == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeWithDict2", HeapCTypeWithDict2); + ADD("HeapCTypeWithDict2", HeapCTypeWithDict2); PyObject *HeapCTypeWithNegativeDict = PyType_FromSpec(&HeapCTypeWithNegativeDict_spec); - if (HeapCTypeWithNegativeDict == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeWithNegativeDict", HeapCTypeWithNegativeDict); + ADD("HeapCTypeWithNegativeDict", HeapCTypeWithNegativeDict); PyObject *HeapCTypeWithManagedDict = PyType_FromSpec(&HeapCTypeWithManagedDict_spec); - if (HeapCTypeWithManagedDict == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeWithManagedDict", HeapCTypeWithManagedDict); + ADD("HeapCTypeWithManagedDict", HeapCTypeWithManagedDict); PyObject *HeapCTypeWithManagedWeakref = PyType_FromSpec(&HeapCTypeWithManagedWeakref_spec); - if (HeapCTypeWithManagedWeakref == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeWithManagedWeakref", HeapCTypeWithManagedWeakref); + ADD("HeapCTypeWithManagedWeakref", HeapCTypeWithManagedWeakref); PyObject *HeapCTypeWithWeakref = PyType_FromSpec(&HeapCTypeWithWeakref_spec); - if (HeapCTypeWithWeakref == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeWithWeakref", HeapCTypeWithWeakref); + ADD("HeapCTypeWithWeakref", HeapCTypeWithWeakref); PyObject *HeapCTypeWithWeakref2 = PyType_FromSpec(&HeapCTypeWithWeakref2_spec); - if (HeapCTypeWithWeakref2 == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeWithWeakref2", HeapCTypeWithWeakref2); + ADD("HeapCTypeWithWeakref2", HeapCTypeWithWeakref2); PyObject *HeapCTypeWithBuffer = PyType_FromSpec(&HeapCTypeWithBuffer_spec); - if (HeapCTypeWithBuffer == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeWithBuffer", HeapCTypeWithBuffer); + ADD("HeapCTypeWithBuffer", HeapCTypeWithBuffer); PyObject *HeapCTypeSetattr = PyType_FromSpec(&HeapCTypeSetattr_spec); - if (HeapCTypeSetattr == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeSetattr", HeapCTypeSetattr); + ADD("HeapCTypeSetattr", HeapCTypeSetattr); PyObject *subclass_with_finalizer_bases = PyTuple_Pack(1, HeapCTypeSubclass); if (subclass_with_finalizer_bases == NULL) { @@ -1217,32 +1185,20 @@ _PyTestCapi_Init_Heaptype(PyObject *m) { } PyObject *HeapCTypeSubclassWithFinalizer = PyType_FromSpecWithBases( &HeapCTypeSubclassWithFinalizer_spec, subclass_with_finalizer_bases); - if (HeapCTypeSubclassWithFinalizer == NULL) { - return -1; - } Py_DECREF(subclass_with_finalizer_bases); - PyModule_AddObject(m, "HeapCTypeSubclassWithFinalizer", HeapCTypeSubclassWithFinalizer); + ADD("HeapCTypeSubclassWithFinalizer", HeapCTypeSubclassWithFinalizer); PyObject *HeapCTypeMetaclass = PyType_FromMetaclass( &PyType_Type, m, &HeapCTypeMetaclass_spec, (PyObject *) &PyType_Type); - if (HeapCTypeMetaclass == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeMetaclass", HeapCTypeMetaclass); + ADD("HeapCTypeMetaclass", HeapCTypeMetaclass); PyObject *HeapCTypeMetaclassCustomNew = PyType_FromMetaclass( &PyType_Type, m, &HeapCTypeMetaclassCustomNew_spec, (PyObject *) &PyType_Type); - if (HeapCTypeMetaclassCustomNew == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeMetaclassCustomNew", HeapCTypeMetaclassCustomNew); + ADD("HeapCTypeMetaclassCustomNew", HeapCTypeMetaclassCustomNew); PyObject *HeapCTypeMetaclassNullNew = PyType_FromMetaclass( &PyType_Type, m, &HeapCTypeMetaclassNullNew_spec, (PyObject *) &PyType_Type); - if (HeapCTypeMetaclassNullNew == NULL) { - return -1; - } - PyModule_AddObject(m, "HeapCTypeMetaclassNullNew", HeapCTypeMetaclassNullNew); + ADD("HeapCTypeMetaclassNullNew", HeapCTypeMetaclassNullNew); PyObject *HeapCCollection = PyType_FromMetaclass( NULL, m, &HeapCCollection_spec, NULL); diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c index ca71b6156b005..fdef06168bfc8 100644 --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -383,32 +383,20 @@ static int execfunc(PyObject *m) /* Add a custom type */ temp = PyType_FromSpec(&Example_Type_spec); - if (temp == NULL) { - goto fail; - } - if (PyModule_AddObject(m, "Example", temp) != 0) { - Py_DECREF(temp); + if (PyModule_Add(m, "Example", temp) != 0) { goto fail; } /* Add an exception type */ temp = PyErr_NewException("_testimportexec.error", NULL, NULL); - if (temp == NULL) { - goto fail; - } - if (PyModule_AddObject(m, "error", temp) != 0) { - Py_DECREF(temp); + if (PyModule_Add(m, "error", temp) != 0) { goto fail; } /* Add Str */ temp = PyType_FromSpec(&Str_Type_spec); - if (temp == NULL) { - goto fail; - } - if (PyModule_AddObject(m, "Str", temp) != 0) { - Py_DECREF(temp); + if (PyModule_Add(m, "Str", temp) != 0) { goto fail; } @@ -857,11 +845,7 @@ meth_state_access_exec(PyObject *m) } temp = PyType_FromModuleAndSpec(m, &StateAccessType_spec, NULL); - if (temp == NULL) { - return -1; - } - if (PyModule_AddObject(m, "StateAccessType", temp) != 0) { - Py_DECREF(temp); + if (PyModule_Add(m, "StateAccessType", temp) != 0) { return -1; } diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c index 76af803bd6eef..1605606e8a470 100644 --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -3197,7 +3197,7 @@ static struct PyModuleDef _tkintermodule = { PyMODINIT_FUNC PyInit__tkinter(void) { - PyObject *m, *uexe, *cexe, *o; + PyObject *m, *uexe, *cexe; tcl_lock = PyThread_allocate_lock(); if (tcl_lock == NULL) @@ -3207,17 +3207,11 @@ PyInit__tkinter(void) if (m == NULL) return NULL; - o = PyErr_NewException("_tkinter.TclError", NULL, NULL); - if (o == NULL) { + Tkinter_TclError = PyErr_NewException("_tkinter.TclError", NULL, NULL); + if (PyModule_AddObjectRef(m, "TclError", Tkinter_TclError)) { Py_DECREF(m); return NULL; } - if (PyModule_AddObject(m, "TclError", Py_NewRef(o))) { - Py_DECREF(o); - Py_DECREF(m); - return NULL; - } - Tkinter_TclError = o; if (PyModule_AddIntConstant(m, "READABLE", TCL_READABLE)) { Py_DECREF(m); @@ -3264,41 +3258,23 @@ PyInit__tkinter(void) return NULL; } - o = PyType_FromSpec(&Tkapp_Type_spec); - if (o == NULL) { - Py_DECREF(m); - return NULL; - } - if (PyModule_AddObject(m, "TkappType", o)) { - Py_DECREF(o); + Tkapp_Type = PyType_FromSpec(&Tkapp_Type_spec); + if (PyModule_AddObjectRef(m, "TkappType", Tkapp_Type)) { Py_DECREF(m); return NULL; } - Tkapp_Type = o; - o = PyType_FromSpec(&Tktt_Type_spec); - if (o == NULL) { + Tktt_Type = PyType_FromSpec(&Tktt_Type_spec); + if (PyModule_AddObjectRef(m, "TkttType", Tktt_Type)) { Py_DECREF(m); return NULL; } - if (PyModule_AddObject(m, "TkttType", o)) { - Py_DECREF(o); - Py_DECREF(m); - return NULL; - } - Tktt_Type = o; - o = PyType_FromSpec(&PyTclObject_Type_spec); - if (o == NULL) { - Py_DECREF(m); - return NULL; - } - if (PyModule_AddObject(m, "Tcl_Obj", o)) { - Py_DECREF(o); + PyTclObject_Type = PyType_FromSpec(&PyTclObject_Type_spec); + if (PyModule_AddObjectRef(m, "Tcl_Obj", PyTclObject_Type)) { Py_DECREF(m); return NULL; } - PyTclObject_Type = o; /* This helps the dynamic loader; in Unicode aware Tcl versions diff --git a/Modules/_weakref.c b/Modules/_weakref.c index dcb2984e3e817..4e2862e7467c3 100644 --- a/Modules/_weakref.c +++ b/Modules/_weakref.c @@ -144,27 +144,19 @@ weakref_functions[] = { static int weakref_exec(PyObject *module) { - Py_INCREF(&_PyWeakref_RefType); - if (PyModule_AddObject(module, "ref", (PyObject *) &_PyWeakref_RefType) < 0) { - Py_DECREF(&_PyWeakref_RefType); + if (PyModule_AddObjectRef(module, "ref", (PyObject *) &_PyWeakref_RefType) < 0) { return -1; } - Py_INCREF(&_PyWeakref_RefType); - if (PyModule_AddObject(module, "ReferenceType", + if (PyModule_AddObjectRef(module, "ReferenceType", (PyObject *) &_PyWeakref_RefType) < 0) { - Py_DECREF(&_PyWeakref_RefType); return -1; } - Py_INCREF(&_PyWeakref_ProxyType); - if (PyModule_AddObject(module, "ProxyType", + if (PyModule_AddObjectRef(module, "ProxyType", (PyObject *) &_PyWeakref_ProxyType) < 0) { - Py_DECREF(&_PyWeakref_ProxyType); return -1; } - Py_INCREF(&_PyWeakref_CallableProxyType); - if (PyModule_AddObject(module, "CallableProxyType", + if (PyModule_AddObjectRef(module, "CallableProxyType", (PyObject *) &_PyWeakref_CallableProxyType) < 0) { - Py_DECREF(&_PyWeakref_CallableProxyType); return -1; } diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c index 0000a8d637eb5..80a95428ebfd0 100644 --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -3163,9 +3163,8 @@ array_modexec(PyObject *m) CREATE_TYPE(m, state->ArrayIterType, &arrayiter_spec); Py_SET_TYPE(state->ArrayIterType, &PyType_Type); - if (PyModule_AddObject(m, "ArrayType", - Py_NewRef((PyObject *)state->ArrayType)) < 0) { - Py_DECREF((PyObject *)state->ArrayType); + if (PyModule_AddObjectRef(m, "ArrayType", + (PyObject *)state->ArrayType) < 0) { return -1; } @@ -3193,8 +3192,7 @@ array_modexec(PyObject *m) *p++ = (char)descr->typecode; } typecodes = PyUnicode_DecodeASCII(buffer, p - buffer, NULL); - if (PyModule_AddObject(m, "typecodes", typecodes) < 0) { - Py_XDECREF(typecodes); + if (PyModule_Add(m, "typecodes", typecodes) < 0) { return -1; } diff --git a/Modules/binascii.c b/Modules/binascii.c index cf9328795c2bc..a87a2ef2e8992 100644 --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -1253,32 +1253,20 @@ static struct PyMethodDef binascii_module_methods[] = { PyDoc_STRVAR(doc_binascii, "Conversion between binary data and ASCII"); static int -binascii_exec(PyObject *module) { - int result; +binascii_exec(PyObject *module) +{ binascii_state *state = PyModule_GetState(module); if (state == NULL) { return -1; } state->Error = PyErr_NewException("binascii.Error", PyExc_ValueError, NULL); - if (state->Error == NULL) { - return -1; - } - Py_INCREF(state->Error); - result = PyModule_AddObject(module, "Error", state->Error); - if (result == -1) { - Py_DECREF(state->Error); + if (PyModule_AddObjectRef(module, "Error", state->Error) < 0) { return -1; } state->Incomplete = PyErr_NewException("binascii.Incomplete", NULL, NULL); - if (state->Incomplete == NULL) { - return -1; - } - Py_INCREF(state->Incomplete); - result = PyModule_AddObject(module, "Incomplete", state->Incomplete); - if (result == -1) { - Py_DECREF(state->Incomplete); + if (PyModule_AddObjectRef(module, "Incomplete", state->Incomplete) < 0) { return -1; } diff --git a/Modules/cjkcodecs/cjkcodecs.h b/Modules/cjkcodecs/cjkcodecs.h index ee588785e7403..766f82983025e 100644 --- a/Modules/cjkcodecs/cjkcodecs.h +++ b/Modules/cjkcodecs/cjkcodecs.h @@ -398,11 +398,7 @@ register_maps(PyObject *module) strcpy(mhname + sizeof("__map_") - 1, h->charset); PyObject *capsule = PyCapsule_New((void *)h, MAP_CAPSULE, NULL); - if (capsule == NULL) { - return -1; - } - if (PyModule_AddObject(module, mhname, capsule) < 0) { - Py_DECREF(capsule); + if (PyModule_Add(module, mhname, capsule) < 0) { return -1; } } diff --git a/Modules/md5module.c b/Modules/md5module.c index 2122f8b18baf6..5463effb507de 100644 --- a/Modules/md5module.c +++ b/Modules/md5module.c @@ -356,13 +356,7 @@ md5_exec(PyObject *m) st->md5_type = (PyTypeObject *)PyType_FromModuleAndSpec( m, &md5_type_spec, NULL); - if (st->md5_type == NULL) { - return -1; - } - - Py_INCREF((PyObject *)st->md5_type); - if (PyModule_AddObject(m, "MD5Type", (PyObject *)st->md5_type) < 0) { - Py_DECREF(st->md5_type); + if (PyModule_AddObjectRef(m, "MD5Type", (PyObject *)st->md5_type) < 0) { return -1; } diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index c1cd5b0efaa3d..cfbd2f41130ac 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -1579,9 +1579,7 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) static int mmap_exec(PyObject *module) { - Py_INCREF(PyExc_OSError); - if (PyModule_AddObject(module, "error", PyExc_OSError) < 0) { - Py_DECREF(PyExc_OSError); + if (PyModule_AddObjectRef(module, "error", PyExc_OSError) < 0) { return -1; } diff --git a/Modules/overlapped.c b/Modules/overlapped.c index 18899509c8771..7d3976faef96b 100644 --- a/Modules/overlapped.c +++ b/Modules/overlapped.c @@ -1996,12 +1996,7 @@ static PyMethodDef overlapped_functions[] = { #define WINAPI_CONSTANT(fmt, con) \ do { \ - PyObject *value = Py_BuildValue(fmt, con); \ - if (value == NULL) { \ - return -1; \ - } \ - if (PyModule_AddObject(module, #con, value) < 0 ) { \ - Py_DECREF(value); \ + if (PyModule_Add(module, #con, Py_BuildValue(fmt, con)) < 0 ) { \ return -1; \ } \ } while (0) diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index 33a56c57a876e..d8395e9a9aa80 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1656,8 +1656,7 @@ add_submodule(PyObject *mod, const char *fullname) Py_DECREF(mod_name); /* gives away the reference to the submodule */ - if (PyModule_AddObject(mod, name, submodule) < 0) { - Py_DECREF(submodule); + if (PyModule_Add(mod, name, submodule) < 0) { return NULL; } @@ -1887,10 +1886,7 @@ add_features(PyObject *mod) goto error; } } - if (PyModule_AddObject(mod, "features", list) < 0) { - goto error; - } - return 0; + return PyModule_Add(mod, "features", list); error: Py_DECREF(list); @@ -1959,8 +1955,7 @@ pyexpat_exec(PyObject *mod) info.major, info.minor, info.micro); - if (PyModule_AddObject(mod, "version_info", versionInfo) < 0) { - Py_DECREF(versionInfo); + if (PyModule_Add(mod, "version_info", versionInfo) < 0) { return -1; } } @@ -2040,8 +2035,7 @@ pyexpat_exec(PyObject *mod) return -1; } - if (PyModule_AddObject(mod, "expat_CAPI", capi_object) < 0) { - Py_DECREF(capi_object); + if (PyModule_Add(mod, "expat_CAPI", capi_object) < 0) { return -1; } diff --git a/Modules/resource.c b/Modules/resource.c index 3c89468c48c56..4614f5e98cc88 100644 --- a/Modules/resource.c +++ b/Modules/resource.c @@ -372,9 +372,7 @@ resource_exec(PyObject *module) } while (0) /* Add some symbolic constants to the module */ - Py_INCREF(PyExc_OSError); - if (PyModule_AddObject(module, "error", PyExc_OSError) < 0) { - Py_DECREF(PyExc_OSError); + if (PyModule_AddObjectRef(module, "error", PyExc_OSError) < 0) { return -1; } @@ -502,12 +500,7 @@ resource_exec(PyObject *module) { v = PyLong_FromLong((long) RLIM_INFINITY); } - if (!v) { - return -1; - } - - if (PyModule_AddObject(module, "RLIM_INFINITY", v) < 0) { - Py_DECREF(v); + if (PyModule_Add(module, "RLIM_INFINITY", v) < 0) { return -1; } return 0; diff --git a/Modules/sha1module.c b/Modules/sha1module.c index ef8e067dd337b..3fd53123229ac 100644 --- a/Modules/sha1module.c +++ b/Modules/sha1module.c @@ -357,16 +357,9 @@ _sha1_exec(PyObject *module) st->sha1_type = (PyTypeObject *)PyType_FromModuleAndSpec( module, &sha1_type_spec, NULL); - - if (st->sha1_type == NULL) { - return -1; - } - - Py_INCREF(st->sha1_type); - if (PyModule_AddObject(module, + if (PyModule_AddObjectRef(module, "SHA1Type", (PyObject *)st->sha1_type) < 0) { - Py_DECREF(st->sha1_type); return -1; } diff --git a/Modules/termios.c b/Modules/termios.c index 6dc8200572bc0..6b254104ed1e2 100644 --- a/Modules/termios.c +++ b/Modules/termios.c @@ -1232,12 +1232,7 @@ termios_exec(PyObject *mod) struct constant *constant = termios_constants; termiosmodulestate *state = get_termios_state(mod); state->TermiosError = PyErr_NewException("termios.error", NULL, NULL); - if (state->TermiosError == NULL) { - return -1; - } - Py_INCREF(state->TermiosError); - if (PyModule_AddObject(mod, "error", state->TermiosError) < 0) { - Py_DECREF(state->TermiosError); + if (PyModule_AddObjectRef(mod, "error", state->TermiosError) < 0) { return -1; } diff --git a/Modules/unicodedata.c b/Modules/unicodedata.c index 966123f4624c0..6267c5ae29ab3 100644 --- a/Modules/unicodedata.c +++ b/Modules/unicodedata.c @@ -1487,11 +1487,7 @@ unicodedata_exec(PyObject *module) v = new_previous_version(ucd_type, "3.2.0", get_change_3_2_0, normalization_3_2_0); Py_DECREF(ucd_type); - if (v == NULL) { - return -1; - } - if (PyModule_AddObject(module, "ucd_3_2_0", v) < 0) { - Py_DECREF(v); + if (PyModule_Add(module, "ucd_3_2_0", v) < 0) { return -1; } diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c index c0f6b96f51bab..9970bc9e5ad97 100644 --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -2030,17 +2030,11 @@ zlib_exec(PyObject *mod) } state->ZlibError = PyErr_NewException("zlib.error", NULL, NULL); - if (state->ZlibError == NULL) { + if (PyModule_AddObjectRef(mod, "error", state->ZlibError) < 0) { return -1; } - - if (PyModule_AddObject(mod, "error", Py_NewRef(state->ZlibError)) < 0) { - Py_DECREF(state->ZlibError); - return -1; - } - if (PyModule_AddObject(mod, "_ZlibDecompressor", - Py_NewRef(state->ZlibDecompressorType)) < 0) { - Py_DECREF(state->ZlibDecompressorType); + if (PyModule_AddObjectRef(mod, "_ZlibDecompressor", + (PyObject *)state->ZlibDecompressorType) < 0) { return -1; } @@ -2082,26 +2076,14 @@ zlib_exec(PyObject *mod) #ifdef Z_TREES // 1.2.3.4, only for inflate ZLIB_ADD_INT_MACRO(Z_TREES); #endif - PyObject *ver = PyUnicode_FromString(ZLIB_VERSION); - if (ver == NULL) { + if (PyModule_Add(mod, "ZLIB_VERSION", + PyUnicode_FromString(ZLIB_VERSION)) < 0) { return -1; } - - if (PyModule_AddObject(mod, "ZLIB_VERSION", ver) < 0) { - Py_DECREF(ver); + if (PyModule_Add(mod, "ZLIB_RUNTIME_VERSION", + PyUnicode_FromString(zlibVersion())) < 0) { return -1; } - - ver = PyUnicode_FromString(zlibVersion()); - if (ver == NULL) { - return -1; - } - - if (PyModule_AddObject(mod, "ZLIB_RUNTIME_VERSION", ver) < 0) { - Py_DECREF(ver); - return -1; - } - if (PyModule_AddStringConstant(mod, "__version__", "1.0") < 0) { return -1; } From webhook-mailer at python.org Tue Jul 25 08:04:01 2023 From: webhook-mailer at python.org (hugovk) Date: Tue, 25 Jul 2023 12:04:01 -0000 Subject: [Python-checkins] gh-106996: Add a how-to section to the turtle documentation (#107153) Message-ID: https://github.com/python/cpython/commit/2425346feeb343db686b2fb071f7bd0130452b84 commit: 2425346feeb343db686b2fb071f7bd0130452b84 branch: main author: Daniele Procida committer: hugovk date: 2023-07-25T15:03:57+03:00 summary: gh-106996: Add a how-to section to the turtle documentation (#107153) Co-authored-by: Hugo van Kemenade files: M Doc/library/turtle.rst diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index 58ac546e75860..c3561a0b2f9ce 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -55,6 +55,8 @@ graphical output it can be a way to do that without the overhead of introducing more complex or external libraries into their work. +.. _turtle-tutorial: + Tutorial ======== @@ -174,6 +176,118 @@ Finally, complete the filling:: ``end_fill()`` command.) +.. _turtle-how-to: + +How to... +========= + +This section covers some typical turtle use-cases and approaches. + + +Get started as quickly as possible +---------------------------------- + +One of the joys of turtle graphics is the immediate, visual feedback that's +available from simple commands - it's an excellent way to introduce children +to programming ideas, with a minimum of overhead (not just children, of +course). + +The turtle module makes this possible by exposing all its basic functionality +as functions, available with ``from turtle import *``. The :ref:`turtle +graphics tutorial ` covers this approach. + +It's worth noting that many of the turtle commands also have even more terse +equivalents, such as ``fd()`` for :func:`forward`. These are especially +useful when working with learners for whom typing is not a skill. + +.. _note: + + You'll need to have the :mod:`Tk interface package ` installed on + your system for turtle graphics to work. Be warned that this is not + always straightforward, so check this in advance if you're planning to + use turtle graphics with a learner. + + +Use the ``turtle`` module namespace +----------------------------------- + +Using ``from turtle import *`` is convenient - but be warned that it imports a +rather large collection of objects, and if you're doing anything but turtle +graphics you run the risk of a name conflict (this becomes even more an issue +if you're using turtle graphics in a script where other modules might be +imported). + +The solution is to use ``import turtle`` - ``fd()`` becomes +``turtle.fd()``, ``width()`` becomes ``turtle.width()`` and so on. (If typing +"turtle" over and over again becomes tedious, use for example ``import turtle +as t`` instead.) + + +Use turtle graphics in a script +------------------------------- + +It's recommended to use the ``turtle`` module namespace as described +immediately above, for example:: + + import turtle as t + from random import random + + for i in range(100): + steps = int(random() * 100) + angle = int(random() * 360) + t.right(angle) + t.fd(steps) + +Another step is also required though - as soon as the script ends, Python +will also close the turtle's window. Add:: + + t.mainloop() + +to the end of the script. The script will now wait to be dismissed and +will not exit until it is terminated, for example by closing the turtle +graphics window. + + +Use object-oriented turtle graphics +----------------------------------- + +.. seealso:: :ref:`Explanation of the object-oriented interface ` + +Other than for very basic introductory purposes, or for trying things out +as quickly as possible, it's more usual and much more powerful to use the +object-oriented approach to turtle graphics. For example, this allows +multiple turtles on screen at once. + +In this approach, the various turtle commands are methods of objects (mostly of +``Turtle`` objects). You *can* use the object-oriented approach in the shell, +but it would be more typical in a Python script. + +The example above then becomes:: + + from turtle import Turtle + from random import random + + t = Turtle() + for i in range(100): + steps = int(random() * 100) + angle = int(random() * 360) + t.right(angle) + t.fd(steps) + + t.screen.mainloop() + +Note the last line. ``t.screen`` is an instance of the :class:`Screen` +that a Turtle instance exists on; it's created automatically along with +the turtle. + +The turtle's screen can be customised, for example:: + + t.screen.title('Object-oriented turtle demo') + t.screen.bgcolor("orange") + + +.. _turtle-explanation: + Explanation =========== From webhook-mailer at python.org Tue Jul 25 08:17:18 2023 From: webhook-mailer at python.org (hugovk) Date: Tue, 25 Jul 2023 12:17:18 -0000 Subject: [Python-checkins] [3.11] gh-106996: Add a how-to section to the turtle documentation (GH-107153) (#107234) Message-ID: https://github.com/python/cpython/commit/1bc160947ea58fb3d11fa8667266bc80e9a14b7f commit: 1bc160947ea58fb3d11fa8667266bc80e9a14b7f branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-25T12:17:14Z summary: [3.11] gh-106996: Add a how-to section to the turtle documentation (GH-107153) (#107234) Co-authored-by: Daniele Procida Co-authored-by: Hugo van Kemenade files: M Doc/library/turtle.rst diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index 03b299ba7279c..262739ec2bbd5 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -46,6 +46,8 @@ graphical output it can be a way to do that without the overhead of introducing more complex or external libraries into their work. +.. _turtle-tutorial: + Tutorial ======== @@ -165,6 +167,118 @@ Finally, complete the filling:: ``end_fill()`` command.) +.. _turtle-how-to: + +How to... +========= + +This section covers some typical turtle use-cases and approaches. + + +Get started as quickly as possible +---------------------------------- + +One of the joys of turtle graphics is the immediate, visual feedback that's +available from simple commands - it's an excellent way to introduce children +to programming ideas, with a minimum of overhead (not just children, of +course). + +The turtle module makes this possible by exposing all its basic functionality +as functions, available with ``from turtle import *``. The :ref:`turtle +graphics tutorial ` covers this approach. + +It's worth noting that many of the turtle commands also have even more terse +equivalents, such as ``fd()`` for :func:`forward`. These are especially +useful when working with learners for whom typing is not a skill. + +.. _note: + + You'll need to have the :mod:`Tk interface package ` installed on + your system for turtle graphics to work. Be warned that this is not + always straightforward, so check this in advance if you're planning to + use turtle graphics with a learner. + + +Use the ``turtle`` module namespace +----------------------------------- + +Using ``from turtle import *`` is convenient - but be warned that it imports a +rather large collection of objects, and if you're doing anything but turtle +graphics you run the risk of a name conflict (this becomes even more an issue +if you're using turtle graphics in a script where other modules might be +imported). + +The solution is to use ``import turtle`` - ``fd()`` becomes +``turtle.fd()``, ``width()`` becomes ``turtle.width()`` and so on. (If typing +"turtle" over and over again becomes tedious, use for example ``import turtle +as t`` instead.) + + +Use turtle graphics in a script +------------------------------- + +It's recommended to use the ``turtle`` module namespace as described +immediately above, for example:: + + import turtle as t + from random import random + + for i in range(100): + steps = int(random() * 100) + angle = int(random() * 360) + t.right(angle) + t.fd(steps) + +Another step is also required though - as soon as the script ends, Python +will also close the turtle's window. Add:: + + t.mainloop() + +to the end of the script. The script will now wait to be dismissed and +will not exit until it is terminated, for example by closing the turtle +graphics window. + + +Use object-oriented turtle graphics +----------------------------------- + +.. seealso:: :ref:`Explanation of the object-oriented interface ` + +Other than for very basic introductory purposes, or for trying things out +as quickly as possible, it's more usual and much more powerful to use the +object-oriented approach to turtle graphics. For example, this allows +multiple turtles on screen at once. + +In this approach, the various turtle commands are methods of objects (mostly of +``Turtle`` objects). You *can* use the object-oriented approach in the shell, +but it would be more typical in a Python script. + +The example above then becomes:: + + from turtle import Turtle + from random import random + + t = Turtle() + for i in range(100): + steps = int(random() * 100) + angle = int(random() * 360) + t.right(angle) + t.fd(steps) + + t.screen.mainloop() + +Note the last line. ``t.screen`` is an instance of the :class:`Screen` +that a Turtle instance exists on; it's created automatically along with +the turtle. + +The turtle's screen can be customised, for example:: + + t.screen.title('Object-oriented turtle demo') + t.screen.bgcolor("orange") + + +.. _turtle-explanation: + Explanation =========== From webhook-mailer at python.org Tue Jul 25 08:20:34 2023 From: webhook-mailer at python.org (hugovk) Date: Tue, 25 Jul 2023 12:20:34 -0000 Subject: [Python-checkins] [3.12] gh-106996: Add a how-to section to the turtle documentation (GH-107153) (#107233) Message-ID: https://github.com/python/cpython/commit/fab36fb63eda4023097cb7f800d672ced26e338f commit: fab36fb63eda4023097cb7f800d672ced26e338f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-25T12:20:30Z summary: [3.12] gh-106996: Add a how-to section to the turtle documentation (GH-107153) (#107233) Co-authored-by: Daniele Procida Co-authored-by: Hugo van Kemenade files: M Doc/library/turtle.rst diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index a9c0fa96207ae..a85c3f466323f 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -46,6 +46,8 @@ graphical output it can be a way to do that without the overhead of introducing more complex or external libraries into their work. +.. _turtle-tutorial: + Tutorial ======== @@ -165,6 +167,118 @@ Finally, complete the filling:: ``end_fill()`` command.) +.. _turtle-how-to: + +How to... +========= + +This section covers some typical turtle use-cases and approaches. + + +Get started as quickly as possible +---------------------------------- + +One of the joys of turtle graphics is the immediate, visual feedback that's +available from simple commands - it's an excellent way to introduce children +to programming ideas, with a minimum of overhead (not just children, of +course). + +The turtle module makes this possible by exposing all its basic functionality +as functions, available with ``from turtle import *``. The :ref:`turtle +graphics tutorial ` covers this approach. + +It's worth noting that many of the turtle commands also have even more terse +equivalents, such as ``fd()`` for :func:`forward`. These are especially +useful when working with learners for whom typing is not a skill. + +.. _note: + + You'll need to have the :mod:`Tk interface package ` installed on + your system for turtle graphics to work. Be warned that this is not + always straightforward, so check this in advance if you're planning to + use turtle graphics with a learner. + + +Use the ``turtle`` module namespace +----------------------------------- + +Using ``from turtle import *`` is convenient - but be warned that it imports a +rather large collection of objects, and if you're doing anything but turtle +graphics you run the risk of a name conflict (this becomes even more an issue +if you're using turtle graphics in a script where other modules might be +imported). + +The solution is to use ``import turtle`` - ``fd()`` becomes +``turtle.fd()``, ``width()`` becomes ``turtle.width()`` and so on. (If typing +"turtle" over and over again becomes tedious, use for example ``import turtle +as t`` instead.) + + +Use turtle graphics in a script +------------------------------- + +It's recommended to use the ``turtle`` module namespace as described +immediately above, for example:: + + import turtle as t + from random import random + + for i in range(100): + steps = int(random() * 100) + angle = int(random() * 360) + t.right(angle) + t.fd(steps) + +Another step is also required though - as soon as the script ends, Python +will also close the turtle's window. Add:: + + t.mainloop() + +to the end of the script. The script will now wait to be dismissed and +will not exit until it is terminated, for example by closing the turtle +graphics window. + + +Use object-oriented turtle graphics +----------------------------------- + +.. seealso:: :ref:`Explanation of the object-oriented interface ` + +Other than for very basic introductory purposes, or for trying things out +as quickly as possible, it's more usual and much more powerful to use the +object-oriented approach to turtle graphics. For example, this allows +multiple turtles on screen at once. + +In this approach, the various turtle commands are methods of objects (mostly of +``Turtle`` objects). You *can* use the object-oriented approach in the shell, +but it would be more typical in a Python script. + +The example above then becomes:: + + from turtle import Turtle + from random import random + + t = Turtle() + for i in range(100): + steps = int(random() * 100) + angle = int(random() * 360) + t.right(angle) + t.fd(steps) + + t.screen.mainloop() + +Note the last line. ``t.screen`` is an instance of the :class:`Screen` +that a Turtle instance exists on; it's created automatically along with +the turtle. + +The turtle's screen can be customised, for example:: + + t.screen.title('Object-oriented turtle demo') + t.screen.bgcolor("orange") + + +.. _turtle-explanation: + Explanation =========== From webhook-mailer at python.org Tue Jul 25 08:27:52 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 12:27:52 -0000 Subject: [Python-checkins] gh-105059: Use GCC/clang extension for PyObject union (#107232) Message-ID: https://github.com/python/cpython/commit/6261585d63a31835b65d445d99dc14cca3fe9cf5 commit: 6261585d63a31835b65d445d99dc14cca3fe9cf5 branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T12:27:48Z summary: gh-105059: Use GCC/clang extension for PyObject union (#107232) Anonymous union is new in C11. To prevent compiler warning when using -pedantic compiler option, use Clang and GCC extension on C99 and older. files: M Include/object.h diff --git a/Include/object.h b/Include/object.h index 7182eba3adfe1..2488d6cd0d40e 100644 --- a/Include/object.h +++ b/Include/object.h @@ -165,6 +165,11 @@ check by comparing the reference count field to the immortality reference count. */ struct _object { _PyObject_HEAD_EXTRA +#if (defined(__GNUC__) || defined(__clang__)) \ + && !(defined __STDC_VERSION__ && __STDC_VERSION__ >= 201112L) + // On C99 and older, anonymous union is a GCC and clang extension + __extension__ +#endif union { Py_ssize_t ob_refcnt; #if SIZEOF_VOID_P > 4 From webhook-mailer at python.org Tue Jul 25 09:01:30 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 13:01:30 -0000 Subject: [Python-checkins] [3.12] gh-105059: Use GCC/clang extension for PyObject union (GH-107232) (#107236) Message-ID: https://github.com/python/cpython/commit/0cf5f6a6db0c50e46687de1fd99f90d7f85f69ea commit: 0cf5f6a6db0c50e46687de1fd99f90d7f85f69ea branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-25T13:01:24Z summary: [3.12] gh-105059: Use GCC/clang extension for PyObject union (GH-107232) (#107236) gh-105059: Use GCC/clang extension for PyObject union (GH-107232) Anonymous union is new in C11. To prevent compiler warning when using -pedantic compiler option, use Clang and GCC extension on C99 and older. (cherry picked from commit 6261585d63a31835b65d445d99dc14cca3fe9cf5) Co-authored-by: Victor Stinner files: M Include/object.h diff --git a/Include/object.h b/Include/object.h index 7564b9623be79..542f8d8c15a7c 100644 --- a/Include/object.h +++ b/Include/object.h @@ -165,6 +165,11 @@ check by comparing the reference count field to the immortality reference count. */ struct _object { _PyObject_HEAD_EXTRA +#if (defined(__GNUC__) || defined(__clang__)) \ + && !(defined __STDC_VERSION__ && __STDC_VERSION__ >= 201112L) + // On C99 and older, anonymous union is a GCC and clang extension + __extension__ +#endif union { Py_ssize_t ob_refcnt; #if SIZEOF_VOID_P > 4 From webhook-mailer at python.org Tue Jul 25 09:14:04 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 13:14:04 -0000 Subject: [Python-checkins] gh-107237: Fix test_udp_reconnection() of test_logging (#107238) Message-ID: https://github.com/python/cpython/commit/ed082383272c2c238e364e9cc83229234aee23cc commit: ed082383272c2c238e364e9cc83229234aee23cc branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T13:13:59Z summary: gh-107237: Fix test_udp_reconnection() of test_logging (#107238) test_logging: Fix test_udp_reconnection() by increasing the timeout from 100 ms to 5 minutes (LONG_TIMEOUT). Replace also blocking wait() with wait(LONG_TIMEOUT) in test_output() to prevent the test to hang. files: A Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst M Lib/test/test_logging.py diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index 18258c22874ae..def976fbe96ba 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -2079,17 +2079,17 @@ def test_output(self): # The log message sent to the SysLogHandler is properly received. logger = logging.getLogger("slh") logger.error("sp\xe4m") - self.handled.wait() + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>sp\xc3\xa4m\x00') self.handled.clear() self.sl_hdlr.append_nul = False logger.error("sp\xe4m") - self.handled.wait() + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>sp\xc3\xa4m') self.handled.clear() self.sl_hdlr.ident = "h\xe4m-" logger.error("sp\xe4m") - self.handled.wait() + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>h\xc3\xa4m-sp\xc3\xa4m') def test_udp_reconnection(self): @@ -2097,7 +2097,7 @@ def test_udp_reconnection(self): self.sl_hdlr.close() self.handled.clear() logger.error("sp\xe4m") - self.handled.wait(0.1) + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>sp\xc3\xa4m\x00') @unittest.skipUnless(hasattr(socket, "AF_UNIX"), "Unix sockets required") diff --git a/Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst b/Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst new file mode 100644 index 0000000000000..a04f7eeddef17 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst @@ -0,0 +1,2 @@ +``test_logging``: Fix ``test_udp_reconnection()`` by increasing the timeout +from 100 ms to 5 minutes (LONG_TIMEOUT). Patch by Victor Stinner. From webhook-mailer at python.org Tue Jul 25 09:28:35 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 13:28:35 -0000 Subject: [Python-checkins] gh-106869: Use new PyMemberDef constant names (#106871) Message-ID: https://github.com/python/cpython/commit/1a3faba9f15e0c03c6cc0d225d377b8910b5379f commit: 1a3faba9f15e0c03c6cc0d225d377b8910b5379f branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T15:28:30+02:00 summary: gh-106869: Use new PyMemberDef constant names (#106871) * Remove '#include "structmember.h"'. * If needed, add to get offsetof() function. * Update Parser/asdl_c.py to regenerate Python/Python-ast.c. * Replace: * T_SHORT => Py_T_SHORT * T_INT => Py_T_INT * T_LONG => Py_T_LONG * T_FLOAT => Py_T_FLOAT * T_DOUBLE => Py_T_DOUBLE * T_STRING => Py_T_STRING * T_OBJECT => _Py_T_OBJECT * T_CHAR => Py_T_CHAR * T_BYTE => Py_T_BYTE * T_UBYTE => Py_T_UBYTE * T_USHORT => Py_T_USHORT * T_UINT => Py_T_UINT * T_ULONG => Py_T_ULONG * T_STRING_INPLACE => Py_T_STRING_INPLACE * T_BOOL => Py_T_BOOL * T_OBJECT_EX => Py_T_OBJECT_EX * T_LONGLONG => Py_T_LONGLONG * T_ULONGLONG => Py_T_ULONGLONG * T_PYSSIZET => Py_T_PYSSIZET * T_NONE => _Py_T_NONE * READONLY => Py_READONLY * PY_AUDIT_READ => Py_AUDIT_READ * READ_RESTRICTED => Py_AUDIT_READ * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED) files: M Include/internal/pycore_frame.h M Modules/_asynciomodule.c M Modules/_bz2module.c M Modules/_collectionsmodule.c M Modules/_csv.c M Modules/_ctypes/_ctypes.c M Modules/_ctypes/callproc.c M Modules/_datetimemodule.c M Modules/_elementtree.c M Modules/_functoolsmodule.c M Modules/_io/bufferedio.c M Modules/_io/bytesio.c M Modules/_io/fileio.c M Modules/_io/iobase.c M Modules/_io/stringio.c M Modules/_io/textio.c M Modules/_io/winconsoleio.c M Modules/_json.c M Modules/_lzmamodule.c M Modules/_multiprocessing/semaphore.c M Modules/_operator.c M Modules/_pickle.c M Modules/_queuemodule.c M Modules/_sqlite/blob.c M Modules/_sqlite/connection.c M Modules/_sqlite/cursor.c M Modules/_sre/sre.c M Modules/_struct.c M Modules/_testcapi/buffer.c M Modules/_testcapi/heaptype.c M Modules/_testcapi/vectorcall.c M Modules/_testcapi/vectorcall_limited.c M Modules/_testcapimodule.c M Modules/_threadmodule.c M Modules/_winapi.c M Modules/_zoneinfo.c M Modules/arraymodule.c M Modules/cjkcodecs/multibytecodec.c M Modules/itertoolsmodule.c M Modules/mmapmodule.c M Modules/overlapped.c M Modules/posixmodule.c M Modules/pyexpat.c M Modules/selectmodule.c M Modules/sha2module.c M Modules/socketmodule.c M Modules/unicodedata.c M Modules/xxsubtype.c M Modules/zlibmodule.c M Objects/classobject.c M Objects/codeobject.c M Objects/complexobject.c M Objects/descrobject.c M Objects/exceptions.c M Objects/frameobject.c M Objects/funcobject.c M Objects/genericaliasobject.c M Objects/genobject.c M Objects/methodobject.c M Objects/moduleobject.c M Objects/namespaceobject.c M Objects/rangeobject.c M Objects/sliceobject.c M Objects/structseq.c M Objects/typeobject.c M Objects/typevarobject.c M Objects/unionobject.c M Objects/weakrefobject.c M PC/winreg.c M Parser/asdl_c.py M Python/Python-ast.c M Python/bytecodes.c M Python/ceval.c M Python/context.c M Python/specialize.c M Python/structmember.c M Python/symtable.c M Python/traceback.c diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h index efc19e33ec5dc..5ff20ef845ab1 100644 --- a/Include/internal/pycore_frame.h +++ b/Include/internal/pycore_frame.h @@ -5,8 +5,8 @@ extern "C" { #endif #include -#include -#include "pycore_code.h" // STATS +#include // offsetof() +#include "pycore_code.h" // STATS /* See Objects/frame_layout.md for an explanation of the frame stack * including explanation of the PyFrameObject and _PyInterpreterFrame diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index ef9f7f8902e09..f5a589b00c48d 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -8,7 +8,7 @@ #include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_runtime_init.h" // _Py_ID() -#include "structmember.h" // PyMemberDef + #include // offsetof() diff --git a/Modules/_bz2module.c b/Modules/_bz2module.c index eeefe6034998c..0a84f25ca4cbe 100644 --- a/Modules/_bz2module.c +++ b/Modules/_bz2module.c @@ -1,10 +1,10 @@ /* _bz2 - Low-level Python interface to libbzip2. */ #include "Python.h" -#include "structmember.h" // PyMemberDef #include #include +#include // offsetof() // Blocks output buffer wrappers #include "pycore_blocks_output_buffer.h" @@ -112,7 +112,7 @@ typedef struct { typedef struct { PyObject_HEAD bz_stream bzs; - char eof; /* T_BOOL expects a char */ + char eof; /* Py_T_BOOL expects a char */ PyObject *unused_data; char needs_input; char *input_buffer; @@ -714,11 +714,11 @@ PyDoc_STRVAR(BZ2Decompressor_needs_input_doc, "True if more input is needed before more decompressed data can be produced."); static PyMemberDef BZ2Decompressor_members[] = { - {"eof", T_BOOL, offsetof(BZ2Decompressor, eof), - READONLY, BZ2Decompressor_eof__doc__}, - {"unused_data", T_OBJECT_EX, offsetof(BZ2Decompressor, unused_data), - READONLY, BZ2Decompressor_unused_data__doc__}, - {"needs_input", T_BOOL, offsetof(BZ2Decompressor, needs_input), READONLY, + {"eof", Py_T_BOOL, offsetof(BZ2Decompressor, eof), + Py_READONLY, BZ2Decompressor_eof__doc__}, + {"unused_data", Py_T_OBJECT_EX, offsetof(BZ2Decompressor, unused_data), + Py_READONLY, BZ2Decompressor_unused_data__doc__}, + {"needs_input", Py_T_BOOL, offsetof(BZ2Decompressor, needs_input), Py_READONLY, BZ2Decompressor_needs_input_doc}, {NULL} }; diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c index 9a81531bdffb1..f2915f83b9d96 100644 --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -3,7 +3,7 @@ #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_typeobject.h" // _PyType_GetModuleState() -#include "structmember.h" // PyMemberDef + #include typedef struct { @@ -1630,7 +1630,7 @@ static PyMethodDef deque_methods[] = { }; static PyMemberDef deque_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(dequeobject, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(dequeobject, weakreflist), Py_READONLY}, {NULL}, }; @@ -2054,7 +2054,7 @@ static PyMethodDef defdict_methods[] = { }; static PyMemberDef defdict_members[] = { - {"default_factory", T_OBJECT, + {"default_factory", _Py_T_OBJECT, offsetof(defdictobject, default_factory), 0, PyDoc_STR("Factory for default value called by __missing__().")}, {NULL} @@ -2466,7 +2466,7 @@ tuplegetter_repr(_tuplegetterobject *self) static PyMemberDef tuplegetter_members[] = { - {"__doc__", T_OBJECT, offsetof(_tuplegetterobject, doc), 0}, + {"__doc__", _Py_T_OBJECT, offsetof(_tuplegetterobject, doc), 0}, {0} }; diff --git a/Modules/_csv.c b/Modules/_csv.c index c36d9805a1284..24a57e362521d 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -11,7 +11,8 @@ module instead. #define MODULE_VERSION "1.0" #include "Python.h" -#include "structmember.h" // PyMemberDef + +#include // offsetof() #include /*[clinic input] @@ -336,9 +337,9 @@ dialect_check_quoting(int quoting) #define D_OFF(x) offsetof(DialectObj, x) static struct PyMemberDef Dialect_memberlist[] = { - { "skipinitialspace", T_BOOL, D_OFF(skipinitialspace), READONLY }, - { "doublequote", T_BOOL, D_OFF(doublequote), READONLY }, - { "strict", T_BOOL, D_OFF(strict), READONLY }, + { "skipinitialspace", Py_T_BOOL, D_OFF(skipinitialspace), Py_READONLY }, + { "doublequote", Py_T_BOOL, D_OFF(doublequote), Py_READONLY }, + { "strict", Py_T_BOOL, D_OFF(strict), Py_READONLY }, { NULL } }; @@ -970,8 +971,8 @@ static struct PyMethodDef Reader_methods[] = { #define R_OFF(x) offsetof(ReaderObj, x) static struct PyMemberDef Reader_memberlist[] = { - { "dialect", T_OBJECT, R_OFF(dialect), READONLY }, - { "line_num", T_ULONG, R_OFF(line_num), READONLY }, + { "dialect", _Py_T_OBJECT, R_OFF(dialect), Py_READONLY }, + { "line_num", Py_T_ULONG, R_OFF(line_num), Py_READONLY }, { NULL } }; @@ -1364,7 +1365,7 @@ static struct PyMethodDef Writer_methods[] = { #define W_OFF(x) offsetof(WriterObj, x) static struct PyMemberDef Writer_memberlist[] = { - { "dialect", T_OBJECT, W_OFF(dialect), READONLY }, + { "dialect", _Py_T_OBJECT, W_OFF(dialect), Py_READONLY }, { NULL } }; diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index 200fd36748c40..c20d6ae55a06e 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -110,7 +110,7 @@ bytes(cdata) #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_ceval.h" // _Py_EnterRecursiveCall() -#include "structmember.h" // PyMemberDef + #include #ifdef MS_WIN32 @@ -2759,14 +2759,14 @@ PyCData_dealloc(PyObject *self) } static PyMemberDef PyCData_members[] = { - { "_b_base_", T_OBJECT, - offsetof(CDataObject, b_base), READONLY, + { "_b_base_", _Py_T_OBJECT, + offsetof(CDataObject, b_base), Py_READONLY, "the base object" }, - { "_b_needsfree_", T_INT, - offsetof(CDataObject, b_needsfree), READONLY, + { "_b_needsfree_", Py_T_INT, + offsetof(CDataObject, b_needsfree), Py_READONLY, "whether the object owns the memory or not" }, - { "_objects", T_OBJECT, - offsetof(CDataObject, b_objects), READONLY, + { "_objects", _Py_T_OBJECT, + offsetof(CDataObject, b_objects), Py_READONLY, "internal objects tree (NEVER CHANGE THIS OBJECT!)"}, { NULL }, }; diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index b3831ae7119a5..69cf8a98af663 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -59,7 +59,7 @@ #endif #include "Python.h" -#include "structmember.h" // PyMemberDef + #include @@ -581,8 +581,8 @@ PyCArg_repr(PyCArgObject *self) } static PyMemberDef PyCArgType_members[] = { - { "_obj", T_OBJECT, - offsetof(PyCArgObject, obj), READONLY, + { "_obj", _Py_T_OBJECT, + offsetof(PyCArgObject, obj), Py_READONLY, "the wrapped object" }, { NULL }, }; diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index e0e34b78d5d84..9002a1de7fb5b 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -15,7 +15,7 @@ #include "pycore_long.h" // _PyLong_GetOne() #include "pycore_object.h" // _PyObject_Init() #include "datetime.h" -#include "structmember.h" // PyMemberDef + #include @@ -2727,13 +2727,13 @@ delta_reduce(PyDateTime_Delta* self, PyObject *Py_UNUSED(ignored)) static PyMemberDef delta_members[] = { - {"days", T_INT, OFFSET(days), READONLY, + {"days", Py_T_INT, OFFSET(days), Py_READONLY, PyDoc_STR("Number of days.")}, - {"seconds", T_INT, OFFSET(seconds), READONLY, + {"seconds", Py_T_INT, OFFSET(seconds), Py_READONLY, PyDoc_STR("Number of seconds (>= 0 and less than 1 day).")}, - {"microseconds", T_INT, OFFSET(microseconds), READONLY, + {"microseconds", Py_T_INT, OFFSET(microseconds), Py_READONLY, PyDoc_STR("Number of microseconds (>= 0 and less than 1 second).")}, {NULL} }; diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 30327e0b6fc31..8cb57e693d81d 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -18,7 +18,8 @@ #include "Python.h" #include "pycore_import.h" // _PyImport_GetModuleAttrString() #include "pycore_pyhash.h" // _Py_HashSecret -#include "structmember.h" // PyMemberDef + +#include // offsetof() #include "expat.h" #include "pyexpat.h" @@ -4134,8 +4135,8 @@ _elementtree_XMLParser__setevents_impl(XMLParserObject *self, } static PyMemberDef xmlparser_members[] = { - {"entity", T_OBJECT, offsetof(XMLParserObject, entity), READONLY, NULL}, - {"target", T_OBJECT, offsetof(XMLParserObject, target), READONLY, NULL}, + {"entity", _Py_T_OBJECT, offsetof(XMLParserObject, entity), Py_READONLY, NULL}, + {"target", _Py_T_OBJECT, offsetof(XMLParserObject, target), Py_READONLY, NULL}, {NULL} }; @@ -4191,7 +4192,7 @@ static PyMethodDef element_methods[] = { }; static struct PyMemberDef element_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(ElementObject, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(ElementObject, weakreflist), Py_READONLY}, {NULL}, }; diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c index c987485e66a48..389ff4391de0b 100644 --- a/Modules/_functoolsmodule.c +++ b/Modules/_functoolsmodule.c @@ -6,7 +6,7 @@ #include "pycore_object.h" // _PyObject_GC_TRACK #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_tuple.h" // _PyTuple_ITEMS() -#include "structmember.h" // PyMemberDef + #include "clinic/_functoolsmodule.c.h" /*[clinic input] @@ -340,18 +340,18 @@ PyDoc_STRVAR(partial_doc, #define OFF(x) offsetof(partialobject, x) static PyMemberDef partial_memberlist[] = { - {"func", T_OBJECT, OFF(fn), READONLY, + {"func", _Py_T_OBJECT, OFF(fn), Py_READONLY, "function object to use in future partial calls"}, - {"args", T_OBJECT, OFF(args), READONLY, + {"args", _Py_T_OBJECT, OFF(args), Py_READONLY, "tuple of arguments to future partial calls"}, - {"keywords", T_OBJECT, OFF(kw), READONLY, + {"keywords", _Py_T_OBJECT, OFF(kw), Py_READONLY, "dictionary of keyword arguments to future partial calls"}, - {"__weaklistoffset__", T_PYSSIZET, - offsetof(partialobject, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, - offsetof(partialobject, dict), READONLY}, - {"__vectorcalloffset__", T_PYSSIZET, - offsetof(partialobject, vectorcall), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, + offsetof(partialobject, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, + offsetof(partialobject, dict), Py_READONLY}, + {"__vectorcalloffset__", Py_T_PYSSIZET, + offsetof(partialobject, vectorcall), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -540,7 +540,7 @@ keyobject_traverse(keyobject *ko, visitproc visit, void *arg) } static PyMemberDef keyobject_members[] = { - {"obj", T_OBJECT, + {"obj", _Py_T_OBJECT, offsetof(keyobject, object), 0, PyDoc_STR("Value wrapped by a key function.")}, {NULL} @@ -1394,10 +1394,10 @@ static PyGetSetDef lru_cache_getsetlist[] = { }; static PyMemberDef lru_cache_memberlist[] = { - {"__dictoffset__", T_PYSSIZET, - offsetof(lru_cache_object, dict), READONLY}, - {"__weaklistoffset__", T_PYSSIZET, - offsetof(lru_cache_object, weakreflist), READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, + offsetof(lru_cache_object, dict), Py_READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, + offsetof(lru_cache_object, weakreflist), Py_READONLY}, {NULL} /* Sentinel */ }; diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c index efc8cb93d8851..0983a7bd151f4 100644 --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -13,7 +13,7 @@ #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "pycore_pyerrors.h" // _Py_FatalErrorFormat() #include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing() -#include "structmember.h" // PyMemberDef + #include "_iomodule.h" /*[clinic input] @@ -2478,10 +2478,10 @@ static PyMethodDef bufferedreader_methods[] = { }; static PyMemberDef bufferedreader_members[] = { - {"raw", T_OBJECT, offsetof(buffered, raw), READONLY}, - {"_finalizing", T_BOOL, offsetof(buffered, finalizing), 0}, - {"__weaklistoffset__", T_PYSSIZET, offsetof(buffered, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(buffered, dict), READONLY}, + {"raw", _Py_T_OBJECT, offsetof(buffered, raw), Py_READONLY}, + {"_finalizing", Py_T_BOOL, offsetof(buffered, finalizing), 0}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(buffered, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(buffered, dict), Py_READONLY}, {NULL} }; @@ -2538,10 +2538,10 @@ static PyMethodDef bufferedwriter_methods[] = { }; static PyMemberDef bufferedwriter_members[] = { - {"raw", T_OBJECT, offsetof(buffered, raw), READONLY}, - {"_finalizing", T_BOOL, offsetof(buffered, finalizing), 0}, - {"__weaklistoffset__", T_PYSSIZET, offsetof(buffered, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(buffered, dict), READONLY}, + {"raw", _Py_T_OBJECT, offsetof(buffered, raw), Py_READONLY}, + {"_finalizing", Py_T_BOOL, offsetof(buffered, finalizing), 0}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(buffered, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(buffered, dict), Py_READONLY}, {NULL} }; @@ -2594,8 +2594,8 @@ static PyMethodDef bufferedrwpair_methods[] = { }; static PyMemberDef bufferedrwpair_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(rwpair, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(rwpair, dict), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(rwpair, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(rwpair, dict), Py_READONLY}, {NULL} }; @@ -2656,10 +2656,10 @@ static PyMethodDef bufferedrandom_methods[] = { }; static PyMemberDef bufferedrandom_members[] = { - {"raw", T_OBJECT, offsetof(buffered, raw), READONLY}, - {"_finalizing", T_BOOL, offsetof(buffered, finalizing), 0}, - {"__weaklistoffset__", T_PYSSIZET, offsetof(buffered, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(buffered, dict), READONLY}, + {"raw", _Py_T_OBJECT, offsetof(buffered, raw), Py_READONLY}, + {"_finalizing", Py_T_BOOL, offsetof(buffered, finalizing), 0}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(buffered, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(buffered, dict), Py_READONLY}, {NULL} }; diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c index 8077305869325..3ab503c9e3998 100644 --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -1028,8 +1028,8 @@ static struct PyMethodDef bytesio_methods[] = { }; static PyMemberDef bytesio_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(bytesio, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(bytesio, dict), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(bytesio, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(bytesio, dict), Py_READONLY}, {NULL} }; diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c index 39709fd293131..7fe37eee787e5 100644 --- a/Modules/_io/fileio.c +++ b/Modules/_io/fileio.c @@ -3,7 +3,7 @@ #include "Python.h" #include "pycore_fileutils.h" // _Py_BEGIN_SUPPRESS_IPH #include "pycore_object.h" // _PyObject_GC_UNTRACK() -#include "structmember.h" // PyMemberDef + #include #ifdef HAVE_SYS_TYPES_H #include @@ -1199,10 +1199,10 @@ static PyGetSetDef fileio_getsetlist[] = { }; static PyMemberDef fileio_members[] = { - {"_blksize", T_UINT, offsetof(fileio, blksize), 0}, - {"_finalizing", T_BOOL, offsetof(fileio, finalizing), 0}, - {"__weaklistoffset__", T_PYSSIZET, offsetof(fileio, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(fileio, dict), READONLY}, + {"_blksize", Py_T_UINT, offsetof(fileio, blksize), 0}, + {"_finalizing", Py_T_BOOL, offsetof(fileio, finalizing), 0}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(fileio, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(fileio, dict), Py_READONLY}, {NULL} }; diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c index e2e8ef46adf90..5fd19895311c0 100644 --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -863,8 +863,8 @@ static PyGetSetDef iobase_getset[] = { }; static struct PyMemberDef iobase_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(iobase, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(iobase, dict), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(iobase, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(iobase, dict), Py_READONLY}, {NULL}, }; diff --git a/Modules/_io/stringio.c b/Modules/_io/stringio.c index 1960002d405ed..1856b07108bab 100644 --- a/Modules/_io/stringio.c +++ b/Modules/_io/stringio.c @@ -1002,8 +1002,8 @@ static PyGetSetDef stringio_getset[] = { }; static struct PyMemberDef stringio_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(stringio, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(stringio, dict), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(stringio, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(stringio, dict), Py_READONLY}, {NULL}, }; diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c index a5cf9fc397f5f..24d846e663437 100644 --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -14,7 +14,7 @@ #include "pycore_fileutils.h" // _Py_GetLocaleEncoding() #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "pycore_pystate.h" // _PyInterpreterState_GET() -#include "structmember.h" // PyMemberDef + #include "_iomodule.h" /*[clinic input] @@ -3230,13 +3230,13 @@ static PyMethodDef textiowrapper_methods[] = { }; static PyMemberDef textiowrapper_members[] = { - {"encoding", T_OBJECT, offsetof(textio, encoding), READONLY}, - {"buffer", T_OBJECT, offsetof(textio, buffer), READONLY}, - {"line_buffering", T_BOOL, offsetof(textio, line_buffering), READONLY}, - {"write_through", T_BOOL, offsetof(textio, write_through), READONLY}, - {"_finalizing", T_BOOL, offsetof(textio, finalizing), 0}, - {"__weaklistoffset__", T_PYSSIZET, offsetof(textio, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(textio, dict), READONLY}, + {"encoding", _Py_T_OBJECT, offsetof(textio, encoding), Py_READONLY}, + {"buffer", _Py_T_OBJECT, offsetof(textio, buffer), Py_READONLY}, + {"line_buffering", Py_T_BOOL, offsetof(textio, line_buffering), Py_READONLY}, + {"write_through", Py_T_BOOL, offsetof(textio, write_through), Py_READONLY}, + {"_finalizing", Py_T_BOOL, offsetof(textio, finalizing), 0}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(textio, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(textio, dict), Py_READONLY}, {NULL} }; diff --git a/Modules/_io/winconsoleio.c b/Modules/_io/winconsoleio.c index 452b12c138fa8..a1ed7eb61e47b 100644 --- a/Modules/_io/winconsoleio.c +++ b/Modules/_io/winconsoleio.c @@ -12,7 +12,7 @@ #ifdef HAVE_WINDOWS_CONSOLE_IO -#include "structmember.h" // PyMemberDef + #ifdef HAVE_SYS_TYPES_H #include #endif @@ -1141,10 +1141,10 @@ static PyGetSetDef winconsoleio_getsetlist[] = { }; static PyMemberDef winconsoleio_members[] = { - {"_blksize", T_UINT, offsetof(winconsoleio, blksize), 0}, - {"_finalizing", T_BOOL, offsetof(winconsoleio, finalizing), 0}, - {"__weaklistoffset__", T_PYSSIZET, offsetof(winconsoleio, weakreflist), READONLY}, - {"__dictoffset__", T_PYSSIZET, offsetof(winconsoleio, dict), READONLY}, + {"_blksize", Py_T_UINT, offsetof(winconsoleio, blksize), 0}, + {"_finalizing", Py_T_BOOL, offsetof(winconsoleio, finalizing), 0}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(winconsoleio, weakreflist), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(winconsoleio, dict), Py_READONLY}, {NULL} }; diff --git a/Modules/_json.c b/Modules/_json.c index 2d0e30d70932b..4fcaa07d9cfd8 100644 --- a/Modules/_json.c +++ b/Modules/_json.c @@ -11,7 +11,7 @@ #include "Python.h" #include "pycore_ceval.h" // _Py_EnterRecursiveCall() #include "pycore_runtime.h" // _PyRuntime -#include "structmember.h" // PyMemberDef + #include "pycore_global_objects.h" // _Py_ID() #include // bool @@ -28,12 +28,12 @@ typedef struct _PyScannerObject { } PyScannerObject; static PyMemberDef scanner_members[] = { - {"strict", T_BOOL, offsetof(PyScannerObject, strict), READONLY, "strict"}, - {"object_hook", T_OBJECT, offsetof(PyScannerObject, object_hook), READONLY, "object_hook"}, - {"object_pairs_hook", T_OBJECT, offsetof(PyScannerObject, object_pairs_hook), READONLY}, - {"parse_float", T_OBJECT, offsetof(PyScannerObject, parse_float), READONLY, "parse_float"}, - {"parse_int", T_OBJECT, offsetof(PyScannerObject, parse_int), READONLY, "parse_int"}, - {"parse_constant", T_OBJECT, offsetof(PyScannerObject, parse_constant), READONLY, "parse_constant"}, + {"strict", Py_T_BOOL, offsetof(PyScannerObject, strict), Py_READONLY, "strict"}, + {"object_hook", _Py_T_OBJECT, offsetof(PyScannerObject, object_hook), Py_READONLY, "object_hook"}, + {"object_pairs_hook", _Py_T_OBJECT, offsetof(PyScannerObject, object_pairs_hook), Py_READONLY}, + {"parse_float", _Py_T_OBJECT, offsetof(PyScannerObject, parse_float), Py_READONLY, "parse_float"}, + {"parse_int", _Py_T_OBJECT, offsetof(PyScannerObject, parse_int), Py_READONLY, "parse_int"}, + {"parse_constant", _Py_T_OBJECT, offsetof(PyScannerObject, parse_constant), Py_READONLY, "parse_constant"}, {NULL} }; @@ -52,14 +52,14 @@ typedef struct _PyEncoderObject { } PyEncoderObject; static PyMemberDef encoder_members[] = { - {"markers", T_OBJECT, offsetof(PyEncoderObject, markers), READONLY, "markers"}, - {"default", T_OBJECT, offsetof(PyEncoderObject, defaultfn), READONLY, "default"}, - {"encoder", T_OBJECT, offsetof(PyEncoderObject, encoder), READONLY, "encoder"}, - {"indent", T_OBJECT, offsetof(PyEncoderObject, indent), READONLY, "indent"}, - {"key_separator", T_OBJECT, offsetof(PyEncoderObject, key_separator), READONLY, "key_separator"}, - {"item_separator", T_OBJECT, offsetof(PyEncoderObject, item_separator), READONLY, "item_separator"}, - {"sort_keys", T_BOOL, offsetof(PyEncoderObject, sort_keys), READONLY, "sort_keys"}, - {"skipkeys", T_BOOL, offsetof(PyEncoderObject, skipkeys), READONLY, "skipkeys"}, + {"markers", _Py_T_OBJECT, offsetof(PyEncoderObject, markers), Py_READONLY, "markers"}, + {"default", _Py_T_OBJECT, offsetof(PyEncoderObject, defaultfn), Py_READONLY, "default"}, + {"encoder", _Py_T_OBJECT, offsetof(PyEncoderObject, encoder), Py_READONLY, "encoder"}, + {"indent", _Py_T_OBJECT, offsetof(PyEncoderObject, indent), Py_READONLY, "indent"}, + {"key_separator", _Py_T_OBJECT, offsetof(PyEncoderObject, key_separator), Py_READONLY, "key_separator"}, + {"item_separator", _Py_T_OBJECT, offsetof(PyEncoderObject, item_separator), Py_READONLY, "item_separator"}, + {"sort_keys", Py_T_BOOL, offsetof(PyEncoderObject, sort_keys), Py_READONLY, "sort_keys"}, + {"skipkeys", Py_T_BOOL, offsetof(PyEncoderObject, skipkeys), Py_READONLY, "skipkeys"}, {NULL} }; diff --git a/Modules/_lzmamodule.c b/Modules/_lzmamodule.c index edb2e3673f0b2..c548f8fa3839e 100644 --- a/Modules/_lzmamodule.c +++ b/Modules/_lzmamodule.c @@ -6,7 +6,7 @@ */ #include "Python.h" -#include "structmember.h" // PyMemberDef + #include // free() #include @@ -1338,13 +1338,13 @@ PyDoc_STRVAR(Decompressor_unused_data_doc, "Data found after the end of the compressed stream."); static PyMemberDef Decompressor_members[] = { - {"check", T_INT, offsetof(Decompressor, check), READONLY, + {"check", Py_T_INT, offsetof(Decompressor, check), Py_READONLY, Decompressor_check_doc}, - {"eof", T_BOOL, offsetof(Decompressor, eof), READONLY, + {"eof", Py_T_BOOL, offsetof(Decompressor, eof), Py_READONLY, Decompressor_eof_doc}, - {"needs_input", T_BOOL, offsetof(Decompressor, needs_input), READONLY, + {"needs_input", Py_T_BOOL, offsetof(Decompressor, needs_input), Py_READONLY, Decompressor_needs_input_doc}, - {"unused_data", T_OBJECT_EX, offsetof(Decompressor, unused_data), READONLY, + {"unused_data", Py_T_OBJECT_EX, offsetof(Decompressor, unused_data), Py_READONLY, Decompressor_unused_data_doc}, {NULL} }; diff --git a/Modules/_multiprocessing/semaphore.c b/Modules/_multiprocessing/semaphore.c index 897b8db7110a4..771f86e5367af 100644 --- a/Modules/_multiprocessing/semaphore.c +++ b/Modules/_multiprocessing/semaphore.c @@ -734,13 +734,13 @@ static PyMethodDef semlock_methods[] = { */ static PyMemberDef semlock_members[] = { - {"handle", T_SEM_HANDLE, offsetof(SemLockObject, handle), READONLY, + {"handle", T_SEM_HANDLE, offsetof(SemLockObject, handle), Py_READONLY, ""}, - {"kind", T_INT, offsetof(SemLockObject, kind), READONLY, + {"kind", Py_T_INT, offsetof(SemLockObject, kind), Py_READONLY, ""}, - {"maxvalue", T_INT, offsetof(SemLockObject, maxvalue), READONLY, + {"maxvalue", Py_T_INT, offsetof(SemLockObject, maxvalue), Py_READONLY, ""}, - {"name", T_STRING, offsetof(SemLockObject, name), READONLY, + {"name", Py_T_STRING, offsetof(SemLockObject, name), Py_READONLY, ""}, {NULL} }; diff --git a/Modules/_operator.c b/Modules/_operator.c index 108f45fb6dad9..b59bfe9ac3217 100644 --- a/Modules/_operator.c +++ b/Modules/_operator.c @@ -3,7 +3,7 @@ #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_runtime.h" // _Py_ID() -#include "structmember.h" // PyMemberDef + #include "clinic/_operator.c.h" typedef struct { @@ -1153,7 +1153,7 @@ static PyMethodDef itemgetter_methods[] = { }; static PyMemberDef itemgetter_members[] = { - {"__vectorcalloffset__", T_PYSSIZET, offsetof(itemgetterobject, vectorcall), READONLY}, + {"__vectorcalloffset__", Py_T_PYSSIZET, offsetof(itemgetterobject, vectorcall), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -1508,7 +1508,7 @@ static PyMethodDef attrgetter_methods[] = { }; static PyMemberDef attrgetter_members[] = { - {"__vectorcalloffset__", T_PYSSIZET, offsetof(attrgetterobject, vectorcall), READONLY}, + {"__vectorcalloffset__", Py_T_PYSSIZET, offsetof(attrgetterobject, vectorcall), Py_READONLY}, {NULL} /* Sentinel*/ }; diff --git a/Modules/_pickle.c b/Modules/_pickle.c index d9395e35daf8b..d4c0be7893523 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -16,7 +16,7 @@ #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_runtime.h" // _Py_ID() #include "pycore_setobject.h" // _PySet_NextEntry() -#include "structmember.h" // PyMemberDef + #include // strtol() @@ -5074,9 +5074,9 @@ Pickler_set_persid(PicklerObject *self, PyObject *value, void *Py_UNUSED(ignored } static PyMemberDef Pickler_members[] = { - {"bin", T_INT, offsetof(PicklerObject, bin)}, - {"fast", T_INT, offsetof(PicklerObject, fast)}, - {"dispatch_table", T_OBJECT_EX, offsetof(PicklerObject, dispatch_table)}, + {"bin", Py_T_INT, offsetof(PicklerObject, bin)}, + {"fast", Py_T_INT, offsetof(PicklerObject, fast)}, + {"dispatch_table", Py_T_OBJECT_EX, offsetof(PicklerObject, dispatch_table)}, {NULL} }; diff --git a/Modules/_queuemodule.c b/Modules/_queuemodule.c index 69cc05135c2a7..b0a36f0369450 100644 --- a/Modules/_queuemodule.c +++ b/Modules/_queuemodule.c @@ -5,7 +5,7 @@ #include "Python.h" #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_time.h" // _PyTime_t -#include "structmember.h" // PyMemberDef + #include // offsetof() typedef struct { @@ -373,7 +373,7 @@ static PyMethodDef simplequeue_methods[] = { }; static struct PyMemberDef simplequeue_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(simplequeueobject, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(simplequeueobject, weakreflist), Py_READONLY}, {NULL}, }; diff --git a/Modules/_sqlite/blob.c b/Modules/_sqlite/blob.c index 989d9a83b590c..f099020c5f4e6 100644 --- a/Modules/_sqlite/blob.c +++ b/Modules/_sqlite/blob.c @@ -577,7 +577,7 @@ static PyMethodDef blob_methods[] = { }; static struct PyMemberDef blob_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(pysqlite_Blob, in_weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(pysqlite_Blob, in_weakreflist), Py_READONLY}, {NULL}, }; diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index bab743674b666..ddd7ace81198b 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -26,7 +26,7 @@ #endif #include "module.h" -#include "structmember.h" // PyMemberDef + #include "connection.h" #include "statement.h" #include "cursor.h" @@ -2511,18 +2511,18 @@ static PyMethodDef connection_methods[] = { static struct PyMemberDef connection_members[] = { - {"Warning", T_OBJECT, offsetof(pysqlite_Connection, Warning), READONLY}, - {"Error", T_OBJECT, offsetof(pysqlite_Connection, Error), READONLY}, - {"InterfaceError", T_OBJECT, offsetof(pysqlite_Connection, InterfaceError), READONLY}, - {"DatabaseError", T_OBJECT, offsetof(pysqlite_Connection, DatabaseError), READONLY}, - {"DataError", T_OBJECT, offsetof(pysqlite_Connection, DataError), READONLY}, - {"OperationalError", T_OBJECT, offsetof(pysqlite_Connection, OperationalError), READONLY}, - {"IntegrityError", T_OBJECT, offsetof(pysqlite_Connection, IntegrityError), READONLY}, - {"InternalError", T_OBJECT, offsetof(pysqlite_Connection, InternalError), READONLY}, - {"ProgrammingError", T_OBJECT, offsetof(pysqlite_Connection, ProgrammingError), READONLY}, - {"NotSupportedError", T_OBJECT, offsetof(pysqlite_Connection, NotSupportedError), READONLY}, - {"row_factory", T_OBJECT, offsetof(pysqlite_Connection, row_factory)}, - {"text_factory", T_OBJECT, offsetof(pysqlite_Connection, text_factory)}, + {"Warning", _Py_T_OBJECT, offsetof(pysqlite_Connection, Warning), Py_READONLY}, + {"Error", _Py_T_OBJECT, offsetof(pysqlite_Connection, Error), Py_READONLY}, + {"InterfaceError", _Py_T_OBJECT, offsetof(pysqlite_Connection, InterfaceError), Py_READONLY}, + {"DatabaseError", _Py_T_OBJECT, offsetof(pysqlite_Connection, DatabaseError), Py_READONLY}, + {"DataError", _Py_T_OBJECT, offsetof(pysqlite_Connection, DataError), Py_READONLY}, + {"OperationalError", _Py_T_OBJECT, offsetof(pysqlite_Connection, OperationalError), Py_READONLY}, + {"IntegrityError", _Py_T_OBJECT, offsetof(pysqlite_Connection, IntegrityError), Py_READONLY}, + {"InternalError", _Py_T_OBJECT, offsetof(pysqlite_Connection, InternalError), Py_READONLY}, + {"ProgrammingError", _Py_T_OBJECT, offsetof(pysqlite_Connection, ProgrammingError), Py_READONLY}, + {"NotSupportedError", _Py_T_OBJECT, offsetof(pysqlite_Connection, NotSupportedError), Py_READONLY}, + {"row_factory", _Py_T_OBJECT, offsetof(pysqlite_Connection, row_factory)}, + {"text_factory", _Py_T_OBJECT, offsetof(pysqlite_Connection, text_factory)}, {NULL} }; diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c index dba8ab61e41e7..618ce532b2518 100644 --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -1325,13 +1325,13 @@ static PyMethodDef cursor_methods[] = { static struct PyMemberDef cursor_members[] = { - {"connection", T_OBJECT, offsetof(pysqlite_Cursor, connection), READONLY}, - {"description", T_OBJECT, offsetof(pysqlite_Cursor, description), READONLY}, - {"arraysize", T_INT, offsetof(pysqlite_Cursor, arraysize), 0}, - {"lastrowid", T_OBJECT, offsetof(pysqlite_Cursor, lastrowid), READONLY}, - {"rowcount", T_LONG, offsetof(pysqlite_Cursor, rowcount), READONLY}, - {"row_factory", T_OBJECT, offsetof(pysqlite_Cursor, row_factory), 0}, - {"__weaklistoffset__", T_PYSSIZET, offsetof(pysqlite_Cursor, in_weakreflist), READONLY}, + {"connection", _Py_T_OBJECT, offsetof(pysqlite_Cursor, connection), Py_READONLY}, + {"description", _Py_T_OBJECT, offsetof(pysqlite_Cursor, description), Py_READONLY}, + {"arraysize", Py_T_INT, offsetof(pysqlite_Cursor, arraysize), 0}, + {"lastrowid", _Py_T_OBJECT, offsetof(pysqlite_Cursor, lastrowid), Py_READONLY}, + {"rowcount", Py_T_LONG, offsetof(pysqlite_Cursor, rowcount), Py_READONLY}, + {"row_factory", _Py_T_OBJECT, offsetof(pysqlite_Cursor, row_factory), 0}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(pysqlite_Cursor, in_weakreflist), Py_READONLY}, {NULL} }; diff --git a/Modules/_sre/sre.c b/Modules/_sre/sre.c index ddbdc9f478aab..c4e43a0db0f5d 100644 --- a/Modules/_sre/sre.c +++ b/Modules/_sre/sre.c @@ -41,7 +41,7 @@ static const char copyright[] = #include "Python.h" #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_moduleobject.h" // _PyModule_GetState() -#include "structmember.h" // PyMemberDef + #include "sre.h" @@ -2994,13 +2994,13 @@ static PyGetSetDef pattern_getset[] = { #define PAT_OFF(x) offsetof(PatternObject, x) static PyMemberDef pattern_members[] = { - {"pattern", T_OBJECT, PAT_OFF(pattern), READONLY, + {"pattern", _Py_T_OBJECT, PAT_OFF(pattern), Py_READONLY, "The pattern string from which the RE object was compiled."}, - {"flags", T_INT, PAT_OFF(flags), READONLY, + {"flags", Py_T_INT, PAT_OFF(flags), Py_READONLY, "The regex matching flags."}, - {"groups", T_PYSSIZET, PAT_OFF(groups), READONLY, + {"groups", Py_T_PYSSIZET, PAT_OFF(groups), Py_READONLY, "The number of capturing groups in the pattern."}, - {"__weaklistoffset__", T_PYSSIZET, offsetof(PatternObject, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(PatternObject, weakreflist), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -3053,13 +3053,13 @@ static PyGetSetDef match_getset[] = { #define MATCH_OFF(x) offsetof(MatchObject, x) static PyMemberDef match_members[] = { - {"string", T_OBJECT, MATCH_OFF(string), READONLY, + {"string", _Py_T_OBJECT, MATCH_OFF(string), Py_READONLY, "The string passed to match() or search()."}, - {"re", T_OBJECT, MATCH_OFF(pattern), READONLY, + {"re", _Py_T_OBJECT, MATCH_OFF(pattern), Py_READONLY, "The regular expression object."}, - {"pos", T_PYSSIZET, MATCH_OFF(pos), READONLY, + {"pos", Py_T_PYSSIZET, MATCH_OFF(pos), Py_READONLY, "The index into the string at which the RE engine started looking for a match."}, - {"endpos", T_PYSSIZET, MATCH_OFF(endpos), READONLY, + {"endpos", Py_T_PYSSIZET, MATCH_OFF(endpos), Py_READONLY, "The index into the string beyond which the RE engine will not go."}, {NULL} }; @@ -3103,7 +3103,7 @@ static PyMethodDef scanner_methods[] = { #define SCAN_OFF(x) offsetof(ScannerObject, x) static PyMemberDef scanner_members[] = { - {"pattern", T_OBJECT, SCAN_OFF(pattern), READONLY}, + {"pattern", _Py_T_OBJECT, SCAN_OFF(pattern), Py_READONLY}, {NULL} /* Sentinel */ }; diff --git a/Modules/_struct.c b/Modules/_struct.c index 31c94927e91d6..425715ad030d4 100644 --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -10,8 +10,9 @@ #include "Python.h" #include "pycore_bytesobject.h" // _PyBytesWriter #include "pycore_moduleobject.h" // _PyModule_GetState() -#include "structmember.h" // PyMemberDef + #include +#include // offsetof() /*[clinic input] class Struct "PyStructObject *" "&PyStructType" @@ -2176,7 +2177,7 @@ static struct PyMethodDef s_methods[] = { }; static PyMemberDef s_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(PyStructObject, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(PyStructObject, weakreflist), Py_READONLY}, {NULL} /* sentinel */ }; diff --git a/Modules/_testcapi/buffer.c b/Modules/_testcapi/buffer.c index aff9a477eff57..942774156c6c4 100644 --- a/Modules/_testcapi/buffer.c +++ b/Modules/_testcapi/buffer.c @@ -2,7 +2,7 @@ #include "parts.h" -#include "structmember.h" // PyMemberDef + #include // offsetof typedef struct { @@ -72,7 +72,7 @@ static PyBufferProcs testbuf_as_buffer = { }; static struct PyMemberDef testbuf_members[] = { - {"references", T_PYSSIZET, offsetof(testBufObject, references), READONLY}, + {"references", Py_T_PYSSIZET, offsetof(testBufObject, references), Py_READONLY}, {NULL}, }; diff --git a/Modules/_testcapi/heaptype.c b/Modules/_testcapi/heaptype.c index 2da06f447afaa..d14a176318420 100644 --- a/Modules/_testcapi/heaptype.c +++ b/Modules/_testcapi/heaptype.c @@ -1,5 +1,6 @@ #include "parts.h" -#include "structmember.h" // PyMemberDef +#include // offsetof() + static struct PyModuleDef *_testcapimodule = NULL; // set at initialization @@ -332,7 +333,7 @@ typedef struct { static struct PyMemberDef members_to_repeat[] = { - {"T_INT", T_INT, offsetof(HeapCTypeWithDataObject, data), 0, NULL}, + {"Py_T_INT", Py_T_INT, offsetof(HeapCTypeWithDataObject, data), 0, NULL}, {NULL} }; @@ -477,7 +478,7 @@ typedef struct { } HeapCTypeObject; static struct PyMemberDef heapctype_members[] = { - {"value", T_INT, offsetof(HeapCTypeObject, value)}, + {"value", Py_T_INT, offsetof(HeapCTypeObject, value)}, {NULL} /* Sentinel */ }; @@ -571,7 +572,7 @@ heapctypesubclass_init(PyObject *self, PyObject *args, PyObject *kwargs) } static struct PyMemberDef heapctypesubclass_members[] = { - {"value2", T_INT, offsetof(HeapCTypeSubclassObject, value2)}, + {"value2", Py_T_INT, offsetof(HeapCTypeSubclassObject, value2)}, {NULL} /* Sentinel */ }; @@ -772,8 +773,8 @@ static PyGetSetDef heapctypewithdict_getsetlist[] = { }; static struct PyMemberDef heapctypewithdict_members[] = { - {"dictobj", T_OBJECT, offsetof(HeapCTypeWithDictObject, dict)}, - {"__dictoffset__", T_PYSSIZET, offsetof(HeapCTypeWithDictObject, dict), READONLY}, + {"dictobj", _Py_T_OBJECT, offsetof(HeapCTypeWithDictObject, dict)}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(HeapCTypeWithDictObject, dict), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -867,8 +868,8 @@ static PyType_Spec HeapCTypeWithManagedWeakref_spec = { }; static struct PyMemberDef heapctypewithnegativedict_members[] = { - {"dictobj", T_OBJECT, offsetof(HeapCTypeWithDictObject, dict)}, - {"__dictoffset__", T_PYSSIZET, -(Py_ssize_t)sizeof(void*), READONLY}, + {"dictobj", _Py_T_OBJECT, offsetof(HeapCTypeWithDictObject, dict)}, + {"__dictoffset__", Py_T_PYSSIZET, -(Py_ssize_t)sizeof(void*), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -893,9 +894,9 @@ typedef struct { } HeapCTypeWithWeakrefObject; static struct PyMemberDef heapctypewithweakref_members[] = { - {"weakreflist", T_OBJECT, offsetof(HeapCTypeWithWeakrefObject, weakreflist)}, - {"__weaklistoffset__", T_PYSSIZET, - offsetof(HeapCTypeWithWeakrefObject, weakreflist), READONLY}, + {"weakreflist", _Py_T_OBJECT, offsetof(HeapCTypeWithWeakrefObject, weakreflist)}, + {"__weaklistoffset__", Py_T_PYSSIZET, + offsetof(HeapCTypeWithWeakrefObject, weakreflist), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -943,7 +944,7 @@ typedef struct { } HeapCTypeSetattrObject; static struct PyMemberDef heapctypesetattr_members[] = { - {"pvalue", T_LONG, offsetof(HeapCTypeSetattrObject, value)}, + {"pvalue", Py_T_LONG, offsetof(HeapCTypeSetattrObject, value)}, {NULL} /* Sentinel */ }; diff --git a/Modules/_testcapi/vectorcall.c b/Modules/_testcapi/vectorcall.c index 5ee468bd28c85..61c6e0f485f47 100644 --- a/Modules/_testcapi/vectorcall.c +++ b/Modules/_testcapi/vectorcall.c @@ -1,7 +1,7 @@ #include "parts.h" #include "clinic/vectorcall.c.h" -#include "structmember.h" // PyMemberDef + #include // offsetof /*[clinic input] @@ -197,7 +197,7 @@ PyMethodDef VectorCallClass_methods[] = { }; PyMemberDef VectorCallClass_members[] = { - {"__vectorcalloffset__", T_PYSSIZET, 0/* set later */, READONLY}, + {"__vectorcalloffset__", Py_T_PYSSIZET, 0/* set later */, Py_READONLY}, {NULL} }; diff --git a/Modules/_testcapi/vectorcall_limited.c b/Modules/_testcapi/vectorcall_limited.c index a69f1d3f2a79b..a96925e840121 100644 --- a/Modules/_testcapi/vectorcall_limited.c +++ b/Modules/_testcapi/vectorcall_limited.c @@ -3,7 +3,7 @@ #ifdef LIMITED_API_AVAILABLE -#include "structmember.h" // PyMemberDef + /* Test Vectorcall in the limited API */ @@ -132,7 +132,7 @@ call_vectorcall_method(PyObject* self, PyObject *callable) } static PyMemberDef LimitedVectorCallClass_members[] = { - {"__vectorcalloffset__", T_PYSSIZET, sizeof(PyObject), READONLY}, + {"__vectorcalloffset__", Py_T_PYSSIZET, sizeof(PyObject), Py_READONLY}, {NULL} }; diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 065a7fb733d43..2286a925e36e2 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -20,15 +20,16 @@ #include "Python.h" #include "frameobject.h" // PyFrame_New #include "marshal.h" // PyMarshal_WriteLongToFile -#include "structmember.h" // for offsetof(), T_OBJECT + #include // FLT_MAX #include +#include // offsetof() #ifndef MS_WINDOWS -#include +# include #endif #ifdef HAVE_SYS_WAIT_H -#include // W_STOPCODE +# include // W_STOPCODE #endif #ifdef Py_BUILD_CORE @@ -4257,7 +4258,7 @@ ContainerNoGC_dealloc(ContainerNoGCobject *self) } static PyMemberDef ContainerNoGC_members[] = { - {"value", T_OBJECT, offsetof(ContainerNoGCobject, value), READONLY, + {"value", _Py_T_OBJECT, offsetof(ContainerNoGCobject, value), Py_READONLY, PyDoc_STR("a container value for test purposes")}, {0} }; diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index d8a797f34dbc4..52f44d0452345 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -9,7 +9,7 @@ #include "pycore_pystate.h" // _PyThreadState_SetCurrent() #include "pycore_weakref.h" // _PyWeakref_GET_REF() #include // offsetof() -#include "structmember.h" // PyMemberDef + #ifdef HAVE_SIGNAL_H # include // SIGINT @@ -293,7 +293,7 @@ unlock it. A thread attempting to lock a lock that it has already locked\n\ will block until another thread unlocks it. Deadlocks may ensue."); static PyMemberDef lock_type_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(lockobject, in_weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(lockobject, in_weakreflist), Py_READONLY}, {NULL}, }; @@ -575,7 +575,7 @@ static PyMethodDef rlock_methods[] = { static PyMemberDef rlock_type_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(rlockobject, in_weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(rlockobject, in_weakreflist), Py_READONLY}, {NULL}, }; @@ -679,7 +679,7 @@ localdummy_dealloc(localdummyobject *self) } static PyMemberDef local_dummy_type_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(localdummyobject, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(localdummyobject, weakreflist), Py_READONLY}, {NULL}, }; @@ -959,7 +959,7 @@ local_setattro(localobject *self, PyObject *name, PyObject *v) static PyObject *local_getattro(localobject *, PyObject *); static PyMemberDef local_type_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(localobject, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(localobject, weakreflist), Py_READONLY}, {NULL}, }; diff --git a/Modules/_winapi.c b/Modules/_winapi.c index 313c12a34c672..0edb36d18dd3f 100644 --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -38,7 +38,7 @@ #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_pylifecycle.h" // _Py_IsInterpreterFinalizing() #include "pycore_pystate.h" // _PyInterpreterState_GET -#include "structmember.h" // PyMemberDef + #ifndef WINDOWS_LEAN_AND_MEAN @@ -54,13 +54,13 @@ PyLong_FromUnsignedLong((unsigned long) handle) #define PYNUM_TO_HANDLE(obj) ((HANDLE)PyLong_AsUnsignedLong(obj)) #define F_POINTER "k" -#define T_POINTER T_ULONG +#define T_POINTER Py_T_ULONG #else #define HANDLE_TO_PYNUM(handle) \ PyLong_FromUnsignedLongLong((unsigned long long) handle) #define PYNUM_TO_HANDLE(obj) ((HANDLE)PyLong_AsUnsignedLongLong(obj)) #define F_POINTER "K" -#define T_POINTER T_ULONGLONG +#define T_POINTER Py_T_ULONGLONG #endif #define F_HANDLE F_POINTER @@ -322,7 +322,7 @@ static PyMethodDef overlapped_methods[] = { static PyMemberDef overlapped_members[] = { {"event", T_HANDLE, offsetof(OverlappedObject, overlapped) + offsetof(OVERLAPPED, hEvent), - READONLY, "overlapped event handle"}, + Py_READONLY, "overlapped event handle"}, {NULL} }; diff --git a/Modules/_zoneinfo.c b/Modules/_zoneinfo.c index 0ced9d08b9eb7..fb0b4b40b2ad5 100644 --- a/Modules/_zoneinfo.c +++ b/Modules/_zoneinfo.c @@ -4,7 +4,7 @@ #include "Python.h" #include "pycore_long.h" // _PyLong_GetOne() -#include "structmember.h" + #include #include @@ -2692,13 +2692,13 @@ static PyMethodDef zoneinfo_methods[] = { static PyMemberDef zoneinfo_members[] = { {.name = "key", .offset = offsetof(PyZoneInfo_ZoneInfo, key), - .type = T_OBJECT_EX, - .flags = READONLY, + .type = Py_T_OBJECT_EX, + .flags = Py_READONLY, .doc = NULL}, {.name = "__weaklistoffset__", .offset = offsetof(PyZoneInfo_ZoneInfo, weakreflist), - .type = T_PYSSIZET, - .flags = READONLY}, + .type = Py_T_PYSSIZET, + .flags = Py_READONLY}, {NULL}, /* Sentinel */ }; diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c index 80a95428ebfd0..a40a5b75b63d4 100644 --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -11,7 +11,7 @@ #include "pycore_call.h" // _PyObject_CallMethod() #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_bytesobject.h" // _PyBytes_Repeat -#include "structmember.h" // PyMemberDef + #include // offsetof() #include @@ -2895,7 +2895,7 @@ itemsize -- the length in bytes of one array item\n\ static PyObject *array_iter(arrayobject *ao); static struct PyMemberDef array_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(arrayobject, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(arrayobject, weakreflist), Py_READONLY}, {NULL}, }; diff --git a/Modules/cjkcodecs/multibytecodec.c b/Modules/cjkcodecs/multibytecodec.c index 3febd1a832f9c..5d3c16a98423b 100644 --- a/Modules/cjkcodecs/multibytecodec.c +++ b/Modules/cjkcodecs/multibytecodec.c @@ -9,10 +9,12 @@ #endif #include "Python.h" -#include "structmember.h" // PyMemberDef + #include "multibytecodec.h" #include "clinic/multibytecodec.c.h" +#include // offsetof() + #define MODULE_NAME "_multibytecodec" typedef struct { @@ -1611,9 +1613,9 @@ static struct PyMethodDef mbstreamreader_methods[] = { }; static PyMemberDef mbstreamreader_members[] = { - {"stream", T_OBJECT, + {"stream", _Py_T_OBJECT, offsetof(MultibyteStreamReaderObject, stream), - READONLY, NULL}, + Py_READONLY, NULL}, {NULL,} }; @@ -1919,9 +1921,9 @@ static struct PyMethodDef mbstreamwriter_methods[] = { }; static PyMemberDef mbstreamwriter_members[] = { - {"stream", T_OBJECT, + {"stream", _Py_T_OBJECT, offsetof(MultibyteStreamWriterObject, stream), - READONLY, NULL}, + Py_READONLY, NULL}, {NULL,} }; diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index f5f7bf33bf8f4..0ab6d330e8779 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -5,7 +5,7 @@ #include "pycore_typeobject.h" // _PyType_GetModuleState() #include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_tuple.h" // _PyTuple_ITEMS() -#include "structmember.h" // PyMemberDef + #include // offsetof() /* Itertools module written and maintained @@ -1090,7 +1090,7 @@ static PyMethodDef tee_methods[] = { }; static PyMemberDef tee_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(teeobject, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(teeobject, weakreflist), Py_READONLY}, {NULL}, }; diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index cfbd2f41130ac..5c13157012356 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -25,7 +25,7 @@ #include #include "pycore_bytesobject.h" // _PyBytes_Find() #include "pycore_fileutils.h" // _Py_stat_struct -#include "structmember.h" // PyMemberDef + #include // offsetof() // to support MS_WINDOWS_SYSTEM OpenFileMappingA / CreateFileMappingA @@ -883,7 +883,7 @@ mmap_madvise_method(mmap_object *self, PyObject *args) #endif // HAVE_MADVISE static struct PyMemberDef mmap_object_members[] = { - {"__weaklistoffset__", T_PYSSIZET, offsetof(mmap_object, weakreflist), READONLY}, + {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(mmap_object, weakreflist), Py_READONLY}, {NULL}, }; diff --git a/Modules/overlapped.c b/Modules/overlapped.c index 7d3976faef96b..271f6ce7e2d91 100644 --- a/Modules/overlapped.c +++ b/Modules/overlapped.c @@ -8,7 +8,7 @@ Check itemsize */ #include "Python.h" -#include "structmember.h" // PyMemberDef + #define WINDOWS_LEAN_AND_MEAN #include @@ -17,10 +17,10 @@ #if defined(MS_WIN32) && !defined(MS_WIN64) # define F_POINTER "k" -# define T_POINTER T_ULONG +# define T_POINTER Py_T_ULONG #else # define F_POINTER "K" -# define T_POINTER T_ULONGLONG +# define T_POINTER Py_T_ULONGLONG #endif #define F_HANDLE F_POINTER @@ -1942,12 +1942,12 @@ static PyMethodDef Overlapped_methods[] = { }; static PyMemberDef Overlapped_members[] = { - {"error", T_ULONG, + {"error", Py_T_ULONG, offsetof(OverlappedObject, error), - READONLY, "Error from last operation"}, + Py_READONLY, "Error from last operation"}, {"event", T_HANDLE, offsetof(OverlappedObject, overlapped) + offsetof(OVERLAPPED, hEvent), - READONLY, "Overlapped event handle"}, + Py_READONLY, "Overlapped event handle"}, {NULL} }; diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 7e04d12a9e4e7..bab2ac2ccf59f 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -36,7 +36,7 @@ # endif /* MS_WINDOWS_DESKTOP | MS_WINDOWS_SYSTEM */ #endif -#include "structmember.h" // PyMemberDef + #ifndef MS_WINDOWS # include "posixmodule.h" #else @@ -14822,9 +14822,9 @@ os_DirEntry___fspath___impl(DirEntry *self) } static PyMemberDef DirEntry_members[] = { - {"name", T_OBJECT_EX, offsetof(DirEntry, name), READONLY, + {"name", Py_T_OBJECT_EX, offsetof(DirEntry, name), Py_READONLY, "the entry's base filename, relative to scandir() \"path\" argument"}, - {"path", T_OBJECT_EX, offsetof(DirEntry, path), READONLY, + {"path", Py_T_OBJECT_EX, offsetof(DirEntry, path), Py_READONLY, "the entry's full path name; equivalent to os.path.join(scandir_path, entry.name)"}, {NULL} }; diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index d8395e9a9aa80..a8ce84c0bb9f0 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -5,11 +5,10 @@ #include "Python.h" #include "pycore_import.h" // _PyImport_SetModule() #include "pycore_pyhash.h" // _Py_HashSecret -#include -#include "structmember.h" // PyMemberDef +#include +#include // offsetof() #include "expat.h" - #include "pyexpat.h" /* Do not emit Clinic output to a file as that wreaks havoc with conditionally @@ -1471,7 +1470,7 @@ xmlparse_specified_attributes_setter(xmlparseobject *self, PyObject *v, void *cl } static PyMemberDef xmlparse_members[] = { - {"intern", T_OBJECT, offsetof(xmlparseobject, intern), READONLY, NULL}, + {"intern", _Py_T_OBJECT, offsetof(xmlparseobject, intern), Py_READONLY, NULL}, {NULL} }; diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c index 7ab0804ad2723..94d246960f441 100644 --- a/Modules/selectmodule.c +++ b/Modules/selectmodule.c @@ -15,7 +15,8 @@ #include "Python.h" #include "pycore_fileutils.h" // _Py_set_inheritable() #include "pycore_time.h" // _PyTime_t -#include "structmember.h" // PyMemberDef + +#include // offsetof() #ifdef HAVE_SYS_DEVPOLL_H #include @@ -1758,18 +1759,18 @@ typedef struct { #if (SIZEOF_UINTPTR_T != SIZEOF_VOID_P) # error uintptr_t does not match void *! #elif (SIZEOF_UINTPTR_T == SIZEOF_LONG_LONG) -# define T_UINTPTRT T_ULONGLONG -# define T_INTPTRT T_LONGLONG +# define T_UINTPTRT Py_T_ULONGLONG +# define T_INTPTRT Py_T_LONGLONG # define UINTPTRT_FMT_UNIT "K" # define INTPTRT_FMT_UNIT "L" #elif (SIZEOF_UINTPTR_T == SIZEOF_LONG) -# define T_UINTPTRT T_ULONG -# define T_INTPTRT T_LONG +# define T_UINTPTRT Py_T_ULONG +# define T_INTPTRT Py_T_LONG # define UINTPTRT_FMT_UNIT "k" # define INTPTRT_FMT_UNIT "l" #elif (SIZEOF_UINTPTR_T == SIZEOF_INT) -# define T_UINTPTRT T_UINT -# define T_INTPTRT T_INT +# define T_UINTPTRT Py_T_UINT +# define T_INTPTRT Py_T_INT # define UINTPTRT_FMT_UNIT "I" # define INTPTRT_FMT_UNIT "i" #else @@ -1777,26 +1778,26 @@ typedef struct { #endif #if SIZEOF_LONG_LONG == 8 -# define T_INT64 T_LONGLONG +# define T_INT64 Py_T_LONGLONG # define INT64_FMT_UNIT "L" #elif SIZEOF_LONG == 8 -# define T_INT64 T_LONG +# define T_INT64 Py_T_LONG # define INT64_FMT_UNIT "l" #elif SIZEOF_INT == 8 -# define T_INT64 T_INT +# define T_INT64 Py_T_INT # define INT64_FMT_UNIT "i" #else # define INT64_FMT_UNIT "_" #endif #if SIZEOF_LONG_LONG == 4 -# define T_UINT32 T_ULONGLONG +# define T_UINT32 Py_T_ULONGLONG # define UINT32_FMT_UNIT "K" #elif SIZEOF_LONG == 4 -# define T_UINT32 T_ULONG +# define T_UINT32 Py_T_ULONG # define UINT32_FMT_UNIT "k" #elif SIZEOF_INT == 4 -# define T_UINT32 T_UINT +# define T_UINT32 Py_T_UINT # define UINT32_FMT_UNIT "I" #else # define UINT32_FMT_UNIT "_" @@ -1813,11 +1814,11 @@ typedef struct { # define FFLAGS_TYPE T_UINT32 # define FFLAGS_FMT_UNIT UINT32_FMT_UNIT #else -# define FILTER_TYPE T_SHORT +# define FILTER_TYPE Py_T_SHORT # define FILTER_FMT_UNIT "h" -# define FLAGS_TYPE T_USHORT +# define FLAGS_TYPE Py_T_USHORT # define FLAGS_FMT_UNIT "H" -# define FFLAGS_TYPE T_UINT +# define FFLAGS_TYPE Py_T_UINT # define FFLAGS_FMT_UNIT "I" #endif @@ -1839,7 +1840,7 @@ static struct PyMemberDef kqueue_event_members[] = { {"ident", T_UINTPTRT, KQ_OFF(e.ident)}, {"filter", FILTER_TYPE, KQ_OFF(e.filter)}, {"flags", FLAGS_TYPE, KQ_OFF(e.flags)}, - {"fflags", T_UINT, KQ_OFF(e.fflags)}, + {"fflags", Py_T_UINT, KQ_OFF(e.fflags)}, {"data", DATA_TYPE, KQ_OFF(e.data)}, {"udata", T_UINTPTRT, KQ_OFF(e.udata)}, {NULL} /* Sentinel */ diff --git a/Modules/sha2module.c b/Modules/sha2module.c index db3774c81e2d9..6ad1ff2e05bfd 100644 --- a/Modules/sha2module.c +++ b/Modules/sha2module.c @@ -25,7 +25,7 @@ #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_typeobject.h" // _PyType_GetModuleState() #include "pycore_strhex.h" // _Py_strhex() -#include "structmember.h" // PyMemberDef + #include "hashlib.h" /*[clinic input] diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 39bbc91171237..fc6a7f994bfd7 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -108,7 +108,7 @@ Local naming conventions: #include "Python.h" #include "pycore_fileutils.h" // _Py_set_inheritable() #include "pycore_moduleobject.h" // _PyModule_GetState -#include "structmember.h" // PyMemberDef + #ifdef _Py_MEMORY_SANITIZER # include @@ -5205,9 +5205,9 @@ static PyMethodDef sock_methods[] = { /* SockObject members */ static PyMemberDef sock_memberlist[] = { - {"family", T_INT, offsetof(PySocketSockObject, sock_family), READONLY, "the socket family"}, - {"type", T_INT, offsetof(PySocketSockObject, sock_type), READONLY, "the socket type"}, - {"proto", T_INT, offsetof(PySocketSockObject, sock_proto), READONLY, "the socket protocol"}, + {"family", Py_T_INT, offsetof(PySocketSockObject, sock_family), Py_READONLY, "the socket family"}, + {"type", Py_T_INT, offsetof(PySocketSockObject, sock_type), Py_READONLY, "the socket type"}, + {"proto", Py_T_INT, offsetof(PySocketSockObject, sock_proto), Py_READONLY, "the socket protocol"}, {0}, }; diff --git a/Modules/unicodedata.c b/Modules/unicodedata.c index 6267c5ae29ab3..c1e22f3868931 100644 --- a/Modules/unicodedata.c +++ b/Modules/unicodedata.c @@ -18,9 +18,9 @@ #include "Python.h" #include "pycore_ucnhash.h" // _PyUnicode_Name_CAPI -#include "structmember.h" // PyMemberDef #include +#include // offsetof() /*[clinic input] module unicodedata @@ -82,7 +82,7 @@ typedef struct previous_version { #define get_old_record(self, v) ((((PreviousDBVersion*)self)->getrecord)(v)) static PyMemberDef DB_members[] = { - {"unidata_version", T_STRING, offsetof(PreviousDBVersion, name), READONLY}, + {"unidata_version", Py_T_STRING, offsetof(PreviousDBVersion, name), Py_READONLY}, {NULL} }; diff --git a/Modules/xxsubtype.c b/Modules/xxsubtype.c index 9e4a3d66ef41b..63b22268c597b 100644 --- a/Modules/xxsubtype.c +++ b/Modules/xxsubtype.c @@ -1,5 +1,6 @@ #include "Python.h" -#include "structmember.h" // PyMemberDef +#include // offsetof() + PyDoc_STRVAR(xxsubtype__doc__, "xxsubtype is an example module showing how to subtype builtin types from C.\n" @@ -181,7 +182,7 @@ spamdict_init(spamdictobject *self, PyObject *args, PyObject *kwds) } static PyMemberDef spamdict_members[] = { - {"state", T_INT, offsetof(spamdictobject, state), READONLY, + {"state", Py_T_INT, offsetof(spamdictobject, state), Py_READONLY, PyDoc_STR("an int variable for demonstration purposes")}, {0} }; diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c index 9970bc9e5ad97..22da50989a88d 100644 --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -4,7 +4,7 @@ /* Windows users: read Python's PCbuild\readme.txt */ #include "Python.h" -#include "structmember.h" // PyMemberDef + #include "zlib.h" #include "stdbool.h" @@ -1344,7 +1344,7 @@ typedef struct { decompress_buf() */ Py_ssize_t avail_in_real; bool is_initialised; - char eof; /* T_BOOL expects a char */ + char eof; /* Py_T_BOOL expects a char */ char needs_input; } ZlibDecompressor; @@ -1801,9 +1801,9 @@ static PyMethodDef ZlibDecompressor_methods[] = { #define COMP_OFF(x) offsetof(compobject, x) static PyMemberDef Decomp_members[] = { - {"unused_data", T_OBJECT, COMP_OFF(unused_data), READONLY}, - {"unconsumed_tail", T_OBJECT, COMP_OFF(unconsumed_tail), READONLY}, - {"eof", T_BOOL, COMP_OFF(eof), READONLY}, + {"unused_data", _Py_T_OBJECT, COMP_OFF(unused_data), Py_READONLY}, + {"unconsumed_tail", _Py_T_OBJECT, COMP_OFF(unconsumed_tail), Py_READONLY}, + {"eof", Py_T_BOOL, COMP_OFF(eof), Py_READONLY}, {NULL}, }; @@ -1817,11 +1817,11 @@ PyDoc_STRVAR(ZlibDecompressor_needs_input_doc, "True if more input is needed before more decompressed data can be produced."); static PyMemberDef ZlibDecompressor_members[] = { - {"eof", T_BOOL, offsetof(ZlibDecompressor, eof), - READONLY, ZlibDecompressor_eof__doc__}, - {"unused_data", T_OBJECT_EX, offsetof(ZlibDecompressor, unused_data), - READONLY, ZlibDecompressor_unused_data__doc__}, - {"needs_input", T_BOOL, offsetof(ZlibDecompressor, needs_input), READONLY, + {"eof", Py_T_BOOL, offsetof(ZlibDecompressor, eof), + Py_READONLY, ZlibDecompressor_eof__doc__}, + {"unused_data", Py_T_OBJECT_EX, offsetof(ZlibDecompressor, unused_data), + Py_READONLY, ZlibDecompressor_unused_data__doc__}, + {"needs_input", Py_T_BOOL, offsetof(ZlibDecompressor, needs_input), Py_READONLY, ZlibDecompressor_needs_input_doc}, {NULL}, }; diff --git a/Objects/classobject.c b/Objects/classobject.c index 548b8672f8632..6f4457d42dda9 100644 --- a/Objects/classobject.c +++ b/Objects/classobject.c @@ -5,7 +5,7 @@ #include "pycore_object.h" #include "pycore_pyerrors.h" #include "pycore_pystate.h" // _PyThreadState_GET() -#include "structmember.h" // PyMemberDef + #include "clinic/classobject.c.h" @@ -150,9 +150,9 @@ static PyMethodDef method_methods[] = { #define MO_OFF(x) offsetof(PyMethodObject, x) static PyMemberDef method_memberlist[] = { - {"__func__", T_OBJECT, MO_OFF(im_func), READONLY, + {"__func__", _Py_T_OBJECT, MO_OFF(im_func), Py_READONLY, "the function (or other callable) implementing a method"}, - {"__self__", T_OBJECT, MO_OFF(im_self), READONLY, + {"__self__", _Py_T_OBJECT, MO_OFF(im_self), Py_READONLY, "the instance to which a method is bound"}, {NULL} /* Sentinel */ }; @@ -372,7 +372,7 @@ PyInstanceMethod_Function(PyObject *im) #define IMO_OFF(x) offsetof(PyInstanceMethodObject, x) static PyMemberDef instancemethod_memberlist[] = { - {"__func__", T_OBJECT, IMO_OFF(func), READONLY, + {"__func__", _Py_T_OBJECT, IMO_OFF(func), Py_READONLY, "the function (or other callable) implementing a method"}, {NULL} /* Sentinel */ }; diff --git a/Objects/codeobject.c b/Objects/codeobject.c index 977e5b05b3d2a..81fa33962e010 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -2,7 +2,7 @@ #include "Python.h" #include "opcode.h" -#include "structmember.h" // PyMemberDef + #include "pycore_code.h" // _PyCodeConstructor #include "pycore_frame.h" // FRAME_SPECIALS_SIZE #include "pycore_interp.h" // PyInterpreterState.co_extra_freefuncs @@ -1877,20 +1877,20 @@ code_hash(PyCodeObject *co) #define OFF(x) offsetof(PyCodeObject, x) static PyMemberDef code_memberlist[] = { - {"co_argcount", T_INT, OFF(co_argcount), READONLY}, - {"co_posonlyargcount", T_INT, OFF(co_posonlyargcount), READONLY}, - {"co_kwonlyargcount", T_INT, OFF(co_kwonlyargcount), READONLY}, - {"co_stacksize", T_INT, OFF(co_stacksize), READONLY}, - {"co_flags", T_INT, OFF(co_flags), READONLY}, - {"co_nlocals", T_INT, OFF(co_nlocals), READONLY}, - {"co_consts", T_OBJECT, OFF(co_consts), READONLY}, - {"co_names", T_OBJECT, OFF(co_names), READONLY}, - {"co_filename", T_OBJECT, OFF(co_filename), READONLY}, - {"co_name", T_OBJECT, OFF(co_name), READONLY}, - {"co_qualname", T_OBJECT, OFF(co_qualname), READONLY}, - {"co_firstlineno", T_INT, OFF(co_firstlineno), READONLY}, - {"co_linetable", T_OBJECT, OFF(co_linetable), READONLY}, - {"co_exceptiontable", T_OBJECT, OFF(co_exceptiontable), READONLY}, + {"co_argcount", Py_T_INT, OFF(co_argcount), Py_READONLY}, + {"co_posonlyargcount", Py_T_INT, OFF(co_posonlyargcount), Py_READONLY}, + {"co_kwonlyargcount", Py_T_INT, OFF(co_kwonlyargcount), Py_READONLY}, + {"co_stacksize", Py_T_INT, OFF(co_stacksize), Py_READONLY}, + {"co_flags", Py_T_INT, OFF(co_flags), Py_READONLY}, + {"co_nlocals", Py_T_INT, OFF(co_nlocals), Py_READONLY}, + {"co_consts", _Py_T_OBJECT, OFF(co_consts), Py_READONLY}, + {"co_names", _Py_T_OBJECT, OFF(co_names), Py_READONLY}, + {"co_filename", _Py_T_OBJECT, OFF(co_filename), Py_READONLY}, + {"co_name", _Py_T_OBJECT, OFF(co_name), Py_READONLY}, + {"co_qualname", _Py_T_OBJECT, OFF(co_qualname), Py_READONLY}, + {"co_firstlineno", Py_T_INT, OFF(co_firstlineno), Py_READONLY}, + {"co_linetable", _Py_T_OBJECT, OFF(co_linetable), Py_READONLY}, + {"co_exceptiontable", _Py_T_OBJECT, OFF(co_exceptiontable), Py_READONLY}, {NULL} /* Sentinel */ }; diff --git a/Objects/complexobject.c b/Objects/complexobject.c index 12968a63cd6fd..0e96f54584677 100644 --- a/Objects/complexobject.c +++ b/Objects/complexobject.c @@ -11,7 +11,7 @@ #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_object.h" // _PyObject_Init() #include "pycore_pymath.h" // _Py_ADJUST_ERANGE2() -#include "structmember.h" // PyMemberDef + /*[clinic input] @@ -720,9 +720,9 @@ static PyMethodDef complex_methods[] = { }; static PyMemberDef complex_members[] = { - {"real", T_DOUBLE, offsetof(PyComplexObject, cval.real), READONLY, + {"real", Py_T_DOUBLE, offsetof(PyComplexObject, cval.real), Py_READONLY, "the real part of a complex number"}, - {"imag", T_DOUBLE, offsetof(PyComplexObject, cval.imag), READONLY, + {"imag", Py_T_DOUBLE, offsetof(PyComplexObject, cval.imag), Py_READONLY, "the imaginary part of a complex number"}, {0}, }; diff --git a/Objects/descrobject.c b/Objects/descrobject.c index 74aa70bed7de9..60383dd06d1ad 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -8,7 +8,7 @@ #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_tuple.h" // _PyTuple_ITEMS() -#include "structmember.h" // PyMemberDef + /*[clinic input] class mappingproxy "mappingproxyobject *" "&PyDictProxy_Type" @@ -182,7 +182,7 @@ member_get(PyMemberDescrObject *descr, PyObject *obj, PyObject *type) return NULL; } - if (descr->d_member->flags & PY_AUDIT_READ) { + if (descr->d_member->flags & Py_AUDIT_READ) { if (PySys_Audit("object.__getattr__", "Os", obj ? obj : Py_None, descr->d_member->name) < 0) { return NULL; @@ -640,8 +640,8 @@ static PyMethodDef descr_methods[] = { }; static PyMemberDef descr_members[] = { - {"__objclass__", T_OBJECT, offsetof(PyDescrObject, d_type), READONLY}, - {"__name__", T_OBJECT, offsetof(PyDescrObject, d_name), READONLY}, + {"__objclass__", _Py_T_OBJECT, offsetof(PyDescrObject, d_type), Py_READONLY}, + {"__name__", _Py_T_OBJECT, offsetof(PyDescrObject, d_name), Py_READONLY}, {0} }; @@ -1355,7 +1355,7 @@ static PyMethodDef wrapper_methods[] = { }; static PyMemberDef wrapper_members[] = { - {"__self__", T_OBJECT, offsetof(wrapperobject, self), READONLY}, + {"__self__", _Py_T_OBJECT, offsetof(wrapperobject, self), Py_READONLY}, {0} }; @@ -1515,10 +1515,10 @@ static PyObject * property_copy(PyObject *, PyObject *, PyObject *, PyObject *); static PyMemberDef property_members[] = { - {"fget", T_OBJECT, offsetof(propertyobject, prop_get), READONLY}, - {"fset", T_OBJECT, offsetof(propertyobject, prop_set), READONLY}, - {"fdel", T_OBJECT, offsetof(propertyobject, prop_del), READONLY}, - {"__doc__", T_OBJECT, offsetof(propertyobject, prop_doc), 0}, + {"fget", _Py_T_OBJECT, offsetof(propertyobject, prop_get), Py_READONLY}, + {"fset", _Py_T_OBJECT, offsetof(propertyobject, prop_set), Py_READONLY}, + {"fdel", _Py_T_OBJECT, offsetof(propertyobject, prop_del), Py_READONLY}, + {"__doc__", _Py_T_OBJECT, offsetof(propertyobject, prop_doc), 0}, {0} }; diff --git a/Objects/exceptions.c b/Objects/exceptions.c index 42c5317d83d0c..62a44234b3404 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -12,7 +12,7 @@ #include "pycore_exceptions.h" // struct _Py_exc_state #include "pycore_initconfig.h" #include "pycore_object.h" -#include "structmember.h" // PyMemberDef + #include "osdefs.h" // SEP @@ -439,7 +439,7 @@ PyExceptionClass_Name(PyObject *ob) } static struct PyMemberDef BaseException_members[] = { - {"__suppress_context__", T_BOOL, + {"__suppress_context__", Py_T_BOOL, offsetof(PyBaseExceptionObject, suppress_context)}, {NULL} }; @@ -569,7 +569,7 @@ SimpleExtendsException(PyExc_Exception, StopAsyncIteration, */ static PyMemberDef StopIteration_members[] = { - {"value", T_OBJECT, offsetof(PyStopIterationObject, value), 0, + {"value", _Py_T_OBJECT, offsetof(PyStopIterationObject, value), 0, PyDoc_STR("generator return value")}, {NULL} /* Sentinel */ }; @@ -671,7 +671,7 @@ SystemExit_traverse(PySystemExitObject *self, visitproc visit, void *arg) } static PyMemberDef SystemExit_members[] = { - {"code", T_OBJECT, offsetof(PySystemExitObject, code), 0, + {"code", _Py_T_OBJECT, offsetof(PySystemExitObject, code), 0, PyDoc_STR("exception code")}, {NULL} /* Sentinel */ }; @@ -1477,9 +1477,9 @@ PyUnstable_Exc_PrepReraiseStar(PyObject *orig, PyObject *excs) } static PyMemberDef BaseExceptionGroup_members[] = { - {"message", T_OBJECT, offsetof(PyBaseExceptionGroupObject, msg), READONLY, + {"message", _Py_T_OBJECT, offsetof(PyBaseExceptionGroupObject, msg), Py_READONLY, PyDoc_STR("exception message")}, - {"exceptions", T_OBJECT, offsetof(PyBaseExceptionGroupObject, excs), READONLY, + {"exceptions", _Py_T_OBJECT, offsetof(PyBaseExceptionGroupObject, excs), Py_READONLY, PyDoc_STR("nested exceptions")}, {NULL} /* Sentinel */ }; @@ -1654,13 +1654,13 @@ ImportError_reduce(PyImportErrorObject *self, PyObject *Py_UNUSED(ignored)) } static PyMemberDef ImportError_members[] = { - {"msg", T_OBJECT, offsetof(PyImportErrorObject, msg), 0, + {"msg", _Py_T_OBJECT, offsetof(PyImportErrorObject, msg), 0, PyDoc_STR("exception message")}, - {"name", T_OBJECT, offsetof(PyImportErrorObject, name), 0, + {"name", _Py_T_OBJECT, offsetof(PyImportErrorObject, name), 0, PyDoc_STR("module name")}, - {"path", T_OBJECT, offsetof(PyImportErrorObject, path), 0, + {"path", _Py_T_OBJECT, offsetof(PyImportErrorObject, path), 0, PyDoc_STR("module path")}, - {"name_from", T_OBJECT, offsetof(PyImportErrorObject, name_from), 0, + {"name_from", _Py_T_OBJECT, offsetof(PyImportErrorObject, name_from), 0, PyDoc_STR("name imported from module")}, {NULL} /* Sentinel */ }; @@ -2103,16 +2103,16 @@ OSError_written_set(PyOSErrorObject *self, PyObject *arg, void *context) } static PyMemberDef OSError_members[] = { - {"errno", T_OBJECT, offsetof(PyOSErrorObject, myerrno), 0, + {"errno", _Py_T_OBJECT, offsetof(PyOSErrorObject, myerrno), 0, PyDoc_STR("POSIX exception code")}, - {"strerror", T_OBJECT, offsetof(PyOSErrorObject, strerror), 0, + {"strerror", _Py_T_OBJECT, offsetof(PyOSErrorObject, strerror), 0, PyDoc_STR("exception strerror")}, - {"filename", T_OBJECT, offsetof(PyOSErrorObject, filename), 0, + {"filename", _Py_T_OBJECT, offsetof(PyOSErrorObject, filename), 0, PyDoc_STR("exception filename")}, - {"filename2", T_OBJECT, offsetof(PyOSErrorObject, filename2), 0, + {"filename2", _Py_T_OBJECT, offsetof(PyOSErrorObject, filename2), 0, PyDoc_STR("second exception filename")}, #ifdef MS_WINDOWS - {"winerror", T_OBJECT, offsetof(PyOSErrorObject, winerror), 0, + {"winerror", _Py_T_OBJECT, offsetof(PyOSErrorObject, winerror), 0, PyDoc_STR("Win32 exception code")}, #endif {NULL} /* Sentinel */ @@ -2249,7 +2249,7 @@ NameError_traverse(PyNameErrorObject *self, visitproc visit, void *arg) } static PyMemberDef NameError_members[] = { - {"name", T_OBJECT, offsetof(PyNameErrorObject, name), 0, PyDoc_STR("name")}, + {"name", _Py_T_OBJECT, offsetof(PyNameErrorObject, name), 0, PyDoc_STR("name")}, {NULL} /* Sentinel */ }; @@ -2368,8 +2368,8 @@ AttributeError_reduce(PyAttributeErrorObject *self, PyObject *Py_UNUSED(ignored) } static PyMemberDef AttributeError_members[] = { - {"name", T_OBJECT, offsetof(PyAttributeErrorObject, name), 0, PyDoc_STR("attribute name")}, - {"obj", T_OBJECT, offsetof(PyAttributeErrorObject, obj), 0, PyDoc_STR("object")}, + {"name", _Py_T_OBJECT, offsetof(PyAttributeErrorObject, name), 0, PyDoc_STR("attribute name")}, + {"obj", _Py_T_OBJECT, offsetof(PyAttributeErrorObject, obj), 0, PyDoc_STR("object")}, {NULL} /* Sentinel */ }; @@ -2541,21 +2541,21 @@ SyntaxError_str(PySyntaxErrorObject *self) } static PyMemberDef SyntaxError_members[] = { - {"msg", T_OBJECT, offsetof(PySyntaxErrorObject, msg), 0, + {"msg", _Py_T_OBJECT, offsetof(PySyntaxErrorObject, msg), 0, PyDoc_STR("exception msg")}, - {"filename", T_OBJECT, offsetof(PySyntaxErrorObject, filename), 0, + {"filename", _Py_T_OBJECT, offsetof(PySyntaxErrorObject, filename), 0, PyDoc_STR("exception filename")}, - {"lineno", T_OBJECT, offsetof(PySyntaxErrorObject, lineno), 0, + {"lineno", _Py_T_OBJECT, offsetof(PySyntaxErrorObject, lineno), 0, PyDoc_STR("exception lineno")}, - {"offset", T_OBJECT, offsetof(PySyntaxErrorObject, offset), 0, + {"offset", _Py_T_OBJECT, offsetof(PySyntaxErrorObject, offset), 0, PyDoc_STR("exception offset")}, - {"text", T_OBJECT, offsetof(PySyntaxErrorObject, text), 0, + {"text", _Py_T_OBJECT, offsetof(PySyntaxErrorObject, text), 0, PyDoc_STR("exception text")}, - {"end_lineno", T_OBJECT, offsetof(PySyntaxErrorObject, end_lineno), 0, + {"end_lineno", _Py_T_OBJECT, offsetof(PySyntaxErrorObject, end_lineno), 0, PyDoc_STR("exception end lineno")}, - {"end_offset", T_OBJECT, offsetof(PySyntaxErrorObject, end_offset), 0, + {"end_offset", _Py_T_OBJECT, offsetof(PySyntaxErrorObject, end_offset), 0, PyDoc_STR("exception end offset")}, - {"print_file_and_line", T_OBJECT, + {"print_file_and_line", _Py_T_OBJECT, offsetof(PySyntaxErrorObject, print_file_and_line), 0, PyDoc_STR("exception print_file_and_line")}, {NULL} /* Sentinel */ @@ -2910,15 +2910,15 @@ UnicodeError_traverse(PyUnicodeErrorObject *self, visitproc visit, void *arg) } static PyMemberDef UnicodeError_members[] = { - {"encoding", T_OBJECT, offsetof(PyUnicodeErrorObject, encoding), 0, + {"encoding", _Py_T_OBJECT, offsetof(PyUnicodeErrorObject, encoding), 0, PyDoc_STR("exception encoding")}, - {"object", T_OBJECT, offsetof(PyUnicodeErrorObject, object), 0, + {"object", _Py_T_OBJECT, offsetof(PyUnicodeErrorObject, object), 0, PyDoc_STR("exception object")}, - {"start", T_PYSSIZET, offsetof(PyUnicodeErrorObject, start), 0, + {"start", Py_T_PYSSIZET, offsetof(PyUnicodeErrorObject, start), 0, PyDoc_STR("exception start")}, - {"end", T_PYSSIZET, offsetof(PyUnicodeErrorObject, end), 0, + {"end", Py_T_PYSSIZET, offsetof(PyUnicodeErrorObject, end), 0, PyDoc_STR("exception end")}, - {"reason", T_OBJECT, offsetof(PyUnicodeErrorObject, reason), 0, + {"reason", _Py_T_OBJECT, offsetof(PyUnicodeErrorObject, reason), 0, PyDoc_STR("exception reason")}, {NULL} /* Sentinel */ }; diff --git a/Objects/frameobject.c b/Objects/frameobject.c index 18820551a0547..cc9ac4b42f479 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -11,12 +11,12 @@ #include "frameobject.h" // PyFrameObject #include "pycore_frame.h" #include "opcode.h" // EXTENDED_ARG -#include "structmember.h" // PyMemberDef + #define OFF(x) offsetof(PyFrameObject, x) static PyMemberDef frame_memberlist[] = { - {"f_trace_lines", T_BOOL, OFF(f_trace_lines), 0}, + {"f_trace_lines", Py_T_BOOL, OFF(f_trace_lines), 0}, {NULL} /* Sentinel */ }; diff --git a/Objects/funcobject.c b/Objects/funcobject.c index 0c69bf4ebcfed..7fffa1c8bbff9 100644 --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -6,7 +6,7 @@ #include "pycore_code.h" // _Py_next_func_version #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "pycore_pyerrors.h" // _PyErr_Occurred() -#include "structmember.h" // PyMemberDef + static PyObject* func_repr(PyFunctionObject *op); @@ -451,11 +451,11 @@ PyFunction_SetAnnotations(PyObject *op, PyObject *annotations) #define OFF(x) offsetof(PyFunctionObject, x) static PyMemberDef func_memberlist[] = { - {"__closure__", T_OBJECT, OFF(func_closure), READONLY}, - {"__doc__", T_OBJECT, OFF(func_doc), 0}, - {"__globals__", T_OBJECT, OFF(func_globals), READONLY}, - {"__module__", T_OBJECT, OFF(func_module), 0}, - {"__builtins__", T_OBJECT, OFF(func_builtins), READONLY}, + {"__closure__", _Py_T_OBJECT, OFF(func_closure), Py_READONLY}, + {"__doc__", _Py_T_OBJECT, OFF(func_doc), 0}, + {"__globals__", _Py_T_OBJECT, OFF(func_globals), Py_READONLY}, + {"__module__", _Py_T_OBJECT, OFF(func_module), 0}, + {"__builtins__", _Py_T_OBJECT, OFF(func_builtins), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -1063,8 +1063,8 @@ cm_init(PyObject *self, PyObject *args, PyObject *kwds) } static PyMemberDef cm_memberlist[] = { - {"__func__", T_OBJECT, offsetof(classmethod, cm_callable), READONLY}, - {"__wrapped__", T_OBJECT, offsetof(classmethod, cm_callable), READONLY}, + {"__func__", _Py_T_OBJECT, offsetof(classmethod, cm_callable), Py_READONLY}, + {"__wrapped__", _Py_T_OBJECT, offsetof(classmethod, cm_callable), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -1258,8 +1258,8 @@ sm_call(PyObject *callable, PyObject *args, PyObject *kwargs) } static PyMemberDef sm_memberlist[] = { - {"__func__", T_OBJECT, offsetof(staticmethod, sm_callable), READONLY}, - {"__wrapped__", T_OBJECT, offsetof(staticmethod, sm_callable), READONLY}, + {"__func__", _Py_T_OBJECT, offsetof(staticmethod, sm_callable), Py_READONLY}, + {"__wrapped__", _Py_T_OBJECT, offsetof(staticmethod, sm_callable), Py_READONLY}, {NULL} /* Sentinel */ }; diff --git a/Objects/genericaliasobject.c b/Objects/genericaliasobject.c index 0c478f3717e03..df8873454aeb3 100644 --- a/Objects/genericaliasobject.c +++ b/Objects/genericaliasobject.c @@ -3,7 +3,7 @@ #include "Python.h" #include "pycore_object.h" #include "pycore_unionobject.h" // _Py_union_type_or, _PyGenericAlias_Check -#include "structmember.h" // PyMemberDef + #include @@ -782,9 +782,9 @@ static PyMethodDef ga_methods[] = { }; static PyMemberDef ga_members[] = { - {"__origin__", T_OBJECT, offsetof(gaobject, origin), READONLY}, - {"__args__", T_OBJECT, offsetof(gaobject, args), READONLY}, - {"__unpacked__", T_BOOL, offsetof(gaobject, starred), READONLY}, + {"__origin__", _Py_T_OBJECT, offsetof(gaobject, origin), Py_READONLY}, + {"__args__", _Py_T_OBJECT, offsetof(gaobject, args), Py_READONLY}, + {"__unpacked__", Py_T_BOOL, offsetof(gaobject, starred), Py_READONLY}, {0} }; diff --git a/Objects/genobject.c b/Objects/genobject.c index 103e8b8bb882f..a630f84fb5a29 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -10,7 +10,7 @@ #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "pycore_pyerrors.h" // _PyErr_ClearExcState() #include "pycore_pystate.h" // _PyThreadState_GET() -#include "structmember.h" // PyMemberDef + #include "opcode.h" // SEND #include "frameobject.h" // _PyInterpreterFrame_GetLine #include "pystats.h" @@ -1144,7 +1144,7 @@ static PyGetSetDef coro_getsetlist[] = { }; static PyMemberDef coro_memberlist[] = { - {"cr_origin", T_OBJECT, offsetof(PyCoroObject, cr_origin_or_finalizer), READONLY}, + {"cr_origin", _Py_T_OBJECT, offsetof(PyCoroObject, cr_origin_or_finalizer), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -1558,8 +1558,8 @@ static PyGetSetDef async_gen_getsetlist[] = { }; static PyMemberDef async_gen_memberlist[] = { - {"ag_running", T_BOOL, offsetof(PyAsyncGenObject, ag_running_async), - READONLY}, + {"ag_running", Py_T_BOOL, offsetof(PyAsyncGenObject, ag_running_async), + Py_READONLY}, {NULL} /* Sentinel */ }; diff --git a/Objects/methodobject.c b/Objects/methodobject.c index fe081992d51fd..628d227ef33c3 100644 --- a/Objects/methodobject.c +++ b/Objects/methodobject.c @@ -7,7 +7,7 @@ #include "pycore_object.h" #include "pycore_pyerrors.h" #include "pycore_pystate.h" // _PyThreadState_GET() -#include "structmember.h" // PyMemberDef + /* undefine macro trampoline to PyCFunction_NewEx */ #undef PyCFunction_New @@ -273,7 +273,7 @@ static PyGetSetDef meth_getsets [] = { #define OFF(x) offsetof(PyCFunctionObject, x) static PyMemberDef meth_members[] = { - {"__module__", T_OBJECT, OFF(m_module), 0}, + {"__module__", _Py_T_OBJECT, OFF(m_module), 0}, {NULL} }; diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index ba20534c3bdd8..7e890d021cb94 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -9,11 +9,11 @@ #include "pycore_object.h" // _PyType_AllocNoTrack #include "pycore_pyerrors.h" // _PyErr_FormatFromCause() #include "pycore_pystate.h" // _PyInterpreterState_GET() -#include "structmember.h" // PyMemberDef + static PyMemberDef module_members[] = { - {"__dict__", T_OBJECT, offsetof(PyModuleObject, md_dict), READONLY}, + {"__dict__", _Py_T_OBJECT, offsetof(PyModuleObject, md_dict), Py_READONLY}, {0} }; diff --git a/Objects/namespaceobject.c b/Objects/namespaceobject.c index 2cc4ddd3c91da..11cf859add3ab 100644 --- a/Objects/namespaceobject.c +++ b/Objects/namespaceobject.c @@ -2,7 +2,9 @@ #include "Python.h" #include "pycore_namespace.h" // _PyNamespace_Type -#include "structmember.h" // PyMemberDef + +#include // offsetof() + typedef struct { @@ -12,7 +14,7 @@ typedef struct { static PyMemberDef namespace_members[] = { - {"__dict__", T_OBJECT, offsetof(_PyNamespaceObject, ns_dict), READONLY}, + {"__dict__", _Py_T_OBJECT, offsetof(_PyNamespaceObject, ns_dict), Py_READONLY}, {NULL} }; diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c index 6dc41d71287ca..1e3d5acc8ae6f 100644 --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -6,7 +6,7 @@ #include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_range.h" #include "pycore_tuple.h" // _PyTuple_ITEMS() -#include "structmember.h" // PyMemberDef + /* Support objects whose length is > PY_SSIZE_T_MAX. @@ -757,9 +757,9 @@ static PyMethodDef range_methods[] = { }; static PyMemberDef range_members[] = { - {"start", T_OBJECT_EX, offsetof(rangeobject, start), READONLY}, - {"stop", T_OBJECT_EX, offsetof(rangeobject, stop), READONLY}, - {"step", T_OBJECT_EX, offsetof(rangeobject, step), READONLY}, + {"start", Py_T_OBJECT_EX, offsetof(rangeobject, start), Py_READONLY}, + {"stop", Py_T_OBJECT_EX, offsetof(rangeobject, stop), Py_READONLY}, + {"step", Py_T_OBJECT_EX, offsetof(rangeobject, step), Py_READONLY}, {0} }; diff --git a/Objects/sliceobject.c b/Objects/sliceobject.c index 5a33977f06442..dc3aad11ce10e 100644 --- a/Objects/sliceobject.c +++ b/Objects/sliceobject.c @@ -17,7 +17,7 @@ this type and there is exactly one in existence. #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_object.h" // _PyObject_GC_TRACK() -#include "structmember.h" // PyMemberDef + static PyObject * ellipsis_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) @@ -377,9 +377,9 @@ slice_repr(PySliceObject *r) } static PyMemberDef slice_members[] = { - {"start", T_OBJECT, offsetof(PySliceObject, start), READONLY}, - {"stop", T_OBJECT, offsetof(PySliceObject, stop), READONLY}, - {"step", T_OBJECT, offsetof(PySliceObject, step), READONLY}, + {"start", _Py_T_OBJECT, offsetof(PySliceObject, start), Py_READONLY}, + {"stop", _Py_T_OBJECT, offsetof(PySliceObject, stop), Py_READONLY}, + {"step", _Py_T_OBJECT, offsetof(PySliceObject, step), Py_READONLY}, {0} }; diff --git a/Objects/structseq.c b/Objects/structseq.c index 49011139b6653..700f67c09c9e5 100644 --- a/Objects/structseq.c +++ b/Objects/structseq.c @@ -10,7 +10,7 @@ #include "Python.h" #include "pycore_tuple.h" // _PyTuple_FromArray() #include "pycore_object.h" // _PyObject_GC_TRACK() -#include "structmember.h" // PyMemberDef + #include "pycore_structseq.h" // PyStructSequence_InitType() #include "pycore_initconfig.h" // _PyStatus_OK() @@ -465,10 +465,10 @@ initialize_members(PyStructSequence_Desc *desc, /* The names and docstrings in these MemberDefs are statically */ /* allocated so it is expected that they'll outlive the MemberDef */ members[k].name = desc->fields[i].name; - members[k].type = T_OBJECT; + members[k].type = _Py_T_OBJECT; members[k].offset = offsetof(PyStructSequence, ob_item) + i * sizeof(PyObject*); - members[k].flags = READONLY; + members[k].flags = Py_READONLY; members[k].doc = desc->fields[i].doc; k++; } diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 7e5282cabd1bf..abe33f1562059 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -18,7 +18,7 @@ #include "pycore_unionobject.h" // _Py_union_type_or #include "pycore_weakref.h" // _PyWeakref_GET_REF() #include "opcode.h" // MAKE_CELL -#include "structmember.h" // PyMemberDef + #include #include // ptrdiff_t @@ -935,16 +935,16 @@ int PyUnstable_Type_AssignVersionTag(PyTypeObject *type) static PyMemberDef type_members[] = { - {"__basicsize__", T_PYSSIZET, offsetof(PyTypeObject,tp_basicsize),READONLY}, - {"__itemsize__", T_PYSSIZET, offsetof(PyTypeObject, tp_itemsize), READONLY}, - {"__flags__", T_ULONG, offsetof(PyTypeObject, tp_flags), READONLY}, + {"__basicsize__", Py_T_PYSSIZET, offsetof(PyTypeObject,tp_basicsize),Py_READONLY}, + {"__itemsize__", Py_T_PYSSIZET, offsetof(PyTypeObject, tp_itemsize), Py_READONLY}, + {"__flags__", Py_T_ULONG, offsetof(PyTypeObject, tp_flags), Py_READONLY}, /* Note that this value is misleading for static builtin types, since the memory at this offset will always be NULL. */ - {"__weakrefoffset__", T_PYSSIZET, - offsetof(PyTypeObject, tp_weaklistoffset), READONLY}, - {"__base__", T_OBJECT, offsetof(PyTypeObject, tp_base), READONLY}, - {"__dictoffset__", T_PYSSIZET, - offsetof(PyTypeObject, tp_dictoffset), READONLY}, + {"__weakrefoffset__", Py_T_PYSSIZET, + offsetof(PyTypeObject, tp_weaklistoffset), Py_READONLY}, + {"__base__", _Py_T_OBJECT, offsetof(PyTypeObject, tp_base), Py_READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, + offsetof(PyTypeObject, tp_dictoffset), Py_READONLY}, {0} }; @@ -1775,7 +1775,7 @@ traverse_slots(PyTypeObject *type, PyObject *self, visitproc visit, void *arg) n = Py_SIZE(type); mp = _PyHeapType_GET_MEMBERS((PyHeapTypeObject *)type); for (i = 0; i < n; i++, mp++) { - if (mp->type == T_OBJECT_EX) { + if (mp->type == Py_T_OBJECT_EX) { char *addr = (char *)self + mp->offset; PyObject *obj = *(PyObject **)addr; if (obj != NULL) { @@ -1850,7 +1850,7 @@ clear_slots(PyTypeObject *type, PyObject *self) n = Py_SIZE(type); mp = _PyHeapType_GET_MEMBERS((PyHeapTypeObject *)type); for (i = 0; i < n; i++, mp++) { - if (mp->type == T_OBJECT_EX && !(mp->flags & READONLY)) { + if (mp->type == Py_T_OBJECT_EX && !(mp->flags & Py_READONLY)) { char *addr = (char *)self + mp->offset; PyObject *obj = *(PyObject **)addr; if (obj != NULL) { @@ -3567,7 +3567,7 @@ type_new_descriptors(const type_new_ctx *ctx, PyTypeObject *type) if (mp->name == NULL) { return -1; } - mp->type = T_OBJECT_EX; + mp->type = Py_T_OBJECT_EX; mp->offset = slotoffset; /* __dict__ and __weakref__ are already filtered out */ @@ -4116,20 +4116,20 @@ _PyType_FromMetaclass_impl( nmembers++; if (strcmp(memb->name, "__weaklistoffset__") == 0) { // The PyMemberDef must be a Py_ssize_t and readonly - assert(memb->type == T_PYSSIZET); - assert(memb->flags == READONLY); + assert(memb->type == Py_T_PYSSIZET); + assert(memb->flags == Py_READONLY); weaklistoffset = memb->offset; } if (strcmp(memb->name, "__dictoffset__") == 0) { // The PyMemberDef must be a Py_ssize_t and readonly - assert(memb->type == T_PYSSIZET); - assert(memb->flags == READONLY); + assert(memb->type == Py_T_PYSSIZET); + assert(memb->flags == Py_READONLY); dictoffset = memb->offset; } if (strcmp(memb->name, "__vectorcalloffset__") == 0) { // The PyMemberDef must be a Py_ssize_t and readonly - assert(memb->type == T_PYSSIZET); - assert(memb->flags == READONLY); + assert(memb->type == Py_T_PYSSIZET); + assert(memb->flags == Py_READONLY); vectorcalloffset = memb->offset; } if (memb->flags & Py_RELATIVE_OFFSET) { @@ -10178,11 +10178,11 @@ typedef struct { } superobject; static PyMemberDef super_members[] = { - {"__thisclass__", T_OBJECT, offsetof(superobject, type), READONLY, + {"__thisclass__", _Py_T_OBJECT, offsetof(superobject, type), Py_READONLY, "the class invoking super()"}, - {"__self__", T_OBJECT, offsetof(superobject, obj), READONLY, + {"__self__", _Py_T_OBJECT, offsetof(superobject, obj), Py_READONLY, "the instance invoking super(); may be None"}, - {"__self_class__", T_OBJECT, offsetof(superobject, obj_type), READONLY, + {"__self_class__", _Py_T_OBJECT, offsetof(superobject, obj_type), Py_READONLY, "the type of the instance invoking super(); may be None"}, {0} }; diff --git a/Objects/typevarobject.c b/Objects/typevarobject.c index 5605662f0e6d5..e09e6a62553cf 100644 --- a/Objects/typevarobject.c +++ b/Objects/typevarobject.c @@ -3,7 +3,7 @@ #include "pycore_object.h" // _PyObject_GC_TRACK/UNTRACK #include "pycore_typevarobject.h" #include "pycore_unionobject.h" // _Py_union_type_or -#include "structmember.h" + /*[clinic input] class typevar "typevarobject *" "&_PyTypeVar_Type" @@ -244,10 +244,10 @@ typevar_repr(PyObject *self) } static PyMemberDef typevar_members[] = { - {"__name__", T_OBJECT, offsetof(typevarobject, name), READONLY}, - {"__covariant__", T_BOOL, offsetof(typevarobject, covariant), READONLY}, - {"__contravariant__", T_BOOL, offsetof(typevarobject, contravariant), READONLY}, - {"__infer_variance__", T_BOOL, offsetof(typevarobject, infer_variance), READONLY}, + {"__name__", _Py_T_OBJECT, offsetof(typevarobject, name), Py_READONLY}, + {"__covariant__", Py_T_BOOL, offsetof(typevarobject, covariant), Py_READONLY}, + {"__contravariant__", Py_T_BOOL, offsetof(typevarobject, contravariant), Py_READONLY}, + {"__infer_variance__", Py_T_BOOL, offsetof(typevarobject, infer_variance), Py_READONLY}, {0} }; @@ -555,7 +555,7 @@ paramspecattr_richcompare(PyObject *a, PyObject *b, int op) } static PyMemberDef paramspecattr_members[] = { - {"__origin__", T_OBJECT, offsetof(paramspecattrobject, __origin__), READONLY}, + {"__origin__", _Py_T_OBJECT, offsetof(paramspecattrobject, __origin__), Py_READONLY}, {0} }; @@ -780,11 +780,11 @@ paramspec_repr(PyObject *self) } static PyMemberDef paramspec_members[] = { - {"__name__", T_OBJECT, offsetof(paramspecobject, name), READONLY}, - {"__bound__", T_OBJECT, offsetof(paramspecobject, bound), READONLY}, - {"__covariant__", T_BOOL, offsetof(paramspecobject, covariant), READONLY}, - {"__contravariant__", T_BOOL, offsetof(paramspecobject, contravariant), READONLY}, - {"__infer_variance__", T_BOOL, offsetof(paramspecobject, infer_variance), READONLY}, + {"__name__", _Py_T_OBJECT, offsetof(paramspecobject, name), Py_READONLY}, + {"__bound__", _Py_T_OBJECT, offsetof(paramspecobject, bound), Py_READONLY}, + {"__covariant__", Py_T_BOOL, offsetof(paramspecobject, covariant), Py_READONLY}, + {"__contravariant__", Py_T_BOOL, offsetof(paramspecobject, contravariant), Py_READONLY}, + {"__infer_variance__", Py_T_BOOL, offsetof(paramspecobject, infer_variance), Py_READONLY}, {0} }; @@ -1054,7 +1054,7 @@ typevartuple_repr(PyObject *self) } static PyMemberDef typevartuple_members[] = { - {"__name__", T_OBJECT, offsetof(typevartupleobject, name), READONLY}, + {"__name__", _Py_T_OBJECT, offsetof(typevartupleobject, name), Py_READONLY}, {0} }; @@ -1292,7 +1292,7 @@ typealias_repr(PyObject *self) } static PyMemberDef typealias_members[] = { - {"__name__", T_OBJECT, offsetof(typealiasobject, name), READONLY}, + {"__name__", _Py_T_OBJECT, offsetof(typealiasobject, name), Py_READONLY}, {0} }; diff --git a/Objects/unionobject.c b/Objects/unionobject.c index 269f46914f263..347945a4c4597 100644 --- a/Objects/unionobject.c +++ b/Objects/unionobject.c @@ -3,7 +3,7 @@ #include "pycore_object.h" // _PyObject_GC_TRACK/UNTRACK #include "pycore_typevarobject.h" // _PyTypeAlias_Type #include "pycore_unionobject.h" -#include "structmember.h" + static PyObject *make_union(PyObject *); @@ -273,7 +273,7 @@ union_repr(PyObject *self) } static PyMemberDef union_members[] = { - {"__args__", T_OBJECT, offsetof(unionobject, args), READONLY}, + {"__args__", _Py_T_OBJECT, offsetof(unionobject, args), Py_READONLY}, {0} }; diff --git a/Objects/weakrefobject.c b/Objects/weakrefobject.c index e9563729bf82b..1814c6eb69c29 100644 --- a/Objects/weakrefobject.c +++ b/Objects/weakrefobject.c @@ -2,7 +2,7 @@ #include "pycore_modsupport.h" // _PyArg_NoKwnames() #include "pycore_object.h" // _PyObject_GET_WEAKREFS_LISTPTR() #include "pycore_weakref.h" // _PyWeakref_GET_REF() -#include "structmember.h" // PyMemberDef + #define GET_WEAKREFS_LISTPTR(o) \ @@ -351,7 +351,7 @@ weakref___init__(PyObject *self, PyObject *args, PyObject *kwargs) static PyMemberDef weakref_members[] = { - {"__callback__", T_OBJECT, offsetof(PyWeakReference, wr_callback), READONLY}, + {"__callback__", _Py_T_OBJECT, offsetof(PyWeakReference, wr_callback), Py_READONLY}, {NULL} /* Sentinel */ }; diff --git a/PC/winreg.c b/PC/winreg.c index 5252f78a9bdf7..ed6258d2b33e5 100644 --- a/PC/winreg.c +++ b/PC/winreg.c @@ -15,7 +15,7 @@ #include "Python.h" #include "pycore_object.h" // _PyObject_Init() #include "pycore_moduleobject.h" -#include "structmember.h" // PyMemberDef + #include #if defined(MS_WINDOWS_DESKTOP) || defined(MS_WINDOWS_SYSTEM) || defined(MS_WINDOWS_GAMES) @@ -352,7 +352,7 @@ static struct PyMethodDef PyHKEY_methods[] = { #define OFF(e) offsetof(PyHKEYObject, e) static PyMemberDef PyHKEY_memberlist[] = { - {"handle", T_INT, OFF(hkey), READONLY}, + {"handle", Py_T_INT, OFF(hkey), Py_READONLY}, {NULL} /* Sentinel */ }; diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py index e9665dd808af3..f159b573ce472 100755 --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -897,7 +897,7 @@ def visitModule(self, mod): } static PyMemberDef ast_type_members[] = { - {"__dictoffset__", T_PYSSIZET, offsetof(AST_object, dict), READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(AST_object, dict), Py_READONLY}, {NULL} /* Sentinel */ }; @@ -1542,7 +1542,6 @@ def generate_module_def(mod, metadata, f, internal_h): #include "pycore_ceval.h" // _Py_EnterRecursiveCall #include "pycore_interp.h" // _PyInterpreterState.ast #include "pycore_pystate.h" // _PyInterpreterState_GET() - #include "structmember.h" #include // Forward declaration diff --git a/Python/Python-ast.c b/Python/Python-ast.c index 55a1370fbd038..412de79397477 100644 --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -6,7 +6,6 @@ #include "pycore_ceval.h" // _Py_EnterRecursiveCall #include "pycore_interp.h" // _PyInterpreterState.ast #include "pycore_pystate.h" // _PyInterpreterState_GET() -#include "structmember.h" #include // Forward declaration @@ -923,7 +922,7 @@ ast_type_reduce(PyObject *self, PyObject *unused) } static PyMemberDef ast_type_members[] = { - {"__dictoffset__", T_PYSSIZET, offsetof(AST_object, dict), READONLY}, + {"__dictoffset__", Py_T_PYSSIZET, offsetof(AST_object, dict), Py_READONLY}, {NULL} /* Sentinel */ }; diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 0705f5b0aac43..0f1e0073a3f60 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -36,7 +36,7 @@ #include "optimizer.h" #include "pydtrace.h" #include "setobject.h" -#include "structmember.h" // struct PyMemberDef, T_OFFSET_EX + #define USE_COMPUTED_GOTOS 0 #include "ceval_macros.h" diff --git a/Python/ceval.c b/Python/ceval.c index a8987efc21a98..688c019e2add1 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -34,7 +34,7 @@ #include "opcode.h" #include "pydtrace.h" #include "setobject.h" -#include "structmember.h" // struct PyMemberDef, T_OFFSET_EX + #include #include diff --git a/Python/context.c b/Python/context.c index 1ffae9871be7b..9ac51874fb5be 100644 --- a/Python/context.c +++ b/Python/context.c @@ -7,7 +7,7 @@ #include "pycore_object.h" #include "pycore_pyerrors.h" #include "pycore_pystate.h" // _PyThreadState_GET() -#include "structmember.h" // PyMemberDef + #include "clinic/context.c.h" @@ -1042,7 +1042,7 @@ _contextvars_ContextVar_reset(PyContextVar *self, PyObject *token) static PyMemberDef PyContextVar_members[] = { - {"name", T_OBJECT, offsetof(PyContextVar, var_name), READONLY}, + {"name", _Py_T_OBJECT, offsetof(PyContextVar, var_name), Py_READONLY}, {NULL} }; diff --git a/Python/specialize.c b/Python/specialize.c index 5892f5441c98e..1669ce17fc804 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -9,7 +9,7 @@ #include "pycore_object.h" #include "pycore_opcode.h" // _PyOpcode_Caches #include "pycore_pylifecycle.h" // _PyOS_URandomNonblock() -#include "structmember.h" // struct PyMemberDef, T_OFFSET_EX + #include // rand() @@ -621,7 +621,7 @@ analyze_descriptor(PyTypeObject *type, PyObject *name, PyObject **descr, int sto if (desc_cls == &PyMemberDescr_Type) { PyMemberDescrObject *member = (PyMemberDescrObject *)descriptor; struct PyMemberDef *dmem = member->d_member; - if (dmem->type == T_OBJECT_EX) { + if (dmem->type == Py_T_OBJECT_EX) { return OBJECT_SLOT; } return OTHER_SLOT; @@ -803,7 +803,7 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name) SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_EXPECTED_ERROR); goto fail; } - if (dmem->flags & PY_AUDIT_READ) { + if (dmem->flags & Py_AUDIT_READ) { SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_AUDITED_SLOT); goto fail; } @@ -811,7 +811,7 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name) SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_OUT_OF_RANGE); goto fail; } - assert(dmem->type == T_OBJECT_EX); + assert(dmem->type == Py_T_OBJECT_EX); assert(offset > 0); cache->index = (uint16_t)offset; write_u32(cache->version, type->tp_version_tag); @@ -939,7 +939,7 @@ _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name) SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_EXPECTED_ERROR); goto fail; } - if (dmem->flags & READONLY) { + if (dmem->flags & Py_READONLY) { SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_ATTR_READ_ONLY); goto fail; } @@ -947,7 +947,7 @@ _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name) SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_OUT_OF_RANGE); goto fail; } - assert(dmem->type == T_OBJECT_EX); + assert(dmem->type == Py_T_OBJECT_EX); assert(offset > 0); cache->index = (uint16_t)offset; write_u32(cache->version, type->tp_version_tag); diff --git a/Python/structmember.c b/Python/structmember.c index 19a75224a0f32..7a5a6a49d2311 100644 --- a/Python/structmember.c +++ b/Python/structmember.c @@ -2,7 +2,7 @@ /* Map C struct members to Python object attributes */ #include "Python.h" -#include "structmember.h" // PyMemberDef + PyObject * PyMember_GetOne(const char *obj_addr, PyMemberDef *l) @@ -17,62 +17,62 @@ PyMember_GetOne(const char *obj_addr, PyMemberDef *l) const char* addr = obj_addr + l->offset; switch (l->type) { - case T_BOOL: + case Py_T_BOOL: v = PyBool_FromLong(*(char*)addr); break; - case T_BYTE: + case Py_T_BYTE: v = PyLong_FromLong(*(char*)addr); break; - case T_UBYTE: + case Py_T_UBYTE: v = PyLong_FromUnsignedLong(*(unsigned char*)addr); break; - case T_SHORT: + case Py_T_SHORT: v = PyLong_FromLong(*(short*)addr); break; - case T_USHORT: + case Py_T_USHORT: v = PyLong_FromUnsignedLong(*(unsigned short*)addr); break; - case T_INT: + case Py_T_INT: v = PyLong_FromLong(*(int*)addr); break; - case T_UINT: + case Py_T_UINT: v = PyLong_FromUnsignedLong(*(unsigned int*)addr); break; - case T_LONG: + case Py_T_LONG: v = PyLong_FromLong(*(long*)addr); break; - case T_ULONG: + case Py_T_ULONG: v = PyLong_FromUnsignedLong(*(unsigned long*)addr); break; - case T_PYSSIZET: + case Py_T_PYSSIZET: v = PyLong_FromSsize_t(*(Py_ssize_t*)addr); break; - case T_FLOAT: + case Py_T_FLOAT: v = PyFloat_FromDouble((double)*(float*)addr); break; - case T_DOUBLE: + case Py_T_DOUBLE: v = PyFloat_FromDouble(*(double*)addr); break; - case T_STRING: + case Py_T_STRING: if (*(char**)addr == NULL) { v = Py_NewRef(Py_None); } else v = PyUnicode_FromString(*(char**)addr); break; - case T_STRING_INPLACE: + case Py_T_STRING_INPLACE: v = PyUnicode_FromString((char*)addr); break; - case T_CHAR: + case Py_T_CHAR: v = PyUnicode_FromStringAndSize((char*)addr, 1); break; - case T_OBJECT: + case _Py_T_OBJECT: v = *(PyObject **)addr; if (v == NULL) v = Py_None; Py_INCREF(v); break; - case T_OBJECT_EX: + case Py_T_OBJECT_EX: v = *(PyObject **)addr; if (v == NULL) { PyObject *obj = (PyObject *)obj_addr; @@ -83,13 +83,13 @@ PyMember_GetOne(const char *obj_addr, PyMemberDef *l) } Py_XINCREF(v); break; - case T_LONGLONG: + case Py_T_LONGLONG: v = PyLong_FromLongLong(*(long long *)addr); break; - case T_ULONGLONG: + case Py_T_ULONGLONG: v = PyLong_FromUnsignedLongLong(*(unsigned long long *)addr); break; - case T_NONE: + case _Py_T_NONE: v = Py_NewRef(Py_None); break; default: @@ -118,27 +118,27 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) addr += l->offset; - if ((l->flags & READONLY)) + if ((l->flags & Py_READONLY)) { PyErr_SetString(PyExc_AttributeError, "readonly attribute"); return -1; } if (v == NULL) { - if (l->type == T_OBJECT_EX) { + if (l->type == Py_T_OBJECT_EX) { /* Check if the attribute is set. */ if (*(PyObject **)addr == NULL) { PyErr_SetString(PyExc_AttributeError, l->name); return -1; } } - else if (l->type != T_OBJECT) { + else if (l->type != _Py_T_OBJECT) { PyErr_SetString(PyExc_TypeError, "can't delete numeric/char attribute"); return -1; } } switch (l->type) { - case T_BOOL:{ + case Py_T_BOOL:{ if (!PyBool_Check(v)) { PyErr_SetString(PyExc_TypeError, "attribute value type must be bool"); @@ -150,7 +150,7 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) *(char*)addr = (char) 0; break; } - case T_BYTE:{ + case Py_T_BYTE:{ long long_val = PyLong_AsLong(v); if ((long_val == -1) && PyErr_Occurred()) return -1; @@ -161,7 +161,7 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) WARN("Truncation of value to char"); break; } - case T_UBYTE:{ + case Py_T_UBYTE:{ long long_val = PyLong_AsLong(v); if ((long_val == -1) && PyErr_Occurred()) return -1; @@ -170,7 +170,7 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) WARN("Truncation of value to unsigned char"); break; } - case T_SHORT:{ + case Py_T_SHORT:{ long long_val = PyLong_AsLong(v); if ((long_val == -1) && PyErr_Occurred()) return -1; @@ -179,7 +179,7 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) WARN("Truncation of value to short"); break; } - case T_USHORT:{ + case Py_T_USHORT:{ long long_val = PyLong_AsLong(v); if ((long_val == -1) && PyErr_Occurred()) return -1; @@ -188,7 +188,7 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) WARN("Truncation of value to unsigned short"); break; } - case T_INT:{ + case Py_T_INT:{ long long_val = PyLong_AsLong(v); if ((long_val == -1) && PyErr_Occurred()) return -1; @@ -197,7 +197,7 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) WARN("Truncation of value to int"); break; } - case T_UINT:{ + case Py_T_UINT:{ unsigned long ulong_val = PyLong_AsUnsignedLong(v); if ((ulong_val == (unsigned long)-1) && PyErr_Occurred()) { /* XXX: For compatibility, accept negative int values @@ -215,13 +215,13 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) WARN("Truncation of value to unsigned int"); break; } - case T_LONG:{ + case Py_T_LONG:{ *(long*)addr = PyLong_AsLong(v); if ((*(long*)addr == -1) && PyErr_Occurred()) return -1; break; } - case T_ULONG:{ + case Py_T_ULONG:{ *(unsigned long*)addr = PyLong_AsUnsignedLong(v); if ((*(unsigned long*)addr == (unsigned long)-1) && PyErr_Occurred()) { @@ -236,32 +236,32 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) } break; } - case T_PYSSIZET:{ + case Py_T_PYSSIZET:{ *(Py_ssize_t*)addr = PyLong_AsSsize_t(v); if ((*(Py_ssize_t*)addr == (Py_ssize_t)-1) && PyErr_Occurred()) return -1; break; } - case T_FLOAT:{ + case Py_T_FLOAT:{ double double_val = PyFloat_AsDouble(v); if ((double_val == -1) && PyErr_Occurred()) return -1; *(float*)addr = (float)double_val; break; } - case T_DOUBLE: + case Py_T_DOUBLE: *(double*)addr = PyFloat_AsDouble(v); if ((*(double*)addr == -1) && PyErr_Occurred()) return -1; break; - case T_OBJECT: - case T_OBJECT_EX: + case _Py_T_OBJECT: + case Py_T_OBJECT_EX: oldv = *(PyObject **)addr; *(PyObject **)addr = Py_XNewRef(v); Py_XDECREF(oldv); break; - case T_CHAR: { + case Py_T_CHAR: { const char *string; Py_ssize_t len; @@ -273,18 +273,18 @@ PyMember_SetOne(char *addr, PyMemberDef *l, PyObject *v) *(char*)addr = string[0]; break; } - case T_STRING: - case T_STRING_INPLACE: + case Py_T_STRING: + case Py_T_STRING_INPLACE: PyErr_SetString(PyExc_TypeError, "readonly attribute"); return -1; - case T_LONGLONG:{ + case Py_T_LONGLONG:{ long long value; *(long long*)addr = value = PyLong_AsLongLong(v); if ((value == -1) && PyErr_Occurred()) return -1; break; } - case T_ULONGLONG:{ + case Py_T_ULONGLONG:{ unsigned long long value; /* ??? PyLong_AsLongLong accepts an int, but PyLong_AsUnsignedLongLong doesn't ??? */ diff --git a/Python/symtable.c b/Python/symtable.c index 2407367a58a9b..04be3192d6c7c 100644 --- a/Python/symtable.c +++ b/Python/symtable.c @@ -3,7 +3,7 @@ #include "pycore_parser.h" // _PyParser_ASTFromString() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_symtable.h" // PySTEntryObject -#include "structmember.h" // PyMemberDef + /* error strings used for warnings */ #define GLOBAL_PARAM \ @@ -171,14 +171,14 @@ ste_dealloc(PySTEntryObject *ste) #define OFF(x) offsetof(PySTEntryObject, x) static PyMemberDef ste_memberlist[] = { - {"id", T_OBJECT, OFF(ste_id), READONLY}, - {"name", T_OBJECT, OFF(ste_name), READONLY}, - {"symbols", T_OBJECT, OFF(ste_symbols), READONLY}, - {"varnames", T_OBJECT, OFF(ste_varnames), READONLY}, - {"children", T_OBJECT, OFF(ste_children), READONLY}, - {"nested", T_INT, OFF(ste_nested), READONLY}, - {"type", T_INT, OFF(ste_type), READONLY}, - {"lineno", T_INT, OFF(ste_lineno), READONLY}, + {"id", _Py_T_OBJECT, OFF(ste_id), Py_READONLY}, + {"name", _Py_T_OBJECT, OFF(ste_name), Py_READONLY}, + {"symbols", _Py_T_OBJECT, OFF(ste_symbols), Py_READONLY}, + {"varnames", _Py_T_OBJECT, OFF(ste_varnames), Py_READONLY}, + {"children", _Py_T_OBJECT, OFF(ste_children), Py_READONLY}, + {"nested", Py_T_INT, OFF(ste_nested), Py_READONLY}, + {"type", Py_T_INT, OFF(ste_type), Py_READONLY}, + {"lineno", Py_T_INT, OFF(ste_lineno), Py_READONLY}, {NULL} }; diff --git a/Python/traceback.c b/Python/traceback.c index c5787b5ea4678..ca524b1b9af78 100644 --- a/Python/traceback.c +++ b/Python/traceback.c @@ -17,7 +17,7 @@ #include "../Parser/pegen.h" // _PyPegen_byte_offset_to_character_offset() #include "frameobject.h" // PyFrame_New() -#include "structmember.h" // PyMemberDef + #include "osdefs.h" // SEP #ifdef HAVE_FCNTL_H # include @@ -148,9 +148,9 @@ static PyMethodDef tb_methods[] = { }; static PyMemberDef tb_memberlist[] = { - {"tb_frame", T_OBJECT, OFF(tb_frame), READONLY|PY_AUDIT_READ}, - {"tb_lasti", T_INT, OFF(tb_lasti), READONLY}, - {"tb_lineno", T_INT, OFF(tb_lineno), READONLY}, + {"tb_frame", _Py_T_OBJECT, OFF(tb_frame), Py_READONLY|Py_AUDIT_READ}, + {"tb_lasti", Py_T_INT, OFF(tb_lasti), Py_READONLY}, + {"tb_lineno", Py_T_INT, OFF(tb_lineno), Py_READONLY}, {NULL} /* Sentinel */ }; From webhook-mailer at python.org Tue Jul 25 09:43:15 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 13:43:15 -0000 Subject: [Python-checkins] [3.12] gh-107237: Fix test_udp_reconnection() of test_logging (GH-107238) (#107242) Message-ID: https://github.com/python/cpython/commit/e788c0aeeb46bdfa3c4d8a39fb9bbb505e6d27a8 commit: e788c0aeeb46bdfa3c4d8a39fb9bbb505e6d27a8 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-25T13:43:11Z summary: [3.12] gh-107237: Fix test_udp_reconnection() of test_logging (GH-107238) (#107242) gh-107237: Fix test_udp_reconnection() of test_logging (GH-107238) test_logging: Fix test_udp_reconnection() by increasing the timeout from 100 ms to 5 minutes (LONG_TIMEOUT). Replace also blocking wait() with wait(LONG_TIMEOUT) in test_output() to prevent the test to hang. (cherry picked from commit ed082383272c2c238e364e9cc83229234aee23cc) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst M Lib/test/test_logging.py diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index 18258c22874ae..def976fbe96ba 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -2079,17 +2079,17 @@ def test_output(self): # The log message sent to the SysLogHandler is properly received. logger = logging.getLogger("slh") logger.error("sp\xe4m") - self.handled.wait() + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>sp\xc3\xa4m\x00') self.handled.clear() self.sl_hdlr.append_nul = False logger.error("sp\xe4m") - self.handled.wait() + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>sp\xc3\xa4m') self.handled.clear() self.sl_hdlr.ident = "h\xe4m-" logger.error("sp\xe4m") - self.handled.wait() + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>h\xc3\xa4m-sp\xc3\xa4m') def test_udp_reconnection(self): @@ -2097,7 +2097,7 @@ def test_udp_reconnection(self): self.sl_hdlr.close() self.handled.clear() logger.error("sp\xe4m") - self.handled.wait(0.1) + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>sp\xc3\xa4m\x00') @unittest.skipUnless(hasattr(socket, "AF_UNIX"), "Unix sockets required") diff --git a/Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst b/Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst new file mode 100644 index 0000000000000..a04f7eeddef17 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst @@ -0,0 +1,2 @@ +``test_logging``: Fix ``test_udp_reconnection()`` by increasing the timeout +from 100 ms to 5 minutes (LONG_TIMEOUT). Patch by Victor Stinner. From webhook-mailer at python.org Tue Jul 25 10:45:42 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 14:45:42 -0000 Subject: [Python-checkins] gh-105059: Fix MSCV compiler warning on PyObject union (#107239) Message-ID: https://github.com/python/cpython/commit/1c8fe9bdb624d356643ee569151a9e4f2963179a commit: 1c8fe9bdb624d356643ee569151a9e4f2963179a branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T16:45:38+02:00 summary: gh-105059: Fix MSCV compiler warning on PyObject union (#107239) Use pragma to ignore the MSCV compiler warning on the PyObject nameless union. files: M Include/object.h diff --git a/Include/object.h b/Include/object.h index 2488d6cd0d40e..e26cedf8ca3c9 100644 --- a/Include/object.h +++ b/Include/object.h @@ -165,10 +165,17 @@ check by comparing the reference count field to the immortality reference count. */ struct _object { _PyObject_HEAD_EXTRA + #if (defined(__GNUC__) || defined(__clang__)) \ && !(defined __STDC_VERSION__ && __STDC_VERSION__ >= 201112L) // On C99 and older, anonymous union is a GCC and clang extension __extension__ +#endif +#ifdef _MSC_VER + // Ignore MSC warning C4201: "nonstandard extension used: + // nameless struct/union" + __pragma(warning(push)) + __pragma(warning(disable: 4201)) #endif union { Py_ssize_t ob_refcnt; @@ -176,6 +183,10 @@ struct _object { PY_UINT32_T ob_refcnt_split[2]; #endif }; +#ifdef _MSC_VER + __pragma(warning(pop)) +#endif + PyTypeObject *ob_type; }; From webhook-mailer at python.org Tue Jul 25 10:49:37 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 14:49:37 -0000 Subject: [Python-checkins] [3.11] gh-107237: Fix test_udp_reconnection() of test_logging (#107238) (#107245) Message-ID: https://github.com/python/cpython/commit/db03cb953a26970e1affe4e8fc7a8da79a3d229a commit: db03cb953a26970e1affe4e8fc7a8da79a3d229a branch: 3.11 author: Victor Stinner committer: vstinner date: 2023-07-25T14:49:32Z summary: [3.11] gh-107237: Fix test_udp_reconnection() of test_logging (#107238) (#107245) gh-107237: Fix test_udp_reconnection() of test_logging (#107238) test_logging: Fix test_udp_reconnection() by increasing the timeout from 100 ms to 5 minutes (LONG_TIMEOUT). Replace also blocking wait() with wait(LONG_TIMEOUT) in test_output() to prevent the test to hang. (cherry picked from commit ed082383272c2c238e364e9cc83229234aee23cc) files: A Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst M Lib/test/test_logging.py diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index 76bd8724e29b4..f1f2b75fef869 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -1981,17 +1981,17 @@ def test_output(self): # The log message sent to the SysLogHandler is properly received. logger = logging.getLogger("slh") logger.error("sp\xe4m") - self.handled.wait() + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>sp\xc3\xa4m\x00') self.handled.clear() self.sl_hdlr.append_nul = False logger.error("sp\xe4m") - self.handled.wait() + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>sp\xc3\xa4m') self.handled.clear() self.sl_hdlr.ident = "h\xe4m-" logger.error("sp\xe4m") - self.handled.wait() + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>h\xc3\xa4m-sp\xc3\xa4m') def test_udp_reconnection(self): @@ -1999,7 +1999,7 @@ def test_udp_reconnection(self): self.sl_hdlr.close() self.handled.clear() logger.error("sp\xe4m") - self.handled.wait(0.1) + self.handled.wait(support.LONG_TIMEOUT) self.assertEqual(self.log_output, b'<11>sp\xc3\xa4m\x00') @unittest.skipUnless(hasattr(socket, "AF_UNIX"), "Unix sockets required") diff --git a/Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst b/Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst new file mode 100644 index 0000000000000..a04f7eeddef17 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2023-07-25-14-36-33.gh-issue-107237.y1pY79.rst @@ -0,0 +1,2 @@ +``test_logging``: Fix ``test_udp_reconnection()`` by increasing the timeout +from 100 ms to 5 minutes (LONG_TIMEOUT). Patch by Victor Stinner. From webhook-mailer at python.org Tue Jul 25 11:35:53 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 15:35:53 -0000 Subject: [Python-checkins] [3.12] gh-105059: Fix MSCV compiler warning on PyObject union (GH-107239) (#107248) Message-ID: https://github.com/python/cpython/commit/9c31d9405027cea9c2d039ade672d604663ed5b0 commit: 9c31d9405027cea9c2d039ade672d604663ed5b0 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-25T15:35:49Z summary: [3.12] gh-105059: Fix MSCV compiler warning on PyObject union (GH-107239) (#107248) gh-105059: Fix MSCV compiler warning on PyObject union (GH-107239) Use pragma to ignore the MSCV compiler warning on the PyObject nameless union. (cherry picked from commit 1c8fe9bdb624d356643ee569151a9e4f2963179a) Co-authored-by: Victor Stinner files: M Include/object.h diff --git a/Include/object.h b/Include/object.h index 542f8d8c15a7c..77434e3bc73c1 100644 --- a/Include/object.h +++ b/Include/object.h @@ -165,10 +165,17 @@ check by comparing the reference count field to the immortality reference count. */ struct _object { _PyObject_HEAD_EXTRA + #if (defined(__GNUC__) || defined(__clang__)) \ && !(defined __STDC_VERSION__ && __STDC_VERSION__ >= 201112L) // On C99 and older, anonymous union is a GCC and clang extension __extension__ +#endif +#ifdef _MSC_VER + // Ignore MSC warning C4201: "nonstandard extension used: + // nameless struct/union" + __pragma(warning(push)) + __pragma(warning(disable: 4201)) #endif union { Py_ssize_t ob_refcnt; @@ -176,6 +183,10 @@ struct _object { PY_UINT32_T ob_refcnt_split[2]; #endif }; +#ifdef _MSC_VER + __pragma(warning(pop)) +#endif + PyTypeObject *ob_type; }; From webhook-mailer at python.org Tue Jul 25 11:42:50 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Tue, 25 Jul 2023 15:42:50 -0000 Subject: [Python-checkins] [3.12] gh-105699: Add some stress tests for subinterpreter creation (GH-106966) (gh-107012) Message-ID: https://github.com/python/cpython/commit/ca42d6720850967698d6e298e37a4225f0c67d8e commit: ca42d6720850967698d6e298e37a4225f0c67d8e branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ericsnowcurrently date: 2023-07-25T09:42:46-06:00 summary: [3.12] gh-105699: Add some stress tests for subinterpreter creation (GH-106966) (gh-107012) gh-105699: Add some stress tests for subinterpreter creation (GH-106966) (cherry picked from commit adda43dc0bcea853cbfa33126e5549c584cef8be) Co-authored-by: Eric Snow files: M Lib/test/test_interpreters.py diff --git a/Lib/test/test_interpreters.py b/Lib/test/test_interpreters.py index d1bebe4715832..5981d96de8de0 100644 --- a/Lib/test/test_interpreters.py +++ b/Lib/test/test_interpreters.py @@ -7,6 +7,7 @@ from test import support from test.support import import_helper +from test.support import threading_helper _interpreters = import_helper.import_module('_xxsubinterpreters') _channels = import_helper.import_module('_xxinterpchannels') from test.support import interpreters @@ -463,6 +464,27 @@ def test_bytes_for_script(self): # test_xxsubinterpreters covers the remaining Interpreter.run() behavior. +class StressTests(TestBase): + + # In these tests we generally want a lot of interpreters, + # but not so many that any test takes too long. + + def test_create_many_sequential(self): + alive = [] + for _ in range(100): + interp = interpreters.create() + alive.append(interp) + + def test_create_many_threaded(self): + alive = [] + def task(): + interp = interpreters.create() + alive.append(interp) + threads = (threading.Thread(target=task) for _ in range(200)) + with threading_helper.start_threads(threads): + pass + + class TestIsShareable(TestBase): def test_default_shareables(self): From webhook-mailer at python.org Tue Jul 25 12:25:11 2023 From: webhook-mailer at python.org (ezio-melotti) Date: Tue, 25 Jul 2023 16:25:11 -0000 Subject: [Python-checkins] [3.12] Remove superflous whitespaces in `layout.html`. (#107251) Message-ID: https://github.com/python/cpython/commit/313284aa423252ebd5d4e761220e0f4fdeac626d commit: 313284aa423252ebd5d4e761220e0f4fdeac626d branch: 3.12 author: Ezio Melotti committer: ezio-melotti date: 2023-07-25T16:25:07Z summary: [3.12] Remove superflous whitespaces in `layout.html`. (#107251) Remove superflous whitespaces in layout.html. files: M Doc/tools/templates/layout.html diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index 10cb6fd880fa1..9832feba14167 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -4,16 +4,16 @@ {%- if outdated %}
{% trans %}This document is for an old version of Python that is no longer supported. - You should upgrade, and read the {% endtrans %} - {% trans %} Python documentation for the current stable release{% endtrans %}. + You should upgrade, and read the{% endtrans %} + {% trans %}Python documentation for the current stable release{% endtrans %}.
{%- endif %} {%- if is_deployment_preview %}
{% trans %}This is a deploy preview created from a pull request. - For authoritative documentation, see the {% endtrans %} - {% trans %} the current stable release{% endtrans %}. + For authoritative documentation, see{% endtrans %} + {% trans %}the current stable release{% endtrans %}.
{%- endif %} {% endblock %} From webhook-mailer at python.org Tue Jul 25 12:36:46 2023 From: webhook-mailer at python.org (ezio-melotti) Date: Tue, 25 Jul 2023 16:36:46 -0000 Subject: [Python-checkins] [3.11] Remove superflous whitespaces in `layout.html`. (GH-107251) (#107252) Message-ID: https://github.com/python/cpython/commit/012bc1329a01bd92b0e09d87db9b70a242e06e3a commit: 012bc1329a01bd92b0e09d87db9b70a242e06e3a branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ezio-melotti date: 2023-07-25T16:34:56Z summary: [3.11] Remove superflous whitespaces in `layout.html`. (GH-107251) (#107252) Remove superflous whitespaces in `layout.html`. (GH-107251) Remove superflous whitespaces in layout.html. (cherry picked from commit 313284aa423252ebd5d4e761220e0f4fdeac626d) Co-authored-by: Ezio Melotti files: M Doc/tools/templates/layout.html diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index 10cb6fd880fa1..9832feba14167 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -4,16 +4,16 @@ {%- if outdated %}
{% trans %}This document is for an old version of Python that is no longer supported. - You should upgrade, and read the {% endtrans %} - {% trans %} Python documentation for the current stable release{% endtrans %}. + You should upgrade, and read the{% endtrans %} + {% trans %}Python documentation for the current stable release{% endtrans %}.
{%- endif %} {%- if is_deployment_preview %}
{% trans %}This is a deploy preview created from a pull request. - For authoritative documentation, see the {% endtrans %} - {% trans %} the current stable release{% endtrans %}. + For authoritative documentation, see{% endtrans %} + {% trans %}the current stable release{% endtrans %}.
{%- endif %} {% endblock %} From webhook-mailer at python.org Tue Jul 25 13:02:17 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 17:02:17 -0000 Subject: [Python-checkins] Remove unused internal _PyImport_GetModuleId() function (#107235) Message-ID: https://github.com/python/cpython/commit/188000ae4b486cfffaeaa0a06902e4844d1b2bbc commit: 188000ae4b486cfffaeaa0a06902e4844d1b2bbc branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T17:02:12Z summary: Remove unused internal _PyImport_GetModuleId() function (#107235) files: M Include/internal/pycore_import.h M Python/import.c diff --git a/Include/internal/pycore_import.h b/Include/internal/pycore_import.h index c12ef7cebadc2..f61f0af6bb263 100644 --- a/Include/internal/pycore_import.h +++ b/Include/internal/pycore_import.h @@ -9,7 +9,6 @@ extern "C" { extern int _PyImport_IsInitialized(PyInterpreterState *); -extern PyObject* _PyImport_GetModuleId(_Py_Identifier *name); PyAPI_FUNC(int) _PyImport_SetModule(PyObject *name, PyObject *module); extern int _PyImport_SetModuleString(const char *name, PyObject* module); diff --git a/Python/import.c b/Python/import.c index cf993cbd62a2e..f8cae641a6454 100644 --- a/Python/import.c +++ b/Python/import.c @@ -210,17 +210,6 @@ PyImport_GetModuleDict(void) return MODULES(interp); } -// This is only kept around for extensions that use _Py_IDENTIFIER. -PyObject * -_PyImport_GetModuleId(_Py_Identifier *nameid) -{ - PyObject *name = _PyUnicode_FromId(nameid); /* borrowed */ - if (name == NULL) { - return NULL; - } - return PyImport_GetModule(name); -} - int _PyImport_SetModule(PyObject *name, PyObject *m) { From webhook-mailer at python.org Tue Jul 25 13:27:40 2023 From: webhook-mailer at python.org (pitrou) Date: Tue, 25 Jul 2023 17:27:40 -0000 Subject: [Python-checkins] gh-106739: Add `rtype_cache` to `warnings.warn` message when leaked objects found (#106740) Message-ID: https://github.com/python/cpython/commit/fabcbe9c12688eb9a902a5c89cb720ed373625c5 commit: fabcbe9c12688eb9a902a5c89cb720ed373625c5 branch: main author: shailshouryya <42100758+shailshouryya at users.noreply.github.com> committer: pitrou date: 2023-07-25T17:27:36Z summary: gh-106739: Add `rtype_cache` to `warnings.warn` message when leaked objects found (#106740) Adding the `rtype_cache` to the `warnings.warn` message improves the previous, somewhat vague message from ``` /Users/username/cpython/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown ``` to ``` /Users/username/cpython/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown: {'/mp-yor5cvj8', '/mp-10jx8eqr', '/mp-eobsx9tt', '/mp-0lml23vl', '/mp-9dgtsa_m', '/mp-frntyv4s'} ``` --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2023-07-20-06-00-35.gh-issue-106739.W1hygr.rst M Lib/multiprocessing/resource_tracker.py diff --git a/Lib/multiprocessing/resource_tracker.py b/Lib/multiprocessing/resource_tracker.py index ea369507297f8..3783c1ffc6e4a 100644 --- a/Lib/multiprocessing/resource_tracker.py +++ b/Lib/multiprocessing/resource_tracker.py @@ -221,9 +221,10 @@ def main(fd): for rtype, rtype_cache in cache.items(): if rtype_cache: try: - warnings.warn('resource_tracker: There appear to be %d ' - 'leaked %s objects to clean up at shutdown' % - (len(rtype_cache), rtype)) + warnings.warn( + f'resource_tracker: There appear to be {len(rtype_cache)} ' + f'leaked {rtype} objects to clean up at shutdown: {rtype_cache}' + ) except Exception: pass for name in rtype_cache: diff --git a/Misc/NEWS.d/next/Library/2023-07-20-06-00-35.gh-issue-106739.W1hygr.rst b/Misc/NEWS.d/next/Library/2023-07-20-06-00-35.gh-issue-106739.W1hygr.rst new file mode 100644 index 0000000000000..168e201939569 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-20-06-00-35.gh-issue-106739.W1hygr.rst @@ -0,0 +1 @@ +Add the ``rtype_cache`` to the warning message (as an addition to the type of leaked objects and the number of leaked objects already included in the message) to make debugging leaked objects easier when the multiprocessing resource tracker process finds leaked objects at shutdown. This helps more quickly identify what was leaked and/or why the leaked object was not properly cleaned up. From webhook-mailer at python.org Tue Jul 25 13:28:20 2023 From: webhook-mailer at python.org (vstinner) Date: Tue, 25 Jul 2023 17:28:20 -0000 Subject: [Python-checkins] gh-107249: Implement Py_UNUSED() for MSVC (#107250) Message-ID: https://github.com/python/cpython/commit/6a43cce32b66e0f66992119dd82959069b6f324a commit: 6a43cce32b66e0f66992119dd82959069b6f324a branch: main author: Victor Stinner committer: vstinner date: 2023-07-25T19:28:16+02:00 summary: gh-107249: Implement Py_UNUSED() for MSVC (#107250) Fix warnings C4100 in Py_UNUSED() when Python is built with "cl /W4". Example with this function included by Python.h: static inline unsigned int PyUnicode_IS_READY(PyObject* Py_UNUSED(op)) { return 1; } Without this change, building a C program with "cl /W4" which just includes Python.h emits the warning: Include\cpython/unicodeobject.h(199): warning C4100: '_unused_op': unreferenced formal parameter This change fix this warning. files: A Misc/NEWS.d/next/C API/2023-07-25-17-23-08.gh-issue-107249.xqk2ke.rst M Include/pymacro.h diff --git a/Include/pymacro.h b/Include/pymacro.h index 342d2a7b844ad..9d264fe6eea1d 100644 --- a/Include/pymacro.h +++ b/Include/pymacro.h @@ -118,6 +118,15 @@ */ #if defined(__GNUC__) || defined(__clang__) # define Py_UNUSED(name) _unused_ ## name __attribute__((unused)) +#elif defined(_MSC_VER) + // Disable warning C4100: unreferenced formal parameter, + // declare the parameter, + // restore old compiler warnings. +# define Py_UNUSED(name) \ + __pragma(warning(push)) \ + __pragma(warning(suppress: 4100)) \ + _unused_ ## name \ + __pragma(warning(pop)) #else # define Py_UNUSED(name) _unused_ ## name #endif diff --git a/Misc/NEWS.d/next/C API/2023-07-25-17-23-08.gh-issue-107249.xqk2ke.rst b/Misc/NEWS.d/next/C API/2023-07-25-17-23-08.gh-issue-107249.xqk2ke.rst new file mode 100644 index 0000000000000..a7139024329fa --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-25-17-23-08.gh-issue-107249.xqk2ke.rst @@ -0,0 +1,2 @@ +Implement the :c:macro:`Py_UNUSED` macro for Windows MSVC compiler. Patch by +Victor Stinner. From webhook-mailer at python.org Tue Jul 25 14:48:11 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 18:48:11 -0000 Subject: [Python-checkins] gh-106004: PyDict_GetItemRef() should only be in the limited API 3.13 (GH-107229) Message-ID: https://github.com/python/cpython/commit/2b1a81e2cfa740609d48ad82627ae251262e6470 commit: 2b1a81e2cfa740609d48ad82627ae251262e6470 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-25T21:48:07+03:00 summary: gh-106004: PyDict_GetItemRef() should only be in the limited API 3.13 (GH-107229) files: M Include/dictobject.h diff --git a/Include/dictobject.h b/Include/dictobject.h index 26434e49f9140..1bbeec1ab699e 100644 --- a/Include/dictobject.h +++ b/Include/dictobject.h @@ -58,6 +58,7 @@ PyAPI_FUNC(PyObject *) PyDict_GetItemString(PyObject *dp, const char *key); PyAPI_FUNC(int) PyDict_SetItemString(PyObject *dp, const char *key, PyObject *item); PyAPI_FUNC(int) PyDict_DelItemString(PyObject *dp, const char *key); +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030D0000 // Return the object from dictionary *op* which has a key *key*. // - If the key is present, set *result to a new strong reference to the value // and return 1. @@ -65,6 +66,7 @@ PyAPI_FUNC(int) PyDict_DelItemString(PyObject *dp, const char *key); // - On error, raise an exception and return -1. PyAPI_FUNC(int) PyDict_GetItemRef(PyObject *mp, PyObject *key, PyObject **result); PyAPI_FUNC(int) PyDict_GetItemStringRef(PyObject *mp, const char *key, PyObject **result); +#endif #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030A0000 PyAPI_FUNC(PyObject *) PyObject_GenericGetDict(PyObject *, void *); From webhook-mailer at python.org Tue Jul 25 14:49:03 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 18:49:03 -0000 Subject: [Python-checkins] [3.12] gh-62519: Make pgettext search plurals when translation is not found (GH-107118) (GH-107134) Message-ID: https://github.com/python/cpython/commit/11d86c5c339abd52d1ab5ce0af4d25c44898d046 commit: 11d86c5c339abd52d1ab5ce0af4d25c44898d046 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-25T21:48:59+03:00 summary: [3.12] gh-62519: Make pgettext search plurals when translation is not found (GH-107118) (GH-107134) (cherry picked from commit b3c34e55c053846beb35f5e4253ef237b3494bd0) Co-authored-by: Tomas R Co-authored-by: ?ukasz Langa files: A Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst M Lib/gettext.py M Lib/test/test_gettext.py diff --git a/Lib/gettext.py b/Lib/gettext.py index cc938e40028c2..b72b15f82d435 100644 --- a/Lib/gettext.py +++ b/Lib/gettext.py @@ -446,10 +446,12 @@ def pgettext(self, context, message): missing = object() tmsg = self._catalog.get(ctxt_msg_id, missing) if tmsg is missing: - if self._fallback: - return self._fallback.pgettext(context, message) - return message - return tmsg + tmsg = self._catalog.get((ctxt_msg_id, self.plural(1)), missing) + if tmsg is not missing: + return tmsg + if self._fallback: + return self._fallback.pgettext(context, message) + return message def npgettext(self, context, msgid1, msgid2, n): ctxt_msg_id = self.CONTEXT % (context, msgid1) diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py index aa3520d2c142e..8430fc234d00e 100644 --- a/Lib/test/test_gettext.py +++ b/Lib/test/test_gettext.py @@ -331,6 +331,8 @@ def test_plural_context_forms1(self): x = gettext.npgettext('With context', 'There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros (context)') + x = gettext.pgettext('With context', 'There is %s file') + eq(x, 'Hay %s fichero (context)') def test_plural_forms2(self): eq = self.assertEqual @@ -353,6 +355,8 @@ def test_plural_context_forms2(self): x = t.npgettext('With context', 'There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros (context)') + x = gettext.pgettext('With context', 'There is %s file') + eq(x, 'Hay %s fichero (context)') # Examples from http://www.gnu.org/software/gettext/manual/gettext.html diff --git a/Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst b/Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst new file mode 100644 index 0000000000000..96e2a3dcc24fb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst @@ -0,0 +1,2 @@ +Make :func:`gettext.pgettext` search plural definitions when +translation is not found. From webhook-mailer at python.org Tue Jul 25 14:49:34 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 18:49:34 -0000 Subject: [Python-checkins] [3.11] gh-62519: Make pgettext search plurals when translation is not found (GH-107118) (GH-107133) Message-ID: https://github.com/python/cpython/commit/41756e3960a38249b9e0076412ef5e08625a7acc commit: 41756e3960a38249b9e0076412ef5e08625a7acc branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-25T21:49:28+03:00 summary: [3.11] gh-62519: Make pgettext search plurals when translation is not found (GH-107118) (GH-107133) (cherry picked from commit b3c34e55c053846beb35f5e4253ef237b3494bd0) Co-authored-by: Tomas R Co-authored-by: ?ukasz Langa files: A Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst M Lib/gettext.py M Lib/test/test_gettext.py diff --git a/Lib/gettext.py b/Lib/gettext.py index 6c5ec4e517f63..57f1a449c28a4 100644 --- a/Lib/gettext.py +++ b/Lib/gettext.py @@ -444,10 +444,12 @@ def pgettext(self, context, message): missing = object() tmsg = self._catalog.get(ctxt_msg_id, missing) if tmsg is missing: - if self._fallback: - return self._fallback.pgettext(context, message) - return message - return tmsg + tmsg = self._catalog.get((ctxt_msg_id, self.plural(1)), missing) + if tmsg is not missing: + return tmsg + if self._fallback: + return self._fallback.pgettext(context, message) + return message def npgettext(self, context, msgid1, msgid2, n): ctxt_msg_id = self.CONTEXT % (context, msgid1) diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py index 1608d1b18e98f..7f7b51c19f35c 100644 --- a/Lib/test/test_gettext.py +++ b/Lib/test/test_gettext.py @@ -329,6 +329,8 @@ def test_plural_context_forms1(self): x = gettext.npgettext('With context', 'There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros (context)') + x = gettext.pgettext('With context', 'There is %s file') + eq(x, 'Hay %s fichero (context)') def test_plural_forms2(self): eq = self.assertEqual @@ -349,6 +351,8 @@ def test_plural_context_forms2(self): x = t.npgettext('With context', 'There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros (context)') + x = gettext.pgettext('With context', 'There is %s file') + eq(x, 'Hay %s fichero (context)') # Examples from http://www.gnu.org/software/gettext/manual/gettext.html diff --git a/Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst b/Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst new file mode 100644 index 0000000000000..96e2a3dcc24fb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-23-12-26-23.gh-issue-62519.w8-81X.rst @@ -0,0 +1,2 @@ +Make :func:`gettext.pgettext` search plural definitions when +translation is not found. From webhook-mailer at python.org Tue Jul 25 14:52:11 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 18:52:11 -0000 Subject: [Python-checkins] gh-106350: Tkinter: do not ignore return value of `mp_init()` (GH-106351) Message-ID: https://github.com/python/cpython/commit/b5ae7c498438657a6ba0bf4cc216b9c2c93a06c7 commit: b5ae7c498438657a6ba0bf4cc216b9c2c93a06c7 branch: main author: Christopher Chavez committer: serhiy-storchaka date: 2023-07-25T21:52:07+03:00 summary: gh-106350: Tkinter: do not ignore return value of `mp_init()` (GH-106351) files: A Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst M Modules/_tkinter.c diff --git a/Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst b/Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst new file mode 100644 index 0000000000000..681d63a6668be --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst @@ -0,0 +1,2 @@ +Detect possible memory allocation failure in the libtommath function :c:func:`mp_init` +used by the ``_tkinter`` module. diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c index 1605606e8a470..145a2940427e6 100644 --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -874,8 +874,9 @@ asBignumObj(PyObject *value) return NULL; } hexchars += neg + 2; /* skip sign and "0x" */ - mp_init(&bigValue); - if (mp_read_radix(&bigValue, hexchars, 16) != MP_OKAY) { + if (mp_init(&bigValue) != MP_OKAY || + mp_read_radix(&bigValue, hexchars, 16) != MP_OKAY) + { mp_clear(&bigValue); Py_DECREF(hexstr); PyErr_NoMemory(); From webhook-mailer at python.org Tue Jul 25 15:01:50 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 19:01:50 -0000 Subject: [Python-checkins] gh-107226: PyModule_AddObjectRef() should only be in the limited API 3.10 (GH-107227) Message-ID: https://github.com/python/cpython/commit/698b01513550798886add5e06a1c3f9a89d7dfc6 commit: 698b01513550798886add5e06a1c3f9a89d7dfc6 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-25T22:01:45+03:00 summary: gh-107226: PyModule_AddObjectRef() should only be in the limited API 3.10 (GH-107227) files: A Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst M Include/modsupport.h diff --git a/Include/modsupport.h b/Include/modsupport.h index 51061c5bc8090..88577e027b527 100644 --- a/Include/modsupport.h +++ b/Include/modsupport.h @@ -22,10 +22,12 @@ PyAPI_FUNC(int) PyArg_UnpackTuple(PyObject *, const char *, Py_ssize_t, Py_ssize PyAPI_FUNC(PyObject *) Py_BuildValue(const char *, ...); PyAPI_FUNC(PyObject *) Py_VaBuildValue(const char *, va_list); +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030a0000 // Add an attribute with name 'name' and value 'obj' to the module 'mod. // On success, return 0. // On error, raise an exception and return -1. PyAPI_FUNC(int) PyModule_AddObjectRef(PyObject *mod, const char *name, PyObject *value); +#endif /* Py_LIMITED_API */ #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030d0000 // Similar to PyModule_AddObjectRef() but steal a reference to 'value'. diff --git a/Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst b/Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst new file mode 100644 index 0000000000000..6178f18517d48 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst @@ -0,0 +1,2 @@ +:c:func:`PyModule_AddObjectRef` is now only available in the limited API +version 3.10 or later. From webhook-mailer at python.org Tue Jul 25 16:01:06 2023 From: webhook-mailer at python.org (gvanrossum) Date: Tue, 25 Jul 2023 20:01:06 -0000 Subject: [Python-checkins] gh-107082: Fix instruction size computation for ENTER_EXECUTOR (#107256) Message-ID: https://github.com/python/cpython/commit/233b8782886939176982a90f563d552757cbf34e commit: 233b8782886939176982a90f563d552757cbf34e branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-25T20:01:02Z summary: gh-107082: Fix instruction size computation for ENTER_EXECUTOR (#107256) Co-authored-by: Victor Stinner files: M Python/instrumentation.c diff --git a/Python/instrumentation.c b/Python/instrumentation.c index e1b62bb32ec8b..c3515d2c5a3ad 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -276,6 +276,13 @@ _PyInstruction_GetLength(PyCodeObject *code, int offset) } assert(opcode != 0); assert(!is_instrumented(opcode)); + if (opcode == ENTER_EXECUTOR) { + int exec_index = _PyCode_CODE(code)[offset].op.arg; + _PyExecutorObject *exec = code->co_executors->executors[exec_index]; + opcode = exec->vm_data.opcode; + + } + assert(opcode != ENTER_EXECUTOR); assert(opcode == _PyOpcode_Deopt[opcode]); return 1 + _PyOpcode_Caches[opcode]; } From webhook-mailer at python.org Tue Jul 25 16:01:22 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 20:01:22 -0000 Subject: [Python-checkins] [3.12] gh-107226: PyModule_AddObjectRef() should only be in the limited API 3.10 (GH-107227) (GH-107260) Message-ID: https://github.com/python/cpython/commit/c3c8916dea0142736efb087ea08bf667dfcf3c50 commit: c3c8916dea0142736efb087ea08bf667dfcf3c50 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-25T23:01:18+03:00 summary: [3.12] gh-107226: PyModule_AddObjectRef() should only be in the limited API 3.10 (GH-107227) (GH-107260) (cherry picked from commit 698b01513550798886add5e06a1c3f9a89d7dfc6) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst M Include/modsupport.h diff --git a/Include/modsupport.h b/Include/modsupport.h index 4e369bd56b4d2..1592bd0db4ffd 100644 --- a/Include/modsupport.h +++ b/Include/modsupport.h @@ -39,10 +39,12 @@ PyAPI_FUNC(PyObject *) _Py_BuildValue_SizeT(const char *, ...); PyAPI_FUNC(PyObject *) Py_VaBuildValue(const char *, va_list); +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030a0000 // Add an attribute with name 'name' and value 'obj' to the module 'mod. // On success, return 0 on success. // On error, raise an exception and return -1. PyAPI_FUNC(int) PyModule_AddObjectRef(PyObject *mod, const char *name, PyObject *value); +#endif /* Py_LIMITED_API */ // Similar to PyModule_AddObjectRef() but steal a reference to 'obj' // (Py_DECREF(obj)) on success (if it returns 0). diff --git a/Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst b/Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst new file mode 100644 index 0000000000000..6178f18517d48 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst @@ -0,0 +1,2 @@ +:c:func:`PyModule_AddObjectRef` is now only available in the limited API +version 3.10 or later. From webhook-mailer at python.org Tue Jul 25 16:02:09 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 20:02:09 -0000 Subject: [Python-checkins] [3.11] gh-107226: PyModule_AddObjectRef() should only be in the limited API 3.10 (GH-107227) (GH-107261) Message-ID: https://github.com/python/cpython/commit/2ce8e133d0ed13d439477659d6349c26728d839c commit: 2ce8e133d0ed13d439477659d6349c26728d839c branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-25T23:02:06+03:00 summary: [3.11] gh-107226: PyModule_AddObjectRef() should only be in the limited API 3.10 (GH-107227) (GH-107261) (cherry picked from commit 698b01513550798886add5e06a1c3f9a89d7dfc6) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst M Include/modsupport.h diff --git a/Include/modsupport.h b/Include/modsupport.h index 0e96a5c988846..b3813a7e31935 100644 --- a/Include/modsupport.h +++ b/Include/modsupport.h @@ -41,10 +41,12 @@ PyAPI_FUNC(PyObject *) _Py_BuildValue_SizeT(const char *, ...); PyAPI_FUNC(PyObject *) Py_VaBuildValue(const char *, va_list); +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030a0000 // Add an attribute with name 'name' and value 'obj' to the module 'mod. // On success, return 0 on success. // On error, raise an exception and return -1. PyAPI_FUNC(int) PyModule_AddObjectRef(PyObject *mod, const char *name, PyObject *value); +#endif /* Py_LIMITED_API */ // Similar to PyModule_AddObjectRef() but steal a reference to 'obj' // (Py_DECREF(obj)) on success (if it returns 0). diff --git a/Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst b/Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst new file mode 100644 index 0000000000000..6178f18517d48 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-07-25-13-41-09.gh-issue-107226.N919zH.rst @@ -0,0 +1,2 @@ +:c:func:`PyModule_AddObjectRef` is now only available in the limited API +version 3.10 or later. From webhook-mailer at python.org Tue Jul 25 16:37:51 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 20:37:51 -0000 Subject: [Python-checkins] gh-106185: Deduplicate `CPythonTracebackErrorCaretTests` in `test_traceback` (GH-106187) Message-ID: https://github.com/python/cpython/commit/7c89f1189229c5c67a3766e24ecf00cde658b7fd commit: 7c89f1189229c5c67a3766e24ecf00cde658b7fd branch: main author: Nikita Sobolev committer: serhiy-storchaka date: 2023-07-25T23:37:47+03:00 summary: gh-106185: Deduplicate `CPythonTracebackErrorCaretTests` in `test_traceback` (GH-106187) files: M Lib/test/test_traceback.py diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index da7d1fb559203..7c6fdbf762921 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -918,7 +918,7 @@ class CPythonTracebackErrorCaretTests( @cpython_only @requires_debug_ranges() -class CPythonTracebackErrorCaretTests( +class CPythonTracebackLegacyErrorCaretTests( CAPIExceptionFormattingLegacyMixin, TracebackErrorLocationCaretTestBase, unittest.TestCase, From webhook-mailer at python.org Tue Jul 25 17:08:55 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Tue, 25 Jul 2023 21:08:55 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: more misc typing improvements (#107264) Message-ID: https://github.com/python/cpython/commit/ee9010784c7d5a084f6e5465bc706717e1a0a151 commit: ee9010784c7d5a084f6e5465bc706717e1a0a151 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-25T22:08:52+01:00 summary: gh-104050: Argument clinic: more misc typing improvements (#107264) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 8c959ed4d0044..c46c6860e45d2 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -45,6 +45,7 @@ Protocol, TypeGuard, TypeVar, + cast, overload, ) @@ -2694,9 +2695,12 @@ def closure(f: CConverterClassT) -> CConverterClassT: return closure class CConverterAutoRegister(type): - def __init__(cls, name, bases, classdict): - add_c_converter(cls) - add_default_legacy_c_converter(cls) + def __init__( + cls, name: str, bases: tuple[type, ...], classdict: dict[str, Any] + ) -> None: + converter_cls = cast(type["CConverter"], cls) + add_c_converter(converter_cls) + add_default_legacy_c_converter(converter_cls) class CConverter(metaclass=CConverterAutoRegister): """ @@ -3174,10 +3178,10 @@ class defining_class_converter(CConverter): def converter_init(self, *, type: str | None = None) -> None: self.specified_type = type - def render(self, parameter, data) -> None: + def render(self, parameter: Parameter, data: CRenderData) -> None: self._render_self(parameter, data) - def set_template_dict(self, template_dict): + def set_template_dict(self, template_dict: TemplateDict) -> None: template_dict['defining_class_name'] = self.name @@ -4048,7 +4052,7 @@ def parser_type(self) -> str: assert isinstance(self.function, Function) return required_type_for_self_for_parser(self.function) or self.type - def render(self, parameter, data): + def render(self, parameter: Parameter, data: CRenderData) -> None: """ parameter is a clinic.Parameter instance. data is a CRenderData instance. @@ -4064,6 +4068,7 @@ def render(self, parameter, data): # because we render parameters in order, and self is always first. assert len(data.impl_arguments) == 1 assert data.impl_arguments[0] == self.name + assert self.type is not None data.impl_arguments[0] = '(' + self.type + ")" + data.impl_arguments[0] def set_template_dict(self, template_dict: TemplateDict) -> None: From webhook-mailer at python.org Tue Jul 25 17:09:29 2023 From: webhook-mailer at python.org (gpshead) Date: Tue, 25 Jul 2023 21:09:29 -0000 Subject: [Python-checkins] gh-106939: document ShareableList nul-strip quirk. (#107266) Message-ID: https://github.com/python/cpython/commit/70dc00946938027d5a79bcb7b65038319040144e commit: 70dc00946938027d5a79bcb7b65038319040144e branch: main author: Gregory P. Smith committer: gpshead date: 2023-07-25T14:09:25-07:00 summary: gh-106939: document ShareableList nul-strip quirk. (#107266) * gh-106939: document ShareableList nul-strip quirk. * Mention the `int` size constraint. files: M Doc/library/multiprocessing.shared_memory.rst diff --git a/Doc/library/multiprocessing.shared_memory.rst b/Doc/library/multiprocessing.shared_memory.rst index 76046b34610ab..f453e6403d932 100644 --- a/Doc/library/multiprocessing.shared_memory.rst +++ b/Doc/library/multiprocessing.shared_memory.rst @@ -255,16 +255,17 @@ shared memory blocks created using that manager are all released when the :keyword:`with` statement's code block finishes execution. -.. class:: ShareableList(sequence=None, *, name=None) +.. class:: ShareableList(sequence=None, \*, name=None) Provides a mutable list-like object where all values stored within are stored in a shared memory block. This constrains storable values to - only the ``int``, ``float``, ``bool``, ``str`` (less than 10M bytes each), - ``bytes`` (less than 10M bytes each), and ``None`` built-in data types. - It also notably differs from the built-in ``list`` type in that these - lists can not change their overall length (i.e. no append, insert, etc.) - and do not support the dynamic creation of new :class:`ShareableList` - instances via slicing. + only the ``int`` (signed 64-bit), ``float``, ``bool``, ``str`` (less + than 10M bytes each when encoded as utf-8), ``bytes`` (less than 10M + bytes each), and ``None`` built-in data types. It also notably + differs from the built-in ``list`` type in that these lists can not + change their overall length (i.e. no append, insert, etc.) and do not + support the dynamic creation of new :class:`ShareableList` instances + via slicing. *sequence* is used in populating a new ``ShareableList`` full of values. Set to ``None`` to instead attach to an already existing @@ -275,6 +276,35 @@ shared memory blocks created using that manager are all released when the existing ``ShareableList``, specify its shared memory block's unique name while leaving ``sequence`` set to ``None``. + .. note:: + + A known issue exists for :class:`bytes` and :class:`str` values. + If they end with ``\x00`` nul bytes or characters, those may be + *silently stripped* when fetching them by index from the + :class:`ShareableList`. This ``.rstrip(b'\x00')`` behavior is + considered a bug and may go away in the future. See :gh:`106939`. + + For applications where rstripping of trailing nulls is a problem, + work around it by always unconditionally appending an extra non-0 + byte to the end of such values when storing and unconditionally + removing it when fetching: + + .. doctest:: + + >>> from multiprocessing import shared_memory + >>> nul_bug_demo = shared_memory.ShareableList(['?\x00', b'\x03\x02\x01\x00\x00\x00']) + >>> nul_bug_demo[0] + '?' + >>> nul_bug_demo[1] + b'\x03\x02\x01' + >>> nul_bug_demo.shm.unlink() + >>> padded = shared_memory.ShareableList(['?\x00\x07', b'\x03\x02\x01\x00\x00\x00\x07']) + >>> padded[0][:-1] + '?\x00' + >>> padded[1][:-1] + b'\x03\x02\x01\x00\x00\x00' + >>> padded.shm.unlink() + .. method:: count(value) Returns the number of occurrences of ``value``. From webhook-mailer at python.org Tue Jul 25 17:18:27 2023 From: webhook-mailer at python.org (gpshead) Date: Tue, 25 Jul 2023 21:18:27 -0000 Subject: [Python-checkins] [3.11] gh-106939: document ShareableList nul-strip quirk. (GH-107266) (#107270) Message-ID: https://github.com/python/cpython/commit/5a89248696ee92e2182c67e366538bb88df5bed7 commit: 5a89248696ee92e2182c67e366538bb88df5bed7 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gpshead date: 2023-07-25T21:18:23Z summary: [3.11] gh-106939: document ShareableList nul-strip quirk. (GH-107266) (#107270) gh-106939: document ShareableList nul-strip quirk. (GH-107266) * gh-106939: document ShareableList nul-strip quirk. * Mention the `int` size constraint. (cherry picked from commit 70dc00946938027d5a79bcb7b65038319040144e) Co-authored-by: Gregory P. Smith files: M Doc/library/multiprocessing.shared_memory.rst diff --git a/Doc/library/multiprocessing.shared_memory.rst b/Doc/library/multiprocessing.shared_memory.rst index 76046b34610ab..f453e6403d932 100644 --- a/Doc/library/multiprocessing.shared_memory.rst +++ b/Doc/library/multiprocessing.shared_memory.rst @@ -255,16 +255,17 @@ shared memory blocks created using that manager are all released when the :keyword:`with` statement's code block finishes execution. -.. class:: ShareableList(sequence=None, *, name=None) +.. class:: ShareableList(sequence=None, \*, name=None) Provides a mutable list-like object where all values stored within are stored in a shared memory block. This constrains storable values to - only the ``int``, ``float``, ``bool``, ``str`` (less than 10M bytes each), - ``bytes`` (less than 10M bytes each), and ``None`` built-in data types. - It also notably differs from the built-in ``list`` type in that these - lists can not change their overall length (i.e. no append, insert, etc.) - and do not support the dynamic creation of new :class:`ShareableList` - instances via slicing. + only the ``int`` (signed 64-bit), ``float``, ``bool``, ``str`` (less + than 10M bytes each when encoded as utf-8), ``bytes`` (less than 10M + bytes each), and ``None`` built-in data types. It also notably + differs from the built-in ``list`` type in that these lists can not + change their overall length (i.e. no append, insert, etc.) and do not + support the dynamic creation of new :class:`ShareableList` instances + via slicing. *sequence* is used in populating a new ``ShareableList`` full of values. Set to ``None`` to instead attach to an already existing @@ -275,6 +276,35 @@ shared memory blocks created using that manager are all released when the existing ``ShareableList``, specify its shared memory block's unique name while leaving ``sequence`` set to ``None``. + .. note:: + + A known issue exists for :class:`bytes` and :class:`str` values. + If they end with ``\x00`` nul bytes or characters, those may be + *silently stripped* when fetching them by index from the + :class:`ShareableList`. This ``.rstrip(b'\x00')`` behavior is + considered a bug and may go away in the future. See :gh:`106939`. + + For applications where rstripping of trailing nulls is a problem, + work around it by always unconditionally appending an extra non-0 + byte to the end of such values when storing and unconditionally + removing it when fetching: + + .. doctest:: + + >>> from multiprocessing import shared_memory + >>> nul_bug_demo = shared_memory.ShareableList(['?\x00', b'\x03\x02\x01\x00\x00\x00']) + >>> nul_bug_demo[0] + '?' + >>> nul_bug_demo[1] + b'\x03\x02\x01' + >>> nul_bug_demo.shm.unlink() + >>> padded = shared_memory.ShareableList(['?\x00\x07', b'\x03\x02\x01\x00\x00\x00\x07']) + >>> padded[0][:-1] + '?\x00' + >>> padded[1][:-1] + b'\x03\x02\x01\x00\x00\x00' + >>> padded.shm.unlink() + .. method:: count(value) Returns the number of occurrences of ``value``. From webhook-mailer at python.org Tue Jul 25 17:19:29 2023 From: webhook-mailer at python.org (gpshead) Date: Tue, 25 Jul 2023 21:19:29 -0000 Subject: [Python-checkins] [3.12] gh-106939: document ShareableList nul-strip quirk. (GH-107266) (#107269) Message-ID: https://github.com/python/cpython/commit/0107731a06db13cf93fc56cc72524f0493c6a04f commit: 0107731a06db13cf93fc56cc72524f0493c6a04f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gpshead date: 2023-07-25T21:19:25Z summary: [3.12] gh-106939: document ShareableList nul-strip quirk. (GH-107266) (#107269) gh-106939: document ShareableList nul-strip quirk. (GH-107266) * gh-106939: document ShareableList nul-strip quirk. * Mention the `int` size constraint. (cherry picked from commit 70dc00946938027d5a79bcb7b65038319040144e) Co-authored-by: Gregory P. Smith files: M Doc/library/multiprocessing.shared_memory.rst diff --git a/Doc/library/multiprocessing.shared_memory.rst b/Doc/library/multiprocessing.shared_memory.rst index 76046b34610ab..f453e6403d932 100644 --- a/Doc/library/multiprocessing.shared_memory.rst +++ b/Doc/library/multiprocessing.shared_memory.rst @@ -255,16 +255,17 @@ shared memory blocks created using that manager are all released when the :keyword:`with` statement's code block finishes execution. -.. class:: ShareableList(sequence=None, *, name=None) +.. class:: ShareableList(sequence=None, \*, name=None) Provides a mutable list-like object where all values stored within are stored in a shared memory block. This constrains storable values to - only the ``int``, ``float``, ``bool``, ``str`` (less than 10M bytes each), - ``bytes`` (less than 10M bytes each), and ``None`` built-in data types. - It also notably differs from the built-in ``list`` type in that these - lists can not change their overall length (i.e. no append, insert, etc.) - and do not support the dynamic creation of new :class:`ShareableList` - instances via slicing. + only the ``int`` (signed 64-bit), ``float``, ``bool``, ``str`` (less + than 10M bytes each when encoded as utf-8), ``bytes`` (less than 10M + bytes each), and ``None`` built-in data types. It also notably + differs from the built-in ``list`` type in that these lists can not + change their overall length (i.e. no append, insert, etc.) and do not + support the dynamic creation of new :class:`ShareableList` instances + via slicing. *sequence* is used in populating a new ``ShareableList`` full of values. Set to ``None`` to instead attach to an already existing @@ -275,6 +276,35 @@ shared memory blocks created using that manager are all released when the existing ``ShareableList``, specify its shared memory block's unique name while leaving ``sequence`` set to ``None``. + .. note:: + + A known issue exists for :class:`bytes` and :class:`str` values. + If they end with ``\x00`` nul bytes or characters, those may be + *silently stripped* when fetching them by index from the + :class:`ShareableList`. This ``.rstrip(b'\x00')`` behavior is + considered a bug and may go away in the future. See :gh:`106939`. + + For applications where rstripping of trailing nulls is a problem, + work around it by always unconditionally appending an extra non-0 + byte to the end of such values when storing and unconditionally + removing it when fetching: + + .. doctest:: + + >>> from multiprocessing import shared_memory + >>> nul_bug_demo = shared_memory.ShareableList(['?\x00', b'\x03\x02\x01\x00\x00\x00']) + >>> nul_bug_demo[0] + '?' + >>> nul_bug_demo[1] + b'\x03\x02\x01' + >>> nul_bug_demo.shm.unlink() + >>> padded = shared_memory.ShareableList(['?\x00\x07', b'\x03\x02\x01\x00\x00\x00\x07']) + >>> padded[0][:-1] + '?\x00' + >>> padded[1][:-1] + b'\x03\x02\x01\x00\x00\x00' + >>> padded.shm.unlink() + .. method:: count(value) Returns the number of occurrences of ``value``. From webhook-mailer at python.org Tue Jul 25 17:23:37 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 25 Jul 2023 21:23:37 -0000 Subject: [Python-checkins] [3.12] gh-106185: Deduplicate `CPythonTracebackErrorCaretTests` in `test_traceback` (GH-106187) (GH-107268) Message-ID: https://github.com/python/cpython/commit/3d15c8b84b334658a5476a9cbf2626fe15fc7344 commit: 3d15c8b84b334658a5476a9cbf2626fe15fc7344 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-25T21:23:33Z summary: [3.12] gh-106185: Deduplicate `CPythonTracebackErrorCaretTests` in `test_traceback` (GH-106187) (GH-107268) (cherry picked from commit 7c89f1189229c5c67a3766e24ecf00cde658b7fd) Co-authored-by: Nikita Sobolev files: M Lib/test/test_traceback.py diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index 19a2be88d2c1b..9476e2c89def4 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -918,7 +918,7 @@ class CPythonTracebackErrorCaretTests( @cpython_only @requires_debug_ranges() -class CPythonTracebackErrorCaretTests( +class CPythonTracebackLegacyErrorCaretTests( CAPIExceptionFormattingLegacyMixin, TracebackErrorLocationCaretTestBase, unittest.TestCase, From webhook-mailer at python.org Tue Jul 25 18:33:06 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Tue, 25 Jul 2023 22:33:06 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: annotate `post_parsing()` and `cleanup()` (#107225) Message-ID: https://github.com/python/cpython/commit/33838fedf7ed6ee907a49d042f8fa123178a1f9c commit: 33838fedf7ed6ee907a49d042f8fa123178a1f9c branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-25T23:33:03+01:00 summary: gh-104050: Argument clinic: annotate `post_parsing()` and `cleanup()` (#107225) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index c46c6860e45d2..d9e893f1f1d35 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -3664,10 +3664,12 @@ def converter_init( if NoneType in accept and self.c_default == "Py_None": self.c_default = "NULL" - def post_parsing(self): + def post_parsing(self) -> str: if self.encoding: name = self.name return f"PyMem_FREE({name});\n" + else: + return "" def parse_arg(self, argname: str, displayname: str) -> str: if self.format_unit == 's': @@ -3845,8 +3847,10 @@ def converter_init( fail("Py_UNICODE_converter: illegal 'accept' argument " + repr(accept)) self.c_default = "NULL" - def cleanup(self): - if not self.length: + def cleanup(self) -> str: + if self.length: + return "" + else: return """\ PyMem_Free((void *){name}); """.format(name=self.name) From webhook-mailer at python.org Wed Jul 26 01:17:47 2023 From: webhook-mailer at python.org (hugovk) Date: Wed, 26 Jul 2023 05:17:47 -0000 Subject: [Python-checkins] [3.11] GH-97950: Fix old-style index directive in Doc/library/imp.rst (#107246) Message-ID: https://github.com/python/cpython/commit/9fa44d7a3045f4571e2cabd53a1cc0f16a0f3723 commit: 9fa44d7a3045f4571e2cabd53a1cc0f16a0f3723 branch: 3.11 author: Wei-Hsiang (Matt) Wang committer: hugovk date: 2023-07-26T08:17:44+03:00 summary: [3.11] GH-97950: Fix old-style index directive in Doc/library/imp.rst (#107246) Use new-style index directive ('statement') - library/imp files: M Doc/library/imp.rst diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst index 000793a7e66ca..cc95cd6502940 100644 --- a/Doc/library/imp.rst +++ b/Doc/library/imp.rst @@ -10,7 +10,7 @@ .. deprecated-removed:: 3.4 3.12 The :mod:`imp` module is deprecated in favor of :mod:`importlib`. -.. index:: statement: import +.. index:: pair: statement; import -------------- From webhook-mailer at python.org Wed Jul 26 02:34:17 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 26 Jul 2023 06:34:17 -0000 Subject: [Python-checkins] gh-106368: Increase Argument Clinic CLI test coverage (#107277) Message-ID: https://github.com/python/cpython/commit/579100f6d75a27429e7f8de74935d7bc3a3e44e6 commit: 579100f6d75a27429e7f8de74935d7bc3a3e44e6 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-26T08:34:14+02:00 summary: gh-106368: Increase Argument Clinic CLI test coverage (#107277) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index f527120336226..d1c61b1d401b9 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1348,18 +1348,18 @@ def test_scaffolding(self): class ClinicExternalTest(TestCase): maxDiff = None + clinic_py = os.path.join(test_tools.toolsdir, "clinic", "clinic.py") def _do_test(self, *args, expect_success=True): - clinic_py = os.path.join(test_tools.toolsdir, "clinic", "clinic.py") with subprocess.Popen( - [sys.executable, "-Xutf8", clinic_py, *args], + [sys.executable, "-Xutf8", self.clinic_py, *args], encoding="utf-8", bufsize=0, stdout=subprocess.PIPE, stderr=subprocess.PIPE, ) as proc: proc.wait() - if expect_success == bool(proc.returncode): + if expect_success and proc.returncode: self.fail("".join(proc.stderr)) stdout = proc.stdout.read() stderr = proc.stderr.read() @@ -1449,6 +1449,49 @@ def test_cli_force(self): generated = f.read() self.assertTrue(generated.endswith(checksum)) + def test_cli_make(self): + c_code = dedent(""" + /*[clinic input] + [clinic start generated code]*/ + """) + py_code = "pass" + c_files = "file1.c", "file2.c" + py_files = "file1.py", "file2.py" + + def create_files(files, srcdir, code): + for fn in files: + path = os.path.join(srcdir, fn) + with open(path, "w", encoding="utf-8") as f: + f.write(code) + + with os_helper.temp_dir() as tmp_dir: + # add some folders, some C files and a Python file + create_files(c_files, tmp_dir, c_code) + create_files(py_files, tmp_dir, py_code) + + # create C files in externals/ dir + ext_path = os.path.join(tmp_dir, "externals") + with os_helper.temp_dir(path=ext_path) as externals: + create_files(c_files, externals, c_code) + + # run clinic in verbose mode with --make on tmpdir + out = self.expect_success("-v", "--make", "--srcdir", tmp_dir) + + # expect verbose mode to only mention the C files in tmp_dir + for filename in c_files: + with self.subTest(filename=filename): + path = os.path.join(tmp_dir, filename) + self.assertIn(path, out) + for filename in py_files: + with self.subTest(filename=filename): + path = os.path.join(tmp_dir, filename) + self.assertNotIn(path, out) + # don't expect C files from the externals dir + for filename in c_files: + with self.subTest(filename=filename): + path = os.path.join(ext_path, filename) + self.assertNotIn(path, out) + def test_cli_verbose(self): with os_helper.temp_dir() as tmp_dir: fn = os.path.join(tmp_dir, "test.c") @@ -1534,6 +1577,35 @@ def test_cli_converters(self): f"expected converter {converter!r}, got {line!r}" ) + def test_cli_fail_converters_and_filename(self): + out = self.expect_failure("--converters", "test.c") + msg = ( + "Usage error: can't specify --converters " + "and a filename at the same time" + ) + self.assertIn(msg, out) + + def test_cli_fail_no_filename(self): + out = self.expect_failure() + self.assertIn("usage: clinic.py", out) + + def test_cli_fail_output_and_multiple_files(self): + out = self.expect_failure("-o", "out.c", "input.c", "moreinput.c") + msg = "Usage error: can't use -o with multiple filenames" + self.assertIn(msg, out) + + def test_cli_fail_filename_or_output_and_make(self): + for opts in ("-o", "out.c"), ("filename.c",): + with self.subTest(opts=opts): + out = self.expect_failure("--make", *opts) + msg = "Usage error: can't use -o or filenames with --make" + self.assertIn(msg, out) + + def test_cli_fail_make_without_srcdir(self): + out = self.expect_failure("--make", "--srcdir", "") + msg = "Usage error: --srcdir must not be empty with --make" + self.assertIn(msg, out) + try: import _testclinic as ac_tester From webhook-mailer at python.org Wed Jul 26 03:13:26 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 26 Jul 2023 07:13:26 -0000 Subject: [Python-checkins] [3.12] gh-106368: Increase Argument Clinic CLI test coverage (GH-107277) (#107282) Message-ID: https://github.com/python/cpython/commit/ef808517f499b25f9f847c813067f24d73af051f commit: ef808517f499b25f9f847c813067f24d73af051f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-26T07:13:22Z summary: [3.12] gh-106368: Increase Argument Clinic CLI test coverage (GH-107277) (#107282) (cherry picked from commit 579100f6d75a27429e7f8de74935d7bc3a3e44e6) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index f527120336226..d1c61b1d401b9 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1348,18 +1348,18 @@ def test_scaffolding(self): class ClinicExternalTest(TestCase): maxDiff = None + clinic_py = os.path.join(test_tools.toolsdir, "clinic", "clinic.py") def _do_test(self, *args, expect_success=True): - clinic_py = os.path.join(test_tools.toolsdir, "clinic", "clinic.py") with subprocess.Popen( - [sys.executable, "-Xutf8", clinic_py, *args], + [sys.executable, "-Xutf8", self.clinic_py, *args], encoding="utf-8", bufsize=0, stdout=subprocess.PIPE, stderr=subprocess.PIPE, ) as proc: proc.wait() - if expect_success == bool(proc.returncode): + if expect_success and proc.returncode: self.fail("".join(proc.stderr)) stdout = proc.stdout.read() stderr = proc.stderr.read() @@ -1449,6 +1449,49 @@ def test_cli_force(self): generated = f.read() self.assertTrue(generated.endswith(checksum)) + def test_cli_make(self): + c_code = dedent(""" + /*[clinic input] + [clinic start generated code]*/ + """) + py_code = "pass" + c_files = "file1.c", "file2.c" + py_files = "file1.py", "file2.py" + + def create_files(files, srcdir, code): + for fn in files: + path = os.path.join(srcdir, fn) + with open(path, "w", encoding="utf-8") as f: + f.write(code) + + with os_helper.temp_dir() as tmp_dir: + # add some folders, some C files and a Python file + create_files(c_files, tmp_dir, c_code) + create_files(py_files, tmp_dir, py_code) + + # create C files in externals/ dir + ext_path = os.path.join(tmp_dir, "externals") + with os_helper.temp_dir(path=ext_path) as externals: + create_files(c_files, externals, c_code) + + # run clinic in verbose mode with --make on tmpdir + out = self.expect_success("-v", "--make", "--srcdir", tmp_dir) + + # expect verbose mode to only mention the C files in tmp_dir + for filename in c_files: + with self.subTest(filename=filename): + path = os.path.join(tmp_dir, filename) + self.assertIn(path, out) + for filename in py_files: + with self.subTest(filename=filename): + path = os.path.join(tmp_dir, filename) + self.assertNotIn(path, out) + # don't expect C files from the externals dir + for filename in c_files: + with self.subTest(filename=filename): + path = os.path.join(ext_path, filename) + self.assertNotIn(path, out) + def test_cli_verbose(self): with os_helper.temp_dir() as tmp_dir: fn = os.path.join(tmp_dir, "test.c") @@ -1534,6 +1577,35 @@ def test_cli_converters(self): f"expected converter {converter!r}, got {line!r}" ) + def test_cli_fail_converters_and_filename(self): + out = self.expect_failure("--converters", "test.c") + msg = ( + "Usage error: can't specify --converters " + "and a filename at the same time" + ) + self.assertIn(msg, out) + + def test_cli_fail_no_filename(self): + out = self.expect_failure() + self.assertIn("usage: clinic.py", out) + + def test_cli_fail_output_and_multiple_files(self): + out = self.expect_failure("-o", "out.c", "input.c", "moreinput.c") + msg = "Usage error: can't use -o with multiple filenames" + self.assertIn(msg, out) + + def test_cli_fail_filename_or_output_and_make(self): + for opts in ("-o", "out.c"), ("filename.c",): + with self.subTest(opts=opts): + out = self.expect_failure("--make", *opts) + msg = "Usage error: can't use -o or filenames with --make" + self.assertIn(msg, out) + + def test_cli_fail_make_without_srcdir(self): + out = self.expect_failure("--make", "--srcdir", "") + msg = "Usage error: --srcdir must not be empty with --make" + self.assertIn(msg, out) + try: import _testclinic as ac_tester From webhook-mailer at python.org Wed Jul 26 03:58:10 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 26 Jul 2023 07:58:10 -0000 Subject: [Python-checkins] [3.11] gh-106350: Tkinter: do not ignore return value of `mp_init()` (GH-106351) (GH-107259) Message-ID: https://github.com/python/cpython/commit/36208b5f6aecf40c325667385cc3afd389efab88 commit: 36208b5f6aecf40c325667385cc3afd389efab88 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-26T10:58:07+03:00 summary: [3.11] gh-106350: Tkinter: do not ignore return value of `mp_init()` (GH-106351) (GH-107259) (cherry picked from commit b5ae7c498438657a6ba0bf4cc216b9c2c93a06c7) Co-authored-by: Christopher Chavez files: A Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst M Modules/_tkinter.c diff --git a/Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst b/Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst new file mode 100644 index 0000000000000..681d63a6668be --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst @@ -0,0 +1,2 @@ +Detect possible memory allocation failure in the libtommath function :c:func:`mp_init` +used by the ``_tkinter`` module. diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c index 2aab68ee1784f..e5377073ff765 100644 --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -906,8 +906,9 @@ asBignumObj(PyObject *value) return NULL; } hexchars += neg + 2; /* skip sign and "0x" */ - mp_init(&bigValue); - if (mp_read_radix(&bigValue, hexchars, 16) != MP_OKAY) { + if (mp_init(&bigValue) != MP_OKAY || + mp_read_radix(&bigValue, hexchars, 16) != MP_OKAY) + { mp_clear(&bigValue); Py_DECREF(hexstr); PyErr_NoMemory(); From webhook-mailer at python.org Wed Jul 26 03:58:36 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 26 Jul 2023 07:58:36 -0000 Subject: [Python-checkins] [3.12] gh-106350: Tkinter: do not ignore return value of `mp_init()` (GH-106351) (GH-107258) Message-ID: https://github.com/python/cpython/commit/d2355426d688eb9c84b3db15bbae21921858afbf commit: d2355426d688eb9c84b3db15bbae21921858afbf branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-26T10:58:33+03:00 summary: [3.12] gh-106350: Tkinter: do not ignore return value of `mp_init()` (GH-106351) (GH-107258) (cherry picked from commit b5ae7c498438657a6ba0bf4cc216b9c2c93a06c7) Co-authored-by: Christopher Chavez files: A Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst M Modules/_tkinter.c diff --git a/Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst b/Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst new file mode 100644 index 0000000000000..681d63a6668be --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-03-03-46-20.gh-issue-106350.LLcTEe.rst @@ -0,0 +1,2 @@ +Detect possible memory allocation failure in the libtommath function :c:func:`mp_init` +used by the ``_tkinter`` module. diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c index 15f9c0465fb04..5abde84ebc2ef 100644 --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -876,8 +876,9 @@ asBignumObj(PyObject *value) return NULL; } hexchars += neg + 2; /* skip sign and "0x" */ - mp_init(&bigValue); - if (mp_read_radix(&bigValue, hexchars, 16) != MP_OKAY) { + if (mp_init(&bigValue) != MP_OKAY || + mp_read_radix(&bigValue, hexchars, 16) != MP_OKAY) + { mp_clear(&bigValue); Py_DECREF(hexstr); PyErr_NoMemory(); From webhook-mailer at python.org Wed Jul 26 05:11:54 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 26 Jul 2023 09:11:54 -0000 Subject: [Python-checkins] gh-103735: Tkinter: remove handling for uninteresting "procbody" Tcl value type (GH-103736) Message-ID: https://github.com/python/cpython/commit/d96ca41688b9c2a8d261c340ae98438ca41796d8 commit: d96ca41688b9c2a8d261c340ae98438ca41796d8 branch: main author: Christopher Chavez committer: serhiy-storchaka date: 2023-07-26T12:11:50+03:00 summary: gh-103735: Tkinter: remove handling for uninteresting "procbody" Tcl value type (GH-103736) files: M Modules/_tkinter.c diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c index 145a2940427e6..406e01cd75ffd 100644 --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -317,7 +317,6 @@ typedef struct { const Tcl_ObjType *WideIntType; const Tcl_ObjType *BignumType; const Tcl_ObjType *ListType; - const Tcl_ObjType *ProcBodyType; const Tcl_ObjType *StringType; } TkappObject; @@ -595,7 +594,6 @@ Tkapp_New(const char *screenName, const char *className, v->WideIntType = Tcl_GetObjType("wideInt"); v->BignumType = Tcl_GetObjType("bignum"); v->ListType = Tcl_GetObjType("list"); - v->ProcBodyType = Tcl_GetObjType("procbody"); v->StringType = Tcl_GetObjType("string"); /* Delete the 'exit' command, which can screw things up */ @@ -1175,10 +1173,6 @@ FromObj(TkappObject *tkapp, Tcl_Obj *value) return result; } - if (value->typePtr == tkapp->ProcBodyType) { - /* fall through: return tcl object. */ - } - if (value->typePtr == tkapp->StringType) { return unicodeFromTclObj(value); } From webhook-mailer at python.org Wed Jul 26 06:51:28 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 26 Jul 2023 10:51:28 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: improve typing around `parse_arg()` methods (#107288) Message-ID: https://github.com/python/cpython/commit/c362678dd289c45ff36ea9c61246a406990befd2 commit: c362678dd289c45ff36ea9c61246a406990befd2 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-26T11:51:24+01:00 summary: gh-104050: Argument clinic: improve typing around `parse_arg()` methods (#107288) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index d9e893f1f1d35..54fd1ae522c0f 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -3046,7 +3046,7 @@ def pre_render(self) -> None: """ pass - def parse_arg(self, argname: str, displayname: str): + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'O&': return """ if (!{converter}({argname}, &{paramname})) {{{{ @@ -3149,7 +3149,7 @@ def converter_init(self, *, accept: TypeSet = {object}) -> None: self.default = bool(self.default) self.c_default = str(int(self.default)) - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'i': return """ {paramname} = _PyLong_AsInt({argname}); @@ -3200,7 +3200,7 @@ def converter_init(self) -> None: if self.c_default == '"\'"': self.c_default = r"'\''" - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'c': return """ if (PyBytes_Check({argname}) && PyBytes_GET_SIZE({argname}) == 1) {{{{ @@ -3229,7 +3229,7 @@ def converter_init(self, *, bitwise: bool = False) -> None: if bitwise: self.format_unit = 'B' - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'b': return """ {{{{ @@ -3274,7 +3274,7 @@ class short_converter(CConverter): format_unit = 'h' c_ignored_default = "0" - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'h': return """ {{{{ @@ -3310,7 +3310,7 @@ def converter_init(self, *, bitwise: bool = False) -> None: else: self.converter = '_PyLong_UnsignedShort_Converter' - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'H': return """ {paramname} = (unsigned short)PyLong_AsUnsignedLongMask({argname}); @@ -3337,7 +3337,7 @@ def converter_init( if type is not None: self.type = type - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'i': return """ {paramname} = _PyLong_AsInt({argname}); @@ -3371,7 +3371,7 @@ def converter_init(self, *, bitwise: bool = False) -> None: else: self.converter = '_PyLong_UnsignedInt_Converter' - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'I': return """ {paramname} = (unsigned int)PyLong_AsUnsignedLongMask({argname}); @@ -3387,7 +3387,7 @@ class long_converter(CConverter): format_unit = 'l' c_ignored_default = "0" - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'l': return """ {paramname} = PyLong_AsLong({argname}); @@ -3408,7 +3408,7 @@ def converter_init(self, *, bitwise: bool = False) -> None: else: self.converter = '_PyLong_UnsignedLong_Converter' - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'k': return """ if (!PyLong_Check({argname})) {{{{ @@ -3426,7 +3426,7 @@ class long_long_converter(CConverter): format_unit = 'L' c_ignored_default = "0" - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'L': return """ {paramname} = PyLong_AsLongLong({argname}); @@ -3447,7 +3447,7 @@ def converter_init(self, *, bitwise: bool = False) -> None: else: self.converter = '_PyLong_UnsignedLongLong_Converter' - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'K': return """ if (!PyLong_Check({argname})) {{{{ @@ -3472,7 +3472,7 @@ def converter_init(self, *, accept: TypeSet = {int}) -> None: else: fail("Py_ssize_t_converter: illegal 'accept' argument " + repr(accept)) - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'n': return """ {{{{ @@ -3507,7 +3507,7 @@ class size_t_converter(CConverter): converter = '_PyLong_Size_t_Converter' c_ignored_default = "0" - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'n': return """ {paramname} = PyNumber_AsSsize_t({argname}, PyExc_OverflowError); @@ -3522,7 +3522,7 @@ class fildes_converter(CConverter): type = 'int' converter = '_PyLong_FileDescriptor_Converter' - def _parse_arg(self, argname: str, displayname: str) -> str: + def _parse_arg(self, argname: str, displayname: str) -> str | None: return """ {paramname} = PyObject_AsFileDescriptor({argname}); if ({paramname} == -1) {{{{ @@ -3537,7 +3537,7 @@ class float_converter(CConverter): format_unit = 'f' c_ignored_default = "0.0" - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'f': return """ if (PyFloat_CheckExact({argname})) {{{{ @@ -3559,7 +3559,7 @@ class double_converter(CConverter): format_unit = 'd' c_ignored_default = "0.0" - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'd': return """ if (PyFloat_CheckExact({argname})) {{{{ @@ -3582,7 +3582,7 @@ class Py_complex_converter(CConverter): format_unit = 'D' c_ignored_default = "{0.0, 0.0}" - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'D': return """ {paramname} = PyComplex_AsCComplex({argname}); @@ -3671,7 +3671,7 @@ def post_parsing(self) -> str: else: return "" - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 's': return """ if (!PyUnicode_Check({argname})) {{{{ @@ -3773,7 +3773,7 @@ class PyBytesObject_converter(CConverter): format_unit = 'S' # accept = {bytes} - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'S': return """ if (!PyBytes_Check({argname})) {{{{ @@ -3790,7 +3790,7 @@ class PyByteArrayObject_converter(CConverter): format_unit = 'Y' # accept = {bytearray} - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'Y': return """ if (!PyByteArray_Check({argname})) {{{{ @@ -3807,7 +3807,7 @@ class unicode_converter(CConverter): default_type = (str, Null, NoneType) format_unit = 'U' - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'U': return """ if (!PyUnicode_Check({argname})) {{{{ @@ -3855,7 +3855,7 @@ def cleanup(self) -> str: PyMem_Free((void *){name}); """.format(name=self.name) - def parse_arg(self, argname: str, argnum: str) -> str: + def parse_arg(self, argname: str, argnum: str) -> str | None: if not self.length: if self.accept == {str}: return """ @@ -3918,7 +3918,7 @@ def cleanup(self) -> str: name = self.name return "".join(["if (", name, ".obj) {\n PyBuffer_Release(&", name, ");\n}\n"]) - def parse_arg(self, argname: str, displayname: str) -> str: + def parse_arg(self, argname: str, displayname: str) -> str | None: if self.format_unit == 'y*': return """ if (PyObject_GetBuffer({argname}, &{paramname}, PyBUF_SIMPLE) != 0) {{{{ From webhook-mailer at python.org Wed Jul 26 06:54:07 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 26 Jul 2023 10:54:07 -0000 Subject: [Python-checkins] gh-104683: Argument Clinic: Make most arguments to `Class` and `Function` required (#107289) Message-ID: https://github.com/python/cpython/commit/14d074e1fb403e54edb345505cfd2c73601e21f3 commit: 14d074e1fb403e54edb345505cfd2c73601e21f3 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-26T11:54:03+01:00 summary: gh-104683: Argument Clinic: Make most arguments to `Class` and `Function` required (#107289) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 54fd1ae522c0f..f557984f7642f 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -1325,7 +1325,6 @@ def parser_body( cpp_endif = "#endif /* " + conditional + " */" assert clinic is not None - assert f.full_name is not None if methoddef_define and f.full_name not in clinic.ifndef_symbols: clinic.ifndef_symbols.add(f.full_name) methoddef_ifndef = normalize_snippet(""" @@ -1541,11 +1540,8 @@ def render_function( '{impl_parameters}' in templates['parser_prototype']): data.declarations.pop(0) - template_dict = {} - - assert isinstance(f.full_name, str) full_name = f.full_name - template_dict['full_name'] = full_name + template_dict = {'full_name': full_name} if new_or_init: assert isinstance(f.cls, Class) @@ -2398,10 +2394,10 @@ def __repr__(self) -> str: @dc.dataclass(repr=False) class Class: name: str - module: Module | None = None - cls: Class | None = None - typedef: str | None = None - type_object: str | None = None + module: Module + cls: Class | None + typedef: str + type_object: str def __post_init__(self) -> None: self.parent = self.cls or self.module @@ -2527,14 +2523,14 @@ class Function: _: dc.KW_ONLY name: str module: Module - cls: Class | None = None - c_basename: str | None = None - full_name: str | None = None + cls: Class | None + c_basename: str | None + full_name: str return_converter: CReturnConverter + kind: FunctionKind + coexist: bool return_annotation: object = inspect.Signature.empty docstring: str = '' - kind: FunctionKind = CALLABLE - coexist: bool = False # docstring_only means "don't generate a machine-readable # signature, just a normal docstring". it's True for # functions with optional groups because we can't represent @@ -5325,7 +5321,6 @@ def state_function_docstring(self, line: str | None) -> None: def format_docstring(self) -> str: f = self.function assert f is not None - assert f.full_name is not None new_or_init = f.kind.new_or_init if new_or_init and not f.docstring: From webhook-mailer at python.org Wed Jul 26 08:32:51 2023 From: webhook-mailer at python.org (iritkatriel) Date: Wed, 26 Jul 2023 12:32:51 -0000 Subject: [Python-checkins] gh-106149: Simplify stack depth calculation. Replace asserts by exceptions. (#107255) Message-ID: https://github.com/python/cpython/commit/b0202a4e5d6b629ba5acbc703e950f08ebaf07df commit: b0202a4e5d6b629ba5acbc703e950f08ebaf07df branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-26T13:32:47+01:00 summary: gh-106149: Simplify stack depth calculation. Replace asserts by exceptions. (#107255) files: M Include/internal/pycore_flowgraph.h M Lib/test/test_peepholer.py M Python/compile.c M Python/flowgraph.c diff --git a/Include/internal/pycore_flowgraph.h b/Include/internal/pycore_flowgraph.h index 83b60ed849a95..cb55f8712e7ad 100644 --- a/Include/internal/pycore_flowgraph.h +++ b/Include/internal/pycore_flowgraph.h @@ -89,8 +89,8 @@ int _PyCfgBuilder_Init(_PyCfgBuilder *g); void _PyCfgBuilder_Fini(_PyCfgBuilder *g); int _PyCfg_OptimizeCodeUnit(_PyCfgBuilder *g, PyObject *consts, PyObject *const_cache, - int code_flags, int nlocals, int nparams, int firstlineno); -int _PyCfg_Stackdepth(_PyCfgBasicblock *entryblock, int code_flags); + int nlocals, int nparams, int firstlineno); +int _PyCfg_Stackdepth(_PyCfgBuilder *g); void _PyCfg_ConvertPseudoOps(_PyCfgBasicblock *entryblock); int _PyCfg_ResolveJumps(_PyCfgBuilder *g); diff --git a/Lib/test/test_peepholer.py b/Lib/test/test_peepholer.py index 82b0b50d0ea43..fba41f0b11979 100644 --- a/Lib/test/test_peepholer.py +++ b/Lib/test/test_peepholer.py @@ -991,6 +991,7 @@ def test_conditional_jump_forward_non_const_condition(self): ('LOAD_NAME', 1, 11), ('POP_JUMP_IF_TRUE', lbl := self.Label(), 12), ('LOAD_CONST', 2, 13), + ('RETURN_VALUE', 13), lbl, ('LOAD_CONST', 3, 14), ('RETURN_VALUE', 14), @@ -998,7 +999,7 @@ def test_conditional_jump_forward_non_const_condition(self): expected_insts = [ ('LOAD_NAME', 1, 11), ('POP_JUMP_IF_TRUE', lbl := self.Label(), 12), - ('LOAD_CONST', 1, 13), + ('RETURN_CONST', 1, 13), lbl, ('RETURN_CONST', 2, 14), ] @@ -1072,6 +1073,7 @@ def test_no_unsafe_static_swap(self): ('STORE_FAST', 1, 4), ('STORE_FAST', 1, 4), ('POP_TOP', 0, 4), + ('LOAD_CONST', 0, 5), ('RETURN_VALUE', 5) ] expected_insts = [ @@ -1080,7 +1082,7 @@ def test_no_unsafe_static_swap(self): ('NOP', 0, 3), ('STORE_FAST', 1, 4), ('POP_TOP', 0, 4), - ('RETURN_VALUE', 5) + ('RETURN_CONST', 0) ] self.cfg_optimization_test(insts, expected_insts, consts=list(range(3)), nlocals=1) @@ -1092,6 +1094,7 @@ def test_dead_store_elimination_in_same_lineno(self): ('STORE_FAST', 1, 4), ('STORE_FAST', 1, 4), ('STORE_FAST', 1, 4), + ('LOAD_CONST', 0, 5), ('RETURN_VALUE', 5) ] expected_insts = [ @@ -1100,7 +1103,7 @@ def test_dead_store_elimination_in_same_lineno(self): ('NOP', 0, 3), ('POP_TOP', 0, 4), ('STORE_FAST', 1, 4), - ('RETURN_VALUE', 5) + ('RETURN_CONST', 0, 5) ] self.cfg_optimization_test(insts, expected_insts, consts=list(range(3)), nlocals=1) @@ -1112,9 +1115,19 @@ def test_no_dead_store_elimination_in_different_lineno(self): ('STORE_FAST', 1, 4), ('STORE_FAST', 1, 5), ('STORE_FAST', 1, 6), + ('LOAD_CONST', 0, 5), ('RETURN_VALUE', 5) ] - self.cfg_optimization_test(insts, insts, consts=list(range(3)), nlocals=1) + expected_insts = [ + ('LOAD_CONST', 0, 1), + ('LOAD_CONST', 1, 2), + ('LOAD_CONST', 2, 3), + ('STORE_FAST', 1, 4), + ('STORE_FAST', 1, 5), + ('STORE_FAST', 1, 6), + ('RETURN_CONST', 0, 5) + ] + self.cfg_optimization_test(insts, expected_insts, consts=list(range(3)), nlocals=1) if __name__ == "__main__": diff --git a/Python/compile.c b/Python/compile.c index 5915a4bc2f81b..b4e06e7cb1408 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -7527,6 +7527,9 @@ build_cellfixedoffsets(_PyCompile_CodeUnitMetadata *umd) return fixed; } +#define IS_GENERATOR(CF) \ + ((CF) & (CO_GENERATOR | CO_COROUTINE | CO_ASYNC_GENERATOR)) + static int insert_prefix_instructions(_PyCompile_CodeUnitMetadata *umd, basicblock *entryblock, int *fixed, int nfreevars, int code_flags) @@ -7534,7 +7537,11 @@ insert_prefix_instructions(_PyCompile_CodeUnitMetadata *umd, basicblock *entrybl assert(umd->u_firstlineno > 0); /* Add the generator prefix instructions. */ - if (code_flags & (CO_GENERATOR | CO_COROUTINE | CO_ASYNC_GENERATOR)) { + if (IS_GENERATOR(code_flags)) { + /* Note that RETURN_GENERATOR + POP_TOP have a net stack effect + * of 0. This is because RETURN_GENERATOR pushes an element + * with _PyFrame_StackPush before switching stacks. + */ cfg_instr make_gen = { .i_opcode = RETURN_GENERATOR, .i_oparg = 0, @@ -7715,19 +7722,27 @@ optimize_and_assemble_code_unit(struct compiler_unit *u, PyObject *const_cache, int nparams = (int)PyList_GET_SIZE(u->u_ste->ste_varnames); int nlocals = (int)PyDict_GET_SIZE(u->u_metadata.u_varnames); assert(u->u_metadata.u_firstlineno); - if (_PyCfg_OptimizeCodeUnit(&g, consts, const_cache, code_flags, nlocals, + if (_PyCfg_OptimizeCodeUnit(&g, consts, const_cache, nlocals, nparams, u->u_metadata.u_firstlineno) < 0) { goto error; } - /** Assembly **/ - int nlocalsplus = prepare_localsplus(&u->u_metadata, &g, code_flags); - if (nlocalsplus < 0) { + int stackdepth = _PyCfg_Stackdepth(&g); + if (stackdepth < 0) { goto error; } - int maxdepth = _PyCfg_Stackdepth(g.g_entryblock, code_flags); - if (maxdepth < 0) { + /* prepare_localsplus adds instructions for generators that push + * and pop an item on the stack. This assertion makes sure there + * is space on the stack for that. + * It should always be true, because at least one expression is + * required to turn a function into a generator. + */ + assert(!(IS_GENERATOR(code_flags) && stackdepth == 0)); + + /** Assembly **/ + int nlocalsplus = prepare_localsplus(&u->u_metadata, &g, code_flags); + if (nlocalsplus < 0) { goto error; } @@ -7746,7 +7761,7 @@ optimize_and_assemble_code_unit(struct compiler_unit *u, PyObject *const_cache, } co = _PyAssemble_MakeCodeObject(&u->u_metadata, const_cache, consts, - maxdepth, &optimized_instrs, nlocalsplus, + stackdepth, &optimized_instrs, nlocalsplus, code_flags, filename); error: @@ -8190,8 +8205,8 @@ _PyCompile_OptimizeCfg(PyObject *instructions, PyObject *consts, int nlocals) if (instructions_to_cfg(instructions, &g) < 0) { goto error; } - int code_flags = 0, nparams = 0, firstlineno = 1; - if (_PyCfg_OptimizeCodeUnit(&g, consts, const_cache, code_flags, nlocals, + int nparams = 0, firstlineno = 1; + if (_PyCfg_OptimizeCodeUnit(&g, consts, const_cache, nlocals, nparams, firstlineno) < 0) { goto error; } @@ -8226,14 +8241,14 @@ _PyCompile_Assemble(_PyCompile_CodeUnitMetadata *umd, PyObject *filename, goto error; } - int code_flags = 0; - int nlocalsplus = prepare_localsplus(umd, &g, code_flags); - if (nlocalsplus < 0) { + int stackdepth = _PyCfg_Stackdepth(&g); + if (stackdepth < 0) { goto error; } - int maxdepth = _PyCfg_Stackdepth(g.g_entryblock, code_flags); - if (maxdepth < 0) { + int code_flags = 0; + int nlocalsplus = prepare_localsplus(umd, &g, code_flags); + if (nlocalsplus < 0) { goto error; } @@ -8256,7 +8271,7 @@ _PyCompile_Assemble(_PyCompile_CodeUnitMetadata *umd, PyObject *filename, goto error; } co = _PyAssemble_MakeCodeObject(umd, const_cache, - consts, maxdepth, &optimized_instrs, + consts, stackdepth, &optimized_instrs, nlocalsplus, code_flags, filename); Py_DECREF(consts); diff --git a/Python/flowgraph.c b/Python/flowgraph.c index f8e30ef7515a2..a72b85d45aa27 100644 --- a/Python/flowgraph.c +++ b/Python/flowgraph.c @@ -615,23 +615,28 @@ make_cfg_traversal_stack(basicblock *entryblock) { return stack; } -Py_LOCAL_INLINE(void) +Py_LOCAL_INLINE(int) stackdepth_push(basicblock ***sp, basicblock *b, int depth) { - assert(b->b_startdepth < 0 || b->b_startdepth == depth); + if (!(b->b_startdepth < 0 || b->b_startdepth == depth)) { + PyErr_Format(PyExc_ValueError, "Invalid CFG, inconsistent stackdepth"); + return ERROR; + } if (b->b_startdepth < depth && b->b_startdepth < 100) { assert(b->b_startdepth < 0); b->b_startdepth = depth; *(*sp)++ = b; } + return SUCCESS; } /* Find the flow path that needs the largest stack. We assume that * cycles in the flow graph have no net effect on the stack depth. */ int -_PyCfg_Stackdepth(basicblock *entryblock, int code_flags) +_PyCfg_Stackdepth(cfg_builder *g) { + basicblock *entryblock = g->g_entryblock; for (basicblock *b = entryblock; b != NULL; b = b->b_next) { b->b_startdepth = INT_MIN; } @@ -640,14 +645,13 @@ _PyCfg_Stackdepth(basicblock *entryblock, int code_flags) return ERROR; } + + int stackdepth = -1; int maxdepth = 0; basicblock **sp = stack; - if (code_flags & (CO_GENERATOR | CO_COROUTINE | CO_ASYNC_GENERATOR)) { - stackdepth_push(&sp, entryblock, 1); - } else { - stackdepth_push(&sp, entryblock, 0); + if (stackdepth_push(&sp, entryblock, 0) < 0) { + goto error; } - while (sp != stack) { basicblock *b = *--sp; int depth = b->b_startdepth; @@ -655,27 +659,40 @@ _PyCfg_Stackdepth(basicblock *entryblock, int code_flags) basicblock *next = b->b_next; for (int i = 0; i < b->b_iused; i++) { cfg_instr *instr = &b->b_instr[i]; - int effect = PyCompile_OpcodeStackEffectWithJump(instr->i_opcode, instr->i_oparg, 0); + int effect = PyCompile_OpcodeStackEffectWithJump( + instr->i_opcode, instr->i_oparg, 0); if (effect == PY_INVALID_STACK_EFFECT) { PyErr_Format(PyExc_SystemError, - "compiler PyCompile_OpcodeStackEffectWithJump(opcode=%d, arg=%i) failed", + "Invalid stack effect for opcode=%d, arg=%i", instr->i_opcode, instr->i_oparg); - return ERROR; + goto error; } int new_depth = depth + effect; - assert(new_depth >= 0); /* invalid code or bug in stackdepth() */ + if (new_depth < 0) { + PyErr_Format(PyExc_ValueError, + "Invalid CFG, stack underflow"); + goto error; + } if (new_depth > maxdepth) { maxdepth = new_depth; } if (HAS_TARGET(instr->i_opcode)) { - effect = PyCompile_OpcodeStackEffectWithJump(instr->i_opcode, instr->i_oparg, 1); - assert(effect != PY_INVALID_STACK_EFFECT); + effect = PyCompile_OpcodeStackEffectWithJump( + instr->i_opcode, instr->i_oparg, 1); + if (effect == PY_INVALID_STACK_EFFECT) { + PyErr_Format(PyExc_SystemError, + "Invalid stack effect for opcode=%d, arg=%i", + instr->i_opcode, instr->i_oparg); + goto error; + } int target_depth = depth + effect; assert(target_depth >= 0); /* invalid code or bug in stackdepth() */ if (target_depth > maxdepth) { maxdepth = target_depth; } - stackdepth_push(&sp, instr->i_target, target_depth); + if (stackdepth_push(&sp, instr->i_target, target_depth) < 0) { + goto error; + } } depth = new_depth; assert(!IS_ASSEMBLER_OPCODE(instr->i_opcode)); @@ -689,11 +706,15 @@ _PyCfg_Stackdepth(basicblock *entryblock, int code_flags) } if (next != NULL) { assert(BB_HAS_FALLTHROUGH(b)); - stackdepth_push(&sp, next, depth); + if (stackdepth_push(&sp, next, depth) < 0) { + goto error; + } } } + stackdepth = maxdepth; +error: PyMem_Free(stack); - return maxdepth; + return stackdepth; } static int @@ -2009,7 +2030,7 @@ mark_cold(basicblock *entryblock) { static int -push_cold_blocks_to_end(cfg_builder *g, int code_flags) { +push_cold_blocks_to_end(cfg_builder *g) { basicblock *entryblock = g->g_entryblock; if (entryblock->b_next == NULL) { /* single basicblock, no need to reorder */ @@ -2251,7 +2272,7 @@ resolve_line_numbers(cfg_builder *g, int firstlineno) int _PyCfg_OptimizeCodeUnit(cfg_builder *g, PyObject *consts, PyObject *const_cache, - int code_flags, int nlocals, int nparams, int firstlineno) + int nlocals, int nparams, int firstlineno) { assert(cfg_builder_check(g)); /** Preprocessing **/ @@ -2268,7 +2289,7 @@ _PyCfg_OptimizeCodeUnit(cfg_builder *g, PyObject *consts, PyObject *const_cache, g->g_entryblock, nlocals, nparams)); insert_superinstructions(g); - RETURN_IF_ERROR(push_cold_blocks_to_end(g, code_flags)); + RETURN_IF_ERROR(push_cold_blocks_to_end(g)); RETURN_IF_ERROR(resolve_line_numbers(g, firstlineno)); return SUCCESS; } From webhook-mailer at python.org Wed Jul 26 11:34:19 2023 From: webhook-mailer at python.org (pablogsal) Date: Wed, 26 Jul 2023 15:34:19 -0000 Subject: [Python-checkins] gh-107015: Remove async_hacks from the tokenizer (#107018) Message-ID: https://github.com/python/cpython/commit/da8f87b7ea421894c41dfc37f578e03409c5d280 commit: da8f87b7ea421894c41dfc37f578e03409c5d280 branch: main author: Pablo Galindo Salgado committer: pablogsal date: 2023-07-26T16:34:15+01:00 summary: gh-107015: Remove async_hacks from the tokenizer (#107018) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-22-14-35-38.gh-issue-107015.Ghp58t.rst M Doc/library/ast.rst M Doc/library/token-list.inc M Doc/library/token.rst M Grammar/Tokens M Grammar/python.gram M Include/internal/pycore_token.h M Lib/test/test_peg_generator/test_c_parser.py M Lib/test/test_tokenize.py M Lib/test/test_type_comments.py M Lib/token.py M Parser/parser.c M Parser/pegen.c M Parser/pegen.h M Parser/token.c M Parser/tokenizer.c M Parser/tokenizer.h M Python/Python-tokenize.c M Tools/peg_generator/pegen/keywordgen.py M Tools/peg_generator/pegen/python_generator.py diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 530cf30643687..cd657aedf6d23 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -2146,7 +2146,7 @@ and classes for traversing abstract syntax trees: Currently ``major`` must equal to ``3``. For example, setting ``feature_version=(3, 4)`` will allow the use of ``async`` and ``await`` as variable names. The lowest supported version is - ``(3, 4)``; the highest is ``sys.version_info[0:2]``. + ``(3, 7)``; the highest is ``sys.version_info[0:2]``. If source contains a null character ('\0'), :exc:`ValueError` is raised. @@ -2169,6 +2169,9 @@ and classes for traversing abstract syntax trees: .. versionchanged:: 3.8 Added ``type_comments``, ``mode='func_type'`` and ``feature_version``. + .. versionchanged:: 3.13 + The minimum supported version for feature_version is now (3,7) + .. function:: unparse(ast_obj) diff --git a/Doc/library/token-list.inc b/Doc/library/token-list.inc index e885de88cad9a..39df2927a0b7f 100644 --- a/Doc/library/token-list.inc +++ b/Doc/library/token-list.inc @@ -207,10 +207,6 @@ .. data:: OP -.. data:: AWAIT - -.. data:: ASYNC - .. data:: TYPE_IGNORE .. data:: TYPE_COMMENT diff --git a/Doc/library/token.rst b/Doc/library/token.rst index 903847bb206d6..e6dc37d7ad852 100644 --- a/Doc/library/token.rst +++ b/Doc/library/token.rst @@ -80,17 +80,21 @@ the :mod:`tokenize` module. .. versionchanged:: 3.5 - Added :data:`AWAIT` and :data:`ASYNC` tokens. + Added :data:`!AWAIT` and :data:`!ASYNC` tokens. .. versionchanged:: 3.7 Added :data:`COMMENT`, :data:`NL` and :data:`ENCODING` tokens. .. versionchanged:: 3.7 - Removed :data:`AWAIT` and :data:`ASYNC` tokens. "async" and "await" are + Removed :data:`!AWAIT` and :data:`!ASYNC` tokens. "async" and "await" are now tokenized as :data:`NAME` tokens. .. versionchanged:: 3.8 Added :data:`TYPE_COMMENT`, :data:`TYPE_IGNORE`, :data:`COLONEQUAL`. - Added :data:`AWAIT` and :data:`ASYNC` tokens back (they're needed + Added :data:`!AWAIT` and :data:`!ASYNC` tokens back (they're needed to support parsing older Python versions for :func:`ast.parse` with ``feature_version`` set to 6 or lower). + +.. versionchanged:: 3.13 + Removed :data:`!AWAIT` and :data:`!ASYNC` tokens again. + diff --git a/Grammar/Tokens b/Grammar/Tokens index 618ae811d824b..20bb803b7d58a 100644 --- a/Grammar/Tokens +++ b/Grammar/Tokens @@ -56,8 +56,6 @@ COLONEQUAL ':=' EXCLAMATION '!' OP -AWAIT -ASYNC TYPE_IGNORE TYPE_COMMENT SOFT_KEYWORD diff --git a/Grammar/python.gram b/Grammar/python.gram index c1863aec67cc2..e7c817856d514 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -127,11 +127,11 @@ simple_stmt[stmt_ty] (memo): | &'nonlocal' nonlocal_stmt compound_stmt[stmt_ty]: - | &('def' | '@' | ASYNC) function_def + | &('def' | '@' | 'async') function_def | &'if' if_stmt | &('class' | '@') class_def - | &('with' | ASYNC) with_stmt - | &('for' | ASYNC) for_stmt + | &('with' | 'async') with_stmt + | &('for' | 'async') for_stmt | &'try' try_stmt | &'while' while_stmt | match_stmt @@ -272,7 +272,7 @@ function_def_raw[stmt_ty]: _PyAST_FunctionDef(n->v.Name.id, (params) ? params : CHECK(arguments_ty, _PyPegen_empty_arguments(p)), b, NULL, a, NEW_TYPE_COMMENT(p, tc), t, EXTRA) } - | ASYNC 'def' n=NAME t=[type_params] &&'(' params=[params] ')' a=['->' z=expression { z }] &&':' tc=[func_type_comment] b=block { + | 'async' 'def' n=NAME t=[type_params] &&'(' params=[params] ')' a=['->' z=expression { z }] &&':' tc=[func_type_comment] b=block { CHECK_VERSION( stmt_ty, 5, @@ -385,7 +385,7 @@ for_stmt[stmt_ty]: | invalid_for_stmt | 'for' t=star_targets 'in' ~ ex=star_expressions ':' tc=[TYPE_COMMENT] b=block el=[else_block] { _PyAST_For(t, ex, b, el, NEW_TYPE_COMMENT(p, tc), EXTRA) } - | ASYNC 'for' t=star_targets 'in' ~ ex=star_expressions ':' tc=[TYPE_COMMENT] b=block el=[else_block] { + | 'async' 'for' t=star_targets 'in' ~ ex=star_expressions ':' tc=[TYPE_COMMENT] b=block el=[else_block] { CHECK_VERSION(stmt_ty, 5, "Async for loops are", _PyAST_AsyncFor(t, ex, b, el, NEW_TYPE_COMMENT(p, tc), EXTRA)) } | invalid_for_target @@ -398,9 +398,9 @@ with_stmt[stmt_ty]: CHECK_VERSION(stmt_ty, 9, "Parenthesized context managers are", _PyAST_With(a, b, NULL, EXTRA)) } | 'with' a[asdl_withitem_seq*]=','.with_item+ ':' tc=[TYPE_COMMENT] b=block { _PyAST_With(a, b, NEW_TYPE_COMMENT(p, tc), EXTRA) } - | ASYNC 'with' '(' a[asdl_withitem_seq*]=','.with_item+ ','? ')' ':' b=block { + | 'async' 'with' '(' a[asdl_withitem_seq*]=','.with_item+ ','? ')' ':' b=block { CHECK_VERSION(stmt_ty, 5, "Async with statements are", _PyAST_AsyncWith(a, b, NULL, EXTRA)) } - | ASYNC 'with' a[asdl_withitem_seq*]=','.with_item+ ':' tc=[TYPE_COMMENT] b=block { + | 'async' 'with' a[asdl_withitem_seq*]=','.with_item+ ':' tc=[TYPE_COMMENT] b=block { CHECK_VERSION(stmt_ty, 5, "Async with statements are", _PyAST_AsyncWith(a, b, NEW_TYPE_COMMENT(p, tc), EXTRA)) } | invalid_with_stmt @@ -814,7 +814,7 @@ power[expr_ty]: # Primary elements are things like "obj.something.something", "obj[something]", "obj(something)", "obj" ... await_primary[expr_ty] (memo): - | AWAIT a=primary { CHECK_VERSION(expr_ty, 5, "Await expressions are", _PyAST_Await(a, EXTRA)) } + | 'await' a=primary { CHECK_VERSION(expr_ty, 5, "Await expressions are", _PyAST_Await(a, EXTRA)) } | primary primary[expr_ty]: @@ -966,7 +966,7 @@ for_if_clauses[asdl_comprehension_seq*]: | a[asdl_comprehension_seq*]=for_if_clause+ { a } for_if_clause[comprehension_ty]: - | ASYNC 'for' a=star_targets 'in' ~ b=disjunction c[asdl_expr_seq*]=('if' z=disjunction { z })* { + | 'async' 'for' a=star_targets 'in' ~ b=disjunction c[asdl_expr_seq*]=('if' z=disjunction { z })* { CHECK_VERSION(comprehension_ty, 6, "Async comprehensions are", _PyAST_comprehension(a, b, c, 1, p->arena)) } | 'for' a=star_targets 'in' ~ b=disjunction c[asdl_expr_seq*]=('if' z=disjunction { z })* { _PyAST_comprehension(a, b, c, 0, p->arena) } @@ -1284,7 +1284,7 @@ invalid_with_item: RAISE_SYNTAX_ERROR_INVALID_TARGET(STAR_TARGETS, a) } invalid_for_target: - | ASYNC? 'for' a=star_expressions { + | 'async'? 'for' a=star_expressions { RAISE_SYNTAX_ERROR_INVALID_TARGET(FOR_TARGETS, a) } invalid_group: @@ -1301,12 +1301,12 @@ invalid_import_from_targets: RAISE_SYNTAX_ERROR("trailing comma not allowed without surrounding parentheses") } invalid_with_stmt: - | [ASYNC] 'with' ','.(expression ['as' star_target])+ NEWLINE { RAISE_SYNTAX_ERROR("expected ':'") } - | [ASYNC] 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE { RAISE_SYNTAX_ERROR("expected ':'") } + | ['async'] 'with' ','.(expression ['as' star_target])+ NEWLINE { RAISE_SYNTAX_ERROR("expected ':'") } + | ['async'] 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE { RAISE_SYNTAX_ERROR("expected ':'") } invalid_with_stmt_indent: - | [ASYNC] a='with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT { + | ['async'] a='with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT { RAISE_INDENTATION_ERROR("expected an indented block after 'with' statement on line %d", a->lineno) } - | [ASYNC] a='with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT { + | ['async'] a='with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT { RAISE_INDENTATION_ERROR("expected an indented block after 'with' statement on line %d", a->lineno) } invalid_try_stmt: @@ -1367,11 +1367,11 @@ invalid_while_stmt: | a='while' named_expression ':' NEWLINE !INDENT { RAISE_INDENTATION_ERROR("expected an indented block after 'while' statement on line %d", a->lineno) } invalid_for_stmt: - | [ASYNC] 'for' star_targets 'in' star_expressions NEWLINE { RAISE_SYNTAX_ERROR("expected ':'") } - | [ASYNC] a='for' star_targets 'in' star_expressions ':' NEWLINE !INDENT { + | ['async'] 'for' star_targets 'in' star_expressions NEWLINE { RAISE_SYNTAX_ERROR("expected ':'") } + | ['async'] a='for' star_targets 'in' star_expressions ':' NEWLINE !INDENT { RAISE_INDENTATION_ERROR("expected an indented block after 'for' statement on line %d", a->lineno) } invalid_def_raw: - | [ASYNC] a='def' NAME '(' [params] ')' ['->' expression] ':' NEWLINE !INDENT { + | ['async'] a='def' NAME '(' [params] ')' ['->' expression] ':' NEWLINE !INDENT { RAISE_INDENTATION_ERROR("expected an indented block after function definition on line %d", a->lineno) } invalid_class_def_raw: | 'class' NAME ['(' [arguments] ')'] NEWLINE { RAISE_SYNTAX_ERROR("expected ':'") } diff --git a/Include/internal/pycore_token.h b/Include/internal/pycore_token.h index c02e637fee1ee..9c65cd802d597 100644 --- a/Include/internal/pycore_token.h +++ b/Include/internal/pycore_token.h @@ -69,18 +69,16 @@ extern "C" { #define COLONEQUAL 53 #define EXCLAMATION 54 #define OP 55 -#define AWAIT 56 -#define ASYNC 57 -#define TYPE_IGNORE 58 -#define TYPE_COMMENT 59 -#define SOFT_KEYWORD 60 -#define FSTRING_START 61 -#define FSTRING_MIDDLE 62 -#define FSTRING_END 63 -#define COMMENT 64 -#define NL 65 -#define ERRORTOKEN 66 -#define N_TOKENS 68 +#define TYPE_IGNORE 56 +#define TYPE_COMMENT 57 +#define SOFT_KEYWORD 58 +#define FSTRING_START 59 +#define FSTRING_MIDDLE 60 +#define FSTRING_END 61 +#define COMMENT 62 +#define NL 63 +#define ERRORTOKEN 64 +#define N_TOKENS 66 #define NT_OFFSET 256 /* Special definitions for cooperation with parser */ diff --git a/Lib/test/test_peg_generator/test_c_parser.py b/Lib/test/test_peg_generator/test_c_parser.py index f9105a9f23bd6..9e273e99e387a 100644 --- a/Lib/test/test_peg_generator/test_c_parser.py +++ b/Lib/test/test_peg_generator/test_c_parser.py @@ -404,7 +404,7 @@ def test_ternary_operator(self) -> None: a='[' b=NAME c=for_if_clauses d=']' { _PyAST_ListComp(b, c, EXTRA) } ) for_if_clauses[asdl_comprehension_seq*]: ( - a[asdl_comprehension_seq*]=(y=[ASYNC] 'for' a=NAME 'in' b=NAME c[asdl_expr_seq*]=('if' z=NAME { z })* + a[asdl_comprehension_seq*]=(y=['async'] 'for' a=NAME 'in' b=NAME c[asdl_expr_seq*]=('if' z=NAME { z })* { _PyAST_comprehension(_PyAST_Name(((expr_ty) a)->v.Name.id, Store, EXTRA), b, c, (y == NULL) ? 0 : 1, p->arena) })+ { a } ) """ diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py index d1552d8a20808..7863e27fccd97 100644 --- a/Lib/test/test_tokenize.py +++ b/Lib/test/test_tokenize.py @@ -2521,7 +2521,7 @@ def test_tabs(self): def test_async(self): self.check_tokenize('async = 1', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) EQUAL '=' (1, 6) (1, 7) NUMBER '1' (1, 8) (1, 9) """) @@ -2530,21 +2530,21 @@ def test_async(self): NAME 'a' (1, 0) (1, 1) EQUAL '=' (1, 2) (1, 3) LPAR '(' (1, 4) (1, 5) - ASYNC 'async' (1, 5) (1, 10) + NAME 'async' (1, 5) (1, 10) EQUAL '=' (1, 11) (1, 12) NUMBER '1' (1, 13) (1, 14) RPAR ')' (1, 14) (1, 15) """) self.check_tokenize('async()', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) LPAR '(' (1, 5) (1, 6) RPAR ')' (1, 6) (1, 7) """) self.check_tokenize('class async(Bar):pass', """\ NAME 'class' (1, 0) (1, 5) - ASYNC 'async' (1, 6) (1, 11) + NAME 'async' (1, 6) (1, 11) LPAR '(' (1, 11) (1, 12) NAME 'Bar' (1, 12) (1, 15) RPAR ')' (1, 15) (1, 16) @@ -2554,13 +2554,13 @@ def test_async(self): self.check_tokenize('class async:pass', """\ NAME 'class' (1, 0) (1, 5) - ASYNC 'async' (1, 6) (1, 11) + NAME 'async' (1, 6) (1, 11) COLON ':' (1, 11) (1, 12) NAME 'pass' (1, 12) (1, 16) """) self.check_tokenize('await = 1', """\ - AWAIT 'await' (1, 0) (1, 5) + NAME 'await' (1, 0) (1, 5) EQUAL '=' (1, 6) (1, 7) NUMBER '1' (1, 8) (1, 9) """) @@ -2568,11 +2568,11 @@ def test_async(self): self.check_tokenize('foo.async', """\ NAME 'foo' (1, 0) (1, 3) DOT '.' (1, 3) (1, 4) - ASYNC 'async' (1, 4) (1, 9) + NAME 'async' (1, 4) (1, 9) """) self.check_tokenize('async for a in b: pass', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NAME 'for' (1, 6) (1, 9) NAME 'a' (1, 10) (1, 11) NAME 'in' (1, 12) (1, 14) @@ -2582,7 +2582,7 @@ def test_async(self): """) self.check_tokenize('async with a as b: pass', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NAME 'with' (1, 6) (1, 10) NAME 'a' (1, 11) (1, 12) NAME 'as' (1, 13) (1, 15) @@ -2592,45 +2592,45 @@ def test_async(self): """) self.check_tokenize('async.foo', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) DOT '.' (1, 5) (1, 6) NAME 'foo' (1, 6) (1, 9) """) self.check_tokenize('async', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) """) self.check_tokenize('async\n#comment\nawait', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NEWLINE '' (1, 5) (1, 5) - AWAIT 'await' (3, 0) (3, 5) + NAME 'await' (3, 0) (3, 5) """) self.check_tokenize('async\n...\nawait', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NEWLINE '' (1, 5) (1, 5) ELLIPSIS '...' (2, 0) (2, 3) NEWLINE '' (2, 3) (2, 3) - AWAIT 'await' (3, 0) (3, 5) + NAME 'await' (3, 0) (3, 5) """) self.check_tokenize('async\nawait', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NEWLINE '' (1, 5) (1, 5) - AWAIT 'await' (2, 0) (2, 5) + NAME 'await' (2, 0) (2, 5) """) self.check_tokenize('foo.async + 1', """\ NAME 'foo' (1, 0) (1, 3) DOT '.' (1, 3) (1, 4) - ASYNC 'async' (1, 4) (1, 9) + NAME 'async' (1, 4) (1, 9) PLUS '+' (1, 10) (1, 11) NUMBER '1' (1, 12) (1, 13) """) self.check_tokenize('async def foo(): pass', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NAME 'def' (1, 6) (1, 9) NAME 'foo' (1, 10) (1, 13) LPAR '(' (1, 13) (1, 14) @@ -2647,7 +2647,7 @@ def foo(await): await async += 1 ''', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NAME 'def' (1, 6) (1, 9) NAME 'foo' (1, 10) (1, 13) LPAR '(' (1, 13) (1, 14) @@ -2658,12 +2658,12 @@ def foo(await): NAME 'def' (2, 2) (2, 5) NAME 'foo' (2, 6) (2, 9) LPAR '(' (2, 9) (2, 10) - AWAIT 'await' (2, 10) (2, 15) + NAME 'await' (2, 10) (2, 15) RPAR ')' (2, 15) (2, 16) COLON ':' (2, 16) (2, 17) NEWLINE '' (2, 17) (2, 17) INDENT '' (3, -1) (3, -1) - AWAIT 'await' (3, 4) (3, 9) + NAME 'await' (3, 4) (3, 9) EQUAL '=' (3, 10) (3, 11) NUMBER '1' (3, 12) (3, 13) NEWLINE '' (3, 13) (3, 13) @@ -2673,18 +2673,18 @@ def foo(await): COLON ':' (4, 6) (4, 7) NEWLINE '' (4, 7) (4, 7) INDENT '' (5, -1) (5, -1) - AWAIT 'await' (5, 4) (5, 9) + NAME 'await' (5, 4) (5, 9) NEWLINE '' (5, 9) (5, 9) DEDENT '' (6, -1) (6, -1) DEDENT '' (6, -1) (6, -1) - ASYNC 'async' (6, 0) (6, 5) + NAME 'async' (6, 0) (6, 5) PLUSEQUAL '+=' (6, 6) (6, 8) NUMBER '1' (6, 9) (6, 10) NEWLINE '' (6, 10) (6, 10) """) self.check_tokenize('async def foo():\n async for i in 1: pass', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NAME 'def' (1, 6) (1, 9) NAME 'foo' (1, 10) (1, 13) LPAR '(' (1, 13) (1, 14) @@ -2692,7 +2692,7 @@ def foo(await): COLON ':' (1, 15) (1, 16) NEWLINE '' (1, 16) (1, 16) INDENT '' (2, -1) (2, -1) - ASYNC 'async' (2, 2) (2, 7) + NAME 'async' (2, 2) (2, 7) NAME 'for' (2, 8) (2, 11) NAME 'i' (2, 12) (2, 13) NAME 'in' (2, 14) (2, 16) @@ -2703,14 +2703,14 @@ def foo(await): """) self.check_tokenize('async def foo(async): await', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NAME 'def' (1, 6) (1, 9) NAME 'foo' (1, 10) (1, 13) LPAR '(' (1, 13) (1, 14) - ASYNC 'async' (1, 14) (1, 19) + NAME 'async' (1, 14) (1, 19) RPAR ')' (1, 19) (1, 20) COLON ':' (1, 20) (1, 21) - AWAIT 'await' (1, 22) (1, 27) + NAME 'await' (1, 22) (1, 27) """) self.check_tokenize('''\ @@ -2734,7 +2734,7 @@ async def bar(): pass COLON ':' (3, 11) (3, 12) NAME 'pass' (3, 13) (3, 17) NEWLINE '' (3, 17) (3, 17) - ASYNC 'async' (4, 2) (4, 7) + NAME 'async' (4, 2) (4, 7) NAME 'def' (4, 8) (4, 11) NAME 'bar' (4, 12) (4, 15) LPAR '(' (4, 15) (4, 16) @@ -2742,7 +2742,7 @@ async def bar(): pass COLON ':' (4, 17) (4, 18) NAME 'pass' (4, 19) (4, 23) NEWLINE '' (4, 23) (4, 23) - AWAIT 'await' (6, 2) (6, 7) + NAME 'await' (6, 2) (6, 7) EQUAL '=' (6, 8) (6, 9) NUMBER '2' (6, 10) (6, 11) DEDENT '' (6, -1) (6, -1) @@ -2755,7 +2755,7 @@ def baz(): pass async def bar(): pass await = 2''', """\ - ASYNC 'async' (1, 0) (1, 5) + NAME 'async' (1, 0) (1, 5) NAME 'def' (1, 6) (1, 9) NAME 'f' (1, 10) (1, 11) LPAR '(' (1, 11) (1, 12) @@ -2770,7 +2770,7 @@ async def bar(): pass COLON ':' (3, 11) (3, 12) NAME 'pass' (3, 13) (3, 17) NEWLINE '' (3, 17) (3, 17) - ASYNC 'async' (4, 2) (4, 7) + NAME 'async' (4, 2) (4, 7) NAME 'def' (4, 8) (4, 11) NAME 'bar' (4, 12) (4, 15) LPAR '(' (4, 15) (4, 16) @@ -2778,7 +2778,7 @@ async def bar(): pass COLON ':' (4, 17) (4, 18) NAME 'pass' (4, 19) (4, 23) NEWLINE '' (4, 23) (4, 23) - AWAIT 'await' (6, 2) (6, 7) + NAME 'await' (6, 2) (6, 7) EQUAL '=' (6, 8) (6, 9) NUMBER '2' (6, 10) (6, 11) DEDENT '' (6, -1) (6, -1) diff --git a/Lib/test/test_type_comments.py b/Lib/test/test_type_comments.py index aba4a44be9da9..9a11fab237235 100644 --- a/Lib/test/test_type_comments.py +++ b/Lib/test/test_type_comments.py @@ -260,8 +260,8 @@ def test_asyncdef(self): self.assertEqual(tree.body[1].type_comment, None) def test_asyncvar(self): - for tree in self.parse_all(asyncvar, maxver=6): - pass + with self.assertRaises(SyntaxError): + self.classic_parse(asyncvar) def test_asynccomp(self): for tree in self.parse_all(asynccomp, minver=6): diff --git a/Lib/token.py b/Lib/token.py index 487f6edd3c951..b620317106e17 100644 --- a/Lib/token.py +++ b/Lib/token.py @@ -59,20 +59,18 @@ COLONEQUAL = 53 EXCLAMATION = 54 OP = 55 -AWAIT = 56 -ASYNC = 57 -TYPE_IGNORE = 58 -TYPE_COMMENT = 59 -SOFT_KEYWORD = 60 -FSTRING_START = 61 -FSTRING_MIDDLE = 62 -FSTRING_END = 63 -COMMENT = 64 -NL = 65 +TYPE_IGNORE = 56 +TYPE_COMMENT = 57 +SOFT_KEYWORD = 58 +FSTRING_START = 59 +FSTRING_MIDDLE = 60 +FSTRING_END = 61 +COMMENT = 62 +NL = 63 # These aren't used by the C tokenizer but are needed for tokenize.py -ERRORTOKEN = 66 -ENCODING = 67 -N_TOKENS = 68 +ERRORTOKEN = 64 +ENCODING = 65 +N_TOKENS = 66 # Special definitions for cooperation with parser NT_OFFSET = 256 diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-22-14-35-38.gh-issue-107015.Ghp58t.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-22-14-35-38.gh-issue-107015.Ghp58t.rst new file mode 100644 index 0000000000000..77618a5bd50f2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-22-14-35-38.gh-issue-107015.Ghp58t.rst @@ -0,0 +1,3 @@ +The ASYNC and AWAIT tokens are removed from the Grammar, which removes the +posibility of making ``async`` and ``await`` soft keywords when using +``feature_version<7`` in :func:`ast.parse`. diff --git a/Parser/parser.c b/Parser/parser.c index f2ea8f59b0056..44312cff125ae 100644 --- a/Parser/parser.c +++ b/Parser/parser.c @@ -17,57 +17,59 @@ static KeywordToken *reserved_keywords[] = { (KeywordToken[]) {{NULL, -1}}, (KeywordToken[]) {{NULL, -1}}, (KeywordToken[]) { - {"if", 642}, - {"as", 640}, - {"in", 651}, - {"or", 574}, - {"is", 582}, + {"if", 656}, + {"as", 654}, + {"in", 667}, + {"or", 581}, + {"is", 589}, {NULL, -1}, }, (KeywordToken[]) { - {"del", 604}, - {"def", 652}, - {"for", 650}, - {"try", 624}, - {"and", 575}, - {"not", 581}, + {"del", 613}, + {"def", 669}, + {"for", 666}, + {"try", 638}, + {"and", 582}, + {"not", 588}, {NULL, -1}, }, (KeywordToken[]) { - {"from", 608}, + {"from", 618}, {"pass", 504}, - {"with", 615}, - {"elif", 644}, - {"else", 645}, - {"None", 602}, - {"True", 601}, + {"with", 629}, + {"elif", 658}, + {"else", 659}, + {"None", 611}, + {"True", 610}, {NULL, -1}, }, (KeywordToken[]) { - {"raise", 522}, - {"yield", 573}, + {"raise", 525}, + {"yield", 580}, {"break", 508}, - {"class", 654}, - {"while", 647}, - {"False", 603}, + {"async", 668}, + {"class", 671}, + {"while", 661}, + {"False", 612}, + {"await", 590}, {NULL, -1}, }, (KeywordToken[]) { - {"return", 519}, - {"import", 607}, - {"assert", 526}, - {"global", 523}, - {"except", 637}, - {"lambda", 600}, + {"return", 522}, + {"import", 617}, + {"assert", 529}, + {"global", 526}, + {"except", 651}, + {"lambda", 609}, {NULL, -1}, }, (KeywordToken[]) { - {"finally", 633}, + {"finally", 647}, {NULL, -1}, }, (KeywordToken[]) { {"continue", 509}, - {"nonlocal", 524}, + {"nonlocal", 527}, {NULL, -1}, }, }; @@ -1820,7 +1822,7 @@ simple_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'return' return_stmt")); stmt_ty return_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 519) // token='return' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 522) // token='return' && (return_stmt_var = return_stmt_rule(p)) // return_stmt ) @@ -1862,7 +1864,7 @@ simple_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'raise' raise_stmt")); stmt_ty raise_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 522) // token='raise' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 525) // token='raise' && (raise_stmt_var = raise_stmt_rule(p)) // raise_stmt ) @@ -1916,7 +1918,7 @@ simple_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'del' del_stmt")); stmt_ty del_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 604) // token='del' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 613) // token='del' && (del_stmt_var = del_stmt_rule(p)) // del_stmt ) @@ -1937,7 +1939,7 @@ simple_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'yield' yield_stmt")); stmt_ty yield_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 573) // token='yield' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 580) // token='yield' && (yield_stmt_var = yield_stmt_rule(p)) // yield_stmt ) @@ -1958,7 +1960,7 @@ simple_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'assert' assert_stmt")); stmt_ty assert_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 526) // token='assert' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 529) // token='assert' && (assert_stmt_var = assert_stmt_rule(p)) // assert_stmt ) @@ -2045,7 +2047,7 @@ simple_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'global' global_stmt")); stmt_ty global_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 523) // token='global' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 526) // token='global' && (global_stmt_var = global_stmt_rule(p)) // global_stmt ) @@ -2066,7 +2068,7 @@ simple_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'nonlocal' nonlocal_stmt")); stmt_ty nonlocal_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 524) // token='nonlocal' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 527) // token='nonlocal' && (nonlocal_stmt_var = nonlocal_stmt_rule(p)) // nonlocal_stmt ) @@ -2087,11 +2089,11 @@ simple_stmt_rule(Parser *p) } // compound_stmt: -// | &('def' | '@' | ASYNC) function_def +// | &('def' | '@' | 'async') function_def // | &'if' if_stmt // | &('class' | '@') class_def -// | &('with' | ASYNC) with_stmt -// | &('for' | ASYNC) for_stmt +// | &('with' | 'async') with_stmt +// | &('for' | 'async') for_stmt // | &'try' try_stmt // | &'while' while_stmt // | match_stmt @@ -2108,12 +2110,12 @@ compound_stmt_rule(Parser *p) } stmt_ty _res = NULL; int _mark = p->mark; - { // &('def' | '@' | ASYNC) function_def + { // &('def' | '@' | 'async') function_def if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> compound_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&('def' | '@' | ASYNC) function_def")); + D(fprintf(stderr, "%*c> compound_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&('def' | '@' | 'async') function_def")); stmt_ty function_def_var; if ( _PyPegen_lookahead(1, _tmp_8_rule, p) @@ -2121,13 +2123,13 @@ compound_stmt_rule(Parser *p) (function_def_var = function_def_rule(p)) // function_def ) { - D(fprintf(stderr, "%*c+ compound_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "&('def' | '@' | ASYNC) function_def")); + D(fprintf(stderr, "%*c+ compound_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "&('def' | '@' | 'async') function_def")); _res = function_def_var; goto done; } p->mark = _mark; D(fprintf(stderr, "%*c%s compound_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&('def' | '@' | ASYNC) function_def")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&('def' | '@' | 'async') function_def")); } { // &'if' if_stmt if (p->error_indicator) { @@ -2137,7 +2139,7 @@ compound_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> compound_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'if' if_stmt")); stmt_ty if_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 642) // token='if' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 656) // token='if' && (if_stmt_var = if_stmt_rule(p)) // if_stmt ) @@ -2171,12 +2173,12 @@ compound_stmt_rule(Parser *p) D(fprintf(stderr, "%*c%s compound_stmt[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&('class' | '@') class_def")); } - { // &('with' | ASYNC) with_stmt + { // &('with' | 'async') with_stmt if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> compound_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&('with' | ASYNC) with_stmt")); + D(fprintf(stderr, "%*c> compound_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&('with' | 'async') with_stmt")); stmt_ty with_stmt_var; if ( _PyPegen_lookahead(1, _tmp_10_rule, p) @@ -2184,20 +2186,20 @@ compound_stmt_rule(Parser *p) (with_stmt_var = with_stmt_rule(p)) // with_stmt ) { - D(fprintf(stderr, "%*c+ compound_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "&('with' | ASYNC) with_stmt")); + D(fprintf(stderr, "%*c+ compound_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "&('with' | 'async') with_stmt")); _res = with_stmt_var; goto done; } p->mark = _mark; D(fprintf(stderr, "%*c%s compound_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&('with' | ASYNC) with_stmt")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&('with' | 'async') with_stmt")); } - { // &('for' | ASYNC) for_stmt + { // &('for' | 'async') for_stmt if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> compound_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&('for' | ASYNC) for_stmt")); + D(fprintf(stderr, "%*c> compound_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&('for' | 'async') for_stmt")); stmt_ty for_stmt_var; if ( _PyPegen_lookahead(1, _tmp_11_rule, p) @@ -2205,13 +2207,13 @@ compound_stmt_rule(Parser *p) (for_stmt_var = for_stmt_rule(p)) // for_stmt ) { - D(fprintf(stderr, "%*c+ compound_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "&('for' | ASYNC) for_stmt")); + D(fprintf(stderr, "%*c+ compound_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "&('for' | 'async') for_stmt")); _res = for_stmt_var; goto done; } p->mark = _mark; D(fprintf(stderr, "%*c%s compound_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&('for' | ASYNC) for_stmt")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&('for' | 'async') for_stmt")); } { // &'try' try_stmt if (p->error_indicator) { @@ -2221,7 +2223,7 @@ compound_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> compound_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'try' try_stmt")); stmt_ty try_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 624) // token='try' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 638) // token='try' && (try_stmt_var = try_stmt_rule(p)) // try_stmt ) @@ -2242,7 +2244,7 @@ compound_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> compound_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&'while' while_stmt")); stmt_ty while_stmt_var; if ( - _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 647) // token='while' + _PyPegen_lookahead_with_int(1, _PyPegen_expect_token, p, 661) // token='while' && (while_stmt_var = while_stmt_rule(p)) // while_stmt ) @@ -2939,7 +2941,7 @@ return_stmt_rule(Parser *p) Token * _keyword; void *a; if ( - (_keyword = _PyPegen_expect_token(p, 519)) // token='return' + (_keyword = _PyPegen_expect_token(p, 522)) // token='return' && (a = star_expressions_rule(p), !p->error_indicator) // star_expressions? ) @@ -3005,7 +3007,7 @@ raise_stmt_rule(Parser *p) expr_ty a; void *b; if ( - (_keyword = _PyPegen_expect_token(p, 522)) // token='raise' + (_keyword = _PyPegen_expect_token(p, 525)) // token='raise' && (a = expression_rule(p)) // expression && @@ -3042,7 +3044,7 @@ raise_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> raise_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'raise'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 522)) // token='raise' + (_keyword = _PyPegen_expect_token(p, 525)) // token='raise' ) { D(fprintf(stderr, "%*c+ raise_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'raise'")); @@ -3105,7 +3107,7 @@ global_stmt_rule(Parser *p) Token * _keyword; asdl_expr_seq* a; if ( - (_keyword = _PyPegen_expect_token(p, 523)) // token='global' + (_keyword = _PyPegen_expect_token(p, 526)) // token='global' && (a = (asdl_expr_seq*)_gather_19_rule(p)) // ','.NAME+ ) @@ -3170,7 +3172,7 @@ nonlocal_stmt_rule(Parser *p) Token * _keyword; asdl_expr_seq* a; if ( - (_keyword = _PyPegen_expect_token(p, 524)) // token='nonlocal' + (_keyword = _PyPegen_expect_token(p, 527)) // token='nonlocal' && (a = (asdl_expr_seq*)_gather_21_rule(p)) // ','.NAME+ ) @@ -3235,7 +3237,7 @@ del_stmt_rule(Parser *p) Token * _keyword; asdl_expr_seq* a; if ( - (_keyword = _PyPegen_expect_token(p, 604)) // token='del' + (_keyword = _PyPegen_expect_token(p, 613)) // token='del' && (a = del_targets_rule(p)) // del_targets && @@ -3384,7 +3386,7 @@ assert_stmt_rule(Parser *p) expr_ty a; void *b; if ( - (_keyword = _PyPegen_expect_token(p, 526)) // token='assert' + (_keyword = _PyPegen_expect_token(p, 529)) // token='assert' && (a = expression_rule(p)) // expression && @@ -3528,7 +3530,7 @@ import_name_rule(Parser *p) Token * _keyword; asdl_alias_seq* a; if ( - (_keyword = _PyPegen_expect_token(p, 607)) // token='import' + (_keyword = _PyPegen_expect_token(p, 617)) // token='import' && (a = dotted_as_names_rule(p)) // dotted_as_names ) @@ -3598,13 +3600,13 @@ import_from_rule(Parser *p) expr_ty b; asdl_alias_seq* c; if ( - (_keyword = _PyPegen_expect_token(p, 608)) // token='from' + (_keyword = _PyPegen_expect_token(p, 618)) // token='from' && (a = _loop0_25_rule(p)) // (('.' | '...'))* && (b = dotted_name_rule(p)) // dotted_name && - (_keyword_1 = _PyPegen_expect_token(p, 607)) // token='import' + (_keyword_1 = _PyPegen_expect_token(p, 617)) // token='import' && (c = import_from_targets_rule(p)) // import_from_targets ) @@ -3642,11 +3644,11 @@ import_from_rule(Parser *p) asdl_seq * a; asdl_alias_seq* b; if ( - (_keyword = _PyPegen_expect_token(p, 608)) // token='from' + (_keyword = _PyPegen_expect_token(p, 618)) // token='from' && (a = _loop1_26_rule(p)) // (('.' | '...'))+ && - (_keyword_1 = _PyPegen_expect_token(p, 607)) // token='import' + (_keyword_1 = _PyPegen_expect_token(p, 617)) // token='import' && (b = import_from_targets_rule(p)) // import_from_targets ) @@ -4406,7 +4408,7 @@ class_def_raw_rule(Parser *p) asdl_stmt_seq* c; void *t; if ( - (_keyword = _PyPegen_expect_token(p, 654)) // token='class' + (_keyword = _PyPegen_expect_token(p, 671)) // token='class' && (a = _PyPegen_name_token(p)) // NAME && @@ -4516,7 +4518,7 @@ function_def_rule(Parser *p) // function_def_raw: // | invalid_def_raw // | 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block -// | ASYNC 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block +// | 'async' 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block static stmt_ty function_def_raw_rule(Parser *p) { @@ -4575,7 +4577,7 @@ function_def_raw_rule(Parser *p) void *t; void *tc; if ( - (_keyword = _PyPegen_expect_token(p, 652)) // token='def' + (_keyword = _PyPegen_expect_token(p, 669)) // token='def' && (n = _PyPegen_name_token(p)) // NAME && @@ -4618,27 +4620,27 @@ function_def_raw_rule(Parser *p) D(fprintf(stderr, "%*c%s function_def_raw[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block")); } - { // ASYNC 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block + { // 'async' 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> function_def_raw[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block")); + D(fprintf(stderr, "%*c> function_def_raw[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async' 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block")); Token * _keyword; + Token * _keyword_1; Token * _literal; Token * _literal_1; Token * _literal_2; void *a; - Token * async_var; asdl_stmt_seq* b; expr_ty n; void *params; void *t; void *tc; if ( - (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' + (_keyword = _PyPegen_expect_token(p, 668)) // token='async' && - (_keyword = _PyPegen_expect_token(p, 652)) // token='def' + (_keyword_1 = _PyPegen_expect_token(p, 669)) // token='def' && (n = _PyPegen_name_token(p)) // NAME && @@ -4659,7 +4661,7 @@ function_def_raw_rule(Parser *p) (b = block_rule(p)) // block ) { - D(fprintf(stderr, "%*c+ function_def_raw[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block")); + D(fprintf(stderr, "%*c+ function_def_raw[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async' 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block")); Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); if (_token == NULL) { p->level--; @@ -4679,7 +4681,7 @@ function_def_raw_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s function_def_raw[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async' 'def' NAME type_params? &&'(' params? ')' ['->' expression] &&':' func_type_comment? block")); } _res = NULL; done: @@ -5992,7 +5994,7 @@ if_stmt_rule(Parser *p) asdl_stmt_seq* b; stmt_ty c; if ( - (_keyword = _PyPegen_expect_token(p, 642)) // token='if' + (_keyword = _PyPegen_expect_token(p, 656)) // token='if' && (a = named_expression_rule(p)) // named_expression && @@ -6037,7 +6039,7 @@ if_stmt_rule(Parser *p) asdl_stmt_seq* b; void *c; if ( - (_keyword = _PyPegen_expect_token(p, 642)) // token='if' + (_keyword = _PyPegen_expect_token(p, 656)) // token='if' && (a = named_expression_rule(p)) // named_expression && @@ -6133,7 +6135,7 @@ elif_stmt_rule(Parser *p) asdl_stmt_seq* b; stmt_ty c; if ( - (_keyword = _PyPegen_expect_token(p, 644)) // token='elif' + (_keyword = _PyPegen_expect_token(p, 658)) // token='elif' && (a = named_expression_rule(p)) // named_expression && @@ -6178,7 +6180,7 @@ elif_stmt_rule(Parser *p) asdl_stmt_seq* b; void *c; if ( - (_keyword = _PyPegen_expect_token(p, 644)) // token='elif' + (_keyword = _PyPegen_expect_token(p, 658)) // token='elif' && (a = named_expression_rule(p)) // named_expression && @@ -6260,7 +6262,7 @@ else_block_rule(Parser *p) Token * _literal; asdl_stmt_seq* b; if ( - (_keyword = _PyPegen_expect_token(p, 645)) // token='else' + (_keyword = _PyPegen_expect_token(p, 659)) // token='else' && (_literal = _PyPegen_expect_forced_token(p, 11, ":")) // forced_token=':' && @@ -6340,7 +6342,7 @@ while_stmt_rule(Parser *p) asdl_stmt_seq* b; void *c; if ( - (_keyword = _PyPegen_expect_token(p, 647)) // token='while' + (_keyword = _PyPegen_expect_token(p, 661)) // token='while' && (a = named_expression_rule(p)) // named_expression && @@ -6382,7 +6384,7 @@ while_stmt_rule(Parser *p) // for_stmt: // | invalid_for_stmt // | 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block? -// | ASYNC 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block? +// | 'async' 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block? // | invalid_for_target static stmt_ty for_stmt_rule(Parser *p) @@ -6441,11 +6443,11 @@ for_stmt_rule(Parser *p) expr_ty t; void *tc; if ( - (_keyword = _PyPegen_expect_token(p, 650)) // token='for' + (_keyword = _PyPegen_expect_token(p, 666)) // token='for' && (t = star_targets_rule(p)) // star_targets && - (_keyword_1 = _PyPegen_expect_token(p, 651)) // token='in' + (_keyword_1 = _PyPegen_expect_token(p, 667)) // token='in' && (_cut_var = 1) && @@ -6486,30 +6488,30 @@ for_stmt_rule(Parser *p) return NULL; } } - { // ASYNC 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block? + { // 'async' 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block? if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> for_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block?")); + D(fprintf(stderr, "%*c> for_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async' 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block?")); int _cut_var = 0; Token * _keyword; Token * _keyword_1; + Token * _keyword_2; Token * _literal; - Token * async_var; asdl_stmt_seq* b; void *el; expr_ty ex; expr_ty t; void *tc; if ( - (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' + (_keyword = _PyPegen_expect_token(p, 668)) // token='async' && - (_keyword = _PyPegen_expect_token(p, 650)) // token='for' + (_keyword_1 = _PyPegen_expect_token(p, 666)) // token='for' && (t = star_targets_rule(p)) // star_targets && - (_keyword_1 = _PyPegen_expect_token(p, 651)) // token='in' + (_keyword_2 = _PyPegen_expect_token(p, 667)) // token='in' && (_cut_var = 1) && @@ -6524,7 +6526,7 @@ for_stmt_rule(Parser *p) (el = else_block_rule(p), !p->error_indicator) // else_block? ) { - D(fprintf(stderr, "%*c+ for_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block?")); + D(fprintf(stderr, "%*c+ for_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async' 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block?")); Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); if (_token == NULL) { p->level--; @@ -6544,7 +6546,7 @@ for_stmt_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s for_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block?")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async' 'for' star_targets 'in' ~ star_expressions ':' TYPE_COMMENT? block else_block?")); if (_cut_var) { p->level--; return NULL; @@ -6579,8 +6581,8 @@ for_stmt_rule(Parser *p) // | invalid_with_stmt_indent // | 'with' '(' ','.with_item+ ','? ')' ':' block // | 'with' ','.with_item+ ':' TYPE_COMMENT? block -// | ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block -// | ASYNC 'with' ','.with_item+ ':' TYPE_COMMENT? block +// | 'async' 'with' '(' ','.with_item+ ','? ')' ':' block +// | 'async' 'with' ','.with_item+ ':' TYPE_COMMENT? block // | invalid_with_stmt static stmt_ty with_stmt_rule(Parser *p) @@ -6638,7 +6640,7 @@ with_stmt_rule(Parser *p) asdl_withitem_seq* a; asdl_stmt_seq* b; if ( - (_keyword = _PyPegen_expect_token(p, 615)) // token='with' + (_keyword = _PyPegen_expect_token(p, 629)) // token='with' && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && @@ -6687,7 +6689,7 @@ with_stmt_rule(Parser *p) asdl_stmt_seq* b; void *tc; if ( - (_keyword = _PyPegen_expect_token(p, 615)) // token='with' + (_keyword = _PyPegen_expect_token(p, 629)) // token='with' && (a = (asdl_withitem_seq*)_gather_54_rule(p)) // ','.with_item+ && @@ -6720,25 +6722,25 @@ with_stmt_rule(Parser *p) D(fprintf(stderr, "%*c%s with_stmt[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'with' ','.with_item+ ':' TYPE_COMMENT? block")); } - { // ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block + { // 'async' 'with' '(' ','.with_item+ ','? ')' ':' block if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block")); + D(fprintf(stderr, "%*c> with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async' 'with' '(' ','.with_item+ ','? ')' ':' block")); Token * _keyword; + Token * _keyword_1; Token * _literal; Token * _literal_1; Token * _literal_2; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings asdl_withitem_seq* a; - Token * async_var; asdl_stmt_seq* b; if ( - (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' + (_keyword = _PyPegen_expect_token(p, 668)) // token='async' && - (_keyword = _PyPegen_expect_token(p, 615)) // token='with' + (_keyword_1 = _PyPegen_expect_token(p, 629)) // token='with' && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && @@ -6753,7 +6755,7 @@ with_stmt_rule(Parser *p) (b = block_rule(p)) // block ) { - D(fprintf(stderr, "%*c+ with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block")); + D(fprintf(stderr, "%*c+ with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async' 'with' '(' ','.with_item+ ','? ')' ':' block")); Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); if (_token == NULL) { p->level--; @@ -6773,24 +6775,24 @@ with_stmt_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s with_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async' 'with' '(' ','.with_item+ ','? ')' ':' block")); } - { // ASYNC 'with' ','.with_item+ ':' TYPE_COMMENT? block + { // 'async' 'with' ','.with_item+ ':' TYPE_COMMENT? block if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC 'with' ','.with_item+ ':' TYPE_COMMENT? block")); + D(fprintf(stderr, "%*c> with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async' 'with' ','.with_item+ ':' TYPE_COMMENT? block")); Token * _keyword; + Token * _keyword_1; Token * _literal; asdl_withitem_seq* a; - Token * async_var; asdl_stmt_seq* b; void *tc; if ( - (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' + (_keyword = _PyPegen_expect_token(p, 668)) // token='async' && - (_keyword = _PyPegen_expect_token(p, 615)) // token='with' + (_keyword_1 = _PyPegen_expect_token(p, 629)) // token='with' && (a = (asdl_withitem_seq*)_gather_58_rule(p)) // ','.with_item+ && @@ -6801,7 +6803,7 @@ with_stmt_rule(Parser *p) (b = block_rule(p)) // block ) { - D(fprintf(stderr, "%*c+ with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC 'with' ','.with_item+ ':' TYPE_COMMENT? block")); + D(fprintf(stderr, "%*c+ with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async' 'with' ','.with_item+ ':' TYPE_COMMENT? block")); Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); if (_token == NULL) { p->level--; @@ -6821,7 +6823,7 @@ with_stmt_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s with_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC 'with' ','.with_item+ ':' TYPE_COMMENT? block")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async' 'with' ','.with_item+ ':' TYPE_COMMENT? block")); } if (p->call_invalid_rules) { // invalid_with_stmt if (p->error_indicator) { @@ -6877,7 +6879,7 @@ with_item_rule(Parser *p) if ( (e = expression_rule(p)) // expression && - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (t = star_target_rule(p)) // star_target && @@ -7003,7 +7005,7 @@ try_stmt_rule(Parser *p) asdl_stmt_seq* b; asdl_stmt_seq* f; if ( - (_keyword = _PyPegen_expect_token(p, 624)) // token='try' + (_keyword = _PyPegen_expect_token(p, 638)) // token='try' && (_literal = _PyPegen_expect_forced_token(p, 11, ":")) // forced_token=':' && @@ -7047,7 +7049,7 @@ try_stmt_rule(Parser *p) asdl_excepthandler_seq* ex; void *f; if ( - (_keyword = _PyPegen_expect_token(p, 624)) // token='try' + (_keyword = _PyPegen_expect_token(p, 638)) // token='try' && (_literal = _PyPegen_expect_forced_token(p, 11, ":")) // forced_token=':' && @@ -7095,7 +7097,7 @@ try_stmt_rule(Parser *p) asdl_excepthandler_seq* ex; void *f; if ( - (_keyword = _PyPegen_expect_token(p, 624)) // token='try' + (_keyword = _PyPegen_expect_token(p, 638)) // token='try' && (_literal = _PyPegen_expect_forced_token(p, 11, ":")) // forced_token=':' && @@ -7194,7 +7196,7 @@ except_block_rule(Parser *p) expr_ty e; void *t; if ( - (_keyword = _PyPegen_expect_token(p, 637)) // token='except' + (_keyword = _PyPegen_expect_token(p, 651)) // token='except' && (e = expression_rule(p)) // expression && @@ -7237,7 +7239,7 @@ except_block_rule(Parser *p) Token * _literal; asdl_stmt_seq* b; if ( - (_keyword = _PyPegen_expect_token(p, 637)) // token='except' + (_keyword = _PyPegen_expect_token(p, 651)) // token='except' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -7349,7 +7351,7 @@ except_star_block_rule(Parser *p) expr_ty e; void *t; if ( - (_keyword = _PyPegen_expect_token(p, 637)) // token='except' + (_keyword = _PyPegen_expect_token(p, 651)) // token='except' && (_literal = _PyPegen_expect_token(p, 16)) // token='*' && @@ -7452,7 +7454,7 @@ finally_block_rule(Parser *p) Token * _literal; asdl_stmt_seq* a; if ( - (_keyword = _PyPegen_expect_token(p, 633)) // token='finally' + (_keyword = _PyPegen_expect_token(p, 647)) // token='finally' && (_literal = _PyPegen_expect_forced_token(p, 11, ":")) // forced_token=':' && @@ -7764,7 +7766,7 @@ guard_rule(Parser *p) Token * _keyword; expr_ty guard; if ( - (_keyword = _PyPegen_expect_token(p, 642)) // token='if' + (_keyword = _PyPegen_expect_token(p, 656)) // token='if' && (guard = named_expression_rule(p)) // named_expression ) @@ -7962,7 +7964,7 @@ as_pattern_rule(Parser *p) if ( (pattern = or_pattern_rule(p)) // or_pattern && - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (target = pattern_capture_target_rule(p)) // pattern_capture_target ) @@ -8399,7 +8401,7 @@ literal_pattern_rule(Parser *p) D(fprintf(stderr, "%*c> literal_pattern[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 602)) // token='None' + (_keyword = _PyPegen_expect_token(p, 611)) // token='None' ) { D(fprintf(stderr, "%*c+ literal_pattern[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); @@ -8432,7 +8434,7 @@ literal_pattern_rule(Parser *p) D(fprintf(stderr, "%*c> literal_pattern[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 601)) // token='True' + (_keyword = _PyPegen_expect_token(p, 610)) // token='True' ) { D(fprintf(stderr, "%*c+ literal_pattern[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); @@ -8465,7 +8467,7 @@ literal_pattern_rule(Parser *p) D(fprintf(stderr, "%*c> literal_pattern[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 603)) // token='False' + (_keyword = _PyPegen_expect_token(p, 612)) // token='False' ) { D(fprintf(stderr, "%*c+ literal_pattern[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); @@ -8592,7 +8594,7 @@ literal_expr_rule(Parser *p) D(fprintf(stderr, "%*c> literal_expr[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 602)) // token='None' + (_keyword = _PyPegen_expect_token(p, 611)) // token='None' ) { D(fprintf(stderr, "%*c+ literal_expr[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); @@ -8625,7 +8627,7 @@ literal_expr_rule(Parser *p) D(fprintf(stderr, "%*c> literal_expr[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 601)) // token='True' + (_keyword = _PyPegen_expect_token(p, 610)) // token='True' ) { D(fprintf(stderr, "%*c+ literal_expr[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); @@ -8658,7 +8660,7 @@ literal_expr_rule(Parser *p) D(fprintf(stderr, "%*c> literal_expr[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 603)) // token='False' + (_keyword = _PyPegen_expect_token(p, 612)) // token='False' ) { D(fprintf(stderr, "%*c+ literal_expr[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); @@ -11222,11 +11224,11 @@ expression_rule(Parser *p) if ( (a = disjunction_rule(p)) // disjunction && - (_keyword = _PyPegen_expect_token(p, 642)) // token='if' + (_keyword = _PyPegen_expect_token(p, 656)) // token='if' && (b = disjunction_rule(p)) // disjunction && - (_keyword_1 = _PyPegen_expect_token(p, 645)) // token='else' + (_keyword_1 = _PyPegen_expect_token(p, 659)) // token='else' && (c = expression_rule(p)) // expression ) @@ -11331,9 +11333,9 @@ yield_expr_rule(Parser *p) Token * _keyword_1; expr_ty a; if ( - (_keyword = _PyPegen_expect_token(p, 573)) // token='yield' + (_keyword = _PyPegen_expect_token(p, 580)) // token='yield' && - (_keyword_1 = _PyPegen_expect_token(p, 608)) // token='from' + (_keyword_1 = _PyPegen_expect_token(p, 618)) // token='from' && (a = expression_rule(p)) // expression ) @@ -11369,7 +11371,7 @@ yield_expr_rule(Parser *p) Token * _keyword; void *a; if ( - (_keyword = _PyPegen_expect_token(p, 573)) // token='yield' + (_keyword = _PyPegen_expect_token(p, 580)) // token='yield' && (a = star_expressions_rule(p), !p->error_indicator) // star_expressions? ) @@ -12118,7 +12120,7 @@ inversion_rule(Parser *p) Token * _keyword; expr_ty a; if ( - (_keyword = _PyPegen_expect_token(p, 581)) // token='not' + (_keyword = _PyPegen_expect_token(p, 588)) // token='not' && (a = inversion_rule(p)) // inversion ) @@ -12781,9 +12783,9 @@ notin_bitwise_or_rule(Parser *p) Token * _keyword_1; expr_ty a; if ( - (_keyword = _PyPegen_expect_token(p, 581)) // token='not' + (_keyword = _PyPegen_expect_token(p, 588)) // token='not' && - (_keyword_1 = _PyPegen_expect_token(p, 651)) // token='in' + (_keyword_1 = _PyPegen_expect_token(p, 667)) // token='in' && (a = bitwise_or_rule(p)) // bitwise_or ) @@ -12830,7 +12832,7 @@ in_bitwise_or_rule(Parser *p) Token * _keyword; expr_ty a; if ( - (_keyword = _PyPegen_expect_token(p, 651)) // token='in' + (_keyword = _PyPegen_expect_token(p, 667)) // token='in' && (a = bitwise_or_rule(p)) // bitwise_or ) @@ -12878,9 +12880,9 @@ isnot_bitwise_or_rule(Parser *p) Token * _keyword_1; expr_ty a; if ( - (_keyword = _PyPegen_expect_token(p, 582)) // token='is' + (_keyword = _PyPegen_expect_token(p, 589)) // token='is' && - (_keyword_1 = _PyPegen_expect_token(p, 581)) // token='not' + (_keyword_1 = _PyPegen_expect_token(p, 588)) // token='not' && (a = bitwise_or_rule(p)) // bitwise_or ) @@ -12927,7 +12929,7 @@ is_bitwise_or_rule(Parser *p) Token * _keyword; expr_ty a; if ( - (_keyword = _PyPegen_expect_token(p, 582)) // token='is' + (_keyword = _PyPegen_expect_token(p, 589)) // token='is' && (a = bitwise_or_rule(p)) // bitwise_or ) @@ -14183,7 +14185,7 @@ power_rule(Parser *p) return _res; } -// await_primary: AWAIT primary | primary +// await_primary: 'await' primary | primary static expr_ty await_primary_rule(Parser *p) { @@ -14210,21 +14212,21 @@ await_primary_rule(Parser *p) UNUSED(_start_lineno); // Only used by EXTRA macro int _start_col_offset = p->tokens[_mark]->col_offset; UNUSED(_start_col_offset); // Only used by EXTRA macro - { // AWAIT primary + { // 'await' primary if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> await_primary[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "AWAIT primary")); + D(fprintf(stderr, "%*c> await_primary[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'await' primary")); + Token * _keyword; expr_ty a; - Token * await_var; if ( - (await_var = _PyPegen_expect_token(p, AWAIT)) // token='AWAIT' + (_keyword = _PyPegen_expect_token(p, 590)) // token='await' && (a = primary_rule(p)) // primary ) { - D(fprintf(stderr, "%*c+ await_primary[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "AWAIT primary")); + D(fprintf(stderr, "%*c+ await_primary[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'await' primary")); Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); if (_token == NULL) { p->level--; @@ -14244,7 +14246,7 @@ await_primary_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s await_primary[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "AWAIT primary")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'await' primary")); } { // primary if (p->error_indicator) { @@ -14768,7 +14770,7 @@ atom_rule(Parser *p) D(fprintf(stderr, "%*c> atom[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 601)) // token='True' + (_keyword = _PyPegen_expect_token(p, 610)) // token='True' ) { D(fprintf(stderr, "%*c+ atom[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); @@ -14801,7 +14803,7 @@ atom_rule(Parser *p) D(fprintf(stderr, "%*c> atom[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 603)) // token='False' + (_keyword = _PyPegen_expect_token(p, 612)) // token='False' ) { D(fprintf(stderr, "%*c+ atom[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); @@ -14834,7 +14836,7 @@ atom_rule(Parser *p) D(fprintf(stderr, "%*c> atom[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 602)) // token='None' + (_keyword = _PyPegen_expect_token(p, 611)) // token='None' ) { D(fprintf(stderr, "%*c+ atom[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); @@ -15104,7 +15106,7 @@ lambdef_rule(Parser *p) void *a; expr_ty b; if ( - (_keyword = _PyPegen_expect_token(p, 600)) // token='lambda' + (_keyword = _PyPegen_expect_token(p, 609)) // token='lambda' && (a = lambda_params_rule(p), !p->error_indicator) // lambda_params? && @@ -16981,7 +16983,7 @@ for_if_clauses_rule(Parser *p) } // for_if_clause: -// | ASYNC 'for' star_targets 'in' ~ disjunction (('if' disjunction))* +// | 'async' 'for' star_targets 'in' ~ disjunction (('if' disjunction))* // | 'for' star_targets 'in' ~ disjunction (('if' disjunction))* // | invalid_for_target static comprehension_ty @@ -16997,27 +16999,27 @@ for_if_clause_rule(Parser *p) } comprehension_ty _res = NULL; int _mark = p->mark; - { // ASYNC 'for' star_targets 'in' ~ disjunction (('if' disjunction))* + { // 'async' 'for' star_targets 'in' ~ disjunction (('if' disjunction))* if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> for_if_clause[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC 'for' star_targets 'in' ~ disjunction (('if' disjunction))*")); + D(fprintf(stderr, "%*c> for_if_clause[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async' 'for' star_targets 'in' ~ disjunction (('if' disjunction))*")); int _cut_var = 0; Token * _keyword; Token * _keyword_1; + Token * _keyword_2; expr_ty a; - Token * async_var; expr_ty b; asdl_expr_seq* c; if ( - (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' + (_keyword = _PyPegen_expect_token(p, 668)) // token='async' && - (_keyword = _PyPegen_expect_token(p, 650)) // token='for' + (_keyword_1 = _PyPegen_expect_token(p, 666)) // token='for' && (a = star_targets_rule(p)) // star_targets && - (_keyword_1 = _PyPegen_expect_token(p, 651)) // token='in' + (_keyword_2 = _PyPegen_expect_token(p, 667)) // token='in' && (_cut_var = 1) && @@ -17026,7 +17028,7 @@ for_if_clause_rule(Parser *p) (c = (asdl_expr_seq*)_loop0_120_rule(p)) // (('if' disjunction))* ) { - D(fprintf(stderr, "%*c+ for_if_clause[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC 'for' star_targets 'in' ~ disjunction (('if' disjunction))*")); + D(fprintf(stderr, "%*c+ for_if_clause[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async' 'for' star_targets 'in' ~ disjunction (('if' disjunction))*")); _res = CHECK_VERSION ( comprehension_ty , 6 , "Async comprehensions are" , _PyAST_comprehension ( a , b , c , 1 , p -> arena ) ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -17037,7 +17039,7 @@ for_if_clause_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s for_if_clause[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC 'for' star_targets 'in' ~ disjunction (('if' disjunction))*")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async' 'for' star_targets 'in' ~ disjunction (('if' disjunction))*")); if (_cut_var) { p->level--; return NULL; @@ -17056,11 +17058,11 @@ for_if_clause_rule(Parser *p) expr_ty b; asdl_expr_seq* c; if ( - (_keyword = _PyPegen_expect_token(p, 650)) // token='for' + (_keyword = _PyPegen_expect_token(p, 666)) // token='for' && (a = star_targets_rule(p)) // star_targets && - (_keyword_1 = _PyPegen_expect_token(p, 651)) // token='in' + (_keyword_1 = _PyPegen_expect_token(p, 667)) // token='in' && (_cut_var = 1) && @@ -20349,11 +20351,11 @@ expression_without_invalid_rule(Parser *p) if ( (a = disjunction_rule(p)) // disjunction && - (_keyword = _PyPegen_expect_token(p, 642)) // token='if' + (_keyword = _PyPegen_expect_token(p, 656)) // token='if' && (b = disjunction_rule(p)) // disjunction && - (_keyword_1 = _PyPegen_expect_token(p, 645)) // token='else' + (_keyword_1 = _PyPegen_expect_token(p, 659)) // token='else' && (c = expression_rule(p)) // expression ) @@ -20536,7 +20538,7 @@ invalid_expression_rule(Parser *p) if ( (a = disjunction_rule(p)) // disjunction && - (_keyword = _PyPegen_expect_token(p, 642)) // token='if' + (_keyword = _PyPegen_expect_token(p, 656)) // token='if' && (b = disjunction_rule(p)) // disjunction && @@ -20567,7 +20569,7 @@ invalid_expression_rule(Parser *p) Token * a; Token * b; if ( - (a = _PyPegen_expect_token(p, 600)) // token='lambda' + (a = _PyPegen_expect_token(p, 609)) // token='lambda' && (_opt_var = lambda_params_rule(p), !p->error_indicator) // lambda_params? && @@ -21042,7 +21044,7 @@ invalid_del_stmt_rule(Parser *p) Token * _keyword; expr_ty a; if ( - (_keyword = _PyPegen_expect_token(p, 604)) // token='del' + (_keyword = _PyPegen_expect_token(p, 613)) // token='del' && (a = star_expressions_rule(p)) // star_expressions ) @@ -22492,7 +22494,7 @@ invalid_with_item_rule(Parser *p) if ( (expression_var = expression_rule(p)) // expression && - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (a = expression_rule(p)) // expression && @@ -22518,7 +22520,7 @@ invalid_with_item_rule(Parser *p) return _res; } -// invalid_for_target: ASYNC? 'for' star_expressions +// invalid_for_target: 'async'? 'for' star_expressions static void * invalid_for_target_rule(Parser *p) { @@ -22532,25 +22534,25 @@ invalid_for_target_rule(Parser *p) } void * _res = NULL; int _mark = p->mark; - { // ASYNC? 'for' star_expressions + { // 'async'? 'for' star_expressions if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_for_target[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'for' star_expressions")); + D(fprintf(stderr, "%*c> invalid_for_target[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'? 'for' star_expressions")); Token * _keyword; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty a; if ( - (_opt_var = _PyPegen_expect_token(p, ASYNC), !p->error_indicator) // ASYNC? + (_opt_var = _PyPegen_expect_token(p, 668), !p->error_indicator) // 'async'? && - (_keyword = _PyPegen_expect_token(p, 650)) // token='for' + (_keyword = _PyPegen_expect_token(p, 666)) // token='for' && (a = star_expressions_rule(p)) // star_expressions ) { - D(fprintf(stderr, "%*c+ invalid_for_target[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'for' star_expressions")); + D(fprintf(stderr, "%*c+ invalid_for_target[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'? 'for' star_expressions")); _res = RAISE_SYNTAX_ERROR_INVALID_TARGET ( FOR_TARGETS , a ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -22561,7 +22563,7 @@ invalid_for_target_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_for_target[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC? 'for' star_expressions")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'? 'for' star_expressions")); } _res = NULL; done: @@ -22677,11 +22679,11 @@ invalid_import_rule(Parser *p) Token * a; expr_ty dotted_name_var; if ( - (a = _PyPegen_expect_token(p, 607)) // token='import' + (a = _PyPegen_expect_token(p, 617)) // token='import' && (_gather_203_var = _gather_203_rule(p)) // ','.dotted_name+ && - (_keyword = _PyPegen_expect_token(p, 608)) // token='from' + (_keyword = _PyPegen_expect_token(p, 618)) // token='from' && (dotted_name_var = dotted_name_rule(p)) // dotted_name ) @@ -22756,8 +22758,8 @@ invalid_import_from_targets_rule(Parser *p) } // invalid_with_stmt: -// | ASYNC? 'with' ','.(expression ['as' star_target])+ NEWLINE -// | ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE +// | 'async'? 'with' ','.(expression ['as' star_target])+ NEWLINE +// | 'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE static void * invalid_with_stmt_rule(Parser *p) { @@ -22771,28 +22773,28 @@ invalid_with_stmt_rule(Parser *p) } void * _res = NULL; int _mark = p->mark; - { // ASYNC? 'with' ','.(expression ['as' star_target])+ NEWLINE + { // 'async'? 'with' ','.(expression ['as' star_target])+ NEWLINE if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ NEWLINE")); + D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'? 'with' ','.(expression ['as' star_target])+ NEWLINE")); asdl_seq * _gather_205_var; Token * _keyword; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings Token * newline_var; if ( - (_opt_var = _PyPegen_expect_token(p, ASYNC), !p->error_indicator) // ASYNC? + (_opt_var = _PyPegen_expect_token(p, 668), !p->error_indicator) // 'async'? && - (_keyword = _PyPegen_expect_token(p, 615)) // token='with' + (_keyword = _PyPegen_expect_token(p, 629)) // token='with' && (_gather_205_var = _gather_205_rule(p)) // ','.(expression ['as' star_target])+ && (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ NEWLINE")); + D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'? 'with' ','.(expression ['as' star_target])+ NEWLINE")); _res = RAISE_SYNTAX_ERROR ( "expected ':'" ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -22803,14 +22805,14 @@ invalid_with_stmt_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_with_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ NEWLINE")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'? 'with' ','.(expression ['as' star_target])+ NEWLINE")); } - { // ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE + { // 'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE")); + D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE")); asdl_seq * _gather_207_var; Token * _keyword; Token * _literal; @@ -22821,9 +22823,9 @@ invalid_with_stmt_rule(Parser *p) UNUSED(_opt_var_1); // Silence compiler warnings Token * newline_var; if ( - (_opt_var = _PyPegen_expect_token(p, ASYNC), !p->error_indicator) // ASYNC? + (_opt_var = _PyPegen_expect_token(p, 668), !p->error_indicator) // 'async'? && - (_keyword = _PyPegen_expect_token(p, 615)) // token='with' + (_keyword = _PyPegen_expect_token(p, 629)) // token='with' && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && @@ -22836,7 +22838,7 @@ invalid_with_stmt_rule(Parser *p) (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE")); + D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE")); _res = RAISE_SYNTAX_ERROR ( "expected ':'" ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -22847,7 +22849,7 @@ invalid_with_stmt_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_with_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' NEWLINE")); } _res = NULL; done: @@ -22856,8 +22858,8 @@ invalid_with_stmt_rule(Parser *p) } // invalid_with_stmt_indent: -// | ASYNC? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT -// | ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT +// | 'async'? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT +// | 'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT static void * invalid_with_stmt_indent_rule(Parser *p) { @@ -22871,12 +22873,12 @@ invalid_with_stmt_indent_rule(Parser *p) } void * _res = NULL; int _mark = p->mark; - { // ASYNC? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT + { // 'async'? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); + D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); asdl_seq * _gather_209_var; Token * _literal; void *_opt_var; @@ -22884,9 +22886,9 @@ invalid_with_stmt_indent_rule(Parser *p) Token * a; Token * newline_var; if ( - (_opt_var = _PyPegen_expect_token(p, ASYNC), !p->error_indicator) // ASYNC? + (_opt_var = _PyPegen_expect_token(p, 668), !p->error_indicator) // 'async'? && - (a = _PyPegen_expect_token(p, 615)) // token='with' + (a = _PyPegen_expect_token(p, 629)) // token='with' && (_gather_209_var = _gather_209_rule(p)) // ','.(expression ['as' star_target])+ && @@ -22897,7 +22899,7 @@ invalid_with_stmt_indent_rule(Parser *p) _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, INDENT) // token=INDENT ) { - D(fprintf(stderr, "%*c+ invalid_with_stmt_indent[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); + D(fprintf(stderr, "%*c+ invalid_with_stmt_indent[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); _res = RAISE_INDENTATION_ERROR ( "expected an indented block after 'with' statement on line %d" , a -> lineno ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -22908,14 +22910,14 @@ invalid_with_stmt_indent_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_with_stmt_indent[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); } - { // ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT + { // 'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); + D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); asdl_seq * _gather_211_var; Token * _literal; Token * _literal_1; @@ -22927,9 +22929,9 @@ invalid_with_stmt_indent_rule(Parser *p) Token * a; Token * newline_var; if ( - (_opt_var = _PyPegen_expect_token(p, ASYNC), !p->error_indicator) // ASYNC? + (_opt_var = _PyPegen_expect_token(p, 668), !p->error_indicator) // 'async'? && - (a = _PyPegen_expect_token(p, 615)) // token='with' + (a = _PyPegen_expect_token(p, 629)) // token='with' && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && @@ -22946,7 +22948,7 @@ invalid_with_stmt_indent_rule(Parser *p) _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, INDENT) // token=INDENT ) { - D(fprintf(stderr, "%*c+ invalid_with_stmt_indent[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); + D(fprintf(stderr, "%*c+ invalid_with_stmt_indent[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); _res = RAISE_INDENTATION_ERROR ( "expected an indented block after 'with' statement on line %d" , a -> lineno ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -22957,7 +22959,7 @@ invalid_with_stmt_indent_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_with_stmt_indent[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); } _res = NULL; done: @@ -22993,7 +22995,7 @@ invalid_try_stmt_rule(Parser *p) Token * a; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 624)) // token='try' + (a = _PyPegen_expect_token(p, 638)) // token='try' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -23025,7 +23027,7 @@ invalid_try_stmt_rule(Parser *p) Token * _literal; asdl_stmt_seq* block_var; if ( - (_keyword = _PyPegen_expect_token(p, 624)) // token='try' + (_keyword = _PyPegen_expect_token(p, 638)) // token='try' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -23064,7 +23066,7 @@ invalid_try_stmt_rule(Parser *p) Token * b; expr_ty expression_var; if ( - (_keyword = _PyPegen_expect_token(p, 624)) // token='try' + (_keyword = _PyPegen_expect_token(p, 638)) // token='try' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -23072,7 +23074,7 @@ invalid_try_stmt_rule(Parser *p) && (_loop1_215_var = _loop1_215_rule(p)) // except_block+ && - (a = _PyPegen_expect_token(p, 637)) // token='except' + (a = _PyPegen_expect_token(p, 651)) // token='except' && (b = _PyPegen_expect_token(p, 16)) // token='*' && @@ -23111,7 +23113,7 @@ invalid_try_stmt_rule(Parser *p) UNUSED(_opt_var); // Silence compiler warnings Token * a; if ( - (_keyword = _PyPegen_expect_token(p, 624)) // token='try' + (_keyword = _PyPegen_expect_token(p, 638)) // token='try' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -23119,7 +23121,7 @@ invalid_try_stmt_rule(Parser *p) && (_loop1_218_var = _loop1_218_rule(p)) // except_star_block+ && - (a = _PyPegen_expect_token(p, 637)) // token='except' + (a = _PyPegen_expect_token(p, 651)) // token='except' && (_opt_var = _tmp_219_rule(p), !p->error_indicator) // [expression ['as' NAME]] && @@ -23179,7 +23181,7 @@ invalid_except_stmt_rule(Parser *p) expr_ty a; expr_ty expressions_var; if ( - (_keyword = _PyPegen_expect_token(p, 637)) // token='except' + (_keyword = _PyPegen_expect_token(p, 651)) // token='except' && (_opt_var = _PyPegen_expect_token(p, 16), !p->error_indicator) // '*'? && @@ -23221,7 +23223,7 @@ invalid_except_stmt_rule(Parser *p) expr_ty expression_var; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 637)) // token='except' + (a = _PyPegen_expect_token(p, 651)) // token='except' && (_opt_var = _PyPegen_expect_token(p, 16), !p->error_indicator) // '*'? && @@ -23254,7 +23256,7 @@ invalid_except_stmt_rule(Parser *p) Token * a; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 637)) // token='except' + (a = _PyPegen_expect_token(p, 651)) // token='except' && (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) @@ -23282,7 +23284,7 @@ invalid_except_stmt_rule(Parser *p) void *_tmp_222_var; Token * a; if ( - (a = _PyPegen_expect_token(p, 637)) // token='except' + (a = _PyPegen_expect_token(p, 651)) // token='except' && (_literal = _PyPegen_expect_token(p, 16)) // token='*' && @@ -23332,7 +23334,7 @@ invalid_finally_stmt_rule(Parser *p) Token * a; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 633)) // token='finally' + (a = _PyPegen_expect_token(p, 647)) // token='finally' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -23389,7 +23391,7 @@ invalid_except_stmt_indent_rule(Parser *p) expr_ty expression_var; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 637)) // token='except' + (a = _PyPegen_expect_token(p, 651)) // token='except' && (expression_var = expression_rule(p)) // expression && @@ -23425,7 +23427,7 @@ invalid_except_stmt_indent_rule(Parser *p) Token * a; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 637)) // token='except' + (a = _PyPegen_expect_token(p, 651)) // token='except' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -23482,7 +23484,7 @@ invalid_except_star_stmt_indent_rule(Parser *p) expr_ty expression_var; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 637)) // token='except' + (a = _PyPegen_expect_token(p, 651)) // token='except' && (_literal = _PyPegen_expect_token(p, 16)) // token='*' && @@ -23724,7 +23726,7 @@ invalid_as_pattern_rule(Parser *p) if ( (or_pattern_var = or_pattern_rule(p)) // or_pattern && - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (a = _PyPegen_expect_soft_keyword(p, "_")) // soft_keyword='"_"' ) @@ -23754,7 +23756,7 @@ invalid_as_pattern_rule(Parser *p) if ( (or_pattern_var = or_pattern_rule(p)) // or_pattern && - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && _PyPegen_lookahead_with_name(0, _PyPegen_name_token, p) && @@ -23911,7 +23913,7 @@ invalid_if_stmt_rule(Parser *p) expr_ty named_expression_var; Token * newline_var; if ( - (_keyword = _PyPegen_expect_token(p, 642)) // token='if' + (_keyword = _PyPegen_expect_token(p, 656)) // token='if' && (named_expression_var = named_expression_rule(p)) // named_expression && @@ -23942,7 +23944,7 @@ invalid_if_stmt_rule(Parser *p) expr_ty a_1; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 642)) // token='if' + (a = _PyPegen_expect_token(p, 656)) // token='if' && (a_1 = named_expression_rule(p)) // named_expression && @@ -23998,7 +24000,7 @@ invalid_elif_stmt_rule(Parser *p) expr_ty named_expression_var; Token * newline_var; if ( - (_keyword = _PyPegen_expect_token(p, 644)) // token='elif' + (_keyword = _PyPegen_expect_token(p, 658)) // token='elif' && (named_expression_var = named_expression_rule(p)) // named_expression && @@ -24029,7 +24031,7 @@ invalid_elif_stmt_rule(Parser *p) expr_ty named_expression_var; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 644)) // token='elif' + (a = _PyPegen_expect_token(p, 658)) // token='elif' && (named_expression_var = named_expression_rule(p)) // named_expression && @@ -24083,7 +24085,7 @@ invalid_else_stmt_rule(Parser *p) Token * a; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 645)) // token='else' + (a = _PyPegen_expect_token(p, 659)) // token='else' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -24137,7 +24139,7 @@ invalid_while_stmt_rule(Parser *p) expr_ty named_expression_var; Token * newline_var; if ( - (_keyword = _PyPegen_expect_token(p, 647)) // token='while' + (_keyword = _PyPegen_expect_token(p, 661)) // token='while' && (named_expression_var = named_expression_rule(p)) // named_expression && @@ -24168,7 +24170,7 @@ invalid_while_stmt_rule(Parser *p) expr_ty named_expression_var; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 647)) // token='while' + (a = _PyPegen_expect_token(p, 661)) // token='while' && (named_expression_var = named_expression_rule(p)) // named_expression && @@ -24199,8 +24201,8 @@ invalid_while_stmt_rule(Parser *p) } // invalid_for_stmt: -// | ASYNC? 'for' star_targets 'in' star_expressions NEWLINE -// | ASYNC? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT +// | 'async'? 'for' star_targets 'in' star_expressions NEWLINE +// | 'async'? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT static void * invalid_for_stmt_rule(Parser *p) { @@ -24214,12 +24216,12 @@ invalid_for_stmt_rule(Parser *p) } void * _res = NULL; int _mark = p->mark; - { // ASYNC? 'for' star_targets 'in' star_expressions NEWLINE + { // 'async'? 'for' star_targets 'in' star_expressions NEWLINE if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_for_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'for' star_targets 'in' star_expressions NEWLINE")); + D(fprintf(stderr, "%*c> invalid_for_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'? 'for' star_targets 'in' star_expressions NEWLINE")); Token * _keyword; Token * _keyword_1; void *_opt_var; @@ -24228,20 +24230,20 @@ invalid_for_stmt_rule(Parser *p) expr_ty star_expressions_var; expr_ty star_targets_var; if ( - (_opt_var = _PyPegen_expect_token(p, ASYNC), !p->error_indicator) // ASYNC? + (_opt_var = _PyPegen_expect_token(p, 668), !p->error_indicator) // 'async'? && - (_keyword = _PyPegen_expect_token(p, 650)) // token='for' + (_keyword = _PyPegen_expect_token(p, 666)) // token='for' && (star_targets_var = star_targets_rule(p)) // star_targets && - (_keyword_1 = _PyPegen_expect_token(p, 651)) // token='in' + (_keyword_1 = _PyPegen_expect_token(p, 667)) // token='in' && (star_expressions_var = star_expressions_rule(p)) // star_expressions && (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ invalid_for_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'for' star_targets 'in' star_expressions NEWLINE")); + D(fprintf(stderr, "%*c+ invalid_for_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'? 'for' star_targets 'in' star_expressions NEWLINE")); _res = RAISE_SYNTAX_ERROR ( "expected ':'" ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24252,14 +24254,14 @@ invalid_for_stmt_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_for_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC? 'for' star_targets 'in' star_expressions NEWLINE")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'? 'for' star_targets 'in' star_expressions NEWLINE")); } - { // ASYNC? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT + { // 'async'? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_for_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT")); + D(fprintf(stderr, "%*c> invalid_for_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT")); Token * _keyword; Token * _literal; void *_opt_var; @@ -24269,13 +24271,13 @@ invalid_for_stmt_rule(Parser *p) expr_ty star_expressions_var; expr_ty star_targets_var; if ( - (_opt_var = _PyPegen_expect_token(p, ASYNC), !p->error_indicator) // ASYNC? + (_opt_var = _PyPegen_expect_token(p, 668), !p->error_indicator) // 'async'? && - (a = _PyPegen_expect_token(p, 650)) // token='for' + (a = _PyPegen_expect_token(p, 666)) // token='for' && (star_targets_var = star_targets_rule(p)) // star_targets && - (_keyword = _PyPegen_expect_token(p, 651)) // token='in' + (_keyword = _PyPegen_expect_token(p, 667)) // token='in' && (star_expressions_var = star_expressions_rule(p)) // star_expressions && @@ -24286,7 +24288,7 @@ invalid_for_stmt_rule(Parser *p) _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, INDENT) // token=INDENT ) { - D(fprintf(stderr, "%*c+ invalid_for_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT")); + D(fprintf(stderr, "%*c+ invalid_for_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT")); _res = RAISE_INDENTATION_ERROR ( "expected an indented block after 'for' statement on line %d" , a -> lineno ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24297,7 +24299,7 @@ invalid_for_stmt_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_for_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'? 'for' star_targets 'in' star_expressions ':' NEWLINE !INDENT")); } _res = NULL; done: @@ -24306,7 +24308,7 @@ invalid_for_stmt_rule(Parser *p) } // invalid_def_raw: -// | ASYNC? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT +// | 'async'? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT static void * invalid_def_raw_rule(Parser *p) { @@ -24320,12 +24322,12 @@ invalid_def_raw_rule(Parser *p) } void * _res = NULL; int _mark = p->mark; - { // ASYNC? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT + { // 'async'? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_def_raw[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT")); + D(fprintf(stderr, "%*c> invalid_def_raw[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT")); Token * _literal; Token * _literal_1; Token * _literal_2; @@ -24339,9 +24341,9 @@ invalid_def_raw_rule(Parser *p) expr_ty name_var; Token * newline_var; if ( - (_opt_var = _PyPegen_expect_token(p, ASYNC), !p->error_indicator) // ASYNC? + (_opt_var = _PyPegen_expect_token(p, 668), !p->error_indicator) // 'async'? && - (a = _PyPegen_expect_token(p, 652)) // token='def' + (a = _PyPegen_expect_token(p, 669)) // token='def' && (name_var = _PyPegen_name_token(p)) // NAME && @@ -24360,7 +24362,7 @@ invalid_def_raw_rule(Parser *p) _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, INDENT) // token=INDENT ) { - D(fprintf(stderr, "%*c+ invalid_def_raw[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT")); + D(fprintf(stderr, "%*c+ invalid_def_raw[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT")); _res = RAISE_INDENTATION_ERROR ( "expected an indented block after function definition on line %d" , a -> lineno ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24371,7 +24373,7 @@ invalid_def_raw_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_def_raw[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'? 'def' NAME '(' params? ')' ['->' expression] ':' NEWLINE !INDENT")); } _res = NULL; done: @@ -24407,7 +24409,7 @@ invalid_class_def_raw_rule(Parser *p) expr_ty name_var; Token * newline_var; if ( - (_keyword = _PyPegen_expect_token(p, 654)) // token='class' + (_keyword = _PyPegen_expect_token(p, 671)) // token='class' && (name_var = _PyPegen_name_token(p)) // NAME && @@ -24442,7 +24444,7 @@ invalid_class_def_raw_rule(Parser *p) expr_ty name_var; Token * newline_var; if ( - (a = _PyPegen_expect_token(p, 654)) // token='class' + (a = _PyPegen_expect_token(p, 671)) // token='class' && (name_var = _PyPegen_name_token(p)) // NAME && @@ -25610,7 +25612,7 @@ _tmp_7_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_7[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'import'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 607)) // token='import' + (_keyword = _PyPegen_expect_token(p, 617)) // token='import' ) { D(fprintf(stderr, "%*c+ _tmp_7[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'import'")); @@ -25629,7 +25631,7 @@ _tmp_7_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_7[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'from'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 608)) // token='from' + (_keyword = _PyPegen_expect_token(p, 618)) // token='from' ) { D(fprintf(stderr, "%*c+ _tmp_7[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'from'")); @@ -25646,7 +25648,7 @@ _tmp_7_rule(Parser *p) return _res; } -// _tmp_8: 'def' | '@' | ASYNC +// _tmp_8: 'def' | '@' | 'async' static void * _tmp_8_rule(Parser *p) { @@ -25668,7 +25670,7 @@ _tmp_8_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_8[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'def'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 652)) // token='def' + (_keyword = _PyPegen_expect_token(p, 669)) // token='def' ) { D(fprintf(stderr, "%*c+ _tmp_8[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'def'")); @@ -25698,24 +25700,24 @@ _tmp_8_rule(Parser *p) D(fprintf(stderr, "%*c%s _tmp_8[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'@'")); } - { // ASYNC + { // 'async' if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_8[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC")); - Token * async_var; + D(fprintf(stderr, "%*c> _tmp_8[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'")); + Token * _keyword; if ( - (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' + (_keyword = _PyPegen_expect_token(p, 668)) // token='async' ) { - D(fprintf(stderr, "%*c+ _tmp_8[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC")); - _res = async_var; + D(fprintf(stderr, "%*c+ _tmp_8[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'")); + _res = _keyword; goto done; } p->mark = _mark; D(fprintf(stderr, "%*c%s _tmp_8[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'")); } _res = NULL; done: @@ -25745,7 +25747,7 @@ _tmp_9_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_9[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'class'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 654)) // token='class' + (_keyword = _PyPegen_expect_token(p, 671)) // token='class' ) { D(fprintf(stderr, "%*c+ _tmp_9[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'class'")); @@ -25781,7 +25783,7 @@ _tmp_9_rule(Parser *p) return _res; } -// _tmp_10: 'with' | ASYNC +// _tmp_10: 'with' | 'async' static void * _tmp_10_rule(Parser *p) { @@ -25803,7 +25805,7 @@ _tmp_10_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_10[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'with'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 615)) // token='with' + (_keyword = _PyPegen_expect_token(p, 629)) // token='with' ) { D(fprintf(stderr, "%*c+ _tmp_10[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'with'")); @@ -25814,24 +25816,24 @@ _tmp_10_rule(Parser *p) D(fprintf(stderr, "%*c%s _tmp_10[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'with'")); } - { // ASYNC + { // 'async' if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_10[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC")); - Token * async_var; + D(fprintf(stderr, "%*c> _tmp_10[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'")); + Token * _keyword; if ( - (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' + (_keyword = _PyPegen_expect_token(p, 668)) // token='async' ) { - D(fprintf(stderr, "%*c+ _tmp_10[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC")); - _res = async_var; + D(fprintf(stderr, "%*c+ _tmp_10[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'")); + _res = _keyword; goto done; } p->mark = _mark; D(fprintf(stderr, "%*c%s _tmp_10[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'")); } _res = NULL; done: @@ -25839,7 +25841,7 @@ _tmp_10_rule(Parser *p) return _res; } -// _tmp_11: 'for' | ASYNC +// _tmp_11: 'for' | 'async' static void * _tmp_11_rule(Parser *p) { @@ -25861,7 +25863,7 @@ _tmp_11_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_11[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'for'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 650)) // token='for' + (_keyword = _PyPegen_expect_token(p, 666)) // token='for' ) { D(fprintf(stderr, "%*c+ _tmp_11[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'for'")); @@ -25872,24 +25874,24 @@ _tmp_11_rule(Parser *p) D(fprintf(stderr, "%*c%s _tmp_11[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'for'")); } - { // ASYNC + { // 'async' if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_11[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC")); - Token * async_var; + D(fprintf(stderr, "%*c> _tmp_11[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'async'")); + Token * _keyword; if ( - (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' + (_keyword = _PyPegen_expect_token(p, 668)) // token='async' ) { - D(fprintf(stderr, "%*c+ _tmp_11[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC")); - _res = async_var; + D(fprintf(stderr, "%*c+ _tmp_11[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'async'")); + _res = _keyword; goto done; } p->mark = _mark; D(fprintf(stderr, "%*c%s _tmp_11[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "ASYNC")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'async'")); } _res = NULL; done: @@ -26272,7 +26274,7 @@ _tmp_18_rule(Parser *p) Token * _keyword; expr_ty z; if ( - (_keyword = _PyPegen_expect_token(p, 608)) // token='from' + (_keyword = _PyPegen_expect_token(p, 618)) // token='from' && (z = expression_rule(p)) // expression ) @@ -26922,7 +26924,7 @@ _tmp_29_rule(Parser *p) Token * _keyword; expr_ty z; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (z = _PyPegen_name_token(p)) // NAME ) @@ -27088,7 +27090,7 @@ _tmp_32_rule(Parser *p) Token * _keyword; expr_ty z; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (z = _PyPegen_name_token(p)) // NAME ) @@ -29106,7 +29108,7 @@ _tmp_63_rule(Parser *p) Token * _keyword; expr_ty z; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (z = _PyPegen_name_token(p)) // NAME ) @@ -29153,7 +29155,7 @@ _tmp_64_rule(Parser *p) Token * _keyword; expr_ty z; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (z = _PyPegen_name_token(p)) // NAME ) @@ -34721,7 +34723,7 @@ _tmp_153_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 601)) // token='True' + (_keyword = _PyPegen_expect_token(p, 610)) // token='True' ) { D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); @@ -34740,7 +34742,7 @@ _tmp_153_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 603)) // token='False' + (_keyword = _PyPegen_expect_token(p, 612)) // token='False' ) { D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); @@ -34759,7 +34761,7 @@ _tmp_153_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 602)) // token='None' + (_keyword = _PyPegen_expect_token(p, 611)) // token='None' ) { D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); @@ -34901,7 +34903,7 @@ _tmp_156_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'else'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 645)) // token='else' + (_keyword = _PyPegen_expect_token(p, 659)) // token='else' ) { D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'else'")); @@ -35132,7 +35134,7 @@ _tmp_159_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 601)) // token='True' + (_keyword = _PyPegen_expect_token(p, 610)) // token='True' ) { D(fprintf(stderr, "%*c+ _tmp_159[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); @@ -35151,7 +35153,7 @@ _tmp_159_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 602)) // token='None' + (_keyword = _PyPegen_expect_token(p, 611)) // token='None' ) { D(fprintf(stderr, "%*c+ _tmp_159[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); @@ -35170,7 +35172,7 @@ _tmp_159_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 603)) // token='False' + (_keyword = _PyPegen_expect_token(p, 612)) // token='False' ) { D(fprintf(stderr, "%*c+ _tmp_159[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); @@ -38608,7 +38610,7 @@ _tmp_213_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_213[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 637)) // token='except' + (_keyword = _PyPegen_expect_token(p, 651)) // token='except' ) { D(fprintf(stderr, "%*c+ _tmp_213[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except'")); @@ -38627,7 +38629,7 @@ _tmp_213_rule(Parser *p) D(fprintf(stderr, "%*c> _tmp_213[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally'")); Token * _keyword; if ( - (_keyword = _PyPegen_expect_token(p, 633)) // token='finally' + (_keyword = _PyPegen_expect_token(p, 647)) // token='finally' ) { D(fprintf(stderr, "%*c+ _tmp_213[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'finally'")); @@ -38808,7 +38810,7 @@ _tmp_216_rule(Parser *p) Token * _keyword; expr_ty name_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (name_var = _PyPegen_name_token(p)) // NAME ) @@ -39034,7 +39036,7 @@ _tmp_220_rule(Parser *p) Token * _keyword; expr_ty name_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (name_var = _PyPegen_name_token(p)) // NAME ) @@ -39076,7 +39078,7 @@ _tmp_221_rule(Parser *p) Token * _keyword; expr_ty name_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (name_var = _PyPegen_name_token(p)) // NAME ) @@ -39176,7 +39178,7 @@ _tmp_223_rule(Parser *p) Token * _keyword; expr_ty name_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (name_var = _PyPegen_name_token(p)) // NAME ) @@ -39218,7 +39220,7 @@ _tmp_224_rule(Parser *p) Token * _keyword; expr_ty name_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (name_var = _PyPegen_name_token(p)) // NAME ) @@ -40867,7 +40869,7 @@ _tmp_254_rule(Parser *p) Token * _keyword; expr_ty c; if ( - (_keyword = _PyPegen_expect_token(p, 574)) // token='or' + (_keyword = _PyPegen_expect_token(p, 581)) // token='or' && (c = conjunction_rule(p)) // conjunction ) @@ -40914,7 +40916,7 @@ _tmp_255_rule(Parser *p) Token * _keyword; expr_ty c; if ( - (_keyword = _PyPegen_expect_token(p, 575)) // token='and' + (_keyword = _PyPegen_expect_token(p, 582)) // token='and' && (c = inversion_rule(p)) // inversion ) @@ -41077,7 +41079,7 @@ _tmp_258_rule(Parser *p) Token * _keyword; expr_ty z; if ( - (_keyword = _PyPegen_expect_token(p, 642)) // token='if' + (_keyword = _PyPegen_expect_token(p, 656)) // token='if' && (z = disjunction_rule(p)) // disjunction ) @@ -41124,7 +41126,7 @@ _tmp_259_rule(Parser *p) Token * _keyword; expr_ty z; if ( - (_keyword = _PyPegen_expect_token(p, 642)) // token='if' + (_keyword = _PyPegen_expect_token(p, 656)) // token='if' && (z = disjunction_rule(p)) // disjunction ) @@ -41697,7 +41699,7 @@ _tmp_271_rule(Parser *p) Token * _keyword; expr_ty name_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (name_var = _PyPegen_name_token(p)) // NAME ) @@ -41799,7 +41801,7 @@ _tmp_273_rule(Parser *p) Token * _keyword; expr_ty star_target_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (star_target_var = star_target_rule(p)) // star_target ) @@ -41841,7 +41843,7 @@ _tmp_274_rule(Parser *p) Token * _keyword; expr_ty star_target_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (star_target_var = star_target_rule(p)) // star_target ) @@ -41883,7 +41885,7 @@ _tmp_275_rule(Parser *p) Token * _keyword; expr_ty star_target_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (star_target_var = star_target_rule(p)) // star_target ) @@ -41925,7 +41927,7 @@ _tmp_276_rule(Parser *p) Token * _keyword; expr_ty star_target_var; if ( - (_keyword = _PyPegen_expect_token(p, 640)) // token='as' + (_keyword = _PyPegen_expect_token(p, 654)) // token='as' && (star_target_var = star_target_rule(p)) // star_target ) diff --git a/Parser/pegen.c b/Parser/pegen.c index 885d423fca66a..bfade3446a57f 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -734,9 +734,6 @@ compute_parser_flags(PyCompilerFlags *flags) if (flags->cf_flags & PyCF_TYPE_COMMENTS) { parser_flags |= PyPARSE_TYPE_COMMENTS; } - if ((flags->cf_flags & PyCF_ONLY_AST) && flags->cf_feature_version < 7) { - parser_flags |= PyPARSE_ASYNC_HACKS; - } if (flags->cf_flags & PyCF_ALLOW_INCOMPLETE_INPUT) { parser_flags |= PyPARSE_ALLOW_INCOMPLETE_INPUT; } @@ -755,7 +752,6 @@ _PyPegen_Parser_New(struct tok_state *tok, int start_rule, int flags, } assert(tok != NULL); tok->type_comments = (flags & PyPARSE_TYPE_COMMENTS) > 0; - tok->async_hacks = (flags & PyPARSE_ASYNC_HACKS) > 0; p->tok = tok; p->keywords = NULL; p->n_keyword_lists = -1; diff --git a/Parser/pegen.h b/Parser/pegen.h index 5f29285951e81..0852bb51d4fe7 100644 --- a/Parser/pegen.h +++ b/Parser/pegen.h @@ -20,7 +20,6 @@ #define PyPARSE_IGNORE_COOKIE 0x0010 #define PyPARSE_BARRY_AS_BDFL 0x0020 #define PyPARSE_TYPE_COMMENTS 0x0040 -#define PyPARSE_ASYNC_HACKS 0x0080 #define PyPARSE_ALLOW_INCOMPLETE_INPUT 0x0100 #define CURRENT_POS (-5) diff --git a/Parser/token.c b/Parser/token.c index 2bc963a91c770..4f163f21609a0 100644 --- a/Parser/token.c +++ b/Parser/token.c @@ -62,8 +62,6 @@ const char * const _PyParser_TokenNames[] = { "COLONEQUAL", "EXCLAMATION", "OP", - "AWAIT", - "ASYNC", "TYPE_IGNORE", "TYPE_COMMENT", "SOFT_KEYWORD", diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index ccff16045233d..5a42f6f357317 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -104,10 +104,6 @@ tok_new(void) tok->decoding_buffer = NULL; tok->readline = NULL; tok->type_comments = 0; - tok->async_hacks = 0; - tok->async_def = 0; - tok->async_def_indent = 0; - tok->async_def_nl = 0; tok->interactive_underflow = IUNDERFLOW_NORMAL; tok->str = NULL; tok->report_warnings = 1; @@ -1925,27 +1921,6 @@ tok_get_normal_mode(struct tok_state *tok, tokenizer_mode* current_tok, struct t /* Peek ahead at the next character */ c = tok_nextc(tok); tok_backup(tok, c); - /* Check if we are closing an async function */ - if (tok->async_def - && !blankline - /* Due to some implementation artifacts of type comments, - * a TYPE_COMMENT at the start of a function won't set an - * indentation level and it will produce a NEWLINE after it. - * To avoid spuriously ending an async function due to this, - * wait until we have some non-newline char in front of us. */ - && c != '\n' - && tok->level == 0 - /* There was a NEWLINE after ASYNC DEF, - so we're past the signature. */ - && tok->async_def_nl - /* Current indentation level is less than where - the async function was defined */ - && tok->async_def_indent >= tok->indent) - { - tok->async_def = 0; - tok->async_def_indent = 0; - tok->async_def_nl = 0; - } again: tok->start = NULL; @@ -2094,54 +2069,6 @@ tok_get_normal_mode(struct tok_state *tok, tokenizer_mode* current_tok, struct t p_start = tok->start; p_end = tok->cur; - /* async/await parsing block. */ - if (tok->cur - tok->start == 5 && tok->start[0] == 'a') { - /* May be an 'async' or 'await' token. For Python 3.7 or - later we recognize them unconditionally. For Python - 3.5 or 3.6 we recognize 'async' in front of 'def', and - either one inside of 'async def'. (Technically we - shouldn't recognize these at all for 3.4 or earlier, - but there's no *valid* Python 3.4 code that would be - rejected, and async functions will be rejected in a - later phase.) */ - if (!tok->async_hacks || tok->async_def) { - /* Always recognize the keywords. */ - if (memcmp(tok->start, "async", 5) == 0) { - return MAKE_TOKEN(ASYNC); - } - if (memcmp(tok->start, "await", 5) == 0) { - return MAKE_TOKEN(AWAIT); - } - } - else if (memcmp(tok->start, "async", 5) == 0) { - /* The current token is 'async'. - Look ahead one token to see if that is 'def'. */ - - struct tok_state ahead_tok; - struct token ahead_token; - _PyToken_Init(&ahead_token); - int ahead_tok_kind; - - memcpy(&ahead_tok, tok, sizeof(ahead_tok)); - ahead_tok_kind = tok_get_normal_mode(&ahead_tok, - current_tok, - &ahead_token); - - if (ahead_tok_kind == NAME - && ahead_tok.cur - ahead_tok.start == 3 - && memcmp(ahead_tok.start, "def", 3) == 0) - { - /* The next token is going to be 'def', so instead of - returning a plain NAME token, return ASYNC. */ - tok->async_def_indent = tok->indent; - tok->async_def = 1; - _PyToken_Free(&ahead_token); - return MAKE_TOKEN(ASYNC); - } - _PyToken_Free(&ahead_token); - } - } - return MAKE_TOKEN(NAME); } @@ -2172,11 +2099,6 @@ tok_get_normal_mode(struct tok_state *tok, tokenizer_mode* current_tok, struct t p_start = tok->start; p_end = tok->cur - 1; /* Leave '\n' out of the string */ tok->cont_line = 0; - if (tok->async_def) { - /* We're somewhere inside an 'async def' function, and - we've encountered a NEWLINE after its signature. */ - tok->async_def_nl = 1; - } return MAKE_TOKEN(NEWLINE); } diff --git a/Parser/tokenizer.h b/Parser/tokenizer.h index 1e1daa3648f5d..11d69fc5b2e15 100644 --- a/Parser/tokenizer.h +++ b/Parser/tokenizer.h @@ -116,12 +116,6 @@ struct tok_state { int type_comments; /* Whether to look for type comments */ - /* async/await related fields (still needed depending on feature_version) */ - int async_hacks; /* =1 if async/await aren't always keywords */ - int async_def; /* =1 if tokens are inside an 'async def' body. */ - int async_def_indent; /* Indentation level of the outermost 'async def'. */ - int async_def_nl; /* =1 if the outermost 'async def' had at least one - NEWLINE token after it. */ /* How to proceed when asked for a new token in interactive mode */ enum interactive_underflow_t interactive_underflow; int report_warnings; diff --git a/Python/Python-tokenize.c b/Python/Python-tokenize.c index 1938562706914..1b021069c5e10 100644 --- a/Python/Python-tokenize.c +++ b/Python/Python-tokenize.c @@ -237,9 +237,6 @@ tokenizeriter_next(tokenizeriterobject *it) if (type > DEDENT && type < OP) { type = OP; } - else if (type == ASYNC || type == AWAIT) { - type = NAME; - } else if (type == NEWLINE) { Py_DECREF(str); if (!it->tok->implicit_newline) { diff --git a/Tools/peg_generator/pegen/keywordgen.py b/Tools/peg_generator/pegen/keywordgen.py index 35a5e1a229cde..bbf13267e5763 100644 --- a/Tools/peg_generator/pegen/keywordgen.py +++ b/Tools/peg_generator/pegen/keywordgen.py @@ -35,9 +35,6 @@ issoftkeyword = frozenset(softkwlist).__contains__ '''.lstrip() -EXTRA_KEYWORDS = ["async", "await"] - - def main() -> None: parser = argparse.ArgumentParser( description="Generate the Lib/keywords.py file from the grammar." @@ -62,7 +59,7 @@ def main() -> None: gen.collect_rules() with args.keyword_file as thefile: - all_keywords = sorted(list(gen.keywords.keys()) + EXTRA_KEYWORDS) + all_keywords = sorted(list(gen.keywords.keys())) all_soft_keywords = sorted(gen.soft_keywords) keywords = "" if not all_keywords else " " + ",\n ".join(map(repr, all_keywords)) diff --git a/Tools/peg_generator/pegen/python_generator.py b/Tools/peg_generator/pegen/python_generator.py index 5329d0ebe5e64..4a2883eb4ee20 100644 --- a/Tools/peg_generator/pegen/python_generator.py +++ b/Tools/peg_generator/pegen/python_generator.py @@ -102,7 +102,7 @@ def visit_NameLeaf(self, node: NameLeaf) -> Tuple[Optional[str], str]: if name in ("NAME", "NUMBER", "STRING", "OP", "TYPE_COMMENT"): name = name.lower() return name, f"self.{name}()" - if name in ("NEWLINE", "DEDENT", "INDENT", "ENDMARKER", "ASYNC", "AWAIT"): + if name in ("NEWLINE", "DEDENT", "INDENT", "ENDMARKER"): # Avoid using names that can be Python keywords return "_" + name.lower(), f"self.expect({name!r})" return name, f"self.{name}()" From webhook-mailer at python.org Wed Jul 26 12:04:50 2023 From: webhook-mailer at python.org (vstinner) Date: Wed, 26 Jul 2023 16:04:50 -0000 Subject: [Python-checkins] gh-106948: Update documentation nitpick_ignore for c:identifer domain (#107295) Message-ID: https://github.com/python/cpython/commit/b1de3807b832b72dfeb66dd5646159d08d2cc74a commit: b1de3807b832b72dfeb66dd5646159d08d2cc74a branch: main author: Victor Stinner committer: vstinner date: 2023-07-26T18:04:46+02:00 summary: gh-106948: Update documentation nitpick_ignore for c:identifer domain (#107295) Update the nitpick_ignore of the documentation configuration to fix Sphinx warnings about standard C types when declaring functions with the "c:function" markups. Copy standard C types declared in the "c:type" domain to the "c:identifier" domain, since "c:function" markup looks for types in the "c:identifier" domain. Co-authored-by: Serhiy Storchaka files: M Doc/conf.py M Doc/tools/.nitignore diff --git a/Doc/conf.py b/Doc/conf.py index 453f4fc63883a..8821a2a172f11 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -153,6 +153,15 @@ ('py:meth', '_SubParsersAction.add_parser'), ] +# gh-106948: Copy standard C types declared in the "c:type" domain to the +# "c:identifier" domain, since "c:function" markup looks for types in the +# "c:identifier" domain. Use list() to not iterate on items which are being +# added +for role, name in list(nitpick_ignore): + if role == 'c:type': + nitpick_ignore.append(('c:identifier', name)) +del role, name + # Disable Docutils smartquotes for several translations smartquotes_excludes = { 'languages': ['ja', 'fr', 'zh_TW', 'zh_CN'], 'builders': ['man', 'text'], diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index fc6de10eb1c2c..07123bcb5ae06 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -8,7 +8,6 @@ Doc/c-api/arg.rst Doc/c-api/bool.rst Doc/c-api/buffer.rst Doc/c-api/bytes.rst -Doc/c-api/call.rst Doc/c-api/capsule.rst Doc/c-api/cell.rst Doc/c-api/code.rst @@ -26,8 +25,6 @@ Doc/c-api/init.rst Doc/c-api/init_config.rst Doc/c-api/intro.rst Doc/c-api/iterator.rst -Doc/c-api/long.rst -Doc/c-api/marshal.rst Doc/c-api/memory.rst Doc/c-api/memoryview.rst Doc/c-api/module.rst From webhook-mailer at python.org Wed Jul 26 12:16:08 2023 From: webhook-mailer at python.org (vstinner) Date: Wed, 26 Jul 2023 16:16:08 -0000 Subject: [Python-checkins] [3.12] gh-106948: Update documentation nitpick_ignore for c:identifer domain (GH-107295) (#107297) Message-ID: https://github.com/python/cpython/commit/0d2e1317bd4d13c1652a08c3ad8199c03387f73b commit: 0d2e1317bd4d13c1652a08c3ad8199c03387f73b branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: vstinner date: 2023-07-26T18:16:04+02:00 summary: [3.12] gh-106948: Update documentation nitpick_ignore for c:identifer domain (GH-107295) (#107297) gh-106948: Update documentation nitpick_ignore for c:identifer domain (GH-107295) Update the nitpick_ignore of the documentation configuration to fix Sphinx warnings about standard C types when declaring functions with the "c:function" markups. Copy standard C types declared in the "c:type" domain to the "c:identifier" domain, since "c:function" markup looks for types in the "c:identifier" domain. (cherry picked from commit b1de3807b832b72dfeb66dd5646159d08d2cc74a) Co-authored-by: Victor Stinner Co-authored-by: Serhiy Storchaka files: M Doc/conf.py M Doc/tools/.nitignore diff --git a/Doc/conf.py b/Doc/conf.py index 067aa1d9e2291..209656be4ad43 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -153,6 +153,15 @@ ('py:meth', '_SubParsersAction.add_parser'), ] +# gh-106948: Copy standard C types declared in the "c:type" domain to the +# "c:identifier" domain, since "c:function" markup looks for types in the +# "c:identifier" domain. Use list() to not iterate on items which are being +# added +for role, name in list(nitpick_ignore): + if role == 'c:type': + nitpick_ignore.append(('c:identifier', name)) +del role, name + # Disable Docutils smartquotes for several translations smartquotes_excludes = { 'languages': ['ja', 'fr', 'zh_TW', 'zh_CN'], 'builders': ['man', 'text'], diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index b6adbcb34663e..849ef1168b455 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -9,7 +9,6 @@ Doc/c-api/arg.rst Doc/c-api/bool.rst Doc/c-api/buffer.rst Doc/c-api/bytes.rst -Doc/c-api/call.rst Doc/c-api/capsule.rst Doc/c-api/cell.rst Doc/c-api/code.rst @@ -28,8 +27,6 @@ Doc/c-api/init.rst Doc/c-api/init_config.rst Doc/c-api/intro.rst Doc/c-api/iterator.rst -Doc/c-api/long.rst -Doc/c-api/marshal.rst Doc/c-api/memory.rst Doc/c-api/memoryview.rst Doc/c-api/module.rst From webhook-mailer at python.org Wed Jul 26 12:27:12 2023 From: webhook-mailer at python.org (vstinner) Date: Wed, 26 Jul 2023 16:27:12 -0000 Subject: [Python-checkins] [3.11] gh-106948: Update documentation nitpick_ignore for c:identifer domain (#107295) (#107299) Message-ID: https://github.com/python/cpython/commit/bd0def00b3d92b30d6d0440a45ab811f3328933e commit: bd0def00b3d92b30d6d0440a45ab811f3328933e branch: 3.11 author: Victor Stinner committer: vstinner date: 2023-07-26T16:27:08Z summary: [3.11] gh-106948: Update documentation nitpick_ignore for c:identifer domain (#107295) (#107299) gh-106948: Update documentation nitpick_ignore for c:identifer domain (#107295) Update the nitpick_ignore of the documentation configuration to fix Sphinx warnings about standard C types when declaring functions with the "c:function" markups. Copy standard C types declared in the "c:type" domain to the "c:identifier" domain, since "c:function" markup looks for types in the "c:identifier" domain. Co-authored-by: Serhiy Storchaka (cherry picked from commit b1de3807b832b72dfeb66dd5646159d08d2cc74a) files: M Doc/conf.py diff --git a/Doc/conf.py b/Doc/conf.py index 28c35f2b0a234..d5eea9cdfb88d 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -148,6 +148,15 @@ ('py:meth', '_SubParsersAction.add_parser'), ] +# gh-106948: Copy standard C types declared in the "c:type" domain to the +# "c:identifier" domain, since "c:function" markup looks for types in the +# "c:identifier" domain. Use list() to not iterate on items which are being +# added +for role, name in list(nitpick_ignore): + if role == 'c:type': + nitpick_ignore.append(('c:identifier', name)) +del role, name + # Disable Docutils smartquotes for several translations smartquotes_excludes = { 'languages': ['ja', 'fr', 'zh_TW', 'zh_CN'], 'builders': ['man', 'text'], From webhook-mailer at python.org Wed Jul 26 12:59:12 2023 From: webhook-mailer at python.org (vstinner) Date: Wed, 26 Jul 2023 16:59:12 -0000 Subject: [Python-checkins] gh-107298: Fix doc references to undocumented modules (#107300) Message-ID: https://github.com/python/cpython/commit/87b39028e5f453a949a1675526c439f6479a04a8 commit: 87b39028e5f453a949a1675526c439f6479a04a8 branch: main author: Victor Stinner committer: vstinner date: 2023-07-26T18:59:06+02:00 summary: gh-107298: Fix doc references to undocumented modules (#107300) Update also Doc/tools/.nitignore. files: M Doc/c-api/arg.rst M Doc/c-api/codec.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/install/index.rst M Doc/tools/.nitignore diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index 9d744a12e374b..ae321ff918d13 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -477,7 +477,7 @@ API Functions will be set if there was a failure. This is an example of the use of this function, taken from the sources for the - :mod:`_weakref` helper module for weak references:: + :mod:`!_weakref` helper module for weak references:: static PyObject * weakref_ref(PyObject *self, PyObject *args) diff --git a/Doc/c-api/codec.rst b/Doc/c-api/codec.rst index 235c77c945cc5..8ae5c4fecd624 100644 --- a/Doc/c-api/codec.rst +++ b/Doc/c-api/codec.rst @@ -7,7 +7,7 @@ Codec registry and support functions Register a new codec search function. - As side effect, this tries to load the :mod:`encodings` package, if not yet + As side effect, this tries to load the :mod:`!encodings` package, if not yet done, to make sure that it is always first in the list of search functions. .. c:function:: int PyCodec_Unregister(PyObject *search_function) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 9aa076baaa977..c0f0d784ddeb4 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -579,7 +579,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) name, followed by a dot, followed by the type name; for built-in types, it should be just the type name. If the module is a submodule of a package, the full package name is part of the full module name. For example, a type named - :class:`T` defined in module :mod:`M` in subpackage :mod:`Q` in package :mod:`P` + :class:`T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``. For :ref:`dynamically allocated type objects `, diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index cf97b11bdbbf4..ad1d5edb901dc 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1205,7 +1205,7 @@ Character Map Codecs This codec is special in that it can be used to implement many different codecs (and this is in fact what was done to obtain most of the standard codecs -included in the :mod:`encodings` package). The codec uses mappings to encode and +included in the :mod:`!encodings` package). The codec uses mappings to encode and decode characters. The mapping objects provided must support the :meth:`__getitem__` mapping interface; dictionaries and sequences work well. diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 00a9edcf2e414..94ead2b60f8d6 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -367,7 +367,7 @@ Note that PyMODINIT_FUNC declares the function as ``PyObject *`` return type, declares any special linkage declarations required by the platform, and for C++ declares the function as ``extern "C"``. -When the Python program imports module :mod:`spam` for the first time, +When the Python program imports module :mod:`!spam` for the first time, :c:func:`PyInit_spam` is called. (See below for comments about embedding Python.) It calls :c:func:`PyModule_Create`, which returns a module object, and inserts built-in function objects into the newly created module based upon the @@ -1219,7 +1219,7 @@ file corresponding to the module provides a macro that takes care of importing the module and retrieving its C API pointers; client modules only have to call this macro before accessing the C API. -The exporting module is a modification of the :mod:`spam` module from section +The exporting module is a modification of the :mod:`!spam` module from section :ref:`extending-simpleexample`. The function :func:`spam.system` does not call the C library function :c:func:`system` directly, but a function :c:func:`PySpam_System`, which would of course do something more complicated in diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index 4f9c8899ffee1..a05bae8e07cd3 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -37,7 +37,7 @@ object. This sort of thing can only be explained by example, so here's a minimal, but complete, module that defines a new type named :class:`Custom` inside a C -extension module :mod:`custom`: +extension module :mod:`!custom`: .. note:: What we're showing here is the traditional way of defining *static* @@ -55,7 +55,7 @@ from the previous chapter. This file defines three things: #. How the :class:`Custom` **type** behaves: this is the ``CustomType`` struct, which defines a set of flags and function pointers that the interpreter inspects when specific operations are requested. -#. How to initialize the :mod:`custom` module: this is the ``PyInit_custom`` +#. How to initialize the :mod:`!custom` module: this is the ``PyInit_custom`` function and the associated ``custommodule`` struct. The first bit is:: @@ -127,7 +127,7 @@ our objects and in some error messages, for example: TypeError: can only concatenate str (not "custom.Custom") to str Note that the name is a dotted name that includes both the module name and the -name of the type within the module. The module in this case is :mod:`custom` and +name of the type within the module. The module in this case is :mod:`!custom` and the type is :class:`Custom`, so we set the type name to :class:`custom.Custom`. Using the real dotted import path is important to make your type compatible with the :mod:`pydoc` and :mod:`pickle` modules. :: @@ -229,7 +229,7 @@ Adding data and methods to the Basic example ============================================ Let's extend the basic example to add some data and methods. Let's also make -the type usable as a base class. We'll create a new module, :mod:`custom2` that +the type usable as a base class. We'll create a new module, :mod:`!custom2` that adds these capabilities: .. literalinclude:: ../includes/custom2.c diff --git a/Doc/install/index.rst b/Doc/install/index.rst index 34d47fdb6a2f0..443c75b7684fb 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -374,7 +374,7 @@ will expand this to your home directory:: To make Python find the distributions installed with this scheme, you may have to :ref:`modify Python's search path ` or edit -:mod:`sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit +:mod:`!sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit :data:`sys.path`. The :option:`!--home` option defines the installation base directory. Files are diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 07123bcb5ae06..2a10db5f3fff8 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -4,14 +4,12 @@ Doc/c-api/allocation.rst Doc/c-api/apiabiversion.rst -Doc/c-api/arg.rst Doc/c-api/bool.rst Doc/c-api/buffer.rst Doc/c-api/bytes.rst Doc/c-api/capsule.rst Doc/c-api/cell.rst Doc/c-api/code.rst -Doc/c-api/codec.rst Doc/c-api/complex.rst Doc/c-api/conversion.rst Doc/c-api/datetime.rst From webhook-mailer at python.org Wed Jul 26 14:39:02 2023 From: webhook-mailer at python.org (brettcannon) Date: Wed, 26 Jul 2023 18:39:02 -0000 Subject: [Python-checkins] Document that `os.link()` is not available on Emscripten (GH-104822) Message-ID: https://github.com/python/cpython/commit/737d1da0746053d515158eac5b115e8bd813f6d3 commit: 737d1da0746053d515158eac5b115e8bd813f6d3 branch: main author: Roman Yurchak committer: brettcannon date: 2023-07-26T11:38:59-07:00 summary: Document that `os.link()` is not available on Emscripten (GH-104822) files: M Doc/library/os.rst diff --git a/Doc/library/os.rst b/Doc/library/os.rst index bbf227aab649e..8b4e7649beacb 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2150,7 +2150,7 @@ features: .. audit-event:: os.link src,dst,src_dir_fd,dst_dir_fd os.link - .. availability:: Unix, Windows. + .. availability:: Unix, Windows, not Emscripten. .. versionchanged:: 3.2 Added Windows support. From webhook-mailer at python.org Wed Jul 26 15:16:11 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 26 Jul 2023 19:16:11 -0000 Subject: [Python-checkins] gh-107091: Fix some uses of :c:member: role (GH-107129) Message-ID: https://github.com/python/cpython/commit/af61cb9c7837ff3c11da79e3ee1cab3fdd0ba4da commit: af61cb9c7837ff3c11da79e3ee1cab3fdd0ba4da branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-26T22:16:06+03:00 summary: gh-107091: Fix some uses of :c:member: role (GH-107129) files: M Doc/c-api/import.rst M Doc/c-api/init_config.rst M Doc/c-api/module.rst M Doc/c-api/structures.rst M Doc/c-api/typeobj.rst M Doc/howto/isolating-extensions.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.7.rst diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index f9c2f4e69ce88..2ed0984dbc9df 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -154,7 +154,7 @@ Importing Modules :class:`SourceFileLoader` otherwise. The module's :attr:`__file__` attribute will be set to the code object's - :c:member:`co_filename`. If applicable, :attr:`__cached__` will also + :attr:`co_filename`. If applicable, :attr:`__cached__` will also be set. This function will reload the module if it was already imported. See diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 5f70c45c54f75..0c2e707c94c8d 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -522,7 +522,7 @@ PyConfig Moreover, if :c:func:`PyConfig_SetArgv` or :c:func:`PyConfig_SetBytesArgv` is used, this method must be called before other methods, since the preinitialization configuration depends on command line arguments (if - :c:member:`parse_argv` is non-zero). + :c:member:`~PyConfig.parse_argv` is non-zero). The caller of these methods is responsible to handle exceptions (error or exit) using ``PyStatus_Exception()`` and ``Py_ExitStatusException()``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index e85baa03a2247..3ecaeee22d7fd 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -164,7 +164,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This memory area is allocated based on *m_size* on module creation, and freed when the module object is deallocated, after the - :c:member:`m_free` function has been called, if present. + :c:member:`~PyModuleDef.m_free` function has been called, if present. Setting ``m_size`` to ``-1`` means that the module does not support sub-interpreters, because it has global state. @@ -202,7 +202,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More - precisely, this function is not called if :c:member:`m_size` is greater + precisely, this function is not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. @@ -217,7 +217,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More - precisely, this function is not called if :c:member:`m_size` is greater + precisely, this function is not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. @@ -238,7 +238,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More - precisely, this function is not called if :c:member:`m_size` is greater + precisely, this function is not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 720ab31f3de85..75e325e8fa83f 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -244,14 +244,16 @@ Implementing functions and methods points to the contents of the docstring -The :c:member:`ml_meth` is a C function pointer. The functions may be of different +The :c:member:`~PyMethodDef.ml_meth` is a C function pointer. +The functions may be of different types, but they always return :c:expr:`PyObject*`. If the function is not of the :c:type:`PyCFunction`, the compiler will require a cast in the method table. Even though :c:type:`PyCFunction` defines the first parameter as :c:expr:`PyObject*`, it is common that the method implementation uses the specific C type of the *self* object. -The :c:member:`ml_flags` field is a bitfield which can include the following flags. +The :c:member:`~PyMethodDef.ml_flags` field is a bitfield which can include +the following flags. The individual flags indicate either a calling convention or a binding convention. @@ -432,7 +434,7 @@ Accessing attributes of extension types The string should be static, no copy is made of it. Typically, it is defined using :c:macro:`PyDoc_STR`. - By default (when :c:member:`flags` is ``0``), members allow + By default (when :c:member:`~PyMemberDef.flags` is ``0``), members allow both read and write access. Use the :c:macro:`Py_READONLY` flag for read-only access. Certain types, like :c:macro:`Py_T_STRING`, imply :c:macro:`Py_READONLY`. @@ -512,7 +514,7 @@ The following flags can be used with :c:member:`PyMemberDef.flags`: Can only be used as part of :c:member:`Py_tp_members ` :c:type:`slot ` when creating a class using negative - :c:member:`~PyTypeDef.basicsize`. + :c:member:`~PyType_Spec.basicsize`. It is mandatory in that case. This flag is only used in :c:type:`PyTypeSlot`. diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index c0f0d784ddeb4..a070c41c58d34 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1176,7 +1176,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:macro:: Py_TPFLAGS_ITEMS_AT_END Only usable with variable-size types, i.e. ones with non-zero - :c:member:`~PyObject.tp_itemsize`. + :c:member:`~PyTypeObject.tp_itemsize`. Indicates that the variable-sized portion of an instance of this type is at the end of the instance's memory area, at an offset of diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index cc4a908febfb7..60854c3c034d7 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -467,7 +467,7 @@ Module State Access from Slot Methods, Getters and Setters Slot methods?the fast C equivalents for special methods, such as :c:member:`~PyNumberMethods.nb_add` for :py:attr:`~object.__add__` or -:c:member:`~PyType.tp_new` for initialization?have a very simple API that +:c:member:`~PyTypeObject.tp_new` for initialization?have a very simple API that doesn't allow passing in the defining class, unlike with :c:type:`PyCMethod`. The same goes for getters and setters defined with :c:type:`PyGetSetDef`. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 1e30569c559b9..86bfdc4d478a0 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2533,7 +2533,7 @@ Changes in the C API * As part of the :pep:`492` implementation, the ``tp_reserved`` slot of :c:type:`PyTypeObject` was replaced with a - :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for + :c:member:`~PyTypeObject.tp_as_async` slot. Refer to :ref:`coro-objects` for new types, structures and functions. diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 09c91fc00dab2..8a4496bc727ef 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1674,7 +1674,7 @@ The new :c:func:`import__find__load__start` and module imports. (Contributed by Christian Heimes in :issue:`31574`.) -The fields :c:member:`name` and :c:member:`doc` of structures +The fields :c:member:`!name` and :c:member:`!doc` of structures :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, and :c:type:`wrapperbase` are now of type ``const char *`` rather of From webhook-mailer at python.org Wed Jul 26 15:29:27 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 26 Jul 2023 19:29:27 -0000 Subject: [Python-checkins] gh-107091: Fix some uses of :c:type: role (GH-107138) Message-ID: https://github.com/python/cpython/commit/6d5b6e71c87fca7c5c26f5dd8f325087962215cc commit: 6d5b6e71c87fca7c5c26f5dd8f325087962215cc branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-26T22:29:23+03:00 summary: gh-107091: Fix some uses of :c:type: role (GH-107138) files: M Doc/c-api/tuple.rst M Doc/c-api/typeobj.rst M Doc/conf.py M Doc/library/os.rst M Doc/whatsnew/2.4.rst M Doc/whatsnew/3.2.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.7.rst M Misc/NEWS.d/3.10.0a7.rst diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index 3fe1062aa8539..2277a39c95906 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -114,6 +114,8 @@ Tuple Objects raises :exc:`MemoryError` or :exc:`SystemError`. +.. _struct-sequence-objects: + Struct Sequence Objects ----------------------- diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index a070c41c58d34..2b21a349ccc4c 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -163,9 +163,9 @@ Quick Reference .. [#cols] Columns: - **"O"**: set on :c:type:`PyBaseObject_Type` + **"O"**: set on :c:data:`PyBaseObject_Type` - **"T"**: set on :c:type:`PyType_Type` + **"T"**: set on :c:data:`PyType_Type` **"D"**: default (if slot is set to ``NULL``) @@ -569,8 +569,8 @@ PyTypeObject Slots Each slot has a section describing inheritance. If :c:func:`PyType_Ready` may set a value when the field is set to ``NULL`` then there will also be -a "Default" section. (Note that many fields set on :c:type:`PyBaseObject_Type` -and :c:type:`PyType_Type` effectively act as defaults.) +a "Default" section. (Note that many fields set on :c:data:`PyBaseObject_Type` +and :c:data:`PyType_Type` effectively act as defaults.) .. c:member:: const char* PyTypeObject.tp_name @@ -964,7 +964,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` uses :c:func:`PyObject_GenericGetAttr`. + :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_GenericGetAttr`. .. c:member:: setattrofunc PyTypeObject.tp_setattro @@ -990,7 +990,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` uses :c:func:`PyObject_GenericSetAttr`. + :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_GenericSetAttr`. .. c:member:: PyBufferProcs* PyTypeObject.tp_as_buffer @@ -1031,7 +1031,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` uses + :c:data:`PyBaseObject_Type` uses ``Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE``. **Bit Masks:** @@ -1556,7 +1556,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` provides a :attr:`tp_richcompare` + :c:data:`PyBaseObject_Type` provides a :attr:`tp_richcompare` implementation, which may be inherited. However, if only :attr:`tp_hash` is defined, not even the inherited function is used and instances of the type will not be able to participate in any @@ -1878,7 +1878,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:func:`PyType_GenericAlloc`, to force a standard heap allocation strategy. - For static subtypes, :c:type:`PyBaseObject_Type` uses + For static subtypes, :c:data:`PyBaseObject_Type` uses :c:func:`PyType_GenericAlloc`. That is the recommended value for all statically defined types. @@ -1941,7 +1941,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) match :c:func:`PyType_GenericAlloc` and the value of the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. - For static subtypes, :c:type:`PyBaseObject_Type` uses PyObject_Del. + For static subtypes, :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_Del`. .. c:member:: inquiry PyTypeObject.tp_is_gc diff --git a/Doc/conf.py b/Doc/conf.py index 8821a2a172f11..93c9bd121cc1e 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -108,6 +108,10 @@ ('c:type', 'uintmax_t'), ('c:type', 'va_list'), ('c:type', 'wchar_t'), + ('c:struct', 'in6_addr'), + ('c:struct', 'in_addr'), + ('c:struct', 'stat'), + ('c:struct', 'statvfs'), # Standard C macros ('c:macro', 'LLONG_MAX'), ('c:macro', 'LLONG_MIN'), diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 8b4e7649beacb..956eeffa1cd45 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2420,13 +2420,13 @@ features: .. function:: major(device, /) Extract the device major number from a raw device number (usually the - :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`). + :attr:`st_dev` or :attr:`st_rdev` field from :c:struct:`stat`). .. function:: minor(device, /) Extract the device minor number from a raw device number (usually the - :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`). + :attr:`st_dev` or :attr:`st_rdev` field from :c:struct:`stat`). .. function:: makedev(major, minor, /) @@ -2937,7 +2937,7 @@ features: .. class:: stat_result Object whose attributes correspond roughly to the members of the - :c:type:`stat` structure. It is used for the result of :func:`os.stat`, + :c:struct:`stat` structure. It is used for the result of :func:`os.stat`, :func:`os.fstat` and :func:`os.lstat`. Attributes: @@ -3117,12 +3117,12 @@ features: See the ``IO_REPARSE_TAG_*`` constants in the :mod:`stat` module. The standard module :mod:`stat` defines functions and constants that are - useful for extracting information from a :c:type:`stat` structure. (On + useful for extracting information from a :c:struct:`stat` structure. (On Windows, some items are filled with dummy values.) For backward compatibility, a :class:`stat_result` instance is also accessible as a tuple of at least 10 integers giving the most important (and - portable) members of the :c:type:`stat` structure, in the order + portable) members of the :c:struct:`stat` structure, in the order :attr:`st_mode`, :attr:`st_ino`, :attr:`st_dev`, :attr:`st_nlink`, :attr:`st_uid`, :attr:`st_gid`, :attr:`st_size`, :attr:`st_atime`, :attr:`st_mtime`, :attr:`st_ctime`. More items may be added at the end by @@ -3174,7 +3174,7 @@ features: Perform a :c:func:`statvfs` system call on the given path. The return value is an object whose attributes describe the filesystem on the given path, and - correspond to the members of the :c:type:`statvfs` structure, namely: + correspond to the members of the :c:struct:`statvfs` structure, namely: :attr:`f_bsize`, :attr:`f_frsize`, :attr:`f_blocks`, :attr:`f_bfree`, :attr:`f_bavail`, :attr:`f_files`, :attr:`f_ffree`, :attr:`f_favail`, :attr:`f_flag`, :attr:`f_namemax`, :attr:`f_fsid`. diff --git a/Doc/whatsnew/2.4.rst b/Doc/whatsnew/2.4.rst index ad46637b8497b..9e8a9e6a622d0 100644 --- a/Doc/whatsnew/2.4.rst +++ b/Doc/whatsnew/2.4.rst @@ -1491,7 +1491,7 @@ Some of the changes to Python's build process and to the C API are: though that processor architecture doesn't call that register "the TSC register". (Contributed by Jeremy Hylton.) -* The :c:type:`tracebackobject` type has been renamed to +* The :c:type:`!tracebackobject` type has been renamed to :c:type:`PyTracebackObject`. .. ====================================================================== diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index b9c6541795e44..ed1c1770fb0f5 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -566,7 +566,7 @@ Some smaller changes made to the core Python language are: (See :issue:`4617`.) -* The internal :c:type:`structsequence` tool now creates subclasses of tuple. +* :ref:`Struct sequence types ` are now subclasses of tuple. This means that C structures like those returned by :func:`os.stat`, :func:`time.gmtime`, and :data:`sys.version_info` now work like a :term:`named tuple` and now work with functions and methods that diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 911e5ba47df41..2b87f9eb9196a 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -2195,7 +2195,7 @@ Changes to Python's build process and to the C API include: * :c:func:`PyUnicode_AsUCS4`, :c:func:`PyUnicode_AsUCS4Copy` * :c:macro:`PyUnicode_DATA`, :c:macro:`PyUnicode_1BYTE_DATA`, :c:macro:`PyUnicode_2BYTE_DATA`, :c:macro:`PyUnicode_4BYTE_DATA` - * :c:macro:`PyUnicode_KIND` with :c:type:`PyUnicode_Kind` enum: + * :c:macro:`PyUnicode_KIND` with :c:enum:`PyUnicode_Kind` enum: :c:data:`PyUnicode_WCHAR_KIND`, :c:data:`PyUnicode_1BYTE_KIND`, :c:data:`PyUnicode_2BYTE_KIND`, :c:data:`PyUnicode_4BYTE_KIND` * :c:macro:`PyUnicode_READ`, :c:macro:`PyUnicode_READ_CHAR`, :c:macro:`PyUnicode_WRITE` diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 8a4496bc727ef..218a37cd264c7 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1677,7 +1677,7 @@ module imports. The fields :c:member:`!name` and :c:member:`!doc` of structures :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, -and :c:type:`wrapperbase` are now of type ``const char *`` rather of +and :c:struct:`wrapperbase` are now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` diff --git a/Misc/NEWS.d/3.10.0a7.rst b/Misc/NEWS.d/3.10.0a7.rst index ff01ee645c0a9..7933f71b01c14 100644 --- a/Misc/NEWS.d/3.10.0a7.rst +++ b/Misc/NEWS.d/3.10.0a7.rst @@ -215,8 +215,8 @@ a non-Python signal handler. .. nonce: VouZjn .. section: Core and Builtins -Add ``__match_args__`` to :c:type:`structsequence` based classes. Patch by -Pablo Galindo. +Add ``__match_args__`` to :ref:`struct sequence objects `. +Patch by Pablo Galindo. .. From webhook-mailer at python.org Wed Jul 26 15:30:50 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 26 Jul 2023 19:30:50 -0000 Subject: [Python-checkins] [3.12] gh-107091: Fix some uses of :c:member: role (GH-107129) (GH-107310) Message-ID: https://github.com/python/cpython/commit/9f5a5f0b33479abcd1619fcc5ea3b8d0a31818f4 commit: 9f5a5f0b33479abcd1619fcc5ea3b8d0a31818f4 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-26T22:30:46+03:00 summary: [3.12] gh-107091: Fix some uses of :c:member: role (GH-107129) (GH-107310) (cherry picked from commit af61cb9c7837ff3c11da79e3ee1cab3fdd0ba4da) Co-authored-by: Serhiy Storchaka files: M Doc/c-api/import.rst M Doc/c-api/init_config.rst M Doc/c-api/module.rst M Doc/c-api/structures.rst M Doc/c-api/typeobj.rst M Doc/howto/isolating-extensions.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.7.rst diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index d32c09ef06c9e..905cab9034bec 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -138,7 +138,7 @@ Importing Modules :class:`SourceFileLoader` otherwise. The module's :attr:`__file__` attribute will be set to the code object's - :c:member:`co_filename`. If applicable, :attr:`__cached__` will also + :attr:`co_filename`. If applicable, :attr:`__cached__` will also be set. This function will reload the module if it was already imported. See diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 161def0b4baf3..18f17e2059b10 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -522,7 +522,7 @@ PyConfig Moreover, if :c:func:`PyConfig_SetArgv` or :c:func:`PyConfig_SetBytesArgv` is used, this method must be called before other methods, since the preinitialization configuration depends on command line arguments (if - :c:member:`parse_argv` is non-zero). + :c:member:`~PyConfig.parse_argv` is non-zero). The caller of these methods is responsible to handle exceptions (error or exit) using ``PyStatus_Exception()`` and ``Py_ExitStatusException()``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index e358c5da14a69..6abd550ab80b2 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -164,7 +164,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This memory area is allocated based on *m_size* on module creation, and freed when the module object is deallocated, after the - :c:member:`m_free` function has been called, if present. + :c:member:`~PyModuleDef.m_free` function has been called, if present. Setting ``m_size`` to ``-1`` means that the module does not support sub-interpreters, because it has global state. @@ -202,7 +202,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More - precisely, this function is not called if :c:member:`m_size` is greater + precisely, this function is not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. @@ -217,7 +217,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More - precisely, this function is not called if :c:member:`m_size` is greater + precisely, this function is not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. @@ -238,7 +238,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More - precisely, this function is not called if :c:member:`m_size` is greater + precisely, this function is not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 720ab31f3de85..75e325e8fa83f 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -244,14 +244,16 @@ Implementing functions and methods points to the contents of the docstring -The :c:member:`ml_meth` is a C function pointer. The functions may be of different +The :c:member:`~PyMethodDef.ml_meth` is a C function pointer. +The functions may be of different types, but they always return :c:expr:`PyObject*`. If the function is not of the :c:type:`PyCFunction`, the compiler will require a cast in the method table. Even though :c:type:`PyCFunction` defines the first parameter as :c:expr:`PyObject*`, it is common that the method implementation uses the specific C type of the *self* object. -The :c:member:`ml_flags` field is a bitfield which can include the following flags. +The :c:member:`~PyMethodDef.ml_flags` field is a bitfield which can include +the following flags. The individual flags indicate either a calling convention or a binding convention. @@ -432,7 +434,7 @@ Accessing attributes of extension types The string should be static, no copy is made of it. Typically, it is defined using :c:macro:`PyDoc_STR`. - By default (when :c:member:`flags` is ``0``), members allow + By default (when :c:member:`~PyMemberDef.flags` is ``0``), members allow both read and write access. Use the :c:macro:`Py_READONLY` flag for read-only access. Certain types, like :c:macro:`Py_T_STRING`, imply :c:macro:`Py_READONLY`. @@ -512,7 +514,7 @@ The following flags can be used with :c:member:`PyMemberDef.flags`: Can only be used as part of :c:member:`Py_tp_members ` :c:type:`slot ` when creating a class using negative - :c:member:`~PyTypeDef.basicsize`. + :c:member:`~PyType_Spec.basicsize`. It is mandatory in that case. This flag is only used in :c:type:`PyTypeSlot`. diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 9aa076baaa977..de190f5ac39ad 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1176,7 +1176,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:macro:: Py_TPFLAGS_ITEMS_AT_END Only usable with variable-size types, i.e. ones with non-zero - :c:member:`~PyObject.tp_itemsize`. + :c:member:`~PyTypeObject.tp_itemsize`. Indicates that the variable-sized portion of an instance of this type is at the end of the instance's memory area, at an offset of diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index cc4a908febfb7..60854c3c034d7 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -467,7 +467,7 @@ Module State Access from Slot Methods, Getters and Setters Slot methods?the fast C equivalents for special methods, such as :c:member:`~PyNumberMethods.nb_add` for :py:attr:`~object.__add__` or -:c:member:`~PyType.tp_new` for initialization?have a very simple API that +:c:member:`~PyTypeObject.tp_new` for initialization?have a very simple API that doesn't allow passing in the defining class, unlike with :c:type:`PyCMethod`. The same goes for getters and setters defined with :c:type:`PyGetSetDef`. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index d23644b5093cb..002d829e78bb3 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2533,7 +2533,7 @@ Changes in the C API * As part of the :pep:`492` implementation, the ``tp_reserved`` slot of :c:type:`PyTypeObject` was replaced with a - :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for + :c:member:`~PyTypeObject.tp_as_async` slot. Refer to :ref:`coro-objects` for new types, structures and functions. diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index eda8bf0137bff..56ffc3ef34570 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1674,7 +1674,7 @@ The new :c:func:`import__find__load__start` and module imports. (Contributed by Christian Heimes in :issue:`31574`.) -The fields :c:member:`name` and :c:member:`doc` of structures +The fields :c:member:`!name` and :c:member:`!doc` of structures :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, and :c:type:`wrapperbase` are now of type ``const char *`` rather of From webhook-mailer at python.org Wed Jul 26 15:35:19 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 26 Jul 2023 19:35:19 -0000 Subject: [Python-checkins] [3.11] gh-107091: Fix some uses of :c:member: role (GH-107129) (GH-107311) Message-ID: https://github.com/python/cpython/commit/cfa9f3b7ccfb0482bb525c73f994cf669c32a11c commit: cfa9f3b7ccfb0482bb525c73f994cf669c32a11c branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-26T19:35:16Z summary: [3.11] gh-107091: Fix some uses of :c:member: role (GH-107129) (GH-107311) (cherry picked from commit af61cb9c7837ff3c11da79e3ee1cab3fdd0ba4da) files: M Doc/c-api/import.rst M Doc/c-api/init_config.rst M Doc/c-api/module.rst M Doc/c-api/structures.rst M Doc/howto/isolating-extensions.rst M Doc/whatsnew/3.5.rst M Doc/whatsnew/3.7.rst diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 4ca1517efcb38..8b2d38582343b 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -138,7 +138,7 @@ Importing Modules :class:`SourceFileLoader` otherwise. The module's :attr:`__file__` attribute will be set to the code object's - :c:member:`co_filename`. If applicable, :attr:`__cached__` will also + :attr:`co_filename`. If applicable, :attr:`__cached__` will also be set. This function will reload the module if it was already imported. See diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 24d66201f68db..715ad47f13eb4 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -522,7 +522,7 @@ PyConfig Moreover, if :c:func:`PyConfig_SetArgv` or :c:func:`PyConfig_SetBytesArgv` is used, this method must be called before other methods, since the preinitialization configuration depends on command line arguments (if - :c:member:`parse_argv` is non-zero). + :c:member:`~PyConfig.parse_argv` is non-zero). The caller of these methods is responsible to handle exceptions (error or exit) using ``PyStatus_Exception()`` and ``Py_ExitStatusException()``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index e358c5da14a69..6abd550ab80b2 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -164,7 +164,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This memory area is allocated based on *m_size* on module creation, and freed when the module object is deallocated, after the - :c:member:`m_free` function has been called, if present. + :c:member:`~PyModuleDef.m_free` function has been called, if present. Setting ``m_size`` to ``-1`` means that the module does not support sub-interpreters, because it has global state. @@ -202,7 +202,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More - precisely, this function is not called if :c:member:`m_size` is greater + precisely, this function is not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. @@ -217,7 +217,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More - precisely, this function is not called if :c:member:`m_size` is greater + precisely, this function is not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. @@ -238,7 +238,7 @@ or request "multi-phase initialization" by returning the definition struct itsel This function is not called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (:c:data:`Py_mod_exec` function). More - precisely, this function is not called if :c:member:`m_size` is greater + precisely, this function is not called if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as returned by :c:func:`PyModule_GetState`) is ``NULL``. diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index e9f158c176924..5c85838d2dbfa 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -263,14 +263,16 @@ Implementing functions and methods points to the contents of the docstring -The :c:member:`ml_meth` is a C function pointer. The functions may be of different +The :c:member:`~PyMethodDef.ml_meth` is a C function pointer. +The functions may be of different types, but they always return :c:expr:`PyObject*`. If the function is not of the :c:type:`PyCFunction`, the compiler will require a cast in the method table. Even though :c:type:`PyCFunction` defines the first parameter as :c:expr:`PyObject*`, it is common that the method implementation uses the specific C type of the *self* object. -The :c:member:`ml_flags` field is a bitfield which can include the following flags. +The :c:member:`~PyMethodDef.ml_flags` field is a bitfield which can include +the following flags. The individual flags indicate either a calling convention or a binding convention. diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index c88c9edc0ccad..0362fa6c68544 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -467,7 +467,7 @@ Module State Access from Slot Methods, Getters and Setters Slot methods?the fast C equivalents for special methods, such as :c:member:`~PyNumberMethods.nb_add` for :py:attr:`~object.__add__` or -:c:member:`~PyType.tp_new` for initialization?have a very simple API that +:c:member:`~PyTypeObject.tp_new` for initialization?have a very simple API that doesn't allow passing in the defining class, unlike with :c:type:`PyCMethod`. The same goes for getters and setters defined with :c:type:`PyGetSetDef`. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 315de6b59fe5f..9bd97e4726ca2 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2533,7 +2533,7 @@ Changes in the C API * As part of the :pep:`492` implementation, the ``tp_reserved`` slot of :c:type:`PyTypeObject` was replaced with a - :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for + :c:member:`~PyTypeObject.tp_as_async` slot. Refer to :ref:`coro-objects` for new types, structures and functions. diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index d00c55a776613..02406691d4ac7 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1674,7 +1674,7 @@ The new :c:func:`import__find__load__start` and module imports. (Contributed by Christian Heimes in :issue:`31574`.) -The fields :c:member:`name` and :c:member:`doc` of structures +The fields :c:member:`!name` and :c:member:`!doc` of structures :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, and :c:type:`wrapperbase` are now of type ``const char *`` rather of From webhook-mailer at python.org Wed Jul 26 15:37:43 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 26 Jul 2023 19:37:43 -0000 Subject: [Python-checkins] [3.12] gh-107091: Fix some uses of :c:type: role (GH-107138) (GH-107312) Message-ID: https://github.com/python/cpython/commit/da2097dffbf1573d34439166468012d8f79d8595 commit: da2097dffbf1573d34439166468012d8f79d8595 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-26T19:37:39Z summary: [3.12] gh-107091: Fix some uses of :c:type: role (GH-107138) (GH-107312) (cherry picked from commit 6d5b6e71c87fca7c5c26f5dd8f325087962215cc) Co-authored-by: Serhiy Storchaka files: M Doc/c-api/tuple.rst M Doc/c-api/typeobj.rst M Doc/conf.py M Doc/library/os.rst M Doc/whatsnew/2.4.rst M Doc/whatsnew/3.2.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.7.rst M Misc/NEWS.d/3.10.0a7.rst diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index ac62058676eee..01b8da6cb4e0e 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -111,6 +111,8 @@ Tuple Objects raises :exc:`MemoryError` or :exc:`SystemError`. +.. _struct-sequence-objects: + Struct Sequence Objects ----------------------- diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index de190f5ac39ad..a6cf8a03d03b9 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -163,9 +163,9 @@ Quick Reference .. [#cols] Columns: - **"O"**: set on :c:type:`PyBaseObject_Type` + **"O"**: set on :c:data:`PyBaseObject_Type` - **"T"**: set on :c:type:`PyType_Type` + **"T"**: set on :c:data:`PyType_Type` **"D"**: default (if slot is set to ``NULL``) @@ -569,8 +569,8 @@ PyTypeObject Slots Each slot has a section describing inheritance. If :c:func:`PyType_Ready` may set a value when the field is set to ``NULL`` then there will also be -a "Default" section. (Note that many fields set on :c:type:`PyBaseObject_Type` -and :c:type:`PyType_Type` effectively act as defaults.) +a "Default" section. (Note that many fields set on :c:data:`PyBaseObject_Type` +and :c:data:`PyType_Type` effectively act as defaults.) .. c:member:: const char* PyTypeObject.tp_name @@ -964,7 +964,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` uses :c:func:`PyObject_GenericGetAttr`. + :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_GenericGetAttr`. .. c:member:: setattrofunc PyTypeObject.tp_setattro @@ -990,7 +990,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` uses :c:func:`PyObject_GenericSetAttr`. + :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_GenericSetAttr`. .. c:member:: PyBufferProcs* PyTypeObject.tp_as_buffer @@ -1031,7 +1031,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` uses + :c:data:`PyBaseObject_Type` uses ``Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE``. **Bit Masks:** @@ -1556,7 +1556,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` provides a :attr:`tp_richcompare` + :c:data:`PyBaseObject_Type` provides a :attr:`tp_richcompare` implementation, which may be inherited. However, if only :attr:`tp_hash` is defined, not even the inherited function is used and instances of the type will not be able to participate in any @@ -1878,7 +1878,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:func:`PyType_GenericAlloc`, to force a standard heap allocation strategy. - For static subtypes, :c:type:`PyBaseObject_Type` uses + For static subtypes, :c:data:`PyBaseObject_Type` uses :c:func:`PyType_GenericAlloc`. That is the recommended value for all statically defined types. @@ -1941,7 +1941,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) match :c:func:`PyType_GenericAlloc` and the value of the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. - For static subtypes, :c:type:`PyBaseObject_Type` uses PyObject_Del. + For static subtypes, :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_Del`. .. c:member:: inquiry PyTypeObject.tp_is_gc diff --git a/Doc/conf.py b/Doc/conf.py index 209656be4ad43..722831cdb54c3 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -108,6 +108,10 @@ ('c:type', 'uintmax_t'), ('c:type', 'va_list'), ('c:type', 'wchar_t'), + ('c:struct', 'in6_addr'), + ('c:struct', 'in_addr'), + ('c:struct', 'stat'), + ('c:struct', 'statvfs'), # Standard C macros ('c:macro', 'LLONG_MAX'), ('c:macro', 'LLONG_MIN'), diff --git a/Doc/library/os.rst b/Doc/library/os.rst index bbf227aab649e..55ebe7e669162 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2420,13 +2420,13 @@ features: .. function:: major(device, /) Extract the device major number from a raw device number (usually the - :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`). + :attr:`st_dev` or :attr:`st_rdev` field from :c:struct:`stat`). .. function:: minor(device, /) Extract the device minor number from a raw device number (usually the - :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`). + :attr:`st_dev` or :attr:`st_rdev` field from :c:struct:`stat`). .. function:: makedev(major, minor, /) @@ -2937,7 +2937,7 @@ features: .. class:: stat_result Object whose attributes correspond roughly to the members of the - :c:type:`stat` structure. It is used for the result of :func:`os.stat`, + :c:struct:`stat` structure. It is used for the result of :func:`os.stat`, :func:`os.fstat` and :func:`os.lstat`. Attributes: @@ -3117,12 +3117,12 @@ features: See the ``IO_REPARSE_TAG_*`` constants in the :mod:`stat` module. The standard module :mod:`stat` defines functions and constants that are - useful for extracting information from a :c:type:`stat` structure. (On + useful for extracting information from a :c:struct:`stat` structure. (On Windows, some items are filled with dummy values.) For backward compatibility, a :class:`stat_result` instance is also accessible as a tuple of at least 10 integers giving the most important (and - portable) members of the :c:type:`stat` structure, in the order + portable) members of the :c:struct:`stat` structure, in the order :attr:`st_mode`, :attr:`st_ino`, :attr:`st_dev`, :attr:`st_nlink`, :attr:`st_uid`, :attr:`st_gid`, :attr:`st_size`, :attr:`st_atime`, :attr:`st_mtime`, :attr:`st_ctime`. More items may be added at the end by @@ -3174,7 +3174,7 @@ features: Perform a :c:func:`statvfs` system call on the given path. The return value is an object whose attributes describe the filesystem on the given path, and - correspond to the members of the :c:type:`statvfs` structure, namely: + correspond to the members of the :c:struct:`statvfs` structure, namely: :attr:`f_bsize`, :attr:`f_frsize`, :attr:`f_blocks`, :attr:`f_bfree`, :attr:`f_bavail`, :attr:`f_files`, :attr:`f_ffree`, :attr:`f_favail`, :attr:`f_flag`, :attr:`f_namemax`, :attr:`f_fsid`. diff --git a/Doc/whatsnew/2.4.rst b/Doc/whatsnew/2.4.rst index d68f600ba44f6..a00c4360b9c68 100644 --- a/Doc/whatsnew/2.4.rst +++ b/Doc/whatsnew/2.4.rst @@ -1491,7 +1491,7 @@ Some of the changes to Python's build process and to the C API are: though that processor architecture doesn't call that register "the TSC register". (Contributed by Jeremy Hylton.) -* The :c:type:`tracebackobject` type has been renamed to +* The :c:type:`!tracebackobject` type has been renamed to :c:type:`PyTracebackObject`. .. ====================================================================== diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index a42775804637e..ec4d8a0312b6b 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -566,7 +566,7 @@ Some smaller changes made to the core Python language are: (See :issue:`4617`.) -* The internal :c:type:`structsequence` tool now creates subclasses of tuple. +* :ref:`Struct sequence types ` are now subclasses of tuple. This means that C structures like those returned by :func:`os.stat`, :func:`time.gmtime`, and :data:`sys.version_info` now work like a :term:`named tuple` and now work with functions and methods that diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index aaa4e873fddd8..6eb5818c317d8 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -2195,7 +2195,7 @@ Changes to Python's build process and to the C API include: * :c:func:`PyUnicode_AsUCS4`, :c:func:`PyUnicode_AsUCS4Copy` * :c:macro:`PyUnicode_DATA`, :c:macro:`PyUnicode_1BYTE_DATA`, :c:macro:`PyUnicode_2BYTE_DATA`, :c:macro:`PyUnicode_4BYTE_DATA` - * :c:macro:`PyUnicode_KIND` with :c:type:`PyUnicode_Kind` enum: + * :c:macro:`PyUnicode_KIND` with :c:enum:`PyUnicode_Kind` enum: :c:data:`PyUnicode_WCHAR_KIND`, :c:data:`PyUnicode_1BYTE_KIND`, :c:data:`PyUnicode_2BYTE_KIND`, :c:data:`PyUnicode_4BYTE_KIND` * :c:macro:`PyUnicode_READ`, :c:macro:`PyUnicode_READ_CHAR`, :c:macro:`PyUnicode_WRITE` diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 56ffc3ef34570..e82bb756b8485 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1677,7 +1677,7 @@ module imports. The fields :c:member:`!name` and :c:member:`!doc` of structures :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, -and :c:type:`wrapperbase` are now of type ``const char *`` rather of +and :c:struct:`wrapperbase` are now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` diff --git a/Misc/NEWS.d/3.10.0a7.rst b/Misc/NEWS.d/3.10.0a7.rst index ff01ee645c0a9..7933f71b01c14 100644 --- a/Misc/NEWS.d/3.10.0a7.rst +++ b/Misc/NEWS.d/3.10.0a7.rst @@ -215,8 +215,8 @@ a non-Python signal handler. .. nonce: VouZjn .. section: Core and Builtins -Add ``__match_args__`` to :c:type:`structsequence` based classes. Patch by -Pablo Galindo. +Add ``__match_args__`` to :ref:`struct sequence objects `. +Patch by Pablo Galindo. .. From webhook-mailer at python.org Wed Jul 26 15:44:58 2023 From: webhook-mailer at python.org (barneygale) Date: Wed, 26 Jul 2023 19:44:58 -0000 Subject: [Python-checkins] gh-105002: [pathlib] Fix relative_to with walk_up=True using ".." (#107014) Message-ID: https://github.com/python/cpython/commit/e7e6e4b035f51ab4a962b45a957254859f264f4f commit: e7e6e4b035f51ab4a962b45a957254859f264f4f branch: main author: J?nos Kukovecz committer: barneygale date: 2023-07-26T20:44:55+01:00 summary: gh-105002: [pathlib] Fix relative_to with walk_up=True using ".." (#107014) It makes sense to raise an Error because ".." can not be resolved and the current working directory is unknown. files: A Misc/NEWS.d/next/Library/2023-07-22-12-53-53.gh-issue-105002.gkfsW0.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Lib/pathlib.py b/Lib/pathlib.py index 8ff4d4ea19168..c83cf3d2ef696 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -633,10 +633,12 @@ def relative_to(self, other, /, *_deprecated, walk_up=False): for step, path in enumerate([other] + list(other.parents)): if self.is_relative_to(path): break + elif not walk_up: + raise ValueError(f"{str(self)!r} is not in the subpath of {str(other)!r}") + elif path.name == '..': + raise ValueError(f"'..' segment in {str(other)!r} cannot be walked") else: raise ValueError(f"{str(self)!r} and {str(other)!r} have different anchors") - if step and not walk_up: - raise ValueError(f"{str(self)!r} is not in the subpath of {str(other)!r}") parts = ['..'] * step + self._tail[len(path._tail):] return self.with_segments(*parts) diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index 78948e3b72032..5789a932c5903 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -693,8 +693,14 @@ def test_relative_to_common(self): self.assertRaises(ValueError, p.relative_to, P('a/b/c')) self.assertRaises(ValueError, p.relative_to, P('a/c')) self.assertRaises(ValueError, p.relative_to, P('/a')) + self.assertRaises(ValueError, p.relative_to, P("../a")) + self.assertRaises(ValueError, p.relative_to, P("a/..")) + self.assertRaises(ValueError, p.relative_to, P("/a/..")) self.assertRaises(ValueError, p.relative_to, P('/'), walk_up=True) self.assertRaises(ValueError, p.relative_to, P('/a'), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("../a"), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("a/.."), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("/a/.."), walk_up=True) p = P('/a/b') self.assertEqual(p.relative_to(P('/')), P('a/b')) self.assertEqual(p.relative_to('/'), P('a/b')) @@ -723,8 +729,14 @@ def test_relative_to_common(self): self.assertRaises(ValueError, p.relative_to, P()) self.assertRaises(ValueError, p.relative_to, '') self.assertRaises(ValueError, p.relative_to, P('a')) + self.assertRaises(ValueError, p.relative_to, P("../a")) + self.assertRaises(ValueError, p.relative_to, P("a/..")) + self.assertRaises(ValueError, p.relative_to, P("/a/..")) self.assertRaises(ValueError, p.relative_to, P(''), walk_up=True) self.assertRaises(ValueError, p.relative_to, P('a'), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("../a"), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("a/.."), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("/a/.."), walk_up=True) def test_is_relative_to_common(self): P = self.cls diff --git a/Misc/NEWS.d/next/Library/2023-07-22-12-53-53.gh-issue-105002.gkfsW0.rst b/Misc/NEWS.d/next/Library/2023-07-22-12-53-53.gh-issue-105002.gkfsW0.rst new file mode 100644 index 0000000000000..b4c133a5cb124 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-12-53-53.gh-issue-105002.gkfsW0.rst @@ -0,0 +1,3 @@ +Fix invalid result from :meth:`PurePath.relative_to` method when attempting to walk +a "``..``" segment in *other* with *walk_up* enabled. A :exc:`ValueError` exception +is now raised in this case. From webhook-mailer at python.org Wed Jul 26 16:00:17 2023 From: webhook-mailer at python.org (brettcannon) Date: Wed, 26 Jul 2023 20:00:17 -0000 Subject: [Python-checkins] [3.11] Document that `os.link()` is not available on Emscripten (GH-104822) (GH-107309) Message-ID: https://github.com/python/cpython/commit/35aa6b80a6018ea12ea54dfb97da46b70c1cc81e commit: 35aa6b80a6018ea12ea54dfb97da46b70c1cc81e branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: brettcannon date: 2023-07-26T13:00:13-07:00 summary: [3.11] Document that `os.link()` is not available on Emscripten (GH-104822) (GH-107309) Document that `os.link()` is not available on Emscripten (GH-104822) (cherry picked from commit 737d1da0746053d515158eac5b115e8bd813f6d3) Co-authored-by: Roman Yurchak files: M Doc/library/os.rst diff --git a/Doc/library/os.rst b/Doc/library/os.rst index dcb4e7d3ab259..56e34a3df8761 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2036,7 +2036,7 @@ features: .. audit-event:: os.link src,dst,src_dir_fd,dst_dir_fd os.link - .. availability:: Unix, Windows. + .. availability:: Unix, Windows, not Emscripten. .. versionchanged:: 3.2 Added Windows support. From webhook-mailer at python.org Wed Jul 26 16:00:44 2023 From: webhook-mailer at python.org (brettcannon) Date: Wed, 26 Jul 2023 20:00:44 -0000 Subject: [Python-checkins] [3.12] Document that `os.link()` is not available on Emscripten (GH-104822) (GH-107308) Message-ID: https://github.com/python/cpython/commit/58af565c5085c427203ee424585a2c3ed0db4981 commit: 58af565c5085c427203ee424585a2c3ed0db4981 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: brettcannon date: 2023-07-26T13:00:40-07:00 summary: [3.12] Document that `os.link()` is not available on Emscripten (GH-104822) (GH-107308) Document that `os.link()` is not available on Emscripten (GH-104822) (cherry picked from commit 737d1da0746053d515158eac5b115e8bd813f6d3) Co-authored-by: Roman Yurchak files: M Doc/library/os.rst diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 55ebe7e669162..956eeffa1cd45 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2150,7 +2150,7 @@ features: .. audit-event:: os.link src,dst,src_dir_fd,dst_dir_fd os.link - .. availability:: Unix, Windows. + .. availability:: Unix, Windows, not Emscripten. .. versionchanged:: 3.2 Added Windows support. From webhook-mailer at python.org Wed Jul 26 16:11:19 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 26 Jul 2023 20:11:19 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: Complete `get_destination_buffer` annotations (#107293) Message-ID: https://github.com/python/cpython/commit/f9bcdf2368d25e6c56193c13d49de031ac2ae2e9 commit: f9bcdf2368d25e6c56193c13d49de031ac2ae2e9 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-26T21:11:15+01:00 summary: gh-104050: Argument clinic: Complete `get_destination_buffer` annotations (#107293) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index f557984f7642f..2d804c96f86c3 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2172,7 +2172,7 @@ def __init__( 'impl_definition': d('block'), } - DestBufferType = dict[str, Callable[..., Any]] + DestBufferType = dict[str, _TextAccumulator] DestBufferList = list[DestBufferType] self.destination_buffers_stack: DestBufferList = [] @@ -2226,7 +2226,7 @@ def get_destination_buffer( self, name: str, item: int = 0 - ): + ) -> _TextAccumulator: d = self.get_destination(name) return d.buffers[item] From webhook-mailer at python.org Wed Jul 26 16:12:22 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Wed, 26 Jul 2023 20:12:22 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: Annotate `str_converter_key()` (#107294) Message-ID: https://github.com/python/cpython/commit/5aa6964a5ca60dff5c5ab40b968d92b49bd13b3d commit: 5aa6964a5ca60dff5c5ab40b968d92b49bd13b3d branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-26T21:12:18+01:00 summary: gh-104050: Argument clinic: Annotate `str_converter_key()` (#107294) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 2d804c96f86c3..34cc4014b35b4 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -3624,10 +3624,14 @@ class buffer: pass class rwbuffer: pass class robuffer: pass -def str_converter_key(types, encoding, zeroes): +StrConverterKeyType = tuple[frozenset[type], bool, bool] + +def str_converter_key( + types: TypeSet, encoding: bool | str | None, zeroes: bool +) -> StrConverterKeyType: return (frozenset(types), bool(encoding), bool(zeroes)) -str_converter_argument_map: dict[str, str] = {} +str_converter_argument_map: dict[StrConverterKeyType, str] = {} class str_converter(CConverter): type = 'const char *' From webhook-mailer at python.org Wed Jul 26 16:17:35 2023 From: webhook-mailer at python.org (barneygale) Date: Wed, 26 Jul 2023 20:17:35 -0000 Subject: [Python-checkins] [3.12] gh-105002: [pathlib] Fix relative_to with walk_up=True using ".." (GH-107014) (#107315) Message-ID: https://github.com/python/cpython/commit/4f6d7a5890760434015d9e90a6e81bf9e9b5c52f commit: 4f6d7a5890760434015d9e90a6e81bf9e9b5c52f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: barneygale date: 2023-07-26T20:17:31Z summary: [3.12] gh-105002: [pathlib] Fix relative_to with walk_up=True using ".." (GH-107014) (#107315) gh-105002: [pathlib] Fix relative_to with walk_up=True using ".." (GH-107014) It makes sense to raise an Error because ".." can not be resolved and the current working directory is unknown. (cherry picked from commit e7e6e4b035f51ab4a962b45a957254859f264f4f) Co-authored-by: J?nos Kukovecz files: A Misc/NEWS.d/next/Library/2023-07-22-12-53-53.gh-issue-105002.gkfsW0.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Lib/pathlib.py b/Lib/pathlib.py index b99bf6e7dd240..65631b7039631 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -679,10 +679,12 @@ def relative_to(self, other, /, *_deprecated, walk_up=False): for step, path in enumerate([other] + list(other.parents)): if self.is_relative_to(path): break + elif not walk_up: + raise ValueError(f"{str(self)!r} is not in the subpath of {str(other)!r}") + elif path.name == '..': + raise ValueError(f"'..' segment in {str(other)!r} cannot be walked") else: raise ValueError(f"{str(self)!r} and {str(other)!r} have different anchors") - if step and not walk_up: - raise ValueError(f"{str(self)!r} is not in the subpath of {str(other)!r}") parts = ['..'] * step + self._tail[len(path._tail):] return self.with_segments(*parts) diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index 1d28a782f44ab..012dacf10ea80 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -626,8 +626,14 @@ def test_relative_to_common(self): self.assertRaises(ValueError, p.relative_to, P('a/b/c')) self.assertRaises(ValueError, p.relative_to, P('a/c')) self.assertRaises(ValueError, p.relative_to, P('/a')) + self.assertRaises(ValueError, p.relative_to, P("../a")) + self.assertRaises(ValueError, p.relative_to, P("a/..")) + self.assertRaises(ValueError, p.relative_to, P("/a/..")) self.assertRaises(ValueError, p.relative_to, P('/'), walk_up=True) self.assertRaises(ValueError, p.relative_to, P('/a'), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("../a"), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("a/.."), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("/a/.."), walk_up=True) p = P('/a/b') self.assertEqual(p.relative_to(P('/')), P('a/b')) self.assertEqual(p.relative_to('/'), P('a/b')) @@ -656,8 +662,14 @@ def test_relative_to_common(self): self.assertRaises(ValueError, p.relative_to, P()) self.assertRaises(ValueError, p.relative_to, '') self.assertRaises(ValueError, p.relative_to, P('a')) + self.assertRaises(ValueError, p.relative_to, P("../a")) + self.assertRaises(ValueError, p.relative_to, P("a/..")) + self.assertRaises(ValueError, p.relative_to, P("/a/..")) self.assertRaises(ValueError, p.relative_to, P(''), walk_up=True) self.assertRaises(ValueError, p.relative_to, P('a'), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("../a"), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("a/.."), walk_up=True) + self.assertRaises(ValueError, p.relative_to, P("/a/.."), walk_up=True) def test_is_relative_to_common(self): P = self.cls diff --git a/Misc/NEWS.d/next/Library/2023-07-22-12-53-53.gh-issue-105002.gkfsW0.rst b/Misc/NEWS.d/next/Library/2023-07-22-12-53-53.gh-issue-105002.gkfsW0.rst new file mode 100644 index 0000000000000..b4c133a5cb124 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-12-53-53.gh-issue-105002.gkfsW0.rst @@ -0,0 +1,3 @@ +Fix invalid result from :meth:`PurePath.relative_to` method when attempting to walk +a "``..``" segment in *other* with *walk_up* enabled. A :exc:`ValueError` exception +is now raised in this case. From webhook-mailer at python.org Wed Jul 26 16:54:29 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 26 Jul 2023 20:54:29 -0000 Subject: [Python-checkins] Docs: Remove the numbered steps from the Argument Clinic tutorial (#107203) Message-ID: https://github.com/python/cpython/commit/592395577c679543d899e68a3cff538b8b4df80d commit: 592395577c679543d899e68a3cff538b8b4df80d branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-26T22:54:25+02:00 summary: Docs: Remove the numbered steps from the Argument Clinic tutorial (#107203) Instead, order the tutorial as one body of prose, making it easier to align the tutorial according to Di?taxis principles. files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 98d3632ff0232..ea3b4537d4d53 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -167,23 +167,23 @@ convert a function to work with it. Here, then, are the bare minimum steps you'd need to follow to convert a function to work with Argument Clinic. Note that for code you plan to check in to CPython, you really should take the conversion farther, -using some of the advanced concepts you'll see later on in -the document (like "return converters" and "self converters"). +using some of the :ref:`advanced concepts ` +you'll see later on in the document, +like :ref:`clinic-howto-return-converters` +and :ref:`clinic-howto-self-converter`. But we'll keep it simple for this walkthrough so you can learn. -Let's dive in! +First, make sure you're working with a freshly updated checkout +of the CPython trunk. -0. Make sure you're working with a freshly updated checkout - of the CPython trunk. +Next, find a Python builtin that calls either :c:func:`PyArg_ParseTuple` +or :c:func:`PyArg_ParseTupleAndKeywords`, and hasn't been converted +to work with Argument Clinic yet. +For this tutorial, we'll be using +:py:meth:`_pickle.Pickler.dump `. -1. Find a Python builtin that calls either :c:func:`PyArg_ParseTuple` - or :c:func:`PyArg_ParseTupleAndKeywords`, and hasn't been converted - to work with Argument Clinic yet. - For my example I'm using - :py:meth:`_pickle.Pickler.dump `. - -2. If the call to the :c:func:`!PyArg_Parse*` function uses any of the - following format units: +If the call to the :c:func:`!PyArg_Parse*` function uses any of the +following format units...: .. code-block:: none @@ -194,388 +194,377 @@ Let's dive in! et et# - or if it has multiple calls to :c:func:`PyArg_ParseTuple`, - you should choose a different function. Argument Clinic *does* - support all of these scenarios. But these are advanced - topics?let's do something simpler for your first function. - - Also, if the function has multiple calls to :c:func:`!PyArg_ParseTuple` - or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different - types for the same argument, or if the function uses something besides - :c:func:`!PyArg_Parse*` functions to parse its arguments, it probably - isn't suitable for conversion to Argument Clinic. Argument Clinic - doesn't support generic functions or polymorphic parameters. - -3. Add the following boilerplate above the function, creating our block:: - - /*[clinic input] - [clinic start generated code]*/ - -4. Cut the docstring and paste it in between the ``[clinic]`` lines, - removing all the junk that makes it a properly quoted C string. - When you're done you should have just the text, based at the left - margin, with no line wider than 80 characters. - (Argument Clinic will preserve indents inside the docstring.) - - If the old docstring had a first line that looked like a function - signature, throw that line away. (The docstring doesn't need it - anymore?when you use :py:func:`help` on your builtin in the future, - the first line will be built automatically based on the function's - signature.) - - Sample:: +... or if it has multiple calls to :c:func:`PyArg_ParseTuple`, +you should choose a different function. +(See :ref:`clinic-howto-advanced-converters` for those scenarios.) - /*[clinic input] - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ - -5. If your docstring doesn't have a "summary" line, Argument Clinic will - complain. So let's make sure it has one. The "summary" line should - be a paragraph consisting of a single 80-column line - at the beginning of the docstring. - - (Our example docstring consists solely of a summary line, so the sample - code doesn't have to change for this step.) +Also, if the function has multiple calls to :c:func:`!PyArg_ParseTuple` +or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different +types for the same argument, or if the function uses something besides +:c:func:`!PyArg_Parse*` functions to parse its arguments, it probably +isn't suitable for conversion to Argument Clinic. Argument Clinic +doesn't support generic functions or polymorphic parameters. -6. Above the docstring, enter the name of the function, followed - by a blank line. This should be the Python name of the function, - and should be the full dotted path - to the function?it should start with the name of the module, - include any sub-modules, and if the function is a method on - a class it should include the class name too. - - Sample:: +Next, add the following boilerplate above the function, +creating our input block:: /*[clinic input] - _pickle.Pickler.dump - - Write a pickled representation of obj to the open file. [clinic start generated code]*/ -7. If this is the first time that module or class has been used with Argument - Clinic in this C file, - you must declare the module and/or class. Proper Argument Clinic hygiene - prefers declaring these in a separate block somewhere near the - top of the C file, in the same way that include files and statics go at - the top. (In our sample code we'll just show the two blocks next to - each other.) +Cut the docstring and paste it in between the ``[clinic]`` lines, +removing all the junk that makes it a properly quoted C string. +When you're done you should have just the text, based at the left +margin, with no line wider than 80 characters. +Argument Clinic will preserve indents inside the docstring. - The name of the class and module should be the same as the one - seen by Python. Check the name defined in the :c:type:`PyModuleDef` - or :c:type:`PyTypeObject` as appropriate. +If the old docstring had a first line that looked like a function +signature, throw that line away; The docstring doesn't need it anymore --- +when you use :py:func:`help` on your builtin in the future, +the first line will be built automatically based on the function's signature. - When you declare a class, you must also specify two aspects of its type - in C: the type declaration you'd use for a pointer to an instance of - this class, and a pointer to the :c:type:`!PyTypeObject` for this class. +Example docstring summary line:: - Sample:: - - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ - - /*[clinic input] - _pickle.Pickler.dump - - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ - - - - -8. Declare each of the parameters to the function. Each parameter - should get its own line. All the parameter lines should be - indented from the function name and the docstring. - - The general form of these parameter lines is as follows: - - .. code-block:: none - - name_of_parameter: converter - - If the parameter has a default value, add that after the - converter: - - .. code-block:: none + /*[clinic input] + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - name_of_parameter: converter = default_value +If your docstring doesn't have a "summary" line, Argument Clinic will +complain, so let's make sure it has one. The "summary" line should +be a paragraph consisting of a single 80-column line +at the beginning of the docstring. +(See :pep:`257` regarding docstring conventions.) - Argument Clinic's support for "default values" is quite sophisticated; - please see :ref:`the section below on default values ` - for more information. +Our example docstring consists solely of a summary line, so the sample +code doesn't have to change for this step. - Add a blank line below the parameters. +Now, above the docstring, enter the name of the function, followed +by a blank line. This should be the Python name of the function, +and should be the full dotted path to the function --- +it should start with the name of the module, +include any sub-modules, and if the function is a method on +a class it should include the class name too. - What's a "converter"? It establishes both the type - of the variable used in C, and the method to convert the Python - value into a C value at runtime. - For now you're going to use what's called a "legacy converter"?a - convenience syntax intended to make porting old code into Argument - Clinic easier. +In our example, :mod:`!_pickle` is the module, :py:class:`!Pickler` is the class, +and :py:meth:`!dump` is the method, so the name becomes +:py:meth:`!_pickle.Pickler.dump`:: - For each parameter, copy the "format unit" for that - parameter from the :c:func:`PyArg_Parse` format argument and - specify *that* as its converter, as a quoted - string. ("format unit" is the formal name for the one-to-three - character substring of the *format* parameter that tells - the argument parsing function what the type of the variable - is and how to convert it. For more on format units please - see :ref:`arg-parsing`.) + /*[clinic input] + _pickle.Pickler.dump - For multicharacter format units like ``z#``, use the - entire two-or-three character string. + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - Sample:: +If this is the first time that module or class has been used with Argument +Clinic in this C file, +you must declare the module and/or class. Proper Argument Clinic hygiene +prefers declaring these in a separate block somewhere near the +top of the C file, in the same way that include files and statics go at +the top. +In our sample code we'll just show the two blocks next to each other. - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ +The name of the class and module should be the same as the one +seen by Python. Check the name defined in the :c:type:`PyModuleDef` +or :c:type:`PyTypeObject` as appropriate. - /*[clinic input] - _pickle.Pickler.dump +When you declare a class, you must also specify two aspects of its type +in C: the type declaration you'd use for a pointer to an instance of +this class, and a pointer to the :c:type:`!PyTypeObject` for this class:: - obj: 'O' + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ + /*[clinic input] + _pickle.Pickler.dump -9. If your function has ``|`` in the format string, meaning some - parameters have default values, you can ignore it. Argument - Clinic infers which parameters are optional based on whether - or not they have default values. + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - If your function has ``$`` in the format string, meaning it - takes keyword-only arguments, specify ``*`` on a line by - itself before the first keyword-only argument, indented the - same as the parameter lines. +Declare each of the parameters to the function. Each parameter +should get its own line. All the parameter lines should be +indented from the function name and the docstring. +The general form of these parameter lines is as follows: - (:py:meth:`!_pickle.Pickler.dump` has neither, so our sample is unchanged.) +.. code-block:: none + name_of_parameter: converter -10. If the existing C function calls :c:func:`PyArg_ParseTuple` - (as opposed to :c:func:`PyArg_ParseTupleAndKeywords`), then all its - arguments are positional-only. +If the parameter has a default value, add that after the +converter: - To mark all parameters as positional-only in Argument Clinic, - add a ``/`` on a line by itself after the last parameter, - indented the same as the parameter lines. +.. code-block:: none - Currently this is all-or-nothing; either all parameters are - positional-only, or none of them are. (In the future Argument - Clinic may relax this restriction.) + name_of_parameter: converter = default_value - Sample:: +Argument Clinic's support for "default values" is quite sophisticated; +see :ref:`clinic-howto-default-values` for more information. - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ +Next, add a blank line below the parameters. - /*[clinic input] - _pickle.Pickler.dump +What's a "converter"? +It establishes both the type of the variable used in C, +and the method to convert the Python value into a C value at runtime. +For now you're going to use what's called a "legacy converter" --- +a convenience syntax intended to make porting old code into Argument +Clinic easier. - obj: 'O' - / +For each parameter, copy the "format unit" for that +parameter from the :c:func:`PyArg_Parse` format argument and +specify *that* as its converter, as a quoted string. +The "format unit" is the formal name for the one-to-three +character substring of the *format* parameter that tells +the argument parsing function what the type of the variable +is and how to convert it. +For more on format units please see :ref:`arg-parsing`. - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +For multicharacter format units like ``z#``, +use the entire two-or-three character string. -11. It's helpful to write a per-parameter docstring for each parameter. - But per-parameter docstrings are optional; you can skip this step - if you prefer. +Sample:: - Here's how to add a per-parameter docstring. The first line - of the per-parameter docstring must be indented further than the - parameter definition. The left margin of this first line establishes - the left margin for the whole per-parameter docstring; all the text - you write will be outdented by this amount. You can write as much - text as you like, across multiple lines if you wish. + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - Sample:: + /*[clinic input] + _pickle.Pickler.dump - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ + obj: 'O' - /*[clinic input] - _pickle.Pickler.dump + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - obj: 'O' - The object to be pickled. - / +If your function has ``|`` in the format string, +meaning some parameters have default values, you can ignore it. +Argument Clinic infers which parameters are optional +based on whether or not they have default values. - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +If your function has ``$`` in the format string, +meaning it takes keyword-only arguments, +specify ``*`` on a line by itself before the first keyword-only argument, +indented the same as the parameter lines. -12. Save and close the file, then run ``Tools/clinic/clinic.py`` on - it. With luck everything worked---your block now has output, and - a :file:`.c.h` file has been generated! Reopen the file in your - text editor to see:: +:py:meth:`!_pickle.Pickler.dump` has neither, so our sample is unchanged. - /*[clinic input] - _pickle.Pickler.dump +Next, if the existing C function calls :c:func:`PyArg_ParseTuple` +(as opposed to :c:func:`PyArg_ParseTupleAndKeywords`), then all its +arguments are positional-only. - obj: 'O' - The object to be pickled. - / +To mark parameters as positional-only in Argument Clinic, +add a ``/`` on a line by itself after the last positional-only parameter, +indented the same as the parameter lines. - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +Sample:: - static PyObject * - _pickle_Pickler_dump(PicklerObject *self, PyObject *obj) - /*[clinic end generated code: output=87ecad1261e02ac7 input=552eb1c0f52260d9]*/ + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - Obviously, if Argument Clinic didn't produce any output, it's because - it found an error in your input. Keep fixing your errors and retrying - until Argument Clinic processes your file without complaint. + /*[clinic input] + _pickle.Pickler.dump - For readability, most of the glue code has been generated to a :file:`.c.h` - file. You'll need to include that in your original :file:`.c` file, - typically right after the clinic module block:: + obj: 'O' + / - #include "clinic/_pickle.c.h" + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ -13. Double-check that the argument-parsing code Argument Clinic generated - looks basically the same as the existing code. +It can be helpful to write a per-parameter docstring for each parameter. +Since per-parameter docstrings are optional, +you can skip this step if you prefer. - First, ensure both places use the same argument-parsing function. - The existing code must call either - :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords`; - ensure that the code generated by Argument Clinic calls the - *exact* same function. +Nevertheless, here's how to add a per-parameter docstring. +The first line of the per-parameter docstring +must be indented further than the parameter definition. +The left margin of this first line establishes +the left margin for the whole per-parameter docstring; +all the text you write will be outdented by this amount. +You can write as much text as you like, across multiple lines if you wish. - Second, the format string passed in to :c:func:`!PyArg_ParseTuple` or - :c:func:`!PyArg_ParseTupleAndKeywords` should be *exactly* the same - as the hand-written one in the existing function, up to the colon - or semi-colon. +Sample:: - (Argument Clinic always generates its format strings - with a ``:`` followed by the name of the function. If the - existing code's format string ends with ``;``, to provide - usage help, this change is harmless?don't worry about it.) + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - Third, for parameters whose format units require two arguments - (like a length variable, or an encoding string, or a pointer - to a conversion function), ensure that the second argument is - *exactly* the same between the two invocations. + /*[clinic input] + _pickle.Pickler.dump - Fourth, inside the output portion of the block you'll find a preprocessor - macro defining the appropriate static :c:type:`PyMethodDef` structure for - this builtin:: + obj: 'O' + The object to be pickled. + / - #define __PICKLE_PICKLER_DUMP_METHODDEF \ - {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - This static structure should be *exactly* the same as the existing static - :c:type:`!PyMethodDef` structure for this builtin. +Save and close the file, then run ``Tools/clinic/clinic.py`` on it. +With luck everything worked---your block now has output, +and a :file:`.c.h` file has been generated! +Reload the file in your text editor to see the generated code:: - If any of these items differ in *any way*, - adjust your Argument Clinic function specification and rerun - ``Tools/clinic/clinic.py`` until they *are* the same. + /*[clinic input] + _pickle.Pickler.dump + obj: 'O' + The object to be pickled. + / -14. Notice that the last line of its output is the declaration - of your "impl" function. This is where the builtin's implementation goes. - Delete the existing prototype of the function you're modifying, but leave - the opening curly brace. Now delete its argument parsing code and the - declarations of all the variables it dumps the arguments into. - Notice how the Python arguments are now arguments to this impl function; - if the implementation used different names for these variables, fix it. + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - Let's reiterate, just because it's kind of weird. Your code should now - look like this:: + static PyObject * + _pickle_Pickler_dump(PicklerObject *self, PyObject *obj) + /*[clinic end generated code: output=87ecad1261e02ac7 input=552eb1c0f52260d9]*/ - static return_type - your_function_impl(...) - /*[clinic end generated code: checksum=...]*/ - { - ... +Obviously, if Argument Clinic didn't produce any output, +it's because it found an error in your input. +Keep fixing your errors and retrying until Argument Clinic processes your file +without complaint. - Argument Clinic generated the checksum line and the function prototype just - above it. You should write the opening (and closing) curly braces for the - function, and the implementation inside. +For readability, most of the glue code has been generated to a :file:`.c.h` +file. You'll need to include that in your original :file:`.c` file, +typically right after the clinic module block:: - Sample:: + #include "clinic/_pickle.c.h" - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ - /*[clinic end generated code: checksum=da39a3ee5e6b4b0d3255bfef95601890afd80709]*/ +Double-check that the argument-parsing code Argument Clinic generated +looks basically the same as the existing code. - /*[clinic input] - _pickle.Pickler.dump +First, ensure both places use the same argument-parsing function. +The existing code must call either +:c:func:`PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords`; +ensure that the code generated by Argument Clinic calls the +*exact* same function. - obj: 'O' - The object to be pickled. - / +Second, the format string passed in to :c:func:`!PyArg_ParseTuple` or +:c:func:`!PyArg_ParseTupleAndKeywords` should be *exactly* the same +as the hand-written one in the existing function, +up to the colon or semi-colon. - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +Argument Clinic always generates its format strings +with a ``:`` followed by the name of the function. +If the existing code's format string ends with ``;``, +to provide usage help, this change is harmless --- don't worry about it. - PyDoc_STRVAR(__pickle_Pickler_dump__doc__, - "Write a pickled representation of obj to the open file.\n" - "\n" - ... - static PyObject * - _pickle_Pickler_dump_impl(PicklerObject *self, PyObject *obj) - /*[clinic end generated code: checksum=3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/ - { - /* Check whether the Pickler was initialized correctly (issue3664). - Developers often forget to call __init__() in their subclasses, which - would trigger a segfault without this check. */ - if (self->write == NULL) { - PyErr_Format(PicklingError, - "Pickler.__init__() was not called by %s.__init__()", - Py_TYPE(self)->tp_name); - return NULL; - } +Third, for parameters whose format units require two arguments, +like a length variable, an encoding string, or a pointer +to a conversion function, ensure that the second argument is +*exactly* the same between the two invocations. - if (_Pickler_ClearBuffer(self) < 0) - return NULL; +Fourth, inside the output portion of the block, +you'll find a preprocessor macro defining the appropriate static +:c:type:`PyMethodDef` structure for this builtin:: - ... + #define __PICKLE_PICKLER_DUMP_METHODDEF \ + {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, -15. Remember the macro with the :c:type:`PyMethodDef` structure for this - function? Find the existing :c:type:`!PyMethodDef` structure for this - function and replace it with a reference to the macro. (If the builtin - is at module scope, this will probably be very near the end of the file; - if the builtin is a class method, this will probably be below but relatively - near to the implementation.) +This static structure should be *exactly* the same as the existing static +:c:type:`!PyMethodDef` structure for this builtin. - Note that the body of the macro contains a trailing comma. So when you - replace the existing static :c:type:`!PyMethodDef` structure with the macro, - *don't* add a comma to the end. +If any of these items differ in *any way*, +adjust your Argument Clinic function specification and rerun +``Tools/clinic/clinic.py`` until they *are* the same. - Sample:: +Notice that the last line of its output is the declaration +of your "impl" function. This is where the builtin's implementation goes. +Delete the existing prototype of the function you're modifying, but leave +the opening curly brace. Now delete its argument parsing code and the +declarations of all the variables it dumps the arguments into. +Notice how the Python arguments are now arguments to this impl function; +if the implementation used different names for these variables, fix it. - static struct PyMethodDef Pickler_methods[] = { - __PICKLE_PICKLER_DUMP_METHODDEF - __PICKLE_PICKLER_CLEAR_MEMO_METHODDEF - {NULL, NULL} /* sentinel */ - }; +Let's reiterate, just because it's kind of weird. +Your code should now look like this:: + static return_type + your_function_impl(...) + /*[clinic end generated code: input=..., output=...]*/ + { + ... -16. Argument Clinic may generate new instances of ``_Py_ID``. For example:: +Argument Clinic generated the checksum line and the function prototype just +above it. You should write the opening and closing curly braces for the +function, and the implementation inside. - &_Py_ID(new_unique_py_id) +Sample:: - If it does, you'll have to run ``make regen-global-objects`` - to regenerate the list of precompiled identifiers at this point. + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ + /*[clinic end generated code: checksum=da39a3ee5e6b4b0d3255bfef95601890afd80709]*/ + /*[clinic input] + _pickle.Pickler.dump -17. Compile, then run the relevant portions of the regression-test suite. - This change should not introduce any new compile-time warnings or errors, - and there should be no externally visible change to Python's behavior. + obj: 'O' + The object to be pickled. + / - Well, except for one difference: :py:func:`inspect.signature` run on your function - should now provide a valid signature! + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - Congratulations, you've ported your first function to work with Argument Clinic! + PyDoc_STRVAR(__pickle_Pickler_dump__doc__, + "Write a pickled representation of obj to the open file.\n" + "\n" + ... + static PyObject * + _pickle_Pickler_dump_impl(PicklerObject *self, PyObject *obj) + /*[clinic end generated code: checksum=3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/ + { + /* Check whether the Pickler was initialized correctly (issue3664). + Developers often forget to call __init__() in their subclasses, which + would trigger a segfault without this check. */ + if (self->write == NULL) { + PyErr_Format(PicklingError, + "Pickler.__init__() was not called by %s.__init__()", + Py_TYPE(self)->tp_name); + return NULL; + } + + if (_Pickler_ClearBuffer(self) < 0) { + return NULL; + } + + ... + +Remember the macro with the :c:type:`PyMethodDef` structure for this function? +Find the existing :c:type:`!PyMethodDef` structure for this +function and replace it with a reference to the macro. If the builtin +is at module scope, this will probably be very near the end of the file; +if the builtin is a class method, this will probably be below but relatively +near to the implementation. + +Note that the body of the macro contains a trailing comma; when you +replace the existing static :c:type:`!PyMethodDef` structure with the macro, +*don't* add a comma to the end. + +Sample:: + + static struct PyMethodDef Pickler_methods[] = { + __PICKLE_PICKLER_DUMP_METHODDEF + __PICKLE_PICKLER_CLEAR_MEMO_METHODDEF + {NULL, NULL} /* sentinel */ + }; + +Argument Clinic may generate new instances of ``_Py_ID``. For example:: + + &_Py_ID(new_unique_py_id) + +If it does, you'll have to run ``make regen-global-objects`` +to regenerate the list of precompiled identifiers at this point. + +Finally, compile, then run the relevant portions of the regression-test suite. +This change should not introduce any new compile-time warnings or errors, +and there should be no externally visible change to Python's behavior, +except for one difference: :py:func:`inspect.signature` run on your function +should now provide a valid signature! + +Congratulations, you've ported your first function to work with Argument Clinic! .. _clinic-howtos: @@ -913,6 +902,8 @@ you *must* not call :c:func:`PyBuffer_Release` on the provided buffer. Argument Clinic generates code that does it for you (in the parsing function). +.. _clinic-howto-advanced-converters: + How to use advanced converters ------------------------------ @@ -943,6 +934,7 @@ This restriction doesn't seem unreasonable; CPython itself always passes in stat hard-coded encoding strings for parameters whose format units start with ``e``. +.. _clinic-howto-default-values: .. _default_values: How to assign default values to parameter @@ -1053,6 +1045,8 @@ you're not permitted to use: * Tuple/list/set/dict literals. +.. _clinic-howto-return-converters: + How to use return converters ---------------------------- @@ -1195,6 +1189,8 @@ variable to the C code:: /*[python checksum:...]*/ +.. _clinic-howto-self-converter: + How to use the "self converter" ------------------------------- From webhook-mailer at python.org Wed Jul 26 17:00:40 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Wed, 26 Jul 2023 21:00:40 -0000 Subject: [Python-checkins] [3.11] gh-107091: Fix some uses of :c:type: role (GH-107138) (GH-107313) Message-ID: https://github.com/python/cpython/commit/603c5d50e89982e9b72a1b4d9edd5b52982b395a commit: 603c5d50e89982e9b72a1b4d9edd5b52982b395a branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-27T00:00:36+03:00 summary: [3.11] gh-107091: Fix some uses of :c:type: role (GH-107138) (GH-107313) (cherry picked from commit 6d5b6e71c87fca7c5c26f5dd8f325087962215cc) Co-authored-by: Serhiy Storchaka files: M Doc/c-api/tuple.rst M Doc/c-api/typeobj.rst M Doc/conf.py M Doc/library/os.rst M Doc/whatsnew/2.4.rst M Doc/whatsnew/3.2.rst M Doc/whatsnew/3.3.rst M Doc/whatsnew/3.7.rst M Misc/NEWS.d/3.10.0a7.rst diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index 0982d29e48dec..689fad683aed6 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -111,6 +111,8 @@ Tuple Objects raises :exc:`MemoryError` or :exc:`SystemError`. +.. _struct-sequence-objects: + Struct Sequence Objects ----------------------- diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 006e11b2e7235..002603857394c 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -161,9 +161,9 @@ Quick Reference .. [#cols] Columns: - **"O"**: set on :c:type:`PyBaseObject_Type` + **"O"**: set on :c:data:`PyBaseObject_Type` - **"T"**: set on :c:type:`PyType_Type` + **"T"**: set on :c:data:`PyType_Type` **"D"**: default (if slot is set to ``NULL``) @@ -567,8 +567,8 @@ PyTypeObject Slots Each slot has a section describing inheritance. If :c:func:`PyType_Ready` may set a value when the field is set to ``NULL`` then there will also be -a "Default" section. (Note that many fields set on :c:type:`PyBaseObject_Type` -and :c:type:`PyType_Type` effectively act as defaults.) +a "Default" section. (Note that many fields set on :c:data:`PyBaseObject_Type` +and :c:data:`PyType_Type` effectively act as defaults.) .. c:member:: const char* PyTypeObject.tp_name @@ -962,7 +962,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` uses :c:func:`PyObject_GenericGetAttr`. + :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_GenericGetAttr`. .. c:member:: setattrofunc PyTypeObject.tp_setattro @@ -988,7 +988,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` uses :c:func:`PyObject_GenericSetAttr`. + :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_GenericSetAttr`. .. c:member:: PyBufferProcs* PyTypeObject.tp_as_buffer @@ -1030,7 +1030,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` uses + :c:data:`PyBaseObject_Type` uses ``Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE``. **Bit Masks:** @@ -1492,7 +1492,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) **Default:** - :c:type:`PyBaseObject_Type` provides a :attr:`tp_richcompare` + :c:data:`PyBaseObject_Type` provides a :attr:`tp_richcompare` implementation, which may be inherited. However, if only :attr:`tp_hash` is defined, not even the inherited function is used and instances of the type will not be able to participate in any @@ -1814,7 +1814,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:func:`PyType_GenericAlloc`, to force a standard heap allocation strategy. - For static subtypes, :c:type:`PyBaseObject_Type` uses + For static subtypes, :c:data:`PyBaseObject_Type` uses :c:func:`PyType_GenericAlloc`. That is the recommended value for all statically defined types. @@ -1877,7 +1877,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) match :c:func:`PyType_GenericAlloc` and the value of the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit. - For static subtypes, :c:type:`PyBaseObject_Type` uses PyObject_Del. + For static subtypes, :c:data:`PyBaseObject_Type` uses :c:func:`PyObject_Del`. .. c:member:: inquiry PyTypeObject.tp_is_gc diff --git a/Doc/conf.py b/Doc/conf.py index d5eea9cdfb88d..c0e20c801f183 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -103,6 +103,10 @@ ('c:type', 'uintmax_t'), ('c:type', 'va_list'), ('c:type', 'wchar_t'), + ('c:struct', 'in6_addr'), + ('c:struct', 'in_addr'), + ('c:struct', 'stat'), + ('c:struct', 'statvfs'), # Standard C macros ('c:macro', 'LLONG_MAX'), ('c:macro', 'LLONG_MIN'), diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 56e34a3df8761..bb9cbae2b1345 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2243,13 +2243,13 @@ features: .. function:: major(device, /) Extract the device major number from a raw device number (usually the - :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`). + :attr:`st_dev` or :attr:`st_rdev` field from :c:struct:`stat`). .. function:: minor(device, /) Extract the device minor number from a raw device number (usually the - :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`). + :attr:`st_dev` or :attr:`st_rdev` field from :c:struct:`stat`). .. function:: makedev(major, minor, /) @@ -2743,7 +2743,7 @@ features: .. class:: stat_result Object whose attributes correspond roughly to the members of the - :c:type:`stat` structure. It is used for the result of :func:`os.stat`, + :c:struct:`stat` structure. It is used for the result of :func:`os.stat`, :func:`os.fstat` and :func:`os.lstat`. Attributes: @@ -2906,12 +2906,12 @@ features: See the ``IO_REPARSE_TAG_*`` constants in the :mod:`stat` module. The standard module :mod:`stat` defines functions and constants that are - useful for extracting information from a :c:type:`stat` structure. (On + useful for extracting information from a :c:struct:`stat` structure. (On Windows, some items are filled with dummy values.) For backward compatibility, a :class:`stat_result` instance is also accessible as a tuple of at least 10 integers giving the most important (and - portable) members of the :c:type:`stat` structure, in the order + portable) members of the :c:struct:`stat` structure, in the order :attr:`st_mode`, :attr:`st_ino`, :attr:`st_dev`, :attr:`st_nlink`, :attr:`st_uid`, :attr:`st_gid`, :attr:`st_size`, :attr:`st_atime`, :attr:`st_mtime`, :attr:`st_ctime`. More items may be added at the end by @@ -2944,7 +2944,7 @@ features: Perform a :c:func:`statvfs` system call on the given path. The return value is an object whose attributes describe the filesystem on the given path, and - correspond to the members of the :c:type:`statvfs` structure, namely: + correspond to the members of the :c:struct:`statvfs` structure, namely: :attr:`f_bsize`, :attr:`f_frsize`, :attr:`f_blocks`, :attr:`f_bfree`, :attr:`f_bavail`, :attr:`f_files`, :attr:`f_ffree`, :attr:`f_favail`, :attr:`f_flag`, :attr:`f_namemax`, :attr:`f_fsid`. diff --git a/Doc/whatsnew/2.4.rst b/Doc/whatsnew/2.4.rst index d68f600ba44f6..a00c4360b9c68 100644 --- a/Doc/whatsnew/2.4.rst +++ b/Doc/whatsnew/2.4.rst @@ -1491,7 +1491,7 @@ Some of the changes to Python's build process and to the C API are: though that processor architecture doesn't call that register "the TSC register". (Contributed by Jeremy Hylton.) -* The :c:type:`tracebackobject` type has been renamed to +* The :c:type:`!tracebackobject` type has been renamed to :c:type:`PyTracebackObject`. .. ====================================================================== diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index 2c176926e55e1..01f5cf5204bcf 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -564,7 +564,7 @@ Some smaller changes made to the core Python language are: (See :issue:`4617`.) -* The internal :c:type:`structsequence` tool now creates subclasses of tuple. +* :ref:`Struct sequence types ` are now subclasses of tuple. This means that C structures like those returned by :func:`os.stat`, :func:`time.gmtime`, and :data:`sys.version_info` now work like a :term:`named tuple` and now work with functions and methods that diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index e3e49640f6c91..1fda66cf9f0ea 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -2194,7 +2194,7 @@ Changes to Python's build process and to the C API include: * :c:func:`PyUnicode_AsUCS4`, :c:func:`PyUnicode_AsUCS4Copy` * :c:macro:`PyUnicode_DATA`, :c:macro:`PyUnicode_1BYTE_DATA`, :c:macro:`PyUnicode_2BYTE_DATA`, :c:macro:`PyUnicode_4BYTE_DATA` - * :c:macro:`PyUnicode_KIND` with :c:type:`PyUnicode_Kind` enum: + * :c:macro:`PyUnicode_KIND` with :c:enum:`PyUnicode_Kind` enum: :c:data:`PyUnicode_WCHAR_KIND`, :c:data:`PyUnicode_1BYTE_KIND`, :c:data:`PyUnicode_2BYTE_KIND`, :c:data:`PyUnicode_4BYTE_KIND` * :c:macro:`PyUnicode_READ`, :c:macro:`PyUnicode_READ_CHAR`, :c:macro:`PyUnicode_WRITE` diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 02406691d4ac7..a3fd6321f3124 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -1677,7 +1677,7 @@ module imports. The fields :c:member:`!name` and :c:member:`!doc` of structures :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, -and :c:type:`wrapperbase` are now of type ``const char *`` rather of +and :c:struct:`wrapperbase` are now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` diff --git a/Misc/NEWS.d/3.10.0a7.rst b/Misc/NEWS.d/3.10.0a7.rst index ff01ee645c0a9..7933f71b01c14 100644 --- a/Misc/NEWS.d/3.10.0a7.rst +++ b/Misc/NEWS.d/3.10.0a7.rst @@ -215,8 +215,8 @@ a non-Python signal handler. .. nonce: VouZjn .. section: Core and Builtins -Add ``__match_args__`` to :c:type:`structsequence` based classes. Patch by -Pablo Galindo. +Add ``__match_args__`` to :ref:`struct sequence objects `. +Patch by Pablo Galindo. .. From webhook-mailer at python.org Wed Jul 26 17:03:39 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 26 Jul 2023 21:03:39 -0000 Subject: [Python-checkins] [3.12] Docs: Remove the numbered steps from the Argument Clinic tutorial (GH-107203) (#107317) Message-ID: https://github.com/python/cpython/commit/6bbcd792f74148273bf14ab2bad87f190d80b076 commit: 6bbcd792f74148273bf14ab2bad87f190d80b076 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-26T21:03:35Z summary: [3.12] Docs: Remove the numbered steps from the Argument Clinic tutorial (GH-107203) (#107317) Instead, order the tutorial as one body of prose, making it easier to align the tutorial according to Di?taxis principles. (cherry picked from commit 592395577c679543d899e68a3cff538b8b4df80d) Co-authored-by: Erlend E. Aasland files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 98d3632ff0232..ea3b4537d4d53 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -167,23 +167,23 @@ convert a function to work with it. Here, then, are the bare minimum steps you'd need to follow to convert a function to work with Argument Clinic. Note that for code you plan to check in to CPython, you really should take the conversion farther, -using some of the advanced concepts you'll see later on in -the document (like "return converters" and "self converters"). +using some of the :ref:`advanced concepts ` +you'll see later on in the document, +like :ref:`clinic-howto-return-converters` +and :ref:`clinic-howto-self-converter`. But we'll keep it simple for this walkthrough so you can learn. -Let's dive in! +First, make sure you're working with a freshly updated checkout +of the CPython trunk. -0. Make sure you're working with a freshly updated checkout - of the CPython trunk. +Next, find a Python builtin that calls either :c:func:`PyArg_ParseTuple` +or :c:func:`PyArg_ParseTupleAndKeywords`, and hasn't been converted +to work with Argument Clinic yet. +For this tutorial, we'll be using +:py:meth:`_pickle.Pickler.dump `. -1. Find a Python builtin that calls either :c:func:`PyArg_ParseTuple` - or :c:func:`PyArg_ParseTupleAndKeywords`, and hasn't been converted - to work with Argument Clinic yet. - For my example I'm using - :py:meth:`_pickle.Pickler.dump `. - -2. If the call to the :c:func:`!PyArg_Parse*` function uses any of the - following format units: +If the call to the :c:func:`!PyArg_Parse*` function uses any of the +following format units...: .. code-block:: none @@ -194,388 +194,377 @@ Let's dive in! et et# - or if it has multiple calls to :c:func:`PyArg_ParseTuple`, - you should choose a different function. Argument Clinic *does* - support all of these scenarios. But these are advanced - topics?let's do something simpler for your first function. - - Also, if the function has multiple calls to :c:func:`!PyArg_ParseTuple` - or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different - types for the same argument, or if the function uses something besides - :c:func:`!PyArg_Parse*` functions to parse its arguments, it probably - isn't suitable for conversion to Argument Clinic. Argument Clinic - doesn't support generic functions or polymorphic parameters. - -3. Add the following boilerplate above the function, creating our block:: - - /*[clinic input] - [clinic start generated code]*/ - -4. Cut the docstring and paste it in between the ``[clinic]`` lines, - removing all the junk that makes it a properly quoted C string. - When you're done you should have just the text, based at the left - margin, with no line wider than 80 characters. - (Argument Clinic will preserve indents inside the docstring.) - - If the old docstring had a first line that looked like a function - signature, throw that line away. (The docstring doesn't need it - anymore?when you use :py:func:`help` on your builtin in the future, - the first line will be built automatically based on the function's - signature.) - - Sample:: +... or if it has multiple calls to :c:func:`PyArg_ParseTuple`, +you should choose a different function. +(See :ref:`clinic-howto-advanced-converters` for those scenarios.) - /*[clinic input] - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ - -5. If your docstring doesn't have a "summary" line, Argument Clinic will - complain. So let's make sure it has one. The "summary" line should - be a paragraph consisting of a single 80-column line - at the beginning of the docstring. - - (Our example docstring consists solely of a summary line, so the sample - code doesn't have to change for this step.) +Also, if the function has multiple calls to :c:func:`!PyArg_ParseTuple` +or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different +types for the same argument, or if the function uses something besides +:c:func:`!PyArg_Parse*` functions to parse its arguments, it probably +isn't suitable for conversion to Argument Clinic. Argument Clinic +doesn't support generic functions or polymorphic parameters. -6. Above the docstring, enter the name of the function, followed - by a blank line. This should be the Python name of the function, - and should be the full dotted path - to the function?it should start with the name of the module, - include any sub-modules, and if the function is a method on - a class it should include the class name too. - - Sample:: +Next, add the following boilerplate above the function, +creating our input block:: /*[clinic input] - _pickle.Pickler.dump - - Write a pickled representation of obj to the open file. [clinic start generated code]*/ -7. If this is the first time that module or class has been used with Argument - Clinic in this C file, - you must declare the module and/or class. Proper Argument Clinic hygiene - prefers declaring these in a separate block somewhere near the - top of the C file, in the same way that include files and statics go at - the top. (In our sample code we'll just show the two blocks next to - each other.) +Cut the docstring and paste it in between the ``[clinic]`` lines, +removing all the junk that makes it a properly quoted C string. +When you're done you should have just the text, based at the left +margin, with no line wider than 80 characters. +Argument Clinic will preserve indents inside the docstring. - The name of the class and module should be the same as the one - seen by Python. Check the name defined in the :c:type:`PyModuleDef` - or :c:type:`PyTypeObject` as appropriate. +If the old docstring had a first line that looked like a function +signature, throw that line away; The docstring doesn't need it anymore --- +when you use :py:func:`help` on your builtin in the future, +the first line will be built automatically based on the function's signature. - When you declare a class, you must also specify two aspects of its type - in C: the type declaration you'd use for a pointer to an instance of - this class, and a pointer to the :c:type:`!PyTypeObject` for this class. +Example docstring summary line:: - Sample:: - - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ - - /*[clinic input] - _pickle.Pickler.dump - - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ - - - - -8. Declare each of the parameters to the function. Each parameter - should get its own line. All the parameter lines should be - indented from the function name and the docstring. - - The general form of these parameter lines is as follows: - - .. code-block:: none - - name_of_parameter: converter - - If the parameter has a default value, add that after the - converter: - - .. code-block:: none + /*[clinic input] + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - name_of_parameter: converter = default_value +If your docstring doesn't have a "summary" line, Argument Clinic will +complain, so let's make sure it has one. The "summary" line should +be a paragraph consisting of a single 80-column line +at the beginning of the docstring. +(See :pep:`257` regarding docstring conventions.) - Argument Clinic's support for "default values" is quite sophisticated; - please see :ref:`the section below on default values ` - for more information. +Our example docstring consists solely of a summary line, so the sample +code doesn't have to change for this step. - Add a blank line below the parameters. +Now, above the docstring, enter the name of the function, followed +by a blank line. This should be the Python name of the function, +and should be the full dotted path to the function --- +it should start with the name of the module, +include any sub-modules, and if the function is a method on +a class it should include the class name too. - What's a "converter"? It establishes both the type - of the variable used in C, and the method to convert the Python - value into a C value at runtime. - For now you're going to use what's called a "legacy converter"?a - convenience syntax intended to make porting old code into Argument - Clinic easier. +In our example, :mod:`!_pickle` is the module, :py:class:`!Pickler` is the class, +and :py:meth:`!dump` is the method, so the name becomes +:py:meth:`!_pickle.Pickler.dump`:: - For each parameter, copy the "format unit" for that - parameter from the :c:func:`PyArg_Parse` format argument and - specify *that* as its converter, as a quoted - string. ("format unit" is the formal name for the one-to-three - character substring of the *format* parameter that tells - the argument parsing function what the type of the variable - is and how to convert it. For more on format units please - see :ref:`arg-parsing`.) + /*[clinic input] + _pickle.Pickler.dump - For multicharacter format units like ``z#``, use the - entire two-or-three character string. + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - Sample:: +If this is the first time that module or class has been used with Argument +Clinic in this C file, +you must declare the module and/or class. Proper Argument Clinic hygiene +prefers declaring these in a separate block somewhere near the +top of the C file, in the same way that include files and statics go at +the top. +In our sample code we'll just show the two blocks next to each other. - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ +The name of the class and module should be the same as the one +seen by Python. Check the name defined in the :c:type:`PyModuleDef` +or :c:type:`PyTypeObject` as appropriate. - /*[clinic input] - _pickle.Pickler.dump +When you declare a class, you must also specify two aspects of its type +in C: the type declaration you'd use for a pointer to an instance of +this class, and a pointer to the :c:type:`!PyTypeObject` for this class:: - obj: 'O' + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ + /*[clinic input] + _pickle.Pickler.dump -9. If your function has ``|`` in the format string, meaning some - parameters have default values, you can ignore it. Argument - Clinic infers which parameters are optional based on whether - or not they have default values. + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - If your function has ``$`` in the format string, meaning it - takes keyword-only arguments, specify ``*`` on a line by - itself before the first keyword-only argument, indented the - same as the parameter lines. +Declare each of the parameters to the function. Each parameter +should get its own line. All the parameter lines should be +indented from the function name and the docstring. +The general form of these parameter lines is as follows: - (:py:meth:`!_pickle.Pickler.dump` has neither, so our sample is unchanged.) +.. code-block:: none + name_of_parameter: converter -10. If the existing C function calls :c:func:`PyArg_ParseTuple` - (as opposed to :c:func:`PyArg_ParseTupleAndKeywords`), then all its - arguments are positional-only. +If the parameter has a default value, add that after the +converter: - To mark all parameters as positional-only in Argument Clinic, - add a ``/`` on a line by itself after the last parameter, - indented the same as the parameter lines. +.. code-block:: none - Currently this is all-or-nothing; either all parameters are - positional-only, or none of them are. (In the future Argument - Clinic may relax this restriction.) + name_of_parameter: converter = default_value - Sample:: +Argument Clinic's support for "default values" is quite sophisticated; +see :ref:`clinic-howto-default-values` for more information. - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ +Next, add a blank line below the parameters. - /*[clinic input] - _pickle.Pickler.dump +What's a "converter"? +It establishes both the type of the variable used in C, +and the method to convert the Python value into a C value at runtime. +For now you're going to use what's called a "legacy converter" --- +a convenience syntax intended to make porting old code into Argument +Clinic easier. - obj: 'O' - / +For each parameter, copy the "format unit" for that +parameter from the :c:func:`PyArg_Parse` format argument and +specify *that* as its converter, as a quoted string. +The "format unit" is the formal name for the one-to-three +character substring of the *format* parameter that tells +the argument parsing function what the type of the variable +is and how to convert it. +For more on format units please see :ref:`arg-parsing`. - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +For multicharacter format units like ``z#``, +use the entire two-or-three character string. -11. It's helpful to write a per-parameter docstring for each parameter. - But per-parameter docstrings are optional; you can skip this step - if you prefer. +Sample:: - Here's how to add a per-parameter docstring. The first line - of the per-parameter docstring must be indented further than the - parameter definition. The left margin of this first line establishes - the left margin for the whole per-parameter docstring; all the text - you write will be outdented by this amount. You can write as much - text as you like, across multiple lines if you wish. + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - Sample:: + /*[clinic input] + _pickle.Pickler.dump - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ + obj: 'O' - /*[clinic input] - _pickle.Pickler.dump + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - obj: 'O' - The object to be pickled. - / +If your function has ``|`` in the format string, +meaning some parameters have default values, you can ignore it. +Argument Clinic infers which parameters are optional +based on whether or not they have default values. - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +If your function has ``$`` in the format string, +meaning it takes keyword-only arguments, +specify ``*`` on a line by itself before the first keyword-only argument, +indented the same as the parameter lines. -12. Save and close the file, then run ``Tools/clinic/clinic.py`` on - it. With luck everything worked---your block now has output, and - a :file:`.c.h` file has been generated! Reopen the file in your - text editor to see:: +:py:meth:`!_pickle.Pickler.dump` has neither, so our sample is unchanged. - /*[clinic input] - _pickle.Pickler.dump +Next, if the existing C function calls :c:func:`PyArg_ParseTuple` +(as opposed to :c:func:`PyArg_ParseTupleAndKeywords`), then all its +arguments are positional-only. - obj: 'O' - The object to be pickled. - / +To mark parameters as positional-only in Argument Clinic, +add a ``/`` on a line by itself after the last positional-only parameter, +indented the same as the parameter lines. - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +Sample:: - static PyObject * - _pickle_Pickler_dump(PicklerObject *self, PyObject *obj) - /*[clinic end generated code: output=87ecad1261e02ac7 input=552eb1c0f52260d9]*/ + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - Obviously, if Argument Clinic didn't produce any output, it's because - it found an error in your input. Keep fixing your errors and retrying - until Argument Clinic processes your file without complaint. + /*[clinic input] + _pickle.Pickler.dump - For readability, most of the glue code has been generated to a :file:`.c.h` - file. You'll need to include that in your original :file:`.c` file, - typically right after the clinic module block:: + obj: 'O' + / - #include "clinic/_pickle.c.h" + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ -13. Double-check that the argument-parsing code Argument Clinic generated - looks basically the same as the existing code. +It can be helpful to write a per-parameter docstring for each parameter. +Since per-parameter docstrings are optional, +you can skip this step if you prefer. - First, ensure both places use the same argument-parsing function. - The existing code must call either - :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords`; - ensure that the code generated by Argument Clinic calls the - *exact* same function. +Nevertheless, here's how to add a per-parameter docstring. +The first line of the per-parameter docstring +must be indented further than the parameter definition. +The left margin of this first line establishes +the left margin for the whole per-parameter docstring; +all the text you write will be outdented by this amount. +You can write as much text as you like, across multiple lines if you wish. - Second, the format string passed in to :c:func:`!PyArg_ParseTuple` or - :c:func:`!PyArg_ParseTupleAndKeywords` should be *exactly* the same - as the hand-written one in the existing function, up to the colon - or semi-colon. +Sample:: - (Argument Clinic always generates its format strings - with a ``:`` followed by the name of the function. If the - existing code's format string ends with ``;``, to provide - usage help, this change is harmless?don't worry about it.) + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - Third, for parameters whose format units require two arguments - (like a length variable, or an encoding string, or a pointer - to a conversion function), ensure that the second argument is - *exactly* the same between the two invocations. + /*[clinic input] + _pickle.Pickler.dump - Fourth, inside the output portion of the block you'll find a preprocessor - macro defining the appropriate static :c:type:`PyMethodDef` structure for - this builtin:: + obj: 'O' + The object to be pickled. + / - #define __PICKLE_PICKLER_DUMP_METHODDEF \ - {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - This static structure should be *exactly* the same as the existing static - :c:type:`!PyMethodDef` structure for this builtin. +Save and close the file, then run ``Tools/clinic/clinic.py`` on it. +With luck everything worked---your block now has output, +and a :file:`.c.h` file has been generated! +Reload the file in your text editor to see the generated code:: - If any of these items differ in *any way*, - adjust your Argument Clinic function specification and rerun - ``Tools/clinic/clinic.py`` until they *are* the same. + /*[clinic input] + _pickle.Pickler.dump + obj: 'O' + The object to be pickled. + / -14. Notice that the last line of its output is the declaration - of your "impl" function. This is where the builtin's implementation goes. - Delete the existing prototype of the function you're modifying, but leave - the opening curly brace. Now delete its argument parsing code and the - declarations of all the variables it dumps the arguments into. - Notice how the Python arguments are now arguments to this impl function; - if the implementation used different names for these variables, fix it. + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - Let's reiterate, just because it's kind of weird. Your code should now - look like this:: + static PyObject * + _pickle_Pickler_dump(PicklerObject *self, PyObject *obj) + /*[clinic end generated code: output=87ecad1261e02ac7 input=552eb1c0f52260d9]*/ - static return_type - your_function_impl(...) - /*[clinic end generated code: checksum=...]*/ - { - ... +Obviously, if Argument Clinic didn't produce any output, +it's because it found an error in your input. +Keep fixing your errors and retrying until Argument Clinic processes your file +without complaint. - Argument Clinic generated the checksum line and the function prototype just - above it. You should write the opening (and closing) curly braces for the - function, and the implementation inside. +For readability, most of the glue code has been generated to a :file:`.c.h` +file. You'll need to include that in your original :file:`.c` file, +typically right after the clinic module block:: - Sample:: + #include "clinic/_pickle.c.h" - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ - /*[clinic end generated code: checksum=da39a3ee5e6b4b0d3255bfef95601890afd80709]*/ +Double-check that the argument-parsing code Argument Clinic generated +looks basically the same as the existing code. - /*[clinic input] - _pickle.Pickler.dump +First, ensure both places use the same argument-parsing function. +The existing code must call either +:c:func:`PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords`; +ensure that the code generated by Argument Clinic calls the +*exact* same function. - obj: 'O' - The object to be pickled. - / +Second, the format string passed in to :c:func:`!PyArg_ParseTuple` or +:c:func:`!PyArg_ParseTupleAndKeywords` should be *exactly* the same +as the hand-written one in the existing function, +up to the colon or semi-colon. - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +Argument Clinic always generates its format strings +with a ``:`` followed by the name of the function. +If the existing code's format string ends with ``;``, +to provide usage help, this change is harmless --- don't worry about it. - PyDoc_STRVAR(__pickle_Pickler_dump__doc__, - "Write a pickled representation of obj to the open file.\n" - "\n" - ... - static PyObject * - _pickle_Pickler_dump_impl(PicklerObject *self, PyObject *obj) - /*[clinic end generated code: checksum=3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/ - { - /* Check whether the Pickler was initialized correctly (issue3664). - Developers often forget to call __init__() in their subclasses, which - would trigger a segfault without this check. */ - if (self->write == NULL) { - PyErr_Format(PicklingError, - "Pickler.__init__() was not called by %s.__init__()", - Py_TYPE(self)->tp_name); - return NULL; - } +Third, for parameters whose format units require two arguments, +like a length variable, an encoding string, or a pointer +to a conversion function, ensure that the second argument is +*exactly* the same between the two invocations. - if (_Pickler_ClearBuffer(self) < 0) - return NULL; +Fourth, inside the output portion of the block, +you'll find a preprocessor macro defining the appropriate static +:c:type:`PyMethodDef` structure for this builtin:: - ... + #define __PICKLE_PICKLER_DUMP_METHODDEF \ + {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, -15. Remember the macro with the :c:type:`PyMethodDef` structure for this - function? Find the existing :c:type:`!PyMethodDef` structure for this - function and replace it with a reference to the macro. (If the builtin - is at module scope, this will probably be very near the end of the file; - if the builtin is a class method, this will probably be below but relatively - near to the implementation.) +This static structure should be *exactly* the same as the existing static +:c:type:`!PyMethodDef` structure for this builtin. - Note that the body of the macro contains a trailing comma. So when you - replace the existing static :c:type:`!PyMethodDef` structure with the macro, - *don't* add a comma to the end. +If any of these items differ in *any way*, +adjust your Argument Clinic function specification and rerun +``Tools/clinic/clinic.py`` until they *are* the same. - Sample:: +Notice that the last line of its output is the declaration +of your "impl" function. This is where the builtin's implementation goes. +Delete the existing prototype of the function you're modifying, but leave +the opening curly brace. Now delete its argument parsing code and the +declarations of all the variables it dumps the arguments into. +Notice how the Python arguments are now arguments to this impl function; +if the implementation used different names for these variables, fix it. - static struct PyMethodDef Pickler_methods[] = { - __PICKLE_PICKLER_DUMP_METHODDEF - __PICKLE_PICKLER_CLEAR_MEMO_METHODDEF - {NULL, NULL} /* sentinel */ - }; +Let's reiterate, just because it's kind of weird. +Your code should now look like this:: + static return_type + your_function_impl(...) + /*[clinic end generated code: input=..., output=...]*/ + { + ... -16. Argument Clinic may generate new instances of ``_Py_ID``. For example:: +Argument Clinic generated the checksum line and the function prototype just +above it. You should write the opening and closing curly braces for the +function, and the implementation inside. - &_Py_ID(new_unique_py_id) +Sample:: - If it does, you'll have to run ``make regen-global-objects`` - to regenerate the list of precompiled identifiers at this point. + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ + /*[clinic end generated code: checksum=da39a3ee5e6b4b0d3255bfef95601890afd80709]*/ + /*[clinic input] + _pickle.Pickler.dump -17. Compile, then run the relevant portions of the regression-test suite. - This change should not introduce any new compile-time warnings or errors, - and there should be no externally visible change to Python's behavior. + obj: 'O' + The object to be pickled. + / - Well, except for one difference: :py:func:`inspect.signature` run on your function - should now provide a valid signature! + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - Congratulations, you've ported your first function to work with Argument Clinic! + PyDoc_STRVAR(__pickle_Pickler_dump__doc__, + "Write a pickled representation of obj to the open file.\n" + "\n" + ... + static PyObject * + _pickle_Pickler_dump_impl(PicklerObject *self, PyObject *obj) + /*[clinic end generated code: checksum=3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/ + { + /* Check whether the Pickler was initialized correctly (issue3664). + Developers often forget to call __init__() in their subclasses, which + would trigger a segfault without this check. */ + if (self->write == NULL) { + PyErr_Format(PicklingError, + "Pickler.__init__() was not called by %s.__init__()", + Py_TYPE(self)->tp_name); + return NULL; + } + + if (_Pickler_ClearBuffer(self) < 0) { + return NULL; + } + + ... + +Remember the macro with the :c:type:`PyMethodDef` structure for this function? +Find the existing :c:type:`!PyMethodDef` structure for this +function and replace it with a reference to the macro. If the builtin +is at module scope, this will probably be very near the end of the file; +if the builtin is a class method, this will probably be below but relatively +near to the implementation. + +Note that the body of the macro contains a trailing comma; when you +replace the existing static :c:type:`!PyMethodDef` structure with the macro, +*don't* add a comma to the end. + +Sample:: + + static struct PyMethodDef Pickler_methods[] = { + __PICKLE_PICKLER_DUMP_METHODDEF + __PICKLE_PICKLER_CLEAR_MEMO_METHODDEF + {NULL, NULL} /* sentinel */ + }; + +Argument Clinic may generate new instances of ``_Py_ID``. For example:: + + &_Py_ID(new_unique_py_id) + +If it does, you'll have to run ``make regen-global-objects`` +to regenerate the list of precompiled identifiers at this point. + +Finally, compile, then run the relevant portions of the regression-test suite. +This change should not introduce any new compile-time warnings or errors, +and there should be no externally visible change to Python's behavior, +except for one difference: :py:func:`inspect.signature` run on your function +should now provide a valid signature! + +Congratulations, you've ported your first function to work with Argument Clinic! .. _clinic-howtos: @@ -913,6 +902,8 @@ you *must* not call :c:func:`PyBuffer_Release` on the provided buffer. Argument Clinic generates code that does it for you (in the parsing function). +.. _clinic-howto-advanced-converters: + How to use advanced converters ------------------------------ @@ -943,6 +934,7 @@ This restriction doesn't seem unreasonable; CPython itself always passes in stat hard-coded encoding strings for parameters whose format units start with ``e``. +.. _clinic-howto-default-values: .. _default_values: How to assign default values to parameter @@ -1053,6 +1045,8 @@ you're not permitted to use: * Tuple/list/set/dict literals. +.. _clinic-howto-return-converters: + How to use return converters ---------------------------- @@ -1195,6 +1189,8 @@ variable to the C code:: /*[python checksum:...]*/ +.. _clinic-howto-self-converter: + How to use the "self converter" ------------------------------- From webhook-mailer at python.org Wed Jul 26 17:21:32 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 26 Jul 2023 21:21:32 -0000 Subject: [Python-checkins] [3.11] Docs: Remove the numbered steps from the Argument Clinic tutorial (#107203) (#107319) Message-ID: https://github.com/python/cpython/commit/55edee247cc305d97a47e7eb428783a405c87a78 commit: 55edee247cc305d97a47e7eb428783a405c87a78 branch: 3.11 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-26T21:21:28Z summary: [3.11] Docs: Remove the numbered steps from the Argument Clinic tutorial (#107203) (#107319) Instead, order the tutorial as one body of prose, making it easier to align the tutorial according to Di?taxis principles. (cherry picked from commit 592395577c679543d899e68a3cff538b8b4df80d) files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 2d332f03d91a2..d14713e925224 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -167,23 +167,23 @@ convert a function to work with it. Here, then, are the bare minimum steps you'd need to follow to convert a function to work with Argument Clinic. Note that for code you plan to check in to CPython, you really should take the conversion farther, -using some of the advanced concepts you'll see later on in -the document (like "return converters" and "self converters"). +using some of the :ref:`advanced concepts ` +you'll see later on in the document, +like :ref:`clinic-howto-return-converters` +and :ref:`clinic-howto-self-converter`. But we'll keep it simple for this walkthrough so you can learn. -Let's dive in! +First, make sure you're working with a freshly updated checkout +of the CPython trunk. -0. Make sure you're working with a freshly updated checkout - of the CPython trunk. +Next, find a Python builtin that calls either :c:func:`PyArg_ParseTuple` +or :c:func:`PyArg_ParseTupleAndKeywords`, and hasn't been converted +to work with Argument Clinic yet. +For this tutorial, we'll be using +:py:meth:`_pickle.Pickler.dump `. -1. Find a Python builtin that calls either :c:func:`PyArg_ParseTuple` - or :c:func:`PyArg_ParseTupleAndKeywords`, and hasn't been converted - to work with Argument Clinic yet. - For my example I'm using - :py:meth:`_pickle.Pickler.dump `. - -2. If the call to the :c:func:`!PyArg_Parse*` function uses any of the - following format units: +If the call to the :c:func:`!PyArg_Parse*` function uses any of the +following format units...: .. code-block:: none @@ -194,380 +194,370 @@ Let's dive in! et et# - or if it has multiple calls to :c:func:`PyArg_ParseTuple`, - you should choose a different function. Argument Clinic *does* - support all of these scenarios. But these are advanced - topics?let's do something simpler for your first function. +... or if it has multiple calls to :c:func:`PyArg_ParseTuple`, +you should choose a different function. +(See :ref:`clinic-howto-advanced-converters` for those scenarios.) - Also, if the function has multiple calls to :c:func:`!PyArg_ParseTuple` - or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different - types for the same argument, or if the function uses something besides - :c:func:`!PyArg_Parse*` functions to parse its arguments, it probably - isn't suitable for conversion to Argument Clinic. Argument Clinic - doesn't support generic functions or polymorphic parameters. +Also, if the function has multiple calls to :c:func:`!PyArg_ParseTuple` +or :c:func:`PyArg_ParseTupleAndKeywords` where it supports different +types for the same argument, or if the function uses something besides +:c:func:`!PyArg_Parse*` functions to parse its arguments, it probably +isn't suitable for conversion to Argument Clinic. Argument Clinic +doesn't support generic functions or polymorphic parameters. -3. Add the following boilerplate above the function, creating our block:: +Next, add the following boilerplate above the function, +creating our input block:: /*[clinic input] [clinic start generated code]*/ -4. Cut the docstring and paste it in between the ``[clinic]`` lines, - removing all the junk that makes it a properly quoted C string. - When you're done you should have just the text, based at the left - margin, with no line wider than 80 characters. - (Argument Clinic will preserve indents inside the docstring.) - - If the old docstring had a first line that looked like a function - signature, throw that line away. (The docstring doesn't need it - anymore?when you use :py:func:`help` on your builtin in the future, - the first line will be built automatically based on the function's - signature.) - - Sample:: - - /*[clinic input] - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ - -5. If your docstring doesn't have a "summary" line, Argument Clinic will - complain. So let's make sure it has one. The "summary" line should - be a paragraph consisting of a single 80-column line - at the beginning of the docstring. - - (Our example docstring consists solely of a summary line, so the sample - code doesn't have to change for this step.) - -6. Above the docstring, enter the name of the function, followed - by a blank line. This should be the Python name of the function, - and should be the full dotted path - to the function?it should start with the name of the module, - include any sub-modules, and if the function is a method on - a class it should include the class name too. - - Sample:: - - /*[clinic input] - _pickle.Pickler.dump - - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +Cut the docstring and paste it in between the ``[clinic]`` lines, +removing all the junk that makes it a properly quoted C string. +When you're done you should have just the text, based at the left +margin, with no line wider than 80 characters. +Argument Clinic will preserve indents inside the docstring. -7. If this is the first time that module or class has been used with Argument - Clinic in this C file, - you must declare the module and/or class. Proper Argument Clinic hygiene - prefers declaring these in a separate block somewhere near the - top of the C file, in the same way that include files and statics go at - the top. (In our sample code we'll just show the two blocks next to - each other.) +If the old docstring had a first line that looked like a function +signature, throw that line away; The docstring doesn't need it anymore --- +when you use :py:func:`help` on your builtin in the future, +the first line will be built automatically based on the function's signature. - The name of the class and module should be the same as the one - seen by Python. Check the name defined in the :c:type:`PyModuleDef` - or :c:type:`PyTypeObject` as appropriate. +Example docstring summary line:: - When you declare a class, you must also specify two aspects of its type - in C: the type declaration you'd use for a pointer to an instance of - this class, and a pointer to the :c:type:`!PyTypeObject` for this class. - - Sample:: - - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ - - /*[clinic input] - _pickle.Pickler.dump - - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ - - - - -8. Declare each of the parameters to the function. Each parameter - should get its own line. All the parameter lines should be - indented from the function name and the docstring. - - The general form of these parameter lines is as follows: + /*[clinic input] + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - .. code-block:: none +If your docstring doesn't have a "summary" line, Argument Clinic will +complain, so let's make sure it has one. The "summary" line should +be a paragraph consisting of a single 80-column line +at the beginning of the docstring. +(See :pep:`257` regarding docstring conventions.) - name_of_parameter: converter +Our example docstring consists solely of a summary line, so the sample +code doesn't have to change for this step. - If the parameter has a default value, add that after the - converter: +Now, above the docstring, enter the name of the function, followed +by a blank line. This should be the Python name of the function, +and should be the full dotted path to the function --- +it should start with the name of the module, +include any sub-modules, and if the function is a method on +a class it should include the class name too. - .. code-block:: none +In our example, :mod:`!_pickle` is the module, :py:class:`!Pickler` is the class, +and :py:meth:`!dump` is the method, so the name becomes +:py:meth:`!_pickle.Pickler.dump`:: - name_of_parameter: converter = default_value + /*[clinic input] + _pickle.Pickler.dump - Argument Clinic's support for "default values" is quite sophisticated; - please see :ref:`the section below on default values ` - for more information. + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - Add a blank line below the parameters. +If this is the first time that module or class has been used with Argument +Clinic in this C file, +you must declare the module and/or class. Proper Argument Clinic hygiene +prefers declaring these in a separate block somewhere near the +top of the C file, in the same way that include files and statics go at +the top. +In our sample code we'll just show the two blocks next to each other. - What's a "converter"? It establishes both the type - of the variable used in C, and the method to convert the Python - value into a C value at runtime. - For now you're going to use what's called a "legacy converter"?a - convenience syntax intended to make porting old code into Argument - Clinic easier. +The name of the class and module should be the same as the one +seen by Python. Check the name defined in the :c:type:`PyModuleDef` +or :c:type:`PyTypeObject` as appropriate. - For each parameter, copy the "format unit" for that - parameter from the :c:func:`PyArg_Parse` format argument and - specify *that* as its converter, as a quoted - string. ("format unit" is the formal name for the one-to-three - character substring of the *format* parameter that tells - the argument parsing function what the type of the variable - is and how to convert it. For more on format units please - see :ref:`arg-parsing`.) +When you declare a class, you must also specify two aspects of its type +in C: the type declaration you'd use for a pointer to an instance of +this class, and a pointer to the :c:type:`!PyTypeObject` for this class:: - For multicharacter format units like ``z#``, use the - entire two-or-three character string. + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - Sample:: + /*[clinic input] + _pickle.Pickler.dump - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - /*[clinic input] - _pickle.Pickler.dump +Declare each of the parameters to the function. Each parameter +should get its own line. All the parameter lines should be +indented from the function name and the docstring. +The general form of these parameter lines is as follows: - obj: 'O' +.. code-block:: none - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ + name_of_parameter: converter -9. If your function has ``|`` in the format string, meaning some - parameters have default values, you can ignore it. Argument - Clinic infers which parameters are optional based on whether - or not they have default values. +If the parameter has a default value, add that after the +converter: - If your function has ``$`` in the format string, meaning it - takes keyword-only arguments, specify ``*`` on a line by - itself before the first keyword-only argument, indented the - same as the parameter lines. +.. code-block:: none - (:py:meth:`!_pickle.Pickler.dump` has neither, so our sample is unchanged.) + name_of_parameter: converter = default_value +Argument Clinic's support for "default values" is quite sophisticated; +see :ref:`clinic-howto-default-values` for more information. -10. If the existing C function calls :c:func:`PyArg_ParseTuple` - (as opposed to :c:func:`PyArg_ParseTupleAndKeywords`), then all its - arguments are positional-only. +Next, add a blank line below the parameters. - To mark all parameters as positional-only in Argument Clinic, - add a ``/`` on a line by itself after the last parameter, - indented the same as the parameter lines. +What's a "converter"? +It establishes both the type of the variable used in C, +and the method to convert the Python value into a C value at runtime. +For now you're going to use what's called a "legacy converter" --- +a convenience syntax intended to make porting old code into Argument +Clinic easier. - Currently this is all-or-nothing; either all parameters are - positional-only, or none of them are. (In the future Argument - Clinic may relax this restriction.) +For each parameter, copy the "format unit" for that +parameter from the :c:func:`PyArg_Parse` format argument and +specify *that* as its converter, as a quoted string. +The "format unit" is the formal name for the one-to-three +character substring of the *format* parameter that tells +the argument parsing function what the type of the variable +is and how to convert it. +For more on format units please see :ref:`arg-parsing`. - Sample:: +For multicharacter format units like ``z#``, +use the entire two-or-three character string. - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ +Sample:: - /*[clinic input] - _pickle.Pickler.dump + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - obj: 'O' - / + /*[clinic input] + _pickle.Pickler.dump - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ + obj: 'O' -11. It's helpful to write a per-parameter docstring for each parameter. - But per-parameter docstrings are optional; you can skip this step - if you prefer. + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - Here's how to add a per-parameter docstring. The first line - of the per-parameter docstring must be indented further than the - parameter definition. The left margin of this first line establishes - the left margin for the whole per-parameter docstring; all the text - you write will be outdented by this amount. You can write as much - text as you like, across multiple lines if you wish. +If your function has ``|`` in the format string, +meaning some parameters have default values, you can ignore it. +Argument Clinic infers which parameters are optional +based on whether or not they have default values. - Sample:: +If your function has ``$`` in the format string, +meaning it takes keyword-only arguments, +specify ``*`` on a line by itself before the first keyword-only argument, +indented the same as the parameter lines. - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ +:py:meth:`!_pickle.Pickler.dump` has neither, so our sample is unchanged. - /*[clinic input] - _pickle.Pickler.dump +Next, if the existing C function calls :c:func:`PyArg_ParseTuple` +(as opposed to :c:func:`PyArg_ParseTupleAndKeywords`), then all its +arguments are positional-only. - obj: 'O' - The object to be pickled. - / +To mark parameters as positional-only in Argument Clinic, +add a ``/`` on a line by itself after the last positional-only parameter, +indented the same as the parameter lines. - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +Sample:: -12. Save and close the file, then run ``Tools/clinic/clinic.py`` on - it. With luck everything worked---your block now has output, and - a :file:`.c.h` file has been generated! Reopen the file in your - text editor to see:: + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ - /*[clinic input] - _pickle.Pickler.dump + /*[clinic input] + _pickle.Pickler.dump - obj: 'O' - The object to be pickled. - / + obj: 'O' + / - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - static PyObject * - _pickle_Pickler_dump(PicklerObject *self, PyObject *obj) - /*[clinic end generated code: output=87ecad1261e02ac7 input=552eb1c0f52260d9]*/ +It can be helpful to write a per-parameter docstring for each parameter. +Since per-parameter docstrings are optional, +you can skip this step if you prefer. - Obviously, if Argument Clinic didn't produce any output, it's because - it found an error in your input. Keep fixing your errors and retrying - until Argument Clinic processes your file without complaint. +Nevertheless, here's how to add a per-parameter docstring. +The first line of the per-parameter docstring +must be indented further than the parameter definition. +The left margin of this first line establishes +the left margin for the whole per-parameter docstring; +all the text you write will be outdented by this amount. +You can write as much text as you like, across multiple lines if you wish. - For readability, most of the glue code has been generated to a :file:`.c.h` - file. You'll need to include that in your original :file:`.c` file, - typically right after the clinic module block:: +Sample:: - #include "clinic/_pickle.c.h" + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ -13. Double-check that the argument-parsing code Argument Clinic generated - looks basically the same as the existing code. + /*[clinic input] + _pickle.Pickler.dump - First, ensure both places use the same argument-parsing function. - The existing code must call either - :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords`; - ensure that the code generated by Argument Clinic calls the - *exact* same function. + obj: 'O' + The object to be pickled. + / - Second, the format string passed in to :c:func:`!PyArg_ParseTuple` or - :c:func:`!PyArg_ParseTupleAndKeywords` should be *exactly* the same - as the hand-written one in the existing function, up to the colon - or semi-colon. + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - (Argument Clinic always generates its format strings - with a ``:`` followed by the name of the function. If the - existing code's format string ends with ``;``, to provide - usage help, this change is harmless?don't worry about it.) +Save and close the file, then run ``Tools/clinic/clinic.py`` on it. +With luck everything worked---your block now has output, +and a :file:`.c.h` file has been generated! +Reload the file in your text editor to see the generated code:: - Third, for parameters whose format units require two arguments - (like a length variable, or an encoding string, or a pointer - to a conversion function), ensure that the second argument is - *exactly* the same between the two invocations. + /*[clinic input] + _pickle.Pickler.dump - Fourth, inside the output portion of the block you'll find a preprocessor - macro defining the appropriate static :c:type:`PyMethodDef` structure for - this builtin:: + obj: 'O' + The object to be pickled. + / - #define __PICKLE_PICKLER_DUMP_METHODDEF \ - {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - This static structure should be *exactly* the same as the existing static - :c:type:`!PyMethodDef` structure for this builtin. + static PyObject * + _pickle_Pickler_dump(PicklerObject *self, PyObject *obj) + /*[clinic end generated code: output=87ecad1261e02ac7 input=552eb1c0f52260d9]*/ - If any of these items differ in *any way*, - adjust your Argument Clinic function specification and rerun - ``Tools/clinic/clinic.py`` until they *are* the same. +Obviously, if Argument Clinic didn't produce any output, +it's because it found an error in your input. +Keep fixing your errors and retrying until Argument Clinic processes your file +without complaint. +For readability, most of the glue code has been generated to a :file:`.c.h` +file. You'll need to include that in your original :file:`.c` file, +typically right after the clinic module block:: -14. Notice that the last line of its output is the declaration - of your "impl" function. This is where the builtin's implementation goes. - Delete the existing prototype of the function you're modifying, but leave - the opening curly brace. Now delete its argument parsing code and the - declarations of all the variables it dumps the arguments into. - Notice how the Python arguments are now arguments to this impl function; - if the implementation used different names for these variables, fix it. + #include "clinic/_pickle.c.h" - Let's reiterate, just because it's kind of weird. Your code should now - look like this:: +Double-check that the argument-parsing code Argument Clinic generated +looks basically the same as the existing code. - static return_type - your_function_impl(...) - /*[clinic end generated code: checksum=...]*/ - { - ... +First, ensure both places use the same argument-parsing function. +The existing code must call either +:c:func:`PyArg_ParseTuple` or :c:func:`PyArg_ParseTupleAndKeywords`; +ensure that the code generated by Argument Clinic calls the +*exact* same function. - Argument Clinic generated the checksum line and the function prototype just - above it. You should write the opening (and closing) curly braces for the - function, and the implementation inside. +Second, the format string passed in to :c:func:`!PyArg_ParseTuple` or +:c:func:`!PyArg_ParseTupleAndKeywords` should be *exactly* the same +as the hand-written one in the existing function, +up to the colon or semi-colon. - Sample:: +Argument Clinic always generates its format strings +with a ``:`` followed by the name of the function. +If the existing code's format string ends with ``;``, +to provide usage help, this change is harmless --- don't worry about it. - /*[clinic input] - module _pickle - class _pickle.Pickler "PicklerObject *" "&Pickler_Type" - [clinic start generated code]*/ - /*[clinic end generated code: checksum=da39a3ee5e6b4b0d3255bfef95601890afd80709]*/ +Third, for parameters whose format units require two arguments, +like a length variable, an encoding string, or a pointer +to a conversion function, ensure that the second argument is +*exactly* the same between the two invocations. - /*[clinic input] - _pickle.Pickler.dump +Fourth, inside the output portion of the block, +you'll find a preprocessor macro defining the appropriate static +:c:type:`PyMethodDef` structure for this builtin:: - obj: 'O' - The object to be pickled. - / + #define __PICKLE_PICKLER_DUMP_METHODDEF \ + {"dump", (PyCFunction)__pickle_Pickler_dump, METH_O, __pickle_Pickler_dump__doc__}, - Write a pickled representation of obj to the open file. - [clinic start generated code]*/ +This static structure should be *exactly* the same as the existing static +:c:type:`!PyMethodDef` structure for this builtin. - PyDoc_STRVAR(__pickle_Pickler_dump__doc__, - "Write a pickled representation of obj to the open file.\n" - "\n" - ... - static PyObject * - _pickle_Pickler_dump_impl(PicklerObject *self, PyObject *obj) - /*[clinic end generated code: checksum=3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/ - { - /* Check whether the Pickler was initialized correctly (issue3664). - Developers often forget to call __init__() in their subclasses, which - would trigger a segfault without this check. */ - if (self->write == NULL) { - PyErr_Format(PicklingError, - "Pickler.__init__() was not called by %s.__init__()", - Py_TYPE(self)->tp_name); - return NULL; - } +If any of these items differ in *any way*, +adjust your Argument Clinic function specification and rerun +``Tools/clinic/clinic.py`` until they *are* the same. - if (_Pickler_ClearBuffer(self) < 0) - return NULL; +Notice that the last line of its output is the declaration +of your "impl" function. This is where the builtin's implementation goes. +Delete the existing prototype of the function you're modifying, but leave +the opening curly brace. Now delete its argument parsing code and the +declarations of all the variables it dumps the arguments into. +Notice how the Python arguments are now arguments to this impl function; +if the implementation used different names for these variables, fix it. - ... +Let's reiterate, just because it's kind of weird. +Your code should now look like this:: -15. Remember the macro with the :c:type:`PyMethodDef` structure for this - function? Find the existing :c:type:`!PyMethodDef` structure for this - function and replace it with a reference to the macro. (If the builtin - is at module scope, this will probably be very near the end of the file; - if the builtin is a class method, this will probably be below but relatively - near to the implementation.) + static return_type + your_function_impl(...) + /*[clinic end generated code: input=..., output=...]*/ + { + ... - Note that the body of the macro contains a trailing comma. So when you - replace the existing static :c:type:`!PyMethodDef` structure with the macro, - *don't* add a comma to the end. +Argument Clinic generated the checksum line and the function prototype just +above it. You should write the opening and closing curly braces for the +function, and the implementation inside. - Sample:: +Sample:: - static struct PyMethodDef Pickler_methods[] = { - __PICKLE_PICKLER_DUMP_METHODDEF - __PICKLE_PICKLER_CLEAR_MEMO_METHODDEF - {NULL, NULL} /* sentinel */ - }; + /*[clinic input] + module _pickle + class _pickle.Pickler "PicklerObject *" "&Pickler_Type" + [clinic start generated code]*/ + /*[clinic end generated code: checksum=da39a3ee5e6b4b0d3255bfef95601890afd80709]*/ + /*[clinic input] + _pickle.Pickler.dump -16. Compile, then run the relevant portions of the regression-test suite. - This change should not introduce any new compile-time warnings or errors, - and there should be no externally visible change to Python's behavior. + obj: 'O' + The object to be pickled. + / - Well, except for one difference: :py:func:`inspect.signature` run on your function - should now provide a valid signature! + Write a pickled representation of obj to the open file. + [clinic start generated code]*/ - Congratulations, you've ported your first function to work with Argument Clinic! + PyDoc_STRVAR(__pickle_Pickler_dump__doc__, + "Write a pickled representation of obj to the open file.\n" + "\n" + ... + static PyObject * + _pickle_Pickler_dump_impl(PicklerObject *self, PyObject *obj) + /*[clinic end generated code: checksum=3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/ + { + /* Check whether the Pickler was initialized correctly (issue3664). + Developers often forget to call __init__() in their subclasses, which + would trigger a segfault without this check. */ + if (self->write == NULL) { + PyErr_Format(PicklingError, + "Pickler.__init__() was not called by %s.__init__()", + Py_TYPE(self)->tp_name); + return NULL; + } + + if (_Pickler_ClearBuffer(self) < 0) { + return NULL; + } + + ... + +Remember the macro with the :c:type:`PyMethodDef` structure for this function? +Find the existing :c:type:`!PyMethodDef` structure for this +function and replace it with a reference to the macro. If the builtin +is at module scope, this will probably be very near the end of the file; +if the builtin is a class method, this will probably be below but relatively +near to the implementation. + +Note that the body of the macro contains a trailing comma; when you +replace the existing static :c:type:`!PyMethodDef` structure with the macro, +*don't* add a comma to the end. + +Sample:: + + static struct PyMethodDef Pickler_methods[] = { + __PICKLE_PICKLER_DUMP_METHODDEF + __PICKLE_PICKLER_CLEAR_MEMO_METHODDEF + {NULL, NULL} /* sentinel */ + }; + +Finally, compile, then run the relevant portions of the regression-test suite. +This change should not introduce any new compile-time warnings or errors, +and there should be no externally visible change to Python's behavior, +except for one difference: :py:func:`inspect.signature` run on your function +should now provide a valid signature! + +Congratulations, you've ported your first function to work with Argument Clinic! .. _clinic-howtos: @@ -902,6 +892,8 @@ you *must* not call :c:func:`PyBuffer_Release` on the provided buffer. Argument Clinic generates code that does it for you (in the parsing function). +.. _clinic-howto-advanced-converters: + How to use advanced converters ------------------------------ @@ -932,6 +924,7 @@ This restriction doesn't seem unreasonable; CPython itself always passes in stat hard-coded encoding strings for parameters whose format units start with ``e``. +.. _clinic-howto-default-values: .. _default_values: How to assign default values to parameter @@ -1042,6 +1035,8 @@ you're not permitted to use: * Tuple/list/set/dict literals. +.. _clinic-howto-return-converters: + How to use return converters ---------------------------- @@ -1189,6 +1184,8 @@ variable to the C code:: /*[python checksum:...]*/ +.. _clinic-howto-self-converter: + How to use the "self converter" ------------------------------- From webhook-mailer at python.org Wed Jul 26 18:08:47 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 26 Jul 2023 22:08:47 -0000 Subject: [Python-checkins] Docs: Argument Clinic: Restructure "Basic concepts and usage" (#106981) Message-ID: https://github.com/python/cpython/commit/2ad699002e3ce09e9fa41e333ac72f16a32d94de commit: 2ad699002e3ce09e9fa41e333ac72f16a32d94de branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-26T22:08:43Z summary: Docs: Argument Clinic: Restructure "Basic concepts and usage" (#106981) Split "Basic concepts and usage" into: - Reference - Terminology - CLI reference - Background - Basic concepts Co-authored-by: Adam Turner <9087854+AA-Turner at users.noreply.github.com> Co-authored-by: Alex Waygood Co-authored-by: Ezio Melotti files: M Doc/howto/clinic.rst M Tools/clinic/clinic.py diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index ea3b4537d4d53..7aafd48711b58 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -8,6 +8,7 @@ Argument Clinic How-To :author: Larry Hastings +**Source code:** :source:`Tools/clinic/clinic.py`. .. topic:: Abstract @@ -15,10 +16,12 @@ Argument Clinic How-To Its purpose is to automate all the boilerplate involved with writing argument parsing code for "builtins", module level functions, and class methods. - This document is divided in three major sections: + This document is divided in four major sections: * :ref:`clinic-background` talks about the basic concepts and goals of Argument Clinic. + * :ref:`clinic-reference` describes the command-line interface and Argument + Clinic terminology. * :ref:`clinic-tutorial` guides you through all the steps required to adapt an existing C function to Argument Clinic. * :ref:`clinic-howtos` details how to handle specific tasks. @@ -93,39 +96,29 @@ and it should be able to do many interesting and smart things with all the information you give it. -Basic concepts and usage ------------------------- +Basic concepts +-------------- -Argument Clinic ships with CPython; you'll find it in -:source:`Tools/clinic/clinic.py`. -If you run that script, specifying a C file as an argument: - -.. code-block:: shell-session - - $ python Tools/clinic/clinic.py foo.c - -Argument Clinic will scan over the file looking for lines that -look exactly like this: +When Argument Clinic is run on a file, either via the :ref:`clinic-cli` +or via ``make clinic``, it will scan over the input files looking for +:term:`start lines `: .. code-block:: none /*[clinic input] -When it finds one, it reads everything up to a line that looks -exactly like this: +When it finds one, it reads everything up to the :term:`end line`: .. code-block:: none [clinic start generated code]*/ -Everything in between these two lines is input for Argument Clinic. -All of these lines, including the beginning and ending comment -lines, are collectively called an Argument Clinic "block". - -When Argument Clinic parses one of these blocks, it -generates output. This output is rewritten into the C file -immediately after the block, followed by a comment containing a checksum. -The Argument Clinic block now looks like this: +Everything in between these two lines is Argument Clinic :term:`input`. +When Argument Clinic parses input, it generates :term:`output`. +The output is rewritten into the C file immediately after the input, +followed by a :term:`checksum line`. +All of these lines, including the :term:`start line` and :term:`checksum line`, +are collectively called an Argument Clinic :term:`block`: .. code-block:: none @@ -133,28 +126,121 @@ The Argument Clinic block now looks like this: ... clinic input goes here ... [clinic start generated code]*/ ... clinic output goes here ... - /*[clinic end generated code: checksum=...]*/ + /*[clinic end generated code: ...]*/ If you run Argument Clinic on the same file a second time, Argument Clinic -will discard the old output and write out the new output with a fresh checksum -line. However, if the input hasn't changed, the output won't change either. - -You should never modify the output portion of an Argument Clinic block. Instead, -change the input until it produces the output you want. (That's the purpose of the -checksum?to detect if someone changed the output, as these edits would be lost -the next time Argument Clinic writes out fresh output.) - -For the sake of clarity, here's the terminology we'll use with Argument Clinic: - -* The first line of the comment (``/*[clinic input]``) is the *start line*. -* The last line of the initial comment (``[clinic start generated code]*/``) is the *end line*. -* The last line (``/*[clinic end generated code: checksum=...]*/``) is the *checksum line*. -* In between the start line and the end line is the *input*. -* In between the end line and the checksum line is the *output*. -* All the text collectively, from the start line to the checksum line inclusively, - is the *block*. (A block that hasn't been successfully processed by Argument - Clinic yet doesn't have output or a checksum line, but it's still considered - a block.) +will discard the old :term:`output` and write out the new output with a fresh +:term:`checksum line`. +If the :term:`input` hasn't changed, the output won't change either. + +.. note:: + + You should never modify the output of an Argument Clinic block, + as any change will be lost in future Argument Clinic runs; + Argument Clinic will detect an output checksum mismatch and regenerate the + correct output. + If you are not happy with the generated output, + you should instead change the input until it produces the output you want. + + +.. _clinic-reference: + +Reference +========= + + +.. _clinic-terminology: + +Terminology +----------- + +.. glossary:: + + start line + The line ``/*[clinic input]``. + This line marks the beginning of Argument Clinic input. + Note that the *start line* opens a C block comment. + + end line + The line ``[clinic start generated code]*/``. + The *end line* marks the _end_ of Argument Clinic :term:`input`, + but at the same time marks the _start_ of Argument Clinic :term:`output`, + thus the text *"clinic start start generated code"* + Note that the *end line* closes the C block comment opened + by the *start line*. + + checksum + A hash to distinguish unique :term:`inputs ` + and :term:`outputs `. + + checksum line + A line that looks like ``/*[clinic end generated code: ...]*/``. + The three dots will be replaced by a :term:`checksum` generated from the + :term:`input`, and a :term:`checksum` generated from the :term:`output`. + The checksum line marks the end of Argument Clinic generated code, + and is used by Argument Clinic to determine if it needs to regenerate + output. + + input + The text between the :term:`start line` and the :term:`end line`. + Note that the start and end lines open and close a C block comment; + the *input* is thus a part of that same C block comment. + + output + The text between the :term:`end line` and the :term:`checksum line`. + + block + All text from the :term:`start line` to the :term:`checksum line` inclusively. + + +.. _clinic-cli: + +Command-line interface +---------------------- + +The Argument Clinic :abbr:`CLI (Command-Line Interface)` is typically used to +process a single source file, like this: + +.. code-block:: shell-session + + $ python3 ./Tools/clinic/clinic.py foo.c + +The CLI supports the following options: + +.. program:: ./Tools/clinic/clinic.py [-h] [-f] [-o OUTPUT] [-v] \ + [--converters] [--make] [--srcdir SRCDIR] [FILE ...] + +.. option:: -h, --help + + Print CLI usage. + +.. option:: -f, --force + + Force output regeneration. + +.. option:: -o, --output OUTPUT + + Redirect file output to OUTPUT + +.. option:: -v, --verbose + + Enable verbose mode. + +.. option:: --converters + + Print a list of all supported converters and return converters. + +.. option:: --make + + Walk :option:`--srcdir` to run over all relevant files. + +.. option:: --srcdir SRCDIR + + The directory tree to walk in :option:`--make` mode. + +.. option:: FILE ... + + The list of files to process. .. _clinic-tutorial: diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 34cc4014b35b4..5d3ed4167b06c 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -5631,15 +5631,21 @@ def main(argv: list[str]) -> None: signatures ("docstrings") for CPython builtins. For more information see https://docs.python.org/3/howto/clinic.html""") - cmdline.add_argument("-f", "--force", action='store_true') - cmdline.add_argument("-o", "--output", type=str) - cmdline.add_argument("-v", "--verbose", action='store_true') - cmdline.add_argument("--converters", action='store_true') + cmdline.add_argument("-f", "--force", action='store_true', + help="force output regeneration") + cmdline.add_argument("-o", "--output", type=str, + help="redirect file output to OUTPUT") + cmdline.add_argument("-v", "--verbose", action='store_true', + help="enable verbose mode") + cmdline.add_argument("--converters", action='store_true', + help=("print a list of all supported converters " + "and return converters")) cmdline.add_argument("--make", action='store_true', - help="Walk --srcdir to run over all relevant files.") + help="walk --srcdir to run over all relevant files") cmdline.add_argument("--srcdir", type=str, default=os.curdir, - help="The directory tree to walk in --make mode.") - cmdline.add_argument("filename", type=str, nargs="*") + help="the directory tree to walk in --make mode") + cmdline.add_argument("filename", metavar="FILE", type=str, nargs="*", + help="the list of files to process") ns = cmdline.parse_args(argv) if ns.converters: From webhook-mailer at python.org Wed Jul 26 18:31:51 2023 From: webhook-mailer at python.org (iritkatriel) Date: Wed, 26 Jul 2023 22:31:51 -0000 Subject: [Python-checkins] gh-106149: fix comment on stackdepth of generators (#107321) Message-ID: https://github.com/python/cpython/commit/507d8bc39a79e965fa32aa08caa568f3399e1472 commit: 507d8bc39a79e965fa32aa08caa568f3399e1472 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-26T22:31:47Z summary: gh-106149: fix comment on stackdepth of generators (#107321) files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index b4e06e7cb1408..b673e3ac6c1cc 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -7735,8 +7735,9 @@ optimize_and_assemble_code_unit(struct compiler_unit *u, PyObject *const_cache, /* prepare_localsplus adds instructions for generators that push * and pop an item on the stack. This assertion makes sure there * is space on the stack for that. - * It should always be true, because at least one expression is - * required to turn a function into a generator. + * It should always be true, because a generator must have at + * least one expression or call to INTRINSIC_STOPITERATION_ERROR, + * which requires stackspace. */ assert(!(IS_GENERATOR(code_flags) && stackdepth == 0)); From webhook-mailer at python.org Wed Jul 26 18:37:17 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 26 Jul 2023 22:37:17 -0000 Subject: [Python-checkins] [3.11] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156) (#107190) Message-ID: https://github.com/python/cpython/commit/6bf2c29878451a4404ab9063664e6c5cde17cc2b commit: 6bf2c29878451a4404ab9063664e6c5cde17cc2b branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-27T00:37:14+02:00 summary: [3.11] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156) (#107190) Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicExternalTest test suite. (cherry picked from commit 83a2837b328c58b243f7d97bec12c64ec66681c5) Co-authored-by: Erlend E. Aasland Co-authored-by: Nikita Sobolev files: From webhook-mailer at python.org Wed Jul 26 19:02:28 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 26 Jul 2023 23:02:28 -0000 Subject: [Python-checkins] [3.11] gh-106368: Increase Argument Clinic CLI test coverage (#107277) (#107326) Message-ID: https://github.com/python/cpython/commit/9edeeb2014c154ad450c1fe6b5d170a2951df14c commit: 9edeeb2014c154ad450c1fe6b5d170a2951df14c branch: 3.11 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-26T23:02:24Z summary: [3.11] gh-106368: Increase Argument Clinic CLI test coverage (#107277) (#107326) (cherry picked from commit 579100f6d75a27429e7f8de74935d7bc3a3e44e6) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 1cf665b03002d..32c4b1e67f957 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1294,18 +1294,18 @@ def test_scaffolding(self): class ClinicExternalTest(TestCase): maxDiff = None + clinic_py = os.path.join(test_tools.toolsdir, "clinic", "clinic.py") def _do_test(self, *args, expect_success=True): - clinic_py = os.path.join(test_tools.toolsdir, "clinic", "clinic.py") with subprocess.Popen( - [sys.executable, "-Xutf8", clinic_py, *args], + [sys.executable, "-Xutf8", self.clinic_py, *args], encoding="utf-8", bufsize=0, stdout=subprocess.PIPE, stderr=subprocess.PIPE, ) as proc: proc.wait() - if expect_success == bool(proc.returncode): + if expect_success and proc.returncode: self.fail("".join(proc.stderr)) stdout = proc.stdout.read() stderr = proc.stderr.read() @@ -1395,6 +1395,49 @@ def test_cli_force(self): generated = f.read() self.assertTrue(generated.endswith(checksum)) + def test_cli_make(self): + c_code = dedent(""" + /*[clinic input] + [clinic start generated code]*/ + """) + py_code = "pass" + c_files = "file1.c", "file2.c" + py_files = "file1.py", "file2.py" + + def create_files(files, srcdir, code): + for fn in files: + path = os.path.join(srcdir, fn) + with open(path, "w", encoding="utf-8") as f: + f.write(code) + + with os_helper.temp_dir() as tmp_dir: + # add some folders, some C files and a Python file + create_files(c_files, tmp_dir, c_code) + create_files(py_files, tmp_dir, py_code) + + # create C files in externals/ dir + ext_path = os.path.join(tmp_dir, "externals") + with os_helper.temp_dir(path=ext_path) as externals: + create_files(c_files, externals, c_code) + + # run clinic in verbose mode with --make on tmpdir + out = self.expect_success("-v", "--make", "--srcdir", tmp_dir) + + # expect verbose mode to only mention the C files in tmp_dir + for filename in c_files: + with self.subTest(filename=filename): + path = os.path.join(tmp_dir, filename) + self.assertIn(path, out) + for filename in py_files: + with self.subTest(filename=filename): + path = os.path.join(tmp_dir, filename) + self.assertNotIn(path, out) + # don't expect C files from the externals dir + for filename in c_files: + with self.subTest(filename=filename): + path = os.path.join(ext_path, filename) + self.assertNotIn(path, out) + def test_cli_verbose(self): with os_helper.temp_dir() as tmp_dir: fn = os.path.join(tmp_dir, "test.c") @@ -1481,6 +1524,35 @@ def test_cli_converters(self): f"expected converter {converter!r}, got {line!r}" ) + def test_cli_fail_converters_and_filename(self): + out = self.expect_failure("--converters", "test.c") + msg = ( + "Usage error: can't specify --converters " + "and a filename at the same time" + ) + self.assertIn(msg, out) + + def test_cli_fail_no_filename(self): + out = self.expect_failure() + self.assertIn("usage: clinic.py", out) + + def test_cli_fail_output_and_multiple_files(self): + out = self.expect_failure("-o", "out.c", "input.c", "moreinput.c") + msg = "Usage error: can't use -o with multiple filenames" + self.assertIn(msg, out) + + def test_cli_fail_filename_or_output_and_make(self): + for opts in ("-o", "out.c"), ("filename.c",): + with self.subTest(opts=opts): + out = self.expect_failure("--make", *opts) + msg = "Usage error: can't use -o or filenames with --make" + self.assertIn(msg, out) + + def test_cli_fail_make_without_srcdir(self): + out = self.expect_failure("--make", "--srcdir", "") + msg = "Usage error: --srcdir must not be empty with --make" + self.assertIn(msg, out) + try: import _testclinic as ac_tester From webhook-mailer at python.org Wed Jul 26 19:26:07 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Wed, 26 Jul 2023 23:26:07 -0000 Subject: [Python-checkins] [3.11] Docs: Argument Clinic: Restructure "Basic concepts and usage" (#106981) (#107327) Message-ID: https://github.com/python/cpython/commit/00b65da4931e233a8dd69686947583ba0aa10e5e commit: 00b65da4931e233a8dd69686947583ba0aa10e5e branch: 3.11 author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-27T01:26:04+02:00 summary: [3.11] Docs: Argument Clinic: Restructure "Basic concepts and usage" (#106981) (#107327) Split "Basic concepts and usage" into: - Reference - Terminology - CLI reference - Background - Basic concepts (cherry picked from commit 2ad699002e3ce09e9fa41e333ac72f16a32d94de) Co-authored-by: Adam Turner <9087854+AA-Turner at users.noreply.github.com> Co-authored-by: Alex Waygood Co-authored-by: Ezio Melotti files: M Doc/howto/clinic.rst M Tools/clinic/clinic.py diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index d14713e925224..1ec8984391bbe 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -8,6 +8,7 @@ Argument Clinic How-To :author: Larry Hastings +**Source code:** :source:`Tools/clinic/clinic.py`. .. topic:: Abstract @@ -15,10 +16,12 @@ Argument Clinic How-To Its purpose is to automate all the boilerplate involved with writing argument parsing code for "builtins", module level functions, and class methods. - This document is divided in three major sections: + This document is divided in four major sections: * :ref:`clinic-background` talks about the basic concepts and goals of Argument Clinic. + * :ref:`clinic-reference` describes the command-line interface and Argument + Clinic terminology. * :ref:`clinic-tutorial` guides you through all the steps required to adapt an existing C function to Argument Clinic. * :ref:`clinic-howtos` details how to handle specific tasks. @@ -93,39 +96,29 @@ and it should be able to do many interesting and smart things with all the information you give it. -Basic concepts and usage ------------------------- +Basic concepts +-------------- -Argument Clinic ships with CPython; you'll find it in -:source:`Tools/clinic/clinic.py`. -If you run that script, specifying a C file as an argument: - -.. code-block:: shell-session - - $ python3 Tools/clinic/clinic.py foo.c - -Argument Clinic will scan over the file looking for lines that -look exactly like this: +When Argument Clinic is run on a file, either via the :ref:`clinic-cli` +or via ``make clinic``, it will scan over the input files looking for +:term:`start lines `: .. code-block:: none /*[clinic input] -When it finds one, it reads everything up to a line that looks -exactly like this: +When it finds one, it reads everything up to the :term:`end line`: .. code-block:: none [clinic start generated code]*/ -Everything in between these two lines is input for Argument Clinic. -All of these lines, including the beginning and ending comment -lines, are collectively called an Argument Clinic "block". - -When Argument Clinic parses one of these blocks, it -generates output. This output is rewritten into the C file -immediately after the block, followed by a comment containing a checksum. -The Argument Clinic block now looks like this: +Everything in between these two lines is Argument Clinic :term:`input`. +When Argument Clinic parses input, it generates :term:`output`. +The output is rewritten into the C file immediately after the input, +followed by a :term:`checksum line`. +All of these lines, including the :term:`start line` and :term:`checksum line`, +are collectively called an Argument Clinic :term:`block`: .. code-block:: none @@ -133,28 +126,121 @@ The Argument Clinic block now looks like this: ... clinic input goes here ... [clinic start generated code]*/ ... clinic output goes here ... - /*[clinic end generated code: checksum=...]*/ + /*[clinic end generated code: ...]*/ If you run Argument Clinic on the same file a second time, Argument Clinic -will discard the old output and write out the new output with a fresh checksum -line. However, if the input hasn't changed, the output won't change either. - -You should never modify the output portion of an Argument Clinic block. Instead, -change the input until it produces the output you want. (That's the purpose of the -checksum?to detect if someone changed the output, as these edits would be lost -the next time Argument Clinic writes out fresh output.) - -For the sake of clarity, here's the terminology we'll use with Argument Clinic: - -* The first line of the comment (``/*[clinic input]``) is the *start line*. -* The last line of the initial comment (``[clinic start generated code]*/``) is the *end line*. -* The last line (``/*[clinic end generated code: checksum=...]*/``) is the *checksum line*. -* In between the start line and the end line is the *input*. -* In between the end line and the checksum line is the *output*. -* All the text collectively, from the start line to the checksum line inclusively, - is the *block*. (A block that hasn't been successfully processed by Argument - Clinic yet doesn't have output or a checksum line, but it's still considered - a block.) +will discard the old :term:`output` and write out the new output with a fresh +:term:`checksum line`. +If the :term:`input` hasn't changed, the output won't change either. + +.. note:: + + You should never modify the output of an Argument Clinic block, + as any change will be lost in future Argument Clinic runs; + Argument Clinic will detect an output checksum mismatch and regenerate the + correct output. + If you are not happy with the generated output, + you should instead change the input until it produces the output you want. + + +.. _clinic-reference: + +Reference +========= + + +.. _clinic-terminology: + +Terminology +----------- + +.. glossary:: + + start line + The line ``/*[clinic input]``. + This line marks the beginning of Argument Clinic input. + Note that the *start line* opens a C block comment. + + end line + The line ``[clinic start generated code]*/``. + The *end line* marks the _end_ of Argument Clinic :term:`input`, + but at the same time marks the _start_ of Argument Clinic :term:`output`, + thus the text *"clinic start start generated code"* + Note that the *end line* closes the C block comment opened + by the *start line*. + + checksum + A hash to distinguish unique :term:`inputs ` + and :term:`outputs `. + + checksum line + A line that looks like ``/*[clinic end generated code: ...]*/``. + The three dots will be replaced by a :term:`checksum` generated from the + :term:`input`, and a :term:`checksum` generated from the :term:`output`. + The checksum line marks the end of Argument Clinic generated code, + and is used by Argument Clinic to determine if it needs to regenerate + output. + + input + The text between the :term:`start line` and the :term:`end line`. + Note that the start and end lines open and close a C block comment; + the *input* is thus a part of that same C block comment. + + output + The text between the :term:`end line` and the :term:`checksum line`. + + block + All text from the :term:`start line` to the :term:`checksum line` inclusively. + + +.. _clinic-cli: + +Command-line interface +---------------------- + +The Argument Clinic :abbr:`CLI (Command-Line Interface)` is typically used to +process a single source file, like this: + +.. code-block:: shell-session + + $ python3 ./Tools/clinic/clinic.py foo.c + +The CLI supports the following options: + +.. program:: ./Tools/clinic/clinic.py [-h] [-f] [-o OUTPUT] [-v] \ + [--converters] [--make] [--srcdir SRCDIR] [FILE ...] + +.. option:: -h, --help + + Print CLI usage. + +.. option:: -f, --force + + Force output regeneration. + +.. option:: -o, --output OUTPUT + + Redirect file output to OUTPUT + +.. option:: -v, --verbose + + Enable verbose mode. + +.. option:: --converters + + Print a list of all supported converters and return converters. + +.. option:: --make + + Walk :option:`--srcdir` to run over all relevant files. + +.. option:: --srcdir SRCDIR + + The directory tree to walk in :option:`--make` mode. + +.. option:: FILE ... + + The list of files to process. .. _clinic-tutorial: diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index f8e89bc74f177..4d30e66b4822a 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -5190,15 +5190,21 @@ def main(argv): signatures ("docstrings") for CPython builtins. For more information see https://docs.python.org/3/howto/clinic.html""") - cmdline.add_argument("-f", "--force", action='store_true') - cmdline.add_argument("-o", "--output", type=str) - cmdline.add_argument("-v", "--verbose", action='store_true') - cmdline.add_argument("--converters", action='store_true') + cmdline.add_argument("-f", "--force", action='store_true', + help="force output regeneration") + cmdline.add_argument("-o", "--output", type=str, + help="redirect file output to OUTPUT") + cmdline.add_argument("-v", "--verbose", action='store_true', + help="enable verbose mode") + cmdline.add_argument("--converters", action='store_true', + help=("print a list of all supported converters " + "and return converters")) cmdline.add_argument("--make", action='store_true', - help="Walk --srcdir to run over all relevant files.") + help="walk --srcdir to run over all relevant files") cmdline.add_argument("--srcdir", type=str, default=os.curdir, - help="The directory tree to walk in --make mode.") - cmdline.add_argument("filename", type=str, nargs="*") + help="the directory tree to walk in --make mode") + cmdline.add_argument("filename", metavar="FILE", type=str, nargs="*", + help="the list of files to process") ns = cmdline.parse_args(argv) if ns.converters: From webhook-mailer at python.org Wed Jul 26 19:41:18 2023 From: webhook-mailer at python.org (vstinner) Date: Wed, 26 Jul 2023 23:41:18 -0000 Subject: [Python-checkins] gh-107298: Fix Sphinx warnings in the C API doc (#107302) Message-ID: https://github.com/python/cpython/commit/391e03fa05b80d17a14ac88d30c974fa2fa00adb commit: 391e03fa05b80d17a14ac88d30c974fa2fa00adb branch: main author: Victor Stinner committer: vstinner date: 2023-07-27T01:41:15+02:00 summary: gh-107298: Fix Sphinx warnings in the C API doc (#107302) * Update Doc/tools/.nitignore * Fix BufferedIOBase.write() link in buffer.rst files: M Doc/c-api/apiabiversion.rst M Doc/c-api/buffer.rst M Doc/c-api/bytes.rst M Doc/c-api/cell.rst M Doc/c-api/code.rst M Doc/c-api/gcsupport.rst M Doc/c-api/iterator.rst M Doc/c-api/type.rst M Doc/c-api/typehints.rst M Doc/c-api/unicode.rst M Doc/c-api/veryhigh.rst M Doc/tools/.nitignore diff --git a/Doc/c-api/apiabiversion.rst b/Doc/c-api/apiabiversion.rst index 62d542966622c..f6c8284daeacb 100644 --- a/Doc/c-api/apiabiversion.rst +++ b/Doc/c-api/apiabiversion.rst @@ -60,7 +60,7 @@ See :ref:`stable` for a discussion of API and ABI stability across versions. Use this for numeric comparisons, e.g. ``#if PY_VERSION_HEX >= ...``. - This version is also available via the symbol :data:`Py_Version`. + This version is also available via the symbol :c:var:`Py_Version`. .. c:var:: const unsigned long Py_Version diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst index 6e5443f0d6cdc..02b53ec149c73 100644 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -44,7 +44,7 @@ the elements exposed by an :class:`array.array` can be multi-byte values. An example consumer of the buffer interface is the :meth:`~io.BufferedIOBase.write` method of file objects: any object that can export a series of bytes through -the buffer interface can be written to a file. While :meth:`write` only +the buffer interface can be written to a file. While :meth:`!write` only needs read-only access to the internal contents of the object passed to it, other methods such as :meth:`~io.BufferedIOBase.readinto` need write access to the contents of their argument. The buffer interface allows objects to diff --git a/Doc/c-api/bytes.rst b/Doc/c-api/bytes.rst index 9f48f2ffafe17..4e3ffc7e23e3f 100644 --- a/Doc/c-api/bytes.rst +++ b/Doc/c-api/bytes.rst @@ -64,39 +64,39 @@ called with a non-bytes parameter. +-------------------+---------------+--------------------------------+ | Format Characters | Type | Comment | +===================+===============+================================+ - | :attr:`%%` | *n/a* | The literal % character. | + | ``%%`` | *n/a* | The literal % character. | +-------------------+---------------+--------------------------------+ - | :attr:`%c` | int | A single byte, | + | ``%c`` | int | A single byte, | | | | represented as a C int. | +-------------------+---------------+--------------------------------+ - | :attr:`%d` | int | Equivalent to | + | ``%d`` | int | Equivalent to | | | | ``printf("%d")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%u` | unsigned int | Equivalent to | + | ``%u`` | unsigned int | Equivalent to | | | | ``printf("%u")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%ld` | long | Equivalent to | + | ``%ld`` | long | Equivalent to | | | | ``printf("%ld")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%lu` | unsigned long | Equivalent to | + | ``%lu`` | unsigned long | Equivalent to | | | | ``printf("%lu")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%zd` | :c:type:`\ | Equivalent to | + | ``%zd`` | :c:type:`\ | Equivalent to | | | Py_ssize_t` | ``printf("%zd")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%zu` | size_t | Equivalent to | + | ``%zu`` | size_t | Equivalent to | | | | ``printf("%zu")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%i` | int | Equivalent to | + | ``%i`` | int | Equivalent to | | | | ``printf("%i")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%x` | int | Equivalent to | + | ``%x`` | int | Equivalent to | | | | ``printf("%x")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%s` | const char\* | A null-terminated C character | + | ``%s`` | const char\* | A null-terminated C character | | | | array. | +-------------------+---------------+--------------------------------+ - | :attr:`%p` | const void\* | The hex representation of a C | + | ``%p`` | const void\* | The hex representation of a C | | | | pointer. Mostly equivalent to | | | | ``printf("%p")`` except that | | | | it is guaranteed to start with | diff --git a/Doc/c-api/cell.rst b/Doc/c-api/cell.rst index ac4ef5adc5cc2..f8cd0344fdd1c 100644 --- a/Doc/c-api/cell.rst +++ b/Doc/c-api/cell.rst @@ -25,7 +25,7 @@ Cell objects are not likely to be useful elsewhere. The type object corresponding to cell objects. -.. c:function:: int PyCell_Check(ob) +.. c:function:: int PyCell_Check(PyObject *ob) Return true if *ob* is a cell object; *ob* must not be ``NULL``. This function always succeeds. diff --git a/Doc/c-api/code.rst b/Doc/c-api/code.rst index a99de9904c074..89fe42d1ff05f 100644 --- a/Doc/c-api/code.rst +++ b/Doc/c-api/code.rst @@ -39,7 +39,7 @@ bound into a function. use :c:func:`PyCode_NewEmpty` instead. Since the definition of the bytecode changes often, calling - :c:func:`PyCode_New` directly can bind you to a precise Python version. + :c:func:`PyUnstable_Code_New` directly can bind you to a precise Python version. The many arguments of this function are inter-dependent in complex ways, meaning that subtle changes to values are likely to result in incorrect @@ -58,8 +58,8 @@ bound into a function. .. c:function:: PyCodeObject* PyUnstable_Code_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable) - Similar to :c:func:`PyCode_New`, but with an extra "posonlyargcount" for positional-only arguments. - The same caveats that apply to ``PyCode_New`` also apply to this function. + Similar to :c:func:`PyUnstable_Code_New`, but with an extra "posonlyargcount" for positional-only arguments. + The same caveats that apply to ``PyUnstable_Code_New`` also apply to this function. .. index:: single: PyCode_NewWithPosOnlyArgs diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index e56414ab9f754..88709879d0163 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -143,7 +143,7 @@ rules: .. versionchanged:: 3.8 - The :c:func:`_PyObject_GC_TRACK` and :c:func:`_PyObject_GC_UNTRACK` macros + The :c:func:`!_PyObject_GC_TRACK` and :c:func:`!_PyObject_GC_UNTRACK` macros have been removed from the public C API. The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type: diff --git a/Doc/c-api/iterator.rst b/Doc/c-api/iterator.rst index 95952237ca746..6b7ba8c997916 100644 --- a/Doc/c-api/iterator.rst +++ b/Doc/c-api/iterator.rst @@ -19,7 +19,7 @@ sentinel value is returned. types. -.. c:function:: int PySeqIter_Check(op) +.. c:function:: int PySeqIter_Check(PyObject *op) Return true if the type of *op* is :c:data:`PySeqIter_Type`. This function always succeeds. @@ -38,7 +38,7 @@ sentinel value is returned. two-argument form of the :func:`iter` built-in function. -.. c:function:: int PyCallIter_Check(op) +.. c:function:: int PyCallIter_Check(PyObject *op) Return true if the type of *op* is :c:data:`PyCallIter_Type`. This function always succeeds. diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index 553d86aa3d997..e7b35c43da32a 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -480,7 +480,7 @@ The following functions and structs are used to create Setting :c:data:`Py_tp_bases` or :c:data:`Py_tp_base` may be problematic on some platforms. To avoid issues, use the *bases* argument of - :py:func:`PyType_FromSpecWithBases` instead. + :c:func:`PyType_FromSpecWithBases` instead. .. versionchanged:: 3.9 diff --git a/Doc/c-api/typehints.rst b/Doc/c-api/typehints.rst index 4c1957a2a1dbc..98fe68737deb8 100644 --- a/Doc/c-api/typehints.rst +++ b/Doc/c-api/typehints.rst @@ -35,7 +35,7 @@ two types exist -- :ref:`GenericAlias ` and ... } - .. seealso:: The data model method :meth:`__class_getitem__`. + .. seealso:: The data model method :meth:`~object.__class_getitem__`. .. versionadded:: 3.9 diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index ad1d5edb901dc..c7f5f0f9e230d 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1207,7 +1207,7 @@ This codec is special in that it can be used to implement many different codecs (and this is in fact what was done to obtain most of the standard codecs included in the :mod:`!encodings` package). The codec uses mappings to encode and decode characters. The mapping objects provided must support the -:meth:`__getitem__` mapping interface; dictionaries and sequences work well. +:meth:`~object.__getitem__` mapping interface; dictionaries and sequences work well. These are the mapping codec APIs: @@ -1250,7 +1250,7 @@ The following codec API is special in that maps Unicode to Unicode. The mapping table must map Unicode ordinal integers to Unicode ordinal integers or ``None`` (causing deletion of the character). - Mapping tables need only provide the :meth:`__getitem__` interface; dictionaries + Mapping tables need only provide the :meth:`~object.__getitem__` interface; dictionaries and sequences work well. Unmapped character ordinals (ones which cause a :exc:`LookupError`) are left untouched and are copied as-is. diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 56fa2d6abd913..a19c274d05f2c 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -353,7 +353,7 @@ the same library that the Python runtime is using. executed, it is passed as ``PyCompilerFlags *flags``. In this case, ``from __future__ import`` can modify *flags*. - Whenever ``PyCompilerFlags *flags`` is ``NULL``, :attr:`cf_flags` is treated as + Whenever ``PyCompilerFlags *flags`` is ``NULL``, :c:member:`cf_flags` is treated as equal to ``0``, and any modification due to ``from __future__ import`` is discarded. diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 2a10db5f3fff8..1ea7c791f6753 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -3,13 +3,9 @@ # Keep lines sorted lexicographically to help avoid merge conflicts. Doc/c-api/allocation.rst -Doc/c-api/apiabiversion.rst Doc/c-api/bool.rst Doc/c-api/buffer.rst -Doc/c-api/bytes.rst Doc/c-api/capsule.rst -Doc/c-api/cell.rst -Doc/c-api/code.rst Doc/c-api/complex.rst Doc/c-api/conversion.rst Doc/c-api/datetime.rst @@ -22,7 +18,6 @@ Doc/c-api/import.rst Doc/c-api/init.rst Doc/c-api/init_config.rst Doc/c-api/intro.rst -Doc/c-api/iterator.rst Doc/c-api/memory.rst Doc/c-api/memoryview.rst Doc/c-api/module.rst @@ -34,10 +29,8 @@ Doc/c-api/structures.rst Doc/c-api/sys.rst Doc/c-api/tuple.rst Doc/c-api/type.rst -Doc/c-api/typehints.rst Doc/c-api/typeobj.rst Doc/c-api/unicode.rst -Doc/c-api/veryhigh.rst Doc/extending/embedding.rst Doc/extending/extending.rst Doc/extending/newtypes.rst From webhook-mailer at python.org Wed Jul 26 20:52:43 2023 From: webhook-mailer at python.org (vstinner) Date: Thu, 27 Jul 2023 00:52:43 -0000 Subject: [Python-checkins] gh-107298: Fix more Sphinx warnings in the C API doc (#107329) Message-ID: https://github.com/python/cpython/commit/8d61a71f9c81619e34d4a30b625922ebc83c561b commit: 8d61a71f9c81619e34d4a30b625922ebc83c561b branch: main author: Victor Stinner committer: vstinner date: 2023-07-27T00:52:40Z summary: gh-107298: Fix more Sphinx warnings in the C API doc (#107329) Declare the following functions as macros, since they are actually macros. It avoids a warning on "TYPE" or "macro" argument. * PyMem_New() * PyMem_Resize() * PyModule_AddIntMacro() * PyModule_AddStringMacro() * PyObject_GC_New() * PyObject_GC_NewVar() * PyObject_New() * PyObject_NewVar() Add C standard C types to nitpick_ignore in Doc/conf.py: * int64_t * uint64_t * uintptr_t No longer ignore non existing "__int" type in nitpick_ignore. Update Doc/tools/.nitignore files: M Doc/c-api/allocation.rst M Doc/c-api/capsule.rst M Doc/c-api/complex.rst M Doc/c-api/conversion.rst M Doc/c-api/exceptions.rst M Doc/c-api/gcsupport.rst M Doc/c-api/import.rst M Doc/c-api/init.rst M Doc/c-api/init_config.rst M Doc/c-api/memory.rst M Doc/c-api/module.rst M Doc/c-api/set.rst M Doc/c-api/structures.rst M Doc/c-api/sys.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/conf.py M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/tools/.nitignore M Doc/whatsnew/2.3.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst diff --git a/Doc/c-api/allocation.rst b/Doc/c-api/allocation.rst index 0a8fcc5ae5fcd..44747e2964366 100644 --- a/Doc/c-api/allocation.rst +++ b/Doc/c-api/allocation.rst @@ -27,22 +27,25 @@ Allocating Objects on the Heap length information for a variable-size object. -.. c:function:: TYPE* PyObject_New(TYPE, PyTypeObject *type) +.. c:macro:: PyObject_New(TYPE, typeobj) Allocate a new Python object using the C structure type *TYPE* and the - Python type object *type*. Fields not defined by the Python object header + Python type object *typeobj* (``PyTypeObject*``). + Fields not defined by the Python object header are not initialized; the object's reference count will be one. The size of the memory allocation is determined from the :c:member:`~PyTypeObject.tp_basicsize` field of the type object. -.. c:function:: TYPE* PyObject_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) +.. c:macro:: PyObject_NewVar(TYPE, typeobj, size) Allocate a new Python object using the C structure type *TYPE* and the - Python type object *type*. Fields not defined by the Python object header + Python type object *typeobj* (``PyTypeObject*``). + Fields not defined by the Python object header are not initialized. The allocated memory allows for the *TYPE* structure - plus *size* fields of the size given by the :c:member:`~PyTypeObject.tp_itemsize` field of - *type*. This is useful for implementing objects like tuples, which are + plus *size* (``Py_ssize_t``) fields of the size + given by the :c:member:`~PyTypeObject.tp_itemsize` field of + *typeobj*. This is useful for implementing objects like tuples, which are able to determine their size at construction time. Embedding the array of fields into the same allocation decreases the number of allocations, improving the memory management efficiency. @@ -50,8 +53,8 @@ Allocating Objects on the Heap .. c:function:: void PyObject_Del(void *op) - Releases memory allocated to an object using :c:func:`PyObject_New` or - :c:func:`PyObject_NewVar`. This is normally called from the + Releases memory allocated to an object using :c:macro:`PyObject_New` or + :c:macro:`PyObject_NewVar`. This is normally called from the :c:member:`~PyTypeObject.tp_dealloc` handler specified in the object's type. The fields of the object should not be accessed after this call as the memory is no longer a valid Python object. diff --git a/Doc/c-api/capsule.rst b/Doc/c-api/capsule.rst index 427ed959c5856..2a1b602dc79c0 100644 --- a/Doc/c-api/capsule.rst +++ b/Doc/c-api/capsule.rst @@ -64,7 +64,7 @@ Refer to :ref:`using-capsules` for more information on using these objects. The *name* parameter must compare exactly to the name stored in the capsule. If the name stored in the capsule is ``NULL``, the *name* passed in must also - be ``NULL``. Python uses the C function :c:func:`strcmp` to compare capsule + be ``NULL``. Python uses the C function :c:func:`!strcmp` to compare capsule names. diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst index 6679ce76f1dc6..e3fd001c599c8 100644 --- a/Doc/c-api/complex.rst +++ b/Doc/c-api/complex.rst @@ -64,7 +64,7 @@ pointers. This is consistent throughout the API. representation. If *divisor* is null, this method returns zero and sets - :c:data:`errno` to :c:macro:`EDOM`. + :c:data:`errno` to :c:macro:`!EDOM`. .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp) @@ -73,7 +73,7 @@ pointers. This is consistent throughout the API. representation. If *num* is null and *exp* is not a positive real number, - this method returns zero and sets :c:data:`errno` to :c:macro:`EDOM`. + this method returns zero and sets :c:data:`errno` to :c:macro:`!EDOM`. Complex Numbers as Python Objects diff --git a/Doc/c-api/conversion.rst b/Doc/c-api/conversion.rst index fdb321fe7ab3f..c5350123dfdfd 100644 --- a/Doc/c-api/conversion.rst +++ b/Doc/c-api/conversion.rst @@ -119,10 +119,10 @@ The following functions provide locale-independent string to number conversions. .. c:function:: int PyOS_stricmp(const char *s1, const char *s2) Case insensitive comparison of strings. The function works almost - identically to :c:func:`strcmp` except that it ignores the case. + identically to :c:func:`!strcmp` except that it ignores the case. .. c:function:: int PyOS_strnicmp(const char *s1, const char *s2, Py_ssize_t size) Case insensitive comparison of strings. The function works almost - identically to :c:func:`strncmp` except that it ignores the case. + identically to :c:func:`!strncmp` except that it ignores the case. diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 4d81b273638b1..a2126ffc559ab 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -83,7 +83,7 @@ Printing and clearing This utility function prints a warning message to ``sys.stderr`` when an exception has been set but it is impossible for the interpreter to actually raise the exception. It is used, for example, when an exception occurs in an - :meth:`__del__` method. + :meth:`~object.__del__` method. The function is called with a single argument *obj* that identifies the context in which the unraisable exception occurred. If possible, @@ -165,7 +165,7 @@ For convenience, some of these functions will always return a tuple object whose first item is the integer :c:data:`errno` value and whose second item is the corresponding error message (gotten from :c:func:`!strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the - :c:data:`errno` value is :c:macro:`EINTR`, indicating an interrupted system call, + :c:data:`errno` value is :c:macro:`!EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, leaves it set to that. The function always returns ``NULL``, so a wrapper function around a system call can write ``return PyErr_SetFromErrno(type);`` @@ -177,7 +177,7 @@ For convenience, some of these functions will always return a Similar to :c:func:`PyErr_SetFromErrno`, with the additional behavior that if *filenameObject* is not ``NULL``, it is passed to the constructor of *type* as a third parameter. In the case of :exc:`OSError` exception, - this is used to define the :attr:`filename` attribute of the + this is used to define the :attr:`!filename` attribute of the exception instance. @@ -200,12 +200,12 @@ For convenience, some of these functions will always return a .. c:function:: PyObject* PyErr_SetFromWindowsErr(int ierr) This is a convenience function to raise :exc:`WindowsError`. If called with - *ierr* of ``0``, the error code returned by a call to :c:func:`GetLastError` - is used instead. It calls the Win32 function :c:func:`FormatMessage` to retrieve - the Windows description of error code given by *ierr* or :c:func:`GetLastError`, + *ierr* of ``0``, the error code returned by a call to :c:func:`!GetLastError` + is used instead. It calls the Win32 function :c:func:`!FormatMessage` to retrieve + the Windows description of error code given by *ierr* or :c:func:`!GetLastError`, then it constructs a tuple object whose first item is the *ierr* value and whose second item is the corresponding error message (gotten from - :c:func:`FormatMessage`), and then calls ``PyErr_SetObject(PyExc_WindowsError, + :c:func:`!FormatMessage`), and then calls ``PyErr_SetObject(PyExc_WindowsError, object)``. This function always returns ``NULL``. .. availability:: Windows. @@ -631,7 +631,7 @@ Signal Handling be interruptible by user requests (such as by pressing Ctrl-C). .. note:: - The default Python signal handler for :c:macro:`SIGINT` raises the + The default Python signal handler for :c:macro:`!SIGINT` raises the :exc:`KeyboardInterrupt` exception. @@ -642,7 +642,7 @@ Signal Handling single: SIGINT single: KeyboardInterrupt (built-in exception) - Simulate the effect of a :c:macro:`SIGINT` signal arriving. + Simulate the effect of a :c:macro:`!SIGINT` signal arriving. This is equivalent to ``PyErr_SetInterruptEx(SIGINT)``. .. note:: diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index 88709879d0163..6b2494ee4f0ed 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -25,8 +25,8 @@ include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the Constructors for container types must conform to two rules: -#. The memory for the object must be allocated using :c:func:`PyObject_GC_New` - or :c:func:`PyObject_GC_NewVar`. +#. The memory for the object must be allocated using :c:macro:`PyObject_GC_New` + or :c:macro:`PyObject_GC_NewVar`. #. Once all the fields which may contain references to other containers are initialized, it must call :c:func:`PyObject_GC_Track`. @@ -52,19 +52,19 @@ rules: class that implements the garbage collector protocol and the child class does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. -.. c:function:: TYPE* PyObject_GC_New(TYPE, PyTypeObject *type) +.. c:macro:: PyObject_GC_New(TYPE, typeobj) - Analogous to :c:func:`PyObject_New` but for container objects with the + Analogous to :c:macro:`PyObject_New` but for container objects with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. -.. c:function:: TYPE* PyObject_GC_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) +.. c:macro:: PyObject_GC_NewVar(TYPE, typeobj, size) - Analogous to :c:func:`PyObject_NewVar` but for container objects with the + Analogous to :c:macro:`PyObject_NewVar` but for container objects with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. .. c:function:: PyObject* PyUnstable_Object_GC_NewWithExtraData(PyTypeObject *type, size_t extra_size) - Analogous to :c:func:`PyObject_GC_New` but allocates *extra_size* + Analogous to :c:macro:`PyObject_GC_New` but allocates *extra_size* bytes at the end of the object (at offset :c:member:`~PyTypeObject.tp_basicsize`). The allocated memory is initialized to zeros, @@ -85,7 +85,7 @@ rules: .. c:function:: TYPE* PyObject_GC_Resize(TYPE, PyVarObject *op, Py_ssize_t newsize) - Resize an object allocated by :c:func:`PyObject_NewVar`. Returns the + Resize an object allocated by :c:macro:`PyObject_NewVar`. Returns the resized object or ``NULL`` on failure. *op* must not be tracked by the collector yet. @@ -128,8 +128,8 @@ rules: .. c:function:: void PyObject_GC_Del(void *op) - Releases memory allocated to an object using :c:func:`PyObject_GC_New` or - :c:func:`PyObject_GC_NewVar`. + Releases memory allocated to an object using :c:macro:`PyObject_GC_New` or + :c:macro:`PyObject_GC_NewVar`. .. c:function:: void PyObject_GC_UnTrack(void *op) diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 2ed0984dbc9df..dd599b6687e88 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -151,10 +151,10 @@ Importing Modules The module's :attr:`__spec__` and :attr:`__loader__` will be set, if not set already, with the appropriate values. The spec's loader will be set to the module's ``__loader__`` (if set) and to an instance of - :class:`SourceFileLoader` otherwise. + :class:`~importlib.machinery.SourceFileLoader` otherwise. The module's :attr:`__file__` attribute will be set to the code object's - :attr:`co_filename`. If applicable, :attr:`__cached__` will also + :attr:`!co_filename`. If applicable, :attr:`__cached__` will also be set. This function will reload the module if it was already imported. See @@ -241,7 +241,7 @@ Importing Modules .. c:function:: PyObject* PyImport_GetImporter(PyObject *path) - Return a finder object for a :data:`sys.path`/:attr:`pkg.__path__` item + Return a finder object for a :data:`sys.path`/:attr:`!pkg.__path__` item *path*, possibly by fetching it from the :data:`sys.path_importer_cache` dict. If it wasn't yet cached, traverse :data:`sys.path_hooks` until a hook is found that can handle the path item. Return ``None`` if no hook could; diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 48cea69c96940..8d495da26e34b 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -25,7 +25,7 @@ The following functions can be safely called before Python is initialized: * :c:func:`PyImport_AppendInittab` * :c:func:`PyImport_ExtendInittab` - * :c:func:`PyInitFrozenExtensions` + * :c:func:`!PyInitFrozenExtensions` * :c:func:`PyMem_SetAllocator` * :c:func:`PyMem_SetupDebugHooks` * :c:func:`PyObject_SetArenaAllocator` @@ -151,7 +151,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. :c:member:`PyConfig.use_environment` should be used instead, see :ref:`Python Initialization Configuration `. - Ignore all :envvar:`PYTHON*` environment variables, e.g. + Ignore all :envvar:`!PYTHON*` environment variables, e.g. :envvar:`PYTHONPATH` and :envvar:`PYTHONHOME`, that might be set. Set by the :option:`-E` and :option:`-I` options. @@ -224,7 +224,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. :ref:`Python Initialization Configuration `. If the flag is non-zero, use :class:`io.FileIO` instead of - :class:`WindowsConsoleIO` for :mod:`sys` standard streams. + :class:`!io._WindowsConsoleIO` for :mod:`sys` standard streams. Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSSTDIO` environment variable is set to a non-empty string. @@ -393,7 +393,7 @@ Initializing and finalizing the interpreter the application. **Bugs and caveats:** The destruction of modules and objects in modules is done - in random order; this may cause destructors (:meth:`__del__` methods) to fail + in random order; this may cause destructors (:meth:`~object.__del__` methods) to fail when they depend on other objects (even functions) or modules. Dynamically loaded extension modules loaded by Python are not unloaded. Small amounts of memory allocated by the Python interpreter may not be freed (if you find a leak, @@ -417,7 +417,7 @@ Process-wide parameters ======================= -.. c:function:: wchar* Py_GetProgramName() +.. c:function:: wchar_t* Py_GetProgramName() Return the program name set with :c:member:`PyConfig.program_name`, or the default. The returned string points into static storage; the caller should not modify its @@ -785,7 +785,7 @@ the fork, and releasing them afterwards. In addition, it resets any :ref:`lock-objects` in the child. When extending or embedding Python, there is no way to inform Python of additional (non-Python) locks that need to be acquired before or reset after a fork. OS facilities such as -:c:func:`pthread_atfork` would need to be used to accomplish the same thing. +:c:func:`!pthread_atfork` would need to be used to accomplish the same thing. Additionally, when extending or embedding Python, calling :c:func:`fork` directly rather than through :func:`os.fork` (and returning to or calling into Python) may result in a deadlock by one of Python's internal locks @@ -849,7 +849,7 @@ code, or when embedding the Python interpreter: .. note:: Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. - You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to + You can use :c:func:`!_Py_IsFinalizing` or :func:`sys.is_finalizing` to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination. @@ -895,7 +895,7 @@ with sub-interpreters: .. note:: Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. - You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to + You can use :c:func:`!_Py_IsFinalizing` or :func:`sys.is_finalizing` to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination. @@ -1177,7 +1177,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. note:: Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. - You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to + You can use :c:func:`!_Py_IsFinalizing` or :func:`sys.is_finalizing` to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination. diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 0c2e707c94c8d..e3d74cc2b2d6b 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -889,7 +889,7 @@ PyConfig .. c:member:: int legacy_windows_stdio If non-zero, use :class:`io.FileIO` instead of - :class:`io.WindowsConsoleIO` for :data:`sys.stdin`, :data:`sys.stdout` + :class:`!io._WindowsConsoleIO` for :data:`sys.stdin`, :data:`sys.stdout` and :data:`sys.stderr`. Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSSTDIO` environment @@ -1139,7 +1139,7 @@ PyConfig Set to ``0`` by the :option:`-S` command line option. - :data:`sys.flags.no_site` is set to the inverted value of + :data:`sys.flags.no_site ` is set to the inverted value of :c:member:`~PyConfig.site_import`. Default: ``1``. diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 4ca3b8804427f..c51aba3f55536 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -136,7 +136,7 @@ need to be held. The :ref:`default raw memory allocator ` uses the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` -and :c:func:`free`; call ``malloc(1)`` (or ``calloc(1, 1)``) when requesting +and :c:func:`!free`; call ``malloc(1)`` (or ``calloc(1, 1)``) when requesting zero bytes. .. versionadded:: 3.4 @@ -264,14 +264,14 @@ The following type-oriented macros are provided for convenience. Note that *TYPE* refers to any C type. -.. c:function:: TYPE* PyMem_New(TYPE, size_t n) +.. c:macro:: PyMem_New(TYPE, n) Same as :c:func:`PyMem_Malloc`, but allocates ``(n * sizeof(TYPE))`` bytes of memory. Returns a pointer cast to :c:expr:`TYPE*`. The memory will not have been initialized in any way. -.. c:function:: TYPE* PyMem_Resize(void *p, TYPE, size_t n) +.. c:macro:: PyMem_Resize(p, TYPE, n) Same as :c:func:`PyMem_Realloc`, but the memory block is resized to ``(n * sizeof(TYPE))`` bytes. Returns a pointer cast to :c:expr:`TYPE*`. On return, @@ -423,7 +423,7 @@ Customize Memory Allocators +----------------------------------------------------------+---------------------------------------+ .. versionchanged:: 3.5 - The :c:type:`PyMemAllocator` structure was renamed to + The :c:type:`!PyMemAllocator` structure was renamed to :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added. @@ -627,8 +627,8 @@ with a fixed size of 256 KiB. It falls back to :c:func:`PyMem_RawMalloc` and The arena allocator uses the following functions: -* :c:func:`VirtualAlloc` and :c:func:`VirtualFree` on Windows, -* :c:func:`mmap` and :c:func:`munmap` if available, +* :c:func:`!VirtualAlloc` and :c:func:`!VirtualFree` on Windows, +* :c:func:`!mmap` and :c:func:`!munmap` if available, * :c:func:`malloc` and :c:func:`free` otherwise. This allocator is disabled if Python is configured with the @@ -732,8 +732,8 @@ allocators operating on different heaps. :: free(buf1); /* Fatal -- should be PyMem_Del() */ In addition to the functions aimed at handling raw memory blocks from the Python -heap, objects in Python are allocated and released with :c:func:`PyObject_New`, -:c:func:`PyObject_NewVar` and :c:func:`PyObject_Del`. +heap, objects in Python are allocated and released with :c:macro:`PyObject_New`, +:c:macro:`PyObject_NewVar` and :c:func:`PyObject_Del`. These will be explained in the next chapter on defining and implementing new object types in C. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index 3ecaeee22d7fd..ad51e7b9b3230 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -282,7 +282,7 @@ An alternate way to specify extensions is to request "multi-phase initialization Extension modules created this way behave more like Python modules: the initialization is split between the *creation phase*, when the module object is created, and the *execution phase*, when it is populated. -The distinction is similar to the :py:meth:`__new__` and :py:meth:`__init__` methods +The distinction is similar to the :py:meth:`!__new__` and :py:meth:`!__init__` methods of classes. Unlike modules created using single-phase initialization, these modules are not @@ -293,7 +293,7 @@ By default, multiple modules created from the same definition should be independent: changes to one should not affect the others. This means that all state should be specific to the module object (using e.g. using :c:func:`PyModule_GetState`), or its contents (such as the module's -:attr:`__dict__` or individual classes created with :c:func:`PyType_FromSpec`). +:attr:`~object.__dict__` or individual classes created with :c:func:`PyType_FromSpec`). All modules created using multi-phase initialization are expected to support :ref:`sub-interpreters `. Making sure multiple modules @@ -552,7 +552,7 @@ state: ``NULL``-terminated. Return ``-1`` on error, ``0`` on success. -.. c:function:: int PyModule_AddIntMacro(PyObject *module, macro) +.. c:macro:: PyModule_AddIntMacro(module, macro) Add an int constant to *module*. The name and the value are taken from *macro*. For example ``PyModule_AddIntMacro(module, AF_INET)`` adds the int @@ -560,7 +560,7 @@ state: Return ``-1`` on error, ``0`` on success. -.. c:function:: int PyModule_AddStringMacro(PyObject *module, macro) +.. c:macro:: PyModule_AddStringMacro(module, macro) Add a string constant to *module*. diff --git a/Doc/c-api/set.rst b/Doc/c-api/set.rst index d642a5f1902e2..7e0ebd2f791a4 100644 --- a/Doc/c-api/set.rst +++ b/Doc/c-api/set.rst @@ -122,7 +122,7 @@ or :class:`frozenset` or instances of their subtypes. .. c:function:: int PySet_Contains(PyObject *anyset, PyObject *key) Return ``1`` if found, ``0`` if not found, and ``-1`` if an error is encountered. Unlike - the Python :meth:`__contains__` method, this function does not automatically + the Python :meth:`~object.__contains__` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise a :exc:`TypeError` if the *key* is unhashable. Raise :exc:`PyExc_SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 75e325e8fa83f..cdf09199ecbb5 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -393,7 +393,7 @@ definition with the same method name. *METH_COEXIST*, the default is to skip repeated definitions. Since slot wrappers are loaded before the method table, the existence of a *sq_contains* slot, for example, would generate a wrapped method named - :meth:`__contains__` and preclude the loading of a corresponding + :meth:`~object.__contains__` and preclude the loading of a corresponding PyCFunction with the same name. With the flag defined, the PyCFunction will be loaded in place of the wrapper object and will co-exist with the slot. This is helpful because calls to PyCFunctions are optimized more diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 6a717122c8a2f..995fca4503ccd 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -363,7 +363,7 @@ Process Control This function should only be invoked when a condition is detected that would make it dangerous to continue using the Python interpreter; e.g., when the object administration appears to be corrupted. On Unix, the standard C library - function :c:func:`abort` is called which will attempt to produce a :file:`core` + function :c:func:`!abort` is called which will attempt to produce a :file:`core` file. The ``Py_FatalError()`` function is replaced with a macro which logs diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 2b21a349ccc4c..7abb4774fb11f 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -579,7 +579,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) name, followed by a dot, followed by the type name; for built-in types, it should be just the type name. If the module is a submodule of a package, the full package name is part of the full module name. For example, a type named - :class:`T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` + :class:`!T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``. For :ref:`dynamically allocated type objects `, @@ -673,9 +673,9 @@ and :c:data:`PyType_Type` effectively act as defaults.) permissible to call the object deallocator directly instead of via :c:member:`~PyTypeObject.tp_free`. The object deallocator should be the one used to allocate the instance; this is normally :c:func:`PyObject_Del` if the instance was allocated - using :c:func:`PyObject_New` or :c:func:`PyObject_VarNew`, or + using :c:macro:`PyObject_New` or :c:macro:`PyObject_NewVar`, or :c:func:`PyObject_GC_Del` if the instance was allocated using - :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar`. + :c:macro:`PyObject_GC_New` or :c:macro:`PyObject_GC_NewVar`. If the type supports garbage collection (has the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit set), the destructor should call :c:func:`PyObject_GC_UnTrack` @@ -1092,7 +1092,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) .. c:macro:: Py_TPFLAGS_HAVE_GC This bit is set when the object supports garbage collection. If this bit - is set, instances must be created using :c:func:`PyObject_GC_New` and + is set, instances must be created using :c:macro:`PyObject_GC_New` and destroyed using :c:func:`PyObject_GC_Del`. More information in section :ref:`supporting-cycle-detection`. This bit also implies that the GC-related fields :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` are present in @@ -1180,7 +1180,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) Indicates that the variable-sized portion of an instance of this type is at the end of the instance's memory area, at an offset of - :c:expr:`Py_TYPE(obj)->tp_basicsize` (which may be different in each + ``Py_TYPE(obj)->tp_basicsize`` (which may be different in each subclass). When setting this flag, be sure that all superclasses either diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index c7f5f0f9e230d..e875cf6ded655 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -601,7 +601,7 @@ APIs: Py_ssize_t how_many) Copy characters from one Unicode object into another. This function performs - character conversion when necessary and falls back to :c:func:`memcpy` if + character conversion when necessary and falls back to :c:func:`!memcpy` if possible. Returns ``-1`` and sets an exception on error, otherwise returns the number of copied characters. @@ -714,7 +714,7 @@ system. .. c:function:: PyObject* PyUnicode_DecodeLocale(const char *str, const char *errors) Similar to :c:func:`PyUnicode_DecodeLocaleAndSize`, but compute the string - length using :c:func:`strlen`. + length using :c:func:`!strlen`. .. versionadded:: 3.3 @@ -872,7 +872,7 @@ wchar_t Support most C functions. If *size* is ``NULL`` and the :c:expr:`wchar_t*` string contains null characters a :exc:`ValueError` is raised. - Returns a buffer allocated by :c:func:`PyMem_New` (use + Returns a buffer allocated by :c:macro:`PyMem_New` (use :c:func:`PyMem_Free` to free it) on success. On error, returns ``NULL`` and *\*size* is undefined. Raises a :exc:`MemoryError` if memory allocation is failed. diff --git a/Doc/conf.py b/Doc/conf.py index 93c9bd121cc1e..1b30b862f9a86 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -97,7 +97,7 @@ ('c:func', 'vsnprintf'), # Standard C types ('c:type', 'FILE'), - ('c:type', '__int'), + ('c:type', 'int64_t'), ('c:type', 'intmax_t'), ('c:type', 'off_t'), ('c:type', 'ptrdiff_t'), @@ -105,9 +105,12 @@ ('c:type', 'size_t'), ('c:type', 'ssize_t'), ('c:type', 'time_t'), + ('c:type', 'uint64_t'), ('c:type', 'uintmax_t'), + ('c:type', 'uintptr_t'), ('c:type', 'va_list'), ('c:type', 'wchar_t'), + # Standard C structures ('c:struct', 'in6_addr'), ('c:struct', 'in_addr'), ('c:struct', 'stat'), diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 94ead2b60f8d6..b9cfc9b4474d9 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -230,7 +230,7 @@ with an exception object:: return m; } -Note that the Python name for the exception object is :exc:`spam.error`. The +Note that the Python name for the exception object is :exc:`!spam.error`. The :c:func:`PyErr_NewException` function may create a class with the base class being :exc:`Exception` (unless another class is passed in instead of ``NULL``), described in :ref:`bltin-exceptions`. @@ -245,7 +245,7 @@ raises the exception could cause a core dump or other unintended side effects. We discuss the use of ``PyMODINIT_FUNC`` as a function return type later in this sample. -The :exc:`spam.error` exception can be raised in your extension module using a +The :exc:`!spam.error` exception can be raised in your extension module using a call to :c:func:`PyErr_SetString` as shown below:: static PyObject * @@ -315,7 +315,7 @@ contexts, as we have seen. The Module's Method Table and Initialization Function ===================================================== -I promised to show how :c:func:`spam_system` is called from Python programs. +I promised to show how :c:func:`!spam_system` is called from Python programs. First, we need to list its name and address in a "method table":: static PyMethodDef SpamMethods[] = { @@ -1041,13 +1041,13 @@ Let's follow the control flow into :c:func:`PyList_SetItem`. The list owns references to all its items, so when item 1 is replaced, it has to dispose of the original item 1. Now let's suppose the original item 1 was an instance of a user-defined class, and let's further suppose that the class defined a -:meth:`__del__` method. If this class instance has a reference count of 1, -disposing of it will call its :meth:`__del__` method. +:meth:`!__del__` method. If this class instance has a reference count of 1, +disposing of it will call its :meth:`!__del__` method. -Since it is written in Python, the :meth:`__del__` method can execute arbitrary +Since it is written in Python, the :meth:`!__del__` method can execute arbitrary Python code. Could it perhaps do something to invalidate the reference to -``item`` in :c:func:`bug`? You bet! Assuming that the list passed into -:c:func:`bug` is accessible to the :meth:`__del__` method, it could execute a +``item`` in :c:func:`!bug`? You bet! Assuming that the list passed into +:c:func:`!bug` is accessible to the :meth:`!__del__` method, it could execute a statement to the effect of ``del list[0]``, and assuming this was the last reference to that object, it would free the memory associated with it, thereby invalidating ``item``. @@ -1068,7 +1068,7 @@ increment the reference count. The correct version of the function reads:: This is a true story. An older version of Python contained variants of this bug and someone spent a considerable amount of time in a C debugger to figure out -why his :meth:`__del__` methods would fail... +why his :meth:`!__del__` methods would fail... The second case of problems with a borrowed reference is a variant involving threads. Normally, multiple threads in the Python interpreter can't get in each @@ -1235,7 +1235,7 @@ The function :c:func:`PySpam_System` is a plain C function, declared return system(command); } -The function :c:func:`spam_system` is modified in a trivial way:: +The function :c:func:`!spam_system` is modified in a trivial way:: static PyObject * spam_system(PyObject *self, PyObject *args) diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index a05bae8e07cd3..67382dfcb759b 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -36,7 +36,7 @@ So, if you want to define a new extension type, you need to create a new type object. This sort of thing can only be explained by example, so here's a minimal, but -complete, module that defines a new type named :class:`Custom` inside a C +complete, module that defines a new type named :class:`!Custom` inside a C extension module :mod:`!custom`: .. note:: @@ -50,9 +50,9 @@ extension module :mod:`!custom`: Now that's quite a bit to take in at once, but hopefully bits will seem familiar from the previous chapter. This file defines three things: -#. What a :class:`Custom` **object** contains: this is the ``CustomObject`` - struct, which is allocated once for each :class:`Custom` instance. -#. How the :class:`Custom` **type** behaves: this is the ``CustomType`` struct, +#. What a :class:`!Custom` **object** contains: this is the ``CustomObject`` + struct, which is allocated once for each :class:`!Custom` instance. +#. How the :class:`!Custom` **type** behaves: this is the ``CustomType`` struct, which defines a set of flags and function pointers that the interpreter inspects when specific operations are requested. #. How to initialize the :mod:`!custom` module: this is the ``PyInit_custom`` @@ -128,7 +128,7 @@ our objects and in some error messages, for example: Note that the name is a dotted name that includes both the module name and the name of the type within the module. The module in this case is :mod:`!custom` and -the type is :class:`Custom`, so we set the type name to :class:`custom.Custom`. +the type is :class:`!Custom`, so we set the type name to :class:`!custom.Custom`. Using the real dotted import path is important to make your type compatible with the :mod:`pydoc` and :mod:`pickle` modules. :: @@ -136,7 +136,7 @@ with the :mod:`pydoc` and :mod:`pickle` modules. :: .tp_itemsize = 0, This is so that Python knows how much memory to allocate when creating -new :class:`Custom` instances. :c:member:`~PyTypeObject.tp_itemsize` is +new :class:`!Custom` instances. :c:member:`~PyTypeObject.tp_itemsize` is only used for variable-sized objects and should otherwise be zero. .. note:: @@ -176,7 +176,7 @@ Everything else in the file should be familiar, except for some code in if (PyType_Ready(&CustomType) < 0) return; -This initializes the :class:`Custom` type, filling in a number of members +This initializes the :class:`!Custom` type, filling in a number of members to the appropriate default values, including :attr:`ob_type` that we initially set to ``NULL``. :: @@ -186,7 +186,7 @@ set to ``NULL``. :: } This adds the type to the module dictionary. This allows us to create -:class:`Custom` instances by calling the :class:`Custom` class: +:class:`!Custom` instances by calling the :class:`!Custom` class: .. code-block:: pycon @@ -237,7 +237,7 @@ adds these capabilities: This version of the module has a number of changes. -The :class:`Custom` type now has three data attributes in its C struct, +The :class:`!Custom` type now has three data attributes in its C struct, *first*, *last*, and *number*. The *first* and *last* variables are Python strings containing first and last names. The *number* attribute is a C integer. @@ -312,7 +312,7 @@ The ``tp_new`` handler is responsible for creating (as opposed to initializing) objects of the type. It is exposed in Python as the :meth:`__new__` method. It is not required to define a ``tp_new`` member, and indeed many extension types will simply reuse :c:func:`PyType_GenericNew` as done in the first -version of the ``Custom`` type above. In this case, we use the ``tp_new`` +version of the :class:`!Custom` type above. In this case, we use the ``tp_new`` handler to initialize the ``first`` and ``last`` attributes to non-``NULL`` default values. @@ -451,7 +451,7 @@ Further, the attributes can be deleted, setting the C pointers to ``NULL``. Eve though we can make sure the members are initialized to non-``NULL`` values, the members can be set to ``NULL`` if the attributes are deleted. -We define a single method, :meth:`Custom.name()`, that outputs the objects name as the +We define a single method, :meth:`!Custom.name()`, that outputs the objects name as the concatenation of the first and last names. :: static PyObject * @@ -468,8 +468,8 @@ concatenation of the first and last names. :: return PyUnicode_FromFormat("%S %S", self->first, self->last); } -The method is implemented as a C function that takes a :class:`Custom` (or -:class:`Custom` subclass) instance as the first argument. Methods always take an +The method is implemented as a C function that takes a :class:`!Custom` (or +:class:`!Custom` subclass) instance as the first argument. Methods always take an instance as the first argument. Methods often take positional and keyword arguments as well, but in this case we don't take any and don't need to accept a positional argument tuple or keyword argument dictionary. This method is @@ -480,8 +480,8 @@ equivalent to the Python method: def name(self): return "%s %s" % (self.first, self.last) -Note that we have to check for the possibility that our :attr:`first` and -:attr:`last` members are ``NULL``. This is because they can be deleted, in which +Note that we have to check for the possibility that our :attr:`!first` and +:attr:`!last` members are ``NULL``. This is because they can be deleted, in which case they are set to ``NULL``. It would be better to prevent deletion of these attributes and to restrict the attribute values to be strings. We'll see how to do that in the next section. @@ -510,7 +510,7 @@ to add the :c:macro:`Py_TPFLAGS_BASETYPE` to our class flag definition:: .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, -We rename :c:func:`PyInit_custom` to :c:func:`PyInit_custom2`, update the +We rename :c:func:`!PyInit_custom` to :c:func:`!PyInit_custom2`, update the module name in the :c:type:`PyModuleDef` struct, and update the full class name in the :c:type:`PyTypeObject` struct. @@ -529,18 +529,18 @@ Finally, we update our :file:`setup.py` file to build the new module: Providing finer control over data attributes ============================================ -In this section, we'll provide finer control over how the :attr:`first` and -:attr:`last` attributes are set in the :class:`Custom` example. In the previous -version of our module, the instance variables :attr:`first` and :attr:`last` +In this section, we'll provide finer control over how the :attr:`!first` and +:attr:`!last` attributes are set in the :class:`!Custom` example. In the previous +version of our module, the instance variables :attr:`!first` and :attr:`!last` could be set to non-string values or even deleted. We want to make sure that these attributes always contain strings. .. literalinclude:: ../includes/custom3.c -To provide greater control, over the :attr:`first` and :attr:`last` attributes, +To provide greater control, over the :attr:`!first` and :attr:`!last` attributes, we'll use custom getter and setter functions. Here are the functions for -getting and setting the :attr:`first` attribute:: +getting and setting the :attr:`!first` attribute:: static PyObject * Custom_getfirst(CustomObject *self, void *closure) @@ -569,13 +569,13 @@ getting and setting the :attr:`first` attribute:: return 0; } -The getter function is passed a :class:`Custom` object and a "closure", which is +The getter function is passed a :class:`!Custom` object and a "closure", which is a void pointer. In this case, the closure is ignored. (The closure supports an advanced usage in which definition data is passed to the getter and setter. This could, for example, be used to allow a single set of getter and setter functions that decide the attribute to get or set based on data in the closure.) -The setter function is passed the :class:`Custom` object, the new value, and the +The setter function is passed the :class:`!Custom` object, the new value, and the closure. The new value may be ``NULL``, in which case the attribute is being deleted. In our setter, we raise an error if the attribute is deleted or if its new value is not a string. @@ -664,11 +664,11 @@ still has a reference from itself. Its reference count doesn't drop to zero. Fortunately, Python's cyclic garbage collector will eventually figure out that the list is garbage and free it. -In the second version of the :class:`Custom` example, we allowed any kind of -object to be stored in the :attr:`first` or :attr:`last` attributes [#]_. +In the second version of the :class:`!Custom` example, we allowed any kind of +object to be stored in the :attr:`!first` or :attr:`!last` attributes [#]_. Besides, in the second and third versions, we allowed subclassing -:class:`Custom`, and subclasses may add arbitrary attributes. For any of -those two reasons, :class:`Custom` objects can participate in cycles: +:class:`!Custom`, and subclasses may add arbitrary attributes. For any of +those two reasons, :class:`!Custom` objects can participate in cycles: .. code-block:: pycon @@ -678,8 +678,8 @@ those two reasons, :class:`Custom` objects can participate in cycles: >>> n = Derived() >>> n.some_attribute = n -To allow a :class:`Custom` instance participating in a reference cycle to -be properly detected and collected by the cyclic GC, our :class:`Custom` type +To allow a :class:`!Custom` instance participating in a reference cycle to +be properly detected and collected by the cyclic GC, our :class:`!Custom` type needs to fill two additional slots and to enable a flag that enables these slots: .. literalinclude:: ../includes/custom4.c @@ -809,7 +809,7 @@ increases an internal counter: .. literalinclude:: ../includes/sublist.c -As you can see, the source code closely resembles the :class:`Custom` examples in +As you can see, the source code closely resembles the :class:`!Custom` examples in previous sections. We will break down the main differences between them. :: typedef struct { @@ -875,7 +875,7 @@ slot with :c:func:`PyType_GenericNew` -- the allocation function from the base type will be inherited. After that, calling :c:func:`PyType_Ready` and adding the type object to the -module is the same as with the basic :class:`Custom` examples. +module is the same as with the basic :class:`!Custom` examples. .. rubric:: Footnotes diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 1ea7c791f6753..cb98e97668d5c 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -2,12 +2,9 @@ # as tested on the CI via check-warnings.py in reusable-docs.yml. # Keep lines sorted lexicographically to help avoid merge conflicts. -Doc/c-api/allocation.rst Doc/c-api/bool.rst Doc/c-api/buffer.rst Doc/c-api/capsule.rst -Doc/c-api/complex.rst -Doc/c-api/conversion.rst Doc/c-api/datetime.rst Doc/c-api/descriptor.rst Doc/c-api/exceptions.rst diff --git a/Doc/whatsnew/2.3.rst b/Doc/whatsnew/2.3.rst index c1722e5ab56fc..a96c1061455e0 100644 --- a/Doc/whatsnew/2.3.rst +++ b/Doc/whatsnew/2.3.rst @@ -1847,7 +1847,7 @@ specifically for allocating Python objects. :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`, and :c:func:`PyObject_Free`. * To allocate and free Python objects, use the "object" family - :c:func:`PyObject_New`, :c:func:`PyObject_NewVar`, and :c:func:`PyObject_Del`. + :c:macro:`PyObject_New`, :c:macro:`PyObject_NewVar`, and :c:func:`PyObject_Del`. Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides debugging features to catch memory overwrites and doubled frees in both extension modules diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index f50c43f724b6b..a57f9bcba6189 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -2061,8 +2061,8 @@ Changes in the C API * Remove :c:macro:`Py_INCREF` on the type object after allocating an instance - if any. - This may happen after calling :c:func:`PyObject_New`, - :c:func:`PyObject_NewVar`, :c:func:`PyObject_GC_New`, + This may happen after calling :c:macro:`PyObject_New`, + :c:macro:`PyObject_NewVar`, :c:func:`PyObject_GC_New`, :c:func:`PyObject_GC_NewVar`, or any other custom allocator that uses :c:func:`PyObject_Init` or :c:func:`PyObject_INIT`. diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index e48cc13eff617..3e3e0ff5c41f4 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1389,8 +1389,8 @@ Porting to Python 3.9 * :c:func:`PyObject_IS_GC` macro was converted to a function. * The :c:func:`PyObject_NEW` macro becomes an alias to the - :c:func:`PyObject_New` macro, and the :c:func:`PyObject_NEW_VAR` macro - becomes an alias to the :c:func:`PyObject_NewVar` macro. They no longer + :c:macro:`PyObject_New` macro, and the :c:func:`PyObject_NEW_VAR` macro + becomes an alias to the :c:macro:`PyObject_NewVar` macro. They no longer access directly the :c:member:`PyTypeObject.tp_basicsize` member. * :c:func:`PyObject_GET_WEAKREFS_LISTPTR` macro was converted to a function: From webhook-mailer at python.org Thu Jul 27 01:52:58 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 27 Jul 2023 05:52:58 -0000 Subject: [Python-checkins] gh-107091: Fix some uses of :attr: role (GH-107318) Message-ID: https://github.com/python/cpython/commit/d363eb5b0255c055e7b43f5e2c0847f555e1982e commit: d363eb5b0255c055e7b43f5e2c0847f555e1982e branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-27T08:52:54+03:00 summary: gh-107091: Fix some uses of :attr: role (GH-107318) Fix also formatting of PyMethodDef members. files: M Doc/c-api/import.rst M Doc/c-api/init.rst M Doc/c-api/structures.rst M Doc/c-api/tuple.rst M Doc/c-api/typeobj.rst M Doc/c-api/veryhigh.rst M Doc/extending/newtypes.rst M Doc/extending/newtypes_tutorial.rst M Doc/tools/.nitignore M Doc/whatsnew/2.5.rst M Misc/NEWS.d/next/Core and Builtins/2023-05-30-08-09-43.gh-issue-105035.OWUlHy.rst diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index dd599b6687e88..137780cc359cf 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -310,23 +310,25 @@ Importing Modules .. c:struct:: _inittab - Structure describing a single entry in the list of built-in modules. Each of - these structures gives the name and initialization function for a module built - into the interpreter. The name is an ASCII encoded string. Programs which + Structure describing a single entry in the list of built-in modules. + Programs which embed Python may use an array of these structures in conjunction with :c:func:`PyImport_ExtendInittab` to provide additional built-in modules. - The structure is defined in :file:`Include/import.h` as:: + The structure consists of two members: - struct _inittab { - const char *name; /* ASCII encoded string */ - PyObject* (*initfunc)(void); - }; + .. c:member:: const char *name + + The module name, as an ASCII encoded string. + + .. c: member:: PyObject* (*initfunc)(void) + + Initialization function for a module built into the interpreter. .. c:function:: int PyImport_ExtendInittab(struct _inittab *newtab) Add a collection of modules to the table of built-in modules. The *newtab* - array must end with a sentinel entry which contains ``NULL`` for the :attr:`name` + array must end with a sentinel entry which contains ``NULL`` for the :c:member:`~_inittab.name` field; failure to provide the sentinel value can result in a memory fault. Returns ``0`` on success or ``-1`` if insufficient memory could be allocated to extend the internal table. In the event of failure, no modules are added to the diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 8d495da26e34b..a447eefc55e44 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -827,8 +827,11 @@ code, or when embedding the Python interpreter: .. c:type:: PyThreadState This data structure represents the state of a single thread. The only public - data member is :attr:`interp` (:c:expr:`PyInterpreterState *`), which points to - this thread's interpreter state. + data member is: + + .. c:member:: PyInterpreterState *interp + + This thread's interpreter state. .. c:function:: PyThreadState* PyEval_SaveThread() diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index cdf09199ecbb5..747cfa62294c2 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -35,7 +35,7 @@ under :ref:`reference counting `. .. c:type:: PyVarObject - This is an extension of :c:type:`PyObject` that adds the :attr:`ob_size` + This is an extension of :c:type:`PyObject` that adds the :c:member:`~PyVarObject.ob_size` field. This is only used for objects that have some notion of *length*. This type does not often appear in the Python/C API. Access to the members must be done by using the macros @@ -152,7 +152,7 @@ under :ref:`reference counting `. .. c:macro:: PyVarObject_HEAD_INIT(type, size) This is a macro which expands to initialization values for a new - :c:type:`PyVarObject` type, including the :attr:`ob_size` field. + :c:type:`PyVarObject` type, including the :c:member:`~PyVarObject.ob_size` field. This macro expands to:: _PyObject_EXTRA_INIT @@ -228,21 +228,21 @@ Implementing functions and methods Structure used to describe a method of an extension type. This structure has four fields: - .. c:member:: const char* ml_name + .. c:member:: const char *ml_name - name of the method + Name of the method. .. c:member:: PyCFunction ml_meth - pointer to the C implementation + Pointer to the C implementation. .. c:member:: int ml_flags - flags bits indicating how the call should be constructed + Flags bits indicating how the call should be constructed. - .. c:member:: const char* ml_doc + .. c:member:: const char *ml_doc - points to the contents of the docstring + Points to the contents of the docstring. The :c:member:`~PyMethodDef.ml_meth` is a C function pointer. The functions may be of different diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index 2277a39c95906..d1c9521944705 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -167,7 +167,8 @@ type. Describes a field of a struct sequence. As a struct sequence is modeled as a tuple, all fields are typed as :c:expr:`PyObject*`. The index in the - :attr:`fields` array of the :c:type:`PyStructSequence_Desc` determines which + :c:member:`~PyStructSequence_Desc.fields` array of + the :c:type:`PyStructSequence_Desc` determines which field of the struct sequence is described. +-----------+------------------+-----------------------------------------+ diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 7abb4774fb11f..04f426d982bbb 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -485,17 +485,17 @@ PyObject Slots -------------- The type object structure extends the :c:type:`PyVarObject` structure. The -:attr:`ob_size` field is used for dynamic types (created by :func:`type_new`, +:c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :func:`type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. -type objects) *must* have the :attr:`ob_size` field. +type objects) *must* have the :c:member:`~PyVarObject.ob_size` field. .. c:member:: Py_ssize_t PyObject.ob_refcnt This is the type object's reference count, initialized to ``1`` by the ``PyObject_HEAD_INIT`` macro. Note that for :ref:`statically allocated type - objects `, the type's instances (objects whose :attr:`ob_type` + objects `, the type's instances (objects whose :c:member:`~PyObject.ob_type` points back to the type) do *not* count as references. But for :ref:`dynamically allocated type objects `, the instances *do* count as references. @@ -519,8 +519,8 @@ type objects) *must* have the :attr:`ob_size` field. Foo_Type.ob_type = &PyType_Type; This should be done before any instances of the type are created. - :c:func:`PyType_Ready` checks if :attr:`ob_type` is ``NULL``, and if so, - initializes it to the :attr:`ob_type` field of the base class. + :c:func:`PyType_Ready` checks if :c:member:`~PyObject.ob_type` is ``NULL``, and if so, + initializes it to the :c:member:`~PyObject.ob_type` field of the base class. :c:func:`PyType_Ready` will not change this field if it is non-zero. **Inheritance:** @@ -619,20 +619,20 @@ and :c:data:`PyType_Type` effectively act as defaults.) instances have the same size, given in :c:member:`~PyTypeObject.tp_basicsize`. For a type with variable-length instances, the instances must have an - :attr:`ob_size` field, and the instance size is :c:member:`~PyTypeObject.tp_basicsize` plus N + :c:member:`~PyVarObject.ob_size` field, and the instance size is :c:member:`~PyTypeObject.tp_basicsize` plus N times :c:member:`~PyTypeObject.tp_itemsize`, where N is the "length" of the object. The value of - N is typically stored in the instance's :attr:`ob_size` field. There are - exceptions: for example, ints use a negative :attr:`ob_size` to indicate a + N is typically stored in the instance's :c:member:`~PyVarObject.ob_size` field. There are + exceptions: for example, ints use a negative :c:member:`~PyVarObject.ob_size` to indicate a negative number, and N is ``abs(ob_size)`` there. Also, the presence of an - :attr:`ob_size` field in the instance layout doesn't mean that the instance + :c:member:`~PyVarObject.ob_size` field in the instance layout doesn't mean that the instance structure is variable-length (for example, the structure for the list type has - fixed-length instances, yet those instances have a meaningful :attr:`ob_size` + fixed-length instances, yet those instances have a meaningful :c:member:`~PyVarObject.ob_size` field). The basic size includes the fields in the instance declared by the macro :c:macro:`PyObject_HEAD` or :c:macro:`PyObject_VAR_HEAD` (whichever is used to - declare the instance struct) and this in turn includes the :attr:`_ob_prev` and - :attr:`_ob_next` fields if they are present. This means that the only correct + declare the instance struct) and this in turn includes the :c:member:`~PyObject._ob_prev` and + :c:member:`~PyObject._ob_next` fields if they are present. This means that the only correct way to get an initializer for the :c:member:`~PyTypeObject.tp_basicsize` is to use the ``sizeof`` operator on the struct used to declare the instance layout. The basic size does not include the GC header size. @@ -764,7 +764,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_getattr`, :attr:`tp_getattro` + Group: :c:member:`~PyTypeObject.tp_getattr`, :c:member:`~PyTypeObject.tp_getattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattro`: a subtype inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when @@ -781,7 +781,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_setattr`, :attr:`tp_setattro` + Group: :c:member:`~PyTypeObject.tp_setattr`, :c:member:`~PyTypeObject.tp_setattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattro`: a subtype inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when @@ -883,7 +883,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) normal return value; when an error occurs during the computation of the hash value, the function should set an exception and return ``-1``. - When this field is not set (*and* :attr:`tp_richcompare` is not set), + When this field is not set (*and* :c:member:`~PyTypeObject.tp_richcompare` is not set), an attempt to take the hash of the object raises :exc:`TypeError`. This is the same as setting it to :c:func:`PyObject_HashNotImplemented`. @@ -897,7 +897,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_hash`, :attr:`tp_richcompare` + Group: :c:member:`~PyTypeObject.tp_hash`, :c:member:`~PyTypeObject.tp_richcompare` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_richcompare`: a subtype inherits both of @@ -956,7 +956,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_getattr`, :attr:`tp_getattro` + Group: :c:member:`~PyTypeObject.tp_getattr`, :c:member:`~PyTypeObject.tp_getattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattr`: a subtype inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when @@ -982,7 +982,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_setattr`, :attr:`tp_setattro` + Group: :c:member:`~PyTypeObject.tp_setattr`, :c:member:`~PyTypeObject.tp_setattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattr`: a subtype inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when @@ -1047,7 +1047,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) This bit is set when the type object itself is allocated on the heap, for example, types created dynamically using :c:func:`PyType_FromSpec`. In this - case, the :attr:`ob_type` field of its instances is considered a reference to + case, the :c:member:`~PyObject.ob_type` field of its instances is considered a reference to the type, and the type object is INCREF'ed when a new instance is created, and DECREF'ed when an instance is destroyed (this does not apply to instances of subtypes; only the type referenced by the instance's ob_type gets INCREF'ed or @@ -1100,13 +1100,13 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited - together with the :attr:`tp_traverse` and :attr:`tp_clear` + together with the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is - clear in the subtype and the :attr:`tp_traverse` and - :attr:`tp_clear` fields in the subtype exist and have ``NULL`` + clear in the subtype and the :c:member:`~PyTypeObject.tp_traverse` and + :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have ``NULL`` values. @@ -1421,7 +1421,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_clear` and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and @@ -1488,7 +1488,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_traverse` and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and @@ -1547,7 +1547,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_hash`, :attr:`tp_richcompare` + Group: :c:member:`~PyTypeObject.tp_hash`, :c:member:`~PyTypeObject.tp_richcompare` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_hash`: a subtype inherits :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` when @@ -1556,9 +1556,9 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Default:** - :c:data:`PyBaseObject_Type` provides a :attr:`tp_richcompare` + :c:data:`PyBaseObject_Type` provides a :c:member:`~PyTypeObject.tp_richcompare` implementation, which may be inherited. However, if only - :attr:`tp_hash` is defined, not even the inherited function is used + :c:member:`~PyTypeObject.tp_hash` is defined, not even the inherited function is used and instances of the type will not be able to participate in any comparisons. @@ -2374,9 +2374,9 @@ Sequence Object Structures This slot must be filled for the :c:func:`PySequence_Check` function to return ``1``, it can be ``NULL`` otherwise. - Negative indexes are handled as follows: if the :attr:`sq_length` slot is + Negative indexes are handled as follows: if the :c:member:`~PySequenceMethods.sq_length` slot is filled, it is called and the sequence length is used to compute a positive - index which is passed to :attr:`sq_item`. If :attr:`sq_length` is ``NULL``, + index which is passed to :c:member:`~PySequenceMethods.sq_item`. If :c:member:`!sq_length` is ``NULL``, the index is passed as is to the function. .. c:member:: ssizeobjargproc PySequenceMethods.sq_ass_item @@ -2586,8 +2586,8 @@ Slot Type typedefs The purpose of this function is to separate memory allocation from memory initialization. It should return a pointer to a block of memory of adequate length for the instance, suitably aligned, and initialized to zeros, but with - :attr:`ob_refcnt` set to ``1`` and :attr:`ob_type` set to the type argument. If - the type's :c:member:`~PyTypeObject.tp_itemsize` is non-zero, the object's :attr:`ob_size` field + :c:member:`~PyObject.ob_refcnt` set to ``1`` and :c:member:`~PyObject.ob_type` set to the type argument. If + the type's :c:member:`~PyTypeObject.tp_itemsize` is non-zero, the object's :c:member:`~PyVarObject.ob_size` field should be initialized to *nitems* and the length of the allocated memory block should be ``tp_basicsize + nitems*tp_itemsize``, rounded up to a multiple of ``sizeof(void*)``; otherwise, *nitems* is not used and the length of the block diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index a19c274d05f2c..324518c035096 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -353,7 +353,7 @@ the same library that the Python runtime is using. executed, it is passed as ``PyCompilerFlags *flags``. In this case, ``from __future__ import`` can modify *flags*. - Whenever ``PyCompilerFlags *flags`` is ``NULL``, :c:member:`cf_flags` is treated as + Whenever ``PyCompilerFlags *flags`` is ``NULL``, :c:member:`~PyCompilerFlags.cf_flags` is treated as equal to ``0``, and any modification due to ``from __future__ import`` is discarded. @@ -367,7 +367,7 @@ the same library that the Python runtime is using. initialized to ``PY_MINOR_VERSION``. The field is ignored by default, it is used if and only if - ``PyCF_ONLY_AST`` flag is set in *cf_flags*. + ``PyCF_ONLY_AST`` flag is set in :c:member:`~PyCompilerFlags.cf_flags`. .. versionchanged:: 3.8 Added *cf_feature_version* field. diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst index 7f8f8ddaaaccd..25822744125f8 100644 --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -270,7 +270,7 @@ structure:: One entry should be defined for each method provided by the type; no entries are needed for methods inherited from a base type. One additional entry is needed at the end; it is a sentinel that marks the end of the array. The -:attr:`ml_name` field of the sentinel must be ``NULL``. +:c:member:`~PyMethodDef.ml_name` field of the sentinel must be ``NULL``. The second table is used to define attributes which map directly to data stored in the instance. A variety of primitive C types are supported, and access may diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index 67382dfcb759b..0cc366b9c0b51 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -177,7 +177,7 @@ Everything else in the file should be familiar, except for some code in return; This initializes the :class:`!Custom` type, filling in a number of members -to the appropriate default values, including :attr:`ob_type` that we initially +to the appropriate default values, including :c:member:`~PyObject.ob_type` that we initially set to ``NULL``. :: if (PyModule_AddObjectRef(m, "Custom", (PyObject *) &CustomType) < 0) { diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index cb98e97668d5c..08f215f035d0b 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -11,7 +11,6 @@ Doc/c-api/exceptions.rst Doc/c-api/file.rst Doc/c-api/float.rst Doc/c-api/gcsupport.rst -Doc/c-api/import.rst Doc/c-api/init.rst Doc/c-api/init_config.rst Doc/c-api/intro.rst diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index ec4a7eb9a2361..2bb681bddedbb 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -954,7 +954,7 @@ The return value must be either a Python integer or long integer. The interpreter will check that the type returned is correct, and raises a :exc:`TypeError` if this requirement isn't met. -A corresponding :attr:`nb_index` slot was added to the C-level +A corresponding :c:member:`~PyNumberMethods.nb_index` slot was added to the C-level :c:type:`PyNumberMethods` structure to let C extensions implement this protocol. ``PyNumber_Index(obj)`` can be used in extension code to call the :meth:`__index__` function and retrieve its result. diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-05-30-08-09-43.gh-issue-105035.OWUlHy.rst b/Misc/NEWS.d/next/Core and Builtins/2023-05-30-08-09-43.gh-issue-105035.OWUlHy.rst index c0ee2da9d4503..dbfcd658d945d 100644 --- a/Misc/NEWS.d/next/Core and Builtins/2023-05-30-08-09-43.gh-issue-105035.OWUlHy.rst +++ b/Misc/NEWS.d/next/Core and Builtins/2023-05-30-08-09-43.gh-issue-105035.OWUlHy.rst @@ -1,2 +1,2 @@ -Fix :func:`super` calls on types with custom :attr:`tp_getattro` +Fix :func:`super` calls on types with custom :c:member:`~PyTypeObject.tp_getattro` implementation (e.g. meta-types.) From webhook-mailer at python.org Thu Jul 27 02:04:06 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 27 Jul 2023 06:04:06 -0000 Subject: [Python-checkins] gh-107298: Docs: add targets for some :c:member: and :c:macro: references (GH-107316) Message-ID: https://github.com/python/cpython/commit/abec9a1b20b70d8ced401d59fc4f02b331c6568b commit: abec9a1b20b70d8ced401d59fc4f02b331c6568b branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-27T09:04:02+03:00 summary: gh-107298: Docs: add targets for some :c:member: and :c:macro: references (GH-107316) Add targets for PyStructSequence_Desc and PyStructSequence_Field members and macros like Py_EQ. Fix target for Py_RETURN_RICHCOMPARE. files: M Doc/c-api/tuple.rst M Doc/c-api/typeobj.rst M Doc/tools/.nitignore diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index d1c9521944705..b3710560ebe7a 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -147,20 +147,21 @@ type. Contains the meta information of a struct sequence type to create. - +-------------------+------------------------------+--------------------------------------+ - | Field | C Type | Meaning | - +===================+==============================+======================================+ - | ``name`` | ``const char *`` | name of the struct sequence type | - +-------------------+------------------------------+--------------------------------------+ - | ``doc`` | ``const char *`` | pointer to docstring for the type | - | | | or ``NULL`` to omit | - +-------------------+------------------------------+--------------------------------------+ - | ``fields`` | ``PyStructSequence_Field *`` | pointer to ``NULL``-terminated array | - | | | with field names of the new type | - +-------------------+------------------------------+--------------------------------------+ - | ``n_in_sequence`` | ``int`` | number of fields visible to the | - | | | Python side (if used as tuple) | - +-------------------+------------------------------+--------------------------------------+ + .. c:member:: const char *name + + Name of the struct sequence type. + + .. c:member:: const char *doc + + Pointer to docstring for the type or ``NULL`` to omit. + + .. c:member:: PyStructSequence_Field *fields + + Pointer to ``NULL``-terminated array with field names of the new type. + + .. c:member:: int n_in_sequence + + Number of fields visible to the Python side (if used as tuple). .. c:type:: PyStructSequence_Field @@ -171,16 +172,14 @@ type. the :c:type:`PyStructSequence_Desc` determines which field of the struct sequence is described. - +-----------+------------------+-----------------------------------------+ - | Field | C Type | Meaning | - +===========+==================+=========================================+ - | ``name`` | ``const char *`` | name for the field or ``NULL`` to end | - | | | the list of named fields, set to | - | | | :c:data:`PyStructSequence_UnnamedField` | - | | | to leave unnamed | - +-----------+------------------+-----------------------------------------+ - | ``doc`` | ``const char *`` | field docstring or ``NULL`` to omit | - +-----------+------------------+-----------------------------------------+ + .. c:member:: const char *name + + Name for the field or ``NULL`` to end the list of named fields, + set to :c:data:`PyStructSequence_UnnamedField` to leave unnamed. + + .. c:member:: const char *doc + + Field docstring or ``NULL`` to omit. .. c:var:: const char * const PyStructSequence_UnnamedField diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 04f426d982bbb..99b025dc41fc2 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1513,21 +1513,23 @@ and :c:data:`PyType_Type` effectively act as defaults.) The following constants are defined to be used as the third argument for :c:member:`~PyTypeObject.tp_richcompare` and for :c:func:`PyObject_RichCompare`: - +------------------+------------+ - | Constant | Comparison | - +==================+============+ - | :c:macro:`Py_LT` | ``<`` | - +------------------+------------+ - | :c:macro:`Py_LE` | ``<=`` | - +------------------+------------+ - | :c:macro:`Py_EQ` | ``==`` | - +------------------+------------+ - | :c:macro:`Py_NE` | ``!=`` | - +------------------+------------+ - | :c:macro:`Py_GT` | ``>`` | - +------------------+------------+ - | :c:macro:`Py_GE` | ``>=`` | - +------------------+------------+ + .. c:namespace:: NULL + + +--------------------+------------+ + | Constant | Comparison | + +====================+============+ + | .. c:macro:: Py_LT | ``<`` | + +--------------------+------------+ + | .. c:macro:: Py_LE | ``<=`` | + +--------------------+------------+ + | .. c:macro:: Py_EQ | ``==`` | + +--------------------+------------+ + | .. c:macro:: Py_NE | ``!=`` | + +--------------------+------------+ + | .. c:macro:: Py_GT | ``>`` | + +--------------------+------------+ + | .. c:macro:: Py_GE | ``>=`` | + +--------------------+------------+ The following macro is defined to ease writing rich comparison functions: diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 08f215f035d0b..1cf8ff5a6e222 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -23,7 +23,6 @@ Doc/c-api/set.rst Doc/c-api/stable.rst Doc/c-api/structures.rst Doc/c-api/sys.rst -Doc/c-api/tuple.rst Doc/c-api/type.rst Doc/c-api/typeobj.rst Doc/c-api/unicode.rst From webhook-mailer at python.org Thu Jul 27 02:24:22 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 27 Jul 2023 06:24:22 -0000 Subject: [Python-checkins] [3.12] gh-107091: Fix some uses of :attr: role (GH-107318) (GH-107330) Message-ID: https://github.com/python/cpython/commit/434e3b89a7dfc5a9cfc644327c1774a50332b126 commit: 434e3b89a7dfc5a9cfc644327c1774a50332b126 branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-27T09:24:18+03:00 summary: [3.12] gh-107091: Fix some uses of :attr: role (GH-107318) (GH-107330) Fix also formatting of PyMethodDef members. (cherry picked from commit d363eb5b0255c055e7b43f5e2c0847f555e1982e) files: M Doc/c-api/import.rst M Doc/c-api/init.rst M Doc/c-api/structures.rst M Doc/c-api/tuple.rst M Doc/c-api/typeobj.rst M Doc/c-api/veryhigh.rst M Doc/extending/newtypes.rst M Doc/extending/newtypes_tutorial.rst M Doc/whatsnew/2.5.rst diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 905cab9034bec..57f94a7f97671 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -294,23 +294,25 @@ Importing Modules .. c:struct:: _inittab - Structure describing a single entry in the list of built-in modules. Each of - these structures gives the name and initialization function for a module built - into the interpreter. The name is an ASCII encoded string. Programs which + Structure describing a single entry in the list of built-in modules. + Programs which embed Python may use an array of these structures in conjunction with :c:func:`PyImport_ExtendInittab` to provide additional built-in modules. - The structure is defined in :file:`Include/import.h` as:: + The structure consists of two members: - struct _inittab { - const char *name; /* ASCII encoded string */ - PyObject* (*initfunc)(void); - }; + .. c:member:: const char *name + + The module name, as an ASCII encoded string. + + .. c: member:: PyObject* (*initfunc)(void) + + Initialization function for a module built into the interpreter. .. c:function:: int PyImport_ExtendInittab(struct _inittab *newtab) Add a collection of modules to the table of built-in modules. The *newtab* - array must end with a sentinel entry which contains ``NULL`` for the :attr:`name` + array must end with a sentinel entry which contains ``NULL`` for the :c:member:`~_inittab.name` field; failure to provide the sentinel value can result in a memory fault. Returns ``0`` on success or ``-1`` if insufficient memory could be allocated to extend the internal table. In the event of failure, no modules are added to the diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index c99377fb45a6b..4144f16f23bff 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1023,8 +1023,11 @@ code, or when embedding the Python interpreter: .. c:type:: PyThreadState This data structure represents the state of a single thread. The only public - data member is :attr:`interp` (:c:expr:`PyInterpreterState *`), which points to - this thread's interpreter state. + data member is: + + .. c:member:: PyInterpreterState *interp + + This thread's interpreter state. .. c:function:: void PyEval_InitThreads() diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 75e325e8fa83f..55f3410728bc3 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -35,7 +35,7 @@ under :ref:`reference counting `. .. c:type:: PyVarObject - This is an extension of :c:type:`PyObject` that adds the :attr:`ob_size` + This is an extension of :c:type:`PyObject` that adds the :c:member:`~PyVarObject.ob_size` field. This is only used for objects that have some notion of *length*. This type does not often appear in the Python/C API. Access to the members must be done by using the macros @@ -152,7 +152,7 @@ under :ref:`reference counting `. .. c:macro:: PyVarObject_HEAD_INIT(type, size) This is a macro which expands to initialization values for a new - :c:type:`PyVarObject` type, including the :attr:`ob_size` field. + :c:type:`PyVarObject` type, including the :c:member:`~PyVarObject.ob_size` field. This macro expands to:: _PyObject_EXTRA_INIT @@ -228,21 +228,21 @@ Implementing functions and methods Structure used to describe a method of an extension type. This structure has four fields: - .. c:member:: const char* ml_name + .. c:member:: const char *ml_name - name of the method + Name of the method. .. c:member:: PyCFunction ml_meth - pointer to the C implementation + Pointer to the C implementation. .. c:member:: int ml_flags - flags bits indicating how the call should be constructed + Flags bits indicating how the call should be constructed. - .. c:member:: const char* ml_doc + .. c:member:: const char *ml_doc - points to the contents of the docstring + Points to the contents of the docstring. The :c:member:`~PyMethodDef.ml_meth` is a C function pointer. The functions may be of different diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index 01b8da6cb4e0e..c3415fdf03d0d 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -164,7 +164,8 @@ type. Describes a field of a struct sequence. As a struct sequence is modeled as a tuple, all fields are typed as :c:expr:`PyObject*`. The index in the - :attr:`fields` array of the :c:type:`PyStructSequence_Desc` determines which + :c:member:`~PyStructSequence_Desc.fields` array of + the :c:type:`PyStructSequence_Desc` determines which field of the struct sequence is described. +-----------+------------------+-----------------------------------------+ diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index a6cf8a03d03b9..8d3455c878545 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -485,17 +485,17 @@ PyObject Slots -------------- The type object structure extends the :c:type:`PyVarObject` structure. The -:attr:`ob_size` field is used for dynamic types (created by :func:`type_new`, +:c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :func:`type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. -type objects) *must* have the :attr:`ob_size` field. +type objects) *must* have the :c:member:`~PyVarObject.ob_size` field. .. c:member:: Py_ssize_t PyObject.ob_refcnt This is the type object's reference count, initialized to ``1`` by the ``PyObject_HEAD_INIT`` macro. Note that for :ref:`statically allocated type - objects `, the type's instances (objects whose :attr:`ob_type` + objects `, the type's instances (objects whose :c:member:`~PyObject.ob_type` points back to the type) do *not* count as references. But for :ref:`dynamically allocated type objects `, the instances *do* count as references. @@ -519,8 +519,8 @@ type objects) *must* have the :attr:`ob_size` field. Foo_Type.ob_type = &PyType_Type; This should be done before any instances of the type are created. - :c:func:`PyType_Ready` checks if :attr:`ob_type` is ``NULL``, and if so, - initializes it to the :attr:`ob_type` field of the base class. + :c:func:`PyType_Ready` checks if :c:member:`~PyObject.ob_type` is ``NULL``, and if so, + initializes it to the :c:member:`~PyObject.ob_type` field of the base class. :c:func:`PyType_Ready` will not change this field if it is non-zero. **Inheritance:** @@ -619,20 +619,20 @@ and :c:data:`PyType_Type` effectively act as defaults.) instances have the same size, given in :c:member:`~PyTypeObject.tp_basicsize`. For a type with variable-length instances, the instances must have an - :attr:`ob_size` field, and the instance size is :c:member:`~PyTypeObject.tp_basicsize` plus N + :c:member:`~PyVarObject.ob_size` field, and the instance size is :c:member:`~PyTypeObject.tp_basicsize` plus N times :c:member:`~PyTypeObject.tp_itemsize`, where N is the "length" of the object. The value of - N is typically stored in the instance's :attr:`ob_size` field. There are - exceptions: for example, ints use a negative :attr:`ob_size` to indicate a + N is typically stored in the instance's :c:member:`~PyVarObject.ob_size` field. There are + exceptions: for example, ints use a negative :c:member:`~PyVarObject.ob_size` to indicate a negative number, and N is ``abs(ob_size)`` there. Also, the presence of an - :attr:`ob_size` field in the instance layout doesn't mean that the instance + :c:member:`~PyVarObject.ob_size` field in the instance layout doesn't mean that the instance structure is variable-length (for example, the structure for the list type has - fixed-length instances, yet those instances have a meaningful :attr:`ob_size` + fixed-length instances, yet those instances have a meaningful :c:member:`~PyVarObject.ob_size` field). The basic size includes the fields in the instance declared by the macro :c:macro:`PyObject_HEAD` or :c:macro:`PyObject_VAR_HEAD` (whichever is used to - declare the instance struct) and this in turn includes the :attr:`_ob_prev` and - :attr:`_ob_next` fields if they are present. This means that the only correct + declare the instance struct) and this in turn includes the :c:member:`~PyObject._ob_prev` and + :c:member:`~PyObject._ob_next` fields if they are present. This means that the only correct way to get an initializer for the :c:member:`~PyTypeObject.tp_basicsize` is to use the ``sizeof`` operator on the struct used to declare the instance layout. The basic size does not include the GC header size. @@ -764,7 +764,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_getattr`, :attr:`tp_getattro` + Group: :c:member:`~PyTypeObject.tp_getattr`, :c:member:`~PyTypeObject.tp_getattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattro`: a subtype inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when @@ -781,7 +781,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_setattr`, :attr:`tp_setattro` + Group: :c:member:`~PyTypeObject.tp_setattr`, :c:member:`~PyTypeObject.tp_setattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattro`: a subtype inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when @@ -883,7 +883,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) normal return value; when an error occurs during the computation of the hash value, the function should set an exception and return ``-1``. - When this field is not set (*and* :attr:`tp_richcompare` is not set), + When this field is not set (*and* :c:member:`~PyTypeObject.tp_richcompare` is not set), an attempt to take the hash of the object raises :exc:`TypeError`. This is the same as setting it to :c:func:`PyObject_HashNotImplemented`. @@ -897,7 +897,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_hash`, :attr:`tp_richcompare` + Group: :c:member:`~PyTypeObject.tp_hash`, :c:member:`~PyTypeObject.tp_richcompare` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_richcompare`: a subtype inherits both of @@ -956,7 +956,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_getattr`, :attr:`tp_getattro` + Group: :c:member:`~PyTypeObject.tp_getattr`, :c:member:`~PyTypeObject.tp_getattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattr`: a subtype inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when @@ -982,7 +982,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_setattr`, :attr:`tp_setattro` + Group: :c:member:`~PyTypeObject.tp_setattr`, :c:member:`~PyTypeObject.tp_setattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattr`: a subtype inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when @@ -1047,7 +1047,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) This bit is set when the type object itself is allocated on the heap, for example, types created dynamically using :c:func:`PyType_FromSpec`. In this - case, the :attr:`ob_type` field of its instances is considered a reference to + case, the :c:member:`~PyObject.ob_type` field of its instances is considered a reference to the type, and the type object is INCREF'ed when a new instance is created, and DECREF'ed when an instance is destroyed (this does not apply to instances of subtypes; only the type referenced by the instance's ob_type gets INCREF'ed or @@ -1100,13 +1100,13 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited - together with the :attr:`tp_traverse` and :attr:`tp_clear` + together with the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is - clear in the subtype and the :attr:`tp_traverse` and - :attr:`tp_clear` fields in the subtype exist and have ``NULL`` + clear in the subtype and the :c:member:`~PyTypeObject.tp_traverse` and + :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have ``NULL`` values. @@ -1421,7 +1421,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_clear` and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and @@ -1488,7 +1488,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_traverse` and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and @@ -1547,7 +1547,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_hash`, :attr:`tp_richcompare` + Group: :c:member:`~PyTypeObject.tp_hash`, :c:member:`~PyTypeObject.tp_richcompare` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_hash`: a subtype inherits :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` when @@ -1556,9 +1556,9 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Default:** - :c:data:`PyBaseObject_Type` provides a :attr:`tp_richcompare` + :c:data:`PyBaseObject_Type` provides a :c:member:`~PyTypeObject.tp_richcompare` implementation, which may be inherited. However, if only - :attr:`tp_hash` is defined, not even the inherited function is used + :c:member:`~PyTypeObject.tp_hash` is defined, not even the inherited function is used and instances of the type will not be able to participate in any comparisons. @@ -2374,9 +2374,9 @@ Sequence Object Structures This slot must be filled for the :c:func:`PySequence_Check` function to return ``1``, it can be ``NULL`` otherwise. - Negative indexes are handled as follows: if the :attr:`sq_length` slot is + Negative indexes are handled as follows: if the :c:member:`~PySequenceMethods.sq_length` slot is filled, it is called and the sequence length is used to compute a positive - index which is passed to :attr:`sq_item`. If :attr:`sq_length` is ``NULL``, + index which is passed to :c:member:`~PySequenceMethods.sq_item`. If :c:member:`!sq_length` is ``NULL``, the index is passed as is to the function. .. c:member:: ssizeobjargproc PySequenceMethods.sq_ass_item @@ -2586,8 +2586,8 @@ Slot Type typedefs The purpose of this function is to separate memory allocation from memory initialization. It should return a pointer to a block of memory of adequate length for the instance, suitably aligned, and initialized to zeros, but with - :attr:`ob_refcnt` set to ``1`` and :attr:`ob_type` set to the type argument. If - the type's :c:member:`~PyTypeObject.tp_itemsize` is non-zero, the object's :attr:`ob_size` field + :c:member:`~PyObject.ob_refcnt` set to ``1`` and :c:member:`~PyObject.ob_type` set to the type argument. If + the type's :c:member:`~PyTypeObject.tp_itemsize` is non-zero, the object's :c:member:`~PyVarObject.ob_size` field should be initialized to *nitems* and the length of the allocated memory block should be ``tp_basicsize + nitems*tp_itemsize``, rounded up to a multiple of ``sizeof(void*)``; otherwise, *nitems* is not used and the length of the block diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 56fa2d6abd913..324518c035096 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -353,7 +353,7 @@ the same library that the Python runtime is using. executed, it is passed as ``PyCompilerFlags *flags``. In this case, ``from __future__ import`` can modify *flags*. - Whenever ``PyCompilerFlags *flags`` is ``NULL``, :attr:`cf_flags` is treated as + Whenever ``PyCompilerFlags *flags`` is ``NULL``, :c:member:`~PyCompilerFlags.cf_flags` is treated as equal to ``0``, and any modification due to ``from __future__ import`` is discarded. @@ -367,7 +367,7 @@ the same library that the Python runtime is using. initialized to ``PY_MINOR_VERSION``. The field is ignored by default, it is used if and only if - ``PyCF_ONLY_AST`` flag is set in *cf_flags*. + ``PyCF_ONLY_AST`` flag is set in :c:member:`~PyCompilerFlags.cf_flags`. .. versionchanged:: 3.8 Added *cf_feature_version* field. diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst index 7f8f8ddaaaccd..25822744125f8 100644 --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -270,7 +270,7 @@ structure:: One entry should be defined for each method provided by the type; no entries are needed for methods inherited from a base type. One additional entry is needed at the end; it is a sentinel that marks the end of the array. The -:attr:`ml_name` field of the sentinel must be ``NULL``. +:c:member:`~PyMethodDef.ml_name` field of the sentinel must be ``NULL``. The second table is used to define attributes which map directly to data stored in the instance. A variety of primitive C types are supported, and access may diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index ba09fb1b05c0b..82671fdc447d9 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -176,8 +176,8 @@ Everything else in the file should be familiar, except for some code in if (PyType_Ready(&CustomType) < 0) return; -This initializes the :class:`Custom` type, filling in a number of members -to the appropriate default values, including :attr:`ob_type` that we initially +This initializes the :class:`!Custom` type, filling in a number of members +to the appropriate default values, including :c:member:`~PyObject.ob_type` that we initially set to ``NULL``. :: Py_INCREF(&CustomType); diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index b410fe1ce3f08..162d2c342f50f 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -954,7 +954,7 @@ The return value must be either a Python integer or long integer. The interpreter will check that the type returned is correct, and raises a :exc:`TypeError` if this requirement isn't met. -A corresponding :attr:`nb_index` slot was added to the C-level +A corresponding :c:member:`~PyNumberMethods.nb_index` slot was added to the C-level :c:type:`PyNumberMethods` structure to let C extensions implement this protocol. ``PyNumber_Index(obj)`` can be used in extension code to call the :meth:`__index__` function and retrieve its result. From webhook-mailer at python.org Thu Jul 27 02:26:18 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 27 Jul 2023 06:26:18 -0000 Subject: [Python-checkins] [3.11] gh-107091: Fix some uses of :attr: role (GH-107318) (GH-107331) Message-ID: https://github.com/python/cpython/commit/17aada0dd0b3478f5fbf3b9c8bbc86cbae266aa2 commit: 17aada0dd0b3478f5fbf3b9c8bbc86cbae266aa2 branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-27T09:26:14+03:00 summary: [3.11] gh-107091: Fix some uses of :attr: role (GH-107318) (GH-107331) Fix also formatting of PyMethodDef members. (cherry picked from commit d363eb5b0255c055e7b43f5e2c0847f555e1982e) files: M Doc/c-api/import.rst M Doc/c-api/init.rst M Doc/c-api/structures.rst M Doc/c-api/tuple.rst M Doc/c-api/typeobj.rst M Doc/c-api/veryhigh.rst M Doc/extending/newtypes.rst M Doc/extending/newtypes_tutorial.rst M Doc/whatsnew/2.5.rst diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 8b2d38582343b..887c1793c3ec7 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -283,23 +283,25 @@ Importing Modules .. c:struct:: _inittab - Structure describing a single entry in the list of built-in modules. Each of - these structures gives the name and initialization function for a module built - into the interpreter. The name is an ASCII encoded string. Programs which + Structure describing a single entry in the list of built-in modules. + Programs which embed Python may use an array of these structures in conjunction with :c:func:`PyImport_ExtendInittab` to provide additional built-in modules. - The structure is defined in :file:`Include/import.h` as:: + The structure consists of two members: - struct _inittab { - const char *name; /* ASCII encoded string */ - PyObject* (*initfunc)(void); - }; + .. c:member:: const char *name + + The module name, as an ASCII encoded string. + + .. c: member:: PyObject* (*initfunc)(void) + + Initialization function for a module built into the interpreter. .. c:function:: int PyImport_ExtendInittab(struct _inittab *newtab) Add a collection of modules to the table of built-in modules. The *newtab* - array must end with a sentinel entry which contains ``NULL`` for the :attr:`name` + array must end with a sentinel entry which contains ``NULL`` for the :c:member:`~_inittab.name` field; failure to provide the sentinel value can result in a memory fault. Returns ``0`` on success or ``-1`` if insufficient memory could be allocated to extend the internal table. In the event of failure, no modules are added to the diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 9a42fc5a25dc2..05b9b77e4e7c9 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -914,8 +914,11 @@ code, or when embedding the Python interpreter: .. c:type:: PyThreadState This data structure represents the state of a single thread. The only public - data member is :attr:`interp` (:c:expr:`PyInterpreterState *`), which points to - this thread's interpreter state. + data member is: + + .. c:member:: PyInterpreterState *interp + + This thread's interpreter state. .. c:function:: void PyEval_InitThreads() diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 5c85838d2dbfa..b892a08ea0837 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -34,7 +34,7 @@ the definition of all other Python objects. .. c:type:: PyVarObject - This is an extension of :c:type:`PyObject` that adds the :attr:`ob_size` + This is an extension of :c:type:`PyObject` that adds the :c:member:`~PyVarObject.ob_size` field. This is only used for objects that have some notion of *length*. This type does not often appear in the Python/C API. Access to the members must be done by using the macros @@ -171,7 +171,7 @@ the definition of all other Python objects. .. c:macro:: PyVarObject_HEAD_INIT(type, size) This is a macro which expands to initialization values for a new - :c:type:`PyVarObject` type, including the :attr:`ob_size` field. + :c:type:`PyVarObject` type, including the :c:member:`~PyVarObject.ob_size` field. This macro expands to:: _PyObject_EXTRA_INIT @@ -247,21 +247,21 @@ Implementing functions and methods Structure used to describe a method of an extension type. This structure has four fields: - .. c:member:: const char* ml_name + .. c:member:: const char *ml_name - name of the method + Name of the method. .. c:member:: PyCFunction ml_meth - pointer to the C implementation + Pointer to the C implementation. .. c:member:: int ml_flags - flags bits indicating how the call should be constructed + Flags bits indicating how the call should be constructed. - .. c:member:: const char* ml_doc + .. c:member:: const char *ml_doc - points to the contents of the docstring + Points to the contents of the docstring. The :c:member:`~PyMethodDef.ml_meth` is a C function pointer. The functions may be of different diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index 689fad683aed6..9fec738e68a9b 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -164,7 +164,8 @@ type. Describes a field of a struct sequence. As a struct sequence is modeled as a tuple, all fields are typed as :c:expr:`PyObject*`. The index in the - :attr:`fields` array of the :c:type:`PyStructSequence_Desc` determines which + :c:member:`~PyStructSequence_Desc.fields` array of + the :c:type:`PyStructSequence_Desc` determines which field of the struct sequence is described. +-----------+------------------+-----------------------------------------+ diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 002603857394c..05b2f8a36c01a 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -483,17 +483,17 @@ PyObject Slots -------------- The type object structure extends the :c:type:`PyVarObject` structure. The -:attr:`ob_size` field is used for dynamic types (created by :func:`type_new`, +:c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :func:`type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. -type objects) *must* have the :attr:`ob_size` field. +type objects) *must* have the :c:member:`~PyVarObject.ob_size` field. .. c:member:: Py_ssize_t PyObject.ob_refcnt This is the type object's reference count, initialized to ``1`` by the ``PyObject_HEAD_INIT`` macro. Note that for :ref:`statically allocated type - objects `, the type's instances (objects whose :attr:`ob_type` + objects `, the type's instances (objects whose :c:member:`~PyObject.ob_type` points back to the type) do *not* count as references. But for :ref:`dynamically allocated type objects `, the instances *do* count as references. @@ -517,8 +517,8 @@ type objects) *must* have the :attr:`ob_size` field. Foo_Type.ob_type = &PyType_Type; This should be done before any instances of the type are created. - :c:func:`PyType_Ready` checks if :attr:`ob_type` is ``NULL``, and if so, - initializes it to the :attr:`ob_type` field of the base class. + :c:func:`PyType_Ready` checks if :c:member:`~PyObject.ob_type` is ``NULL``, and if so, + initializes it to the :c:member:`~PyObject.ob_type` field of the base class. :c:func:`PyType_Ready` will not change this field if it is non-zero. **Inheritance:** @@ -617,20 +617,20 @@ and :c:data:`PyType_Type` effectively act as defaults.) instances have the same size, given in :c:member:`~PyTypeObject.tp_basicsize`. For a type with variable-length instances, the instances must have an - :attr:`ob_size` field, and the instance size is :c:member:`~PyTypeObject.tp_basicsize` plus N + :c:member:`~PyVarObject.ob_size` field, and the instance size is :c:member:`~PyTypeObject.tp_basicsize` plus N times :c:member:`~PyTypeObject.tp_itemsize`, where N is the "length" of the object. The value of - N is typically stored in the instance's :attr:`ob_size` field. There are - exceptions: for example, ints use a negative :attr:`ob_size` to indicate a + N is typically stored in the instance's :c:member:`~PyVarObject.ob_size` field. There are + exceptions: for example, ints use a negative :c:member:`~PyVarObject.ob_size` to indicate a negative number, and N is ``abs(ob_size)`` there. Also, the presence of an - :attr:`ob_size` field in the instance layout doesn't mean that the instance + :c:member:`~PyVarObject.ob_size` field in the instance layout doesn't mean that the instance structure is variable-length (for example, the structure for the list type has - fixed-length instances, yet those instances have a meaningful :attr:`ob_size` + fixed-length instances, yet those instances have a meaningful :c:member:`~PyVarObject.ob_size` field). The basic size includes the fields in the instance declared by the macro :c:macro:`PyObject_HEAD` or :c:macro:`PyObject_VAR_HEAD` (whichever is used to - declare the instance struct) and this in turn includes the :attr:`_ob_prev` and - :attr:`_ob_next` fields if they are present. This means that the only correct + declare the instance struct) and this in turn includes the :c:member:`~PyObject._ob_prev` and + :c:member:`~PyObject._ob_next` fields if they are present. This means that the only correct way to get an initializer for the :c:member:`~PyTypeObject.tp_basicsize` is to use the ``sizeof`` operator on the struct used to declare the instance layout. The basic size does not include the GC header size. @@ -762,7 +762,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_getattr`, :attr:`tp_getattro` + Group: :c:member:`~PyTypeObject.tp_getattr`, :c:member:`~PyTypeObject.tp_getattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattro`: a subtype inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when @@ -779,7 +779,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_setattr`, :attr:`tp_setattro` + Group: :c:member:`~PyTypeObject.tp_setattr`, :c:member:`~PyTypeObject.tp_setattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattro`: a subtype inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when @@ -881,7 +881,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) normal return value; when an error occurs during the computation of the hash value, the function should set an exception and return ``-1``. - When this field is not set (*and* :attr:`tp_richcompare` is not set), + When this field is not set (*and* :c:member:`~PyTypeObject.tp_richcompare` is not set), an attempt to take the hash of the object raises :exc:`TypeError`. This is the same as setting it to :c:func:`PyObject_HashNotImplemented`. @@ -895,7 +895,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_hash`, :attr:`tp_richcompare` + Group: :c:member:`~PyTypeObject.tp_hash`, :c:member:`~PyTypeObject.tp_richcompare` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_richcompare`: a subtype inherits both of @@ -954,7 +954,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_getattr`, :attr:`tp_getattro` + Group: :c:member:`~PyTypeObject.tp_getattr`, :c:member:`~PyTypeObject.tp_getattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_getattr`: a subtype inherits both :c:member:`~PyTypeObject.tp_getattr` and :c:member:`~PyTypeObject.tp_getattro` from its base type when @@ -980,7 +980,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_setattr`, :attr:`tp_setattro` + Group: :c:member:`~PyTypeObject.tp_setattr`, :c:member:`~PyTypeObject.tp_setattro` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_setattr`: a subtype inherits both :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` from its base type when @@ -1046,7 +1046,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) This bit is set when the type object itself is allocated on the heap, for example, types created dynamically using :c:func:`PyType_FromSpec`. In this - case, the :attr:`ob_type` field of its instances is considered a reference to + case, the :c:member:`~PyObject.ob_type` field of its instances is considered a reference to the type, and the type object is INCREF'ed when a new instance is created, and DECREF'ed when an instance is destroyed (this does not apply to instances of subtypes; only the type referenced by the instance's ob_type gets INCREF'ed or @@ -1099,13 +1099,13 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` The :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is inherited - together with the :attr:`tp_traverse` and :attr:`tp_clear` + together with the :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields, i.e. if the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit is - clear in the subtype and the :attr:`tp_traverse` and - :attr:`tp_clear` fields in the subtype exist and have ``NULL`` + clear in the subtype and the :c:member:`~PyTypeObject.tp_traverse` and + :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have ``NULL`` values. @@ -1357,7 +1357,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_clear` and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and @@ -1424,7 +1424,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :attr:`tp_traverse`, :attr:`tp_clear` + Group: :c:macro:`Py_TPFLAGS_HAVE_GC`, :c:member:`~PyTypeObject.tp_traverse`, :c:member:`~PyTypeObject.tp_clear` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_traverse` and the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit: the flag bit, :c:member:`~PyTypeObject.tp_traverse`, and @@ -1483,7 +1483,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Inheritance:** - Group: :attr:`tp_hash`, :attr:`tp_richcompare` + Group: :c:member:`~PyTypeObject.tp_hash`, :c:member:`~PyTypeObject.tp_richcompare` This field is inherited by subtypes together with :c:member:`~PyTypeObject.tp_hash`: a subtype inherits :c:member:`~PyTypeObject.tp_richcompare` and :c:member:`~PyTypeObject.tp_hash` when @@ -1492,9 +1492,9 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Default:** - :c:data:`PyBaseObject_Type` provides a :attr:`tp_richcompare` + :c:data:`PyBaseObject_Type` provides a :c:member:`~PyTypeObject.tp_richcompare` implementation, which may be inherited. However, if only - :attr:`tp_hash` is defined, not even the inherited function is used + :c:member:`~PyTypeObject.tp_hash` is defined, not even the inherited function is used and instances of the type will not be able to participate in any comparisons. @@ -2288,9 +2288,9 @@ Sequence Object Structures This slot must be filled for the :c:func:`PySequence_Check` function to return ``1``, it can be ``NULL`` otherwise. - Negative indexes are handled as follows: if the :attr:`sq_length` slot is + Negative indexes are handled as follows: if the :c:member:`~PySequenceMethods.sq_length` slot is filled, it is called and the sequence length is used to compute a positive - index which is passed to :attr:`sq_item`. If :attr:`sq_length` is ``NULL``, + index which is passed to :c:member:`~PySequenceMethods.sq_item`. If :c:member:`!sq_length` is ``NULL``, the index is passed as is to the function. .. c:member:: ssizeobjargproc PySequenceMethods.sq_ass_item @@ -2500,8 +2500,8 @@ Slot Type typedefs The purpose of this function is to separate memory allocation from memory initialization. It should return a pointer to a block of memory of adequate length for the instance, suitably aligned, and initialized to zeros, but with - :attr:`ob_refcnt` set to ``1`` and :attr:`ob_type` set to the type argument. If - the type's :c:member:`~PyTypeObject.tp_itemsize` is non-zero, the object's :attr:`ob_size` field + :c:member:`~PyObject.ob_refcnt` set to ``1`` and :c:member:`~PyObject.ob_type` set to the type argument. If + the type's :c:member:`~PyTypeObject.tp_itemsize` is non-zero, the object's :c:member:`~PyVarObject.ob_size` field should be initialized to *nitems* and the length of the allocated memory block should be ``tp_basicsize + nitems*tp_itemsize``, rounded up to a multiple of ``sizeof(void*)``; otherwise, *nitems* is not used and the length of the block diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 79515c9df07ed..9a1295b558f63 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -345,7 +345,7 @@ the same library that the Python runtime is using. executed, it is passed as ``PyCompilerFlags *flags``. In this case, ``from __future__ import`` can modify *flags*. - Whenever ``PyCompilerFlags *flags`` is ``NULL``, :attr:`cf_flags` is treated as + Whenever ``PyCompilerFlags *flags`` is ``NULL``, :c:member:`~PyCompilerFlags.cf_flags` is treated as equal to ``0``, and any modification due to ``from __future__ import`` is discarded. @@ -359,7 +359,7 @@ the same library that the Python runtime is using. initialized to ``PY_MINOR_VERSION``. The field is ignored by default, it is used if and only if - ``PyCF_ONLY_AST`` flag is set in *cf_flags*. + ``PyCF_ONLY_AST`` flag is set in :c:member:`~PyCompilerFlags.cf_flags`. .. versionchanged:: 3.8 Added *cf_feature_version* field. diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst index e60db11b5f10f..1f2a7d12bc654 100644 --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -270,7 +270,7 @@ structure:: One entry should be defined for each method provided by the type; no entries are needed for methods inherited from a base type. One additional entry is needed at the end; it is a sentinel that marks the end of the array. The -:attr:`ml_name` field of the sentinel must be ``NULL``. +:c:member:`~PyMethodDef.ml_name` field of the sentinel must be ``NULL``. The second table is used to define attributes which map directly to data stored in the instance. A variety of primitive C types are supported, and access may diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index b1bf0535048eb..92e9b97e36d77 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -176,8 +176,8 @@ Everything else in the file should be familiar, except for some code in if (PyType_Ready(&CustomType) < 0) return; -This initializes the :class:`Custom` type, filling in a number of members -to the appropriate default values, including :attr:`ob_type` that we initially +This initializes the :class:`!Custom` type, filling in a number of members +to the appropriate default values, including :c:member:`~PyObject.ob_type` that we initially set to ``NULL``. :: Py_INCREF(&CustomType); diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index b410fe1ce3f08..162d2c342f50f 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -954,7 +954,7 @@ The return value must be either a Python integer or long integer. The interpreter will check that the type returned is correct, and raises a :exc:`TypeError` if this requirement isn't met. -A corresponding :attr:`nb_index` slot was added to the C-level +A corresponding :c:member:`~PyNumberMethods.nb_index` slot was added to the C-level :c:type:`PyNumberMethods` structure to let C extensions implement this protocol. ``PyNumber_Index(obj)`` can be used in extension code to call the :meth:`__index__` function and retrieve its result. From webhook-mailer at python.org Thu Jul 27 02:27:06 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 27 Jul 2023 06:27:06 -0000 Subject: [Python-checkins] [3.11] gh-107298: Docs: add targets for some :c:member: and :c:macro: references (GH-107316) (GH-107333) Message-ID: https://github.com/python/cpython/commit/9513acf737946e756cea48deb9aa1b51138f7c9b commit: 9513acf737946e756cea48deb9aa1b51138f7c9b branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-27T09:27:02+03:00 summary: [3.11] gh-107298: Docs: add targets for some :c:member: and :c:macro: references (GH-107316) (GH-107333) Add targets for PyStructSequence_Desc and PyStructSequence_Field members and macros like Py_EQ. Fix target for Py_RETURN_RICHCOMPARE. (cherry picked from commit abec9a1b20b70d8ced401d59fc4f02b331c6568b) files: M Doc/c-api/tuple.rst M Doc/c-api/typeobj.rst diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index 9fec738e68a9b..d298aaf3ab548 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -144,20 +144,21 @@ type. Contains the meta information of a struct sequence type to create. - +-------------------+------------------------------+--------------------------------------+ - | Field | C Type | Meaning | - +===================+==============================+======================================+ - | ``name`` | ``const char *`` | name of the struct sequence type | - +-------------------+------------------------------+--------------------------------------+ - | ``doc`` | ``const char *`` | pointer to docstring for the type | - | | | or ``NULL`` to omit | - +-------------------+------------------------------+--------------------------------------+ - | ``fields`` | ``PyStructSequence_Field *`` | pointer to ``NULL``-terminated array | - | | | with field names of the new type | - +-------------------+------------------------------+--------------------------------------+ - | ``n_in_sequence`` | ``int`` | number of fields visible to the | - | | | Python side (if used as tuple) | - +-------------------+------------------------------+--------------------------------------+ + .. c:member:: const char *name + + Name of the struct sequence type. + + .. c:member:: const char *doc + + Pointer to docstring for the type or ``NULL`` to omit. + + .. c:member:: PyStructSequence_Field *fields + + Pointer to ``NULL``-terminated array with field names of the new type. + + .. c:member:: int n_in_sequence + + Number of fields visible to the Python side (if used as tuple). .. c:type:: PyStructSequence_Field @@ -168,16 +169,14 @@ type. the :c:type:`PyStructSequence_Desc` determines which field of the struct sequence is described. - +-----------+------------------+-----------------------------------------+ - | Field | C Type | Meaning | - +===========+==================+=========================================+ - | ``name`` | ``const char *`` | name for the field or ``NULL`` to end | - | | | the list of named fields, set to | - | | | :c:data:`PyStructSequence_UnnamedField` | - | | | to leave unnamed | - +-----------+------------------+-----------------------------------------+ - | ``doc`` | ``const char *`` | field docstring or ``NULL`` to omit | - +-----------+------------------+-----------------------------------------+ + .. c:member:: const char *name + + Name for the field or ``NULL`` to end the list of named fields, + set to :c:data:`PyStructSequence_UnnamedField` to leave unnamed. + + .. c:member:: const char *doc + + Field docstring or ``NULL`` to omit. .. c:var:: const char * const PyStructSequence_UnnamedField diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 05b2f8a36c01a..ec357450d004a 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1449,21 +1449,23 @@ and :c:data:`PyType_Type` effectively act as defaults.) The following constants are defined to be used as the third argument for :c:member:`~PyTypeObject.tp_richcompare` and for :c:func:`PyObject_RichCompare`: - +------------------+------------+ - | Constant | Comparison | - +==================+============+ - | :c:macro:`Py_LT` | ``<`` | - +------------------+------------+ - | :c:macro:`Py_LE` | ``<=`` | - +------------------+------------+ - | :c:macro:`Py_EQ` | ``==`` | - +------------------+------------+ - | :c:macro:`Py_NE` | ``!=`` | - +------------------+------------+ - | :c:macro:`Py_GT` | ``>`` | - +------------------+------------+ - | :c:macro:`Py_GE` | ``>=`` | - +------------------+------------+ + .. c:namespace:: NULL + + +--------------------+------------+ + | Constant | Comparison | + +====================+============+ + | .. c:macro:: Py_LT | ``<`` | + +--------------------+------------+ + | .. c:macro:: Py_LE | ``<=`` | + +--------------------+------------+ + | .. c:macro:: Py_EQ | ``==`` | + +--------------------+------------+ + | .. c:macro:: Py_NE | ``!=`` | + +--------------------+------------+ + | .. c:macro:: Py_GT | ``>`` | + +--------------------+------------+ + | .. c:macro:: Py_GE | ``>=`` | + +--------------------+------------+ The following macro is defined to ease writing rich comparison functions: From webhook-mailer at python.org Thu Jul 27 02:37:58 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 27 Jul 2023 06:37:58 -0000 Subject: [Python-checkins] [3.12] gh-107298: Docs: add targets for some :c:member: and :c:macro: references (GH-107316) (GH-107332) Message-ID: https://github.com/python/cpython/commit/36e96baa33cb5aa6cb03a1555017dd66f4244183 commit: 36e96baa33cb5aa6cb03a1555017dd66f4244183 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-27T09:37:54+03:00 summary: [3.12] gh-107298: Docs: add targets for some :c:member: and :c:macro: references (GH-107316) (GH-107332) Add targets for PyStructSequence_Desc and PyStructSequence_Field members and macros like Py_EQ. Fix target for Py_RETURN_RICHCOMPARE. (cherry picked from commit abec9a1b20b70d8ced401d59fc4f02b331c6568b) Co-authored-by: Serhiy Storchaka files: M Doc/c-api/tuple.rst M Doc/c-api/typeobj.rst M Doc/tools/.nitignore diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index c3415fdf03d0d..9bc3dab0c9c12 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -144,20 +144,21 @@ type. Contains the meta information of a struct sequence type to create. - +-------------------+------------------------------+--------------------------------------+ - | Field | C Type | Meaning | - +===================+==============================+======================================+ - | ``name`` | ``const char *`` | name of the struct sequence type | - +-------------------+------------------------------+--------------------------------------+ - | ``doc`` | ``const char *`` | pointer to docstring for the type | - | | | or ``NULL`` to omit | - +-------------------+------------------------------+--------------------------------------+ - | ``fields`` | ``PyStructSequence_Field *`` | pointer to ``NULL``-terminated array | - | | | with field names of the new type | - +-------------------+------------------------------+--------------------------------------+ - | ``n_in_sequence`` | ``int`` | number of fields visible to the | - | | | Python side (if used as tuple) | - +-------------------+------------------------------+--------------------------------------+ + .. c:member:: const char *name + + Name of the struct sequence type. + + .. c:member:: const char *doc + + Pointer to docstring for the type or ``NULL`` to omit. + + .. c:member:: PyStructSequence_Field *fields + + Pointer to ``NULL``-terminated array with field names of the new type. + + .. c:member:: int n_in_sequence + + Number of fields visible to the Python side (if used as tuple). .. c:type:: PyStructSequence_Field @@ -168,16 +169,14 @@ type. the :c:type:`PyStructSequence_Desc` determines which field of the struct sequence is described. - +-----------+------------------+-----------------------------------------+ - | Field | C Type | Meaning | - +===========+==================+=========================================+ - | ``name`` | ``const char *`` | name for the field or ``NULL`` to end | - | | | the list of named fields, set to | - | | | :c:data:`PyStructSequence_UnnamedField` | - | | | to leave unnamed | - +-----------+------------------+-----------------------------------------+ - | ``doc`` | ``const char *`` | field docstring or ``NULL`` to omit | - +-----------+------------------+-----------------------------------------+ + .. c:member:: const char *name + + Name for the field or ``NULL`` to end the list of named fields, + set to :c:data:`PyStructSequence_UnnamedField` to leave unnamed. + + .. c:member:: const char *doc + + Field docstring or ``NULL`` to omit. .. c:var:: const char * const PyStructSequence_UnnamedField diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 8d3455c878545..e70b82405a4cf 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1513,21 +1513,23 @@ and :c:data:`PyType_Type` effectively act as defaults.) The following constants are defined to be used as the third argument for :c:member:`~PyTypeObject.tp_richcompare` and for :c:func:`PyObject_RichCompare`: - +------------------+------------+ - | Constant | Comparison | - +==================+============+ - | :c:macro:`Py_LT` | ``<`` | - +------------------+------------+ - | :c:macro:`Py_LE` | ``<=`` | - +------------------+------------+ - | :c:macro:`Py_EQ` | ``==`` | - +------------------+------------+ - | :c:macro:`Py_NE` | ``!=`` | - +------------------+------------+ - | :c:macro:`Py_GT` | ``>`` | - +------------------+------------+ - | :c:macro:`Py_GE` | ``>=`` | - +------------------+------------+ + .. c:namespace:: NULL + + +--------------------+------------+ + | Constant | Comparison | + +====================+============+ + | .. c:macro:: Py_LT | ``<`` | + +--------------------+------------+ + | .. c:macro:: Py_LE | ``<=`` | + +--------------------+------------+ + | .. c:macro:: Py_EQ | ``==`` | + +--------------------+------------+ + | .. c:macro:: Py_NE | ``!=`` | + +--------------------+------------+ + | .. c:macro:: Py_GT | ``>`` | + +--------------------+------------+ + | .. c:macro:: Py_GE | ``>=`` | + +--------------------+------------+ The following macro is defined to ease writing rich comparison functions: diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 849ef1168b455..25d0f27e18311 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -36,7 +36,6 @@ Doc/c-api/set.rst Doc/c-api/stable.rst Doc/c-api/structures.rst Doc/c-api/sys.rst -Doc/c-api/tuple.rst Doc/c-api/type.rst Doc/c-api/typehints.rst Doc/c-api/typeobj.rst From webhook-mailer at python.org Thu Jul 27 02:56:43 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Thu, 27 Jul 2023 06:56:43 -0000 Subject: [Python-checkins] gh-107279 Add `` to `Modules/zlibmodule.c` to fix failing builds (#107280) Message-ID: https://github.com/python/cpython/commit/4b2e54bd3c23da37923a18ae5e82cfd574e9a439 commit: 4b2e54bd3c23da37923a18ae5e82cfd574e9a439 branch: main author: shailshouryya <42100758+shailshouryya at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-27T12:26:39+05:30 summary: gh-107279 Add `` to `Modules/zlibmodule.c` to fix failing builds (#107280) files: M Modules/zlibmodule.c diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c index 22da50989a88d..a98a37adadcff 100644 --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -7,6 +7,7 @@ #include "zlib.h" #include "stdbool.h" +#include // offsetof() #if defined(ZLIB_VERNUM) && ZLIB_VERNUM < 0x1221 #error "At least zlib version 1.2.2.1 is required" From webhook-mailer at python.org Thu Jul 27 03:07:59 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Thu, 27 Jul 2023 07:07:59 -0000 Subject: [Python-checkins] [3.12] Docs: Argument Clinic: Restructure "Basic concepts and usage" (GH-106981) (#107325) Message-ID: https://github.com/python/cpython/commit/5f3e371e2d837d4a06eaee43fbfdf589aca2fc0f commit: 5f3e371e2d837d4a06eaee43fbfdf589aca2fc0f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-27T09:07:55+02:00 summary: [3.12] Docs: Argument Clinic: Restructure "Basic concepts and usage" (GH-106981) (#107325) Split "Basic concepts and usage" into: - Reference - Terminology - CLI reference - Background - Basic concepts (cherry picked from commit 2ad699002e3ce09e9fa41e333ac72f16a32d94de) Co-authored-by: Erlend E. Aasland Co-authored-by: Adam Turner <9087854+AA-Turner at users.noreply.github.com> Co-authored-by: Alex Waygood Co-authored-by: Ezio Melotti files: M Doc/howto/clinic.rst M Tools/clinic/clinic.py diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index ea3b4537d4d53..7aafd48711b58 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -8,6 +8,7 @@ Argument Clinic How-To :author: Larry Hastings +**Source code:** :source:`Tools/clinic/clinic.py`. .. topic:: Abstract @@ -15,10 +16,12 @@ Argument Clinic How-To Its purpose is to automate all the boilerplate involved with writing argument parsing code for "builtins", module level functions, and class methods. - This document is divided in three major sections: + This document is divided in four major sections: * :ref:`clinic-background` talks about the basic concepts and goals of Argument Clinic. + * :ref:`clinic-reference` describes the command-line interface and Argument + Clinic terminology. * :ref:`clinic-tutorial` guides you through all the steps required to adapt an existing C function to Argument Clinic. * :ref:`clinic-howtos` details how to handle specific tasks. @@ -93,39 +96,29 @@ and it should be able to do many interesting and smart things with all the information you give it. -Basic concepts and usage ------------------------- +Basic concepts +-------------- -Argument Clinic ships with CPython; you'll find it in -:source:`Tools/clinic/clinic.py`. -If you run that script, specifying a C file as an argument: - -.. code-block:: shell-session - - $ python Tools/clinic/clinic.py foo.c - -Argument Clinic will scan over the file looking for lines that -look exactly like this: +When Argument Clinic is run on a file, either via the :ref:`clinic-cli` +or via ``make clinic``, it will scan over the input files looking for +:term:`start lines `: .. code-block:: none /*[clinic input] -When it finds one, it reads everything up to a line that looks -exactly like this: +When it finds one, it reads everything up to the :term:`end line`: .. code-block:: none [clinic start generated code]*/ -Everything in between these two lines is input for Argument Clinic. -All of these lines, including the beginning and ending comment -lines, are collectively called an Argument Clinic "block". - -When Argument Clinic parses one of these blocks, it -generates output. This output is rewritten into the C file -immediately after the block, followed by a comment containing a checksum. -The Argument Clinic block now looks like this: +Everything in between these two lines is Argument Clinic :term:`input`. +When Argument Clinic parses input, it generates :term:`output`. +The output is rewritten into the C file immediately after the input, +followed by a :term:`checksum line`. +All of these lines, including the :term:`start line` and :term:`checksum line`, +are collectively called an Argument Clinic :term:`block`: .. code-block:: none @@ -133,28 +126,121 @@ The Argument Clinic block now looks like this: ... clinic input goes here ... [clinic start generated code]*/ ... clinic output goes here ... - /*[clinic end generated code: checksum=...]*/ + /*[clinic end generated code: ...]*/ If you run Argument Clinic on the same file a second time, Argument Clinic -will discard the old output and write out the new output with a fresh checksum -line. However, if the input hasn't changed, the output won't change either. - -You should never modify the output portion of an Argument Clinic block. Instead, -change the input until it produces the output you want. (That's the purpose of the -checksum?to detect if someone changed the output, as these edits would be lost -the next time Argument Clinic writes out fresh output.) - -For the sake of clarity, here's the terminology we'll use with Argument Clinic: - -* The first line of the comment (``/*[clinic input]``) is the *start line*. -* The last line of the initial comment (``[clinic start generated code]*/``) is the *end line*. -* The last line (``/*[clinic end generated code: checksum=...]*/``) is the *checksum line*. -* In between the start line and the end line is the *input*. -* In between the end line and the checksum line is the *output*. -* All the text collectively, from the start line to the checksum line inclusively, - is the *block*. (A block that hasn't been successfully processed by Argument - Clinic yet doesn't have output or a checksum line, but it's still considered - a block.) +will discard the old :term:`output` and write out the new output with a fresh +:term:`checksum line`. +If the :term:`input` hasn't changed, the output won't change either. + +.. note:: + + You should never modify the output of an Argument Clinic block, + as any change will be lost in future Argument Clinic runs; + Argument Clinic will detect an output checksum mismatch and regenerate the + correct output. + If you are not happy with the generated output, + you should instead change the input until it produces the output you want. + + +.. _clinic-reference: + +Reference +========= + + +.. _clinic-terminology: + +Terminology +----------- + +.. glossary:: + + start line + The line ``/*[clinic input]``. + This line marks the beginning of Argument Clinic input. + Note that the *start line* opens a C block comment. + + end line + The line ``[clinic start generated code]*/``. + The *end line* marks the _end_ of Argument Clinic :term:`input`, + but at the same time marks the _start_ of Argument Clinic :term:`output`, + thus the text *"clinic start start generated code"* + Note that the *end line* closes the C block comment opened + by the *start line*. + + checksum + A hash to distinguish unique :term:`inputs ` + and :term:`outputs `. + + checksum line + A line that looks like ``/*[clinic end generated code: ...]*/``. + The three dots will be replaced by a :term:`checksum` generated from the + :term:`input`, and a :term:`checksum` generated from the :term:`output`. + The checksum line marks the end of Argument Clinic generated code, + and is used by Argument Clinic to determine if it needs to regenerate + output. + + input + The text between the :term:`start line` and the :term:`end line`. + Note that the start and end lines open and close a C block comment; + the *input* is thus a part of that same C block comment. + + output + The text between the :term:`end line` and the :term:`checksum line`. + + block + All text from the :term:`start line` to the :term:`checksum line` inclusively. + + +.. _clinic-cli: + +Command-line interface +---------------------- + +The Argument Clinic :abbr:`CLI (Command-Line Interface)` is typically used to +process a single source file, like this: + +.. code-block:: shell-session + + $ python3 ./Tools/clinic/clinic.py foo.c + +The CLI supports the following options: + +.. program:: ./Tools/clinic/clinic.py [-h] [-f] [-o OUTPUT] [-v] \ + [--converters] [--make] [--srcdir SRCDIR] [FILE ...] + +.. option:: -h, --help + + Print CLI usage. + +.. option:: -f, --force + + Force output regeneration. + +.. option:: -o, --output OUTPUT + + Redirect file output to OUTPUT + +.. option:: -v, --verbose + + Enable verbose mode. + +.. option:: --converters + + Print a list of all supported converters and return converters. + +.. option:: --make + + Walk :option:`--srcdir` to run over all relevant files. + +.. option:: --srcdir SRCDIR + + The directory tree to walk in :option:`--make` mode. + +.. option:: FILE ... + + The list of files to process. .. _clinic-tutorial: diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 3e60328082d66..e775b012d0b6d 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -5431,15 +5431,21 @@ def main(argv): signatures ("docstrings") for CPython builtins. For more information see https://docs.python.org/3/howto/clinic.html""") - cmdline.add_argument("-f", "--force", action='store_true') - cmdline.add_argument("-o", "--output", type=str) - cmdline.add_argument("-v", "--verbose", action='store_true') - cmdline.add_argument("--converters", action='store_true') + cmdline.add_argument("-f", "--force", action='store_true', + help="force output regeneration") + cmdline.add_argument("-o", "--output", type=str, + help="redirect file output to OUTPUT") + cmdline.add_argument("-v", "--verbose", action='store_true', + help="enable verbose mode") + cmdline.add_argument("--converters", action='store_true', + help=("print a list of all supported converters " + "and return converters")) cmdline.add_argument("--make", action='store_true', - help="Walk --srcdir to run over all relevant files.") + help="walk --srcdir to run over all relevant files") cmdline.add_argument("--srcdir", type=str, default=os.curdir, - help="The directory tree to walk in --make mode.") - cmdline.add_argument("filename", type=str, nargs="*") + help="the directory tree to walk in --make mode") + cmdline.add_argument("filename", metavar="FILE", type=str, nargs="*", + help="the list of files to process") ns = cmdline.parse_args(argv) if ns.converters: From webhook-mailer at python.org Thu Jul 27 03:24:48 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Thu, 27 Jul 2023 07:24:48 -0000 Subject: [Python-checkins] remove outdated `_asyncio` globals from globals-to-fix.tsv (#107334) Message-ID: https://github.com/python/cpython/commit/71b3eda02c08383bcacc97fd2f74ea9145b7cc96 commit: 71b3eda02c08383bcacc97fd2f74ea9145b7cc96 branch: main author: Kumar Aditya committer: kumaraditya303 date: 2023-07-27T07:24:44Z summary: remove outdated `_asyncio` globals from globals-to-fix.tsv (#107334) files: M Tools/c-analyzer/cpython/globals-to-fix.tsv diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 90bbc8928b428..033bd7a2ba515 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -424,8 +424,6 @@ Modules/_datetimemodule.c - seconds_per_day - Modules/_decimal/_decimal.c - global_state - ## state -Modules/_asynciomodule.c - fi_freelist - -Modules/_asynciomodule.c - fi_freelist_len - Modules/_ctypes/_ctypes.c - _ctypes_ptrtype_cache - Modules/_ctypes/_ctypes.c - global_state - Modules/_ctypes/ctypes.h - global_state - From webhook-mailer at python.org Thu Jul 27 04:02:23 2023 From: webhook-mailer at python.org (hugovk) Date: Thu, 27 Jul 2023 08:02:23 -0000 Subject: [Python-checkins] gh-106996: Rewrite turtle explanation (#107244) Message-ID: https://github.com/python/cpython/commit/ccd4253752ae2d84a5448281db1a1298ca8d8610 commit: ccd4253752ae2d84a5448281db1a1298ca8d8610 branch: main author: Daniele Procida committer: hugovk date: 2023-07-27T11:02:19+03:00 summary: gh-106996: Rewrite turtle explanation (#107244) Co-authored-by: Hugo van Kemenade files: M Doc/library/turtle.rst diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index c3561a0b2f9ce..4c84ae6a820f0 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -286,67 +286,16 @@ The turtle's screen can be customised, for example:: t.screen.bgcolor("orange") -.. _turtle-explanation: - -Explanation -=========== - -The :mod:`turtle` module is an extended reimplementation of the same-named -module from the Python standard distribution up to version Python 2.5. - -It tries to keep the merits of the old turtle module and to be (nearly) 100% -compatible with it. This means in the first place to enable the learning -programmer to use all the commands, classes and methods interactively when using -the module from within IDLE run with the ``-n`` switch. - -The turtle module provides turtle graphics primitives, in both object-oriented -and procedure-oriented ways. Because it uses :mod:`tkinter` for the underlying -graphics, it needs a version of Python installed with Tk support. - -The object-oriented interface uses essentially two+two classes: - -1. The :class:`TurtleScreen` class defines graphics windows as a playground for - the drawing turtles. Its constructor needs a :class:`tkinter.Canvas` or a - :class:`ScrolledCanvas` as argument. It should be used when :mod:`turtle` is - used as part of some application. - - The function :func:`Screen` returns a singleton object of a - :class:`TurtleScreen` subclass. This function should be used when - :mod:`turtle` is used as a standalone tool for doing graphics. - As a singleton object, inheriting from its class is not possible. - - All methods of TurtleScreen/Screen also exist as functions, i.e. as part of - the procedure-oriented interface. - -2. :class:`RawTurtle` (alias: :class:`RawPen`) defines Turtle objects which draw - on a :class:`TurtleScreen`. Its constructor needs a Canvas, ScrolledCanvas - or TurtleScreen as argument, so the RawTurtle objects know where to draw. - - Derived from RawTurtle is the subclass :class:`Turtle` (alias: :class:`Pen`), - which draws on "the" :class:`Screen` instance which is automatically - created, if not already present. - - All methods of RawTurtle/Turtle also exist as functions, i.e. part of the - procedure-oriented interface. - -The procedural interface provides functions which are derived from the methods -of the classes :class:`Screen` and :class:`Turtle`. They have the same names as -the corresponding methods. A screen object is automatically created whenever a -function derived from a Screen method is called. An (unnamed) turtle object is -automatically created whenever any of the functions derived from a Turtle method -is called. - -To use multiple turtles on a screen one has to use the object-oriented interface. +Turtle graphics reference +========================= .. note:: + In the following documentation the argument list for functions is given. Methods, of course, have the additional first argument *self* which is omitted here. -Turtle graphics reference -========================= - Turtle methods -------------- @@ -2451,6 +2400,41 @@ Public classes * ``a.rotate(angle)`` rotation +.. _turtle-explanation: + +Explanation +=========== + +A turtle object draws on a screen object, and there a number of key classes in +the turtle object-oriented interface that can be used to create them and relate +them to each other. + +A :class:`Turtle` instance will automatically create a :class:`Screen` +instance if one is not already present. + +``Turtle`` is a subclass of :class:`RawTurtle`, which *doesn't* automatically +create a drawing surface - a *canvas* will need to be provided or created for +it. The *canvas* can be a :class:`tkinter.Canvas`, :class:`ScrolledCanvas` +or :class:`TurtleScreen`. + + +:class:`TurtleScreen` is the basic drawing surface for a +turtle. :class:`Screen` is a subclass of ``TurtleScreen``, and +includes :ref:`some additional methods ` for managing its +appearance (including size and title) and behaviour. ``TurtleScreen``'s +constructor needs a :class:`tkinter.Canvas` or a +:class:`ScrolledCanvas` as an argument. + +The functional interface for turtle graphics uses the various methods of +``Turtle`` and ``TurtleScreen``/``Screen``. Behind the scenes, a screen +object is automatically created whenever a function derived from a ``Screen`` +method is called. Similarly, a turtle object is automatically created +whenever any of the functions derived from a Turtle method is called. + +To use multiple turtles on a screen, the object-oriented interface must be +used. + + Help and configuration ====================== From webhook-mailer at python.org Thu Jul 27 04:11:54 2023 From: webhook-mailer at python.org (hugovk) Date: Thu, 27 Jul 2023 08:11:54 -0000 Subject: [Python-checkins] [3.12] gh-106996: Rewrite turtle explanation (GH-107244) (#107335) Message-ID: https://github.com/python/cpython/commit/38c982d92eef1b2fa9418fc0127442690515cdbe commit: 38c982d92eef1b2fa9418fc0127442690515cdbe branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-27T08:11:50Z summary: [3.12] gh-106996: Rewrite turtle explanation (GH-107244) (#107335) Co-authored-by: Daniele Procida Co-authored-by: Hugo van Kemenade files: M Doc/library/turtle.rst diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index a85c3f466323f..cd8254153f765 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -277,67 +277,16 @@ The turtle's screen can be customised, for example:: t.screen.bgcolor("orange") -.. _turtle-explanation: - -Explanation -=========== - -The :mod:`turtle` module is an extended reimplementation of the same-named -module from the Python standard distribution up to version Python 2.5. - -It tries to keep the merits of the old turtle module and to be (nearly) 100% -compatible with it. This means in the first place to enable the learning -programmer to use all the commands, classes and methods interactively when using -the module from within IDLE run with the ``-n`` switch. - -The turtle module provides turtle graphics primitives, in both object-oriented -and procedure-oriented ways. Because it uses :mod:`tkinter` for the underlying -graphics, it needs a version of Python installed with Tk support. - -The object-oriented interface uses essentially two+two classes: - -1. The :class:`TurtleScreen` class defines graphics windows as a playground for - the drawing turtles. Its constructor needs a :class:`tkinter.Canvas` or a - :class:`ScrolledCanvas` as argument. It should be used when :mod:`turtle` is - used as part of some application. - - The function :func:`Screen` returns a singleton object of a - :class:`TurtleScreen` subclass. This function should be used when - :mod:`turtle` is used as a standalone tool for doing graphics. - As a singleton object, inheriting from its class is not possible. - - All methods of TurtleScreen/Screen also exist as functions, i.e. as part of - the procedure-oriented interface. - -2. :class:`RawTurtle` (alias: :class:`RawPen`) defines Turtle objects which draw - on a :class:`TurtleScreen`. Its constructor needs a Canvas, ScrolledCanvas - or TurtleScreen as argument, so the RawTurtle objects know where to draw. - - Derived from RawTurtle is the subclass :class:`Turtle` (alias: :class:`Pen`), - which draws on "the" :class:`Screen` instance which is automatically - created, if not already present. - - All methods of RawTurtle/Turtle also exist as functions, i.e. part of the - procedure-oriented interface. - -The procedural interface provides functions which are derived from the methods -of the classes :class:`Screen` and :class:`Turtle`. They have the same names as -the corresponding methods. A screen object is automatically created whenever a -function derived from a Screen method is called. An (unnamed) turtle object is -automatically created whenever any of the functions derived from a Turtle method -is called. - -To use multiple turtles on a screen one has to use the object-oriented interface. +Turtle graphics reference +========================= .. note:: + In the following documentation the argument list for functions is given. Methods, of course, have the additional first argument *self* which is omitted here. -Turtle graphics reference -========================= - Turtle methods -------------- @@ -2465,6 +2414,41 @@ Public classes * ``a.rotate(angle)`` rotation +.. _turtle-explanation: + +Explanation +=========== + +A turtle object draws on a screen object, and there a number of key classes in +the turtle object-oriented interface that can be used to create them and relate +them to each other. + +A :class:`Turtle` instance will automatically create a :class:`Screen` +instance if one is not already present. + +``Turtle`` is a subclass of :class:`RawTurtle`, which *doesn't* automatically +create a drawing surface - a *canvas* will need to be provided or created for +it. The *canvas* can be a :class:`tkinter.Canvas`, :class:`ScrolledCanvas` +or :class:`TurtleScreen`. + + +:class:`TurtleScreen` is the basic drawing surface for a +turtle. :class:`Screen` is a subclass of ``TurtleScreen``, and +includes :ref:`some additional methods ` for managing its +appearance (including size and title) and behaviour. ``TurtleScreen``'s +constructor needs a :class:`tkinter.Canvas` or a +:class:`ScrolledCanvas` as an argument. + +The functional interface for turtle graphics uses the various methods of +``Turtle`` and ``TurtleScreen``/``Screen``. Behind the scenes, a screen +object is automatically created whenever a function derived from a ``Screen`` +method is called. Similarly, a turtle object is automatically created +whenever any of the functions derived from a Turtle method is called. + +To use multiple turtles on a screen, the object-oriented interface must be +used. + + Help and configuration ====================== From webhook-mailer at python.org Thu Jul 27 04:12:21 2023 From: webhook-mailer at python.org (hugovk) Date: Thu, 27 Jul 2023 08:12:21 -0000 Subject: [Python-checkins] [3.11] gh-106996: Rewrite turtle explanation (GH-107244) (#107336) Message-ID: https://github.com/python/cpython/commit/8f0dc1878c37c7fd565371deb34fd1b3b36fdf9b commit: 8f0dc1878c37c7fd565371deb34fd1b3b36fdf9b branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: hugovk date: 2023-07-27T08:12:17Z summary: [3.11] gh-106996: Rewrite turtle explanation (GH-107244) (#107336) Co-authored-by: Daniele Procida Co-authored-by: Hugo van Kemenade files: M Doc/library/turtle.rst diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst index 262739ec2bbd5..a36948360de49 100644 --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -277,67 +277,16 @@ The turtle's screen can be customised, for example:: t.screen.bgcolor("orange") -.. _turtle-explanation: - -Explanation -=========== - -The :mod:`turtle` module is an extended reimplementation of the same-named -module from the Python standard distribution up to version Python 2.5. - -It tries to keep the merits of the old turtle module and to be (nearly) 100% -compatible with it. This means in the first place to enable the learning -programmer to use all the commands, classes and methods interactively when using -the module from within IDLE run with the ``-n`` switch. - -The turtle module provides turtle graphics primitives, in both object-oriented -and procedure-oriented ways. Because it uses :mod:`tkinter` for the underlying -graphics, it needs a version of Python installed with Tk support. - -The object-oriented interface uses essentially two+two classes: - -1. The :class:`TurtleScreen` class defines graphics windows as a playground for - the drawing turtles. Its constructor needs a :class:`tkinter.Canvas` or a - :class:`ScrolledCanvas` as argument. It should be used when :mod:`turtle` is - used as part of some application. - - The function :func:`Screen` returns a singleton object of a - :class:`TurtleScreen` subclass. This function should be used when - :mod:`turtle` is used as a standalone tool for doing graphics. - As a singleton object, inheriting from its class is not possible. - - All methods of TurtleScreen/Screen also exist as functions, i.e. as part of - the procedure-oriented interface. - -2. :class:`RawTurtle` (alias: :class:`RawPen`) defines Turtle objects which draw - on a :class:`TurtleScreen`. Its constructor needs a Canvas, ScrolledCanvas - or TurtleScreen as argument, so the RawTurtle objects know where to draw. - - Derived from RawTurtle is the subclass :class:`Turtle` (alias: :class:`Pen`), - which draws on "the" :class:`Screen` instance which is automatically - created, if not already present. - - All methods of RawTurtle/Turtle also exist as functions, i.e. part of the - procedure-oriented interface. - -The procedural interface provides functions which are derived from the methods -of the classes :class:`Screen` and :class:`Turtle`. They have the same names as -the corresponding methods. A screen object is automatically created whenever a -function derived from a Screen method is called. An (unnamed) turtle object is -automatically created whenever any of the functions derived from a Turtle method -is called. - -To use multiple turtles on a screen one has to use the object-oriented interface. +Turtle graphics reference +========================= .. note:: + In the following documentation the argument list for functions is given. Methods, of course, have the additional first argument *self* which is omitted here. -Turtle graphics reference -========================= - Turtle methods -------------- @@ -2434,6 +2383,41 @@ Public classes * ``a.rotate(angle)`` rotation +.. _turtle-explanation: + +Explanation +=========== + +A turtle object draws on a screen object, and there a number of key classes in +the turtle object-oriented interface that can be used to create them and relate +them to each other. + +A :class:`Turtle` instance will automatically create a :class:`Screen` +instance if one is not already present. + +``Turtle`` is a subclass of :class:`RawTurtle`, which *doesn't* automatically +create a drawing surface - a *canvas* will need to be provided or created for +it. The *canvas* can be a :class:`tkinter.Canvas`, :class:`ScrolledCanvas` +or :class:`TurtleScreen`. + + +:class:`TurtleScreen` is the basic drawing surface for a +turtle. :class:`Screen` is a subclass of ``TurtleScreen``, and +includes :ref:`some additional methods ` for managing its +appearance (including size and title) and behaviour. ``TurtleScreen``'s +constructor needs a :class:`tkinter.Canvas` or a +:class:`ScrolledCanvas` as an argument. + +The functional interface for turtle graphics uses the various methods of +``Turtle`` and ``TurtleScreen``/``Screen``. Behind the scenes, a screen +object is automatically created whenever a function derived from a ``Screen`` +method is called. Similarly, a turtle object is automatically created +whenever any of the functions derived from a Turtle method is called. + +To use multiple turtles on a screen, the object-oriented interface must be +used. + + Help and configuration ====================== From webhook-mailer at python.org Thu Jul 27 04:25:23 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 27 Jul 2023 08:25:23 -0000 Subject: [Python-checkins] gh-103731: Remove unneeded checks for TCL_WIDE_INT_TYPE (GH-103732) Message-ID: https://github.com/python/cpython/commit/6e850c30bb7085c80a97982d00c9c20e7d23ff84 commit: 6e850c30bb7085c80a97982d00c9c20e7d23ff84 branch: main author: Christopher Chavez committer: serhiy-storchaka date: 2023-07-27T11:25:19+03:00 summary: gh-103731: Remove unneeded checks for TCL_WIDE_INT_TYPE (GH-103732) files: M Modules/_tkinter.c diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c index 406e01cd75ffd..663b411768362 100644 --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -911,16 +911,13 @@ AsObj(PyObject *value) if (PyLong_CheckExact(value)) { int overflow; long longValue; -#ifdef TCL_WIDE_INT_TYPE Tcl_WideInt wideValue; -#endif longValue = PyLong_AsLongAndOverflow(value, &overflow); if (!overflow) { return Tcl_NewLongObj(longValue); } /* If there is an overflow in the long conversion, fall through to wideInt handling. */ -#ifdef TCL_WIDE_INT_TYPE if (_PyLong_AsByteArray((PyLongObject *)value, (unsigned char *)(void *)&wideValue, sizeof(wideValue), @@ -929,7 +926,6 @@ AsObj(PyObject *value) return Tcl_NewWideIntObj(wideValue); } PyErr_Clear(); -#endif /* If there is an overflow in the wideInt conversion, fall through to bignum handling. */ return asBignumObj(value); From webhook-mailer at python.org Thu Jul 27 07:53:25 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 27 Jul 2023 11:53:25 -0000 Subject: [Python-checkins] Bump some docs dependencies to resolve a Dependabot security alert (#107341) Message-ID: https://github.com/python/cpython/commit/f84d77b4e07aeb6241c1ff9932627d3ba059efa8 commit: f84d77b4e07aeb6241c1ff9932627d3ba059efa8 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-27T11:53:21Z summary: Bump some docs dependencies to resolve a Dependabot security alert (#107341) files: M Doc/requirements-oldest-sphinx.txt diff --git a/Doc/requirements-oldest-sphinx.txt b/Doc/requirements-oldest-sphinx.txt index f7e0665bde445..94611ca22f09f 100644 --- a/Doc/requirements-oldest-sphinx.txt +++ b/Doc/requirements-oldest-sphinx.txt @@ -14,11 +14,10 @@ python-docs-theme>=2022.1 # Docutils<0.17, Jinja2<3, and MarkupSafe<2 are additionally specified as # Sphinx 3.2 is incompatible with newer releases of these packages. -Sphinx==3.2.1 alabaster==0.7.13 Babel==2.12.1 -certifi==2022.12.7 -charset-normalizer==3.1.0 +certifi==2023.7.22 +charset-normalizer==3.2.0 colorama==0.4.6 docutils==0.16 idna==3.4 @@ -29,10 +28,11 @@ packaging==23.1 Pygments==2.15.1 requests==2.31.0 snowballstemmer==2.2.0 +Sphinx==3.2.1 sphinxcontrib-applehelp==1.0.4 sphinxcontrib-devhelp==1.0.2 sphinxcontrib-htmlhelp==2.0.1 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 -urllib3==1.26.15 +urllib3==2.0.4 From webhook-mailer at python.org Thu Jul 27 08:13:06 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 27 Jul 2023 12:13:06 -0000 Subject: [Python-checkins] [3.12] Bump some docs dependencies to resolve a Dependabot security alert (GH-107341) (#107342) Message-ID: https://github.com/python/cpython/commit/0063ad8189cd4ac84775eb9150b7f3373842568a commit: 0063ad8189cd4ac84775eb9150b7f3373842568a branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-27T13:13:02+01:00 summary: [3.12] Bump some docs dependencies to resolve a Dependabot security alert (GH-107341) (#107342) Bump some docs dependencies to resolve a Dependabot security alert (GH-107341) (cherry picked from commit f84d77b4e07aeb6241c1ff9932627d3ba059efa8) Co-authored-by: Alex Waygood files: M Doc/requirements-oldest-sphinx.txt diff --git a/Doc/requirements-oldest-sphinx.txt b/Doc/requirements-oldest-sphinx.txt index d0390a04ea6dd..94611ca22f09f 100644 --- a/Doc/requirements-oldest-sphinx.txt +++ b/Doc/requirements-oldest-sphinx.txt @@ -14,11 +14,10 @@ python-docs-theme>=2022.1 # Docutils<0.17, Jinja2<3, and MarkupSafe<2 are additionally specified as # Sphinx 3.2 is incompatible with newer releases of these packages. -Sphinx==3.2.1 alabaster==0.7.13 Babel==2.12.1 -certifi==2022.12.7 -charset-normalizer==3.1.0 +certifi==2023.7.22 +charset-normalizer==3.2.0 colorama==0.4.6 docutils==0.16 idna==3.4 @@ -27,12 +26,13 @@ Jinja2==2.11.3 MarkupSafe==1.1.1 packaging==23.1 Pygments==2.15.1 -requests==2.29.0 +requests==2.31.0 snowballstemmer==2.2.0 +Sphinx==3.2.1 sphinxcontrib-applehelp==1.0.4 sphinxcontrib-devhelp==1.0.2 sphinxcontrib-htmlhelp==2.0.1 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 -urllib3==1.26.15 +urllib3==2.0.4 From webhook-mailer at python.org Thu Jul 27 08:15:10 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 27 Jul 2023 12:15:10 -0000 Subject: [Python-checkins] [3.11] Bump some docs dependencies to resolve a Dependabot security alert (GH-107341) (#107343) Message-ID: https://github.com/python/cpython/commit/f106254e0b3c83d9064763d6132b3fe997da901b commit: f106254e0b3c83d9064763d6132b3fe997da901b branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-27T12:15:06Z summary: [3.11] Bump some docs dependencies to resolve a Dependabot security alert (GH-107341) (#107343) Bump some docs dependencies to resolve a Dependabot security alert (GH-107341) (cherry picked from commit f84d77b4e07aeb6241c1ff9932627d3ba059efa8) Co-authored-by: Alex Waygood files: M Doc/requirements-oldest-sphinx.txt diff --git a/Doc/requirements-oldest-sphinx.txt b/Doc/requirements-oldest-sphinx.txt index d0390a04ea6dd..94611ca22f09f 100644 --- a/Doc/requirements-oldest-sphinx.txt +++ b/Doc/requirements-oldest-sphinx.txt @@ -14,11 +14,10 @@ python-docs-theme>=2022.1 # Docutils<0.17, Jinja2<3, and MarkupSafe<2 are additionally specified as # Sphinx 3.2 is incompatible with newer releases of these packages. -Sphinx==3.2.1 alabaster==0.7.13 Babel==2.12.1 -certifi==2022.12.7 -charset-normalizer==3.1.0 +certifi==2023.7.22 +charset-normalizer==3.2.0 colorama==0.4.6 docutils==0.16 idna==3.4 @@ -27,12 +26,13 @@ Jinja2==2.11.3 MarkupSafe==1.1.1 packaging==23.1 Pygments==2.15.1 -requests==2.29.0 +requests==2.31.0 snowballstemmer==2.2.0 +Sphinx==3.2.1 sphinxcontrib-applehelp==1.0.4 sphinxcontrib-devhelp==1.0.2 sphinxcontrib-htmlhelp==2.0.1 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 -urllib3==1.26.15 +urllib3==2.0.4 From webhook-mailer at python.org Thu Jul 27 08:32:34 2023 From: webhook-mailer at python.org (markshannon) Date: Thu, 27 Jul 2023 12:32:34 -0000 Subject: [Python-checkins] GH-106897: Add `RERAISE` event to `sys.monitoring`. (GH-107291) Message-ID: https://github.com/python/cpython/commit/766d2518ae8384c6bd7f82727defeb86847ccf64 commit: 766d2518ae8384c6bd7f82727defeb86847ccf64 branch: main author: Mark Shannon committer: markshannon date: 2023-07-27T13:32:30+01:00 summary: GH-106897: Add `RERAISE` event to `sys.monitoring`. (GH-107291) * Ensures that exception handling events are balanced. Each [re]raise event has a matching unwind/handled event. files: A Misc/NEWS.d/next/Core and Builtins/2023-07-26-12-18-10.gh-issue-106897.EsGurc.rst M Include/cpython/code.h M Include/internal/pycore_instruments.h M Lib/test/test_monitoring.py M Python/bytecodes.c M Python/ceval.c M Python/generated_cases.c.h M Python/instrumentation.c diff --git a/Include/cpython/code.h b/Include/cpython/code.h index b8953e26ecddc..24c5ec23590c9 100644 --- a/Include/cpython/code.h +++ b/Include/cpython/code.h @@ -10,9 +10,9 @@ extern "C" { /* Count of all "real" monitoring events (not derived from other events) */ -#define _PY_MONITORING_UNGROUPED_EVENTS 14 +#define _PY_MONITORING_UNGROUPED_EVENTS 15 /* Count of all monitoring events */ -#define _PY_MONITORING_EVENTS 16 +#define _PY_MONITORING_EVENTS 17 /* Table of which tools are active for each monitored event. */ typedef struct _Py_Monitors { diff --git a/Include/internal/pycore_instruments.h b/Include/internal/pycore_instruments.h index 9fb3952227af1..cfa5d09a4704f 100644 --- a/Include/internal/pycore_instruments.h +++ b/Include/internal/pycore_instruments.h @@ -36,12 +36,13 @@ extern "C" { #define PY_MONITORING_EVENT_EXCEPTION_HANDLED 11 #define PY_MONITORING_EVENT_PY_UNWIND 12 #define PY_MONITORING_EVENT_PY_THROW 13 +#define PY_MONITORING_EVENT_RERAISE 14 /* Ancilliary events */ -#define PY_MONITORING_EVENT_C_RETURN 14 -#define PY_MONITORING_EVENT_C_RAISE 15 +#define PY_MONITORING_EVENT_C_RETURN 15 +#define PY_MONITORING_EVENT_C_RAISE 16 typedef uint32_t _PyMonitoringEventSet; diff --git a/Lib/test/test_monitoring.py b/Lib/test/test_monitoring.py index f854c58284666..7538786ccf218 100644 --- a/Lib/test/test_monitoring.py +++ b/Lib/test/test_monitoring.py @@ -8,7 +8,7 @@ import textwrap import types import unittest - +import asyncio PAIR = (0,1) @@ -243,7 +243,6 @@ def check_events(self, func, expected=None): expected = func.events self.assertEqual(events, expected) - class MonitoringEventsTest(MonitoringEventsBase, unittest.TestCase): def test_just_pass(self): @@ -632,7 +631,7 @@ def __call__(self, code, offset, exc): class CheckEvents(MonitoringTestBase, unittest.TestCase): - def check_events(self, func, expected, tool=TEST_TOOL, recorders=(ExceptionRecorder,)): + def get_events(self, func, tool, recorders): try: self.assertEqual(sys.monitoring._all_events(), {}) event_list = [] @@ -646,19 +645,63 @@ def check_events(self, func, expected, tool=TEST_TOOL, recorders=(ExceptionRecor sys.monitoring.set_events(tool, 0) for recorder in recorders: sys.monitoring.register_callback(tool, recorder.event_type, None) - self.assertEqual(event_list, expected) + return event_list finally: sys.monitoring.set_events(tool, 0) for recorder in recorders: sys.monitoring.register_callback(tool, recorder.event_type, None) + def check_events(self, func, expected, tool=TEST_TOOL, recorders=(ExceptionRecorder,)): + events = self.get_events(func, tool, recorders) + if events != expected: + print(events, file = sys.stderr) + self.assertEqual(events, expected) + + def check_balanced(self, func, recorders): + events = self.get_events(func, TEST_TOOL, recorders) + self.assertEqual(len(events)%2, 0) + for r, h in zip(events[::2],events[1::2]): + r0 = r[0] + self.assertIn(r0, ("raise", "reraise")) + h0 = h[0] + self.assertIn(h0, ("handled", "unwind")) + + + class StopiterationRecorder(ExceptionRecorder): event_type = E.STOP_ITERATION -class ExceptionMontoringTest(CheckEvents): +class ReraiseRecorder(ExceptionRecorder): + + event_type = E.RERAISE + + def __call__(self, code, offset, exc): + self.events.append(("reraise", type(exc))) + +class UnwindRecorder(ExceptionRecorder): + + event_type = E.PY_UNWIND + + def __call__(self, *args): + self.events.append(("unwind", None)) + +class ExceptionHandledRecorder(ExceptionRecorder): + + event_type = E.EXCEPTION_HANDLED + + def __call__(self, code, offset, exc): + self.events.append(("handled", type(exc))) - recorder = ExceptionRecorder +class ExceptionMonitoringTest(CheckEvents): + + + exception_recorders = ( + ExceptionRecorder, + ReraiseRecorder, + ExceptionHandledRecorder, + UnwindRecorder + ) def test_simple_try_except(self): @@ -672,6 +715,8 @@ def func1(): self.check_events(func1, [("raise", KeyError)]) + def test_implicit_stop_iteration(self): + def gen(): yield 1 return 2 @@ -682,6 +727,117 @@ def implicit_stop_iteration(): self.check_events(implicit_stop_iteration, [("raise", StopIteration)], recorders=(StopiterationRecorder,)) + initial = [ + ("raise", ZeroDivisionError), + ("handled", ZeroDivisionError) + ] + + reraise = [ + ("reraise", ZeroDivisionError), + ("handled", ZeroDivisionError) + ] + + def test_explicit_reraise(self): + + def func(): + try: + try: + 1/0 + except: + raise + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + def test_explicit_reraise_named(self): + + def func(): + try: + try: + 1/0 + except Exception as ex: + raise + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + def test_implicit_reraise(self): + + def func(): + try: + try: + 1/0 + except ValueError: + pass + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + + def test_implicit_reraise_named(self): + + def func(): + try: + try: + 1/0 + except ValueError as ex: + pass + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + def test_try_finally(self): + + def func(): + try: + try: + 1/0 + finally: + pass + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + def test_async_for(self): + + def func(): + + async def async_generator(): + for i in range(1): + raise ZeroDivisionError + yield i + + async def async_loop(): + try: + async for item in async_generator(): + pass + except Exception: + pass + + try: + async_loop().send(None) + except StopIteration: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + class LineRecorder: event_type = E.LINE @@ -733,12 +889,12 @@ def func1(): line3 = 3 self.check_events(func1, recorders = MANY_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'func1', sys.monitoring.MISSING), ('line', 'func1', 1), ('line', 'func1', 2), ('line', 'func1', 3), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2)]) def test_c_call(self): @@ -749,14 +905,14 @@ def func2(): line3 = 3 self.check_events(func2, recorders = MANY_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'func2', sys.monitoring.MISSING), ('line', 'func2', 1), ('line', 'func2', 2), ('call', 'append', [2]), ('C return', 'append', [2]), ('line', 'func2', 3), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2)]) def test_try_except(self): @@ -770,7 +926,7 @@ def func3(): line = 6 self.check_events(func3, recorders = MANY_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'func3', sys.monitoring.MISSING), ('line', 'func3', 1), ('line', 'func3', 2), @@ -779,7 +935,7 @@ def func3(): ('line', 'func3', 4), ('line', 'func3', 5), ('line', 'func3', 6), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2)]) class InstructionRecorder: @@ -791,7 +947,7 @@ def __init__(self, events): def __call__(self, code, offset): # Filter out instructions in check_events to lower noise - if code.co_name != "check_events": + if code.co_name != "get_events": self.events.append(("instruction", code.co_name, offset)) @@ -808,7 +964,7 @@ def func1(): line3 = 3 self.check_events(func1, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func1', 1), ('instruction', 'func1', 2), ('instruction', 'func1', 4), @@ -819,7 +975,7 @@ def func1(): ('instruction', 'func1', 10), ('instruction', 'func1', 12), ('instruction', 'func1', 14), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) def test_c_call(self): @@ -829,7 +985,7 @@ def func2(): line3 = 3 self.check_events(func2, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func2', 1), ('instruction', 'func2', 2), ('instruction', 'func2', 4), @@ -843,7 +999,7 @@ def func2(): ('instruction', 'func2', 40), ('instruction', 'func2', 42), ('instruction', 'func2', 44), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) def test_try_except(self): @@ -856,7 +1012,7 @@ def func3(): line = 6 self.check_events(func3, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func3', 1), ('instruction', 'func3', 2), ('line', 'func3', 2), @@ -876,7 +1032,7 @@ def func3(): ('instruction', 'func3', 30), ('instruction', 'func3', 32), ('instruction', 'func3', 34), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) def test_with_restart(self): def func1(): @@ -885,7 +1041,7 @@ def func1(): line3 = 3 self.check_events(func1, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func1', 1), ('instruction', 'func1', 2), ('instruction', 'func1', 4), @@ -896,12 +1052,12 @@ def func1(): ('instruction', 'func1', 10), ('instruction', 'func1', 12), ('instruction', 'func1', 14), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) sys.monitoring.restart_events() self.check_events(func1, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func1', 1), ('instruction', 'func1', 2), ('instruction', 'func1', 4), @@ -912,7 +1068,7 @@ def func1(): ('instruction', 'func1', 10), ('instruction', 'func1', 12), ('instruction', 'func1', 14), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) class TestInstallIncrementallly(MonitoringTestBase, unittest.TestCase): @@ -1114,7 +1270,7 @@ def func(): ('branch', 'func', 2, 2)]) self.check_events(func, recorders = JUMP_BRANCH_AND_LINE_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func', 1), ('line', 'func', 2), ('branch', 'func', 2, 2), @@ -1130,7 +1286,7 @@ def func(): ('jump', 'func', 4, 2), ('line', 'func', 2), ('branch', 'func', 2, 2), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) def test_except_star(self): @@ -1149,7 +1305,7 @@ def func(): self.check_events(func, recorders = JUMP_BRANCH_AND_LINE_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func', 1), ('line', 'func', 2), ('line', 'func', 3), @@ -1160,10 +1316,10 @@ def func(): ('jump', 'func', 5, 5), ('jump', 'func', 5, '[offset=112]'), ('branch', 'func', '[offset=118]', '[offset=120]'), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) self.check_events(func, recorders = FLOW_AND_LINE_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func', 1), ('line', 'func', 2), ('line', 'func', 3), @@ -1177,7 +1333,7 @@ def func(): ('jump', 'func', 5, '[offset=112]'), ('branch', 'func', '[offset=118]', '[offset=120]'), ('return', None), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) class TestLoadSuperAttr(CheckEvents): RECORDERS = CallRecorder, LineRecorder, CRaiseRecorder, CReturnRecorder @@ -1229,7 +1385,7 @@ def f(): """ d = self._exec_super(codestr, optimized) expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'f', sys.monitoring.MISSING), ('line', 'f', 1), ('call', 'method', d["b"]), @@ -1242,7 +1398,7 @@ def f(): ('call', 'method', 1), ('line', 'method', 1), ('line', 'method', 1), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2), ] return d["f"], expected @@ -1280,7 +1436,7 @@ def f(): """ d = self._exec_super(codestr, optimized) expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'f', sys.monitoring.MISSING), ('line', 'f', 1), ('line', 'f', 2), @@ -1293,7 +1449,7 @@ def f(): ('C raise', 'super', 1), ('line', 'f', 3), ('line', 'f', 4), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2), ] return d["f"], expected @@ -1321,7 +1477,7 @@ def f(): """ d = self._exec_super(codestr, optimized) expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'f', sys.monitoring.MISSING), ('line', 'f', 1), ('call', 'method', d["b"]), @@ -1330,7 +1486,7 @@ def f(): ('C return', 'super', sys.monitoring.MISSING), ('line', 'method', 2), ('line', 'method', 1), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2) ] return d["f"], expected @@ -1355,7 +1511,7 @@ def f(): def get_expected(name, call_method, ns): repr_arg = 0 if name == "int" else sys.monitoring.MISSING return [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'f', sys.monitoring.MISSING), ('line', 'f', 1), ('call', 'method', ns["c"]), @@ -1368,7 +1524,7 @@ def get_expected(name, call_method, ns): ('C return', '__repr__', repr_arg), ] if call_method else [] ), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2), ] diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-26-12-18-10.gh-issue-106897.EsGurc.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-12-18-10.gh-issue-106897.EsGurc.rst new file mode 100644 index 0000000000000..d787dc4aad2d2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-12-18-10.gh-issue-106897.EsGurc.rst @@ -0,0 +1,3 @@ +Add a ``RERAISE`` event to ``sys.monitoring``, which occurs when an +exception is reraise, either explicitly by a plain ``raise`` statement, or +implicitly in an ``except`` or ``finally`` block. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 0f1e0073a3f60..da9d8e2a56a41 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -720,7 +720,11 @@ dummy_func( exc = args[0]; /* fall through */ case 0: - ERROR_IF(do_raise(tstate, exc, cause), exception_unwind); + if (do_raise(tstate, exc, cause)) { + assert(oparg == 0); + monitor_reraise(tstate, frame, next_instr-1); + goto exception_unwind; + } break; default: _PyErr_SetString(tstate, PyExc_SystemError, @@ -1047,6 +1051,7 @@ dummy_func( assert(exc && PyExceptionInstance_Check(exc)); Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } @@ -1058,6 +1063,7 @@ dummy_func( else { Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } } @@ -1072,6 +1078,7 @@ dummy_func( } else { _PyErr_SetRaisedException(tstate, Py_NewRef(exc_value)); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } } diff --git a/Python/ceval.c b/Python/ceval.c index 688c019e2add1..a5d37c0629df9 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -184,6 +184,9 @@ lltrace_resume_frame(_PyInterpreterFrame *frame) static void monitor_raise(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr); +static void monitor_reraise(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr); static int monitor_stop_iteration(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr); @@ -840,7 +843,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int } } monitor_raise(tstate, frame, next_instr-1); - exception_unwind: { /* We can't use frame->f_lasti here, as RERAISE may have set it */ @@ -1954,6 +1956,16 @@ monitor_raise(PyThreadState *tstate, _PyInterpreterFrame *frame, do_monitor_exc(tstate, frame, instr, PY_MONITORING_EVENT_RAISE); } +static void +monitor_reraise(PyThreadState *tstate, _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr) +{ + if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_RERAISE)) { + return; + } + do_monitor_exc(tstate, frame, instr, PY_MONITORING_EVENT_RERAISE); +} + static int monitor_stop_iteration(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr) diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 02ad69a6bc4da..37ffb560f1457 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -881,7 +881,11 @@ exc = args[0]; /* fall through */ case 0: - if (do_raise(tstate, exc, cause)) { STACK_SHRINK(oparg); goto exception_unwind; } + if (do_raise(tstate, exc, cause)) { + assert(oparg == 0); + monitor_reraise(tstate, frame, next_instr-1); + goto exception_unwind; + } break; default: _PyErr_SetString(tstate, PyExc_SystemError, @@ -1237,6 +1241,7 @@ assert(exc && PyExceptionInstance_Check(exc)); Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } @@ -1251,6 +1256,7 @@ else { Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } STACK_SHRINK(2); @@ -1274,6 +1280,7 @@ } else { _PyErr_SetRaisedException(tstate, Py_NewRef(exc_value)); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } STACK_SHRINK(1); diff --git a/Python/instrumentation.c b/Python/instrumentation.c index c3515d2c5a3ad..280e13d65526b 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -2030,6 +2030,7 @@ static const char *const event_names [] = { [PY_MONITORING_EVENT_C_RETURN] = "C_RETURN", [PY_MONITORING_EVENT_PY_THROW] = "PY_THROW", [PY_MONITORING_EVENT_RAISE] = "RAISE", + [PY_MONITORING_EVENT_RERAISE] = "RERAISE", [PY_MONITORING_EVENT_EXCEPTION_HANDLED] = "EXCEPTION_HANDLED", [PY_MONITORING_EVENT_C_RAISE] = "C_RAISE", [PY_MONITORING_EVENT_PY_UNWIND] = "PY_UNWIND", From webhook-mailer at python.org Thu Jul 27 09:15:29 2023 From: webhook-mailer at python.org (iritkatriel) Date: Thu, 27 Jul 2023 13:15:29 -0000 Subject: [Python-checkins] gh-105481: remove dependency of _inline_cache_entries on opname (#107339) Message-ID: https://github.com/python/cpython/commit/d77d973335835bd744be8106010061cb338b0ae1 commit: d77d973335835bd744be8106010061cb338b0ae1 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2023-07-27T14:15:25+01:00 summary: gh-105481: remove dependency of _inline_cache_entries on opname (#107339) files: M Include/internal/pycore_opcode.h M Lib/dis.py M Lib/opcode.py M Lib/test/test__opcode.py M Tools/build/generate_opcode_h.py diff --git a/Include/internal/pycore_opcode.h b/Include/internal/pycore_opcode.h index d7f6b84e95c4f..aff09a2a926ae 100644 --- a/Include/internal/pycore_opcode.h +++ b/Include/internal/pycore_opcode.h @@ -19,20 +19,20 @@ extern const uint8_t _PyOpcode_Deopt[256]; #ifdef NEED_OPCODE_TABLES const uint8_t _PyOpcode_Caches[256] = { - [TO_BOOL] = 3, - [BINARY_SUBSCR] = 1, - [STORE_SUBSCR] = 1, + [LOAD_GLOBAL] = 4, + [BINARY_OP] = 1, [UNPACK_SEQUENCE] = 1, + [COMPARE_OP] = 1, + [BINARY_SUBSCR] = 1, [FOR_ITER] = 1, - [STORE_ATTR] = 4, + [LOAD_SUPER_ATTR] = 1, [LOAD_ATTR] = 9, - [COMPARE_OP] = 1, - [LOAD_GLOBAL] = 4, - [BINARY_OP] = 1, + [STORE_ATTR] = 4, + [CALL] = 3, + [STORE_SUBSCR] = 1, [SEND] = 1, [JUMP_BACKWARD] = 1, - [LOAD_SUPER_ATTR] = 1, - [CALL] = 3, + [TO_BOOL] = 3, }; const uint8_t _PyOpcode_Deopt[256] = { diff --git a/Lib/dis.py b/Lib/dis.py index 184eb14a5b2b2..b167773dfe7b0 100644 --- a/Lib/dis.py +++ b/Lib/dis.py @@ -288,13 +288,16 @@ def show_code(co, *, file=None): _OPNAME_WIDTH = 20 _OPARG_WIDTH = 5 +def _get_cache_size(opname): + return _inline_cache_entries.get(opname, 0) + def _get_jump_target(op, arg, offset): """Gets the bytecode offset of the jump target if this is a jump instruction. Otherwise return None. """ deop = _deoptop(op) - caches = _inline_cache_entries[deop] + caches = _get_cache_size(_all_opname[deop]) if deop in hasjrel: if _is_backward_jump(deop): arg = -arg @@ -353,7 +356,7 @@ def cache_offset(self): @property def end_offset(self): """End index of the cache entries following the operation.""" - return self.cache_offset + _inline_cache_entries[self.opcode]*2 + return self.cache_offset + _get_cache_size(_all_opname[self.opcode])*2 @property def jump_target(self): @@ -535,7 +538,7 @@ def _get_instructions_bytes(code, varname_from_oparg=None, argrepr = '' positions = Positions(*next(co_positions, ())) deop = _deoptop(op) - caches = _inline_cache_entries[deop] + caches = _get_cache_size(_all_opname[deop]) op = code[offset] if arg is not None: # Set argval to the dereferenced value of the argument when @@ -679,7 +682,7 @@ def _disassemble_bytes(code, lasti=-1, varname_from_oparg=None, else: # Each CACHE takes 2 bytes is_current_instr = instr.offset <= lasti \ - <= instr.offset + 2 * _inline_cache_entries[_deoptop(instr.opcode)] + <= instr.offset + 2 * _get_cache_size(_all_opname[_deoptop(instr.opcode)]) print(instr._disassemble(lineno_width, is_current_instr, offset_width), file=file) if exception_entries: @@ -712,7 +715,7 @@ def _unpack_opargs(code): continue op = code[i] deop = _deoptop(op) - caches = _inline_cache_entries[deop] + caches = _get_cache_size(_all_opname[deop]) if deop in hasarg: arg = code[i+1] | extended_arg extended_arg = (arg << 8) if deop == EXTENDED_ARG else 0 diff --git a/Lib/opcode.py b/Lib/opcode.py index 36831d8a122bf..bed922399821a 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -347,6 +347,6 @@ def pseudo_op(name, op, real_ops): }, } -_inline_cache_entries = [ - sum(_cache_format.get(opname[opcode], {}).values()) for opcode in range(256) -] +_inline_cache_entries = { + name : sum(value.values()) for (name, value) in _cache_format.items() +} diff --git a/Lib/test/test__opcode.py b/Lib/test/test__opcode.py index b3a9bcbe16045..c1f612dc4a63c 100644 --- a/Lib/test/test__opcode.py +++ b/Lib/test/test__opcode.py @@ -106,7 +106,7 @@ def test_specialization_stats(self): specialized_opcodes = [ op.lower() for op in opcode._specializations - if opcode._inline_cache_entries[opcode.opmap[op]] + if opcode._inline_cache_entries.get(op, 0) ] self.assertIn('load_attr', specialized_opcodes) self.assertIn('binary_subscr', specialized_opcodes) diff --git a/Tools/build/generate_opcode_h.py b/Tools/build/generate_opcode_h.py index 179abcf989e9b..19e5eab822a23 100644 --- a/Tools/build/generate_opcode_h.py +++ b/Tools/build/generate_opcode_h.py @@ -120,9 +120,8 @@ def main(opcode_py, iobj.write("\n#ifdef NEED_OPCODE_TABLES\n") iobj.write("\nconst uint8_t _PyOpcode_Caches[256] = {\n") - for i, entries in enumerate(opcode["_inline_cache_entries"]): - if entries: - iobj.write(f" [{opname[i]}] = {entries},\n") + for name, entries in opcode["_inline_cache_entries"].items(): + iobj.write(f" [{name}] = {entries},\n") iobj.write("};\n") deoptcodes = {} From webhook-mailer at python.org Thu Jul 27 10:27:16 2023 From: webhook-mailer at python.org (markshannon) Date: Thu, 27 Jul 2023 14:27:16 -0000 Subject: [Python-checkins] GH-106895: Raise a `ValueError` when attempting to disable events that cannot be disabled. (GH-107337) Message-ID: https://github.com/python/cpython/commit/c6539b36c163efff3d6ed02b938a6151325f4db7 commit: c6539b36c163efff3d6ed02b938a6151325f4db7 branch: main author: Mark Shannon committer: markshannon date: 2023-07-27T15:27:11+01:00 summary: GH-106895: Raise a `ValueError` when attempting to disable events that cannot be disabled. (GH-107337) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-26-18-53-34.gh-issue-106895.DdEwV8.rst M Include/internal/pycore_instruments.h M Lib/test/test_monitoring.py M Objects/classobject.c M Python/bytecodes.c M Python/ceval.c M Python/executor_cases.c.h M Python/generated_cases.c.h M Python/instrumentation.c diff --git a/Include/internal/pycore_instruments.h b/Include/internal/pycore_instruments.h index cfa5d09a4704f..ccccd54a2f70a 100644 --- a/Include/internal/pycore_instruments.h +++ b/Include/internal/pycore_instruments.h @@ -28,7 +28,8 @@ extern "C" { #define PY_MONITORING_EVENT_BRANCH 8 #define PY_MONITORING_EVENT_STOP_ITERATION 9 -#define PY_MONITORING_INSTRUMENTED_EVENTS 10 +#define PY_MONITORING_IS_INSTRUMENTED_EVENT(ev) \ + ((ev) <= PY_MONITORING_EVENT_STOP_ITERATION) /* Other events, mainly exceptions */ diff --git a/Lib/test/test_monitoring.py b/Lib/test/test_monitoring.py index 7538786ccf218..7098f483e0ee7 100644 --- a/Lib/test/test_monitoring.py +++ b/Lib/test/test_monitoring.py @@ -136,20 +136,27 @@ def test_c_return_count(self): E = sys.monitoring.events -SIMPLE_EVENTS = [ +INSTRUMENTED_EVENTS = [ (E.PY_START, "start"), (E.PY_RESUME, "resume"), (E.PY_RETURN, "return"), (E.PY_YIELD, "yield"), (E.JUMP, "jump"), (E.BRANCH, "branch"), +] + +EXCEPT_EVENTS = [ (E.RAISE, "raise"), (E.PY_UNWIND, "unwind"), (E.EXCEPTION_HANDLED, "exception_handled"), +] + +SIMPLE_EVENTS = INSTRUMENTED_EVENTS + EXCEPT_EVENTS + [ (E.C_RAISE, "c_raise"), (E.C_RETURN, "c_return"), ] + SIMPLE_EVENT_SET = functools.reduce(operator.or_, [ev for (ev, _) in SIMPLE_EVENTS], 0) | E.CALL @@ -618,6 +625,49 @@ def func2(): self.check_lines(func2, [1,2,3,4,5,6]) +class TestDisable(MonitoringTestBase, unittest.TestCase): + + def gen(self, cond): + for i in range(10): + if cond: + yield 1 + else: + yield 2 + + def raise_handle_reraise(self): + try: + 1/0 + except: + raise + + def test_disable_legal_events(self): + for event, name in INSTRUMENTED_EVENTS: + try: + counter = CounterWithDisable() + counter.disable = True + sys.monitoring.register_callback(TEST_TOOL, event, counter) + sys.monitoring.set_events(TEST_TOOL, event) + for _ in self.gen(1): + pass + self.assertLess(counter.count, 4) + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.register_callback(TEST_TOOL, event, None) + + + def test_disable_illegal_events(self): + for event, name in EXCEPT_EVENTS: + try: + counter = CounterWithDisable() + counter.disable = True + sys.monitoring.register_callback(TEST_TOOL, event, counter) + sys.monitoring.set_events(TEST_TOOL, event) + with self.assertRaises(ValueError): + self.raise_handle_reraise() + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.register_callback(TEST_TOOL, event, None) + class ExceptionRecorder: diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-26-18-53-34.gh-issue-106895.DdEwV8.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-18-53-34.gh-issue-106895.DdEwV8.rst new file mode 100644 index 0000000000000..370a29d34c860 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-18-53-34.gh-issue-106895.DdEwV8.rst @@ -0,0 +1,2 @@ +Raise a ``ValueError`` when a monitoring callback funtion returns +``DISABLE`` for events that cannot be disabled locally. diff --git a/Objects/classobject.c b/Objects/classobject.c index 6f4457d42dda9..5471045d777c9 100644 --- a/Objects/classobject.c +++ b/Objects/classobject.c @@ -48,6 +48,7 @@ method_vectorcall(PyObject *method, PyObject *const *args, PyObject *self = PyMethod_GET_SELF(method); PyObject *func = PyMethod_GET_FUNCTION(method); Py_ssize_t nargs = PyVectorcall_NARGS(nargsf); + assert(nargs == 0 || args[nargs-1]); PyObject *result; if (nargsf & PY_VECTORCALL_ARGUMENTS_OFFSET) { @@ -56,6 +57,7 @@ method_vectorcall(PyObject *method, PyObject *const *args, nargs += 1; PyObject *tmp = newargs[0]; newargs[0] = self; + assert(newargs[nargs-1]); result = _PyObject_VectorcallTstate(tstate, func, newargs, nargs, kwnames); newargs[0] = tmp; diff --git a/Python/bytecodes.c b/Python/bytecodes.c index da9d8e2a56a41..2b871e8546efb 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2672,7 +2672,12 @@ dummy_func( assert(val && PyExceptionInstance_Check(val)); exc = PyExceptionInstance_Class(val); tb = PyException_GetTraceback(val); - Py_XDECREF(tb); + if (tb == NULL) { + tb = Py_None; + } + else { + Py_DECREF(tb); + } assert(PyLong_Check(lasti)); (void)lasti; // Shut up compiler warning if asserts are off PyObject *stack[4] = {NULL, exc, val, tb}; diff --git a/Python/ceval.c b/Python/ceval.c index a5d37c0629df9..c0b37b328a25f 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -193,7 +193,7 @@ static int monitor_stop_iteration(PyThreadState *tstate, static void monitor_unwind(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr); -static void monitor_handled(PyThreadState *tstate, +static int monitor_handled(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *exc); static void monitor_throw(PyThreadState *tstate, @@ -886,7 +886,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int PyObject *exc = _PyErr_GetRaisedException(tstate); PUSH(exc); JUMPTO(handler); - monitor_handled(tstate, frame, next_instr, exc); + if (monitor_handled(tstate, frame, next_instr, exc) < 0) { + goto exception_unwind; + } /* Resume normal execution */ DISPATCH(); } @@ -1924,6 +1926,7 @@ do_monitor_exc(PyThreadState *tstate, _PyInterpreterFrame *frame, PyErr_SetRaisedException(exc); } else { + assert(PyErr_Occurred()); Py_DECREF(exc); } return err; @@ -1988,15 +1991,15 @@ monitor_unwind(PyThreadState *tstate, } -static void +static int monitor_handled(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *exc) { if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_EXCEPTION_HANDLED)) { - return; + return 0; } - _Py_call_instrumentation_arg(tstate, PY_MONITORING_EVENT_EXCEPTION_HANDLED, frame, instr, exc); + return _Py_call_instrumentation_arg(tstate, PY_MONITORING_EVENT_EXCEPTION_HANDLED, frame, instr, exc); } static void diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 0f04b428ba805..f3e24bc0479ec 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1949,7 +1949,12 @@ assert(val && PyExceptionInstance_Check(val)); exc = PyExceptionInstance_Class(val); tb = PyException_GetTraceback(val); - Py_XDECREF(tb); + if (tb == NULL) { + tb = Py_None; + } + else { + Py_DECREF(tb); + } assert(PyLong_Check(lasti)); (void)lasti; // Shut up compiler warning if asserts are off PyObject *stack[4] = {NULL, exc, val, tb}; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 37ffb560f1457..6ac622b11acac 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3315,7 +3315,12 @@ assert(val && PyExceptionInstance_Check(val)); exc = PyExceptionInstance_Class(val); tb = PyException_GetTraceback(val); - Py_XDECREF(tb); + if (tb == NULL) { + tb = Py_None; + } + else { + Py_DECREF(tb); + } assert(PyLong_Check(lasti)); (void)lasti; // Shut up compiler warning if asserts are off PyObject *stack[4] = {NULL, exc, val, tb}; diff --git a/Python/instrumentation.c b/Python/instrumentation.c index 280e13d65526b..123c20dfe1a99 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -702,29 +702,13 @@ instrument_per_instruction(PyCodeObject *code, int i) *opcode_ptr = INSTRUMENTED_INSTRUCTION; } -#ifndef NDEBUG -static bool -instruction_has_event(PyCodeObject *code, int offset) -{ - _Py_CODEUNIT instr = _PyCode_CODE(code)[offset]; - int opcode = instr.op.code; - if (opcode == INSTRUMENTED_LINE) { - opcode = code->_co_monitoring->lines[offset].original_opcode; - } - if (opcode == INSTRUMENTED_INSTRUCTION) { - opcode = code->_co_monitoring->per_instruction_opcodes[offset]; - } - return opcode_has_event(opcode); -} -#endif - static void remove_tools(PyCodeObject * code, int offset, int event, int tools) { assert(event != PY_MONITORING_EVENT_LINE); assert(event != PY_MONITORING_EVENT_INSTRUCTION); - assert(event < PY_MONITORING_INSTRUMENTED_EVENTS); - assert(instruction_has_event(code, offset)); + assert(PY_MONITORING_IS_INSTRUMENTED_EVENT(event)); + assert(opcode_has_event(_Py_GetBaseOpcode(code, offset))); _PyCoMonitoringData *monitoring = code->_co_monitoring; if (monitoring && monitoring->tools) { monitoring->tools[offset] &= ~tools; @@ -779,7 +763,7 @@ add_tools(PyCodeObject * code, int offset, int event, int tools) { assert(event != PY_MONITORING_EVENT_LINE); assert(event != PY_MONITORING_EVENT_INSTRUCTION); - assert(event < PY_MONITORING_INSTRUMENTED_EVENTS); + assert(PY_MONITORING_IS_INSTRUMENTED_EVENT(event)); assert(code->_co_monitoring); if (code->_co_monitoring && code->_co_monitoring->tools @@ -919,7 +903,7 @@ get_tools_for_instruction(PyCodeObject *code, PyInterpreterState *interp, int i, event == PY_MONITORING_EVENT_C_RETURN); event = PY_MONITORING_EVENT_CALL; } - if (event < PY_MONITORING_INSTRUMENTED_EVENTS) { + if (PY_MONITORING_IS_INSTRUMENTED_EVENT(event)) { CHECK(is_version_up_to_date(code, interp)); CHECK(instrumentation_cross_checks(interp, code)); if (code->_co_monitoring->tools) { @@ -940,6 +924,26 @@ get_tools_for_instruction(PyCodeObject *code, PyInterpreterState *interp, int i, return tools; } +static const char *const event_names [] = { + [PY_MONITORING_EVENT_PY_START] = "PY_START", + [PY_MONITORING_EVENT_PY_RESUME] = "PY_RESUME", + [PY_MONITORING_EVENT_PY_RETURN] = "PY_RETURN", + [PY_MONITORING_EVENT_PY_YIELD] = "PY_YIELD", + [PY_MONITORING_EVENT_CALL] = "CALL", + [PY_MONITORING_EVENT_LINE] = "LINE", + [PY_MONITORING_EVENT_INSTRUCTION] = "INSTRUCTION", + [PY_MONITORING_EVENT_JUMP] = "JUMP", + [PY_MONITORING_EVENT_BRANCH] = "BRANCH", + [PY_MONITORING_EVENT_C_RETURN] = "C_RETURN", + [PY_MONITORING_EVENT_PY_THROW] = "PY_THROW", + [PY_MONITORING_EVENT_RAISE] = "RAISE", + [PY_MONITORING_EVENT_RERAISE] = "RERAISE", + [PY_MONITORING_EVENT_EXCEPTION_HANDLED] = "EXCEPTION_HANDLED", + [PY_MONITORING_EVENT_C_RAISE] = "C_RAISE", + [PY_MONITORING_EVENT_PY_UNWIND] = "PY_UNWIND", + [PY_MONITORING_EVENT_STOP_ITERATION] = "STOP_ITERATION", +}; + static int call_instrumentation_vector( PyThreadState *tstate, int event, @@ -984,7 +988,18 @@ call_instrumentation_vector( } else { /* DISABLE */ - remove_tools(code, offset, event, 1 << tool); + if (!PY_MONITORING_IS_INSTRUMENTED_EVENT(event)) { + PyErr_Format(PyExc_ValueError, + "Cannot disable %s events. Callback removed.", + event_names[event]); + /* Clear tool to prevent infinite loop */ + Py_CLEAR(interp->monitoring_callables[tool][event]); + err = -1; + break; + } + else { + remove_tools(code, offset, event, 1 << tool); + } } } Py_DECREF(offset_obj); @@ -1262,7 +1277,7 @@ initialize_tools(PyCodeObject *code) assert(event > 0); } assert(event >= 0); - assert(event < PY_MONITORING_INSTRUMENTED_EVENTS); + assert(PY_MONITORING_IS_INSTRUMENTED_EVENT(event)); tools[i] = code->_co_monitoring->active_monitors.tools[event]; CHECK(tools[i] != 0); } @@ -2017,26 +2032,6 @@ add_power2_constant(PyObject *obj, const char *name, int i) return err; } -static const char *const event_names [] = { - [PY_MONITORING_EVENT_PY_START] = "PY_START", - [PY_MONITORING_EVENT_PY_RESUME] = "PY_RESUME", - [PY_MONITORING_EVENT_PY_RETURN] = "PY_RETURN", - [PY_MONITORING_EVENT_PY_YIELD] = "PY_YIELD", - [PY_MONITORING_EVENT_CALL] = "CALL", - [PY_MONITORING_EVENT_LINE] = "LINE", - [PY_MONITORING_EVENT_INSTRUCTION] = "INSTRUCTION", - [PY_MONITORING_EVENT_JUMP] = "JUMP", - [PY_MONITORING_EVENT_BRANCH] = "BRANCH", - [PY_MONITORING_EVENT_C_RETURN] = "C_RETURN", - [PY_MONITORING_EVENT_PY_THROW] = "PY_THROW", - [PY_MONITORING_EVENT_RAISE] = "RAISE", - [PY_MONITORING_EVENT_RERAISE] = "RERAISE", - [PY_MONITORING_EVENT_EXCEPTION_HANDLED] = "EXCEPTION_HANDLED", - [PY_MONITORING_EVENT_C_RAISE] = "C_RAISE", - [PY_MONITORING_EVENT_PY_UNWIND] = "PY_UNWIND", - [PY_MONITORING_EVENT_STOP_ITERATION] = "STOP_ITERATION", -}; - /*[clinic input] monitoring._all_events [clinic start generated code]*/ From webhook-mailer at python.org Thu Jul 27 10:36:58 2023 From: webhook-mailer at python.org (vstinner) Date: Thu, 27 Jul 2023 14:36:58 -0000 Subject: [Python-checkins] gh-107196: Remove _PyArg_VaParseTupleAndKeywordsFast() function (#107197) Message-ID: https://github.com/python/cpython/commit/1dbb427dd67a1de5b3662cbda0277ff2c3b18095 commit: 1dbb427dd67a1de5b3662cbda0277ff2c3b18095 branch: main author: Victor Stinner committer: vstinner date: 2023-07-27T16:36:54+02:00 summary: gh-107196: Remove _PyArg_VaParseTupleAndKeywordsFast() function (#107197) Remove the private _PyArg_VaParseTupleAndKeywordsFast() function: it is no longer used. files: M Include/cpython/modsupport.h M Python/getargs.c diff --git a/Include/cpython/modsupport.h b/Include/cpython/modsupport.h index 376336b13dcf8..cfc2c2cdb5a7f 100644 --- a/Include/cpython/modsupport.h +++ b/Include/cpython/modsupport.h @@ -52,8 +52,6 @@ PyAPI_FUNC(int) _PyArg_ParseStackAndKeywords( PyObject *kwnames, struct _PyArg_Parser *, ...); -PyAPI_FUNC(int) _PyArg_VaParseTupleAndKeywordsFast(PyObject *, PyObject *, - struct _PyArg_Parser *, va_list); PyAPI_FUNC(PyObject * const *) _PyArg_UnpackKeywords( PyObject *const *args, Py_ssize_t nargs, PyObject *kwargs, PyObject *kwnames, diff --git a/Python/getargs.c b/Python/getargs.c index efcf2e333af8d..916e46578a454 100644 --- a/Python/getargs.c +++ b/Python/getargs.c @@ -1418,20 +1418,6 @@ _PyArg_ParseStackAndKeywords_SizeT(PyObject *const *args, Py_ssize_t nargs, PyOb } -PyAPI_FUNC(int) -_PyArg_VaParseTupleAndKeywordsFast(PyObject *args, PyObject *keywords, - struct _PyArg_Parser *parser, va_list va) -{ - int retval; - va_list lva; - - va_copy(lva, va); - - retval = vgetargskeywordsfast(args, keywords, parser, &lva, 0); - va_end(lva); - return retval; -} - static void error_unexpected_keyword_arg(PyObject *kwargs, PyObject *kwnames, PyObject *kwtuple, const char *fname) { From webhook-mailer at python.org Thu Jul 27 10:43:12 2023 From: webhook-mailer at python.org (vstinner) Date: Thu, 27 Jul 2023 14:43:12 -0000 Subject: [Python-checkins] gh-105268: _PyGC_FINALIZED() removal is already documented in 3.12 (#107350) Message-ID: https://github.com/python/cpython/commit/9a7204b86bdb3e26c2a62aeaafb875275500b9f7 commit: 9a7204b86bdb3e26c2a62aeaafb875275500b9f7 branch: main author: Victor Stinner committer: vstinner date: 2023-07-27T14:43:09Z summary: gh-105268: _PyGC_FINALIZED() removal is already documented in 3.12 (#107350) files: M Doc/whatsnew/3.13.rst diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 8ca0abf552576..7cad5d1ec02aa 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -940,13 +940,6 @@ Removed (Contributed by Victor Stinner in :gh:`105182`.) -* Remove the old private, undocumented and untested ``_PyGC_FINALIZED()`` macro - which was kept for backward compatibility with Python 3.8 and older: use - :c:func:`PyObject_GC_IsFinalized()` instead. The `pythoncapi-compat project - `__ can be used to get this - function on Python 3.8 and older. - (Contributed by Victor Stinner in :gh:`105268`.) - * Remove the old aliases to functions calling functions which were kept for backward compatibility with Python 3.8 provisional API: From webhook-mailer at python.org Thu Jul 27 10:47:37 2023 From: webhook-mailer at python.org (markshannon) Date: Thu, 27 Jul 2023 14:47:37 -0000 Subject: [Python-checkins] GH-106898: Add the exception as an argument to the `PY_UNWIND` event callback function. (GH-107347) Message-ID: https://github.com/python/cpython/commit/ac7a0f858a8d0c6ca2e64bb880fca40e229d267a commit: ac7a0f858a8d0c6ca2e64bb880fca40e229d267a branch: main author: Mark Shannon committer: markshannon date: 2023-07-27T15:47:33+01:00 summary: GH-106898: Add the exception as an argument to the `PY_UNWIND` event callback function. (GH-107347) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-26-21-28-06.gh-issue-106898.8Wjuiv.rst M Lib/test/test_monitoring.py M Python/ceval.c M Python/legacy_tracing.c diff --git a/Lib/test/test_monitoring.py b/Lib/test/test_monitoring.py index 7098f483e0ee7..b36382c8d0982 100644 --- a/Lib/test/test_monitoring.py +++ b/Lib/test/test_monitoring.py @@ -715,7 +715,7 @@ def check_balanced(self, func, recorders): self.assertIn(r0, ("raise", "reraise")) h0 = h[0] self.assertIn(h0, ("handled", "unwind")) - + self.assertEqual(r[1], h[1]) class StopiterationRecorder(ExceptionRecorder): @@ -733,8 +733,8 @@ class UnwindRecorder(ExceptionRecorder): event_type = E.PY_UNWIND - def __call__(self, *args): - self.events.append(("unwind", None)) + def __call__(self, code, offset, exc): + self.events.append(("unwind", type(exc))) class ExceptionHandledRecorder(ExceptionRecorder): diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-26-21-28-06.gh-issue-106898.8Wjuiv.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-21-28-06.gh-issue-106898.8Wjuiv.rst new file mode 100644 index 0000000000000..f1b1c4c64b4ac --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-21-28-06.gh-issue-106898.8Wjuiv.rst @@ -0,0 +1,3 @@ +Add the exception as the third argument to ``PY_UNIND`` callbacks in +``sys.monitoring``. This makes the ``PY_UNWIND`` callback consistent with +the other exception hanlding callbacks. diff --git a/Python/ceval.c b/Python/ceval.c index c0b37b328a25f..17818a0d3c17e 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1987,7 +1987,7 @@ monitor_unwind(PyThreadState *tstate, if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_PY_UNWIND)) { return; } - _Py_call_instrumentation_exc0(tstate, PY_MONITORING_EVENT_PY_UNWIND, frame, instr); + do_monitor_exc(tstate, frame, instr, PY_MONITORING_EVENT_PY_UNWIND); } diff --git a/Python/legacy_tracing.c b/Python/legacy_tracing.c index 9cc48fc9493a0..48db51731cfdb 100644 --- a/Python/legacy_tracing.c +++ b/Python/legacy_tracing.c @@ -64,6 +64,16 @@ sys_profile_func3( return call_profile_func(self, args[2]); } +static PyObject * +sys_profile_unwind( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 3); + return call_profile_func(self, Py_None); +} + static PyObject * sys_profile_call_or_return( _PyLegacyEventHandler *self, PyObject *const *args, @@ -152,6 +162,16 @@ sys_trace_func2( return call_trace_func(self, Py_None); } +static PyObject * +sys_trace_unwind( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 3); + return call_trace_func(self, Py_None); +} + static PyObject * sys_trace_return( _PyLegacyEventHandler *self, PyObject *const *args, @@ -362,7 +382,7 @@ _PyEval_SetProfile(PyThreadState *tstate, Py_tracefunc func, PyObject *arg) return -1; } if (set_callbacks(PY_MONITORING_SYS_PROFILE_ID, - (vectorcallfunc)sys_profile_func2, PyTrace_RETURN, + (vectorcallfunc)sys_profile_unwind, PyTrace_RETURN, PY_MONITORING_EVENT_PY_UNWIND, -1)) { return -1; } @@ -450,7 +470,7 @@ _PyEval_SetTrace(PyThreadState *tstate, Py_tracefunc func, PyObject *arg) return -1; } if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, - (vectorcallfunc)sys_trace_func2, PyTrace_RETURN, + (vectorcallfunc)sys_trace_unwind, PyTrace_RETURN, PY_MONITORING_EVENT_PY_UNWIND, -1)) { return -1; } From webhook-mailer at python.org Thu Jul 27 11:44:35 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 27 Jul 2023 15:44:35 -0000 Subject: [Python-checkins] gh-107298: Fix yet more Sphinx warnings in the C API doc (GH-107345) Message-ID: https://github.com/python/cpython/commit/983305268e2291b0a7835621b81bf40cba7c27f3 commit: 983305268e2291b0a7835621b81bf40cba7c27f3 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-27T18:44:32+03:00 summary: gh-107298: Fix yet more Sphinx warnings in the C API doc (GH-107345) files: M Doc/c-api/capsule.rst M Doc/c-api/init_config.rst M Doc/c-api/intro.rst M Doc/c-api/memory.rst M Doc/c-api/module.rst M Doc/c-api/none.rst M Doc/c-api/object.rst M Doc/c-api/set.rst M Doc/c-api/sys.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Doc/extending/embedding.rst M Doc/extending/extending.rst M Doc/extending/newtypes.rst M Doc/extending/newtypes_tutorial.rst M Doc/howto/descriptor.rst M Doc/tools/.nitignore M Doc/whatsnew/2.5.rst M Doc/whatsnew/3.8.rst diff --git a/Doc/c-api/capsule.rst b/Doc/c-api/capsule.rst index 2a1b602dc79c0..cdb8aa33e9fd3 100644 --- a/Doc/c-api/capsule.rst +++ b/Doc/c-api/capsule.rst @@ -121,7 +121,7 @@ Refer to :ref:`using-capsules` for more information on using these objects. compared.) In other words, if :c:func:`PyCapsule_IsValid` returns a true value, calls to - any of the accessors (any function starting with :c:func:`PyCapsule_Get`) are + any of the accessors (any function starting with ``PyCapsule_Get``) are guaranteed to succeed. Return a nonzero value if the object is valid and matches the name passed in. diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index e3d74cc2b2d6b..ac7b357e08eeb 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -135,6 +135,8 @@ PyStatus Name of the function which created an error, can be ``NULL``. + .. c:namespace:: NULL + Functions to create a status: .. c:function:: PyStatus PyStatus_Ok(void) @@ -210,6 +212,8 @@ PyPreConfig Structure used to preinitialize Python. + .. c:namespace:: NULL + Function to initialize a preconfiguration: .. c:function:: void PyPreConfig_InitPythonConfig(PyPreConfig *preconfig) @@ -222,6 +226,8 @@ PyPreConfig Initialize the preconfiguration with :ref:`Isolated Configuration `. + .. c:namespace:: PyPreConfig + Structure fields: .. c:member:: int allocator @@ -429,6 +435,8 @@ PyConfig When done, the :c:func:`PyConfig_Clear` function must be used to release the configuration memory. + .. c:namespace:: NULL + Structure methods: .. c:function:: void PyConfig_InitPythonConfig(PyConfig *config) @@ -527,6 +535,8 @@ PyConfig The caller of these methods is responsible to handle exceptions (error or exit) using ``PyStatus_Exception()`` and ``Py_ExitStatusException()``. + .. c:namespace:: PyConfig + Structure fields: .. c:member:: PyWideStringList argv @@ -938,7 +948,7 @@ PyConfig .. c:member:: wchar_t* pythonpath_env Module search paths (:data:`sys.path`) as a string separated by ``DELIM`` - (:data:`os.path.pathsep`). + (:data:`os.pathsep`). Set by the :envvar:`PYTHONPATH` environment variable. diff --git a/Doc/c-api/intro.rst b/Doc/c-api/intro.rst index 9014f7e03b360..1739eabd2a303 100644 --- a/Doc/c-api/intro.rst +++ b/Doc/c-api/intro.rst @@ -616,7 +616,7 @@ and lose important information about the exact cause of the error. .. index:: single: sum_sequence() A simple example of detecting exceptions and passing them on is shown in the -:c:func:`sum_sequence` example above. It so happens that this example doesn't +:c:func:`!sum_sequence` example above. It so happens that this example doesn't need to clean up any owned references when it detects an error. The following example function shows some error cleanup. First, to remind you why you like Python, we show the equivalent Python code:: diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index c51aba3f55536..8968b26b64320 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -431,6 +431,8 @@ Customize Memory Allocators Enum used to identify an allocator domain. Domains: + .. c:namespace:: NULL + .. c:macro:: PYMEM_DOMAIN_RAW Functions: diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index ad51e7b9b3230..fa5e97846e7bc 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -119,7 +119,7 @@ Module Objects encoded to 'utf-8'. .. deprecated:: 3.2 - :c:func:`PyModule_GetFilename` raises :c:type:`UnicodeEncodeError` on + :c:func:`PyModule_GetFilename` raises :exc:`UnicodeEncodeError` on unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead. diff --git a/Doc/c-api/none.rst b/Doc/c-api/none.rst index 1a497652ac565..dd8bfb5610425 100644 --- a/Doc/c-api/none.rst +++ b/Doc/c-api/none.rst @@ -9,7 +9,7 @@ The ``None`` Object Note that the :c:type:`PyTypeObject` for ``None`` is not directly exposed in the Python/C API. Since ``None`` is a singleton, testing for object identity (using -``==`` in C) is sufficient. There is no :c:func:`PyNone_Check` function for the +``==`` in C) is sufficient. There is no :c:func:`!PyNone_Check` function for the same reason. diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 284f75470d524..1b7e05e7c53ed 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -293,7 +293,7 @@ Object Protocol Normally only class objects, i.e. instances of :class:`type` or a derived class, are considered classes. However, objects can override this by having - a :attr:`__bases__` attribute (which must be a tuple of base classes). + a :attr:`~class.__bases__` attribute (which must be a tuple of base classes). .. c:function:: int PyObject_IsInstance(PyObject *inst, PyObject *cls) @@ -310,10 +310,10 @@ Object Protocol is an instance of *cls* if its class is a subclass of *cls*. An instance *inst* can override what is considered its class by having a - :attr:`__class__` attribute. + :attr:`~instance.__class__` attribute. An object *cls* can override if it is considered a class, and what its base - classes are, by having a :attr:`__bases__` attribute (which must be a tuple + classes are, by having a :attr:`~class.__bases__` attribute (which must be a tuple of base classes). diff --git a/Doc/c-api/set.rst b/Doc/c-api/set.rst index 7e0ebd2f791a4..1e8a09509032f 100644 --- a/Doc/c-api/set.rst +++ b/Doc/c-api/set.rst @@ -110,7 +110,7 @@ or :class:`frozenset` or instances of their subtypes. .. index:: pair: built-in function; len Return the length of a :class:`set` or :class:`frozenset` object. Equivalent to - ``len(anyset)``. Raises a :exc:`PyExc_SystemError` if *anyset* is not a + ``len(anyset)``. Raises a :exc:`SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. @@ -124,7 +124,7 @@ or :class:`frozenset` or instances of their subtypes. Return ``1`` if found, ``0`` if not found, and ``-1`` if an error is encountered. Unlike the Python :meth:`~object.__contains__` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise a :exc:`TypeError` if - the *key* is unhashable. Raise :exc:`PyExc_SystemError` if *anyset* is not a + the *key* is unhashable. Raise :exc:`SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. @@ -149,7 +149,7 @@ subtypes but not for instances of :class:`frozenset` or its subtypes. error is encountered. Does not raise :exc:`KeyError` for missing keys. Raise a :exc:`TypeError` if the *key* is unhashable. Unlike the Python :meth:`~set.discard` method, this function does not automatically convert unhashable sets into - temporary frozensets. Raise :exc:`PyExc_SystemError` if *set* is not an + temporary frozensets. Raise :exc:`SystemError` if *set* is not an instance of :class:`set` or its subtype. diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 995fca4503ccd..5457d17159f8d 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -167,7 +167,7 @@ Operating System Utilities .. versionchanged:: 3.8 The function now uses the UTF-8 encoding on Windows if - :c:member:`PyConfig.legacy_windows_fs_encoding` is zero; + :c:member:`PyPreConfig.legacy_windows_fs_encoding` is zero; .. c:function:: char* Py_EncodeLocale(const wchar_t *text, size_t *error_pos) @@ -209,7 +209,7 @@ Operating System Utilities .. versionchanged:: 3.8 The function now uses the UTF-8 encoding on Windows if - :c:member:`PyConfig.legacy_windows_fs_encoding` is zero. + :c:member:`PyPreConfig.legacy_windows_fs_encoding` is zero. .. _systemfunctions: diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index e7b35c43da32a..0f58326f6c06b 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -103,7 +103,7 @@ Type Objects :c:func:`PyType_AddWatcher` will be called whenever :c:func:`PyType_Modified` reports a change to *type*. (The callback may be called only once for a series of consecutive modifications to *type*, if - :c:func:`PyType_Lookup` is not called on *type* between the modifications; + :c:func:`!_PyType_Lookup` is not called on *type* between the modifications; this is an implementation detail and subject to change.) An extension should never call ``PyType_Watch`` with a *watcher_id* that was diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 99b025dc41fc2..26e6133aebaa8 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -485,7 +485,7 @@ PyObject Slots -------------- The type object structure extends the :c:type:`PyVarObject` structure. The -:c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :func:`type_new`, +:c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :c:func:`!type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. type objects) *must* have the :c:member:`~PyVarObject.ob_size` field. @@ -740,7 +740,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) Before version 3.12, it was not recommended for :ref:`mutable heap types ` to implement the vectorcall protocol. - When a user sets :attr:`~type.__call__` in Python code, only *tp_call* is + When a user sets :attr:`~object.__call__` in Python code, only *tp_call* is updated, likely making it inconsistent with the vectorcall function. Since 3.12, setting ``__call__`` will disable vectorcall optimization by clearing the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag. @@ -1369,8 +1369,8 @@ and :c:data:`PyType_Type` effectively act as defaults.) The :c:member:`~PyTypeObject.tp_traverse` pointer is used by the garbage collector to detect reference cycles. A typical implementation of a :c:member:`~PyTypeObject.tp_traverse` function simply calls :c:func:`Py_VISIT` on each of the instance's members that are Python - objects that the instance owns. For example, this is function :c:func:`local_traverse` from the - :mod:`_thread` extension module:: + objects that the instance owns. For example, this is function :c:func:`!local_traverse` from the + :mod:`!_thread` extension module:: static int local_traverse(localobject *self, visitproc visit, void *arg) @@ -1721,7 +1721,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) called; it may also be initialized to a dictionary containing initial attributes for the type. Once :c:func:`PyType_Ready` has initialized the type, extra attributes for the type may be added to this dictionary only if they don't - correspond to overloaded operations (like :meth:`__add__`). Once + correspond to overloaded operations (like :meth:`~object.__add__`). Once initialization for the type has finished, this field should be treated as read-only. @@ -1818,7 +1818,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Default:** This slot has no default. For :ref:`static types `, if the - field is ``NULL`` then no :attr:`__dict__` gets created for instances. + field is ``NULL`` then no :attr:`~object.__dict__` gets created for instances. If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the :c:member:`~PyTypeObject.tp_dict` field, then @@ -1830,10 +1830,10 @@ and :c:data:`PyType_Type` effectively act as defaults.) An optional pointer to an instance initialization function. - This function corresponds to the :meth:`__init__` method of classes. Like - :meth:`__init__`, it is possible to create an instance without calling - :meth:`__init__`, and it is possible to reinitialize an instance by calling its - :meth:`__init__` method again. + This function corresponds to the :meth:`~object.__init__` method of classes. Like + :meth:`!__init__`, it is possible to create an instance without calling + :meth:`!__init__`, and it is possible to reinitialize an instance by calling its + :meth:`!__init__` method again. The function signature is:: @@ -1841,7 +1841,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) The self argument is the instance to be initialized; the *args* and *kwds* arguments represent positional and keyword arguments of the call to - :meth:`__init__`. + :meth:`~object.__init__`. The :c:member:`~PyTypeObject.tp_init` function, if not ``NULL``, is called when an instance is created normally by calling its type, after the type's :c:member:`~PyTypeObject.tp_new` function @@ -2130,7 +2130,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) In other words, it is used to implement :ref:`vectorcall ` for ``type.__call__``. If ``tp_vectorcall`` is ``NULL``, the default call implementation - using :attr:`__new__` and :attr:`__init__` is used. + using :meth:`~object.__new__` and :meth:`~object.__init__` is used. **Inheritance:** @@ -2329,8 +2329,8 @@ Mapping Object Structures .. c:member:: objobjargproc PyMappingMethods.mp_ass_subscript This function is used by :c:func:`PyObject_SetItem`, - :c:func:`PyObject_DelItem`, :c:func:`PyObject_SetSlice` and - :c:func:`PyObject_DelSlice`. It has the same signature as + :c:func:`PyObject_DelItem`, :c:func:`PySequence_SetSlice` and + :c:func:`PySequence_DelSlice`. It has the same signature as :c:func:`!PyObject_SetItem`, but *v* can also be set to ``NULL`` to delete an item. If this slot is ``NULL``, the object does not support item assignment and deletion. @@ -2552,7 +2552,7 @@ Async Object Structures PyObject *am_aiter(PyObject *self); Must return an :term:`asynchronous iterator` object. - See :meth:`__anext__` for details. + See :meth:`~object.__anext__` for details. This slot may be set to ``NULL`` if an object does not implement asynchronous iteration protocol. @@ -2563,7 +2563,8 @@ Async Object Structures PyObject *am_anext(PyObject *self); - Must return an :term:`awaitable` object. See :meth:`__anext__` for details. + Must return an :term:`awaitable` object. + See :meth:`~object.__anext__` for details. This slot may be set to ``NULL``. .. c:member:: sendfunc PyAsyncMethods.am_send diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst index bd1abe36cbb80..20397dc5add5d 100644 --- a/Doc/extending/embedding.rst +++ b/Doc/extending/embedding.rst @@ -269,7 +269,7 @@ following two statements before the call to :c:func:`Py_Initialize`:: PyImport_AppendInittab("emb", &PyInit_emb); These two lines initialize the ``numargs`` variable, and make the -:func:`emb.numargs` function accessible to the embedded Python interpreter. +:func:`!emb.numargs` function accessible to the embedded Python interpreter. With these extensions, the Python script can do things like .. code-block:: python diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index b9cfc9b4474d9..f58b4f28113e8 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -197,7 +197,7 @@ The choice of which exception to raise is entirely yours. There are predeclared C objects corresponding to all built-in Python exceptions, such as :c:data:`PyExc_ZeroDivisionError`, which you can use directly. Of course, you should choose exceptions wisely --- don't use :c:data:`PyExc_TypeError` to mean -that a file couldn't be opened (that should probably be :c:data:`PyExc_IOError`). +that a file couldn't be opened (that should probably be :c:data:`PyExc_OSError`). If something's wrong with the argument list, the :c:func:`PyArg_ParseTuple` function usually raises :c:data:`PyExc_TypeError`. If you have an argument whose value must be in a particular range or must satisfy other conditions, @@ -208,7 +208,7 @@ usually declare a static object variable at the beginning of your file:: static PyObject *SpamError; -and initialize it in your module's initialization function (:c:func:`PyInit_spam`) +and initialize it in your module's initialization function (:c:func:`!PyInit_spam`) with an exception object:: PyMODINIT_FUNC @@ -354,7 +354,7 @@ The method table must be referenced in the module definition structure:: This structure, in turn, must be passed to the interpreter in the module's initialization function. The initialization function must be named -:c:func:`PyInit_name`, where *name* is the name of the module, and should be the +:c:func:`!PyInit_name`, where *name* is the name of the module, and should be the only non-\ ``static`` item defined in the module file:: PyMODINIT_FUNC @@ -368,7 +368,7 @@ declares any special linkage declarations required by the platform, and for C++ declares the function as ``extern "C"``. When the Python program imports module :mod:`!spam` for the first time, -:c:func:`PyInit_spam` is called. (See below for comments about embedding Python.) +:c:func:`!PyInit_spam` is called. (See below for comments about embedding Python.) It calls :c:func:`PyModule_Create`, which returns a module object, and inserts built-in function objects into the newly created module based upon the table (an array of :c:type:`PyMethodDef` structures) found in the module definition. @@ -378,7 +378,7 @@ certain errors, or return ``NULL`` if the module could not be initialized satisfactorily. The init function must return the module object to its caller, so that it then gets inserted into ``sys.modules``. -When embedding Python, the :c:func:`PyInit_spam` function is not called +When embedding Python, the :c:func:`!PyInit_spam` function is not called automatically unless there's an entry in the :c:data:`PyImport_Inittab` table. To add the module to the initialization table, use :c:func:`PyImport_AppendInittab`, optionally followed by an import of the module:: @@ -1220,13 +1220,13 @@ the module and retrieving its C API pointers; client modules only have to call this macro before accessing the C API. The exporting module is a modification of the :mod:`!spam` module from section -:ref:`extending-simpleexample`. The function :func:`spam.system` does not call +:ref:`extending-simpleexample`. The function :func:`!spam.system` does not call the C library function :c:func:`system` directly, but a function -:c:func:`PySpam_System`, which would of course do something more complicated in +:c:func:`!PySpam_System`, which would of course do something more complicated in reality (such as adding "spam" to every command). This function -:c:func:`PySpam_System` is also exported to other extension modules. +:c:func:`!PySpam_System` is also exported to other extension modules. -The function :c:func:`PySpam_System` is a plain C function, declared +The function :c:func:`!PySpam_System` is a plain C function, declared ``static`` like everything else:: static int @@ -1288,7 +1288,7 @@ function must take care of initializing the C API pointer array:: } Note that ``PySpam_API`` is declared ``static``; otherwise the pointer -array would disappear when :func:`PyInit_spam` terminates! +array would disappear when :c:func:`!PyInit_spam` terminates! The bulk of the work is in the header file :file:`spammodule.h`, which looks like this:: @@ -1342,8 +1342,8 @@ like this:: #endif /* !defined(Py_SPAMMODULE_H) */ All that a client module must do in order to have access to the function -:c:func:`PySpam_System` is to call the function (or rather macro) -:c:func:`import_spam` in its initialization function:: +:c:func:`!PySpam_System` is to call the function (or rather macro) +:c:func:`!import_spam` in its initialization function:: PyMODINIT_FUNC PyInit_client(void) diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst index 25822744125f8..386b3c8f4452c 100644 --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -286,9 +286,9 @@ be read-only or read-write. The structures in the table are defined as:: For each entry in the table, a :term:`descriptor` will be constructed and added to the type which will be able to extract a value from the instance structure. The -:attr:`type` field should contain a type code like :c:macro:`Py_T_INT` or +:c:member:`~PyMemberDef.type` field should contain a type code like :c:macro:`Py_T_INT` or :c:macro:`Py_T_DOUBLE`; the value will be used to determine how to -convert Python values to and from C values. The :attr:`flags` field is used to +convert Python values to and from C values. The :c:member:`~PyMemberDef.flags` field is used to store flags which control how the attribute can be accessed: you can set it to :c:macro:`Py_READONLY` to prevent Python code from setting it. @@ -298,7 +298,7 @@ have an associated doc string simply by providing the text in the table. An application can use the introspection API to retrieve the descriptor from the class object, and get the doc string using its :attr:`__doc__` attribute. -As with the :c:member:`~PyTypeObject.tp_methods` table, a sentinel entry with a :attr:`name` value +As with the :c:member:`~PyTypeObject.tp_methods` table, a sentinel entry with a :c:member:`~PyMethodDef.name` value of ``NULL`` is required. .. XXX Descriptors need to be explained in more detail somewhere, but not here. @@ -323,7 +323,7 @@ called, so that if you do need to extend their functionality, you'll understand what needs to be done. The :c:member:`~PyTypeObject.tp_getattr` handler is called when the object requires an attribute -look-up. It is called in the same situations where the :meth:`__getattr__` +look-up. It is called in the same situations where the :meth:`~object.__getattr__` method of a class would be called. Here is an example:: @@ -342,8 +342,8 @@ Here is an example:: return NULL; } -The :c:member:`~PyTypeObject.tp_setattr` handler is called when the :meth:`__setattr__` or -:meth:`__delattr__` method of a class instance would be called. When an +The :c:member:`~PyTypeObject.tp_setattr` handler is called when the :meth:`~object.__setattr__` or +:meth:`~object.__delattr__` method of a class instance would be called. When an attribute should be deleted, the third parameter will be ``NULL``. Here is an example that simply raises an exception; if this were really all you wanted, the :c:member:`~PyTypeObject.tp_setattr` handler should be set to ``NULL``. :: @@ -364,7 +364,7 @@ Object Comparison The :c:member:`~PyTypeObject.tp_richcompare` handler is called when comparisons are needed. It is analogous to the :ref:`rich comparison methods `, like -:meth:`__lt__`, and also called by :c:func:`PyObject_RichCompare` and +:meth:`!__lt__`, and also called by :c:func:`PyObject_RichCompare` and :c:func:`PyObject_RichCompareBool`. This function is called with two Python objects and the operator as arguments, @@ -505,7 +505,7 @@ These functions provide support for the iterator protocol. Both handlers take exactly one parameter, the instance for which they are being called, and return a new reference. In the case of an error, they should set an exception and return ``NULL``. :c:member:`~PyTypeObject.tp_iter` corresponds -to the Python :meth:`__iter__` method, while :c:member:`~PyTypeObject.tp_iternext` +to the Python :meth:`~object.__iter__` method, while :c:member:`~PyTypeObject.tp_iternext` corresponds to the Python :meth:`~iterator.__next__` method. Any :term:`iterable` object must implement the :c:member:`~PyTypeObject.tp_iter` diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index 0cc366b9c0b51..209a4ab76d226 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -145,7 +145,7 @@ only used for variable-sized objects and should otherwise be zero. :c:member:`~PyTypeObject.tp_basicsize` as its base type, you may have problems with multiple inheritance. A Python subclass of your type will have to list your type first in its :attr:`~class.__bases__`, or else it will not be able to call your type's - :meth:`__new__` method without getting an error. You can avoid this problem by + :meth:`~object.__new__` method without getting an error. You can avoid this problem by ensuring that your type has a larger value for :c:member:`~PyTypeObject.tp_basicsize` than its base type does. Most of the time, this will be true anyway, because either your base type will be :class:`object`, or else you will be adding data members to @@ -164,14 +164,14 @@ We provide a doc string for the type in :c:member:`~PyTypeObject.tp_doc`. :: .tp_doc = PyDoc_STR("Custom objects"), To enable object creation, we have to provide a :c:member:`~PyTypeObject.tp_new` -handler. This is the equivalent of the Python method :meth:`__new__`, but +handler. This is the equivalent of the Python method :meth:`~object.__new__`, but has to be specified explicitly. In this case, we can just use the default implementation provided by the API function :c:func:`PyType_GenericNew`. :: .tp_new = PyType_GenericNew, Everything else in the file should be familiar, except for some code in -:c:func:`PyInit_custom`:: +:c:func:`!PyInit_custom`:: if (PyType_Ready(&CustomType) < 0) return; @@ -218,7 +218,7 @@ Of course, the current Custom type is pretty uninteresting. It has no data and doesn't do anything. It can't even be subclassed. .. note:: - While this documentation showcases the standard :mod:`distutils` module + While this documentation showcases the standard :mod:`!distutils` module for building C extensions, it is recommended in real-world use cases to use the newer and better-maintained ``setuptools`` library. Documentation on how to do this is out of scope for this document and can be found in @@ -270,7 +270,7 @@ This method first clears the reference counts of the two Python attributes. ``NULL`` (which might happen here if ``tp_new`` failed midway). It then calls the :c:member:`~PyTypeObject.tp_free` member of the object's type (computed by ``Py_TYPE(self)``) to free the object's memory. Note that -the object's type might not be :class:`CustomType`, because the object may +the object's type might not be :class:`!CustomType`, because the object may be an instance of a subclass. .. note:: @@ -309,7 +309,7 @@ and install it in the :c:member:`~PyTypeObject.tp_new` member:: .tp_new = Custom_new, The ``tp_new`` handler is responsible for creating (as opposed to initializing) -objects of the type. It is exposed in Python as the :meth:`__new__` method. +objects of the type. It is exposed in Python as the :meth:`~object.__new__` method. It is not required to define a ``tp_new`` member, and indeed many extension types will simply reuse :c:func:`PyType_GenericNew` as done in the first version of the :class:`!Custom` type above. In this case, we use the ``tp_new`` @@ -343,7 +343,7 @@ result against ``NULL`` before proceeding. .. note:: If you are creating a co-operative :c:member:`~PyTypeObject.tp_new` (one - that calls a base type's :c:member:`~PyTypeObject.tp_new` or :meth:`__new__`), + that calls a base type's :c:member:`~PyTypeObject.tp_new` or :meth:`~object.__new__`), you must *not* try to determine what method to call using method resolution order at runtime. Always statically determine what type you are going to call, and call its :c:member:`~PyTypeObject.tp_new` directly, or via @@ -386,14 +386,14 @@ by filling the :c:member:`~PyTypeObject.tp_init` slot. :: .tp_init = (initproc) Custom_init, The :c:member:`~PyTypeObject.tp_init` slot is exposed in Python as the -:meth:`__init__` method. It is used to initialize an object after it's +:meth:`~object.__init__` method. It is used to initialize an object after it's created. Initializers always accept positional and keyword arguments, and they should return either ``0`` on success or ``-1`` on error. Unlike the ``tp_new`` handler, there is no guarantee that ``tp_init`` is called at all (for example, the :mod:`pickle` module by default -doesn't call :meth:`__init__` on unpickled instances). It can also be -called multiple times. Anyone can call the :meth:`__init__` method on +doesn't call :meth:`~object.__init__` on unpickled instances). It can also be +called multiple times. Anyone can call the :meth:`!__init__` method on our objects. For this reason, we have to be extra careful when assigning the new attribute values. We might be tempted, for example to assign the ``first`` member like this:: @@ -706,8 +706,8 @@ participate in cycles:: } For each subobject that can participate in cycles, we need to call the -:c:func:`visit` function, which is passed to the traversal method. The -:c:func:`visit` function takes as arguments the subobject and the extra argument +:c:func:`!visit` function, which is passed to the traversal method. The +:c:func:`!visit` function takes as arguments the subobject and the extra argument *arg* passed to the traversal method. It returns an integer value that must be returned if it is non-zero. @@ -789,9 +789,9 @@ types. It is easiest to inherit from the built in types, since an extension can easily use the :c:type:`PyTypeObject` it needs. It can be difficult to share these :c:type:`PyTypeObject` structures between extension modules. -In this example we will create a :class:`SubList` type that inherits from the +In this example we will create a :class:`!SubList` type that inherits from the built-in :class:`list` type. The new type will be completely compatible with -regular lists, but will have an additional :meth:`increment` method that +regular lists, but will have an additional :meth:`!increment` method that increases an internal counter: .. code-block:: pycon @@ -821,7 +821,7 @@ The primary difference for derived type objects is that the base type's object structure must be the first value. The base type will already include the :c:func:`PyObject_HEAD` at the beginning of its structure. -When a Python object is a :class:`SubList` instance, its ``PyObject *`` pointer +When a Python object is a :class:`!SubList` instance, its ``PyObject *`` pointer can be safely cast to both ``PyListObject *`` and ``SubListObject *``:: static int @@ -833,7 +833,7 @@ can be safely cast to both ``PyListObject *`` and ``SubListObject *``:: return 0; } -We see above how to call through to the :attr:`__init__` method of the base +We see above how to call through to the :meth:`~object.__init__` method of the base type. This pattern is important when writing a type with custom diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 3688c47f0d6ec..1d9424cb735a4 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -779,8 +779,8 @@ by a search through the class's :term:`method resolution order`. If a descriptor is found, it is invoked with ``desc.__get__(None, A)``. -The full C implementation can be found in :c:func:`type_getattro()` and -:c:func:`_PyType_Lookup()` in :source:`Objects/typeobject.c`. +The full C implementation can be found in :c:func:`!type_getattro` and +:c:func:`!_PyType_Lookup` in :source:`Objects/typeobject.c`. Invocation from super @@ -794,7 +794,7 @@ for the base class ``B`` immediately following ``A`` and then returns ``B.__dict__['m'].__get__(obj, A)``. If not a descriptor, ``m`` is returned unchanged. -The full C implementation can be found in :c:func:`super_getattro()` in +The full C implementation can be found in :c:func:`!super_getattro` in :source:`Objects/typeobject.c`. A pure Python equivalent can be found in `Guido's Tutorial `_. @@ -836,8 +836,8 @@ and if they define :meth:`__set_name__`, that method is called with two arguments. The *owner* is the class where the descriptor is used, and the *name* is the class variable the descriptor was assigned to. -The implementation details are in :c:func:`type_new()` and -:c:func:`set_names()` in :source:`Objects/typeobject.c`. +The implementation details are in :c:func:`!type_new` and +:c:func:`!set_names` in :source:`Objects/typeobject.c`. Since the update logic is in :meth:`type.__new__`, notifications only take place at the time of class creation. If descriptors are added to the class diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 1cf8ff5a6e222..27f483b74184b 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -4,7 +4,6 @@ Doc/c-api/bool.rst Doc/c-api/buffer.rst -Doc/c-api/capsule.rst Doc/c-api/datetime.rst Doc/c-api/descriptor.rst Doc/c-api/exceptions.rst @@ -17,7 +16,6 @@ Doc/c-api/intro.rst Doc/c-api/memory.rst Doc/c-api/memoryview.rst Doc/c-api/module.rst -Doc/c-api/none.rst Doc/c-api/object.rst Doc/c-api/set.rst Doc/c-api/stable.rst @@ -26,10 +24,8 @@ Doc/c-api/sys.rst Doc/c-api/type.rst Doc/c-api/typeobj.rst Doc/c-api/unicode.rst -Doc/extending/embedding.rst Doc/extending/extending.rst Doc/extending/newtypes.rst -Doc/extending/newtypes_tutorial.rst Doc/faq/design.rst Doc/faq/extending.rst Doc/faq/gui.rst diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index 2bb681bddedbb..a47327d15fd79 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -2151,8 +2151,8 @@ Changes to Python's build process and to the C API include: Previously these different families all reduced to the platform's :c:func:`malloc` and :c:func:`free` functions. This meant it didn't matter if - you got things wrong and allocated memory with the :c:func:`PyMem` function but - freed it with the :c:func:`PyObject` function. With 2.5's changes to obmalloc, + you got things wrong and allocated memory with the ``PyMem`` function but + freed it with the ``PyObject`` function. With 2.5's changes to obmalloc, these families now do different things and mismatches will probably result in a segfault. You should carefully test your C extension modules with Python 2.5. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index a57f9bcba6189..c5fb5c53dfe35 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1850,7 +1850,7 @@ Changes in Python behavior finalizing, making them consistent with :c:func:`PyEval_RestoreThread`, :c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`. If this behavior is not desired, guard the call by checking :c:func:`_Py_IsFinalizing` - or :c:func:`sys.is_finalizing`. + or :func:`sys.is_finalizing`. (Contributed by Joannah Nanjekye in :issue:`36475`.) From webhook-mailer at python.org Thu Jul 27 15:15:50 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 27 Jul 2023 19:15:50 -0000 Subject: [Python-checkins] [3.12] gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-105258) (gh-107303) Message-ID: https://github.com/python/cpython/commit/57ef065eb3a9200aece38dfb6b90055500f076d3 commit: 57ef065eb3a9200aece38dfb6b90055500f076d3 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ericsnowcurrently date: 2023-07-27T13:15:47-06:00 summary: [3.12] gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-105258) (gh-107303) The _xxsubinterpreters module was meant to only use public API. Some internal C-API usage snuck in over the last few years (e.g. gh-28969). This fixes that. (cherry picked from commit e6373c0d8b59512aa7f0dea7f3fb162b6ed10fa4) Co-authored-by: Eric Snow files: A Include/cpython/interpreteridobject.h A Include/interpreteridobject.h D Include/internal/pycore_interpreteridobject.h M Include/internal/pycore_pymem.h M Makefile.pre.in M Modules/_testinternalcapi.c M Modules/_xxinterpchannelsmodule.c M Modules/_xxsubinterpretersmodule.c M Objects/interpreteridobject.c M Objects/object.c M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters M Tools/c-analyzer/cpython/_parser.py diff --git a/Include/internal/pycore_interpreteridobject.h b/Include/cpython/interpreteridobject.h similarity index 51% rename from Include/internal/pycore_interpreteridobject.h rename to Include/cpython/interpreteridobject.h index 804831e76deae..5076584209b90 100644 --- a/Include/internal/pycore_interpreteridobject.h +++ b/Include/cpython/interpreteridobject.h @@ -1,22 +1,11 @@ -/* Interpreter ID Object */ - -#ifndef Py_INTERNAL_INTERPRETERIDOBJECT_H -#define Py_INTERNAL_INTERPRETERIDOBJECT_H -#ifdef __cplusplus -extern "C" { +#ifndef Py_CPYTHON_INTERPRETERIDOBJECT_H +# error "this header file must not be included directly" #endif -#ifndef Py_BUILD_CORE -# error "this header requires Py_BUILD_CORE define" -#endif +/* Interpreter ID Object */ PyAPI_DATA(PyTypeObject) _PyInterpreterID_Type; PyAPI_FUNC(PyObject *) _PyInterpreterID_New(int64_t); PyAPI_FUNC(PyObject *) _PyInterpreterState_GetIDObject(PyInterpreterState *); PyAPI_FUNC(PyInterpreterState *) _PyInterpreterID_LookUp(PyObject *); - -#ifdef __cplusplus -} -#endif -#endif // !Py_INTERNAL_INTERPRETERIDOBJECT_H diff --git a/Include/internal/pycore_pymem.h b/Include/internal/pycore_pymem.h index c2f03254bb876..81a707a0a5ddf 100644 --- a/Include/internal/pycore_pymem.h +++ b/Include/internal/pycore_pymem.h @@ -95,4 +95,4 @@ PyAPI_FUNC(int) _PyMem_SetupAllocators(PyMemAllocatorName allocator); #ifdef __cplusplus } #endif -#endif // !Py_INTERNAL_PYMEM_H +#endif /* !Py_INTERNAL_PYMEM_H */ diff --git a/Include/interpreteridobject.h b/Include/interpreteridobject.h new file mode 100644 index 0000000000000..8432632f339e9 --- /dev/null +++ b/Include/interpreteridobject.h @@ -0,0 +1,17 @@ +#ifndef Py_INTERPRETERIDOBJECT_H +#define Py_INTERPRETERIDOBJECT_H + +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef Py_LIMITED_API +# define Py_CPYTHON_INTERPRETERIDOBJECT_H +# include "cpython/interpreteridobject.h" +# undef Py_CPYTHON_INTERPRETERIDOBJECT_H +#endif + +#ifdef __cplusplus +} +#endif +#endif /* !Py_INTERPRETERIDOBJECT_H */ diff --git a/Makefile.pre.in b/Makefile.pre.in index 393eec7b8d324..a74f4fd66d323 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1616,6 +1616,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/floatobject.h \ $(srcdir)/Include/frameobject.h \ $(srcdir)/Include/import.h \ + $(srcdir)/Include/interpreteridobject.h \ $(srcdir)/Include/intrcheck.h \ $(srcdir)/Include/iterobject.h \ $(srcdir)/Include/listobject.h \ @@ -1686,6 +1687,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/cpython/genobject.h \ $(srcdir)/Include/cpython/import.h \ $(srcdir)/Include/cpython/initconfig.h \ + $(srcdir)/Include/cpython/interpreteridobject.h \ $(srcdir)/Include/cpython/listobject.h \ $(srcdir)/Include/cpython/longintrepr.h \ $(srcdir)/Include/cpython/longobject.h \ @@ -1754,7 +1756,6 @@ PYTHON_HEADERS= \ $(srcdir)/Include/internal/pycore_import.h \ $(srcdir)/Include/internal/pycore_initconfig.h \ $(srcdir)/Include/internal/pycore_interp.h \ - $(srcdir)/Include/internal/pycore_interpreteridobject.h \ $(srcdir)/Include/internal/pycore_intrinsics.h \ $(srcdir)/Include/internal/pycore_list.h \ $(srcdir)/Include/internal/pycore_long.h \ diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 6fc31f61c144d..4e063a8615293 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -13,6 +13,7 @@ #include "Python.h" #include "frameobject.h" +#include "interpreteridobject.h" // _PyInterpreterID_LookUp() #include "pycore_atomic_funcs.h" // _Py_atomic_int_get() #include "pycore_bitutils.h" // _Py_bswap32() #include "pycore_bytesobject.h" // _PyBytes_Find() @@ -24,7 +25,6 @@ #include "pycore_hashtable.h" // _Py_hashtable_new() #include "pycore_initconfig.h" // _Py_GetConfigsAsDict() #include "pycore_interp.h" // _PyInterpreterState_GetConfigCopy() -#include "pycore_interpreteridobject.h" // _PyInterpreterID_LookUp() #include "pycore_pathconfig.h" // _PyPathConfig_ClearGlobal() #include "pycore_pyerrors.h" // _Py_UTF8_Edit_Cost() #include "pycore_pystate.h" // _PyThreadState_GET() diff --git a/Modules/_xxinterpchannelsmodule.c b/Modules/_xxinterpchannelsmodule.c index 616dd57768811..1d7e7f1d71af3 100644 --- a/Modules/_xxinterpchannelsmodule.c +++ b/Modules/_xxinterpchannelsmodule.c @@ -1,13 +1,9 @@ /* interpreters module */ /* low-level access to interpreter primitives */ -#ifndef Py_BUILD_CORE_BUILTIN -# define Py_BUILD_CORE_MODULE 1 -#endif #include "Python.h" -#include "pycore_pystate.h" // _PyThreadState_GET() -#include "pycore_interpreteridobject.h" +#include "interpreteridobject.h" /* diff --git a/Modules/_xxsubinterpretersmodule.c b/Modules/_xxsubinterpretersmodule.c index d7daae254638e..4801f37d6f6c5 100644 --- a/Modules/_xxsubinterpretersmodule.c +++ b/Modules/_xxsubinterpretersmodule.c @@ -1,15 +1,9 @@ /* interpreters module */ /* low-level access to interpreter primitives */ -#ifndef Py_BUILD_CORE_BUILTIN -# define Py_BUILD_CORE_MODULE 1 -#endif #include "Python.h" -// XXX This module should not rely on internal API. -#include "pycore_frame.h" -#include "pycore_pystate.h" // _PyThreadState_GET() -#include "pycore_interpreteridobject.h" +#include "interpreteridobject.h" #define MODULE_NAME "_xxsubinterpreters" @@ -376,7 +370,7 @@ _is_running(PyInterpreterState *interp) } assert(!PyErr_Occurred()); - _PyInterpreterFrame *frame = tstate->cframe->current_frame; + struct _PyInterpreterFrame *frame = tstate->cframe->current_frame; if (frame == NULL) { return 0; } @@ -512,7 +506,7 @@ interp_create(PyObject *self, PyObject *args, PyObject *kwds) } // Create and initialize the new interpreter. - PyThreadState *save_tstate = _PyThreadState_GET(); + PyThreadState *save_tstate = PyThreadState_Get(); assert(save_tstate != NULL); const PyInterpreterConfig config = isolated ? (PyInterpreterConfig)_PyInterpreterConfig_INIT diff --git a/Objects/interpreteridobject.c b/Objects/interpreteridobject.c index 7b3e31beded59..46239100dcb7b 100644 --- a/Objects/interpreteridobject.c +++ b/Objects/interpreteridobject.c @@ -3,7 +3,7 @@ #include "Python.h" #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_interp.h" // _PyInterpreterState_LookUpID() -#include "pycore_interpreteridobject.h" +#include "interpreteridobject.h" typedef struct interpid { diff --git a/Objects/object.c b/Objects/object.c index 6a73c3588da79..bd0fa40bd2a85 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -17,7 +17,7 @@ #include "pycore_typevarobject.h" // _PyTypeAlias_Type, _Py_initialize_generic #include "pycore_typeobject.h" // _PyBufferWrapper_Type #include "pycore_unionobject.h" // _PyUnion_Type -#include "pycore_interpreteridobject.h" // _PyInterpreterID_Type +#include "interpreteridobject.h" // _PyInterpreterID_Type #ifdef Py_LIMITED_API // Prevent recursive call _Py_IncRef() <=> Py_INCREF() diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 43716487f91bd..b265264dc161a 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -152,6 +152,7 @@ + @@ -234,7 +235,6 @@ - @@ -275,6 +275,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 22eb70a0f2dde..67a32f653de1f 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -315,6 +315,9 @@ Include + + Include + Modules @@ -459,6 +462,9 @@ Include + + Include\cpython + Include\cpython @@ -603,9 +609,6 @@ Include\internal - - Include\cpython - Include\cpython diff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.py index 2ee341f8dd137..1587a19716fe0 100644 --- a/Tools/c-analyzer/cpython/_parser.py +++ b/Tools/c-analyzer/cpython/_parser.py @@ -227,6 +227,7 @@ def clean_lines(text): Include/cpython/fileutils.h Py_CPYTHON_FILEUTILS_H 1 Include/cpython/frameobject.h Py_CPYTHON_FRAMEOBJECT_H 1 Include/cpython/import.h Py_CPYTHON_IMPORT_H 1 +Include/cpython/interpreteridobject.h Py_CPYTHON_INTERPRETERIDOBJECT_H 1 Include/cpython/listobject.h Py_CPYTHON_LISTOBJECT_H 1 Include/cpython/methodobject.h Py_CPYTHON_METHODOBJECT_H 1 Include/cpython/object.h Py_CPYTHON_OBJECT_H 1 From webhook-mailer at python.org Thu Jul 27 15:20:29 2023 From: webhook-mailer at python.org (gpshead) Date: Thu, 27 Jul 2023 19:20:29 -0000 Subject: [Python-checkins] gh-104432: Use `memcpy()` to avoid misaligned loads (#104433) Message-ID: https://github.com/python/cpython/commit/f01e4cedba1a17d321664834bb255d9d04ad16ce commit: f01e4cedba1a17d321664834bb255d9d04ad16ce branch: main author: Christopher Chavez committer: gpshead date: 2023-07-27T19:20:25Z summary: gh-104432: Use `memcpy()` to avoid misaligned loads (#104433) Fix potential unaligned memory access on C APIs involving returned sequences of `char *` pointers within the :mod:`grp` and :mod:`socket` modules. These were revealed using a ``-fsaniziter=alignment`` build on ARM macOS. files: A Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst M Modules/grpmodule.c M Modules/socketmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst new file mode 100644 index 0000000000000..e47927b4e1188 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst @@ -0,0 +1,4 @@ +Fix potential unaligned memory access on C APIs involving returned sequences +of `char *` pointers within the :mod:`grp` and :mod:`socket` modules. These +were revealed using a ``-fsaniziter=alignment`` build on ARM macOS. Patch by +Christopher Chavez. diff --git a/Modules/grpmodule.c b/Modules/grpmodule.c index 57cdde6064c24..f5709296334a8 100644 --- a/Modules/grpmodule.c +++ b/Modules/grpmodule.c @@ -65,8 +65,14 @@ mkgrent(PyObject *module, struct group *p) Py_DECREF(v); return NULL; } - for (member = p->gr_mem; *member != NULL; member++) { - PyObject *x = PyUnicode_DecodeFSDefault(*member); + for (member = p->gr_mem; ; member++) { + char *group_member; + // member can be misaligned + memcpy(&group_member, member, sizeof(group_member)); + if (group_member == NULL) { + break; + } + PyObject *x = PyUnicode_DecodeFSDefault(group_member); if (x == NULL || PyList_Append(w, x) != 0) { Py_XDECREF(x); Py_DECREF(w); diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index fc6a7f994bfd7..bb5edc368decb 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -5779,9 +5779,15 @@ gethost_common(socket_state *state, struct hostent *h, struct sockaddr *addr, /* SF #1511317: h_aliases can be NULL */ if (h->h_aliases) { - for (pch = h->h_aliases; *pch != NULL; pch++) { + for (pch = h->h_aliases; ; pch++) { int status; - tmp = PyUnicode_FromString(*pch); + char *host_alias; + // pch can be misaligned + memcpy(&host_alias, pch, sizeof(host_alias)); + if (host_alias == NULL) { + break; + } + tmp = PyUnicode_FromString(host_alias); if (tmp == NULL) goto err; @@ -5793,8 +5799,14 @@ gethost_common(socket_state *state, struct hostent *h, struct sockaddr *addr, } } - for (pch = h->h_addr_list; *pch != NULL; pch++) { + for (pch = h->h_addr_list; ; pch++) { int status; + char *host_address; + // pch can be misaligned + memcpy(&host_address, pch, sizeof(host_address)); + if (host_address == NULL) { + break; + } switch (af) { @@ -5806,7 +5818,7 @@ gethost_common(socket_state *state, struct hostent *h, struct sockaddr *addr, #ifdef HAVE_SOCKADDR_SA_LEN sin.sin_len = sizeof(sin); #endif - memcpy(&sin.sin_addr, *pch, sizeof(sin.sin_addr)); + memcpy(&sin.sin_addr, host_address, sizeof(sin.sin_addr)); tmp = make_ipv4_addr(&sin); if (pch == h->h_addr_list && alen >= sizeof(sin)) @@ -5823,7 +5835,7 @@ gethost_common(socket_state *state, struct hostent *h, struct sockaddr *addr, #ifdef HAVE_SOCKADDR_SA_LEN sin6.sin6_len = sizeof(sin6); #endif - memcpy(&sin6.sin6_addr, *pch, sizeof(sin6.sin6_addr)); + memcpy(&sin6.sin6_addr, host_address, sizeof(sin6.sin6_addr)); tmp = make_ipv6_addr(&sin6); if (pch == h->h_addr_list && alen >= sizeof(sin6)) From webhook-mailer at python.org Thu Jul 27 15:39:30 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 27 Jul 2023 19:39:30 -0000 Subject: [Python-checkins] gh-105699: Disable the Interpreters Stress Tests (gh-107354) Message-ID: https://github.com/python/cpython/commit/4f67921ad28194155e3d4c16255fb140a6a4d89a commit: 4f67921ad28194155e3d4c16255fb140a6a4d89a branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-27T19:39:26Z summary: gh-105699: Disable the Interpreters Stress Tests (gh-107354) The two tests are crashing periodically in CI and on buildbots. I suspect the problem is in the _xxsubinterpreters module. Regardless, I'm disabling the tests temporarily, to reduce the noise as we approach 3.12rc1. I'll be investigating the crashes separately. files: M Lib/test/test_interpreters.py diff --git a/Lib/test/test_interpreters.py b/Lib/test/test_interpreters.py index 5981d96de8de0..662eafa3b7a10 100644 --- a/Lib/test/test_interpreters.py +++ b/Lib/test/test_interpreters.py @@ -464,6 +464,7 @@ def test_bytes_for_script(self): # test_xxsubinterpreters covers the remaining Interpreter.run() behavior. + at unittest.skip('these are crashing, likely just due just to _xxsubinterpreters (see gh-105699)') class StressTests(TestBase): # In these tests we generally want a lot of interpreters, From webhook-mailer at python.org Thu Jul 27 15:57:02 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 27 Jul 2023 19:57:02 -0000 Subject: [Python-checkins] gh-106931: Intern Statically Allocated Strings Globally (gh-107272) Message-ID: https://github.com/python/cpython/commit/b72947a8d26915156323ccfd04d273199ecb870c commit: b72947a8d26915156323ccfd04d273199ecb870c branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-27T13:56:59-06:00 summary: gh-106931: Intern Statically Allocated Strings Globally (gh-107272) We tried this before with a dict and for all interned strings. That ran into problems due to interpreter isolation. However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning. Here we circle back to using a global cache, but only for statically allocated strings. We also use a more-basic _Py_hashtable_t for that global cache instead of a dict. Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter. Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter. files: A Misc/NEWS.d/next/Core and Builtins/2023-07-25-15-29-26.gh-issue-106931.kKU1le.rst M Include/cpython/unicodeobject.h M Include/internal/pycore_global_objects.h M Include/internal/pycore_hashtable.h M Include/internal/pycore_runtime.h M Include/internal/pycore_runtime_init.h M Lib/test/test_sys.py M Objects/unicodeobject.c M Python/hashtable.c M Tools/build/deepfreeze.py diff --git a/Include/cpython/unicodeobject.h b/Include/cpython/unicodeobject.h index fa00b350a799f..859ab7178e920 100644 --- a/Include/cpython/unicodeobject.h +++ b/Include/cpython/unicodeobject.h @@ -140,9 +140,11 @@ typedef struct { and the kind is PyUnicode_1BYTE_KIND. If ascii is set and compact is set, use the PyASCIIObject structure. */ unsigned int ascii:1; + /* The object is statically allocated. */ + unsigned int statically_allocated:1; /* Padding to ensure that PyUnicode_DATA() is always aligned to 4 bytes (see issue #19537 on m68k). */ - unsigned int :25; + unsigned int :24; } state; } PyASCIIObject; diff --git a/Include/internal/pycore_global_objects.h b/Include/internal/pycore_global_objects.h index 5a3fb132c745a..442f8516278b0 100644 --- a/Include/internal/pycore_global_objects.h +++ b/Include/internal/pycore_global_objects.h @@ -8,6 +8,7 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +#include "pycore_hashtable.h" // _Py_hashtable_t #include "pycore_gc.h" // PyGC_Head #include "pycore_global_strings.h" // struct _Py_global_strings #include "pycore_hamt.h" // PyHamtNode_Bitmap @@ -28,6 +29,11 @@ extern "C" { #define _Py_SINGLETON(NAME) \ _Py_GLOBAL_OBJECT(singletons.NAME) +struct _Py_cached_objects { + // XXX We could statically allocate the hashtable. + _Py_hashtable_t *interned_strings; +}; + struct _Py_static_objects { struct { /* Small integers are preallocated in this array so that they diff --git a/Include/internal/pycore_hashtable.h b/Include/internal/pycore_hashtable.h index 6501ab14d2768..f57978a8d614f 100644 --- a/Include/internal/pycore_hashtable.h +++ b/Include/internal/pycore_hashtable.h @@ -106,6 +106,7 @@ PyAPI_FUNC(int) _Py_hashtable_foreach( void *user_data); PyAPI_FUNC(size_t) _Py_hashtable_size(const _Py_hashtable_t *ht); +PyAPI_FUNC(size_t) _Py_hashtable_len(const _Py_hashtable_t *ht); /* Add a new entry to the hash. The key must not be present in the hash table. Return 0 on success, -1 on memory error. */ diff --git a/Include/internal/pycore_runtime.h b/Include/internal/pycore_runtime.h index eb6b66b335f4b..0ec86ee6c50ca 100644 --- a/Include/internal/pycore_runtime.h +++ b/Include/internal/pycore_runtime.h @@ -249,6 +249,7 @@ typedef struct pyruntimestate { struct _types_runtime_state types; /* All the objects that are shared by the runtime's interpreters. */ + struct _Py_cached_objects cached_objects; struct _Py_static_objects static_objects; /* The following fields are here to avoid allocation during init. diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h index e72e7422c7207..840e41c59ca73 100644 --- a/Include/internal/pycore_runtime_init.h +++ b/Include/internal/pycore_runtime_init.h @@ -214,6 +214,7 @@ extern PyTypeObject _PyExc_MemoryError; .kind = 1, \ .compact = 1, \ .ascii = (ASCII), \ + .statically_allocated = 1, \ }, \ } #define _PyASCIIObject_INIT(LITERAL) \ diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index 7861fa26c8007..78ed4bbaad4eb 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -14,6 +14,7 @@ from test.support.script_helper import assert_python_ok, assert_python_failure from test.support import threading_helper from test.support import import_helper +from test.support import interpreters import textwrap import unittest import warnings @@ -699,6 +700,35 @@ def __hash__(self): self.assertRaises(TypeError, sys.intern, S("abc")) + def test_subinterp_intern_dynamically_allocated(self): + global INTERN_NUMRUNS + INTERN_NUMRUNS += 1 + s = "never interned before" + str(INTERN_NUMRUNS) + t = sys.intern(s) + self.assertIs(t, s) + + interp = interpreters.create() + interp.run(textwrap.dedent(f''' + import sys + t = sys.intern({s!r}) + assert id(t) != {id(s)}, (id(t), {id(s)}) + assert id(t) != {id(t)}, (id(t), {id(t)}) + ''')) + + def test_subinterp_intern_statically_allocated(self): + # See Tools/build/generate_global_objects.py for the list + # of strings that are always statically allocated. + s = '__init__' + t = sys.intern(s) + + print('------------------------') + interp = interpreters.create() + interp.run(textwrap.dedent(f''' + import sys + t = sys.intern({s!r}) + assert id(t) == {id(t)}, (id(t), {id(t)}) + ''')) + def test_sys_flags(self): self.assertTrue(sys.flags) attrs = ("debug", diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-25-15-29-26.gh-issue-106931.kKU1le.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-25-15-29-26.gh-issue-106931.kKU1le.rst new file mode 100644 index 0000000000000..e0def5331b6c8 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-25-15-29-26.gh-issue-106931.kKU1le.rst @@ -0,0 +1,3 @@ +Statically allocated string objects are now interned globally instead of +per-interpreter. This fixes a situation where such a string would only be +interned in a single interpreter. Normal string objects are unaffected. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index fe2660c6ce605..cc979b2ef28d7 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -236,15 +236,54 @@ static inline PyObject *get_interned_dict(PyInterpreterState *interp) return _Py_INTERP_CACHED_OBJECT(interp, interned_strings); } +#define INTERNED_STRINGS _PyRuntime.cached_objects.interned_strings + Py_ssize_t _PyUnicode_InternedSize(void) { - return PyObject_Length(get_interned_dict(_PyInterpreterState_GET())); + PyObject *dict = get_interned_dict(_PyInterpreterState_GET()); + return _Py_hashtable_len(INTERNED_STRINGS) + PyDict_GET_SIZE(dict); +} + +static Py_hash_t unicode_hash(PyObject *); +static int unicode_compare_eq(PyObject *, PyObject *); + +static Py_uhash_t +hashtable_unicode_hash(const void *key) +{ + return unicode_hash((PyObject *)key); +} + +static int +hashtable_unicode_compare(const void *key1, const void *key2) +{ + PyObject *obj1 = (PyObject *)key1; + PyObject *obj2 = (PyObject *)key2; + if (obj1 != NULL && obj2 != NULL) { + return unicode_compare_eq(obj1, obj2); + } + else { + return obj1 == obj2; + } } static int init_interned_dict(PyInterpreterState *interp) { + if (_Py_IsMainInterpreter(interp)) { + assert(INTERNED_STRINGS == NULL); + _Py_hashtable_allocator_t hashtable_alloc = {PyMem_RawMalloc, PyMem_RawFree}; + INTERNED_STRINGS = _Py_hashtable_new_full( + hashtable_unicode_hash, + hashtable_unicode_compare, + NULL, + NULL, + &hashtable_alloc + ); + if (INTERNED_STRINGS == NULL) { + return -1; + } + } assert(get_interned_dict(interp) == NULL); PyObject *interned = interned = PyDict_New(); if (interned == NULL) { @@ -263,6 +302,10 @@ clear_interned_dict(PyInterpreterState *interp) Py_DECREF(interned); _Py_INTERP_CACHED_OBJECT(interp, interned_strings) = NULL; } + if (_Py_IsMainInterpreter(interp) && INTERNED_STRINGS != NULL) { + _Py_hashtable_destroy(INTERNED_STRINGS); + INTERNED_STRINGS = NULL; + } } #define _Py_RETURN_UNICODE_EMPTY() \ @@ -1223,6 +1266,7 @@ PyUnicode_New(Py_ssize_t size, Py_UCS4 maxchar) _PyUnicode_STATE(unicode).kind = kind; _PyUnicode_STATE(unicode).compact = 1; _PyUnicode_STATE(unicode).ascii = is_ascii; + _PyUnicode_STATE(unicode).statically_allocated = 0; if (is_ascii) { ((char*)data)[size] = 0; } @@ -1553,7 +1597,9 @@ unicode_dealloc(PyObject *unicode) * we accidentally decref an immortal string out of existence. Since * the string is an immortal object, just re-set the reference count. */ - if (PyUnicode_CHECK_INTERNED(unicode)) { + if (PyUnicode_CHECK_INTERNED(unicode) + || _PyUnicode_STATE(unicode).statically_allocated) + { _Py_SetImmortal(unicode); return; } @@ -14503,6 +14549,7 @@ unicode_subtype_new(PyTypeObject *type, PyObject *unicode) _PyUnicode_STATE(self).kind = kind; _PyUnicode_STATE(self).compact = 0; _PyUnicode_STATE(self).ascii = _PyUnicode_STATE(unicode).ascii; + _PyUnicode_STATE(self).statically_allocated = 0; _PyUnicode_UTF8_LENGTH(self) = 0; _PyUnicode_UTF8(self) = NULL; _PyUnicode_DATA_ANY(self) = NULL; @@ -14726,6 +14773,23 @@ _PyUnicode_InternInPlace(PyInterpreterState *interp, PyObject **p) return; } + /* Look in the global cache first. */ + PyObject *r = (PyObject *)_Py_hashtable_get(INTERNED_STRINGS, s); + if (r != NULL && r != s) { + Py_SETREF(*p, Py_NewRef(r)); + return; + } + + /* Handle statically allocated strings. */ + if (_PyUnicode_STATE(s).statically_allocated) { + assert(_Py_IsImmortal(s)); + if (_Py_hashtable_set(INTERNED_STRINGS, s, s) == 0) { + _PyUnicode_STATE(*p).interned = SSTATE_INTERNED_IMMORTAL_STATIC; + } + return; + } + + /* Look in the per-interpreter cache. */ PyObject *interned = get_interned_dict(interp); assert(interned != NULL); @@ -14741,9 +14805,11 @@ _PyUnicode_InternInPlace(PyInterpreterState *interp, PyObject **p) } if (_Py_IsImmortal(s)) { + // XXX Restrict this to the main interpreter? _PyUnicode_STATE(*p).interned = SSTATE_INTERNED_IMMORTAL_STATIC; - return; + return; } + #ifdef Py_REF_DEBUG /* The reference count value excluding the 2 references from the interned dictionary should be excluded from the RefTotal. The diff --git a/Python/hashtable.c b/Python/hashtable.c index 0f07cd20b1a95..4e22a1a5509eb 100644 --- a/Python/hashtable.c +++ b/Python/hashtable.c @@ -129,6 +129,13 @@ _Py_hashtable_size(const _Py_hashtable_t *ht) } +size_t +_Py_hashtable_len(const _Py_hashtable_t *ht) +{ + return ht->nentries; +} + + _Py_hashtable_entry_t * _Py_hashtable_get_entry_generic(_Py_hashtable_t *ht, const void *key) { diff --git a/Tools/build/deepfreeze.py b/Tools/build/deepfreeze.py index b084d3e457f78..a11fe6a62811a 100644 --- a/Tools/build/deepfreeze.py +++ b/Tools/build/deepfreeze.py @@ -208,6 +208,7 @@ def generate_unicode(self, name: str, s: str) -> str: self.write(".kind = 1,") self.write(".compact = 1,") self.write(".ascii = 1,") + self.write(".statically_allocated = 1,") self.write(f"._data = {make_string_literal(s.encode('ascii'))},") return f"& {name}._ascii.ob_base" else: @@ -220,6 +221,7 @@ def generate_unicode(self, name: str, s: str) -> str: self.write(f".kind = {kind},") self.write(".compact = 1,") self.write(".ascii = 0,") + self.write(".statically_allocated = 1,") utf8 = s.encode('utf-8') self.write(f'.utf8 = {make_string_literal(utf8)},') self.write(f'.utf8_length = {len(utf8)},') From webhook-mailer at python.org Thu Jul 27 16:09:10 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 27 Jul 2023 20:09:10 -0000 Subject: [Python-checkins] [3.12] gh-105699: Disable the Interpreters Stress Tests (gh-107354) (gh-107357) Message-ID: https://github.com/python/cpython/commit/c580527d9230212e8f8d697a18b86120109aa653 commit: c580527d9230212e8f8d697a18b86120109aa653 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ericsnowcurrently date: 2023-07-27T20:09:05Z summary: [3.12] gh-105699: Disable the Interpreters Stress Tests (gh-107354) (gh-107357) gh-105699: Disable the Interpreters Stress Tests (gh-107354) The two tests are crashing periodically in CI and on buildbots. I suspect the problem is in the _xxsubinterpreters module. Regardless, I'm disabling the tests temporarily, to reduce the noise as we approach 3.12rc1. I'll be investigating the crashes separately. (cherry picked from commit 4f67921ad28194155e3d4c16255fb140a6a4d89a) Co-authored-by: Eric Snow files: M Lib/test/test_interpreters.py diff --git a/Lib/test/test_interpreters.py b/Lib/test/test_interpreters.py index 5981d96de8de0..662eafa3b7a10 100644 --- a/Lib/test/test_interpreters.py +++ b/Lib/test/test_interpreters.py @@ -464,6 +464,7 @@ def test_bytes_for_script(self): # test_xxsubinterpreters covers the remaining Interpreter.run() behavior. + at unittest.skip('these are crashing, likely just due just to _xxsubinterpreters (see gh-105699)') class StressTests(TestBase): # In these tests we generally want a lot of interpreters, From webhook-mailer at python.org Thu Jul 27 16:52:44 2023 From: webhook-mailer at python.org (gpshead) Date: Thu, 27 Jul 2023 20:52:44 -0000 Subject: [Python-checkins] [3.12] gh-104432: Use `memcpy()` to avoid misaligned loads (GH-104433) (#107355) Message-ID: https://github.com/python/cpython/commit/5daf19d763826a977a596b6fbc035ee03c0deafc commit: 5daf19d763826a977a596b6fbc035ee03c0deafc branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gpshead date: 2023-07-27T13:52:39-07:00 summary: [3.12] gh-104432: Use `memcpy()` to avoid misaligned loads (GH-104433) (#107355) gh-104432: Use `memcpy()` to avoid misaligned loads (GH-104433) Fix potential unaligned memory access on C APIs involving returned sequences of `char *` pointers within the :mod:`grp` and :mod:`socket` modules. These were revealed using a ``-fsaniziter=alignment`` build on ARM macOS. (cherry picked from commit f01e4cedba1a17d321664834bb255d9d04ad16ce) Co-authored-by: Christopher Chavez files: A Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst M Modules/grpmodule.c M Modules/socketmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst new file mode 100644 index 0000000000000..e47927b4e1188 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst @@ -0,0 +1,4 @@ +Fix potential unaligned memory access on C APIs involving returned sequences +of `char *` pointers within the :mod:`grp` and :mod:`socket` modules. These +were revealed using a ``-fsaniziter=alignment`` build on ARM macOS. Patch by +Christopher Chavez. diff --git a/Modules/grpmodule.c b/Modules/grpmodule.c index 57cdde6064c24..f5709296334a8 100644 --- a/Modules/grpmodule.c +++ b/Modules/grpmodule.c @@ -65,8 +65,14 @@ mkgrent(PyObject *module, struct group *p) Py_DECREF(v); return NULL; } - for (member = p->gr_mem; *member != NULL; member++) { - PyObject *x = PyUnicode_DecodeFSDefault(*member); + for (member = p->gr_mem; ; member++) { + char *group_member; + // member can be misaligned + memcpy(&group_member, member, sizeof(group_member)); + if (group_member == NULL) { + break; + } + PyObject *x = PyUnicode_DecodeFSDefault(group_member); if (x == NULL || PyList_Append(w, x) != 0) { Py_XDECREF(x); Py_DECREF(w); diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index a86aaed501fa3..1ae7ab778290f 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -5783,9 +5783,15 @@ gethost_common(socket_state *state, struct hostent *h, struct sockaddr *addr, /* SF #1511317: h_aliases can be NULL */ if (h->h_aliases) { - for (pch = h->h_aliases; *pch != NULL; pch++) { + for (pch = h->h_aliases; ; pch++) { int status; - tmp = PyUnicode_FromString(*pch); + char *host_alias; + // pch can be misaligned + memcpy(&host_alias, pch, sizeof(host_alias)); + if (host_alias == NULL) { + break; + } + tmp = PyUnicode_FromString(host_alias); if (tmp == NULL) goto err; @@ -5797,8 +5803,14 @@ gethost_common(socket_state *state, struct hostent *h, struct sockaddr *addr, } } - for (pch = h->h_addr_list; *pch != NULL; pch++) { + for (pch = h->h_addr_list; ; pch++) { int status; + char *host_address; + // pch can be misaligned + memcpy(&host_address, pch, sizeof(host_address)); + if (host_address == NULL) { + break; + } switch (af) { @@ -5810,7 +5822,7 @@ gethost_common(socket_state *state, struct hostent *h, struct sockaddr *addr, #ifdef HAVE_SOCKADDR_SA_LEN sin.sin_len = sizeof(sin); #endif - memcpy(&sin.sin_addr, *pch, sizeof(sin.sin_addr)); + memcpy(&sin.sin_addr, host_address, sizeof(sin.sin_addr)); tmp = make_ipv4_addr(&sin); if (pch == h->h_addr_list && alen >= sizeof(sin)) @@ -5827,7 +5839,7 @@ gethost_common(socket_state *state, struct hostent *h, struct sockaddr *addr, #ifdef HAVE_SOCKADDR_SA_LEN sin6.sin6_len = sizeof(sin6); #endif - memcpy(&sin6.sin6_addr, *pch, sizeof(sin6.sin6_addr)); + memcpy(&sin6.sin6_addr, host_address, sizeof(sin6.sin6_addr)); tmp = make_ipv6_addr(&sin6); if (pch == h->h_addr_list && alen >= sizeof(sin6)) From webhook-mailer at python.org Thu Jul 27 17:08:41 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 27 Jul 2023 21:08:41 -0000 Subject: [Python-checkins] gh-104621: Check for Incompatible Extensions in import_find_extension() (gh-107184) Message-ID: https://github.com/python/cpython/commit/75c974f5353685f338344618ad7344e64c2293d0 commit: 75c974f5353685f338344618ad7344e64c2293d0 branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-27T15:08:38-06:00 summary: gh-104621: Check for Incompatible Extensions in import_find_extension() (gh-107184) This fixes a bug where incompatible modules could still be imported if attempted multiple times. files: A Misc/NEWS.d/next/Core and Builtins/2023-07-24-11-11-41.gh-issue-104621.vM8Y_l.rst M Lib/test/test_capi/check_config.py M Lib/test/test_import/__init__.py M Python/import.c diff --git a/Lib/test/test_capi/check_config.py b/Lib/test/test_capi/check_config.py index aaedd82f39af5..eb99ae16f2b69 100644 --- a/Lib/test/test_capi/check_config.py +++ b/Lib/test/test_capi/check_config.py @@ -12,7 +12,7 @@ def import_singlephase(): try: import _testsinglephase except ImportError: - sys.modules.pop('_testsinglephase') + sys.modules.pop('_testsinglephase', None) return False else: del sys.modules['_testsinglephase'] diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py index ec8ccf0bd78d3..7a3fcc22be8d1 100644 --- a/Lib/test/test_import/__init__.py +++ b/Lib/test/test_import/__init__.py @@ -97,7 +97,6 @@ def require_frozen(module, *, skip=True): def require_pure_python(module, *, skip=False): _require_loader(module, SourceFileLoader, skip) - def remove_files(name): for f in (name + ".py", name + ".pyc", @@ -147,19 +146,34 @@ def _ready_to_import(name=None, source=""): del sys.modules[name] -def requires_subinterpreters(meth): - """Decorator to skip a test if subinterpreters are not supported.""" - return unittest.skipIf(_interpreters is None, - 'subinterpreters required')(meth) +if _testsinglephase is not None: + def restore__testsinglephase(*, _orig=_testsinglephase): + # We started with the module imported and want to restore + # it to its nominal state. + _orig._clear_globals() + _testinternalcapi.clear_extension('_testsinglephase', _orig.__file__) + import _testsinglephase def requires_singlephase_init(meth): """Decorator to skip if single-phase init modules are not supported.""" + if not isinstance(meth, type): + def meth(self, _meth=meth): + try: + return _meth(self) + finally: + restore__testsinglephase() meth = cpython_only(meth) return unittest.skipIf(_testsinglephase is None, 'test requires _testsinglephase module')(meth) +def requires_subinterpreters(meth): + """Decorator to skip a test if subinterpreters are not supported.""" + return unittest.skipIf(_interpreters is None, + 'subinterpreters required')(meth) + + class ModuleSnapshot(types.SimpleNamespace): """A representation of a module for testing. @@ -1962,6 +1976,20 @@ def test_isolated_config(self): with self.subTest(f'{module}: strict, fresh'): self.check_compatible_fresh(module, strict=True, isolated=True) + @requires_subinterpreters + @requires_singlephase_init + def test_disallowed_reimport(self): + # See https://github.com/python/cpython/issues/104621. + script = textwrap.dedent(''' + import _testsinglephase + print(_testsinglephase) + ''') + interpid = _interpreters.create() + with self.assertRaises(_interpreters.RunFailedError): + _interpreters.run_string(interpid, script) + with self.assertRaises(_interpreters.RunFailedError): + _interpreters.run_string(interpid, script) + class TestSinglePhaseSnapshot(ModuleSnapshot): @@ -2017,6 +2045,10 @@ def setUpClass(cls): # Start fresh. cls.clean_up() + @classmethod + def tearDownClass(cls): + restore__testsinglephase() + def tearDown(self): # Clean up the module. self.clean_up() diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-24-11-11-41.gh-issue-104621.vM8Y_l.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-24-11-11-41.gh-issue-104621.vM8Y_l.rst new file mode 100644 index 0000000000000..86c976295f262 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-24-11-11-41.gh-issue-104621.vM8Y_l.rst @@ -0,0 +1 @@ +Unsupported modules now always fail to be imported. diff --git a/Python/import.c b/Python/import.c index f8cae641a6454..711ed5d32ca97 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1215,6 +1215,15 @@ import_find_extension(PyThreadState *tstate, PyObject *name, return NULL; } + /* It may have been successfully imported previously + in an interpreter that allows legacy modules + but is not allowed in the current interpreter. */ + const char *name_buf = PyUnicode_AsUTF8(name); + assert(name_buf != NULL); + if (_PyImport_CheckSubinterpIncompatibleExtensionAllowed(name_buf) < 0) { + return NULL; + } + PyObject *mod, *mdict; PyObject *modules = MODULES(tstate->interp); @@ -3704,16 +3713,8 @@ _imp_create_dynamic_impl(PyObject *module, PyObject *spec, PyObject *file) PyThreadState *tstate = _PyThreadState_GET(); mod = import_find_extension(tstate, name, path); - if (mod != NULL) { - const char *name_buf = PyUnicode_AsUTF8(name); - assert(name_buf != NULL); - if (_PyImport_CheckSubinterpIncompatibleExtensionAllowed(name_buf) < 0) { - Py_DECREF(mod); - mod = NULL; - } - goto finally; - } - else if (PyErr_Occurred()) { + if (mod != NULL || _PyErr_Occurred(tstate)) { + assert(mod == NULL || !_PyErr_Occurred(tstate)); goto finally; } From webhook-mailer at python.org Thu Jul 27 17:30:20 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 27 Jul 2023 21:30:20 -0000 Subject: [Python-checkins] gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359) Message-ID: https://github.com/python/cpython/commit/8bdae1424b54e5106782f2b9e2fadce444dc84cd commit: 8bdae1424b54e5106782f2b9e2fadce444dc84cd branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-27T15:30:16-06:00 summary: gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359) The _xxsubinterpreters module should not rely on internal API. Some of the functions it uses were recently moved there however. Here we move them back (and expose them properly). files: A Include/cpython/interpreteridobject.h A Include/interpreteridobject.h D Include/internal/pycore_interp_id.h M Include/cpython/pylifecycle.h M Include/cpython/pystate.h M Include/internal/pycore_interp.h M Makefile.pre.in M Modules/_testinternalcapi.c M Modules/_xxinterpchannelsmodule.c M Modules/_xxsubinterpretersmodule.c M Modules/atexitmodule.c M Objects/interpreteridobject.c M Objects/object.c M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters M Python/pystate.c M Tools/c-analyzer/cpython/_parser.py M Tools/c-analyzer/cpython/globals-to-fix.tsv diff --git a/Include/cpython/interpreteridobject.h b/Include/cpython/interpreteridobject.h new file mode 100644 index 0000000000000..4ab9ad5d315f8 --- /dev/null +++ b/Include/cpython/interpreteridobject.h @@ -0,0 +1,11 @@ +#ifndef Py_CPYTHON_INTERPRETERIDOBJECT_H +# error "this header file must not be included directly" +#endif + +/* Interpreter ID Object */ + +PyAPI_DATA(PyTypeObject) PyInterpreterID_Type; + +PyAPI_FUNC(PyObject *) PyInterpreterID_New(int64_t); +PyAPI_FUNC(PyObject *) PyInterpreterState_GetIDObject(PyInterpreterState *); +PyAPI_FUNC(PyInterpreterState *) PyInterpreterID_LookUp(PyObject *); diff --git a/Include/cpython/pylifecycle.h b/Include/cpython/pylifecycle.h index a78cec9525d5f..d425a233f7100 100644 --- a/Include/cpython/pylifecycle.h +++ b/Include/cpython/pylifecycle.h @@ -77,3 +77,7 @@ typedef struct { PyAPI_FUNC(PyStatus) Py_NewInterpreterFromConfig( PyThreadState **tstate_p, const PyInterpreterConfig *config); + +typedef void (*atexit_datacallbackfunc)(void *); +PyAPI_FUNC(int) PyUnstable_AtExit( + PyInterpreterState *, atexit_datacallbackfunc, void *); diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index 4254110889fc6..30de4ee4b6c58 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -8,6 +8,7 @@ PyAPI_FUNC(int) _PyInterpreterState_RequiresIDRef(PyInterpreterState *); PyAPI_FUNC(void) _PyInterpreterState_RequireIDRef(PyInterpreterState *, int); +PyAPI_FUNC(PyObject *) PyUnstable_InterpreterState_GetMainModule(PyInterpreterState *); /* State unique per thread */ diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index 522ce4a80ecc2..bd6a9f28f468a 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -238,9 +238,6 @@ extern int _PyInterpreterState_IDInitref(PyInterpreterState *); extern int _PyInterpreterState_IDIncref(PyInterpreterState *); extern void _PyInterpreterState_IDDecref(PyInterpreterState *); -// Export for '_xxsubinterpreters' shared extension -PyAPI_FUNC(PyObject*) _PyInterpreterState_GetMainModule(PyInterpreterState *); - extern const PyConfig* _PyInterpreterState_GetConfig(PyInterpreterState *interp); /* Get a copy of the current interpreter configuration. diff --git a/Include/internal/pycore_interp_id.h b/Include/internal/pycore_interp_id.h deleted file mode 100644 index 8c6f8315cac39..0000000000000 --- a/Include/internal/pycore_interp_id.h +++ /dev/null @@ -1,28 +0,0 @@ -/* Interpreter ID Object */ - -#ifndef Py_INTERNAL_INTERPRETERIDOBJECT_H -#define Py_INTERNAL_INTERPRETERIDOBJECT_H -#ifdef __cplusplus -extern "C" { -#endif - -#ifndef Py_BUILD_CORE -# error "this header requires Py_BUILD_CORE define" -#endif - -// Export for '_xxsubinterpreters' shared extension -PyAPI_DATA(PyTypeObject) _PyInterpreterID_Type; - -// Export for '_xxsubinterpreters' shared extension -PyAPI_FUNC(PyObject*) _PyInterpreterID_New(int64_t); - -// Export for '_xxinterpchannels' shared extension -PyAPI_FUNC(PyObject*) _PyInterpreterState_GetIDObject(PyInterpreterState *); - -// Export for '_testinternalcapi' shared extension -PyAPI_FUNC(PyInterpreterState*) _PyInterpreterID_LookUp(PyObject *); - -#ifdef __cplusplus -} -#endif -#endif // !Py_INTERNAL_INTERPRETERIDOBJECT_H diff --git a/Include/interpreteridobject.h b/Include/interpreteridobject.h new file mode 100644 index 0000000000000..8432632f339e9 --- /dev/null +++ b/Include/interpreteridobject.h @@ -0,0 +1,17 @@ +#ifndef Py_INTERPRETERIDOBJECT_H +#define Py_INTERPRETERIDOBJECT_H + +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef Py_LIMITED_API +# define Py_CPYTHON_INTERPRETERIDOBJECT_H +# include "cpython/interpreteridobject.h" +# undef Py_CPYTHON_INTERPRETERIDOBJECT_H +#endif + +#ifdef __cplusplus +} +#endif +#endif /* !Py_INTERPRETERIDOBJECT_H */ diff --git a/Makefile.pre.in b/Makefile.pre.in index 5f1988b0d1721..3725feaca66ce 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1631,6 +1631,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/floatobject.h \ $(srcdir)/Include/frameobject.h \ $(srcdir)/Include/import.h \ + $(srcdir)/Include/interpreteridobject.h \ $(srcdir)/Include/intrcheck.h \ $(srcdir)/Include/iterobject.h \ $(srcdir)/Include/listobject.h \ @@ -1700,6 +1701,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/cpython/genobject.h \ $(srcdir)/Include/cpython/import.h \ $(srcdir)/Include/cpython/initconfig.h \ + $(srcdir)/Include/cpython/interpreteridobject.h \ $(srcdir)/Include/cpython/listobject.h \ $(srcdir)/Include/cpython/longintrepr.h \ $(srcdir)/Include/cpython/longobject.h \ @@ -1771,7 +1773,6 @@ PYTHON_HEADERS= \ $(srcdir)/Include/internal/pycore_import.h \ $(srcdir)/Include/internal/pycore_initconfig.h \ $(srcdir)/Include/internal/pycore_interp.h \ - $(srcdir)/Include/internal/pycore_interp_id.h \ $(srcdir)/Include/internal/pycore_intrinsics.h \ $(srcdir)/Include/internal/pycore_list.h \ $(srcdir)/Include/internal/pycore_long.h \ diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c index 98902d6134be8..f28b46dd9846c 100644 --- a/Modules/_testinternalcapi.c +++ b/Modules/_testinternalcapi.c @@ -13,27 +13,27 @@ #include "pycore_atomic_funcs.h" // _Py_atomic_int_get() #include "pycore_bitutils.h" // _Py_bswap32() #include "pycore_bytesobject.h" // _PyBytes_Find() -#include "pycore_ceval.h" // _PyEval_AddPendingCall #include "pycore_compile.h" // _PyCompile_CodeGen, _PyCompile_OptimizeCfg, _PyCompile_Assemble, _PyCompile_CleanDoc +#include "pycore_ceval.h" // _PyEval_AddPendingCall #include "pycore_fileutils.h" // _Py_normpath #include "pycore_frame.h" // _PyInterpreterFrame #include "pycore_gc.h" // PyGC_Head #include "pycore_hashtable.h" // _Py_hashtable_new() #include "pycore_initconfig.h" // _Py_GetConfigsAsDict() #include "pycore_interp.h" // _PyInterpreterState_GetConfigCopy() -#include "pycore_interp_id.h" // _PyInterpreterID_LookUp() -#include "pycore_object.h" // _PyObject_IsFreed() +#include "pycore_object.h" // _PyObject_IsFreed() #include "pycore_pathconfig.h" // _PyPathConfig_ClearGlobal() #include "pycore_pyerrors.h" // _Py_UTF8_Edit_Cost() #include "pycore_pystate.h" // _PyThreadState_GET() #include "frameobject.h" +#include "interpreteridobject.h" // PyInterpreterID_LookUp() #include "osdefs.h" // MAXPATHLEN #include "clinic/_testinternalcapi.c.h" #ifdef MS_WINDOWS -# include // struct timeval +# include // struct timeval #endif @@ -1083,7 +1083,7 @@ pending_identify(PyObject *self, PyObject *args) if (!PyArg_ParseTuple(args, "O:pending_identify", &interpid)) { return NULL; } - PyInterpreterState *interp = _PyInterpreterID_LookUp(interpid); + PyInterpreterState *interp = PyInterpreterID_LookUp(interpid); if (interp == NULL) { if (!PyErr_Occurred()) { PyErr_SetString(PyExc_ValueError, "interpreter not found"); @@ -1433,7 +1433,7 @@ test_atexit(PyObject *self, PyObject *Py_UNUSED(args)) PyThreadState *tstate = Py_NewInterpreter(); struct atexit_data data = {0}; - int res = _Py_AtExit(tstate->interp, callback, (void *)&data); + int res = PyUnstable_AtExit(tstate->interp, callback, (void *)&data); Py_EndInterpreter(tstate); PyThreadState_Swap(oldts); if (res < 0) { diff --git a/Modules/_xxinterpchannelsmodule.c b/Modules/_xxinterpchannelsmodule.c index bdffdba52aa02..1e418414767db 100644 --- a/Modules/_xxinterpchannelsmodule.c +++ b/Modules/_xxinterpchannelsmodule.c @@ -1,13 +1,8 @@ /* interpreters module */ /* low-level access to interpreter primitives */ -#ifndef Py_BUILD_CORE_BUILTIN -# define Py_BUILD_CORE_MODULE 1 -#endif - #include "Python.h" -#include "pycore_atexit.h" // _Py_AtExit() -#include "pycore_interp_id.h" // _PyInterpreterState_GetIDObject() +#include "interpreteridobject.h" /* @@ -2140,7 +2135,7 @@ channel_list_interpreters(PyObject *self, PyObject *args, PyObject *kwds) goto except; } if (res) { - id_obj = _PyInterpreterState_GetIDObject(interp); + id_obj = PyInterpreterState_GetIDObject(interp); if (id_obj == NULL) { goto except; } @@ -2407,7 +2402,7 @@ module_exec(PyObject *mod) // Make sure chnnels drop objects owned by this interpreter PyInterpreterState *interp = _get_current_interp(); - _Py_AtExit(interp, clear_interpreter, (void *)interp); + PyUnstable_AtExit(interp, clear_interpreter, (void *)interp); return 0; diff --git a/Modules/_xxsubinterpretersmodule.c b/Modules/_xxsubinterpretersmodule.c index bf01aaf6ed040..31373f8fdf8c7 100644 --- a/Modules/_xxsubinterpretersmodule.c +++ b/Modules/_xxsubinterpretersmodule.c @@ -1,13 +1,8 @@ /* interpreters module */ /* low-level access to interpreter primitives */ -#ifndef Py_BUILD_CORE_BUILTIN -# define Py_BUILD_CORE_MODULE 1 -#endif - #include "Python.h" -#include "pycore_interp.h" // _PyInterpreterState_GetMainModule() -#include "pycore_interp_id.h" // _PyInterpreterState_GetIDObject() +#include "interpreteridobject.h" #define MODULE_NAME "_xxsubinterpreters" @@ -400,7 +395,7 @@ _run_script(PyInterpreterState *interp, const char *codestr, _sharedns *shared, _sharedexception *sharedexc) { PyObject *excval = NULL; - PyObject *main_mod = _PyInterpreterState_GetMainModule(interp); + PyObject *main_mod = PyUnstable_InterpreterState_GetMainModule(interp); if (main_mod == NULL) { goto error; } @@ -531,7 +526,7 @@ interp_create(PyObject *self, PyObject *args, PyObject *kwds) } assert(tstate != NULL); PyInterpreterState *interp = PyThreadState_GetInterpreter(tstate); - PyObject *idobj = _PyInterpreterState_GetIDObject(interp); + PyObject *idobj = PyInterpreterState_GetIDObject(interp); if (idobj == NULL) { // XXX Possible GILState issues? save_tstate = PyThreadState_Swap(tstate); @@ -561,7 +556,7 @@ interp_destroy(PyObject *self, PyObject *args, PyObject *kwds) } // Look up the interpreter. - PyInterpreterState *interp = _PyInterpreterID_LookUp(id); + PyInterpreterState *interp = PyInterpreterID_LookUp(id); if (interp == NULL) { return NULL; } @@ -616,7 +611,7 @@ interp_list_all(PyObject *self, PyObject *Py_UNUSED(ignored)) interp = PyInterpreterState_Head(); while (interp != NULL) { - id = _PyInterpreterState_GetIDObject(interp); + id = PyInterpreterState_GetIDObject(interp); if (id == NULL) { Py_DECREF(ids); return NULL; @@ -648,7 +643,7 @@ interp_get_current(PyObject *self, PyObject *Py_UNUSED(ignored)) if (interp == NULL) { return NULL; } - return _PyInterpreterState_GetIDObject(interp); + return PyInterpreterState_GetIDObject(interp); } PyDoc_STRVAR(get_current_doc, @@ -662,7 +657,7 @@ interp_get_main(PyObject *self, PyObject *Py_UNUSED(ignored)) { // Currently, 0 is always the main interpreter. int64_t id = 0; - return _PyInterpreterID_New(id); + return PyInterpreterID_New(id); } PyDoc_STRVAR(get_main_doc, @@ -684,7 +679,7 @@ interp_run_string(PyObject *self, PyObject *args, PyObject *kwds) } // Look up the interpreter. - PyInterpreterState *interp = _PyInterpreterID_LookUp(id); + PyInterpreterState *interp = PyInterpreterID_LookUp(id); if (interp == NULL) { return NULL; } @@ -750,7 +745,7 @@ interp_is_running(PyObject *self, PyObject *args, PyObject *kwds) return NULL; } - PyInterpreterState *interp = _PyInterpreterID_LookUp(id); + PyInterpreterState *interp = PyInterpreterID_LookUp(id); if (interp == NULL) { return NULL; } @@ -808,7 +803,7 @@ module_exec(PyObject *mod) } // PyInterpreterID - if (PyModule_AddType(mod, &_PyInterpreterID_Type) < 0) { + if (PyModule_AddType(mod, &PyInterpreterID_Type) < 0) { goto error; } diff --git a/Modules/atexitmodule.c b/Modules/atexitmodule.c index 5882d40563640..cec177cfc2f9c 100644 --- a/Modules/atexitmodule.c +++ b/Modules/atexitmodule.c @@ -24,8 +24,8 @@ get_atexit_state(void) int -_Py_AtExit(PyInterpreterState *interp, - atexit_datacallbackfunc func, void *data) +PyUnstable_AtExit(PyInterpreterState *interp, + atexit_datacallbackfunc func, void *data) { assert(interp == _PyInterpreterState_GET()); atexit_callback *callback = PyMem_Malloc(sizeof(atexit_callback)); diff --git a/Objects/interpreteridobject.c b/Objects/interpreteridobject.c index 7a3e245ce3357..16e27b64c0c9c 100644 --- a/Objects/interpreteridobject.c +++ b/Objects/interpreteridobject.c @@ -3,7 +3,7 @@ #include "Python.h" #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_interp.h" // _PyInterpreterState_LookUpID() -#include "pycore_interp_id.h" // _PyInterpreterID_Type +#include "interpreteridobject.h" typedef struct interpid { @@ -46,7 +46,7 @@ static int interp_id_converter(PyObject *arg, void *ptr) { int64_t id; - if (PyObject_TypeCheck(arg, &_PyInterpreterID_Type)) { + if (PyObject_TypeCheck(arg, &PyInterpreterID_Type)) { id = ((interpid *)arg)->id; } else if (_PyIndex_Check(arg)) { @@ -183,13 +183,13 @@ interpid_richcompare(PyObject *self, PyObject *other, int op) Py_RETURN_NOTIMPLEMENTED; } - if (!PyObject_TypeCheck(self, &_PyInterpreterID_Type)) { + if (!PyObject_TypeCheck(self, &PyInterpreterID_Type)) { Py_RETURN_NOTIMPLEMENTED; } interpid *id = (interpid *)self; int equal; - if (PyObject_TypeCheck(other, &_PyInterpreterID_Type)) { + if (PyObject_TypeCheck(other, &PyInterpreterID_Type)) { interpid *otherid = (interpid *)other; equal = (id->id == otherid->id); } @@ -224,7 +224,7 @@ interpid_richcompare(PyObject *self, PyObject *other, int op) PyDoc_STRVAR(interpid_doc, "A interpreter ID identifies a interpreter and may be used as an int."); -PyTypeObject _PyInterpreterID_Type = { +PyTypeObject PyInterpreterID_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "InterpreterID", /* tp_name */ sizeof(interpid), /* tp_basicsize */ @@ -265,13 +265,13 @@ PyTypeObject _PyInterpreterID_Type = { interpid_new, /* tp_new */ }; -PyObject *_PyInterpreterID_New(int64_t id) +PyObject *PyInterpreterID_New(int64_t id) { - return (PyObject *)newinterpid(&_PyInterpreterID_Type, id, 0); + return (PyObject *)newinterpid(&PyInterpreterID_Type, id, 0); } PyObject * -_PyInterpreterState_GetIDObject(PyInterpreterState *interp) +PyInterpreterState_GetIDObject(PyInterpreterState *interp) { if (_PyInterpreterState_IDInitref(interp) != 0) { return NULL; @@ -280,11 +280,11 @@ _PyInterpreterState_GetIDObject(PyInterpreterState *interp) if (id < 0) { return NULL; } - return (PyObject *)newinterpid(&_PyInterpreterID_Type, id, 0); + return (PyObject *)newinterpid(&PyInterpreterID_Type, id, 0); } PyInterpreterState * -_PyInterpreterID_LookUp(PyObject *requested_id) +PyInterpreterID_LookUp(PyObject *requested_id) { int64_t id; if (!interp_id_converter(requested_id, &id)) { diff --git a/Objects/object.c b/Objects/object.c index b724eb5f8f079..8caa5fd0af3cc 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -9,16 +9,16 @@ #include "pycore_dict.h" // _PyObject_MakeDictFromInstanceAttributes() #include "pycore_floatobject.h" // _PyFloat_DebugMallocStats() #include "pycore_initconfig.h" // _PyStatus_EXCEPTION() -#include "pycore_interp_id.h" // _PyInterpreterID_Type #include "pycore_namespace.h" // _PyNamespace_Type #include "pycore_object.h" // PyAPI_DATA() _Py_SwappedOp definition #include "pycore_pyerrors.h" // _PyErr_Occurred() #include "pycore_pymem.h" // _PyMem_IsPtrFreed() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_symtable.h" // PySTEntry_Type -#include "pycore_typeobject.h" // _PyBufferWrapper_Type #include "pycore_typevarobject.h" // _PyTypeAlias_Type, _Py_initialize_generic +#include "pycore_typeobject.h" // _PyBufferWrapper_Type #include "pycore_unionobject.h" // _PyUnion_Type +#include "interpreteridobject.h" // _PyInterpreterID_Type #ifdef Py_LIMITED_API // Prevent recursive call _Py_IncRef() <=> Py_INCREF() @@ -2072,6 +2072,7 @@ static PyTypeObject* static_types[] = { &PyGen_Type, &PyGetSetDescr_Type, &PyInstanceMethod_Type, + &PyInterpreterID_Type, &PyListIter_Type, &PyListRevIter_Type, &PyList_Type, @@ -2122,7 +2123,6 @@ static PyTypeObject* static_types[] = { &_PyHamt_CollisionNode_Type, &_PyHamt_Type, &_PyLegacyEventHandler_Type, - &_PyInterpreterID_Type, &_PyLineIterator, &_PyManagedBuffer_Type, &_PyMemoryIter_Type, diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 5ccc895833065..bfe59acf12a69 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -152,6 +152,7 @@ + @@ -237,7 +238,6 @@ - @@ -281,6 +281,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 54a77f81a9a1a..0a8b0c3faf51e 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -312,6 +312,9 @@ Include + + Include + Modules @@ -459,6 +462,9 @@ Include + + Include\cpython + Include\cpython @@ -612,9 +618,6 @@ Include\internal - - Include\internal - Include\cpython diff --git a/Python/pystate.c b/Python/pystate.c index cdd975f3672b9..3ec131b6cba99 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -1133,7 +1133,7 @@ _PyInterpreterState_RequireIDRef(PyInterpreterState *interp, int required) } PyObject * -_PyInterpreterState_GetMainModule(PyInterpreterState *interp) +PyUnstable_InterpreterState_GetMainModule(PyInterpreterState *interp) { PyObject *modules = _PyImport_GetModules(interp); if (modules == NULL) { diff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.py index 64ce4cda7d6ed..9bc7285e18b2f 100644 --- a/Tools/c-analyzer/cpython/_parser.py +++ b/Tools/c-analyzer/cpython/_parser.py @@ -227,6 +227,7 @@ def clean_lines(text): Include/cpython/fileutils.h Py_CPYTHON_FILEUTILS_H 1 Include/cpython/frameobject.h Py_CPYTHON_FRAMEOBJECT_H 1 Include/cpython/import.h Py_CPYTHON_IMPORT_H 1 +Include/cpython/interpreteridobject.h Py_CPYTHON_INTERPRETERIDOBJECT_H 1 Include/cpython/listobject.h Py_CPYTHON_LISTOBJECT_H 1 Include/cpython/methodobject.h Py_CPYTHON_METHODOBJECT_H 1 Include/cpython/object.h Py_CPYTHON_OBJECT_H 1 diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 033bd7a2ba515..80e9d35afd9be 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -54,7 +54,7 @@ Objects/genobject.c - _PyAsyncGenASend_Type - Objects/genobject.c - _PyAsyncGenAThrow_Type - Objects/genobject.c - _PyAsyncGenWrappedValue_Type - Objects/genobject.c - _PyCoroWrapper_Type - -Objects/interpreteridobject.c - _PyInterpreterID_Type - +Objects/interpreteridobject.c - PyInterpreterID_Type - Objects/iterobject.c - PyCallIter_Type - Objects/iterobject.c - PySeqIter_Type - Objects/listobject.c - PyListIter_Type - From webhook-mailer at python.org Thu Jul 27 17:46:06 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 27 Jul 2023 21:46:06 -0000 Subject: [Python-checkins] gh-106931: Fix the WASM Buildbots (gh-107362) Message-ID: https://github.com/python/cpython/commit/2f9bb77764c3b41867f79d6df6e2ed71715dad63 commit: 2f9bb77764c3b41867f79d6df6e2ed71715dad63 branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-27T21:46:02Z summary: gh-106931: Fix the WASM Buildbots (gh-107362) Skip subinterpreter tests when not supported. files: M Lib/test/test_sys.py diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index 78ed4bbaad4eb..9dce15ed1529e 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -14,12 +14,21 @@ from test.support.script_helper import assert_python_ok, assert_python_failure from test.support import threading_helper from test.support import import_helper -from test.support import interpreters +try: + from test.support import interpreters +except ImportError: + interpreters = None import textwrap import unittest import warnings +def requires_subinterpreters(meth): + """Decorator to skip a test if subinterpreters are not supported.""" + return unittest.skipIf(interpreters is None, + 'subinterpreters required')(meth) + + # count the number of test runs, used to create unique # strings to intern in test_intern() INTERN_NUMRUNS = 0 @@ -700,6 +709,7 @@ def __hash__(self): self.assertRaises(TypeError, sys.intern, S("abc")) + @requires_subinterpreters def test_subinterp_intern_dynamically_allocated(self): global INTERN_NUMRUNS INTERN_NUMRUNS += 1 @@ -715,6 +725,7 @@ def test_subinterp_intern_dynamically_allocated(self): assert id(t) != {id(t)}, (id(t), {id(t)}) ''')) + @requires_subinterpreters def test_subinterp_intern_statically_allocated(self): # See Tools/build/generate_global_objects.py for the list # of strings that are always statically allocated. From webhook-mailer at python.org Thu Jul 27 17:51:22 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 27 Jul 2023 21:51:22 -0000 Subject: [Python-checkins] gh-104683: Argument clinic: cleanup `state_modulename_name()` (#107340) Message-ID: https://github.com/python/cpython/commit/c2b1689abcfa80ad959862168b5260fbbd478484 commit: c2b1689abcfa80ad959862168b5260fbbd478484 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-27T22:51:18+01:00 summary: gh-104683: Argument clinic: cleanup `state_modulename_name()` (#107340) files: M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index d1c61b1d401b9..d21c7d84c88d2 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -679,6 +679,32 @@ def test_return_converter(self): """) self.assertIsInstance(function.return_converter, clinic.int_return_converter) + def test_return_converter_invalid_syntax(self): + stdout = self.parse_function_should_fail(""" + module os + os.stat -> invalid syntax + """) + expected_error = "Badly formed annotation for os.stat: 'invalid syntax'" + self.assertIn(expected_error, stdout) + + def test_legacy_converter_disallowed_in_return_annotation(self): + stdout = self.parse_function_should_fail(""" + module os + os.stat -> "s" + """) + expected_error = "Legacy converter 's' not allowed as a return converter" + self.assertIn(expected_error, stdout) + + def test_unknown_return_converter(self): + stdout = self.parse_function_should_fail(""" + module os + os.stat -> foooooooooooooooooooooooo + """) + expected_error = ( + "No available return converter called 'foooooooooooooooooooooooo'" + ) + self.assertIn(expected_error, stdout) + def test_star(self): function = self.parse_function(""" module os diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 5d3ed4167b06c..a343dc5c7fc08 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4730,6 +4730,7 @@ def state_modulename_name(self, line: str | None) -> None: return line, _, returns = line.partition('->') + returns = returns.strip() full_name, _, c_basename = line.partition(' as ') full_name = full_name.strip() @@ -4743,23 +4744,21 @@ def state_modulename_name(self, line: str | None) -> None: return_converter = None if returns: ast_input = f"def x() -> {returns}: pass" - module = None try: - module = ast.parse(ast_input) + module_node = ast.parse(ast_input) except SyntaxError: - pass - if not module: - fail("Badly-formed annotation for " + full_name + ": " + returns) + fail(f"Badly formed annotation for {full_name}: {returns!r}") + function_node = module_node.body[0] + assert isinstance(function_node, ast.FunctionDef) try: - name, legacy, kwargs = self.parse_converter(module.body[0].returns) + name, legacy, kwargs = self.parse_converter(function_node.returns) if legacy: - fail("Legacy converter {!r} not allowed as a return converter" - .format(name)) + fail(f"Legacy converter {name!r} not allowed as a return converter") if name not in return_converters: - fail("No available return converter called " + repr(name)) + fail(f"No available return converter called {name!r}") return_converter = return_converters[name](**kwargs) except ValueError: - fail("Badly-formed annotation for " + full_name + ": " + returns) + fail(f"Badly formed annotation for {full_name}: {returns!r}") fields = [x.strip() for x in full_name.split('.')] function_name = fields.pop() From webhook-mailer at python.org Thu Jul 27 17:51:37 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 27 Jul 2023 21:51:37 -0000 Subject: [Python-checkins] [3.12] gh-104621: Check for Incompatible Extensions in import_find_extension() (gh-107184) (gh-107360) Message-ID: https://github.com/python/cpython/commit/abaf89d908304891c99c6a0450cf7e160c17cdd3 commit: abaf89d908304891c99c6a0450cf7e160c17cdd3 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ericsnowcurrently date: 2023-07-27T21:51:34Z summary: [3.12] gh-104621: Check for Incompatible Extensions in import_find_extension() (gh-107184) (gh-107360) gh-104621: Check for Incompatible Extensions in import_find_extension() (gh-107184) This fixes a bug where incompatible modules could still be imported if attempted multiple times. (cherry picked from commit 75c974f5353685f338344618ad7344e64c2293d0) Co-authored-by: Eric Snow files: A Misc/NEWS.d/next/Core and Builtins/2023-07-24-11-11-41.gh-issue-104621.vM8Y_l.rst M Lib/test/test_capi/check_config.py M Lib/test/test_import/__init__.py M Python/import.c diff --git a/Lib/test/test_capi/check_config.py b/Lib/test/test_capi/check_config.py index aaedd82f39af5..eb99ae16f2b69 100644 --- a/Lib/test/test_capi/check_config.py +++ b/Lib/test/test_capi/check_config.py @@ -12,7 +12,7 @@ def import_singlephase(): try: import _testsinglephase except ImportError: - sys.modules.pop('_testsinglephase') + sys.modules.pop('_testsinglephase', None) return False else: del sys.modules['_testsinglephase'] diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py index ee9fe4d574dd4..cc6e8d4e640cd 100644 --- a/Lib/test/test_import/__init__.py +++ b/Lib/test/test_import/__init__.py @@ -97,7 +97,6 @@ def require_frozen(module, *, skip=True): def require_pure_python(module, *, skip=False): _require_loader(module, SourceFileLoader, skip) - def remove_files(name): for f in (name + ".py", name + ".pyc", @@ -128,19 +127,34 @@ def _ready_to_import(name=None, source=""): del sys.modules[name] -def requires_subinterpreters(meth): - """Decorator to skip a test if subinterpreters are not supported.""" - return unittest.skipIf(_interpreters is None, - 'subinterpreters required')(meth) +if _testsinglephase is not None: + def restore__testsinglephase(*, _orig=_testsinglephase): + # We started with the module imported and want to restore + # it to its nominal state. + _orig._clear_globals() + _testinternalcapi.clear_extension('_testsinglephase', _orig.__file__) + import _testsinglephase def requires_singlephase_init(meth): """Decorator to skip if single-phase init modules are not supported.""" + if not isinstance(meth, type): + def meth(self, _meth=meth): + try: + return _meth(self) + finally: + restore__testsinglephase() meth = cpython_only(meth) return unittest.skipIf(_testsinglephase is None, 'test requires _testsinglephase module')(meth) +def requires_subinterpreters(meth): + """Decorator to skip a test if subinterpreters are not supported.""" + return unittest.skipIf(_interpreters is None, + 'subinterpreters required')(meth) + + class ModuleSnapshot(types.SimpleNamespace): """A representation of a module for testing. @@ -1943,6 +1957,20 @@ def test_isolated_config(self): with self.subTest(f'{module}: strict, fresh'): self.check_compatible_fresh(module, strict=True, isolated=True) + @requires_subinterpreters + @requires_singlephase_init + def test_disallowed_reimport(self): + # See https://github.com/python/cpython/issues/104621. + script = textwrap.dedent(''' + import _testsinglephase + print(_testsinglephase) + ''') + interpid = _interpreters.create() + with self.assertRaises(_interpreters.RunFailedError): + _interpreters.run_string(interpid, script) + with self.assertRaises(_interpreters.RunFailedError): + _interpreters.run_string(interpid, script) + class TestSinglePhaseSnapshot(ModuleSnapshot): @@ -2002,6 +2030,10 @@ def setUpClass(cls): # Start fresh. cls.clean_up() + @classmethod + def tearDownClass(cls): + restore__testsinglephase() + def tearDown(self): # Clean up the module. self.clean_up() diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-24-11-11-41.gh-issue-104621.vM8Y_l.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-24-11-11-41.gh-issue-104621.vM8Y_l.rst new file mode 100644 index 0000000000000..86c976295f262 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-24-11-11-41.gh-issue-104621.vM8Y_l.rst @@ -0,0 +1 @@ +Unsupported modules now always fail to be imported. diff --git a/Python/import.c b/Python/import.c index d10c5ce63a2c8..a93a6450285cc 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1222,6 +1222,15 @@ import_find_extension(PyThreadState *tstate, PyObject *name, return NULL; } + /* It may have been successfully imported previously + in an interpreter that allows legacy modules + but is not allowed in the current interpreter. */ + const char *name_buf = PyUnicode_AsUTF8(name); + assert(name_buf != NULL); + if (_PyImport_CheckSubinterpIncompatibleExtensionAllowed(name_buf) < 0) { + return NULL; + } + PyObject *mod, *mdict; PyObject *modules = MODULES(tstate->interp); @@ -3712,16 +3721,8 @@ _imp_create_dynamic_impl(PyObject *module, PyObject *spec, PyObject *file) PyThreadState *tstate = _PyThreadState_GET(); mod = import_find_extension(tstate, name, path); - if (mod != NULL) { - const char *name_buf = PyUnicode_AsUTF8(name); - assert(name_buf != NULL); - if (_PyImport_CheckSubinterpIncompatibleExtensionAllowed(name_buf) < 0) { - Py_DECREF(mod); - mod = NULL; - } - goto finally; - } - else if (PyErr_Occurred()) { + if (mod != NULL || _PyErr_Occurred(tstate)) { + assert(mod == NULL || !_PyErr_Occurred(tstate)); goto finally; } From webhook-mailer at python.org Thu Jul 27 19:57:59 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Thu, 27 Jul 2023 23:57:59 -0000 Subject: [Python-checkins] gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (#107364) Message-ID: https://github.com/python/cpython/commit/76c26eaca4147ba7e3e8d7379c5a828f0b512a46 commit: 76c26eaca4147ba7e3e8d7379c5a828f0b512a46 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-27T23:57:55Z summary: gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (#107364) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index d21c7d84c88d2..2f74ee29b204d 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1386,7 +1386,7 @@ def _do_test(self, *args, expect_success=True): ) as proc: proc.wait() if expect_success and proc.returncode: - self.fail("".join(proc.stderr)) + self.fail("".join([*proc.stdout, *proc.stderr])) stdout = proc.stdout.read() stderr = proc.stderr.read() # Clinic never writes to stderr. From webhook-mailer at python.org Thu Jul 27 20:36:38 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Fri, 28 Jul 2023 00:36:38 -0000 Subject: [Python-checkins] [3.11] gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (GH-107364) (#107366) Message-ID: https://github.com/python/cpython/commit/51b302e3f322eba7933da87ff1e6dd462c925859 commit: 51b302e3f322eba7933da87ff1e6dd462c925859 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-28T00:36:34Z summary: [3.11] gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (GH-107364) (#107366) gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (GH-107364) (cherry picked from commit 76c26eaca4147ba7e3e8d7379c5a828f0b512a46) Co-authored-by: Alex Waygood files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 32c4b1e67f957..1c01fb0ed3688 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1306,7 +1306,7 @@ def _do_test(self, *args, expect_success=True): ) as proc: proc.wait() if expect_success and proc.returncode: - self.fail("".join(proc.stderr)) + self.fail("".join([*proc.stdout, *proc.stderr])) stdout = proc.stdout.read() stderr = proc.stderr.read() # Clinic never writes to stderr. From webhook-mailer at python.org Thu Jul 27 20:39:55 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Fri, 28 Jul 2023 00:39:55 -0000 Subject: [Python-checkins] [3.12] gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (GH-107364) (#107365) Message-ID: https://github.com/python/cpython/commit/2827ad2f317fceaa618bbdb5900b76613af3aa69 commit: 2827ad2f317fceaa618bbdb5900b76613af3aa69 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-28T00:39:51Z summary: [3.12] gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (GH-107364) (#107365) gh-106368: Argument clinic tests: improve failure message when tests in `ClinicExternalTests` fail (GH-107364) (cherry picked from commit 76c26eaca4147ba7e3e8d7379c5a828f0b512a46) Co-authored-by: Alex Waygood files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index d1c61b1d401b9..3d740910a57ae 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -1360,7 +1360,7 @@ def _do_test(self, *args, expect_success=True): ) as proc: proc.wait() if expect_success and proc.returncode: - self.fail("".join(proc.stderr)) + self.fail("".join([*proc.stdout, *proc.stderr])) stdout = proc.stdout.read() stderr = proc.stderr.read() # Clinic never writes to stderr. From webhook-mailer at python.org Fri Jul 28 01:04:29 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 28 Jul 2023 05:04:29 -0000 Subject: [Python-checkins] gh-106723: forward -Xfrozen_modules option to spawned process interpreters (#106724) Message-ID: https://github.com/python/cpython/commit/3dcac785810df4d9db50abe90847eaf03bbdaaf4 commit: 3dcac785810df4d9db50abe90847eaf03bbdaaf4 branch: main author: Felipe A. Hernandez committer: gpshead date: 2023-07-28T05:04:26Z summary: gh-106723: forward -Xfrozen_modules option to spawned process interpreters (#106724) Co-authored-by: Kumar Aditya Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst M Lib/subprocess.py diff --git a/Lib/subprocess.py b/Lib/subprocess.py index fbc76b8d0f14b..6df5dd551ea67 100644 --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -346,7 +346,7 @@ def _args_from_interpreter_flags(): if dev_mode: args.extend(('-X', 'dev')) for opt in ('faulthandler', 'tracemalloc', 'importtime', - 'showrefcount', 'utf8'): + 'frozen_modules', 'showrefcount', 'utf8'): if opt in xoptions: value = xoptions[opt] if value is True: diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst new file mode 100644 index 0000000000000..207f397f17d3f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst @@ -0,0 +1 @@ +Propagate ``frozen_modules`` to multiprocessing spawned process interpreters. From webhook-mailer at python.org Fri Jul 28 01:50:59 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Fri, 28 Jul 2023 05:50:59 -0000 Subject: [Python-checkins] [3.11] gh-106723: forward -Xfrozen_modules option to spawned process interpreters (GH-106724) (#107368) Message-ID: https://github.com/python/cpython/commit/8492aae9bd60174386a374563594a2ba6512f1f6 commit: 8492aae9bd60174386a374563594a2ba6512f1f6 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-28T11:20:56+05:30 summary: [3.11] gh-106723: forward -Xfrozen_modules option to spawned process interpreters (GH-106724) (#107368) gh-106723: forward -Xfrozen_modules option to spawned process interpreters (GH-106724) (cherry picked from commit 3dcac785810df4d9db50abe90847eaf03bbdaaf4) Co-authored-by: Felipe A. Hernandez Co-authored-by: Kumar Aditya Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst M Lib/subprocess.py diff --git a/Lib/subprocess.py b/Lib/subprocess.py index fbc76b8d0f14b..6df5dd551ea67 100644 --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -346,7 +346,7 @@ def _args_from_interpreter_flags(): if dev_mode: args.extend(('-X', 'dev')) for opt in ('faulthandler', 'tracemalloc', 'importtime', - 'showrefcount', 'utf8'): + 'frozen_modules', 'showrefcount', 'utf8'): if opt in xoptions: value = xoptions[opt] if value is True: diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst new file mode 100644 index 0000000000000..207f397f17d3f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst @@ -0,0 +1 @@ +Propagate ``frozen_modules`` to multiprocessing spawned process interpreters. From webhook-mailer at python.org Fri Jul 28 01:58:18 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Fri, 28 Jul 2023 05:58:18 -0000 Subject: [Python-checkins] gh-106078: Isolate `decimal` module (#107287) Message-ID: https://github.com/python/cpython/commit/a43cc3fa1ffebfd15eff2c6d8e5433a17969e348 commit: a43cc3fa1ffebfd15eff2c6d8e5433a17969e348 branch: main author: Charlie Zhao committer: kumaraditya303 date: 2023-07-28T11:28:14+05:30 summary: gh-106078: Isolate `decimal` module (#107287) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-18-04.gh-issue-106078.WEy2Yn.rst M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/globals-to-fix.tsv diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-18-04.gh-issue-106078.WEy2Yn.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-18-04.gh-issue-106078.WEy2Yn.rst new file mode 100644 index 0000000000000..f5a0e539e5d05 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-18-04.gh-issue-106078.WEy2Yn.rst @@ -0,0 +1 @@ +Isolate :mod:`!_decimal` (apply :pep:`687`). Patch by Charlie Zhao. \ No newline at end of file diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 671976a220a64..7a206973975cc 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -95,9 +95,36 @@ typedef struct { PyCFunction _py_float_as_integer_ratio; } decimal_state; -static decimal_state global_state; +static inline decimal_state * +get_module_state(PyObject *mod) +{ + decimal_state *state = _PyModule_GetState(mod); + assert(state != NULL); + return state; +} + +static struct PyModuleDef _decimal_module; + +static inline decimal_state * +get_module_state_by_def(PyTypeObject *tp) +{ + PyObject *mod = PyType_GetModuleByDef(tp, &_decimal_module); + assert(mod != NULL); + return get_module_state(mod); +} + +static inline decimal_state * +find_state_left_or_right(PyObject *left, PyObject *right) +{ + PyObject *mod = PyType_GetModuleByDef(Py_TYPE(left), &_decimal_module); + if (mod == NULL) { + PyErr_Clear(); + mod = PyType_GetModuleByDef(Py_TYPE(right), &_decimal_module); + } + assert(mod != NULL); + return get_module_state(mod); +} -#define GLOBAL_STATE() (&global_state) #if !defined(MPD_VERSION_HEX) || MPD_VERSION_HEX < 0x02050000 #error "libmpdec version >= 2.5.0 required" @@ -328,10 +355,9 @@ dec_traphandler(mpd_context_t *ctx UNUSED) /* GCOV_NOT_REACHED */ } static PyObject * -flags_as_exception(uint32_t flags) +flags_as_exception(decimal_state *state, uint32_t flags) { DecCondMap *cm; - decimal_state *state = GLOBAL_STATE(); for (cm = state->signal_map; cm->name != NULL; cm++) { if (flags&cm->flag) { @@ -343,10 +369,9 @@ flags_as_exception(uint32_t flags) } Py_LOCAL_INLINE(uint32_t) -exception_as_flag(PyObject *ex) +exception_as_flag(decimal_state *state, PyObject *ex) { DecCondMap *cm; - decimal_state *state = GLOBAL_STATE(); for (cm = state->signal_map; cm->name != NULL; cm++) { if (cm->ex == ex) { @@ -359,11 +384,10 @@ exception_as_flag(PyObject *ex) } static PyObject * -flags_as_list(uint32_t flags) +flags_as_list(decimal_state *state, uint32_t flags) { PyObject *list; DecCondMap *cm; - decimal_state *state = GLOBAL_STATE(); list = PyList_New(0); if (list == NULL) { @@ -393,11 +417,10 @@ flags_as_list(uint32_t flags) } static PyObject * -signals_as_list(uint32_t flags) +signals_as_list(decimal_state *state, uint32_t flags) { PyObject *list; DecCondMap *cm; - decimal_state *state = GLOBAL_STATE(); list = PyList_New(0); if (list == NULL) { @@ -417,7 +440,7 @@ signals_as_list(uint32_t flags) } static uint32_t -list_as_flags(PyObject *list) +list_as_flags(decimal_state *state, PyObject *list) { PyObject *item; uint32_t flags, x; @@ -429,7 +452,7 @@ list_as_flags(PyObject *list) flags = 0; for (j = 0; j < n; j++) { item = PyList_GetItem(list, j); - x = exception_as_flag(item); + x = exception_as_flag(state, item); if (x & DEC_ERRORS) { return x; } @@ -440,11 +463,10 @@ list_as_flags(PyObject *list) } static PyObject * -flags_as_dict(uint32_t flags) +flags_as_dict(decimal_state *state, uint32_t flags) { DecCondMap *cm; PyObject *dict; - decimal_state *state = GLOBAL_STATE(); dict = PyDict_New(); if (dict == NULL) { @@ -463,13 +485,12 @@ flags_as_dict(uint32_t flags) } static uint32_t -dict_as_flags(PyObject *val) +dict_as_flags(decimal_state *state, PyObject *val) { PyObject *b; DecCondMap *cm; uint32_t flags = 0; int x; - decimal_state *state = GLOBAL_STATE(); if (!PyDict_Check(val)) { PyErr_SetString(PyExc_TypeError, @@ -529,6 +550,7 @@ static int dec_addstatus(PyObject *context, uint32_t status) { mpd_context_t *ctx = CTX(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); ctx->status |= status; if (status & (ctx->traps|MPD_Malloc_error)) { @@ -539,11 +561,11 @@ dec_addstatus(PyObject *context, uint32_t status) return 1; } - ex = flags_as_exception(ctx->traps&status); + ex = flags_as_exception(state, ctx->traps&status); if (ex == NULL) { return 1; /* GCOV_NOT_REACHED */ } - siglist = flags_as_list(ctx->traps&status); + siglist = flags_as_list(state, ctx->traps&status); if (siglist == NULL) { return 1; } @@ -556,11 +578,9 @@ dec_addstatus(PyObject *context, uint32_t status) } static int -getround(PyObject *v) +getround(decimal_state *state, PyObject *v) { int i; - decimal_state *state = GLOBAL_STATE(); - if (PyUnicode_Check(v)) { for (i = 0; i < _PY_DEC_ROUND_GUARD; i++) { if (v == state->round_map[i]) { @@ -602,9 +622,9 @@ signaldict_len(PyObject *self UNUSED) } static PyObject * -signaldict_iter(PyObject *self UNUSED) +signaldict_iter(PyObject *self) { - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); return PyTuple_Type.tp_iter(state->SignalTuple); } @@ -612,8 +632,9 @@ static PyObject * signaldict_getitem(PyObject *self, PyObject *key) { uint32_t flag; + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); - flag = exception_as_flag(key); + flag = exception_as_flag(state, key); if (flag & DEC_ERRORS) { return NULL; } @@ -627,11 +648,12 @@ signaldict_setitem(PyObject *self, PyObject *key, PyObject *value) uint32_t flag; int x; + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); if (value == NULL) { return value_error_int("signal keys cannot be deleted"); } - flag = exception_as_flag(key); + flag = exception_as_flag(state, key); if (flag & DEC_ERRORS) { return -1; } @@ -677,7 +699,7 @@ signaldict_repr(PyObject *self) assert(SIGNAL_MAP_LEN == 9); - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); for (cm=state->signal_map, i=0; cm->name != NULL; cm++, i++) { n[i] = cm->fqname; b[i] = SdFlags(self)&cm->flag ? "True" : "False"; @@ -696,7 +718,7 @@ signaldict_richcompare(PyObject *v, PyObject *w, int op) { PyObject *res = Py_NotImplemented; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = find_state_left_or_right(v, w); assert(PyDecSignalDict_Check(state, v)); if (op == Py_EQ || op == Py_NE) { @@ -704,7 +726,7 @@ signaldict_richcompare(PyObject *v, PyObject *w, int op) res = (SdFlags(v)==SdFlags(w)) ^ (op==Py_NE) ? Py_True : Py_False; } else if (PyDict_Check(w)) { - uint32_t flags = dict_as_flags(w); + uint32_t flags = dict_as_flags(state, w); if (flags & DEC_ERRORS) { if (flags & DEC_INVALID_SIGNALS) { /* non-comparable: Py_NotImplemented */ @@ -726,7 +748,8 @@ signaldict_richcompare(PyObject *v, PyObject *w, int op) static PyObject * signaldict_copy(PyObject *self, PyObject *args UNUSED) { - return flags_as_dict(SdFlags(self)); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); + return flags_as_dict(state, SdFlags(self)); } @@ -795,7 +818,7 @@ static PyObject * context_getround(PyObject *self, void *closure UNUSED) { int i = mpd_getround(CTX(self)); - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); return Py_NewRef(state->round_map[i]); } @@ -954,7 +977,8 @@ context_setround(PyObject *self, PyObject *value, void *closure UNUSED) mpd_context_t *ctx; int x; - x = getround(value); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); + x = getround(state, value); if (x == -1) { return -1; } @@ -1012,8 +1036,8 @@ context_settraps_list(PyObject *self, PyObject *value) { mpd_context_t *ctx; uint32_t flags; - - flags = list_as_flags(value); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); + flags = list_as_flags(state, value); if (flags & DEC_ERRORS) { return -1; } @@ -1032,12 +1056,12 @@ context_settraps_dict(PyObject *self, PyObject *value) mpd_context_t *ctx; uint32_t flags; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); if (PyDecSignalDict_Check(state, value)) { flags = SdFlags(value); } else { - flags = dict_as_flags(value); + flags = dict_as_flags(state, value); if (flags & DEC_ERRORS) { return -1; } @@ -1077,8 +1101,9 @@ context_setstatus_list(PyObject *self, PyObject *value) { mpd_context_t *ctx; uint32_t flags; + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); - flags = list_as_flags(value); + flags = list_as_flags(state, value); if (flags & DEC_ERRORS) { return -1; } @@ -1097,12 +1122,12 @@ context_setstatus_dict(PyObject *self, PyObject *value) mpd_context_t *ctx; uint32_t flags; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); if (PyDecSignalDict_Check(state, value)) { flags = SdFlags(value); } else { - flags = dict_as_flags(value); + flags = dict_as_flags(state, value); if (flags & DEC_ERRORS) { return -1; } @@ -1288,7 +1313,7 @@ context_new(PyTypeObject *type, PyObject *args UNUSED, PyObject *kwds UNUSED) PyDecContextObject *self = NULL; mpd_context_t *ctx; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(type); if (type == state->PyDecContext_Type) { self = PyObject_GC_New(PyDecContextObject, state->PyDecContext_Type); } @@ -1353,7 +1378,7 @@ context_dealloc(PyDecContextObject *self) PyTypeObject *tp = Py_TYPE(self); PyObject_GC_UnTrack(self); #ifndef WITH_DECIMAL_CONTEXTVAR - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); if (self == state->cached_context) { state->cached_context = NULL; } @@ -1405,7 +1430,7 @@ context_repr(PyDecContextObject *self) int n, mem; #ifdef Py_DEBUG - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); assert(PyDecContext_Check(state, self)); #endif ctx = CTX(self); @@ -1473,7 +1498,7 @@ ieee_context(PyObject *dummy UNUSED, PyObject *v) goto error; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(v)); context = PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL); if (context == NULL) { return NULL; @@ -1496,7 +1521,7 @@ context_copy(PyObject *self, PyObject *args UNUSED) { PyObject *copy; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); copy = PyObject_CallObject((PyObject *)state->PyDecContext_Type, NULL); if (copy == NULL) { return NULL; @@ -1516,14 +1541,15 @@ context_reduce(PyObject *self, PyObject *args UNUSED) PyObject *traps; PyObject *ret; mpd_context_t *ctx; + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); ctx = CTX(self); - flags = signals_as_list(ctx->status); + flags = signals_as_list(state, ctx->status); if (flags == NULL) { return NULL; } - traps = signals_as_list(ctx->traps); + traps = signals_as_list(state, ctx->traps); if (traps == NULL) { Py_DECREF(flags); return NULL; @@ -1568,7 +1594,7 @@ static PyGetSetDef context_getsets [] = #define CONTEXT_CHECK_VA(state, obj) \ if (obj == Py_None) { \ - CURRENT_CONTEXT(obj); \ + CURRENT_CONTEXT(state, obj); \ } \ else if (!PyDecContext_Check(state, obj)) { \ PyErr_SetString(PyExc_TypeError, \ @@ -1591,10 +1617,9 @@ static PyGetSetDef context_getsets [] = #ifndef WITH_DECIMAL_CONTEXTVAR /* Get the context from the thread state dictionary. */ static PyObject * -current_context_from_dict(void) +current_context_from_dict(decimal_state *modstate) { PyThreadState *tstate = _PyThreadState_GET(); - decimal_state *modstate = GLOBAL_STATE(); #ifdef Py_DEBUG // The caller must hold the GIL _Py_EnsureTstateNotNULL(tstate); @@ -1643,45 +1668,41 @@ current_context_from_dict(void) /* Return borrowed reference to thread local context. */ static PyObject * -current_context(void) +current_context(decimal_state *modstate) { PyThreadState *tstate = _PyThreadState_GET(); - decimal_state *modstate = GLOBAL_STATE(); if (modstate->cached_context && modstate->cached_context->tstate == tstate) { return (PyObject *)(modstate->cached_context); } - return current_context_from_dict(); + return current_context_from_dict(modstate); } /* ctxobj := borrowed reference to the current context */ -#define CURRENT_CONTEXT(ctxobj) \ - ctxobj = current_context(); \ +#define CURRENT_CONTEXT(state, ctxobj) \ + ctxobj = current_context(state); \ if (ctxobj == NULL) { \ return NULL; \ } /* Return a new reference to the current context */ static PyObject * -PyDec_GetCurrentContext(PyObject *self UNUSED, PyObject *args UNUSED) +PyDec_GetCurrentContext(PyObject *self, PyObject *args UNUSED) { PyObject *context; + decimal_state *state = get_module_state(self); - context = current_context(); - if (context == NULL) { - return NULL; - } - + CURRENT_CONTEXT(state, context); return Py_NewRef(context); } /* Set the thread local context to a new context, decrement old reference */ static PyObject * -PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) +PyDec_SetCurrentContext(PyObject *self, PyObject *v) { PyObject *dict; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state(self); CONTEXT_CHECK(state, v); dict = PyThreadState_GetDict(); @@ -1717,9 +1738,8 @@ PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) } #else static PyObject * -init_current_context(void) +init_current_context(decimal_state *state) { - decimal_state *state = GLOBAL_STATE(); PyObject *tl_context = context_copy(state->default_context_template, NULL); if (tl_context == NULL) { return NULL; @@ -1737,10 +1757,9 @@ init_current_context(void) } static inline PyObject * -current_context(void) +current_context(decimal_state *state) { PyObject *tl_context; - decimal_state *state = GLOBAL_STATE(); if (PyContextVar_Get(state->current_context_var, NULL, &tl_context) < 0) { return NULL; } @@ -1749,12 +1768,12 @@ current_context(void) return tl_context; } - return init_current_context(); + return init_current_context(state); } /* ctxobj := borrowed reference to the current context */ -#define CURRENT_CONTEXT(ctxobj) \ - ctxobj = current_context(); \ +#define CURRENT_CONTEXT(state, ctxobj) \ + ctxobj = current_context(state); \ if (ctxobj == NULL) { \ return NULL; \ } \ @@ -1762,16 +1781,17 @@ current_context(void) /* Return a new reference to the current context */ static PyObject * -PyDec_GetCurrentContext(PyObject *self UNUSED, PyObject *args UNUSED) +PyDec_GetCurrentContext(PyObject *self, PyObject *args UNUSED) { - return current_context(); + decimal_state *state = get_module_state(self); + return current_context(state); } /* Set the thread local context to a new context, decrement old reference */ static PyObject * -PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) +PyDec_SetCurrentContext(PyObject *self, PyObject *v) { - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state(self); CONTEXT_CHECK(state, v); /* If the new context is one of the templates, make a copy. @@ -1804,7 +1824,7 @@ PyDec_SetCurrentContext(PyObject *self UNUSED, PyObject *v) * owns one reference to the global (outer) context and one * to the local (inner) context. */ static PyObject * -ctxmanager_new(PyTypeObject *type UNUSED, PyObject *args, PyObject *kwds) +ctxmanager_new(PyObject *m, PyObject *args, PyObject *kwds) { static char *kwlist[] = { "ctx", "prec", "rounding", @@ -1824,8 +1844,8 @@ ctxmanager_new(PyTypeObject *type UNUSED, PyObject *args, PyObject *kwds) PyObject *flags = Py_None; PyObject *traps = Py_None; - decimal_state *state = GLOBAL_STATE(); - CURRENT_CONTEXT(global); + decimal_state *state = get_module_state(m); + CURRENT_CONTEXT(state, global); if (!PyArg_ParseTupleAndKeywords(args, kwds, "|OOOOOOOOO", kwlist, &local, &prec, &rounding, &Emin, &Emax, &capitals, &clamp, &flags, &traps)) { return NULL; @@ -1902,7 +1922,7 @@ ctxmanager_set_local(PyDecContextManagerObject *self, PyObject *args UNUSED) { PyObject *ret; - ret = PyDec_SetCurrentContext(NULL, self->local); + ret = PyDec_SetCurrentContext(PyType_GetModule(Py_TYPE(self)), self->local); if (ret == NULL) { return NULL; } @@ -1917,7 +1937,7 @@ ctxmanager_restore_global(PyDecContextManagerObject *self, { PyObject *ret; - ret = PyDec_SetCurrentContext(NULL, self->global); + ret = PyDec_SetCurrentContext(PyType_GetModule(Py_TYPE(self)), self->global); if (ret == NULL) { return NULL; } @@ -1960,7 +1980,7 @@ PyDecType_New(PyTypeObject *type) { PyDecObject *dec; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(type); if (type == state->PyDec_Type) { dec = PyObject_GC_New(PyDecObject, state->PyDec_Type); } @@ -2347,8 +2367,8 @@ PyDecType_FromFloatExact(PyTypeObject *type, PyObject *v, mpd_t *d1, *d2; uint32_t status = 0; mpd_context_t maxctx; + decimal_state *state = get_module_state_by_def(type); - decimal_state *state = GLOBAL_STATE(); #ifdef Py_DEBUG assert(PyType_IsSubtype(type, state->PyDec_Type)); #endif @@ -2485,7 +2505,7 @@ PyDecType_FromDecimalExact(PyTypeObject *type, PyObject *v, PyObject *context) PyObject *dec; uint32_t status = 0; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(type); if (type == state->PyDec_Type && PyDec_CheckExact(state, v)) { return Py_NewRef(v); } @@ -2760,8 +2780,8 @@ dec_from_float(PyObject *type, PyObject *pyfloat) PyObject *context; PyObject *result; - CURRENT_CONTEXT(context); - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def((PyTypeObject *)type); + CURRENT_CONTEXT(state, context); result = PyDecType_FromFloatExact(state->PyDec_Type, pyfloat, context); if (type != (PyObject *)state->PyDec_Type && result != NULL) { Py_SETREF(result, PyObject_CallFunctionObjArgs(type, result, NULL)); @@ -2774,7 +2794,7 @@ dec_from_float(PyObject *type, PyObject *pyfloat) static PyObject * ctx_from_float(PyObject *context, PyObject *v) { - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); return PyDec_FromFloat(state, v, context); } @@ -2785,7 +2805,7 @@ dec_apply(PyObject *v, PyObject *context) PyObject *result; uint32_t status = 0; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); result = dec_alloc(state); if (result == NULL) { return NULL; @@ -2812,7 +2832,7 @@ dec_apply(PyObject *v, PyObject *context) static PyObject * PyDecType_FromObjectExact(PyTypeObject *type, PyObject *v, PyObject *context) { - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(type); if (v == NULL) { return PyDecType_FromSsizeExact(type, 0, context); } @@ -2847,7 +2867,7 @@ PyDecType_FromObjectExact(PyTypeObject *type, PyObject *v, PyObject *context) static PyObject * PyDec_FromObject(PyObject *v, PyObject *context) { - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); if (v == NULL) { return PyDec_FromSsize(state, 0, context); } @@ -2903,7 +2923,7 @@ dec_new(PyTypeObject *type, PyObject *args, PyObject *kwds) &v, &context)) { return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(type); CONTEXT_CHECK_VA(state, context); return PyDecType_FromObjectExact(type, v, context); @@ -2934,7 +2954,7 @@ ctx_create_decimal(PyObject *context, PyObject *args) Py_LOCAL_INLINE(int) convert_op(int type_err, PyObject **conv, PyObject *v, PyObject *context) { - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); if (PyDec_Check(state, v)) { *conv = Py_NewRef(v); return 1; @@ -3037,7 +3057,7 @@ multiply_by_denominator(PyObject *v, PyObject *r, PyObject *context) if (tmp == NULL) { return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); denom = PyDec_FromLongExact(state, tmp, context); Py_DECREF(tmp); if (denom == NULL) { @@ -3092,7 +3112,7 @@ numerator_as_decimal(PyObject *r, PyObject *context) return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); num = PyDec_FromLongExact(state, tmp, context); Py_DECREF(tmp); return num; @@ -3111,7 +3131,7 @@ convert_op_cmp(PyObject **vcmp, PyObject **wcmp, PyObject *v, PyObject *w, *vcmp = v; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); if (PyDec_Check(state, w)) { *wcmp = Py_NewRef(w); } @@ -3209,7 +3229,8 @@ dec_str(PyObject *dec) mpd_ssize_t size; char *cp; - CURRENT_CONTEXT(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(dec)); + CURRENT_CONTEXT(state, context); size = mpd_to_sci_size(&cp, MPD(dec), CtxCaps(context)); if (size < 0) { PyErr_NoMemory(); @@ -3227,8 +3248,8 @@ dec_repr(PyObject *dec) { PyObject *res, *context; char *cp; - - CURRENT_CONTEXT(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(dec)); + CURRENT_CONTEXT(state, context); cp = mpd_to_sci(MPD(dec), CtxCaps(context)); if (cp == NULL) { PyErr_NoMemory(); @@ -3389,7 +3410,8 @@ dec_format(PyObject *dec, PyObject *args) mpd_t tmp = {MPD_STATIC|MPD_STATIC_DATA,0,0,0,MPD_MINALLOC_MAX,dt}; - CURRENT_CONTEXT(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(dec)); + CURRENT_CONTEXT(state, context); if (!PyArg_ParseTuple(args, "O|O", &fmtarg, &override)) { return NULL; } @@ -3652,9 +3674,9 @@ dec_as_integer_ratio(PyObject *self, PyObject *args UNUSED) return NULL; } - CURRENT_CONTEXT(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); + CURRENT_CONTEXT(state, context); - decimal_state *state = GLOBAL_STATE(); tmp = dec_alloc(state); if (tmp == NULL) { return NULL; @@ -3745,12 +3767,12 @@ PyDec_ToIntegralValue(PyObject *dec, PyObject *args, PyObject *kwds) &rounding, &context)) { return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(dec)); CONTEXT_CHECK_VA(state, context); workctx = *CTX(context); if (rounding != Py_None) { - int round = getround(rounding); + int round = getround(state, rounding); if (round < 0) { return NULL; } @@ -3787,12 +3809,12 @@ PyDec_ToIntegralExact(PyObject *dec, PyObject *args, PyObject *kwds) &rounding, &context)) { return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(dec)); CONTEXT_CHECK_VA(state, context); workctx = *CTX(context); if (rounding != Py_None) { - int round = getround(rounding); + int round = getround(state, rounding); if (round < 0) { return NULL; } @@ -3855,7 +3877,8 @@ PyDec_Round(PyObject *dec, PyObject *args) uint32_t status = 0; PyObject *context; - CURRENT_CONTEXT(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(dec)); + CURRENT_CONTEXT(state, context); if (!PyArg_ParseTuple(args, "|O", &x)) { return NULL; } @@ -3875,7 +3898,6 @@ PyDec_Round(PyObject *dec, PyObject *args) if (y == -1 && PyErr_Occurred()) { return NULL; } - decimal_state *state = GLOBAL_STATE(); result = dec_alloc(state); if (result == NULL) { return NULL; @@ -3977,7 +3999,7 @@ PyDec_AsTuple(PyObject *dec, PyObject *dummy UNUSED) } } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(dec)); result = PyObject_CallFunctionObjArgs((PyObject *)state->DecimalTuple, sign, coeff, expt, NULL); @@ -4004,8 +4026,8 @@ nm_##MPDFUNC(PyObject *self) \ PyObject *context; \ uint32_t status = 0; \ \ - decimal_state *state = GLOBAL_STATE(); \ - CURRENT_CONTEXT(context); \ + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); \ + CURRENT_CONTEXT(state, context); \ if ((result = dec_alloc(state)) == NULL) { \ return NULL; \ } \ @@ -4029,8 +4051,8 @@ nm_##MPDFUNC(PyObject *self, PyObject *other) \ PyObject *context; \ uint32_t status = 0; \ \ - decimal_state *state = GLOBAL_STATE(); \ - CURRENT_CONTEXT(context) ; \ + decimal_state *state = find_state_left_or_right(self, other); \ + CURRENT_CONTEXT(state, context) ; \ CONVERT_BINOP(&a, &b, self, other, context); \ \ if ((result = dec_alloc(state)) == NULL) { \ @@ -4070,7 +4092,7 @@ dec_##MPDFUNC(PyObject *self, PyObject *args, PyObject *kwds) \ &context)) { \ return NULL; \ } \ - decimal_state *state = GLOBAL_STATE(); \ + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); \ CONTEXT_CHECK_VA(state, context); \ \ return MPDFUNC(MPD(self), CTX(context)) ? incr_true() : incr_false(); \ @@ -4090,7 +4112,8 @@ dec_##MPDFUNC(PyObject *self, PyObject *args, PyObject *kwds) \ &context)) { \ return NULL; \ } \ - decimal_state *state = GLOBAL_STATE(); \ + decimal_state *state = \ + get_module_state_by_def(Py_TYPE(self)); \ CONTEXT_CHECK_VA(state, context); \ \ if ((result = dec_alloc(state)) == NULL) { \ @@ -4122,7 +4145,8 @@ dec_##MPDFUNC(PyObject *self, PyObject *args, PyObject *kwds) \ &other, &context)) { \ return NULL; \ } \ - decimal_state *state = GLOBAL_STATE(); \ + decimal_state *state = \ + get_module_state_by_def(Py_TYPE(self)); \ CONTEXT_CHECK_VA(state, context); \ CONVERT_BINOP_RAISE(&a, &b, self, other, context); \ \ @@ -4160,7 +4184,8 @@ dec_##MPDFUNC(PyObject *self, PyObject *args, PyObject *kwds) \ &other, &context)) { \ return NULL; \ } \ - decimal_state *state = GLOBAL_STATE(); \ + decimal_state *state = \ + get_module_state_by_def(Py_TYPE(self)); \ CONTEXT_CHECK_VA(state, context); \ CONVERT_BINOP_RAISE(&a, &b, self, other, context); \ \ @@ -4193,7 +4218,7 @@ dec_##MPDFUNC(PyObject *self, PyObject *args, PyObject *kwds) \ &other, &third, &context)) { \ return NULL; \ } \ - decimal_state *state = GLOBAL_STATE(); \ + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); \ CONTEXT_CHECK_VA(state, context); \ CONVERT_TERNOP_RAISE(&a, &b, &c, self, other, third, context); \ \ @@ -4236,8 +4261,8 @@ static PyObject * nm_dec_as_long(PyObject *dec) { PyObject *context; - - CURRENT_CONTEXT(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(dec)); + CURRENT_CONTEXT(state, context); return dec_as_long(dec, context, MPD_ROUND_DOWN); } @@ -4256,10 +4281,10 @@ nm_mpd_qdivmod(PyObject *v, PyObject *w) uint32_t status = 0; PyObject *ret; - CURRENT_CONTEXT(context); + decimal_state *state = find_state_left_or_right(v, w); + CURRENT_CONTEXT(state, context); CONVERT_BINOP(&a, &b, v, w, context); - decimal_state *state = GLOBAL_STATE(); q = dec_alloc(state); if (q == NULL) { Py_DECREF(a); @@ -4297,7 +4322,8 @@ nm_mpd_qpow(PyObject *base, PyObject *exp, PyObject *mod) PyObject *context; uint32_t status = 0; - CURRENT_CONTEXT(context); + decimal_state *state = find_state_left_or_right(base, exp); + CURRENT_CONTEXT(state, context); CONVERT_BINOP(&a, &b, base, exp, context); if (mod != Py_None) { @@ -4308,7 +4334,6 @@ nm_mpd_qpow(PyObject *base, PyObject *exp, PyObject *mod) } } - decimal_state *state = GLOBAL_STATE(); result = dec_alloc(state); if (result == NULL) { Py_DECREF(a); @@ -4406,11 +4431,11 @@ dec_conjugate(PyObject *self, PyObject *dummy UNUSED) } static PyObject * -dec_mpd_radix(PyObject *self UNUSED, PyObject *dummy UNUSED) +dec_mpd_radix(PyObject *self, PyObject *dummy UNUSED) { PyObject *result; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); result = dec_alloc(state); if (result == NULL) { return NULL; @@ -4426,7 +4451,7 @@ dec_mpd_qcopy_abs(PyObject *self, PyObject *dummy UNUSED) PyObject *result; uint32_t status = 0; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); if ((result = dec_alloc(state)) == NULL) { return NULL; } @@ -4447,7 +4472,7 @@ dec_mpd_qcopy_negate(PyObject *self, PyObject *dummy UNUSED) PyObject *result; uint32_t status = 0; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); if ((result = dec_alloc(state)) == NULL) { return NULL; } @@ -4477,7 +4502,7 @@ dec_mpd_class(PyObject *self, PyObject *args, PyObject *kwds) &context)) { return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); CONTEXT_CHECK_VA(state, context); cp = mpd_class(MPD(self), CTX(context)); @@ -4497,7 +4522,7 @@ dec_mpd_to_eng(PyObject *self, PyObject *args, PyObject *kwds) &context)) { return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); CONTEXT_CHECK_VA(state, context); size = mpd_to_eng_size(&s, MPD(self), CtxCaps(context)); @@ -4530,7 +4555,7 @@ dec_mpd_qcopy_sign(PyObject *self, PyObject *args, PyObject *kwds) &other, &context)) { return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); CONTEXT_CHECK_VA(state, context); CONVERT_BINOP_RAISE(&a, &b, self, other, context); @@ -4565,7 +4590,7 @@ dec_mpd_same_quantum(PyObject *self, PyObject *args, PyObject *kwds) &other, &context)) { return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); CONTEXT_CHECK_VA(state, context); CONVERT_BINOP_RAISE(&a, &b, self, other, context); @@ -4600,12 +4625,12 @@ dec_mpd_qquantize(PyObject *v, PyObject *args, PyObject *kwds) &w, &rounding, &context)) { return NULL; } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(v)); CONTEXT_CHECK_VA(state, context); workctx = *CTX(context); if (rounding != Py_None) { - int round = getround(rounding); + int round = getround(state, rounding); if (round < 0) { return NULL; } @@ -4644,12 +4669,12 @@ dec_richcompare(PyObject *v, PyObject *w, int op) uint32_t status = 0; int a_issnan, b_issnan; int r; + decimal_state *state = find_state_left_or_right(v, w); #ifdef Py_DEBUG - decimal_state *state = GLOBAL_STATE(); assert(PyDec_Check(state, v)); #endif - CURRENT_CONTEXT(context); + CURRENT_CONTEXT(state, context); CONVERT_BINOP_CMP(&a, &b, v, w, op, context); a_issnan = mpd_issnan(MPD(a)); @@ -4700,7 +4725,8 @@ dec_ceil(PyObject *self, PyObject *dummy UNUSED) { PyObject *context; - CURRENT_CONTEXT(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); + CURRENT_CONTEXT(state, context); return dec_as_long(self, context, MPD_ROUND_CEILING); } @@ -4738,7 +4764,8 @@ dec_floor(PyObject *self, PyObject *dummy UNUSED) { PyObject *context; - CURRENT_CONTEXT(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); + CURRENT_CONTEXT(state, context); return dec_as_long(self, context, MPD_ROUND_FLOOR); } @@ -4902,7 +4929,8 @@ dec_trunc(PyObject *self, PyObject *dummy UNUSED) { PyObject *context; - CURRENT_CONTEXT(context); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); + CURRENT_CONTEXT(state, context); return dec_as_long(self, context, MPD_ROUND_DOWN); } @@ -4918,7 +4946,7 @@ dec_imag(PyObject *self UNUSED, void *closure UNUSED) { PyObject *result; - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); result = dec_alloc(state); if (result == NULL) { return NULL; @@ -5117,7 +5145,8 @@ ctx_##MPDFUNC(PyObject *context, PyObject *v) \ uint32_t status = 0; \ \ CONVERT_OP_RAISE(&a, v, context); \ - decimal_state *state = GLOBAL_STATE(); \ + decimal_state *state = \ + get_module_state_by_def(Py_TYPE(context)); \ if ((result = dec_alloc(state)) == NULL) { \ Py_DECREF(a); \ return NULL; \ @@ -5148,7 +5177,8 @@ ctx_##MPDFUNC(PyObject *context, PyObject *args) \ } \ \ CONVERT_BINOP_RAISE(&a, &b, v, w, context); \ - decimal_state *state = GLOBAL_STATE(); \ + decimal_state *state = \ + get_module_state_by_def(Py_TYPE(context)); \ if ((result = dec_alloc(state)) == NULL) { \ Py_DECREF(a); \ Py_DECREF(b); \ @@ -5183,7 +5213,8 @@ ctx_##MPDFUNC(PyObject *context, PyObject *args) \ } \ \ CONVERT_BINOP_RAISE(&a, &b, v, w, context); \ - decimal_state *state = GLOBAL_STATE(); \ + decimal_state *state = \ + get_module_state_by_def(Py_TYPE(context)); \ if ((result = dec_alloc(state)) == NULL) { \ Py_DECREF(a); \ Py_DECREF(b); \ @@ -5212,7 +5243,7 @@ ctx_##MPDFUNC(PyObject *context, PyObject *args) \ } \ \ CONVERT_TERNOP_RAISE(&a, &b, &c, v, w, x, context); \ - decimal_state *state = GLOBAL_STATE(); \ + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); \ if ((result = dec_alloc(state)) == NULL) { \ Py_DECREF(a); \ Py_DECREF(b); \ @@ -5278,7 +5309,7 @@ ctx_mpd_qdivmod(PyObject *context, PyObject *args) } CONVERT_BINOP_RAISE(&a, &b, v, w, context); - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); q = dec_alloc(state); if (q == NULL) { Py_DECREF(a); @@ -5333,7 +5364,7 @@ ctx_mpd_qpow(PyObject *context, PyObject *args, PyObject *kwds) } } - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); result = dec_alloc(state); if (result == NULL) { Py_DECREF(a); @@ -5383,9 +5414,9 @@ DecCtx_BoolFunc_NO_CTX(mpd_issnan) DecCtx_BoolFunc_NO_CTX(mpd_iszero) static PyObject * -ctx_iscanonical(PyObject *context UNUSED, PyObject *v) +ctx_iscanonical(PyObject *context, PyObject *v) { - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); if (!PyDec_Check(state, v)) { PyErr_SetString(PyExc_TypeError, "argument must be a Decimal"); @@ -5409,9 +5440,9 @@ PyDecContext_Apply(PyObject *context, PyObject *v) } static PyObject * -ctx_canonical(PyObject *context UNUSED, PyObject *v) +ctx_canonical(PyObject *context, PyObject *v) { - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); if (!PyDec_Check(state, v)) { PyErr_SetString(PyExc_TypeError, "argument must be a Decimal"); @@ -5428,7 +5459,7 @@ ctx_mpd_qcopy_abs(PyObject *context, PyObject *v) uint32_t status = 0; CONVERT_OP_RAISE(&a, v, context); - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); result = dec_alloc(state); if (result == NULL) { Py_DECREF(a); @@ -5461,7 +5492,7 @@ ctx_mpd_qcopy_negate(PyObject *context, PyObject *v) uint32_t status = 0; CONVERT_OP_RAISE(&a, v, context); - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); result = dec_alloc(state); if (result == NULL) { Py_DECREF(a); @@ -5558,7 +5589,7 @@ ctx_mpd_qcopy_sign(PyObject *context, PyObject *args) } CONVERT_BINOP_RAISE(&a, &b, v, w, context); - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state_by_def(Py_TYPE(context)); result = dec_alloc(state); if (result == NULL) { Py_DECREF(a); @@ -5747,17 +5778,6 @@ static PyMethodDef _decimal_methods [] = { NULL, NULL, 1, NULL } }; -static struct PyModuleDef _decimal_module = { - PyModuleDef_HEAD_INIT, - "decimal", - doc__decimal, - -1, - _decimal_methods, - NULL, - NULL, - NULL, - NULL -}; struct ssize_constmap { const char *name; mpd_ssize_t val; }; static struct ssize_constmap ssize_constants [] = { @@ -5833,10 +5853,9 @@ cfunc_noargs(PyTypeObject *t, const char *name) } -PyMODINIT_FUNC -PyInit__decimal(void) +static int +_decimal_exec(PyObject *m) { - PyObject *m = NULL; PyObject *numbers = NULL; PyObject *Number = NULL; PyObject *collections = NULL; @@ -5857,7 +5876,7 @@ PyInit__decimal(void) mpd_free = PyMem_Free; mpd_setminalloc(_Py_DEC_MINALLOC); - decimal_state *state = GLOBAL_STATE(); + decimal_state *state = get_module_state(m); /* Init external C-API functions */ state->_py_long_multiply = PyLong_Type.tp_as_number->nb_multiply; @@ -5929,10 +5948,6 @@ PyInit__decimal(void) Py_CLEAR(collections_abc); Py_CLEAR(MutableMapping); - - /* Create the module */ - ASSIGN_PTR(m, PyModule_Create(&_decimal_module)); - /* Add types to the module */ CHECK_INT(PyModule_AddType(m, state->PyDec_Type)); CHECK_INT(PyModule_AddType(m, state->PyDecContext_Type)); @@ -6073,31 +6088,103 @@ PyInit__decimal(void) CHECK_INT(PyModule_AddStringConstant(m, "__version__", "1.70")); CHECK_INT(PyModule_AddStringConstant(m, "__libmpdec_version__", mpd_version())); - - return m; - + return 0; error: Py_CLEAR(obj); /* GCOV_NOT_REACHED */ Py_CLEAR(numbers); /* GCOV_NOT_REACHED */ Py_CLEAR(Number); /* GCOV_NOT_REACHED */ - Py_CLEAR(state->Rational); /* GCOV_NOT_REACHED */ Py_CLEAR(collections); /* GCOV_NOT_REACHED */ Py_CLEAR(collections_abc); /* GCOV_NOT_REACHED */ Py_CLEAR(MutableMapping); /* GCOV_NOT_REACHED */ - Py_CLEAR(state->SignalTuple); /* GCOV_NOT_REACHED */ - PyMem_Free(state->signal_map); /* GCOV_NOT_REACHED */ - PyMem_Free(state->cond_map); /* GCOV_NOT_REACHED */ - Py_CLEAR(state->DecimalTuple); /* GCOV_NOT_REACHED */ - Py_CLEAR(state->default_context_template); /* GCOV_NOT_REACHED */ + + return -1; +} + +static int +decimal_traverse(PyObject *module, visitproc visit, void *arg) +{ + decimal_state *state = get_module_state(module); + Py_VISIT(state->PyDecContextManager_Type); + Py_VISIT(state->PyDecContext_Type); + Py_VISIT(state->PyDecSignalDictMixin_Type); + Py_VISIT(state->PyDec_Type); + Py_VISIT(state->PyDecSignalDict_Type); + Py_VISIT(state->DecimalTuple); + Py_VISIT(state->DecimalException); + +#ifndef WITH_DECIMAL_CONTEXTVAR + Py_VISIT(state->tls_context_key); + Py_VISIT(state->cached_context); +#else + Py_VISIT(state->current_context_var); +#endif + + Py_VISIT(state->default_context_template); + Py_VISIT(state->basic_context_template); + Py_VISIT(state->extended_context_template); + Py_VISIT(state->Rational); + Py_VISIT(state->SignalTuple); + + return 0; +} + +static int +decimal_clear(PyObject *module) +{ + decimal_state *state = get_module_state(module); + Py_CLEAR(state->PyDecContextManager_Type); + Py_CLEAR(state->PyDecContext_Type); + Py_CLEAR(state->PyDecSignalDictMixin_Type); + Py_CLEAR(state->PyDec_Type); + Py_CLEAR(state->PyDecSignalDict_Type); + Py_CLEAR(state->DecimalTuple); + Py_CLEAR(state->DecimalException); + #ifndef WITH_DECIMAL_CONTEXTVAR - Py_CLEAR(state->tls_context_key); /* GCOV_NOT_REACHED */ + Py_CLEAR(state->tls_context_key); + Py_CLEAR(state->cached_context); #else - Py_CLEAR(state->current_context_var); /* GCOV_NOT_REACHED */ + Py_CLEAR(state->current_context_var); #endif - Py_CLEAR(state->basic_context_template); /* GCOV_NOT_REACHED */ - Py_CLEAR(state->extended_context_template); /* GCOV_NOT_REACHED */ - Py_CLEAR(m); /* GCOV_NOT_REACHED */ - return NULL; /* GCOV_NOT_REACHED */ + Py_CLEAR(state->default_context_template); + Py_CLEAR(state->basic_context_template); + Py_CLEAR(state->extended_context_template); + Py_CLEAR(state->Rational); + Py_CLEAR(state->SignalTuple); + + PyMem_Free(state->signal_map); + PyMem_Free(state->cond_map); + return 0; +} + +static void +decimal_free(void *module) +{ + (void)decimal_clear((PyObject *)module); +} + +static struct PyModuleDef_Slot _decimal_slots[] = { + {Py_mod_exec, _decimal_exec}, + {Py_mod_multiple_interpreters, Py_MOD_PER_INTERPRETER_GIL_SUPPORTED}, + {0, NULL}, +}; + +static struct PyModuleDef _decimal_module = { + PyModuleDef_HEAD_INIT, + .m_name = "decimal", + .m_doc = doc__decimal, + .m_size = sizeof(decimal_state), + .m_methods = _decimal_methods, + .m_slots = _decimal_slots, + .m_traverse = decimal_traverse, + .m_clear = decimal_clear, + .m_free = decimal_free, +}; + +PyMODINIT_FUNC +PyInit__decimal(void) +{ + return PyModuleDef_Init(&_decimal_module); } diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 80e9d35afd9be..dedcb3141457c 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -421,7 +421,6 @@ Modules/_datetimemodule.c - us_per_hour - Modules/_datetimemodule.c - us_per_day - Modules/_datetimemodule.c - us_per_week - Modules/_datetimemodule.c - seconds_per_day - -Modules/_decimal/_decimal.c - global_state - ## state Modules/_ctypes/_ctypes.c - _ctypes_ptrtype_cache - From webhook-mailer at python.org Fri Jul 28 02:17:53 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 06:17:53 -0000 Subject: [Python-checkins] [3.12] gh-107298: Fix doc references to undocumented modules (GH-107300) (GH-107370) Message-ID: https://github.com/python/cpython/commit/17ce87ba7f8e4f8c9a5620ebf24a968f742b6e28 commit: 17ce87ba7f8e4f8c9a5620ebf24a968f742b6e28 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-28T06:17:49Z summary: [3.12] gh-107298: Fix doc references to undocumented modules (GH-107300) (GH-107370) Update also Doc/tools/.nitignore. (cherry picked from commit 87b39028e5f453a949a1675526c439f6479a04a8) Co-authored-by: Victor Stinner files: M Doc/c-api/arg.rst M Doc/c-api/codec.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/install/index.rst M Doc/tools/.nitignore diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index ea176bd7d6a51..c09c2d5261ca0 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -469,7 +469,7 @@ API Functions will be set if there was a failure. This is an example of the use of this function, taken from the sources for the - :mod:`_weakref` helper module for weak references:: + :mod:`!_weakref` helper module for weak references:: static PyObject * weakref_ref(PyObject *self, PyObject *args) diff --git a/Doc/c-api/codec.rst b/Doc/c-api/codec.rst index 235c77c945cc5..8ae5c4fecd624 100644 --- a/Doc/c-api/codec.rst +++ b/Doc/c-api/codec.rst @@ -7,7 +7,7 @@ Codec registry and support functions Register a new codec search function. - As side effect, this tries to load the :mod:`encodings` package, if not yet + As side effect, this tries to load the :mod:`!encodings` package, if not yet done, to make sure that it is always first in the list of search functions. .. c:function:: int PyCodec_Unregister(PyObject *search_function) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index e70b82405a4cf..40d21050446e5 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -579,7 +579,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) name, followed by a dot, followed by the type name; for built-in types, it should be just the type name. If the module is a submodule of a package, the full package name is part of the full module name. For example, a type named - :class:`T` defined in module :mod:`M` in subpackage :mod:`Q` in package :mod:`P` + :class:`T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``. For :ref:`dynamically allocated type objects `, diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 957934c78595e..6ae652e7e243c 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1212,7 +1212,7 @@ Character Map Codecs This codec is special in that it can be used to implement many different codecs (and this is in fact what was done to obtain most of the standard codecs -included in the :mod:`encodings` package). The codec uses mappings to encode and +included in the :mod:`!encodings` package). The codec uses mappings to encode and decode characters. The mapping objects provided must support the :meth:`__getitem__` mapping interface; dictionaries and sequences work well. diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 097d86e30269c..068f6fcdcaccd 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -367,7 +367,7 @@ Note that PyMODINIT_FUNC declares the function as ``PyObject *`` return type, declares any special linkage declarations required by the platform, and for C++ declares the function as ``extern "C"``. -When the Python program imports module :mod:`spam` for the first time, +When the Python program imports module :mod:`!spam` for the first time, :c:func:`PyInit_spam` is called. (See below for comments about embedding Python.) It calls :c:func:`PyModule_Create`, which returns a module object, and inserts built-in function objects into the newly created module based upon the @@ -1208,7 +1208,7 @@ file corresponding to the module provides a macro that takes care of importing the module and retrieving its C API pointers; client modules only have to call this macro before accessing the C API. -The exporting module is a modification of the :mod:`spam` module from section +The exporting module is a modification of the :mod:`!spam` module from section :ref:`extending-simpleexample`. The function :func:`spam.system` does not call the C library function :c:func:`system` directly, but a function :c:func:`PySpam_System`, which would of course do something more complicated in diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index 82671fdc447d9..80c8ea73e343e 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -37,7 +37,7 @@ object. This sort of thing can only be explained by example, so here's a minimal, but complete, module that defines a new type named :class:`Custom` inside a C -extension module :mod:`custom`: +extension module :mod:`!custom`: .. note:: What we're showing here is the traditional way of defining *static* @@ -55,7 +55,7 @@ from the previous chapter. This file defines three things: #. How the :class:`Custom` **type** behaves: this is the ``CustomType`` struct, which defines a set of flags and function pointers that the interpreter inspects when specific operations are requested. -#. How to initialize the :mod:`custom` module: this is the ``PyInit_custom`` +#. How to initialize the :mod:`!custom` module: this is the ``PyInit_custom`` function and the associated ``custommodule`` struct. The first bit is:: @@ -127,7 +127,7 @@ our objects and in some error messages, for example: TypeError: can only concatenate str (not "custom.Custom") to str Note that the name is a dotted name that includes both the module name and the -name of the type within the module. The module in this case is :mod:`custom` and +name of the type within the module. The module in this case is :mod:`!custom` and the type is :class:`Custom`, so we set the type name to :class:`custom.Custom`. Using the real dotted import path is important to make your type compatible with the :mod:`pydoc` and :mod:`pickle` modules. :: @@ -231,7 +231,7 @@ Adding data and methods to the Basic example ============================================ Let's extend the basic example to add some data and methods. Let's also make -the type usable as a base class. We'll create a new module, :mod:`custom2` that +the type usable as a base class. We'll create a new module, :mod:`!custom2` that adds these capabilities: .. literalinclude:: ../includes/custom2.c diff --git a/Doc/install/index.rst b/Doc/install/index.rst index 34d47fdb6a2f0..443c75b7684fb 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -374,7 +374,7 @@ will expand this to your home directory:: To make Python find the distributions installed with this scheme, you may have to :ref:`modify Python's search path ` or edit -:mod:`sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit +:mod:`!sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit :data:`sys.path`. The :option:`!--home` option defines the installation base directory. Files are diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 25d0f27e18311..6a165d626e087 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -12,7 +12,6 @@ Doc/c-api/bytes.rst Doc/c-api/capsule.rst Doc/c-api/cell.rst Doc/c-api/code.rst -Doc/c-api/codec.rst Doc/c-api/complex.rst Doc/c-api/conversion.rst Doc/c-api/datetime.rst From webhook-mailer at python.org Fri Jul 28 02:21:54 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 06:21:54 -0000 Subject: [Python-checkins] [3.11] gh-107298: Fix doc references to undocumented modules (GH-107300) (GH-107371) Message-ID: https://github.com/python/cpython/commit/50d26138e67ce30cb61f271ae35f60b2e43d7d38 commit: 50d26138e67ce30cb61f271ae35f60b2e43d7d38 branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-28T09:21:50+03:00 summary: [3.11] gh-107298: Fix doc references to undocumented modules (GH-107300) (GH-107371) (cherry picked from commit 87b39028e5f453a949a1675526c439f6479a04a8) Co-authored-by: Victor Stinner files: M Doc/c-api/arg.rst M Doc/c-api/codec.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/install/index.rst diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst index 9f7fd2fa8629e..dfbec82c457bc 100644 --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -506,7 +506,7 @@ API Functions will be set if there was a failure. This is an example of the use of this function, taken from the sources for the - :mod:`_weakref` helper module for weak references:: + :mod:`!_weakref` helper module for weak references:: static PyObject * weakref_ref(PyObject *self, PyObject *args) diff --git a/Doc/c-api/codec.rst b/Doc/c-api/codec.rst index 235c77c945cc5..8ae5c4fecd624 100644 --- a/Doc/c-api/codec.rst +++ b/Doc/c-api/codec.rst @@ -7,7 +7,7 @@ Codec registry and support functions Register a new codec search function. - As side effect, this tries to load the :mod:`encodings` package, if not yet + As side effect, this tries to load the :mod:`!encodings` package, if not yet done, to make sure that it is always first in the list of search functions. .. c:function:: int PyCodec_Unregister(PyObject *search_function) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index ec357450d004a..d4b3293ae59e2 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -577,7 +577,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) name, followed by a dot, followed by the type name; for built-in types, it should be just the type name. If the module is a submodule of a package, the full package name is part of the full module name. For example, a type named - :class:`T` defined in module :mod:`M` in subpackage :mod:`Q` in package :mod:`P` + :class:`T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``. For :ref:`dynamically allocated type objects `, diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index a96e0738f716e..62628c33a2bbd 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1304,7 +1304,7 @@ Character Map Codecs This codec is special in that it can be used to implement many different codecs (and this is in fact what was done to obtain most of the standard codecs -included in the :mod:`encodings` package). The codec uses mappings to encode and +included in the :mod:`!encodings` package). The codec uses mappings to encode and decode characters. The mapping objects provided must support the :meth:`__getitem__` mapping interface; dictionaries and sequences work well. diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 097d86e30269c..068f6fcdcaccd 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -367,7 +367,7 @@ Note that PyMODINIT_FUNC declares the function as ``PyObject *`` return type, declares any special linkage declarations required by the platform, and for C++ declares the function as ``extern "C"``. -When the Python program imports module :mod:`spam` for the first time, +When the Python program imports module :mod:`!spam` for the first time, :c:func:`PyInit_spam` is called. (See below for comments about embedding Python.) It calls :c:func:`PyModule_Create`, which returns a module object, and inserts built-in function objects into the newly created module based upon the @@ -1208,7 +1208,7 @@ file corresponding to the module provides a macro that takes care of importing the module and retrieving its C API pointers; client modules only have to call this macro before accessing the C API. -The exporting module is a modification of the :mod:`spam` module from section +The exporting module is a modification of the :mod:`!spam` module from section :ref:`extending-simpleexample`. The function :func:`spam.system` does not call the C library function :c:func:`system` directly, but a function :c:func:`PySpam_System`, which would of course do something more complicated in diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index 92e9b97e36d77..6bf63c8b9d7bf 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -37,7 +37,7 @@ object. This sort of thing can only be explained by example, so here's a minimal, but complete, module that defines a new type named :class:`Custom` inside a C -extension module :mod:`custom`: +extension module :mod:`!custom`: .. note:: What we're showing here is the traditional way of defining *static* @@ -55,7 +55,7 @@ from the previous chapter. This file defines three things: #. How the :class:`Custom` **type** behaves: this is the ``CustomType`` struct, which defines a set of flags and function pointers that the interpreter inspects when specific operations are requested. -#. How to initialize the :mod:`custom` module: this is the ``PyInit_custom`` +#. How to initialize the :mod:`!custom` module: this is the ``PyInit_custom`` function and the associated ``custommodule`` struct. The first bit is:: @@ -127,7 +127,7 @@ our objects and in some error messages, for example: TypeError: can only concatenate str (not "custom.Custom") to str Note that the name is a dotted name that includes both the module name and the -name of the type within the module. The module in this case is :mod:`custom` and +name of the type within the module. The module in this case is :mod:`!custom` and the type is :class:`Custom`, so we set the type name to :class:`custom.Custom`. Using the real dotted import path is important to make your type compatible with the :mod:`pydoc` and :mod:`pickle` modules. :: @@ -231,7 +231,7 @@ Adding data and methods to the Basic example ============================================ Let's extend the basic example to add some data and methods. Let's also make -the type usable as a base class. We'll create a new module, :mod:`custom2` that +the type usable as a base class. We'll create a new module, :mod:`!custom2` that adds these capabilities: .. literalinclude:: ../includes/custom2.c diff --git a/Doc/install/index.rst b/Doc/install/index.rst index 6c401c4c13d1c..fc4377e0d2b82 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -370,7 +370,7 @@ will expand this to your home directory:: To make Python find the distributions installed with this scheme, you may have to :ref:`modify Python's search path ` or edit -:mod:`sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit +:mod:`!sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit :data:`sys.path`. The :option:`!--home` option defines the installation base directory. Files are From webhook-mailer at python.org Fri Jul 28 02:24:33 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Fri, 28 Jul 2023 06:24:33 -0000 Subject: [Python-checkins] [3.12] gh-106723: forward -Xfrozen_modules option to spawned process interpreters (GH-106724) (#107367) Message-ID: https://github.com/python/cpython/commit/4f72a9a2e0b5b44a1a590b112cec2533ca182dff commit: 4f72a9a2e0b5b44a1a590b112cec2533ca182dff branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-28T06:24:29Z summary: [3.12] gh-106723: forward -Xfrozen_modules option to spawned process interpreters (GH-106724) (#107367) gh-106723: forward -Xfrozen_modules option to spawned process interpreters (GH-106724) (cherry picked from commit 3dcac785810df4d9db50abe90847eaf03bbdaaf4) Co-authored-by: Felipe A. Hernandez Co-authored-by: Kumar Aditya Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst M Lib/subprocess.py diff --git a/Lib/subprocess.py b/Lib/subprocess.py index fbc76b8d0f14b..6df5dd551ea67 100644 --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -346,7 +346,7 @@ def _args_from_interpreter_flags(): if dev_mode: args.extend(('-X', 'dev')) for opt in ('faulthandler', 'tracemalloc', 'importtime', - 'showrefcount', 'utf8'): + 'frozen_modules', 'showrefcount', 'utf8'): if opt in xoptions: value = xoptions[opt] if value is True: diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst new file mode 100644 index 0000000000000..207f397f17d3f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-13-14-55-45.gh-issue-106723.KsMufQ.rst @@ -0,0 +1 @@ +Propagate ``frozen_modules`` to multiprocessing spawned process interpreters. From webhook-mailer at python.org Fri Jul 28 02:36:52 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 28 Jul 2023 06:36:52 -0000 Subject: [Python-checkins] [3.11] gh-104432: Use `memcpy()` to avoid misaligned loads (GH-104433) (#107356) Message-ID: https://github.com/python/cpython/commit/c7d9976094e5a16140b5279b1197678529121516 commit: c7d9976094e5a16140b5279b1197678529121516 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gpshead date: 2023-07-28T06:36:45Z summary: [3.11] gh-104432: Use `memcpy()` to avoid misaligned loads (GH-104433) (#107356) gh-104432: Use `memcpy()` to avoid misaligned loads (GH-104433) Fix potential unaligned memory access on C APIs involving returned sequences of `char *` pointers within the :mod:`grp` and :mod:`socket` modules. These were revealed using a ``-fsaniziter=alignment`` build on ARM macOS. (cherry picked from commit f01e4cedba1a17d321664834bb255d9d04ad16ce) Co-authored-by: Christopher Chavez files: A Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst M Modules/grpmodule.c M Modules/socketmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst new file mode 100644 index 0000000000000..a9ab5cd43f0ff --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-27-11-47-29.gh-issue-104432.oGHF-z.rst @@ -0,0 +1,4 @@ +Fix potential unaligned memory access on C APIs involving returned sequences +of ``char *`` pointers within the :mod:`grp` and :mod:`socket` modules. These +were revealed using a ``-fsaniziter=alignment`` build on ARM macOS. Patch by +Christopher Chavez. diff --git a/Modules/grpmodule.c b/Modules/grpmodule.c index f6298ca0ee84c..738076c9d1730 100644 --- a/Modules/grpmodule.c +++ b/Modules/grpmodule.c @@ -65,8 +65,14 @@ mkgrent(PyObject *module, struct group *p) Py_DECREF(v); return NULL; } - for (member = p->gr_mem; *member != NULL; member++) { - PyObject *x = PyUnicode_DecodeFSDefault(*member); + for (member = p->gr_mem; ; member++) { + char *group_member; + // member can be misaligned + memcpy(&group_member, member, sizeof(group_member)); + if (group_member == NULL) { + break; + } + PyObject *x = PyUnicode_DecodeFSDefault(group_member); if (x == NULL || PyList_Append(w, x) != 0) { Py_XDECREF(x); Py_DECREF(w); diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 65d0e10fd253a..e6d983afa7da8 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -5657,9 +5657,15 @@ gethost_common(struct hostent *h, struct sockaddr *addr, size_t alen, int af) /* SF #1511317: h_aliases can be NULL */ if (h->h_aliases) { - for (pch = h->h_aliases; *pch != NULL; pch++) { + for (pch = h->h_aliases; ; pch++) { int status; - tmp = PyUnicode_FromString(*pch); + char *host_alias; + // pch can be misaligned + memcpy(&host_alias, pch, sizeof(host_alias)); + if (host_alias == NULL) { + break; + } + tmp = PyUnicode_FromString(host_alias); if (tmp == NULL) goto err; @@ -5671,8 +5677,14 @@ gethost_common(struct hostent *h, struct sockaddr *addr, size_t alen, int af) } } - for (pch = h->h_addr_list; *pch != NULL; pch++) { + for (pch = h->h_addr_list; ; pch++) { int status; + char *host_address; + // pch can be misaligned + memcpy(&host_address, pch, sizeof(host_address)); + if (host_address == NULL) { + break; + } switch (af) { @@ -5684,7 +5696,7 @@ gethost_common(struct hostent *h, struct sockaddr *addr, size_t alen, int af) #ifdef HAVE_SOCKADDR_SA_LEN sin.sin_len = sizeof(sin); #endif - memcpy(&sin.sin_addr, *pch, sizeof(sin.sin_addr)); + memcpy(&sin.sin_addr, host_address, sizeof(sin.sin_addr)); tmp = make_ipv4_addr(&sin); if (pch == h->h_addr_list && alen >= sizeof(sin)) @@ -5701,7 +5713,7 @@ gethost_common(struct hostent *h, struct sockaddr *addr, size_t alen, int af) #ifdef HAVE_SOCKADDR_SA_LEN sin6.sin6_len = sizeof(sin6); #endif - memcpy(&sin6.sin6_addr, *pch, sizeof(sin6.sin6_addr)); + memcpy(&sin6.sin6_addr, host_address, sizeof(sin6.sin6_addr)); tmp = make_ipv6_addr(&sin6); if (pch == h->h_addr_list && alen >= sizeof(sin6)) From webhook-mailer at python.org Fri Jul 28 02:40:19 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 06:40:19 -0000 Subject: [Python-checkins] [3.12] gh-107298: Fix Sphinx warnings in the C API doc (GH-107302) (GH-107375) Message-ID: https://github.com/python/cpython/commit/ef7422a1b98a9c84db5d2d4e16eafe01f8b4680a commit: ef7422a1b98a9c84db5d2d4e16eafe01f8b4680a branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-28T06:40:16Z summary: [3.12] gh-107298: Fix Sphinx warnings in the C API doc (GH-107302) (GH-107375) (cherry picked from commit 391e03fa05b80d17a14ac88d30c974fa2fa00adb) Co-authored-by: Victor Stinner files: M Doc/c-api/apiabiversion.rst M Doc/c-api/buffer.rst M Doc/c-api/bytes.rst M Doc/c-api/cell.rst M Doc/c-api/code.rst M Doc/c-api/gcsupport.rst M Doc/c-api/iterator.rst M Doc/c-api/type.rst M Doc/c-api/typehints.rst M Doc/c-api/unicode.rst M Doc/c-api/weakref.rst M Doc/tools/.nitignore diff --git a/Doc/c-api/apiabiversion.rst b/Doc/c-api/apiabiversion.rst index 62d542966622c..f6c8284daeacb 100644 --- a/Doc/c-api/apiabiversion.rst +++ b/Doc/c-api/apiabiversion.rst @@ -60,7 +60,7 @@ See :ref:`stable` for a discussion of API and ABI stability across versions. Use this for numeric comparisons, e.g. ``#if PY_VERSION_HEX >= ...``. - This version is also available via the symbol :data:`Py_Version`. + This version is also available via the symbol :c:var:`Py_Version`. .. c:var:: const unsigned long Py_Version diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst index 6e5443f0d6cdc..02b53ec149c73 100644 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -44,7 +44,7 @@ the elements exposed by an :class:`array.array` can be multi-byte values. An example consumer of the buffer interface is the :meth:`~io.BufferedIOBase.write` method of file objects: any object that can export a series of bytes through -the buffer interface can be written to a file. While :meth:`write` only +the buffer interface can be written to a file. While :meth:`!write` only needs read-only access to the internal contents of the object passed to it, other methods such as :meth:`~io.BufferedIOBase.readinto` need write access to the contents of their argument. The buffer interface allows objects to diff --git a/Doc/c-api/bytes.rst b/Doc/c-api/bytes.rst index 9f48f2ffafe17..4e3ffc7e23e3f 100644 --- a/Doc/c-api/bytes.rst +++ b/Doc/c-api/bytes.rst @@ -64,39 +64,39 @@ called with a non-bytes parameter. +-------------------+---------------+--------------------------------+ | Format Characters | Type | Comment | +===================+===============+================================+ - | :attr:`%%` | *n/a* | The literal % character. | + | ``%%`` | *n/a* | The literal % character. | +-------------------+---------------+--------------------------------+ - | :attr:`%c` | int | A single byte, | + | ``%c`` | int | A single byte, | | | | represented as a C int. | +-------------------+---------------+--------------------------------+ - | :attr:`%d` | int | Equivalent to | + | ``%d`` | int | Equivalent to | | | | ``printf("%d")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%u` | unsigned int | Equivalent to | + | ``%u`` | unsigned int | Equivalent to | | | | ``printf("%u")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%ld` | long | Equivalent to | + | ``%ld`` | long | Equivalent to | | | | ``printf("%ld")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%lu` | unsigned long | Equivalent to | + | ``%lu`` | unsigned long | Equivalent to | | | | ``printf("%lu")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%zd` | :c:type:`\ | Equivalent to | + | ``%zd`` | :c:type:`\ | Equivalent to | | | Py_ssize_t` | ``printf("%zd")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%zu` | size_t | Equivalent to | + | ``%zu`` | size_t | Equivalent to | | | | ``printf("%zu")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%i` | int | Equivalent to | + | ``%i`` | int | Equivalent to | | | | ``printf("%i")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%x` | int | Equivalent to | + | ``%x`` | int | Equivalent to | | | | ``printf("%x")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%s` | const char\* | A null-terminated C character | + | ``%s`` | const char\* | A null-terminated C character | | | | array. | +-------------------+---------------+--------------------------------+ - | :attr:`%p` | const void\* | The hex representation of a C | + | ``%p`` | const void\* | The hex representation of a C | | | | pointer. Mostly equivalent to | | | | ``printf("%p")`` except that | | | | it is guaranteed to start with | diff --git a/Doc/c-api/cell.rst b/Doc/c-api/cell.rst index ac4ef5adc5cc2..f8cd0344fdd1c 100644 --- a/Doc/c-api/cell.rst +++ b/Doc/c-api/cell.rst @@ -25,7 +25,7 @@ Cell objects are not likely to be useful elsewhere. The type object corresponding to cell objects. -.. c:function:: int PyCell_Check(ob) +.. c:function:: int PyCell_Check(PyObject *ob) Return true if *ob* is a cell object; *ob* must not be ``NULL``. This function always succeeds. diff --git a/Doc/c-api/code.rst b/Doc/c-api/code.rst index a99de9904c074..89fe42d1ff05f 100644 --- a/Doc/c-api/code.rst +++ b/Doc/c-api/code.rst @@ -39,7 +39,7 @@ bound into a function. use :c:func:`PyCode_NewEmpty` instead. Since the definition of the bytecode changes often, calling - :c:func:`PyCode_New` directly can bind you to a precise Python version. + :c:func:`PyUnstable_Code_New` directly can bind you to a precise Python version. The many arguments of this function are inter-dependent in complex ways, meaning that subtle changes to values are likely to result in incorrect @@ -58,8 +58,8 @@ bound into a function. .. c:function:: PyCodeObject* PyUnstable_Code_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable) - Similar to :c:func:`PyCode_New`, but with an extra "posonlyargcount" for positional-only arguments. - The same caveats that apply to ``PyCode_New`` also apply to this function. + Similar to :c:func:`PyUnstable_Code_New`, but with an extra "posonlyargcount" for positional-only arguments. + The same caveats that apply to ``PyUnstable_Code_New`` also apply to this function. .. index:: single: PyCode_NewWithPosOnlyArgs diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index e56414ab9f754..88709879d0163 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -143,7 +143,7 @@ rules: .. versionchanged:: 3.8 - The :c:func:`_PyObject_GC_TRACK` and :c:func:`_PyObject_GC_UNTRACK` macros + The :c:func:`!_PyObject_GC_TRACK` and :c:func:`!_PyObject_GC_UNTRACK` macros have been removed from the public C API. The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type: diff --git a/Doc/c-api/iterator.rst b/Doc/c-api/iterator.rst index 95952237ca746..6b7ba8c997916 100644 --- a/Doc/c-api/iterator.rst +++ b/Doc/c-api/iterator.rst @@ -19,7 +19,7 @@ sentinel value is returned. types. -.. c:function:: int PySeqIter_Check(op) +.. c:function:: int PySeqIter_Check(PyObject *op) Return true if the type of *op* is :c:data:`PySeqIter_Type`. This function always succeeds. @@ -38,7 +38,7 @@ sentinel value is returned. two-argument form of the :func:`iter` built-in function. -.. c:function:: int PyCallIter_Check(op) +.. c:function:: int PyCallIter_Check(PyObject *op) Return true if the type of *op* is :c:data:`PyCallIter_Type`. This function always succeeds. diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index 553d86aa3d997..e7b35c43da32a 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -480,7 +480,7 @@ The following functions and structs are used to create Setting :c:data:`Py_tp_bases` or :c:data:`Py_tp_base` may be problematic on some platforms. To avoid issues, use the *bases* argument of - :py:func:`PyType_FromSpecWithBases` instead. + :c:func:`PyType_FromSpecWithBases` instead. .. versionchanged:: 3.9 diff --git a/Doc/c-api/typehints.rst b/Doc/c-api/typehints.rst index 4c1957a2a1dbc..98fe68737deb8 100644 --- a/Doc/c-api/typehints.rst +++ b/Doc/c-api/typehints.rst @@ -35,7 +35,7 @@ two types exist -- :ref:`GenericAlias ` and ... } - .. seealso:: The data model method :meth:`__class_getitem__`. + .. seealso:: The data model method :meth:`~object.__class_getitem__`. .. versionadded:: 3.9 diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 6ae652e7e243c..fb4cfc64c8d00 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1214,7 +1214,7 @@ This codec is special in that it can be used to implement many different codecs (and this is in fact what was done to obtain most of the standard codecs included in the :mod:`!encodings` package). The codec uses mappings to encode and decode characters. The mapping objects provided must support the -:meth:`__getitem__` mapping interface; dictionaries and sequences work well. +:meth:`~object.__getitem__` mapping interface; dictionaries and sequences work well. These are the mapping codec APIs: @@ -1257,7 +1257,7 @@ The following codec API is special in that maps Unicode to Unicode. The mapping table must map Unicode ordinal integers to Unicode ordinal integers or ``None`` (causing deletion of the character). - Mapping tables need only provide the :meth:`__getitem__` interface; dictionaries + Mapping tables need only provide the :meth:`~object.__getitem__` interface; dictionaries and sequences work well. Unmapped character ordinals (ones which cause a :exc:`LookupError`) are left untouched and are copied as-is. diff --git a/Doc/c-api/weakref.rst b/Doc/c-api/weakref.rst index f27ec4411b4a2..f46507608606b 100644 --- a/Doc/c-api/weakref.rst +++ b/Doc/c-api/weakref.rst @@ -11,18 +11,18 @@ simple reference object, and the second acts as a proxy for the original object as much as it can. -.. c:function:: int PyWeakref_Check(ob) +.. c:function:: int PyWeakref_Check(PyObject *ob) Return true if *ob* is either a reference or proxy object. This function always succeeds. -.. c:function:: int PyWeakref_CheckRef(ob) +.. c:function:: int PyWeakref_CheckRef(PyObject *ob) Return true if *ob* is a reference object. This function always succeeds. -.. c:function:: int PyWeakref_CheckProxy(ob) +.. c:function:: int PyWeakref_CheckProxy(PyObject *ob) Return true if *ob* is a proxy object. This function always succeeds. @@ -54,7 +54,7 @@ as much as it can. .. c:function:: PyObject* PyWeakref_GetObject(PyObject *ref) Return the referenced object from a weak reference, *ref*. If the referent is - no longer live, returns :const:`Py_None`. + no longer live, returns ``Py_None``. .. note:: diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 6a165d626e087..b02f6eaea1325 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -4,14 +4,10 @@ # to help avoid merge conflicts. Doc/c-api/allocation.rst -Doc/c-api/apiabiversion.rst Doc/c-api/arg.rst Doc/c-api/bool.rst Doc/c-api/buffer.rst -Doc/c-api/bytes.rst Doc/c-api/capsule.rst -Doc/c-api/cell.rst -Doc/c-api/code.rst Doc/c-api/complex.rst Doc/c-api/conversion.rst Doc/c-api/datetime.rst @@ -25,7 +21,6 @@ Doc/c-api/import.rst Doc/c-api/init.rst Doc/c-api/init_config.rst Doc/c-api/intro.rst -Doc/c-api/iterator.rst Doc/c-api/memory.rst Doc/c-api/memoryview.rst Doc/c-api/module.rst @@ -36,11 +31,8 @@ Doc/c-api/stable.rst Doc/c-api/structures.rst Doc/c-api/sys.rst Doc/c-api/type.rst -Doc/c-api/typehints.rst Doc/c-api/typeobj.rst Doc/c-api/unicode.rst -Doc/c-api/veryhigh.rst -Doc/c-api/weakref.rst Doc/extending/embedding.rst Doc/extending/extending.rst Doc/extending/newtypes.rst From webhook-mailer at python.org Fri Jul 28 02:44:58 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 06:44:58 -0000 Subject: [Python-checkins] [3.11] gh-107298: Fix Sphinx warnings in the C API doc (GH-107302) (GH-107373) Message-ID: https://github.com/python/cpython/commit/dcfdfa539964a66fcc39722bada52e440f06a877 commit: dcfdfa539964a66fcc39722bada52e440f06a877 branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-28T06:44:54Z summary: [3.11] gh-107298: Fix Sphinx warnings in the C API doc (GH-107302) (GH-107373) (cherry picked from commit 391e03fa05b80d17a14ac88d30c974fa2fa00adb) Co-authored-by: Victor Stinner files: M Doc/c-api/apiabiversion.rst M Doc/c-api/buffer.rst M Doc/c-api/bytes.rst M Doc/c-api/cell.rst M Doc/c-api/gcsupport.rst M Doc/c-api/iterator.rst M Doc/c-api/type.rst M Doc/c-api/typehints.rst M Doc/c-api/unicode.rst M Doc/c-api/weakref.rst diff --git a/Doc/c-api/apiabiversion.rst b/Doc/c-api/apiabiversion.rst index 62d542966622c..f6c8284daeacb 100644 --- a/Doc/c-api/apiabiversion.rst +++ b/Doc/c-api/apiabiversion.rst @@ -60,7 +60,7 @@ See :ref:`stable` for a discussion of API and ABI stability across versions. Use this for numeric comparisons, e.g. ``#if PY_VERSION_HEX >= ...``. - This version is also available via the symbol :data:`Py_Version`. + This version is also available via the symbol :c:var:`Py_Version`. .. c:var:: const unsigned long Py_Version diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst index 6e5443f0d6cdc..02b53ec149c73 100644 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -44,7 +44,7 @@ the elements exposed by an :class:`array.array` can be multi-byte values. An example consumer of the buffer interface is the :meth:`~io.BufferedIOBase.write` method of file objects: any object that can export a series of bytes through -the buffer interface can be written to a file. While :meth:`write` only +the buffer interface can be written to a file. While :meth:`!write` only needs read-only access to the internal contents of the object passed to it, other methods such as :meth:`~io.BufferedIOBase.readinto` need write access to the contents of their argument. The buffer interface allows objects to diff --git a/Doc/c-api/bytes.rst b/Doc/c-api/bytes.rst index 21a5ab931c8bf..110a98e19d823 100644 --- a/Doc/c-api/bytes.rst +++ b/Doc/c-api/bytes.rst @@ -67,39 +67,39 @@ called with a non-bytes parameter. +-------------------+---------------+--------------------------------+ | Format Characters | Type | Comment | +===================+===============+================================+ - | :attr:`%%` | *n/a* | The literal % character. | + | ``%%`` | *n/a* | The literal % character. | +-------------------+---------------+--------------------------------+ - | :attr:`%c` | int | A single byte, | + | ``%c`` | int | A single byte, | | | | represented as a C int. | +-------------------+---------------+--------------------------------+ - | :attr:`%d` | int | Equivalent to | + | ``%d`` | int | Equivalent to | | | | ``printf("%d")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%u` | unsigned int | Equivalent to | + | ``%u`` | unsigned int | Equivalent to | | | | ``printf("%u")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%ld` | long | Equivalent to | + | ``%ld`` | long | Equivalent to | | | | ``printf("%ld")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%lu` | unsigned long | Equivalent to | + | ``%lu`` | unsigned long | Equivalent to | | | | ``printf("%lu")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%zd` | :c:type:`\ | Equivalent to | + | ``%zd`` | :c:type:`\ | Equivalent to | | | Py_ssize_t` | ``printf("%zd")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%zu` | size_t | Equivalent to | + | ``%zu`` | size_t | Equivalent to | | | | ``printf("%zu")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%i` | int | Equivalent to | + | ``%i`` | int | Equivalent to | | | | ``printf("%i")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%x` | int | Equivalent to | + | ``%x`` | int | Equivalent to | | | | ``printf("%x")``. [1]_ | +-------------------+---------------+--------------------------------+ - | :attr:`%s` | const char\* | A null-terminated C character | + | ``%s`` | const char\* | A null-terminated C character | | | | array. | +-------------------+---------------+--------------------------------+ - | :attr:`%p` | const void\* | The hex representation of a C | + | ``%p`` | const void\* | The hex representation of a C | | | | pointer. Mostly equivalent to | | | | ``printf("%p")`` except that | | | | it is guaranteed to start with | diff --git a/Doc/c-api/cell.rst b/Doc/c-api/cell.rst index ac4ef5adc5cc2..f8cd0344fdd1c 100644 --- a/Doc/c-api/cell.rst +++ b/Doc/c-api/cell.rst @@ -25,7 +25,7 @@ Cell objects are not likely to be useful elsewhere. The type object corresponding to cell objects. -.. c:function:: int PyCell_Check(ob) +.. c:function:: int PyCell_Check(PyObject *ob) Return true if *ob* is a cell object; *ob* must not be ``NULL``. This function always succeeds. diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index fc690fd85c9f7..ac938735a67dd 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -124,7 +124,7 @@ rules: .. versionchanged:: 3.8 - The :c:func:`_PyObject_GC_TRACK` and :c:func:`_PyObject_GC_UNTRACK` macros + The :c:func:`!_PyObject_GC_TRACK` and :c:func:`!_PyObject_GC_UNTRACK` macros have been removed from the public C API. The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type: diff --git a/Doc/c-api/iterator.rst b/Doc/c-api/iterator.rst index 95952237ca746..6b7ba8c997916 100644 --- a/Doc/c-api/iterator.rst +++ b/Doc/c-api/iterator.rst @@ -19,7 +19,7 @@ sentinel value is returned. types. -.. c:function:: int PySeqIter_Check(op) +.. c:function:: int PySeqIter_Check(PyObject *op) Return true if the type of *op* is :c:data:`PySeqIter_Type`. This function always succeeds. @@ -38,7 +38,7 @@ sentinel value is returned. two-argument form of the :func:`iter` built-in function. -.. c:function:: int PyCallIter_Check(op) +.. c:function:: int PyCallIter_Check(PyObject *op) Return true if the type of *op* is :c:data:`PyCallIter_Type`. This function always succeeds. diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index d6f32ed88b673..01d7fc4dc1ecf 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -292,7 +292,7 @@ The following functions and structs are used to create Setting :c:data:`Py_tp_bases` or :c:data:`Py_tp_base` may be problematic on some platforms. To avoid issues, use the *bases* argument of - :py:func:`PyType_FromSpecWithBases` instead. + :c:func:`PyType_FromSpecWithBases` instead. .. versionchanged:: 3.9 diff --git a/Doc/c-api/typehints.rst b/Doc/c-api/typehints.rst index 4c1957a2a1dbc..98fe68737deb8 100644 --- a/Doc/c-api/typehints.rst +++ b/Doc/c-api/typehints.rst @@ -35,7 +35,7 @@ two types exist -- :ref:`GenericAlias ` and ... } - .. seealso:: The data model method :meth:`__class_getitem__`. + .. seealso:: The data model method :meth:`~object.__class_getitem__`. .. versionadded:: 3.9 diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 62628c33a2bbd..1ed1f06f4595d 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1306,7 +1306,7 @@ This codec is special in that it can be used to implement many different codecs (and this is in fact what was done to obtain most of the standard codecs included in the :mod:`!encodings` package). The codec uses mappings to encode and decode characters. The mapping objects provided must support the -:meth:`__getitem__` mapping interface; dictionaries and sequences work well. +:meth:`~object.__getitem__` mapping interface; dictionaries and sequences work well. These are the mapping codec APIs: @@ -1349,7 +1349,7 @@ The following codec API is special in that maps Unicode to Unicode. The mapping table must map Unicode ordinal integers to Unicode ordinal integers or ``None`` (causing deletion of the character). - Mapping tables need only provide the :meth:`__getitem__` interface; dictionaries + Mapping tables need only provide the :meth:`~object.__getitem__` interface; dictionaries and sequences work well. Unmapped character ordinals (ones which cause a :exc:`LookupError`) are left untouched and are copied as-is. diff --git a/Doc/c-api/weakref.rst b/Doc/c-api/weakref.rst index f27ec4411b4a2..f46507608606b 100644 --- a/Doc/c-api/weakref.rst +++ b/Doc/c-api/weakref.rst @@ -11,18 +11,18 @@ simple reference object, and the second acts as a proxy for the original object as much as it can. -.. c:function:: int PyWeakref_Check(ob) +.. c:function:: int PyWeakref_Check(PyObject *ob) Return true if *ob* is either a reference or proxy object. This function always succeeds. -.. c:function:: int PyWeakref_CheckRef(ob) +.. c:function:: int PyWeakref_CheckRef(PyObject *ob) Return true if *ob* is a reference object. This function always succeeds. -.. c:function:: int PyWeakref_CheckProxy(ob) +.. c:function:: int PyWeakref_CheckProxy(PyObject *ob) Return true if *ob* is a proxy object. This function always succeeds. @@ -54,7 +54,7 @@ as much as it can. .. c:function:: PyObject* PyWeakref_GetObject(PyObject *ref) Return the referenced object from a weak reference, *ref*. If the referent is - no longer live, returns :const:`Py_None`. + no longer live, returns ``Py_None``. .. note:: From webhook-mailer at python.org Fri Jul 28 02:56:56 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 06:56:56 -0000 Subject: [Python-checkins] [3.12] gh-107298: Fix more Sphinx warnings in the C API doc (GH-107329) (GH-107376) Message-ID: https://github.com/python/cpython/commit/e6a4b10820768f7a3ca9b919a8d8961bc92c6af4 commit: e6a4b10820768f7a3ca9b919a8d8961bc92c6af4 branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-28T09:56:52+03:00 summary: [3.12] gh-107298: Fix more Sphinx warnings in the C API doc (GH-107329) (GH-107376) Declare the following functions as macros, since they are actually macros. It avoids a warning on "TYPE" or "macro" argument. * PyMem_New() * PyMem_Resize() * PyModule_AddIntMacro() * PyModule_AddStringMacro() * PyObject_GC_New() * PyObject_GC_NewVar() * PyObject_New() * PyObject_NewVar() Add C standard C types to nitpick_ignore in Doc/conf.py: * int64_t * uint64_t * uintptr_t No longer ignore non existing "__int" type in nitpick_ignore. Update Doc/tools/.nitignore. (cherry picked from commit 8d61a71f9c81619e34d4a30b625922ebc83c561b) Co-authored-by: Victor Stinner files: M Doc/c-api/allocation.rst M Doc/c-api/capsule.rst M Doc/c-api/complex.rst M Doc/c-api/conversion.rst M Doc/c-api/exceptions.rst M Doc/c-api/gcsupport.rst M Doc/c-api/import.rst M Doc/c-api/init.rst M Doc/c-api/init_config.rst M Doc/c-api/memory.rst M Doc/c-api/module.rst M Doc/c-api/set.rst M Doc/c-api/structures.rst M Doc/c-api/sys.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/conf.py M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/tools/.nitignore M Doc/whatsnew/2.3.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst diff --git a/Doc/c-api/allocation.rst b/Doc/c-api/allocation.rst index 0a8fcc5ae5fcd..44747e2964366 100644 --- a/Doc/c-api/allocation.rst +++ b/Doc/c-api/allocation.rst @@ -27,22 +27,25 @@ Allocating Objects on the Heap length information for a variable-size object. -.. c:function:: TYPE* PyObject_New(TYPE, PyTypeObject *type) +.. c:macro:: PyObject_New(TYPE, typeobj) Allocate a new Python object using the C structure type *TYPE* and the - Python type object *type*. Fields not defined by the Python object header + Python type object *typeobj* (``PyTypeObject*``). + Fields not defined by the Python object header are not initialized; the object's reference count will be one. The size of the memory allocation is determined from the :c:member:`~PyTypeObject.tp_basicsize` field of the type object. -.. c:function:: TYPE* PyObject_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) +.. c:macro:: PyObject_NewVar(TYPE, typeobj, size) Allocate a new Python object using the C structure type *TYPE* and the - Python type object *type*. Fields not defined by the Python object header + Python type object *typeobj* (``PyTypeObject*``). + Fields not defined by the Python object header are not initialized. The allocated memory allows for the *TYPE* structure - plus *size* fields of the size given by the :c:member:`~PyTypeObject.tp_itemsize` field of - *type*. This is useful for implementing objects like tuples, which are + plus *size* (``Py_ssize_t``) fields of the size + given by the :c:member:`~PyTypeObject.tp_itemsize` field of + *typeobj*. This is useful for implementing objects like tuples, which are able to determine their size at construction time. Embedding the array of fields into the same allocation decreases the number of allocations, improving the memory management efficiency. @@ -50,8 +53,8 @@ Allocating Objects on the Heap .. c:function:: void PyObject_Del(void *op) - Releases memory allocated to an object using :c:func:`PyObject_New` or - :c:func:`PyObject_NewVar`. This is normally called from the + Releases memory allocated to an object using :c:macro:`PyObject_New` or + :c:macro:`PyObject_NewVar`. This is normally called from the :c:member:`~PyTypeObject.tp_dealloc` handler specified in the object's type. The fields of the object should not be accessed after this call as the memory is no longer a valid Python object. diff --git a/Doc/c-api/capsule.rst b/Doc/c-api/capsule.rst index 427ed959c5856..2a1b602dc79c0 100644 --- a/Doc/c-api/capsule.rst +++ b/Doc/c-api/capsule.rst @@ -64,7 +64,7 @@ Refer to :ref:`using-capsules` for more information on using these objects. The *name* parameter must compare exactly to the name stored in the capsule. If the name stored in the capsule is ``NULL``, the *name* passed in must also - be ``NULL``. Python uses the C function :c:func:`strcmp` to compare capsule + be ``NULL``. Python uses the C function :c:func:`!strcmp` to compare capsule names. diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst index 6679ce76f1dc6..e3fd001c599c8 100644 --- a/Doc/c-api/complex.rst +++ b/Doc/c-api/complex.rst @@ -64,7 +64,7 @@ pointers. This is consistent throughout the API. representation. If *divisor* is null, this method returns zero and sets - :c:data:`errno` to :c:macro:`EDOM`. + :c:data:`errno` to :c:macro:`!EDOM`. .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp) @@ -73,7 +73,7 @@ pointers. This is consistent throughout the API. representation. If *num* is null and *exp* is not a positive real number, - this method returns zero and sets :c:data:`errno` to :c:macro:`EDOM`. + this method returns zero and sets :c:data:`errno` to :c:macro:`!EDOM`. Complex Numbers as Python Objects diff --git a/Doc/c-api/conversion.rst b/Doc/c-api/conversion.rst index fdb321fe7ab3f..c5350123dfdfd 100644 --- a/Doc/c-api/conversion.rst +++ b/Doc/c-api/conversion.rst @@ -119,10 +119,10 @@ The following functions provide locale-independent string to number conversions. .. c:function:: int PyOS_stricmp(const char *s1, const char *s2) Case insensitive comparison of strings. The function works almost - identically to :c:func:`strcmp` except that it ignores the case. + identically to :c:func:`!strcmp` except that it ignores the case. .. c:function:: int PyOS_strnicmp(const char *s1, const char *s2, Py_ssize_t size) Case insensitive comparison of strings. The function works almost - identically to :c:func:`strncmp` except that it ignores the case. + identically to :c:func:`!strncmp` except that it ignores the case. diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 4d81b273638b1..a2126ffc559ab 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -83,7 +83,7 @@ Printing and clearing This utility function prints a warning message to ``sys.stderr`` when an exception has been set but it is impossible for the interpreter to actually raise the exception. It is used, for example, when an exception occurs in an - :meth:`__del__` method. + :meth:`~object.__del__` method. The function is called with a single argument *obj* that identifies the context in which the unraisable exception occurred. If possible, @@ -165,7 +165,7 @@ For convenience, some of these functions will always return a tuple object whose first item is the integer :c:data:`errno` value and whose second item is the corresponding error message (gotten from :c:func:`!strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the - :c:data:`errno` value is :c:macro:`EINTR`, indicating an interrupted system call, + :c:data:`errno` value is :c:macro:`!EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, leaves it set to that. The function always returns ``NULL``, so a wrapper function around a system call can write ``return PyErr_SetFromErrno(type);`` @@ -177,7 +177,7 @@ For convenience, some of these functions will always return a Similar to :c:func:`PyErr_SetFromErrno`, with the additional behavior that if *filenameObject* is not ``NULL``, it is passed to the constructor of *type* as a third parameter. In the case of :exc:`OSError` exception, - this is used to define the :attr:`filename` attribute of the + this is used to define the :attr:`!filename` attribute of the exception instance. @@ -200,12 +200,12 @@ For convenience, some of these functions will always return a .. c:function:: PyObject* PyErr_SetFromWindowsErr(int ierr) This is a convenience function to raise :exc:`WindowsError`. If called with - *ierr* of ``0``, the error code returned by a call to :c:func:`GetLastError` - is used instead. It calls the Win32 function :c:func:`FormatMessage` to retrieve - the Windows description of error code given by *ierr* or :c:func:`GetLastError`, + *ierr* of ``0``, the error code returned by a call to :c:func:`!GetLastError` + is used instead. It calls the Win32 function :c:func:`!FormatMessage` to retrieve + the Windows description of error code given by *ierr* or :c:func:`!GetLastError`, then it constructs a tuple object whose first item is the *ierr* value and whose second item is the corresponding error message (gotten from - :c:func:`FormatMessage`), and then calls ``PyErr_SetObject(PyExc_WindowsError, + :c:func:`!FormatMessage`), and then calls ``PyErr_SetObject(PyExc_WindowsError, object)``. This function always returns ``NULL``. .. availability:: Windows. @@ -631,7 +631,7 @@ Signal Handling be interruptible by user requests (such as by pressing Ctrl-C). .. note:: - The default Python signal handler for :c:macro:`SIGINT` raises the + The default Python signal handler for :c:macro:`!SIGINT` raises the :exc:`KeyboardInterrupt` exception. @@ -642,7 +642,7 @@ Signal Handling single: SIGINT single: KeyboardInterrupt (built-in exception) - Simulate the effect of a :c:macro:`SIGINT` signal arriving. + Simulate the effect of a :c:macro:`!SIGINT` signal arriving. This is equivalent to ``PyErr_SetInterruptEx(SIGINT)``. .. note:: diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index 88709879d0163..6b2494ee4f0ed 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -25,8 +25,8 @@ include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the Constructors for container types must conform to two rules: -#. The memory for the object must be allocated using :c:func:`PyObject_GC_New` - or :c:func:`PyObject_GC_NewVar`. +#. The memory for the object must be allocated using :c:macro:`PyObject_GC_New` + or :c:macro:`PyObject_GC_NewVar`. #. Once all the fields which may contain references to other containers are initialized, it must call :c:func:`PyObject_GC_Track`. @@ -52,19 +52,19 @@ rules: class that implements the garbage collector protocol and the child class does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. -.. c:function:: TYPE* PyObject_GC_New(TYPE, PyTypeObject *type) +.. c:macro:: PyObject_GC_New(TYPE, typeobj) - Analogous to :c:func:`PyObject_New` but for container objects with the + Analogous to :c:macro:`PyObject_New` but for container objects with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. -.. c:function:: TYPE* PyObject_GC_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) +.. c:macro:: PyObject_GC_NewVar(TYPE, typeobj, size) - Analogous to :c:func:`PyObject_NewVar` but for container objects with the + Analogous to :c:macro:`PyObject_NewVar` but for container objects with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. .. c:function:: PyObject* PyUnstable_Object_GC_NewWithExtraData(PyTypeObject *type, size_t extra_size) - Analogous to :c:func:`PyObject_GC_New` but allocates *extra_size* + Analogous to :c:macro:`PyObject_GC_New` but allocates *extra_size* bytes at the end of the object (at offset :c:member:`~PyTypeObject.tp_basicsize`). The allocated memory is initialized to zeros, @@ -85,7 +85,7 @@ rules: .. c:function:: TYPE* PyObject_GC_Resize(TYPE, PyVarObject *op, Py_ssize_t newsize) - Resize an object allocated by :c:func:`PyObject_NewVar`. Returns the + Resize an object allocated by :c:macro:`PyObject_NewVar`. Returns the resized object or ``NULL`` on failure. *op* must not be tracked by the collector yet. @@ -128,8 +128,8 @@ rules: .. c:function:: void PyObject_GC_Del(void *op) - Releases memory allocated to an object using :c:func:`PyObject_GC_New` or - :c:func:`PyObject_GC_NewVar`. + Releases memory allocated to an object using :c:macro:`PyObject_GC_New` or + :c:macro:`PyObject_GC_NewVar`. .. c:function:: void PyObject_GC_UnTrack(void *op) diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 57f94a7f97671..7bc93a81e2c78 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -135,10 +135,10 @@ Importing Modules The module's :attr:`__spec__` and :attr:`__loader__` will be set, if not set already, with the appropriate values. The spec's loader will be set to the module's ``__loader__`` (if set) and to an instance of - :class:`SourceFileLoader` otherwise. + :class:`~importlib.machinery.SourceFileLoader` otherwise. The module's :attr:`__file__` attribute will be set to the code object's - :attr:`co_filename`. If applicable, :attr:`__cached__` will also + :attr:`!co_filename`. If applicable, :attr:`__cached__` will also be set. This function will reload the module if it was already imported. See @@ -225,7 +225,7 @@ Importing Modules .. c:function:: PyObject* PyImport_GetImporter(PyObject *path) - Return a finder object for a :data:`sys.path`/:attr:`pkg.__path__` item + Return a finder object for a :data:`sys.path`/:attr:`!pkg.__path__` item *path*, possibly by fetching it from the :data:`sys.path_importer_cache` dict. If it wasn't yet cached, traverse :data:`sys.path_hooks` until a hook is found that can handle the path item. Return ``None`` if no hook could; diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 4144f16f23bff..8a2543affc4d1 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -25,7 +25,7 @@ The following functions can be safely called before Python is initialized: * :c:func:`PyImport_AppendInittab` * :c:func:`PyImport_ExtendInittab` - * :c:func:`PyInitFrozenExtensions` + * :c:func:`!PyInitFrozenExtensions` * :c:func:`PyMem_SetAllocator` * :c:func:`PyMem_SetupDebugHooks` * :c:func:`PyObject_SetArenaAllocator` @@ -157,7 +157,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. :c:member:`PyConfig.use_environment` should be used instead, see :ref:`Python Initialization Configuration `. - Ignore all :envvar:`PYTHON*` environment variables, e.g. + Ignore all :envvar:`!PYTHON*` environment variables, e.g. :envvar:`PYTHONPATH` and :envvar:`PYTHONHOME`, that might be set. Set by the :option:`-E` and :option:`-I` options. @@ -230,7 +230,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. :ref:`Python Initialization Configuration `. If the flag is non-zero, use :class:`io.FileIO` instead of - :class:`WindowsConsoleIO` for :mod:`sys` standard streams. + :class:`!io._WindowsConsoleIO` for :mod:`sys` standard streams. Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSSTDIO` environment variable is set to a non-empty string. @@ -401,7 +401,7 @@ Initializing and finalizing the interpreter the application. **Bugs and caveats:** The destruction of modules and objects in modules is done - in random order; this may cause destructors (:meth:`__del__` methods) to fail + in random order; this may cause destructors (:meth:`~object.__del__` methods) to fail when they depend on other objects (even functions) or modules. Dynamically loaded extension modules loaded by Python are not unloaded. Small amounts of memory allocated by the Python interpreter may not be freed (if you find a leak, @@ -485,12 +485,12 @@ Process-wide parameters interpreter will change the contents of this storage. Use :c:func:`Py_DecodeLocale` to decode a bytes string to get a - :c:expr:`wchar_*` string. + :c:expr:`wchar_t *` string. .. deprecated:: 3.11 -.. c:function:: wchar* Py_GetProgramName() +.. c:function:: wchar_t* Py_GetProgramName() .. index:: single: Py_SetProgramName() @@ -981,7 +981,7 @@ the fork, and releasing them afterwards. In addition, it resets any :ref:`lock-objects` in the child. When extending or embedding Python, there is no way to inform Python of additional (non-Python) locks that need to be acquired before or reset after a fork. OS facilities such as -:c:func:`pthread_atfork` would need to be used to accomplish the same thing. +:c:func:`!pthread_atfork` would need to be used to accomplish the same thing. Additionally, when extending or embedding Python, calling :c:func:`fork` directly rather than through :func:`os.fork` (and returning to or calling into Python) may result in a deadlock by one of Python's internal locks @@ -1087,7 +1087,7 @@ code, or when embedding the Python interpreter: .. note:: Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. - You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to + You can use :c:func:`!_Py_IsFinalizing` or :func:`sys.is_finalizing` to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination. @@ -1133,7 +1133,7 @@ with sub-interpreters: .. note:: Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. - You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to + You can use :c:func:`!_Py_IsFinalizing` or :func:`sys.is_finalizing` to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination. @@ -1415,7 +1415,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. note:: Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. - You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to + You can use :c:func:`!_Py_IsFinalizing` or :func:`sys.is_finalizing` to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination. diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 18f17e2059b10..cf7337f679e69 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -871,7 +871,7 @@ PyConfig .. c:member:: int legacy_windows_stdio If non-zero, use :class:`io.FileIO` instead of - :class:`io.WindowsConsoleIO` for :data:`sys.stdin`, :data:`sys.stdout` + :class:`!io._WindowsConsoleIO` for :data:`sys.stdin`, :data:`sys.stdout` and :data:`sys.stderr`. Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSSTDIO` environment @@ -1120,7 +1120,7 @@ PyConfig Set to ``0`` by the :option:`-S` command line option. - :data:`sys.flags.no_site` is set to the inverted value of + :data:`sys.flags.no_site ` is set to the inverted value of :c:member:`~PyConfig.site_import`. Default: ``1``. diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 4ca3b8804427f..c51aba3f55536 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -136,7 +136,7 @@ need to be held. The :ref:`default raw memory allocator ` uses the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` -and :c:func:`free`; call ``malloc(1)`` (or ``calloc(1, 1)``) when requesting +and :c:func:`!free`; call ``malloc(1)`` (or ``calloc(1, 1)``) when requesting zero bytes. .. versionadded:: 3.4 @@ -264,14 +264,14 @@ The following type-oriented macros are provided for convenience. Note that *TYPE* refers to any C type. -.. c:function:: TYPE* PyMem_New(TYPE, size_t n) +.. c:macro:: PyMem_New(TYPE, n) Same as :c:func:`PyMem_Malloc`, but allocates ``(n * sizeof(TYPE))`` bytes of memory. Returns a pointer cast to :c:expr:`TYPE*`. The memory will not have been initialized in any way. -.. c:function:: TYPE* PyMem_Resize(void *p, TYPE, size_t n) +.. c:macro:: PyMem_Resize(p, TYPE, n) Same as :c:func:`PyMem_Realloc`, but the memory block is resized to ``(n * sizeof(TYPE))`` bytes. Returns a pointer cast to :c:expr:`TYPE*`. On return, @@ -423,7 +423,7 @@ Customize Memory Allocators +----------------------------------------------------------+---------------------------------------+ .. versionchanged:: 3.5 - The :c:type:`PyMemAllocator` structure was renamed to + The :c:type:`!PyMemAllocator` structure was renamed to :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added. @@ -627,8 +627,8 @@ with a fixed size of 256 KiB. It falls back to :c:func:`PyMem_RawMalloc` and The arena allocator uses the following functions: -* :c:func:`VirtualAlloc` and :c:func:`VirtualFree` on Windows, -* :c:func:`mmap` and :c:func:`munmap` if available, +* :c:func:`!VirtualAlloc` and :c:func:`!VirtualFree` on Windows, +* :c:func:`!mmap` and :c:func:`!munmap` if available, * :c:func:`malloc` and :c:func:`free` otherwise. This allocator is disabled if Python is configured with the @@ -732,8 +732,8 @@ allocators operating on different heaps. :: free(buf1); /* Fatal -- should be PyMem_Del() */ In addition to the functions aimed at handling raw memory blocks from the Python -heap, objects in Python are allocated and released with :c:func:`PyObject_New`, -:c:func:`PyObject_NewVar` and :c:func:`PyObject_Del`. +heap, objects in Python are allocated and released with :c:macro:`PyObject_New`, +:c:macro:`PyObject_NewVar` and :c:func:`PyObject_Del`. These will be explained in the next chapter on defining and implementing new object types in C. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index 6abd550ab80b2..fc4dd4ce053e6 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -282,7 +282,7 @@ An alternate way to specify extensions is to request "multi-phase initialization Extension modules created this way behave more like Python modules: the initialization is split between the *creation phase*, when the module object is created, and the *execution phase*, when it is populated. -The distinction is similar to the :py:meth:`__new__` and :py:meth:`__init__` methods +The distinction is similar to the :py:meth:`!__new__` and :py:meth:`!__init__` methods of classes. Unlike modules created using single-phase initialization, these modules are not @@ -293,7 +293,7 @@ By default, multiple modules created from the same definition should be independent: changes to one should not affect the others. This means that all state should be specific to the module object (using e.g. using :c:func:`PyModule_GetState`), or its contents (such as the module's -:attr:`__dict__` or individual classes created with :c:func:`PyType_FromSpec`). +:attr:`~object.__dict__` or individual classes created with :c:func:`PyType_FromSpec`). All modules created using multi-phase initialization are expected to support :ref:`sub-interpreters `. Making sure multiple modules @@ -555,7 +555,7 @@ state: ``NULL``-terminated. Return ``-1`` on error, ``0`` on success. -.. c:function:: int PyModule_AddIntMacro(PyObject *module, macro) +.. c:macro:: PyModule_AddIntMacro(module, macro) Add an int constant to *module*. The name and the value are taken from *macro*. For example ``PyModule_AddIntMacro(module, AF_INET)`` adds the int @@ -563,7 +563,7 @@ state: Return ``-1`` on error, ``0`` on success. -.. c:function:: int PyModule_AddStringMacro(PyObject *module, macro) +.. c:macro:: PyModule_AddStringMacro(module, macro) Add a string constant to *module*. diff --git a/Doc/c-api/set.rst b/Doc/c-api/set.rst index d642a5f1902e2..7e0ebd2f791a4 100644 --- a/Doc/c-api/set.rst +++ b/Doc/c-api/set.rst @@ -122,7 +122,7 @@ or :class:`frozenset` or instances of their subtypes. .. c:function:: int PySet_Contains(PyObject *anyset, PyObject *key) Return ``1`` if found, ``0`` if not found, and ``-1`` if an error is encountered. Unlike - the Python :meth:`__contains__` method, this function does not automatically + the Python :meth:`~object.__contains__` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise a :exc:`TypeError` if the *key* is unhashable. Raise :exc:`PyExc_SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 55f3410728bc3..747cfa62294c2 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -393,7 +393,7 @@ definition with the same method name. *METH_COEXIST*, the default is to skip repeated definitions. Since slot wrappers are loaded before the method table, the existence of a *sq_contains* slot, for example, would generate a wrapped method named - :meth:`__contains__` and preclude the loading of a corresponding + :meth:`~object.__contains__` and preclude the loading of a corresponding PyCFunction with the same name. With the flag defined, the PyCFunction will be loaded in place of the wrapper object and will co-exist with the slot. This is helpful because calls to PyCFunctions are optimized more diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 7e0e982a04ec3..2c45ea1427a11 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -414,7 +414,7 @@ Process Control This function should only be invoked when a condition is detected that would make it dangerous to continue using the Python interpreter; e.g., when the object administration appears to be corrupted. On Unix, the standard C library - function :c:func:`abort` is called which will attempt to produce a :file:`core` + function :c:func:`!abort` is called which will attempt to produce a :file:`core` file. The ``Py_FatalError()`` function is replaced with a macro which logs diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 40d21050446e5..99b025dc41fc2 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -579,7 +579,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) name, followed by a dot, followed by the type name; for built-in types, it should be just the type name. If the module is a submodule of a package, the full package name is part of the full module name. For example, a type named - :class:`T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` + :class:`!T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``. For :ref:`dynamically allocated type objects `, @@ -673,9 +673,9 @@ and :c:data:`PyType_Type` effectively act as defaults.) permissible to call the object deallocator directly instead of via :c:member:`~PyTypeObject.tp_free`. The object deallocator should be the one used to allocate the instance; this is normally :c:func:`PyObject_Del` if the instance was allocated - using :c:func:`PyObject_New` or :c:func:`PyObject_VarNew`, or + using :c:macro:`PyObject_New` or :c:macro:`PyObject_NewVar`, or :c:func:`PyObject_GC_Del` if the instance was allocated using - :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar`. + :c:macro:`PyObject_GC_New` or :c:macro:`PyObject_GC_NewVar`. If the type supports garbage collection (has the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit set), the destructor should call :c:func:`PyObject_GC_UnTrack` @@ -1092,7 +1092,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) .. c:macro:: Py_TPFLAGS_HAVE_GC This bit is set when the object supports garbage collection. If this bit - is set, instances must be created using :c:func:`PyObject_GC_New` and + is set, instances must be created using :c:macro:`PyObject_GC_New` and destroyed using :c:func:`PyObject_GC_Del`. More information in section :ref:`supporting-cycle-detection`. This bit also implies that the GC-related fields :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` are present in @@ -1180,7 +1180,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) Indicates that the variable-sized portion of an instance of this type is at the end of the instance's memory area, at an offset of - :c:expr:`Py_TYPE(obj)->tp_basicsize` (which may be different in each + ``Py_TYPE(obj)->tp_basicsize`` (which may be different in each subclass). When setting this flag, be sure that all superclasses either diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index fb4cfc64c8d00..02f9597c345a0 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -608,7 +608,7 @@ APIs: Py_ssize_t how_many) Copy characters from one Unicode object into another. This function performs - character conversion when necessary and falls back to :c:func:`memcpy` if + character conversion when necessary and falls back to :c:func:`!memcpy` if possible. Returns ``-1`` and sets an exception on error, otherwise returns the number of copied characters. @@ -721,7 +721,7 @@ system. .. c:function:: PyObject* PyUnicode_DecodeLocale(const char *str, const char *errors) Similar to :c:func:`PyUnicode_DecodeLocaleAndSize`, but compute the string - length using :c:func:`strlen`. + length using :c:func:`!strlen`. .. versionadded:: 3.3 @@ -879,7 +879,7 @@ wchar_t Support most C functions. If *size* is ``NULL`` and the :c:expr:`wchar_t*` string contains null characters a :exc:`ValueError` is raised. - Returns a buffer allocated by :c:func:`PyMem_New` (use + Returns a buffer allocated by :c:macro:`PyMem_New` (use :c:func:`PyMem_Free` to free it) on success. On error, returns ``NULL`` and *\*size* is undefined. Raises a :exc:`MemoryError` if memory allocation is failed. diff --git a/Doc/conf.py b/Doc/conf.py index 722831cdb54c3..a8fd853ebb1f0 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -97,7 +97,7 @@ ('c:func', 'vsnprintf'), # Standard C types ('c:type', 'FILE'), - ('c:type', '__int'), + ('c:type', 'int64_t'), ('c:type', 'intmax_t'), ('c:type', 'off_t'), ('c:type', 'ptrdiff_t'), @@ -105,9 +105,12 @@ ('c:type', 'size_t'), ('c:type', 'ssize_t'), ('c:type', 'time_t'), + ('c:type', 'uint64_t'), ('c:type', 'uintmax_t'), + ('c:type', 'uintptr_t'), ('c:type', 'va_list'), ('c:type', 'wchar_t'), + # Standard C structures ('c:struct', 'in6_addr'), ('c:struct', 'in_addr'), ('c:struct', 'stat'), diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 068f6fcdcaccd..c13b9371297ee 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -230,7 +230,7 @@ with an exception object:: return m; } -Note that the Python name for the exception object is :exc:`spam.error`. The +Note that the Python name for the exception object is :exc:`!spam.error`. The :c:func:`PyErr_NewException` function may create a class with the base class being :exc:`Exception` (unless another class is passed in instead of ``NULL``), described in :ref:`bltin-exceptions`. @@ -245,7 +245,7 @@ raises the exception could cause a core dump or other unintended side effects. We discuss the use of ``PyMODINIT_FUNC`` as a function return type later in this sample. -The :exc:`spam.error` exception can be raised in your extension module using a +The :exc:`!spam.error` exception can be raised in your extension module using a call to :c:func:`PyErr_SetString` as shown below:: static PyObject * @@ -315,7 +315,7 @@ contexts, as we have seen. The Module's Method Table and Initialization Function ===================================================== -I promised to show how :c:func:`spam_system` is called from Python programs. +I promised to show how :c:func:`!spam_system` is called from Python programs. First, we need to list its name and address in a "method table":: static PyMethodDef SpamMethods[] = { @@ -1030,13 +1030,13 @@ Let's follow the control flow into :c:func:`PyList_SetItem`. The list owns references to all its items, so when item 1 is replaced, it has to dispose of the original item 1. Now let's suppose the original item 1 was an instance of a user-defined class, and let's further suppose that the class defined a -:meth:`__del__` method. If this class instance has a reference count of 1, -disposing of it will call its :meth:`__del__` method. +:meth:`!__del__` method. If this class instance has a reference count of 1, +disposing of it will call its :meth:`!__del__` method. -Since it is written in Python, the :meth:`__del__` method can execute arbitrary +Since it is written in Python, the :meth:`!__del__` method can execute arbitrary Python code. Could it perhaps do something to invalidate the reference to -``item`` in :c:func:`bug`? You bet! Assuming that the list passed into -:c:func:`bug` is accessible to the :meth:`__del__` method, it could execute a +``item`` in :c:func:`!bug`? You bet! Assuming that the list passed into +:c:func:`!bug` is accessible to the :meth:`!__del__` method, it could execute a statement to the effect of ``del list[0]``, and assuming this was the last reference to that object, it would free the memory associated with it, thereby invalidating ``item``. @@ -1057,7 +1057,7 @@ increment the reference count. The correct version of the function reads:: This is a true story. An older version of Python contained variants of this bug and someone spent a considerable amount of time in a C debugger to figure out -why his :meth:`__del__` methods would fail... +why his :meth:`!__del__` methods would fail... The second case of problems with a borrowed reference is a variant involving threads. Normally, multiple threads in the Python interpreter can't get in each @@ -1224,7 +1224,7 @@ The function :c:func:`PySpam_System` is a plain C function, declared return system(command); } -The function :c:func:`spam_system` is modified in a trivial way:: +The function :c:func:`!spam_system` is modified in a trivial way:: static PyObject * spam_system(PyObject *self, PyObject *args) diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index 80c8ea73e343e..7180191d40f18 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -36,7 +36,7 @@ So, if you want to define a new extension type, you need to create a new type object. This sort of thing can only be explained by example, so here's a minimal, but -complete, module that defines a new type named :class:`Custom` inside a C +complete, module that defines a new type named :class:`!Custom` inside a C extension module :mod:`!custom`: .. note:: @@ -50,9 +50,9 @@ extension module :mod:`!custom`: Now that's quite a bit to take in at once, but hopefully bits will seem familiar from the previous chapter. This file defines three things: -#. What a :class:`Custom` **object** contains: this is the ``CustomObject`` - struct, which is allocated once for each :class:`Custom` instance. -#. How the :class:`Custom` **type** behaves: this is the ``CustomType`` struct, +#. What a :class:`!Custom` **object** contains: this is the ``CustomObject`` + struct, which is allocated once for each :class:`!Custom` instance. +#. How the :class:`!Custom` **type** behaves: this is the ``CustomType`` struct, which defines a set of flags and function pointers that the interpreter inspects when specific operations are requested. #. How to initialize the :mod:`!custom` module: this is the ``PyInit_custom`` @@ -128,7 +128,7 @@ our objects and in some error messages, for example: Note that the name is a dotted name that includes both the module name and the name of the type within the module. The module in this case is :mod:`!custom` and -the type is :class:`Custom`, so we set the type name to :class:`custom.Custom`. +the type is :class:`!Custom`, so we set the type name to :class:`!custom.Custom`. Using the real dotted import path is important to make your type compatible with the :mod:`pydoc` and :mod:`pickle` modules. :: @@ -136,7 +136,7 @@ with the :mod:`pydoc` and :mod:`pickle` modules. :: .tp_itemsize = 0, This is so that Python knows how much memory to allocate when creating -new :class:`Custom` instances. :c:member:`~PyTypeObject.tp_itemsize` is +new :class:`!Custom` instances. :c:member:`~PyTypeObject.tp_itemsize` is only used for variable-sized objects and should otherwise be zero. .. note:: @@ -188,7 +188,7 @@ set to ``NULL``. :: } This adds the type to the module dictionary. This allows us to create -:class:`Custom` instances by calling the :class:`Custom` class: +:class:`!Custom` instances by calling the :class:`!Custom` class: .. code-block:: pycon @@ -239,7 +239,7 @@ adds these capabilities: This version of the module has a number of changes. -The :class:`Custom` type now has three data attributes in its C struct, +The :class:`!Custom` type now has three data attributes in its C struct, *first*, *last*, and *number*. The *first* and *last* variables are Python strings containing first and last names. The *number* attribute is a C integer. @@ -314,7 +314,7 @@ The ``tp_new`` handler is responsible for creating (as opposed to initializing) objects of the type. It is exposed in Python as the :meth:`__new__` method. It is not required to define a ``tp_new`` member, and indeed many extension types will simply reuse :c:func:`PyType_GenericNew` as done in the first -version of the ``Custom`` type above. In this case, we use the ``tp_new`` +version of the :class:`!Custom` type above. In this case, we use the ``tp_new`` handler to initialize the ``first`` and ``last`` attributes to non-``NULL`` default values. @@ -453,7 +453,7 @@ Further, the attributes can be deleted, setting the C pointers to ``NULL``. Eve though we can make sure the members are initialized to non-``NULL`` values, the members can be set to ``NULL`` if the attributes are deleted. -We define a single method, :meth:`Custom.name()`, that outputs the objects name as the +We define a single method, :meth:`!Custom.name()`, that outputs the objects name as the concatenation of the first and last names. :: static PyObject * @@ -470,8 +470,8 @@ concatenation of the first and last names. :: return PyUnicode_FromFormat("%S %S", self->first, self->last); } -The method is implemented as a C function that takes a :class:`Custom` (or -:class:`Custom` subclass) instance as the first argument. Methods always take an +The method is implemented as a C function that takes a :class:`!Custom` (or +:class:`!Custom` subclass) instance as the first argument. Methods always take an instance as the first argument. Methods often take positional and keyword arguments as well, but in this case we don't take any and don't need to accept a positional argument tuple or keyword argument dictionary. This method is @@ -482,8 +482,8 @@ equivalent to the Python method: def name(self): return "%s %s" % (self.first, self.last) -Note that we have to check for the possibility that our :attr:`first` and -:attr:`last` members are ``NULL``. This is because they can be deleted, in which +Note that we have to check for the possibility that our :attr:`!first` and +:attr:`!last` members are ``NULL``. This is because they can be deleted, in which case they are set to ``NULL``. It would be better to prevent deletion of these attributes and to restrict the attribute values to be strings. We'll see how to do that in the next section. @@ -512,7 +512,7 @@ to add the :c:macro:`Py_TPFLAGS_BASETYPE` to our class flag definition:: .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, -We rename :c:func:`PyInit_custom` to :c:func:`PyInit_custom2`, update the +We rename :c:func:`!PyInit_custom` to :c:func:`!PyInit_custom2`, update the module name in the :c:type:`PyModuleDef` struct, and update the full class name in the :c:type:`PyTypeObject` struct. @@ -531,18 +531,18 @@ Finally, we update our :file:`setup.py` file to build the new module: Providing finer control over data attributes ============================================ -In this section, we'll provide finer control over how the :attr:`first` and -:attr:`last` attributes are set in the :class:`Custom` example. In the previous -version of our module, the instance variables :attr:`first` and :attr:`last` +In this section, we'll provide finer control over how the :attr:`!first` and +:attr:`!last` attributes are set in the :class:`!Custom` example. In the previous +version of our module, the instance variables :attr:`!first` and :attr:`!last` could be set to non-string values or even deleted. We want to make sure that these attributes always contain strings. .. literalinclude:: ../includes/custom3.c -To provide greater control, over the :attr:`first` and :attr:`last` attributes, +To provide greater control, over the :attr:`!first` and :attr:`!last` attributes, we'll use custom getter and setter functions. Here are the functions for -getting and setting the :attr:`first` attribute:: +getting and setting the :attr:`!first` attribute:: static PyObject * Custom_getfirst(CustomObject *self, void *closure) @@ -571,13 +571,13 @@ getting and setting the :attr:`first` attribute:: return 0; } -The getter function is passed a :class:`Custom` object and a "closure", which is +The getter function is passed a :class:`!Custom` object and a "closure", which is a void pointer. In this case, the closure is ignored. (The closure supports an advanced usage in which definition data is passed to the getter and setter. This could, for example, be used to allow a single set of getter and setter functions that decide the attribute to get or set based on data in the closure.) -The setter function is passed the :class:`Custom` object, the new value, and the +The setter function is passed the :class:`!Custom` object, the new value, and the closure. The new value may be ``NULL``, in which case the attribute is being deleted. In our setter, we raise an error if the attribute is deleted or if its new value is not a string. @@ -666,11 +666,11 @@ still has a reference from itself. Its reference count doesn't drop to zero. Fortunately, Python's cyclic garbage collector will eventually figure out that the list is garbage and free it. -In the second version of the :class:`Custom` example, we allowed any kind of -object to be stored in the :attr:`first` or :attr:`last` attributes [#]_. +In the second version of the :class:`!Custom` example, we allowed any kind of +object to be stored in the :attr:`!first` or :attr:`!last` attributes [#]_. Besides, in the second and third versions, we allowed subclassing -:class:`Custom`, and subclasses may add arbitrary attributes. For any of -those two reasons, :class:`Custom` objects can participate in cycles: +:class:`!Custom`, and subclasses may add arbitrary attributes. For any of +those two reasons, :class:`!Custom` objects can participate in cycles: .. code-block:: pycon @@ -680,8 +680,8 @@ those two reasons, :class:`Custom` objects can participate in cycles: >>> n = Derived() >>> n.some_attribute = n -To allow a :class:`Custom` instance participating in a reference cycle to -be properly detected and collected by the cyclic GC, our :class:`Custom` type +To allow a :class:`!Custom` instance participating in a reference cycle to +be properly detected and collected by the cyclic GC, our :class:`!Custom` type needs to fill two additional slots and to enable a flag that enables these slots: .. literalinclude:: ../includes/custom4.c @@ -811,7 +811,7 @@ increases an internal counter: .. literalinclude:: ../includes/sublist.c -As you can see, the source code closely resembles the :class:`Custom` examples in +As you can see, the source code closely resembles the :class:`!Custom` examples in previous sections. We will break down the main differences between them. :: typedef struct { @@ -879,7 +879,7 @@ slot with :c:func:`PyType_GenericNew` -- the allocation function from the base type will be inherited. After that, calling :c:func:`PyType_Ready` and adding the type object to the -module is the same as with the basic :class:`Custom` examples. +module is the same as with the basic :class:`!Custom` examples. .. rubric:: Footnotes diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index b02f6eaea1325..ab906b3817670 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -3,13 +3,10 @@ # Add blank lines between files and keep them sorted lexicographically # to help avoid merge conflicts. -Doc/c-api/allocation.rst Doc/c-api/arg.rst Doc/c-api/bool.rst Doc/c-api/buffer.rst Doc/c-api/capsule.rst -Doc/c-api/complex.rst -Doc/c-api/conversion.rst Doc/c-api/datetime.rst Doc/c-api/descriptor.rst Doc/c-api/dict.rst diff --git a/Doc/whatsnew/2.3.rst b/Doc/whatsnew/2.3.rst index 14f951174ba54..3dfe509e1d1c0 100644 --- a/Doc/whatsnew/2.3.rst +++ b/Doc/whatsnew/2.3.rst @@ -1847,7 +1847,7 @@ specifically for allocating Python objects. :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`, and :c:func:`PyObject_Free`. * To allocate and free Python objects, use the "object" family - :c:func:`PyObject_New`, :c:func:`PyObject_NewVar`, and :c:func:`PyObject_Del`. + :c:macro:`PyObject_New`, :c:macro:`PyObject_NewVar`, and :c:func:`PyObject_Del`. Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides debugging features to catch memory overwrites and doubled frees in both extension modules diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 5df2cef7e898a..98d97268b5901 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -2061,8 +2061,8 @@ Changes in the C API * Remove :c:macro:`Py_INCREF` on the type object after allocating an instance - if any. - This may happen after calling :c:func:`PyObject_New`, - :c:func:`PyObject_NewVar`, :c:func:`PyObject_GC_New`, + This may happen after calling :c:macro:`PyObject_New`, + :c:macro:`PyObject_NewVar`, :c:func:`PyObject_GC_New`, :c:func:`PyObject_GC_NewVar`, or any other custom allocator that uses :c:func:`PyObject_Init` or :c:func:`PyObject_INIT`. diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 6de432d6c036b..666d53ed6da7b 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1389,8 +1389,8 @@ Porting to Python 3.9 * :c:func:`PyObject_IS_GC` macro was converted to a function. * The :c:func:`PyObject_NEW` macro becomes an alias to the - :c:func:`PyObject_New` macro, and the :c:func:`PyObject_NEW_VAR` macro - becomes an alias to the :c:func:`PyObject_NewVar` macro. They no longer + :c:macro:`PyObject_New` macro, and the :c:func:`PyObject_NEW_VAR` macro + becomes an alias to the :c:macro:`PyObject_NewVar` macro. They no longer access directly the :c:member:`PyTypeObject.tp_basicsize` member. * :c:func:`PyObject_GET_WEAKREFS_LISTPTR` macro was converted to a function: From webhook-mailer at python.org Fri Jul 28 03:03:52 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 07:03:52 -0000 Subject: [Python-checkins] [3.11] gh-107298: Fix more Sphinx warnings in the C API doc (GH-107329) (GH-107377) Message-ID: https://github.com/python/cpython/commit/32e17d4a3c1da2a8ff082b5cb4f3cd13688a95a1 commit: 32e17d4a3c1da2a8ff082b5cb4f3cd13688a95a1 branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-28T07:03:47Z summary: [3.11] gh-107298: Fix more Sphinx warnings in the C API doc (GH-107329) (GH-107377) Declare the following functions as macros, since they are actually macros. It avoids a warning on "TYPE" or "macro" argument. * PyMem_New() * PyMem_Resize() * PyModule_AddIntMacro() * PyModule_AddStringMacro() * PyObject_GC_New() * PyObject_GC_NewVar() * PyObject_New() * PyObject_NewVar() (cherry picked from commit 8d61a71f9c81619e34d4a30b625922ebc83c561b) Co-authored-by: Victor Stinner files: M Doc/c-api/allocation.rst M Doc/c-api/capsule.rst M Doc/c-api/complex.rst M Doc/c-api/conversion.rst M Doc/c-api/exceptions.rst M Doc/c-api/gcsupport.rst M Doc/c-api/import.rst M Doc/c-api/init.rst M Doc/c-api/init_config.rst M Doc/c-api/memory.rst M Doc/c-api/module.rst M Doc/c-api/set.rst M Doc/c-api/structures.rst M Doc/c-api/sys.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/conf.py M Doc/extending/extending.rst M Doc/extending/newtypes_tutorial.rst M Doc/whatsnew/2.3.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst diff --git a/Doc/c-api/allocation.rst b/Doc/c-api/allocation.rst index 0a8fcc5ae5fcd..44747e2964366 100644 --- a/Doc/c-api/allocation.rst +++ b/Doc/c-api/allocation.rst @@ -27,22 +27,25 @@ Allocating Objects on the Heap length information for a variable-size object. -.. c:function:: TYPE* PyObject_New(TYPE, PyTypeObject *type) +.. c:macro:: PyObject_New(TYPE, typeobj) Allocate a new Python object using the C structure type *TYPE* and the - Python type object *type*. Fields not defined by the Python object header + Python type object *typeobj* (``PyTypeObject*``). + Fields not defined by the Python object header are not initialized; the object's reference count will be one. The size of the memory allocation is determined from the :c:member:`~PyTypeObject.tp_basicsize` field of the type object. -.. c:function:: TYPE* PyObject_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) +.. c:macro:: PyObject_NewVar(TYPE, typeobj, size) Allocate a new Python object using the C structure type *TYPE* and the - Python type object *type*. Fields not defined by the Python object header + Python type object *typeobj* (``PyTypeObject*``). + Fields not defined by the Python object header are not initialized. The allocated memory allows for the *TYPE* structure - plus *size* fields of the size given by the :c:member:`~PyTypeObject.tp_itemsize` field of - *type*. This is useful for implementing objects like tuples, which are + plus *size* (``Py_ssize_t``) fields of the size + given by the :c:member:`~PyTypeObject.tp_itemsize` field of + *typeobj*. This is useful for implementing objects like tuples, which are able to determine their size at construction time. Embedding the array of fields into the same allocation decreases the number of allocations, improving the memory management efficiency. @@ -50,8 +53,8 @@ Allocating Objects on the Heap .. c:function:: void PyObject_Del(void *op) - Releases memory allocated to an object using :c:func:`PyObject_New` or - :c:func:`PyObject_NewVar`. This is normally called from the + Releases memory allocated to an object using :c:macro:`PyObject_New` or + :c:macro:`PyObject_NewVar`. This is normally called from the :c:member:`~PyTypeObject.tp_dealloc` handler specified in the object's type. The fields of the object should not be accessed after this call as the memory is no longer a valid Python object. diff --git a/Doc/c-api/capsule.rst b/Doc/c-api/capsule.rst index 427ed959c5856..2a1b602dc79c0 100644 --- a/Doc/c-api/capsule.rst +++ b/Doc/c-api/capsule.rst @@ -64,7 +64,7 @@ Refer to :ref:`using-capsules` for more information on using these objects. The *name* parameter must compare exactly to the name stored in the capsule. If the name stored in the capsule is ``NULL``, the *name* passed in must also - be ``NULL``. Python uses the C function :c:func:`strcmp` to compare capsule + be ``NULL``. Python uses the C function :c:func:`!strcmp` to compare capsule names. diff --git a/Doc/c-api/complex.rst b/Doc/c-api/complex.rst index 6679ce76f1dc6..e3fd001c599c8 100644 --- a/Doc/c-api/complex.rst +++ b/Doc/c-api/complex.rst @@ -64,7 +64,7 @@ pointers. This is consistent throughout the API. representation. If *divisor* is null, this method returns zero and sets - :c:data:`errno` to :c:macro:`EDOM`. + :c:data:`errno` to :c:macro:`!EDOM`. .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp) @@ -73,7 +73,7 @@ pointers. This is consistent throughout the API. representation. If *num* is null and *exp* is not a positive real number, - this method returns zero and sets :c:data:`errno` to :c:macro:`EDOM`. + this method returns zero and sets :c:data:`errno` to :c:macro:`!EDOM`. Complex Numbers as Python Objects diff --git a/Doc/c-api/conversion.rst b/Doc/c-api/conversion.rst index fdb321fe7ab3f..c5350123dfdfd 100644 --- a/Doc/c-api/conversion.rst +++ b/Doc/c-api/conversion.rst @@ -119,10 +119,10 @@ The following functions provide locale-independent string to number conversions. .. c:function:: int PyOS_stricmp(const char *s1, const char *s2) Case insensitive comparison of strings. The function works almost - identically to :c:func:`strcmp` except that it ignores the case. + identically to :c:func:`!strcmp` except that it ignores the case. .. c:function:: int PyOS_strnicmp(const char *s1, const char *s2, Py_ssize_t size) Case insensitive comparison of strings. The function works almost - identically to :c:func:`strncmp` except that it ignores the case. + identically to :c:func:`!strncmp` except that it ignores the case. diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 6ca53bd29f9e8..8d8b0ac7dd9a9 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -78,7 +78,7 @@ Printing and clearing This utility function prints a warning message to ``sys.stderr`` when an exception has been set but it is impossible for the interpreter to actually raise the exception. It is used, for example, when an exception occurs in an - :meth:`__del__` method. + :meth:`~object.__del__` method. The function is called with a single argument *obj* that identifies the context in which the unraisable exception occurred. If possible, @@ -154,7 +154,7 @@ For convenience, some of these functions will always return a tuple object whose first item is the integer :c:data:`errno` value and whose second item is the corresponding error message (gotten from :c:func:`!strerror`), and then calls ``PyErr_SetObject(type, object)``. On Unix, when the - :c:data:`errno` value is :c:macro:`EINTR`, indicating an interrupted system call, + :c:data:`errno` value is :c:macro:`!EINTR`, indicating an interrupted system call, this calls :c:func:`PyErr_CheckSignals`, and if that set the error indicator, leaves it set to that. The function always returns ``NULL``, so a wrapper function around a system call can write ``return PyErr_SetFromErrno(type);`` @@ -166,7 +166,7 @@ For convenience, some of these functions will always return a Similar to :c:func:`PyErr_SetFromErrno`, with the additional behavior that if *filenameObject* is not ``NULL``, it is passed to the constructor of *type* as a third parameter. In the case of :exc:`OSError` exception, - this is used to define the :attr:`filename` attribute of the + this is used to define the :attr:`!filename` attribute of the exception instance. @@ -189,12 +189,12 @@ For convenience, some of these functions will always return a .. c:function:: PyObject* PyErr_SetFromWindowsErr(int ierr) This is a convenience function to raise :exc:`WindowsError`. If called with - *ierr* of ``0``, the error code returned by a call to :c:func:`GetLastError` - is used instead. It calls the Win32 function :c:func:`FormatMessage` to retrieve - the Windows description of error code given by *ierr* or :c:func:`GetLastError`, + *ierr* of ``0``, the error code returned by a call to :c:func:`!GetLastError` + is used instead. It calls the Win32 function :c:func:`!FormatMessage` to retrieve + the Windows description of error code given by *ierr* or :c:func:`!GetLastError`, then it constructs a tuple object whose first item is the *ierr* value and whose second item is the corresponding error message (gotten from - :c:func:`FormatMessage`), and then calls ``PyErr_SetObject(PyExc_WindowsError, + :c:func:`!FormatMessage`), and then calls ``PyErr_SetObject(PyExc_WindowsError, object)``. This function always returns ``NULL``. .. availability:: Windows. @@ -567,7 +567,7 @@ Signal Handling be interruptible by user requests (such as by pressing Ctrl-C). .. note:: - The default Python signal handler for :c:macro:`SIGINT` raises the + The default Python signal handler for :c:macro:`!SIGINT` raises the :exc:`KeyboardInterrupt` exception. @@ -578,7 +578,7 @@ Signal Handling single: SIGINT single: KeyboardInterrupt (built-in exception) - Simulate the effect of a :c:macro:`SIGINT` signal arriving. + Simulate the effect of a :c:macro:`!SIGINT` signal arriving. This is equivalent to ``PyErr_SetInterruptEx(SIGINT)``. .. note:: diff --git a/Doc/c-api/gcsupport.rst b/Doc/c-api/gcsupport.rst index ac938735a67dd..6bdbb008f8e74 100644 --- a/Doc/c-api/gcsupport.rst +++ b/Doc/c-api/gcsupport.rst @@ -25,8 +25,8 @@ include the :c:macro:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the Constructors for container types must conform to two rules: -#. The memory for the object must be allocated using :c:func:`PyObject_GC_New` - or :c:func:`PyObject_GC_NewVar`. +#. The memory for the object must be allocated using :c:macro:`PyObject_GC_New` + or :c:macro:`PyObject_GC_NewVar`. #. Once all the fields which may contain references to other containers are initialized, it must call :c:func:`PyObject_GC_Track`. @@ -52,21 +52,21 @@ rules: class that implements the garbage collector protocol and the child class does *not* include the :c:macro:`Py_TPFLAGS_HAVE_GC` flag. -.. c:function:: TYPE* PyObject_GC_New(TYPE, PyTypeObject *type) +.. c:macro:: PyObject_GC_New(TYPE, typeobj) - Analogous to :c:func:`PyObject_New` but for container objects with the + Analogous to :c:macro:`PyObject_New` but for container objects with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. -.. c:function:: TYPE* PyObject_GC_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) +.. c:macro:: PyObject_GC_NewVar(TYPE, typeobj, size) - Analogous to :c:func:`PyObject_NewVar` but for container objects with the + Analogous to :c:macro:`PyObject_NewVar` but for container objects with the :c:macro:`Py_TPFLAGS_HAVE_GC` flag set. .. c:function:: TYPE* PyObject_GC_Resize(TYPE, PyVarObject *op, Py_ssize_t newsize) - Resize an object allocated by :c:func:`PyObject_NewVar`. Returns the + Resize an object allocated by :c:macro:`PyObject_NewVar`. Returns the resized object or ``NULL`` on failure. *op* must not be tracked by the collector yet. @@ -109,8 +109,8 @@ rules: .. c:function:: void PyObject_GC_Del(void *op) - Releases memory allocated to an object using :c:func:`PyObject_GC_New` or - :c:func:`PyObject_GC_NewVar`. + Releases memory allocated to an object using :c:macro:`PyObject_GC_New` or + :c:macro:`PyObject_GC_NewVar`. .. c:function:: void PyObject_GC_UnTrack(void *op) diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 887c1793c3ec7..f2a60bdb02364 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -135,10 +135,10 @@ Importing Modules The module's :attr:`__spec__` and :attr:`__loader__` will be set, if not set already, with the appropriate values. The spec's loader will be set to the module's ``__loader__`` (if set) and to an instance of - :class:`SourceFileLoader` otherwise. + :class:`~importlib.machinery.SourceFileLoader` otherwise. The module's :attr:`__file__` attribute will be set to the code object's - :attr:`co_filename`. If applicable, :attr:`__cached__` will also + :attr:`!co_filename`. If applicable, :attr:`__cached__` will also be set. This function will reload the module if it was already imported. See @@ -214,7 +214,7 @@ Importing Modules .. c:function:: PyObject* PyImport_GetImporter(PyObject *path) - Return a finder object for a :data:`sys.path`/:attr:`pkg.__path__` item + Return a finder object for a :data:`sys.path`/:attr:`!pkg.__path__` item *path*, possibly by fetching it from the :data:`sys.path_importer_cache` dict. If it wasn't yet cached, traverse :data:`sys.path_hooks` until a hook is found that can handle the path item. Return ``None`` if no hook could; diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 05b9b77e4e7c9..88f5944db97c4 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -25,7 +25,7 @@ The following functions can be safely called before Python is initialized: * :c:func:`PyImport_AppendInittab` * :c:func:`PyImport_ExtendInittab` - * :c:func:`PyInitFrozenExtensions` + * :c:func:`!PyInitFrozenExtensions` * :c:func:`PyMem_SetAllocator` * :c:func:`PyMem_SetupDebugHooks` * :c:func:`PyObject_SetArenaAllocator` @@ -122,7 +122,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. c:var:: int Py_IgnoreEnvironmentFlag - Ignore all :envvar:`PYTHON*` environment variables, e.g. + Ignore all :envvar:`!PYTHON*` environment variables, e.g. :envvar:`PYTHONPATH` and :envvar:`PYTHONHOME`, that might be set. Set by the :option:`-E` and :option:`-I` options. @@ -165,7 +165,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. c:var:: int Py_LegacyWindowsStdioFlag If the flag is non-zero, use :class:`io.FileIO` instead of - :class:`WindowsConsoleIO` for :mod:`sys` standard streams. + :class:`!io._WindowsConsoleIO` for :mod:`sys` standard streams. Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSSTDIO` environment variable is set to a non-empty string. @@ -292,7 +292,7 @@ Initializing and finalizing the interpreter the application. **Bugs and caveats:** The destruction of modules and objects in modules is done - in random order; this may cause destructors (:meth:`__del__` methods) to fail + in random order; this may cause destructors (:meth:`~object.__del__` methods) to fail when they depend on other objects (even functions) or modules. Dynamically loaded extension modules loaded by Python are not unloaded. Small amounts of memory allocated by the Python interpreter may not be freed (if you find a leak, @@ -381,7 +381,7 @@ Process-wide parameters .. deprecated:: 3.11 -.. c:function:: wchar* Py_GetProgramName() +.. c:function:: wchar_t* Py_GetProgramName() .. index:: single: Py_SetProgramName() @@ -872,7 +872,7 @@ the fork, and releasing them afterwards. In addition, it resets any :ref:`lock-objects` in the child. When extending or embedding Python, there is no way to inform Python of additional (non-Python) locks that need to be acquired before or reset after a fork. OS facilities such as -:c:func:`pthread_atfork` would need to be used to accomplish the same thing. +:c:func:`!pthread_atfork` would need to be used to accomplish the same thing. Additionally, when extending or embedding Python, calling :c:func:`fork` directly rather than through :func:`os.fork` (and returning to or calling into Python) may result in a deadlock by one of Python's internal locks @@ -978,7 +978,7 @@ code, or when embedding the Python interpreter: .. note:: Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. - You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to + You can use :c:func:`!_Py_IsFinalizing` or :func:`sys.is_finalizing` to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination. @@ -1024,7 +1024,7 @@ with sub-interpreters: .. note:: Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. - You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to + You can use :c:func:`!_Py_IsFinalizing` or :func:`sys.is_finalizing` to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination. @@ -1306,7 +1306,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. note:: Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. - You can use :c:func:`_Py_IsFinalizing` or :func:`sys.is_finalizing` to + You can use :c:func:`!_Py_IsFinalizing` or :func:`sys.is_finalizing` to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination. diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 715ad47f13eb4..216755a1f948d 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -850,7 +850,7 @@ PyConfig .. c:member:: int legacy_windows_stdio If non-zero, use :class:`io.FileIO` instead of - :class:`io.WindowsConsoleIO` for :data:`sys.stdin`, :data:`sys.stdout` + :class:`!io._WindowsConsoleIO` for :data:`sys.stdin`, :data:`sys.stdout` and :data:`sys.stderr`. Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSSTDIO` environment @@ -1096,7 +1096,7 @@ PyConfig Set to ``0`` by the :option:`-S` command line option. - :data:`sys.flags.no_site` is set to the inverted value of + :data:`sys.flags.no_site ` is set to the inverted value of :c:member:`~PyConfig.site_import`. Default: ``1``. diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 4ca3b8804427f..c51aba3f55536 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -136,7 +136,7 @@ need to be held. The :ref:`default raw memory allocator ` uses the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` -and :c:func:`free`; call ``malloc(1)`` (or ``calloc(1, 1)``) when requesting +and :c:func:`!free`; call ``malloc(1)`` (or ``calloc(1, 1)``) when requesting zero bytes. .. versionadded:: 3.4 @@ -264,14 +264,14 @@ The following type-oriented macros are provided for convenience. Note that *TYPE* refers to any C type. -.. c:function:: TYPE* PyMem_New(TYPE, size_t n) +.. c:macro:: PyMem_New(TYPE, n) Same as :c:func:`PyMem_Malloc`, but allocates ``(n * sizeof(TYPE))`` bytes of memory. Returns a pointer cast to :c:expr:`TYPE*`. The memory will not have been initialized in any way. -.. c:function:: TYPE* PyMem_Resize(void *p, TYPE, size_t n) +.. c:macro:: PyMem_Resize(p, TYPE, n) Same as :c:func:`PyMem_Realloc`, but the memory block is resized to ``(n * sizeof(TYPE))`` bytes. Returns a pointer cast to :c:expr:`TYPE*`. On return, @@ -423,7 +423,7 @@ Customize Memory Allocators +----------------------------------------------------------+---------------------------------------+ .. versionchanged:: 3.5 - The :c:type:`PyMemAllocator` structure was renamed to + The :c:type:`!PyMemAllocator` structure was renamed to :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added. @@ -627,8 +627,8 @@ with a fixed size of 256 KiB. It falls back to :c:func:`PyMem_RawMalloc` and The arena allocator uses the following functions: -* :c:func:`VirtualAlloc` and :c:func:`VirtualFree` on Windows, -* :c:func:`mmap` and :c:func:`munmap` if available, +* :c:func:`!VirtualAlloc` and :c:func:`!VirtualFree` on Windows, +* :c:func:`!mmap` and :c:func:`!munmap` if available, * :c:func:`malloc` and :c:func:`free` otherwise. This allocator is disabled if Python is configured with the @@ -732,8 +732,8 @@ allocators operating on different heaps. :: free(buf1); /* Fatal -- should be PyMem_Del() */ In addition to the functions aimed at handling raw memory blocks from the Python -heap, objects in Python are allocated and released with :c:func:`PyObject_New`, -:c:func:`PyObject_NewVar` and :c:func:`PyObject_Del`. +heap, objects in Python are allocated and released with :c:macro:`PyObject_New`, +:c:macro:`PyObject_NewVar` and :c:func:`PyObject_Del`. These will be explained in the next chapter on defining and implementing new object types in C. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index 6abd550ab80b2..fc4dd4ce053e6 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -282,7 +282,7 @@ An alternate way to specify extensions is to request "multi-phase initialization Extension modules created this way behave more like Python modules: the initialization is split between the *creation phase*, when the module object is created, and the *execution phase*, when it is populated. -The distinction is similar to the :py:meth:`__new__` and :py:meth:`__init__` methods +The distinction is similar to the :py:meth:`!__new__` and :py:meth:`!__init__` methods of classes. Unlike modules created using single-phase initialization, these modules are not @@ -293,7 +293,7 @@ By default, multiple modules created from the same definition should be independent: changes to one should not affect the others. This means that all state should be specific to the module object (using e.g. using :c:func:`PyModule_GetState`), or its contents (such as the module's -:attr:`__dict__` or individual classes created with :c:func:`PyType_FromSpec`). +:attr:`~object.__dict__` or individual classes created with :c:func:`PyType_FromSpec`). All modules created using multi-phase initialization are expected to support :ref:`sub-interpreters `. Making sure multiple modules @@ -555,7 +555,7 @@ state: ``NULL``-terminated. Return ``-1`` on error, ``0`` on success. -.. c:function:: int PyModule_AddIntMacro(PyObject *module, macro) +.. c:macro:: PyModule_AddIntMacro(module, macro) Add an int constant to *module*. The name and the value are taken from *macro*. For example ``PyModule_AddIntMacro(module, AF_INET)`` adds the int @@ -563,7 +563,7 @@ state: Return ``-1`` on error, ``0`` on success. -.. c:function:: int PyModule_AddStringMacro(PyObject *module, macro) +.. c:macro:: PyModule_AddStringMacro(module, macro) Add a string constant to *module*. diff --git a/Doc/c-api/set.rst b/Doc/c-api/set.rst index d642a5f1902e2..7e0ebd2f791a4 100644 --- a/Doc/c-api/set.rst +++ b/Doc/c-api/set.rst @@ -122,7 +122,7 @@ or :class:`frozenset` or instances of their subtypes. .. c:function:: int PySet_Contains(PyObject *anyset, PyObject *key) Return ``1`` if found, ``0`` if not found, and ``-1`` if an error is encountered. Unlike - the Python :meth:`__contains__` method, this function does not automatically + the Python :meth:`~object.__contains__` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise a :exc:`TypeError` if the *key* is unhashable. Raise :exc:`PyExc_SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index b892a08ea0837..c2903b9ec3285 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -412,7 +412,7 @@ definition with the same method name. *METH_COEXIST*, the default is to skip repeated definitions. Since slot wrappers are loaded before the method table, the existence of a *sq_contains* slot, for example, would generate a wrapped method named - :meth:`__contains__` and preclude the loading of a corresponding + :meth:`~object.__contains__` and preclude the loading of a corresponding PyCFunction with the same name. With the flag defined, the PyCFunction will be loaded in place of the wrapper object and will co-exist with the slot. This is helpful because calls to PyCFunctions are optimized more diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 57a36c038b592..34028d6c87e97 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -412,7 +412,7 @@ Process Control This function should only be invoked when a condition is detected that would make it dangerous to continue using the Python interpreter; e.g., when the object administration appears to be corrupted. On Unix, the standard C library - function :c:func:`abort` is called which will attempt to produce a :file:`core` + function :c:func:`!abort` is called which will attempt to produce a :file:`core` file. The ``Py_FatalError()`` function is replaced with a macro which logs diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index d4b3293ae59e2..3577e8dc84502 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -577,7 +577,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) name, followed by a dot, followed by the type name; for built-in types, it should be just the type name. If the module is a submodule of a package, the full package name is part of the full module name. For example, a type named - :class:`T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` + :class:`!T` defined in module :mod:`!M` in subpackage :mod:`!Q` in package :mod:`!P` should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``. For :ref:`dynamically allocated type objects `, @@ -671,9 +671,9 @@ and :c:data:`PyType_Type` effectively act as defaults.) permissible to call the object deallocator directly instead of via :c:member:`~PyTypeObject.tp_free`. The object deallocator should be the one used to allocate the instance; this is normally :c:func:`PyObject_Del` if the instance was allocated - using :c:func:`PyObject_New` or :c:func:`PyObject_VarNew`, or + using :c:macro:`PyObject_New` or :c:macro:`PyObject_NewVar`, or :c:func:`PyObject_GC_Del` if the instance was allocated using - :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar`. + :c:macro:`PyObject_GC_New` or :c:macro:`PyObject_GC_NewVar`. If the type supports garbage collection (has the :c:macro:`Py_TPFLAGS_HAVE_GC` flag bit set), the destructor should call :c:func:`PyObject_GC_UnTrack` @@ -1091,7 +1091,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) .. c:macro:: Py_TPFLAGS_HAVE_GC This bit is set when the object supports garbage collection. If this bit - is set, instances must be created using :c:func:`PyObject_GC_New` and + is set, instances must be created using :c:macro:`PyObject_GC_New` and destroyed using :c:func:`PyObject_GC_Del`. More information in section :ref:`supporting-cycle-detection`. This bit also implies that the GC-related fields :c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` are present in diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 1ed1f06f4595d..8b4252f08dd4c 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -613,7 +613,7 @@ APIs: Py_ssize_t how_many) Copy characters from one Unicode object into another. This function performs - character conversion when necessary and falls back to :c:func:`memcpy` if + character conversion when necessary and falls back to :c:func:`!memcpy` if possible. Returns ``-1`` and sets an exception on error, otherwise returns the number of copied characters. @@ -800,7 +800,7 @@ system. .. c:function:: PyObject* PyUnicode_DecodeLocale(const char *str, const char *errors) Similar to :c:func:`PyUnicode_DecodeLocaleAndSize`, but compute the string - length using :c:func:`strlen`. + length using :c:func:`!strlen`. .. versionadded:: 3.3 @@ -968,7 +968,7 @@ wchar_t Support most C functions. If *size* is ``NULL`` and the :c:expr:`wchar_t*` string contains null characters a :exc:`ValueError` is raised. - Returns a buffer allocated by :c:func:`PyMem_Alloc` (use + Returns a buffer allocated by :c:macro:`PyMem_New` (use :c:func:`PyMem_Free` to free it) on success. On error, returns ``NULL`` and *\*size* is undefined. Raises a :exc:`MemoryError` if memory allocation is failed. diff --git a/Doc/conf.py b/Doc/conf.py index c0e20c801f183..d9e7c9f11e7c8 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -92,7 +92,7 @@ ('c:func', 'vsnprintf'), # Standard C types ('c:type', 'FILE'), - ('c:type', '__int'), + ('c:type', 'int64_t'), ('c:type', 'intmax_t'), ('c:type', 'off_t'), ('c:type', 'ptrdiff_t'), @@ -100,9 +100,12 @@ ('c:type', 'size_t'), ('c:type', 'ssize_t'), ('c:type', 'time_t'), + ('c:type', 'uint64_t'), ('c:type', 'uintmax_t'), + ('c:type', 'uintptr_t'), ('c:type', 'va_list'), ('c:type', 'wchar_t'), + # Standard C structures ('c:struct', 'in6_addr'), ('c:struct', 'in_addr'), ('c:struct', 'stat'), diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 068f6fcdcaccd..c13b9371297ee 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -230,7 +230,7 @@ with an exception object:: return m; } -Note that the Python name for the exception object is :exc:`spam.error`. The +Note that the Python name for the exception object is :exc:`!spam.error`. The :c:func:`PyErr_NewException` function may create a class with the base class being :exc:`Exception` (unless another class is passed in instead of ``NULL``), described in :ref:`bltin-exceptions`. @@ -245,7 +245,7 @@ raises the exception could cause a core dump or other unintended side effects. We discuss the use of ``PyMODINIT_FUNC`` as a function return type later in this sample. -The :exc:`spam.error` exception can be raised in your extension module using a +The :exc:`!spam.error` exception can be raised in your extension module using a call to :c:func:`PyErr_SetString` as shown below:: static PyObject * @@ -315,7 +315,7 @@ contexts, as we have seen. The Module's Method Table and Initialization Function ===================================================== -I promised to show how :c:func:`spam_system` is called from Python programs. +I promised to show how :c:func:`!spam_system` is called from Python programs. First, we need to list its name and address in a "method table":: static PyMethodDef SpamMethods[] = { @@ -1030,13 +1030,13 @@ Let's follow the control flow into :c:func:`PyList_SetItem`. The list owns references to all its items, so when item 1 is replaced, it has to dispose of the original item 1. Now let's suppose the original item 1 was an instance of a user-defined class, and let's further suppose that the class defined a -:meth:`__del__` method. If this class instance has a reference count of 1, -disposing of it will call its :meth:`__del__` method. +:meth:`!__del__` method. If this class instance has a reference count of 1, +disposing of it will call its :meth:`!__del__` method. -Since it is written in Python, the :meth:`__del__` method can execute arbitrary +Since it is written in Python, the :meth:`!__del__` method can execute arbitrary Python code. Could it perhaps do something to invalidate the reference to -``item`` in :c:func:`bug`? You bet! Assuming that the list passed into -:c:func:`bug` is accessible to the :meth:`__del__` method, it could execute a +``item`` in :c:func:`!bug`? You bet! Assuming that the list passed into +:c:func:`!bug` is accessible to the :meth:`!__del__` method, it could execute a statement to the effect of ``del list[0]``, and assuming this was the last reference to that object, it would free the memory associated with it, thereby invalidating ``item``. @@ -1057,7 +1057,7 @@ increment the reference count. The correct version of the function reads:: This is a true story. An older version of Python contained variants of this bug and someone spent a considerable amount of time in a C debugger to figure out -why his :meth:`__del__` methods would fail... +why his :meth:`!__del__` methods would fail... The second case of problems with a borrowed reference is a variant involving threads. Normally, multiple threads in the Python interpreter can't get in each @@ -1224,7 +1224,7 @@ The function :c:func:`PySpam_System` is a plain C function, declared return system(command); } -The function :c:func:`spam_system` is modified in a trivial way:: +The function :c:func:`!spam_system` is modified in a trivial way:: static PyObject * spam_system(PyObject *self, PyObject *args) diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index 6bf63c8b9d7bf..e35175c982599 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -36,7 +36,7 @@ So, if you want to define a new extension type, you need to create a new type object. This sort of thing can only be explained by example, so here's a minimal, but -complete, module that defines a new type named :class:`Custom` inside a C +complete, module that defines a new type named :class:`!Custom` inside a C extension module :mod:`!custom`: .. note:: @@ -50,9 +50,9 @@ extension module :mod:`!custom`: Now that's quite a bit to take in at once, but hopefully bits will seem familiar from the previous chapter. This file defines three things: -#. What a :class:`Custom` **object** contains: this is the ``CustomObject`` - struct, which is allocated once for each :class:`Custom` instance. -#. How the :class:`Custom` **type** behaves: this is the ``CustomType`` struct, +#. What a :class:`!Custom` **object** contains: this is the ``CustomObject`` + struct, which is allocated once for each :class:`!Custom` instance. +#. How the :class:`!Custom` **type** behaves: this is the ``CustomType`` struct, which defines a set of flags and function pointers that the interpreter inspects when specific operations are requested. #. How to initialize the :mod:`!custom` module: this is the ``PyInit_custom`` @@ -128,7 +128,7 @@ our objects and in some error messages, for example: Note that the name is a dotted name that includes both the module name and the name of the type within the module. The module in this case is :mod:`!custom` and -the type is :class:`Custom`, so we set the type name to :class:`custom.Custom`. +the type is :class:`!Custom`, so we set the type name to :class:`!custom.Custom`. Using the real dotted import path is important to make your type compatible with the :mod:`pydoc` and :mod:`pickle` modules. :: @@ -136,7 +136,7 @@ with the :mod:`pydoc` and :mod:`pickle` modules. :: .tp_itemsize = 0, This is so that Python knows how much memory to allocate when creating -new :class:`Custom` instances. :c:member:`~PyTypeObject.tp_itemsize` is +new :class:`!Custom` instances. :c:member:`~PyTypeObject.tp_itemsize` is only used for variable-sized objects and should otherwise be zero. .. note:: @@ -188,7 +188,7 @@ set to ``NULL``. :: } This adds the type to the module dictionary. This allows us to create -:class:`Custom` instances by calling the :class:`Custom` class: +:class:`!Custom` instances by calling the :class:`!Custom` class: .. code-block:: pycon @@ -246,7 +246,7 @@ We've added an extra include:: This include provides declarations that we use to handle attributes, as described a bit later. -The :class:`Custom` type now has three data attributes in its C struct, +The :class:`!Custom` type now has three data attributes in its C struct, *first*, *last*, and *number*. The *first* and *last* variables are Python strings containing first and last names. The *number* attribute is a C integer. @@ -321,7 +321,7 @@ The ``tp_new`` handler is responsible for creating (as opposed to initializing) objects of the type. It is exposed in Python as the :meth:`__new__` method. It is not required to define a ``tp_new`` member, and indeed many extension types will simply reuse :c:func:`PyType_GenericNew` as done in the first -version of the ``Custom`` type above. In this case, we use the ``tp_new`` +version of the :class:`!Custom` type above. In this case, we use the ``tp_new`` handler to initialize the ``first`` and ``last`` attributes to non-``NULL`` default values. @@ -460,7 +460,7 @@ Further, the attributes can be deleted, setting the C pointers to ``NULL``. Eve though we can make sure the members are initialized to non-``NULL`` values, the members can be set to ``NULL`` if the attributes are deleted. -We define a single method, :meth:`Custom.name()`, that outputs the objects name as the +We define a single method, :meth:`!Custom.name()`, that outputs the objects name as the concatenation of the first and last names. :: static PyObject * @@ -477,8 +477,8 @@ concatenation of the first and last names. :: return PyUnicode_FromFormat("%S %S", self->first, self->last); } -The method is implemented as a C function that takes a :class:`Custom` (or -:class:`Custom` subclass) instance as the first argument. Methods always take an +The method is implemented as a C function that takes a :class:`!Custom` (or +:class:`!Custom` subclass) instance as the first argument. Methods always take an instance as the first argument. Methods often take positional and keyword arguments as well, but in this case we don't take any and don't need to accept a positional argument tuple or keyword argument dictionary. This method is @@ -489,8 +489,8 @@ equivalent to the Python method: def name(self): return "%s %s" % (self.first, self.last) -Note that we have to check for the possibility that our :attr:`first` and -:attr:`last` members are ``NULL``. This is because they can be deleted, in which +Note that we have to check for the possibility that our :attr:`!first` and +:attr:`!last` members are ``NULL``. This is because they can be deleted, in which case they are set to ``NULL``. It would be better to prevent deletion of these attributes and to restrict the attribute values to be strings. We'll see how to do that in the next section. @@ -519,7 +519,7 @@ to add the :c:macro:`Py_TPFLAGS_BASETYPE` to our class flag definition:: .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, -We rename :c:func:`PyInit_custom` to :c:func:`PyInit_custom2`, update the +We rename :c:func:`!PyInit_custom` to :c:func:`!PyInit_custom2`, update the module name in the :c:type:`PyModuleDef` struct, and update the full class name in the :c:type:`PyTypeObject` struct. @@ -538,18 +538,18 @@ Finally, we update our :file:`setup.py` file to build the new module: Providing finer control over data attributes ============================================ -In this section, we'll provide finer control over how the :attr:`first` and -:attr:`last` attributes are set in the :class:`Custom` example. In the previous -version of our module, the instance variables :attr:`first` and :attr:`last` +In this section, we'll provide finer control over how the :attr:`!first` and +:attr:`!last` attributes are set in the :class:`!Custom` example. In the previous +version of our module, the instance variables :attr:`!first` and :attr:`!last` could be set to non-string values or even deleted. We want to make sure that these attributes always contain strings. .. literalinclude:: ../includes/custom3.c -To provide greater control, over the :attr:`first` and :attr:`last` attributes, +To provide greater control, over the :attr:`!first` and :attr:`!last` attributes, we'll use custom getter and setter functions. Here are the functions for -getting and setting the :attr:`first` attribute:: +getting and setting the :attr:`!first` attribute:: static PyObject * Custom_getfirst(CustomObject *self, void *closure) @@ -578,13 +578,13 @@ getting and setting the :attr:`first` attribute:: return 0; } -The getter function is passed a :class:`Custom` object and a "closure", which is +The getter function is passed a :class:`!Custom` object and a "closure", which is a void pointer. In this case, the closure is ignored. (The closure supports an advanced usage in which definition data is passed to the getter and setter. This could, for example, be used to allow a single set of getter and setter functions that decide the attribute to get or set based on data in the closure.) -The setter function is passed the :class:`Custom` object, the new value, and the +The setter function is passed the :class:`!Custom` object, the new value, and the closure. The new value may be ``NULL``, in which case the attribute is being deleted. In our setter, we raise an error if the attribute is deleted or if its new value is not a string. @@ -673,11 +673,11 @@ still has a reference from itself. Its reference count doesn't drop to zero. Fortunately, Python's cyclic garbage collector will eventually figure out that the list is garbage and free it. -In the second version of the :class:`Custom` example, we allowed any kind of -object to be stored in the :attr:`first` or :attr:`last` attributes [#]_. +In the second version of the :class:`!Custom` example, we allowed any kind of +object to be stored in the :attr:`!first` or :attr:`!last` attributes [#]_. Besides, in the second and third versions, we allowed subclassing -:class:`Custom`, and subclasses may add arbitrary attributes. For any of -those two reasons, :class:`Custom` objects can participate in cycles: +:class:`!Custom`, and subclasses may add arbitrary attributes. For any of +those two reasons, :class:`!Custom` objects can participate in cycles: .. code-block:: pycon @@ -687,8 +687,8 @@ those two reasons, :class:`Custom` objects can participate in cycles: >>> n = Derived() >>> n.some_attribute = n -To allow a :class:`Custom` instance participating in a reference cycle to -be properly detected and collected by the cyclic GC, our :class:`Custom` type +To allow a :class:`!Custom` instance participating in a reference cycle to +be properly detected and collected by the cyclic GC, our :class:`!Custom` type needs to fill two additional slots and to enable a flag that enables these slots: .. literalinclude:: ../includes/custom4.c @@ -818,7 +818,7 @@ increases an internal counter: .. literalinclude:: ../includes/sublist.c -As you can see, the source code closely resembles the :class:`Custom` examples in +As you can see, the source code closely resembles the :class:`!Custom` examples in previous sections. We will break down the main differences between them. :: typedef struct { @@ -886,7 +886,7 @@ slot with :c:func:`PyType_GenericNew` -- the allocation function from the base type will be inherited. After that, calling :c:func:`PyType_Ready` and adding the type object to the -module is the same as with the basic :class:`Custom` examples. +module is the same as with the basic :class:`!Custom` examples. .. rubric:: Footnotes diff --git a/Doc/whatsnew/2.3.rst b/Doc/whatsnew/2.3.rst index 8d5603073005b..68a8917a943e7 100644 --- a/Doc/whatsnew/2.3.rst +++ b/Doc/whatsnew/2.3.rst @@ -1847,7 +1847,7 @@ specifically for allocating Python objects. :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`, and :c:func:`PyObject_Free`. * To allocate and free Python objects, use the "object" family - :c:func:`PyObject_New`, :c:func:`PyObject_NewVar`, and :c:func:`PyObject_Del`. + :c:macro:`PyObject_New`, :c:macro:`PyObject_NewVar`, and :c:func:`PyObject_Del`. Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides debugging features to catch memory overwrites and doubled frees in both extension modules diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 99291ba48390c..7bfff94c423f8 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -2058,8 +2058,8 @@ Changes in the C API * Remove :c:macro:`Py_INCREF` on the type object after allocating an instance - if any. - This may happen after calling :c:func:`PyObject_New`, - :c:func:`PyObject_NewVar`, :c:func:`PyObject_GC_New`, + This may happen after calling :c:macro:`PyObject_New`, + :c:macro:`PyObject_NewVar`, :c:func:`PyObject_GC_New`, :c:func:`PyObject_GC_NewVar`, or any other custom allocator that uses :c:func:`PyObject_Init` or :c:func:`PyObject_INIT`. diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 6de432d6c036b..666d53ed6da7b 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -1389,8 +1389,8 @@ Porting to Python 3.9 * :c:func:`PyObject_IS_GC` macro was converted to a function. * The :c:func:`PyObject_NEW` macro becomes an alias to the - :c:func:`PyObject_New` macro, and the :c:func:`PyObject_NEW_VAR` macro - becomes an alias to the :c:func:`PyObject_NewVar` macro. They no longer + :c:macro:`PyObject_New` macro, and the :c:func:`PyObject_NEW_VAR` macro + becomes an alias to the :c:macro:`PyObject_NewVar` macro. They no longer access directly the :c:member:`PyTypeObject.tp_basicsize` member. * :c:func:`PyObject_GET_WEAKREFS_LISTPTR` macro was converted to a function: From webhook-mailer at python.org Fri Jul 28 04:20:57 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 08:20:57 -0000 Subject: [Python-checkins] [3.12] gh-107298: Fix yet more Sphinx warnings in the C API doc (GH-107345) (GH-107380) Message-ID: https://github.com/python/cpython/commit/4014869b4b2456b3da1118a37332f9e9e851aebf commit: 4014869b4b2456b3da1118a37332f9e9e851aebf branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-28T08:20:53Z summary: [3.12] gh-107298: Fix yet more Sphinx warnings in the C API doc (GH-107345) (GH-107380) (cherry picked from commit 983305268e2291b0a7835621b81bf40cba7c27f3) files: M Doc/c-api/capsule.rst M Doc/c-api/init_config.rst M Doc/c-api/intro.rst M Doc/c-api/memory.rst M Doc/c-api/module.rst M Doc/c-api/none.rst M Doc/c-api/object.rst M Doc/c-api/set.rst M Doc/c-api/sys.rst M Doc/c-api/type.rst M Doc/c-api/typeobj.rst M Doc/extending/embedding.rst M Doc/extending/extending.rst M Doc/extending/newtypes.rst M Doc/extending/newtypes_tutorial.rst M Doc/howto/descriptor.rst M Doc/tools/.nitignore M Doc/whatsnew/2.5.rst M Doc/whatsnew/3.8.rst diff --git a/Doc/c-api/capsule.rst b/Doc/c-api/capsule.rst index 2a1b602dc79c0..cdb8aa33e9fd3 100644 --- a/Doc/c-api/capsule.rst +++ b/Doc/c-api/capsule.rst @@ -121,7 +121,7 @@ Refer to :ref:`using-capsules` for more information on using these objects. compared.) In other words, if :c:func:`PyCapsule_IsValid` returns a true value, calls to - any of the accessors (any function starting with :c:func:`PyCapsule_Get`) are + any of the accessors (any function starting with ``PyCapsule_Get``) are guaranteed to succeed. Return a nonzero value if the object is valid and matches the name passed in. diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index cf7337f679e69..9da9f873e7448 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -135,6 +135,8 @@ PyStatus Name of the function which created an error, can be ``NULL``. + .. c:namespace:: NULL + Functions to create a status: .. c:function:: PyStatus PyStatus_Ok(void) @@ -210,6 +212,8 @@ PyPreConfig Structure used to preinitialize Python. + .. c:namespace:: NULL + Function to initialize a preconfiguration: .. c:function:: void PyPreConfig_InitPythonConfig(PyPreConfig *preconfig) @@ -222,6 +226,8 @@ PyPreConfig Initialize the preconfiguration with :ref:`Isolated Configuration `. + .. c:namespace:: PyPreConfig + Structure fields: .. c:member:: int allocator @@ -429,6 +435,8 @@ PyConfig When done, the :c:func:`PyConfig_Clear` function must be used to release the configuration memory. + .. c:namespace:: NULL + Structure methods: .. c:function:: void PyConfig_InitPythonConfig(PyConfig *config) @@ -527,6 +535,8 @@ PyConfig The caller of these methods is responsible to handle exceptions (error or exit) using ``PyStatus_Exception()`` and ``Py_ExitStatusException()``. + .. c:namespace:: PyConfig + Structure fields: .. c:member:: PyWideStringList argv @@ -920,7 +930,7 @@ PyConfig .. c:member:: wchar_t* pythonpath_env Module search paths (:data:`sys.path`) as a string separated by ``DELIM`` - (:data:`os.path.pathsep`). + (:data:`os.pathsep`). Set by the :envvar:`PYTHONPATH` environment variable. diff --git a/Doc/c-api/intro.rst b/Doc/c-api/intro.rst index 8de76e55cd058..42a5db1893472 100644 --- a/Doc/c-api/intro.rst +++ b/Doc/c-api/intro.rst @@ -616,7 +616,7 @@ and lose important information about the exact cause of the error. .. index:: single: sum_sequence() A simple example of detecting exceptions and passing them on is shown in the -:c:func:`sum_sequence` example above. It so happens that this example doesn't +:c:func:`!sum_sequence` example above. It so happens that this example doesn't need to clean up any owned references when it detects an error. The following example function shows some error cleanup. First, to remind you why you like Python, we show the equivalent Python code:: diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index c51aba3f55536..8968b26b64320 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -431,6 +431,8 @@ Customize Memory Allocators Enum used to identify an allocator domain. Domains: + .. c:namespace:: NULL + .. c:macro:: PYMEM_DOMAIN_RAW Functions: diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index fc4dd4ce053e6..8ca48c852d4e6 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -119,7 +119,7 @@ Module Objects encoded to 'utf-8'. .. deprecated:: 3.2 - :c:func:`PyModule_GetFilename` raises :c:type:`UnicodeEncodeError` on + :c:func:`PyModule_GetFilename` raises :exc:`UnicodeEncodeError` on unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead. diff --git a/Doc/c-api/none.rst b/Doc/c-api/none.rst index 1a497652ac565..dd8bfb5610425 100644 --- a/Doc/c-api/none.rst +++ b/Doc/c-api/none.rst @@ -9,7 +9,7 @@ The ``None`` Object Note that the :c:type:`PyTypeObject` for ``None`` is not directly exposed in the Python/C API. Since ``None`` is a singleton, testing for object identity (using -``==`` in C) is sufficient. There is no :c:func:`PyNone_Check` function for the +``==`` in C) is sufficient. There is no :c:func:`!PyNone_Check` function for the same reason. diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 3b922f3b810c7..5b3c3615f7581 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -256,7 +256,7 @@ Object Protocol Normally only class objects, i.e. instances of :class:`type` or a derived class, are considered classes. However, objects can override this by having - a :attr:`__bases__` attribute (which must be a tuple of base classes). + a :attr:`~class.__bases__` attribute (which must be a tuple of base classes). .. c:function:: int PyObject_IsInstance(PyObject *inst, PyObject *cls) @@ -273,10 +273,10 @@ Object Protocol is an instance of *cls* if its class is a subclass of *cls*. An instance *inst* can override what is considered its class by having a - :attr:`__class__` attribute. + :attr:`~instance.__class__` attribute. An object *cls* can override if it is considered a class, and what its base - classes are, by having a :attr:`__bases__` attribute (which must be a tuple + classes are, by having a :attr:`~class.__bases__` attribute (which must be a tuple of base classes). diff --git a/Doc/c-api/set.rst b/Doc/c-api/set.rst index 7e0ebd2f791a4..1e8a09509032f 100644 --- a/Doc/c-api/set.rst +++ b/Doc/c-api/set.rst @@ -110,7 +110,7 @@ or :class:`frozenset` or instances of their subtypes. .. index:: pair: built-in function; len Return the length of a :class:`set` or :class:`frozenset` object. Equivalent to - ``len(anyset)``. Raises a :exc:`PyExc_SystemError` if *anyset* is not a + ``len(anyset)``. Raises a :exc:`SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. @@ -124,7 +124,7 @@ or :class:`frozenset` or instances of their subtypes. Return ``1`` if found, ``0`` if not found, and ``-1`` if an error is encountered. Unlike the Python :meth:`~object.__contains__` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise a :exc:`TypeError` if - the *key* is unhashable. Raise :exc:`PyExc_SystemError` if *anyset* is not a + the *key* is unhashable. Raise :exc:`SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. @@ -149,7 +149,7 @@ subtypes but not for instances of :class:`frozenset` or its subtypes. error is encountered. Does not raise :exc:`KeyError` for missing keys. Raise a :exc:`TypeError` if the *key* is unhashable. Unlike the Python :meth:`~set.discard` method, this function does not automatically convert unhashable sets into - temporary frozensets. Raise :exc:`PyExc_SystemError` if *set* is not an + temporary frozensets. Raise :exc:`SystemError` if *set* is not an instance of :class:`set` or its subtype. diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 2c45ea1427a11..1cec0666f5319 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -167,7 +167,7 @@ Operating System Utilities .. versionchanged:: 3.8 The function now uses the UTF-8 encoding on Windows if - :c:member:`PyConfig.legacy_windows_fs_encoding` is zero; + :c:member:`PyPreConfig.legacy_windows_fs_encoding` is zero; .. c:function:: char* Py_EncodeLocale(const wchar_t *text, size_t *error_pos) @@ -209,7 +209,7 @@ Operating System Utilities .. versionchanged:: 3.8 The function now uses the UTF-8 encoding on Windows if - :c:member:`PyConfig.legacy_windows_fs_encoding` is zero. + :c:member:`PyPreConfig.legacy_windows_fs_encoding` is zero. .. _systemfunctions: diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index e7b35c43da32a..0f58326f6c06b 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -103,7 +103,7 @@ Type Objects :c:func:`PyType_AddWatcher` will be called whenever :c:func:`PyType_Modified` reports a change to *type*. (The callback may be called only once for a series of consecutive modifications to *type*, if - :c:func:`PyType_Lookup` is not called on *type* between the modifications; + :c:func:`!_PyType_Lookup` is not called on *type* between the modifications; this is an implementation detail and subject to change.) An extension should never call ``PyType_Watch`` with a *watcher_id* that was diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 99b025dc41fc2..26e6133aebaa8 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -485,7 +485,7 @@ PyObject Slots -------------- The type object structure extends the :c:type:`PyVarObject` structure. The -:c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :func:`type_new`, +:c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :c:func:`!type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. type objects) *must* have the :c:member:`~PyVarObject.ob_size` field. @@ -740,7 +740,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) Before version 3.12, it was not recommended for :ref:`mutable heap types ` to implement the vectorcall protocol. - When a user sets :attr:`~type.__call__` in Python code, only *tp_call* is + When a user sets :attr:`~object.__call__` in Python code, only *tp_call* is updated, likely making it inconsistent with the vectorcall function. Since 3.12, setting ``__call__`` will disable vectorcall optimization by clearing the :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag. @@ -1369,8 +1369,8 @@ and :c:data:`PyType_Type` effectively act as defaults.) The :c:member:`~PyTypeObject.tp_traverse` pointer is used by the garbage collector to detect reference cycles. A typical implementation of a :c:member:`~PyTypeObject.tp_traverse` function simply calls :c:func:`Py_VISIT` on each of the instance's members that are Python - objects that the instance owns. For example, this is function :c:func:`local_traverse` from the - :mod:`_thread` extension module:: + objects that the instance owns. For example, this is function :c:func:`!local_traverse` from the + :mod:`!_thread` extension module:: static int local_traverse(localobject *self, visitproc visit, void *arg) @@ -1721,7 +1721,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) called; it may also be initialized to a dictionary containing initial attributes for the type. Once :c:func:`PyType_Ready` has initialized the type, extra attributes for the type may be added to this dictionary only if they don't - correspond to overloaded operations (like :meth:`__add__`). Once + correspond to overloaded operations (like :meth:`~object.__add__`). Once initialization for the type has finished, this field should be treated as read-only. @@ -1818,7 +1818,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Default:** This slot has no default. For :ref:`static types `, if the - field is ``NULL`` then no :attr:`__dict__` gets created for instances. + field is ``NULL`` then no :attr:`~object.__dict__` gets created for instances. If the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit is set in the :c:member:`~PyTypeObject.tp_dict` field, then @@ -1830,10 +1830,10 @@ and :c:data:`PyType_Type` effectively act as defaults.) An optional pointer to an instance initialization function. - This function corresponds to the :meth:`__init__` method of classes. Like - :meth:`__init__`, it is possible to create an instance without calling - :meth:`__init__`, and it is possible to reinitialize an instance by calling its - :meth:`__init__` method again. + This function corresponds to the :meth:`~object.__init__` method of classes. Like + :meth:`!__init__`, it is possible to create an instance without calling + :meth:`!__init__`, and it is possible to reinitialize an instance by calling its + :meth:`!__init__` method again. The function signature is:: @@ -1841,7 +1841,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) The self argument is the instance to be initialized; the *args* and *kwds* arguments represent positional and keyword arguments of the call to - :meth:`__init__`. + :meth:`~object.__init__`. The :c:member:`~PyTypeObject.tp_init` function, if not ``NULL``, is called when an instance is created normally by calling its type, after the type's :c:member:`~PyTypeObject.tp_new` function @@ -2130,7 +2130,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) In other words, it is used to implement :ref:`vectorcall ` for ``type.__call__``. If ``tp_vectorcall`` is ``NULL``, the default call implementation - using :attr:`__new__` and :attr:`__init__` is used. + using :meth:`~object.__new__` and :meth:`~object.__init__` is used. **Inheritance:** @@ -2329,8 +2329,8 @@ Mapping Object Structures .. c:member:: objobjargproc PyMappingMethods.mp_ass_subscript This function is used by :c:func:`PyObject_SetItem`, - :c:func:`PyObject_DelItem`, :c:func:`PyObject_SetSlice` and - :c:func:`PyObject_DelSlice`. It has the same signature as + :c:func:`PyObject_DelItem`, :c:func:`PySequence_SetSlice` and + :c:func:`PySequence_DelSlice`. It has the same signature as :c:func:`!PyObject_SetItem`, but *v* can also be set to ``NULL`` to delete an item. If this slot is ``NULL``, the object does not support item assignment and deletion. @@ -2552,7 +2552,7 @@ Async Object Structures PyObject *am_aiter(PyObject *self); Must return an :term:`asynchronous iterator` object. - See :meth:`__anext__` for details. + See :meth:`~object.__anext__` for details. This slot may be set to ``NULL`` if an object does not implement asynchronous iteration protocol. @@ -2563,7 +2563,8 @@ Async Object Structures PyObject *am_anext(PyObject *self); - Must return an :term:`awaitable` object. See :meth:`__anext__` for details. + Must return an :term:`awaitable` object. + See :meth:`~object.__anext__` for details. This slot may be set to ``NULL``. .. c:member:: sendfunc PyAsyncMethods.am_send diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst index e64db37334403..4c7c7ec98a6c9 100644 --- a/Doc/extending/embedding.rst +++ b/Doc/extending/embedding.rst @@ -250,7 +250,7 @@ following two statements before the call to :c:func:`Py_Initialize`:: PyImport_AppendInittab("emb", &PyInit_emb); These two lines initialize the ``numargs`` variable, and make the -:func:`emb.numargs` function accessible to the embedded Python interpreter. +:func:`!emb.numargs` function accessible to the embedded Python interpreter. With these extensions, the Python script can do things like .. code-block:: python diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index c13b9371297ee..72e30a6882807 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -195,7 +195,7 @@ The choice of which exception to raise is entirely yours. There are predeclared C objects corresponding to all built-in Python exceptions, such as :c:data:`PyExc_ZeroDivisionError`, which you can use directly. Of course, you should choose exceptions wisely --- don't use :c:data:`PyExc_TypeError` to mean -that a file couldn't be opened (that should probably be :c:data:`PyExc_IOError`). +that a file couldn't be opened (that should probably be :c:data:`PyExc_OSError`). If something's wrong with the argument list, the :c:func:`PyArg_ParseTuple` function usually raises :c:data:`PyExc_TypeError`. If you have an argument whose value must be in a particular range or must satisfy other conditions, @@ -206,7 +206,7 @@ usually declare a static object variable at the beginning of your file:: static PyObject *SpamError; -and initialize it in your module's initialization function (:c:func:`PyInit_spam`) +and initialize it in your module's initialization function (:c:func:`!PyInit_spam`) with an exception object:: PyMODINIT_FUNC @@ -354,7 +354,7 @@ The method table must be referenced in the module definition structure:: This structure, in turn, must be passed to the interpreter in the module's initialization function. The initialization function must be named -:c:func:`PyInit_name`, where *name* is the name of the module, and should be the +:c:func:`!PyInit_name`, where *name* is the name of the module, and should be the only non-\ ``static`` item defined in the module file:: PyMODINIT_FUNC @@ -368,7 +368,7 @@ declares any special linkage declarations required by the platform, and for C++ declares the function as ``extern "C"``. When the Python program imports module :mod:`!spam` for the first time, -:c:func:`PyInit_spam` is called. (See below for comments about embedding Python.) +:c:func:`!PyInit_spam` is called. (See below for comments about embedding Python.) It calls :c:func:`PyModule_Create`, which returns a module object, and inserts built-in function objects into the newly created module based upon the table (an array of :c:type:`PyMethodDef` structures) found in the module definition. @@ -378,7 +378,7 @@ certain errors, or return ``NULL`` if the module could not be initialized satisfactorily. The init function must return the module object to its caller, so that it then gets inserted into ``sys.modules``. -When embedding Python, the :c:func:`PyInit_spam` function is not called +When embedding Python, the :c:func:`!PyInit_spam` function is not called automatically unless there's an entry in the :c:data:`PyImport_Inittab` table. To add the module to the initialization table, use :c:func:`PyImport_AppendInittab`, optionally followed by an import of the module:: @@ -1209,13 +1209,13 @@ the module and retrieving its C API pointers; client modules only have to call this macro before accessing the C API. The exporting module is a modification of the :mod:`!spam` module from section -:ref:`extending-simpleexample`. The function :func:`spam.system` does not call +:ref:`extending-simpleexample`. The function :func:`!spam.system` does not call the C library function :c:func:`system` directly, but a function -:c:func:`PySpam_System`, which would of course do something more complicated in +:c:func:`!PySpam_System`, which would of course do something more complicated in reality (such as adding "spam" to every command). This function -:c:func:`PySpam_System` is also exported to other extension modules. +:c:func:`!PySpam_System` is also exported to other extension modules. -The function :c:func:`PySpam_System` is a plain C function, declared +The function :c:func:`!PySpam_System` is a plain C function, declared ``static`` like everything else:: static int @@ -1278,7 +1278,7 @@ function must take care of initializing the C API pointer array:: } Note that ``PySpam_API`` is declared ``static``; otherwise the pointer -array would disappear when :func:`PyInit_spam` terminates! +array would disappear when :c:func:`!PyInit_spam` terminates! The bulk of the work is in the header file :file:`spammodule.h`, which looks like this:: @@ -1332,8 +1332,8 @@ like this:: #endif /* !defined(Py_SPAMMODULE_H) */ All that a client module must do in order to have access to the function -:c:func:`PySpam_System` is to call the function (or rather macro) -:c:func:`import_spam` in its initialization function:: +:c:func:`!PySpam_System` is to call the function (or rather macro) +:c:func:`!import_spam` in its initialization function:: PyMODINIT_FUNC PyInit_client(void) diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst index 25822744125f8..386b3c8f4452c 100644 --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -286,9 +286,9 @@ be read-only or read-write. The structures in the table are defined as:: For each entry in the table, a :term:`descriptor` will be constructed and added to the type which will be able to extract a value from the instance structure. The -:attr:`type` field should contain a type code like :c:macro:`Py_T_INT` or +:c:member:`~PyMemberDef.type` field should contain a type code like :c:macro:`Py_T_INT` or :c:macro:`Py_T_DOUBLE`; the value will be used to determine how to -convert Python values to and from C values. The :attr:`flags` field is used to +convert Python values to and from C values. The :c:member:`~PyMemberDef.flags` field is used to store flags which control how the attribute can be accessed: you can set it to :c:macro:`Py_READONLY` to prevent Python code from setting it. @@ -298,7 +298,7 @@ have an associated doc string simply by providing the text in the table. An application can use the introspection API to retrieve the descriptor from the class object, and get the doc string using its :attr:`__doc__` attribute. -As with the :c:member:`~PyTypeObject.tp_methods` table, a sentinel entry with a :attr:`name` value +As with the :c:member:`~PyTypeObject.tp_methods` table, a sentinel entry with a :c:member:`~PyMethodDef.name` value of ``NULL`` is required. .. XXX Descriptors need to be explained in more detail somewhere, but not here. @@ -323,7 +323,7 @@ called, so that if you do need to extend their functionality, you'll understand what needs to be done. The :c:member:`~PyTypeObject.tp_getattr` handler is called when the object requires an attribute -look-up. It is called in the same situations where the :meth:`__getattr__` +look-up. It is called in the same situations where the :meth:`~object.__getattr__` method of a class would be called. Here is an example:: @@ -342,8 +342,8 @@ Here is an example:: return NULL; } -The :c:member:`~PyTypeObject.tp_setattr` handler is called when the :meth:`__setattr__` or -:meth:`__delattr__` method of a class instance would be called. When an +The :c:member:`~PyTypeObject.tp_setattr` handler is called when the :meth:`~object.__setattr__` or +:meth:`~object.__delattr__` method of a class instance would be called. When an attribute should be deleted, the third parameter will be ``NULL``. Here is an example that simply raises an exception; if this were really all you wanted, the :c:member:`~PyTypeObject.tp_setattr` handler should be set to ``NULL``. :: @@ -364,7 +364,7 @@ Object Comparison The :c:member:`~PyTypeObject.tp_richcompare` handler is called when comparisons are needed. It is analogous to the :ref:`rich comparison methods `, like -:meth:`__lt__`, and also called by :c:func:`PyObject_RichCompare` and +:meth:`!__lt__`, and also called by :c:func:`PyObject_RichCompare` and :c:func:`PyObject_RichCompareBool`. This function is called with two Python objects and the operator as arguments, @@ -505,7 +505,7 @@ These functions provide support for the iterator protocol. Both handlers take exactly one parameter, the instance for which they are being called, and return a new reference. In the case of an error, they should set an exception and return ``NULL``. :c:member:`~PyTypeObject.tp_iter` corresponds -to the Python :meth:`__iter__` method, while :c:member:`~PyTypeObject.tp_iternext` +to the Python :meth:`~object.__iter__` method, while :c:member:`~PyTypeObject.tp_iternext` corresponds to the Python :meth:`~iterator.__next__` method. Any :term:`iterable` object must implement the :c:member:`~PyTypeObject.tp_iter` diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index 7180191d40f18..f8684df5dd829 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -145,7 +145,7 @@ only used for variable-sized objects and should otherwise be zero. :c:member:`~PyTypeObject.tp_basicsize` as its base type, you may have problems with multiple inheritance. A Python subclass of your type will have to list your type first in its :attr:`~class.__bases__`, or else it will not be able to call your type's - :meth:`__new__` method without getting an error. You can avoid this problem by + :meth:`~object.__new__` method without getting an error. You can avoid this problem by ensuring that your type has a larger value for :c:member:`~PyTypeObject.tp_basicsize` than its base type does. Most of the time, this will be true anyway, because either your base type will be :class:`object`, or else you will be adding data members to @@ -164,14 +164,14 @@ We provide a doc string for the type in :c:member:`~PyTypeObject.tp_doc`. :: .tp_doc = PyDoc_STR("Custom objects"), To enable object creation, we have to provide a :c:member:`~PyTypeObject.tp_new` -handler. This is the equivalent of the Python method :meth:`__new__`, but +handler. This is the equivalent of the Python method :meth:`~object.__new__`, but has to be specified explicitly. In this case, we can just use the default implementation provided by the API function :c:func:`PyType_GenericNew`. :: .tp_new = PyType_GenericNew, Everything else in the file should be familiar, except for some code in -:c:func:`PyInit_custom`:: +:c:func:`!PyInit_custom`:: if (PyType_Ready(&CustomType) < 0) return; @@ -220,7 +220,7 @@ Of course, the current Custom type is pretty uninteresting. It has no data and doesn't do anything. It can't even be subclassed. .. note:: - While this documentation showcases the standard :mod:`distutils` module + While this documentation showcases the standard :mod:`!distutils` module for building C extensions, it is recommended in real-world use cases to use the newer and better-maintained ``setuptools`` library. Documentation on how to do this is out of scope for this document and can be found in @@ -272,7 +272,7 @@ This method first clears the reference counts of the two Python attributes. ``NULL`` (which might happen here if ``tp_new`` failed midway). It then calls the :c:member:`~PyTypeObject.tp_free` member of the object's type (computed by ``Py_TYPE(self)``) to free the object's memory. Note that -the object's type might not be :class:`CustomType`, because the object may +the object's type might not be :class:`!CustomType`, because the object may be an instance of a subclass. .. note:: @@ -311,7 +311,7 @@ and install it in the :c:member:`~PyTypeObject.tp_new` member:: .tp_new = Custom_new, The ``tp_new`` handler is responsible for creating (as opposed to initializing) -objects of the type. It is exposed in Python as the :meth:`__new__` method. +objects of the type. It is exposed in Python as the :meth:`~object.__new__` method. It is not required to define a ``tp_new`` member, and indeed many extension types will simply reuse :c:func:`PyType_GenericNew` as done in the first version of the :class:`!Custom` type above. In this case, we use the ``tp_new`` @@ -345,7 +345,7 @@ result against ``NULL`` before proceeding. .. note:: If you are creating a co-operative :c:member:`~PyTypeObject.tp_new` (one - that calls a base type's :c:member:`~PyTypeObject.tp_new` or :meth:`__new__`), + that calls a base type's :c:member:`~PyTypeObject.tp_new` or :meth:`~object.__new__`), you must *not* try to determine what method to call using method resolution order at runtime. Always statically determine what type you are going to call, and call its :c:member:`~PyTypeObject.tp_new` directly, or via @@ -388,14 +388,14 @@ by filling the :c:member:`~PyTypeObject.tp_init` slot. :: .tp_init = (initproc) Custom_init, The :c:member:`~PyTypeObject.tp_init` slot is exposed in Python as the -:meth:`__init__` method. It is used to initialize an object after it's +:meth:`~object.__init__` method. It is used to initialize an object after it's created. Initializers always accept positional and keyword arguments, and they should return either ``0`` on success or ``-1`` on error. Unlike the ``tp_new`` handler, there is no guarantee that ``tp_init`` is called at all (for example, the :mod:`pickle` module by default -doesn't call :meth:`__init__` on unpickled instances). It can also be -called multiple times. Anyone can call the :meth:`__init__` method on +doesn't call :meth:`~object.__init__` on unpickled instances). It can also be +called multiple times. Anyone can call the :meth:`!__init__` method on our objects. For this reason, we have to be extra careful when assigning the new attribute values. We might be tempted, for example to assign the ``first`` member like this:: @@ -708,8 +708,8 @@ participate in cycles:: } For each subobject that can participate in cycles, we need to call the -:c:func:`visit` function, which is passed to the traversal method. The -:c:func:`visit` function takes as arguments the subobject and the extra argument +:c:func:`!visit` function, which is passed to the traversal method. The +:c:func:`!visit` function takes as arguments the subobject and the extra argument *arg* passed to the traversal method. It returns an integer value that must be returned if it is non-zero. @@ -791,9 +791,9 @@ types. It is easiest to inherit from the built in types, since an extension can easily use the :c:type:`PyTypeObject` it needs. It can be difficult to share these :c:type:`PyTypeObject` structures between extension modules. -In this example we will create a :class:`SubList` type that inherits from the +In this example we will create a :class:`!SubList` type that inherits from the built-in :class:`list` type. The new type will be completely compatible with -regular lists, but will have an additional :meth:`increment` method that +regular lists, but will have an additional :meth:`!increment` method that increases an internal counter: .. code-block:: pycon @@ -823,7 +823,7 @@ The primary difference for derived type objects is that the base type's object structure must be the first value. The base type will already include the :c:func:`PyObject_HEAD` at the beginning of its structure. -When a Python object is a :class:`SubList` instance, its ``PyObject *`` pointer +When a Python object is a :class:`!SubList` instance, its ``PyObject *`` pointer can be safely cast to both ``PyListObject *`` and ``SubListObject *``:: static int @@ -835,7 +835,7 @@ can be safely cast to both ``PyListObject *`` and ``SubListObject *``:: return 0; } -We see above how to call through to the :attr:`__init__` method of the base +We see above how to call through to the :meth:`~object.__init__` method of the base type. This pattern is important when writing a type with custom diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 3688c47f0d6ec..1d9424cb735a4 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -779,8 +779,8 @@ by a search through the class's :term:`method resolution order`. If a descriptor is found, it is invoked with ``desc.__get__(None, A)``. -The full C implementation can be found in :c:func:`type_getattro()` and -:c:func:`_PyType_Lookup()` in :source:`Objects/typeobject.c`. +The full C implementation can be found in :c:func:`!type_getattro` and +:c:func:`!_PyType_Lookup` in :source:`Objects/typeobject.c`. Invocation from super @@ -794,7 +794,7 @@ for the base class ``B`` immediately following ``A`` and then returns ``B.__dict__['m'].__get__(obj, A)``. If not a descriptor, ``m`` is returned unchanged. -The full C implementation can be found in :c:func:`super_getattro()` in +The full C implementation can be found in :c:func:`!super_getattro` in :source:`Objects/typeobject.c`. A pure Python equivalent can be found in `Guido's Tutorial `_. @@ -836,8 +836,8 @@ and if they define :meth:`__set_name__`, that method is called with two arguments. The *owner* is the class where the descriptor is used, and the *name* is the class variable the descriptor was assigned to. -The implementation details are in :c:func:`type_new()` and -:c:func:`set_names()` in :source:`Objects/typeobject.c`. +The implementation details are in :c:func:`!type_new` and +:c:func:`!set_names` in :source:`Objects/typeobject.c`. Since the update logic is in :meth:`type.__new__`, notifications only take place at the time of class creation. If descriptors are added to the class diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index ab906b3817670..63223de613ff7 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -6,7 +6,6 @@ Doc/c-api/arg.rst Doc/c-api/bool.rst Doc/c-api/buffer.rst -Doc/c-api/capsule.rst Doc/c-api/datetime.rst Doc/c-api/descriptor.rst Doc/c-api/dict.rst @@ -21,7 +20,6 @@ Doc/c-api/intro.rst Doc/c-api/memory.rst Doc/c-api/memoryview.rst Doc/c-api/module.rst -Doc/c-api/none.rst Doc/c-api/object.rst Doc/c-api/set.rst Doc/c-api/stable.rst @@ -30,10 +28,8 @@ Doc/c-api/sys.rst Doc/c-api/type.rst Doc/c-api/typeobj.rst Doc/c-api/unicode.rst -Doc/extending/embedding.rst Doc/extending/extending.rst Doc/extending/newtypes.rst -Doc/extending/newtypes_tutorial.rst Doc/faq/design.rst Doc/faq/extending.rst Doc/faq/gui.rst diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index 162d2c342f50f..a15eefc2953f4 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -2151,8 +2151,8 @@ Changes to Python's build process and to the C API include: Previously these different families all reduced to the platform's :c:func:`malloc` and :c:func:`free` functions. This meant it didn't matter if - you got things wrong and allocated memory with the :c:func:`PyMem` function but - freed it with the :c:func:`PyObject` function. With 2.5's changes to obmalloc, + you got things wrong and allocated memory with the ``PyMem`` function but + freed it with the ``PyObject`` function. With 2.5's changes to obmalloc, these families now do different things and mismatches will probably result in a segfault. You should carefully test your C extension modules with Python 2.5. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 98d97268b5901..3397de5d3e8c4 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1850,7 +1850,7 @@ Changes in Python behavior finalizing, making them consistent with :c:func:`PyEval_RestoreThread`, :c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`. If this behavior is not desired, guard the call by checking :c:func:`_Py_IsFinalizing` - or :c:func:`sys.is_finalizing`. + or :func:`sys.is_finalizing`. (Contributed by Joannah Nanjekye in :issue:`36475`.) From webhook-mailer at python.org Fri Jul 28 04:31:54 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 08:31:54 -0000 Subject: [Python-checkins] [3.11] gh-107298: Fix yet more Sphinx warnings in the C API doc (GH-107345) (GH-107381) Message-ID: https://github.com/python/cpython/commit/d9392c0c0aeedd4d23027ecf20e14af4c227435a commit: d9392c0c0aeedd4d23027ecf20e14af4c227435a branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-28T08:31:49Z summary: [3.11] gh-107298: Fix yet more Sphinx warnings in the C API doc (GH-107345) (GH-107381) (cherry picked from commit 983305268e2291b0a7835621b81bf40cba7c27f3) files: A Doc/tools/.nitignore M Doc/c-api/capsule.rst M Doc/c-api/init_config.rst M Doc/c-api/intro.rst M Doc/c-api/memory.rst M Doc/c-api/module.rst M Doc/c-api/none.rst M Doc/c-api/object.rst M Doc/c-api/set.rst M Doc/c-api/typeobj.rst M Doc/extending/embedding.rst M Doc/extending/extending.rst M Doc/extending/newtypes.rst M Doc/extending/newtypes_tutorial.rst M Doc/howto/descriptor.rst M Doc/whatsnew/2.5.rst M Doc/whatsnew/3.8.rst diff --git a/Doc/c-api/capsule.rst b/Doc/c-api/capsule.rst index 2a1b602dc79c0..cdb8aa33e9fd3 100644 --- a/Doc/c-api/capsule.rst +++ b/Doc/c-api/capsule.rst @@ -121,7 +121,7 @@ Refer to :ref:`using-capsules` for more information on using these objects. compared.) In other words, if :c:func:`PyCapsule_IsValid` returns a true value, calls to - any of the accessors (any function starting with :c:func:`PyCapsule_Get`) are + any of the accessors (any function starting with ``PyCapsule_Get``) are guaranteed to succeed. Return a nonzero value if the object is valid and matches the name passed in. diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 216755a1f948d..c6798d68b0e8a 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -135,6 +135,8 @@ PyStatus Name of the function which created an error, can be ``NULL``. + .. c:namespace:: NULL + Functions to create a status: .. c:function:: PyStatus PyStatus_Ok(void) @@ -210,6 +212,8 @@ PyPreConfig Structure used to preinitialize Python. + .. c:namespace:: NULL + Function to initialize a preconfiguration: .. c:function:: void PyPreConfig_InitPythonConfig(PyPreConfig *preconfig) @@ -222,6 +226,8 @@ PyPreConfig Initialize the preconfiguration with :ref:`Isolated Configuration `. + .. c:namespace:: PyPreConfig + Structure fields: .. c:member:: int allocator @@ -429,6 +435,8 @@ PyConfig When done, the :c:func:`PyConfig_Clear` function must be used to release the configuration memory. + .. c:namespace:: NULL + Structure methods: .. c:function:: void PyConfig_InitPythonConfig(PyConfig *config) @@ -527,6 +535,8 @@ PyConfig The caller of these methods is responsible to handle exceptions (error or exit) using ``PyStatus_Exception()`` and ``Py_ExitStatusException()``. + .. c:namespace:: PyConfig + Structure fields: .. c:member:: PyWideStringList argv @@ -899,7 +909,7 @@ PyConfig .. c:member:: wchar_t* pythonpath_env Module search paths (:data:`sys.path`) as a string separated by ``DELIM`` - (:data:`os.path.pathsep`). + (:data:`os.pathsep`). Set by the :envvar:`PYTHONPATH` environment variable. diff --git a/Doc/c-api/intro.rst b/Doc/c-api/intro.rst index 17710b02abbc4..a3ccfa08baf70 100644 --- a/Doc/c-api/intro.rst +++ b/Doc/c-api/intro.rst @@ -616,7 +616,7 @@ and lose important information about the exact cause of the error. .. index:: single: sum_sequence() A simple example of detecting exceptions and passing them on is shown in the -:c:func:`sum_sequence` example above. It so happens that this example doesn't +:c:func:`!sum_sequence` example above. It so happens that this example doesn't need to clean up any owned references when it detects an error. The following example function shows some error cleanup. First, to remind you why you like Python, we show the equivalent Python code:: diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index c51aba3f55536..8968b26b64320 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -431,6 +431,8 @@ Customize Memory Allocators Enum used to identify an allocator domain. Domains: + .. c:namespace:: NULL + .. c:macro:: PYMEM_DOMAIN_RAW Functions: diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index fc4dd4ce053e6..8ca48c852d4e6 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -119,7 +119,7 @@ Module Objects encoded to 'utf-8'. .. deprecated:: 3.2 - :c:func:`PyModule_GetFilename` raises :c:type:`UnicodeEncodeError` on + :c:func:`PyModule_GetFilename` raises :exc:`UnicodeEncodeError` on unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead. diff --git a/Doc/c-api/none.rst b/Doc/c-api/none.rst index b84a16a28ead5..e238fd1c2d8dc 100644 --- a/Doc/c-api/none.rst +++ b/Doc/c-api/none.rst @@ -9,7 +9,7 @@ The ``None`` Object Note that the :c:type:`PyTypeObject` for ``None`` is not directly exposed in the Python/C API. Since ``None`` is a singleton, testing for object identity (using -``==`` in C) is sufficient. There is no :c:func:`PyNone_Check` function for the +``==`` in C) is sufficient. There is no :c:func:`!PyNone_Check` function for the same reason. diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 191a252808bda..e7d80dbf8c5c1 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -256,7 +256,7 @@ Object Protocol Normally only class objects, i.e. instances of :class:`type` or a derived class, are considered classes. However, objects can override this by having - a :attr:`__bases__` attribute (which must be a tuple of base classes). + a :attr:`~class.__bases__` attribute (which must be a tuple of base classes). .. c:function:: int PyObject_IsInstance(PyObject *inst, PyObject *cls) @@ -273,10 +273,10 @@ Object Protocol is an instance of *cls* if its class is a subclass of *cls*. An instance *inst* can override what is considered its class by having a - :attr:`__class__` attribute. + :attr:`~instance.__class__` attribute. An object *cls* can override if it is considered a class, and what its base - classes are, by having a :attr:`__bases__` attribute (which must be a tuple + classes are, by having a :attr:`~class.__bases__` attribute (which must be a tuple of base classes). diff --git a/Doc/c-api/set.rst b/Doc/c-api/set.rst index 7e0ebd2f791a4..1e8a09509032f 100644 --- a/Doc/c-api/set.rst +++ b/Doc/c-api/set.rst @@ -110,7 +110,7 @@ or :class:`frozenset` or instances of their subtypes. .. index:: pair: built-in function; len Return the length of a :class:`set` or :class:`frozenset` object. Equivalent to - ``len(anyset)``. Raises a :exc:`PyExc_SystemError` if *anyset* is not a + ``len(anyset)``. Raises a :exc:`SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. @@ -124,7 +124,7 @@ or :class:`frozenset` or instances of their subtypes. Return ``1`` if found, ``0`` if not found, and ``-1`` if an error is encountered. Unlike the Python :meth:`~object.__contains__` method, this function does not automatically convert unhashable sets into temporary frozensets. Raise a :exc:`TypeError` if - the *key* is unhashable. Raise :exc:`PyExc_SystemError` if *anyset* is not a + the *key* is unhashable. Raise :exc:`SystemError` if *anyset* is not a :class:`set`, :class:`frozenset`, or an instance of a subtype. @@ -149,7 +149,7 @@ subtypes but not for instances of :class:`frozenset` or its subtypes. error is encountered. Does not raise :exc:`KeyError` for missing keys. Raise a :exc:`TypeError` if the *key* is unhashable. Unlike the Python :meth:`~set.discard` method, this function does not automatically convert unhashable sets into - temporary frozensets. Raise :exc:`PyExc_SystemError` if *set* is not an + temporary frozensets. Raise :exc:`SystemError` if *set* is not an instance of :class:`set` or its subtype. diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 3577e8dc84502..11f6034b9e330 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -483,7 +483,7 @@ PyObject Slots -------------- The type object structure extends the :c:type:`PyVarObject` structure. The -:c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :func:`type_new`, +:c:member:`~PyVarObject.ob_size` field is used for dynamic types (created by :c:func:`!type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. type objects) *must* have the :c:member:`~PyVarObject.ob_size` field. @@ -1305,8 +1305,8 @@ and :c:data:`PyType_Type` effectively act as defaults.) The :c:member:`~PyTypeObject.tp_traverse` pointer is used by the garbage collector to detect reference cycles. A typical implementation of a :c:member:`~PyTypeObject.tp_traverse` function simply calls :c:func:`Py_VISIT` on each of the instance's members that are Python - objects that the instance owns. For example, this is function :c:func:`local_traverse` from the - :mod:`_thread` extension module:: + objects that the instance owns. For example, this is function :c:func:`!local_traverse` from the + :mod:`!_thread` extension module:: static int local_traverse(localobject *self, visitproc visit, void *arg) @@ -1658,7 +1658,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) called; it may also be initialized to a dictionary containing initial attributes for the type. Once :c:func:`PyType_Ready` has initialized the type, extra attributes for the type may be added to this dictionary only if they don't - correspond to overloaded operations (like :meth:`__add__`). + correspond to overloaded operations (like :meth:`~object.__add__`). **Inheritance:** @@ -1759,17 +1759,17 @@ and :c:data:`PyType_Type` effectively act as defaults.) **Default:** This slot has no default. For :ref:`static types `, if the - field is ``NULL`` then no :attr:`__dict__` gets created for instances. + field is ``NULL`` then no :attr:`~object.__dict__` gets created for instances. .. c:member:: initproc PyTypeObject.tp_init An optional pointer to an instance initialization function. - This function corresponds to the :meth:`__init__` method of classes. Like - :meth:`__init__`, it is possible to create an instance without calling - :meth:`__init__`, and it is possible to reinitialize an instance by calling its - :meth:`__init__` method again. + This function corresponds to the :meth:`~object.__init__` method of classes. Like + :meth:`!__init__`, it is possible to create an instance without calling + :meth:`!__init__`, and it is possible to reinitialize an instance by calling its + :meth:`!__init__` method again. The function signature is:: @@ -1777,7 +1777,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) The self argument is the instance to be initialized; the *args* and *kwds* arguments represent positional and keyword arguments of the call to - :meth:`__init__`. + :meth:`~object.__init__`. The :c:member:`~PyTypeObject.tp_init` function, if not ``NULL``, is called when an instance is created normally by calling its type, after the type's :c:member:`~PyTypeObject.tp_new` function @@ -2051,7 +2051,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) In other words, it is used to implement :ref:`vectorcall ` for ``type.__call__``. If ``tp_vectorcall`` is ``NULL``, the default call implementation - using :attr:`__new__` and :attr:`__init__` is used. + using :meth:`~object.__new__` and :meth:`~object.__init__` is used. **Inheritance:** @@ -2243,8 +2243,8 @@ Mapping Object Structures .. c:member:: objobjargproc PyMappingMethods.mp_ass_subscript This function is used by :c:func:`PyObject_SetItem`, - :c:func:`PyObject_DelItem`, :c:func:`PyObject_SetSlice` and - :c:func:`PyObject_DelSlice`. It has the same signature as + :c:func:`PyObject_DelItem`, :c:func:`PySequence_SetSlice` and + :c:func:`PySequence_DelSlice`. It has the same signature as :c:func:`!PyObject_SetItem`, but *v* can also be set to ``NULL`` to delete an item. If this slot is ``NULL``, the object does not support item assignment and deletion. @@ -2466,7 +2466,7 @@ Async Object Structures PyObject *am_aiter(PyObject *self); Must return an :term:`asynchronous iterator` object. - See :meth:`__anext__` for details. + See :meth:`~object.__anext__` for details. This slot may be set to ``NULL`` if an object does not implement asynchronous iteration protocol. @@ -2477,7 +2477,8 @@ Async Object Structures PyObject *am_anext(PyObject *self); - Must return an :term:`awaitable` object. See :meth:`__anext__` for details. + Must return an :term:`awaitable` object. + See :meth:`~object.__anext__` for details. This slot may be set to ``NULL``. .. c:member:: sendfunc PyAsyncMethods.am_send diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst index e64db37334403..4c7c7ec98a6c9 100644 --- a/Doc/extending/embedding.rst +++ b/Doc/extending/embedding.rst @@ -250,7 +250,7 @@ following two statements before the call to :c:func:`Py_Initialize`:: PyImport_AppendInittab("emb", &PyInit_emb); These two lines initialize the ``numargs`` variable, and make the -:func:`emb.numargs` function accessible to the embedded Python interpreter. +:func:`!emb.numargs` function accessible to the embedded Python interpreter. With these extensions, the Python script can do things like .. code-block:: python diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index c13b9371297ee..72e30a6882807 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -195,7 +195,7 @@ The choice of which exception to raise is entirely yours. There are predeclared C objects corresponding to all built-in Python exceptions, such as :c:data:`PyExc_ZeroDivisionError`, which you can use directly. Of course, you should choose exceptions wisely --- don't use :c:data:`PyExc_TypeError` to mean -that a file couldn't be opened (that should probably be :c:data:`PyExc_IOError`). +that a file couldn't be opened (that should probably be :c:data:`PyExc_OSError`). If something's wrong with the argument list, the :c:func:`PyArg_ParseTuple` function usually raises :c:data:`PyExc_TypeError`. If you have an argument whose value must be in a particular range or must satisfy other conditions, @@ -206,7 +206,7 @@ usually declare a static object variable at the beginning of your file:: static PyObject *SpamError; -and initialize it in your module's initialization function (:c:func:`PyInit_spam`) +and initialize it in your module's initialization function (:c:func:`!PyInit_spam`) with an exception object:: PyMODINIT_FUNC @@ -354,7 +354,7 @@ The method table must be referenced in the module definition structure:: This structure, in turn, must be passed to the interpreter in the module's initialization function. The initialization function must be named -:c:func:`PyInit_name`, where *name* is the name of the module, and should be the +:c:func:`!PyInit_name`, where *name* is the name of the module, and should be the only non-\ ``static`` item defined in the module file:: PyMODINIT_FUNC @@ -368,7 +368,7 @@ declares any special linkage declarations required by the platform, and for C++ declares the function as ``extern "C"``. When the Python program imports module :mod:`!spam` for the first time, -:c:func:`PyInit_spam` is called. (See below for comments about embedding Python.) +:c:func:`!PyInit_spam` is called. (See below for comments about embedding Python.) It calls :c:func:`PyModule_Create`, which returns a module object, and inserts built-in function objects into the newly created module based upon the table (an array of :c:type:`PyMethodDef` structures) found in the module definition. @@ -378,7 +378,7 @@ certain errors, or return ``NULL`` if the module could not be initialized satisfactorily. The init function must return the module object to its caller, so that it then gets inserted into ``sys.modules``. -When embedding Python, the :c:func:`PyInit_spam` function is not called +When embedding Python, the :c:func:`!PyInit_spam` function is not called automatically unless there's an entry in the :c:data:`PyImport_Inittab` table. To add the module to the initialization table, use :c:func:`PyImport_AppendInittab`, optionally followed by an import of the module:: @@ -1209,13 +1209,13 @@ the module and retrieving its C API pointers; client modules only have to call this macro before accessing the C API. The exporting module is a modification of the :mod:`!spam` module from section -:ref:`extending-simpleexample`. The function :func:`spam.system` does not call +:ref:`extending-simpleexample`. The function :func:`!spam.system` does not call the C library function :c:func:`system` directly, but a function -:c:func:`PySpam_System`, which would of course do something more complicated in +:c:func:`!PySpam_System`, which would of course do something more complicated in reality (such as adding "spam" to every command). This function -:c:func:`PySpam_System` is also exported to other extension modules. +:c:func:`!PySpam_System` is also exported to other extension modules. -The function :c:func:`PySpam_System` is a plain C function, declared +The function :c:func:`!PySpam_System` is a plain C function, declared ``static`` like everything else:: static int @@ -1278,7 +1278,7 @@ function must take care of initializing the C API pointer array:: } Note that ``PySpam_API`` is declared ``static``; otherwise the pointer -array would disappear when :func:`PyInit_spam` terminates! +array would disappear when :c:func:`!PyInit_spam` terminates! The bulk of the work is in the header file :file:`spammodule.h`, which looks like this:: @@ -1332,8 +1332,8 @@ like this:: #endif /* !defined(Py_SPAMMODULE_H) */ All that a client module must do in order to have access to the function -:c:func:`PySpam_System` is to call the function (or rather macro) -:c:func:`import_spam` in its initialization function:: +:c:func:`!PySpam_System` is to call the function (or rather macro) +:c:func:`!import_spam` in its initialization function:: PyMODINIT_FUNC PyInit_client(void) diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst index 1f2a7d12bc654..c9d2301d1470c 100644 --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -286,9 +286,9 @@ be read-only or read-write. The structures in the table are defined as:: For each entry in the table, a :term:`descriptor` will be constructed and added to the type which will be able to extract a value from the instance structure. The -:attr:`type` field should contain one of the type codes defined in the +:c:member:`~PyMemberDef.type` field should contain one of the type codes defined in the :file:`structmember.h` header; the value will be used to determine how to -convert Python values to and from C values. The :attr:`flags` field is used to +convert Python values to and from C values. The :c:member:`~PyMemberDef.flags` field is used to store flags which control how the attribute can be accessed. The following flag constants are defined in :file:`structmember.h`; they may be @@ -305,10 +305,10 @@ combined using bitwise-OR. +---------------------------+----------------------------------------------+ .. versionchanged:: 3.10 - :const:`RESTRICTED`, :const:`READ_RESTRICTED` and :const:`WRITE_RESTRICTED` - are deprecated. However, :const:`READ_RESTRICTED` is an alias for - :const:`PY_AUDIT_READ`, so fields that specify either :const:`RESTRICTED` - or :const:`READ_RESTRICTED` will also raise an audit event. + :c:macro:`RESTRICTED`, :c:macro:`READ_RESTRICTED` and :c:macro:`WRITE_RESTRICTED` + are deprecated. However, :c:macro:`READ_RESTRICTED` is an alias for + :c:macro:`PY_AUDIT_READ`, so fields that specify either :c:macro:`RESTRICTED` + or :c:macro:`READ_RESTRICTED` will also raise an audit event. .. index:: single: READONLY @@ -323,7 +323,7 @@ have an associated doc string simply by providing the text in the table. An application can use the introspection API to retrieve the descriptor from the class object, and get the doc string using its :attr:`__doc__` attribute. -As with the :c:member:`~PyTypeObject.tp_methods` table, a sentinel entry with a :attr:`name` value +As with the :c:member:`~PyTypeObject.tp_methods` table, a sentinel entry with a :c:member:`~PyMethodDef.name` value of ``NULL`` is required. .. XXX Descriptors need to be explained in more detail somewhere, but not here. @@ -348,7 +348,7 @@ called, so that if you do need to extend their functionality, you'll understand what needs to be done. The :c:member:`~PyTypeObject.tp_getattr` handler is called when the object requires an attribute -look-up. It is called in the same situations where the :meth:`__getattr__` +look-up. It is called in the same situations where the :meth:`~object.__getattr__` method of a class would be called. Here is an example:: @@ -367,8 +367,8 @@ Here is an example:: return NULL; } -The :c:member:`~PyTypeObject.tp_setattr` handler is called when the :meth:`__setattr__` or -:meth:`__delattr__` method of a class instance would be called. When an +The :c:member:`~PyTypeObject.tp_setattr` handler is called when the :meth:`~object.__setattr__` or +:meth:`~object.__delattr__` method of a class instance would be called. When an attribute should be deleted, the third parameter will be ``NULL``. Here is an example that simply raises an exception; if this were really all you wanted, the :c:member:`~PyTypeObject.tp_setattr` handler should be set to ``NULL``. :: @@ -389,7 +389,7 @@ Object Comparison The :c:member:`~PyTypeObject.tp_richcompare` handler is called when comparisons are needed. It is analogous to the :ref:`rich comparison methods `, like -:meth:`__lt__`, and also called by :c:func:`PyObject_RichCompare` and +:meth:`!__lt__`, and also called by :c:func:`PyObject_RichCompare` and :c:func:`PyObject_RichCompareBool`. This function is called with two Python objects and the operator as arguments, @@ -530,7 +530,7 @@ These functions provide support for the iterator protocol. Both handlers take exactly one parameter, the instance for which they are being called, and return a new reference. In the case of an error, they should set an exception and return ``NULL``. :c:member:`~PyTypeObject.tp_iter` corresponds -to the Python :meth:`__iter__` method, while :c:member:`~PyTypeObject.tp_iternext` +to the Python :meth:`~object.__iter__` method, while :c:member:`~PyTypeObject.tp_iternext` corresponds to the Python :meth:`~iterator.__next__` method. Any :term:`iterable` object must implement the :c:member:`~PyTypeObject.tp_iter` diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index e35175c982599..f4684ceb1a72a 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -145,7 +145,7 @@ only used for variable-sized objects and should otherwise be zero. :c:member:`~PyTypeObject.tp_basicsize` as its base type, you may have problems with multiple inheritance. A Python subclass of your type will have to list your type first in its :attr:`~class.__bases__`, or else it will not be able to call your type's - :meth:`__new__` method without getting an error. You can avoid this problem by + :meth:`~object.__new__` method without getting an error. You can avoid this problem by ensuring that your type has a larger value for :c:member:`~PyTypeObject.tp_basicsize` than its base type does. Most of the time, this will be true anyway, because either your base type will be :class:`object`, or else you will be adding data members to @@ -164,14 +164,14 @@ We provide a doc string for the type in :c:member:`~PyTypeObject.tp_doc`. :: .tp_doc = PyDoc_STR("Custom objects"), To enable object creation, we have to provide a :c:member:`~PyTypeObject.tp_new` -handler. This is the equivalent of the Python method :meth:`__new__`, but +handler. This is the equivalent of the Python method :meth:`~object.__new__`, but has to be specified explicitly. In this case, we can just use the default implementation provided by the API function :c:func:`PyType_GenericNew`. :: .tp_new = PyType_GenericNew, Everything else in the file should be familiar, except for some code in -:c:func:`PyInit_custom`:: +:c:func:`!PyInit_custom`:: if (PyType_Ready(&CustomType) < 0) return; @@ -220,7 +220,7 @@ Of course, the current Custom type is pretty uninteresting. It has no data and doesn't do anything. It can't even be subclassed. .. note:: - While this documentation showcases the standard :mod:`distutils` module + While this documentation showcases the standard :mod:`!distutils` module for building C extensions, it is recommended in real-world use cases to use the newer and better-maintained ``setuptools`` library. Documentation on how to do this is out of scope for this document and can be found in @@ -279,7 +279,7 @@ This method first clears the reference counts of the two Python attributes. ``NULL`` (which might happen here if ``tp_new`` failed midway). It then calls the :c:member:`~PyTypeObject.tp_free` member of the object's type (computed by ``Py_TYPE(self)``) to free the object's memory. Note that -the object's type might not be :class:`CustomType`, because the object may +the object's type might not be :class:`!CustomType`, because the object may be an instance of a subclass. .. note:: @@ -318,7 +318,7 @@ and install it in the :c:member:`~PyTypeObject.tp_new` member:: .tp_new = Custom_new, The ``tp_new`` handler is responsible for creating (as opposed to initializing) -objects of the type. It is exposed in Python as the :meth:`__new__` method. +objects of the type. It is exposed in Python as the :meth:`~object.__new__` method. It is not required to define a ``tp_new`` member, and indeed many extension types will simply reuse :c:func:`PyType_GenericNew` as done in the first version of the :class:`!Custom` type above. In this case, we use the ``tp_new`` @@ -352,7 +352,7 @@ result against ``NULL`` before proceeding. .. note:: If you are creating a co-operative :c:member:`~PyTypeObject.tp_new` (one - that calls a base type's :c:member:`~PyTypeObject.tp_new` or :meth:`__new__`), + that calls a base type's :c:member:`~PyTypeObject.tp_new` or :meth:`~object.__new__`), you must *not* try to determine what method to call using method resolution order at runtime. Always statically determine what type you are going to call, and call its :c:member:`~PyTypeObject.tp_new` directly, or via @@ -395,14 +395,14 @@ by filling the :c:member:`~PyTypeObject.tp_init` slot. :: .tp_init = (initproc) Custom_init, The :c:member:`~PyTypeObject.tp_init` slot is exposed in Python as the -:meth:`__init__` method. It is used to initialize an object after it's +:meth:`~object.__init__` method. It is used to initialize an object after it's created. Initializers always accept positional and keyword arguments, and they should return either ``0`` on success or ``-1`` on error. Unlike the ``tp_new`` handler, there is no guarantee that ``tp_init`` is called at all (for example, the :mod:`pickle` module by default -doesn't call :meth:`__init__` on unpickled instances). It can also be -called multiple times. Anyone can call the :meth:`__init__` method on +doesn't call :meth:`~object.__init__` on unpickled instances). It can also be +called multiple times. Anyone can call the :meth:`!__init__` method on our objects. For this reason, we have to be extra careful when assigning the new attribute values. We might be tempted, for example to assign the ``first`` member like this:: @@ -715,8 +715,8 @@ participate in cycles:: } For each subobject that can participate in cycles, we need to call the -:c:func:`visit` function, which is passed to the traversal method. The -:c:func:`visit` function takes as arguments the subobject and the extra argument +:c:func:`!visit` function, which is passed to the traversal method. The +:c:func:`!visit` function takes as arguments the subobject and the extra argument *arg* passed to the traversal method. It returns an integer value that must be returned if it is non-zero. @@ -798,9 +798,9 @@ types. It is easiest to inherit from the built in types, since an extension can easily use the :c:type:`PyTypeObject` it needs. It can be difficult to share these :c:type:`PyTypeObject` structures between extension modules. -In this example we will create a :class:`SubList` type that inherits from the +In this example we will create a :class:`!SubList` type that inherits from the built-in :class:`list` type. The new type will be completely compatible with -regular lists, but will have an additional :meth:`increment` method that +regular lists, but will have an additional :meth:`!increment` method that increases an internal counter: .. code-block:: pycon @@ -830,7 +830,7 @@ The primary difference for derived type objects is that the base type's object structure must be the first value. The base type will already include the :c:func:`PyObject_HEAD` at the beginning of its structure. -When a Python object is a :class:`SubList` instance, its ``PyObject *`` pointer +When a Python object is a :class:`!SubList` instance, its ``PyObject *`` pointer can be safely cast to both ``PyListObject *`` and ``SubListObject *``:: static int @@ -842,7 +842,7 @@ can be safely cast to both ``PyListObject *`` and ``SubListObject *``:: return 0; } -We see above how to call through to the :attr:`__init__` method of the base +We see above how to call through to the :meth:`~object.__init__` method of the base type. This pattern is important when writing a type with custom diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 3688c47f0d6ec..1d9424cb735a4 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -779,8 +779,8 @@ by a search through the class's :term:`method resolution order`. If a descriptor is found, it is invoked with ``desc.__get__(None, A)``. -The full C implementation can be found in :c:func:`type_getattro()` and -:c:func:`_PyType_Lookup()` in :source:`Objects/typeobject.c`. +The full C implementation can be found in :c:func:`!type_getattro` and +:c:func:`!_PyType_Lookup` in :source:`Objects/typeobject.c`. Invocation from super @@ -794,7 +794,7 @@ for the base class ``B`` immediately following ``A`` and then returns ``B.__dict__['m'].__get__(obj, A)``. If not a descriptor, ``m`` is returned unchanged. -The full C implementation can be found in :c:func:`super_getattro()` in +The full C implementation can be found in :c:func:`!super_getattro` in :source:`Objects/typeobject.c`. A pure Python equivalent can be found in `Guido's Tutorial `_. @@ -836,8 +836,8 @@ and if they define :meth:`__set_name__`, that method is called with two arguments. The *owner* is the class where the descriptor is used, and the *name* is the class variable the descriptor was assigned to. -The implementation details are in :c:func:`type_new()` and -:c:func:`set_names()` in :source:`Objects/typeobject.c`. +The implementation details are in :c:func:`!type_new` and +:c:func:`!set_names` in :source:`Objects/typeobject.c`. Since the update logic is in :meth:`type.__new__`, notifications only take place at the time of class creation. If descriptors are added to the class diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore new file mode 100644 index 0000000000000..27f483b74184b --- /dev/null +++ b/Doc/tools/.nitignore @@ -0,0 +1,239 @@ +# All RST files under Doc/ -- except these -- must pass Sphinx nit-picky mode, +# as tested on the CI via check-warnings.py in reusable-docs.yml. +# Keep lines sorted lexicographically to help avoid merge conflicts. + +Doc/c-api/bool.rst +Doc/c-api/buffer.rst +Doc/c-api/datetime.rst +Doc/c-api/descriptor.rst +Doc/c-api/exceptions.rst +Doc/c-api/file.rst +Doc/c-api/float.rst +Doc/c-api/gcsupport.rst +Doc/c-api/init.rst +Doc/c-api/init_config.rst +Doc/c-api/intro.rst +Doc/c-api/memory.rst +Doc/c-api/memoryview.rst +Doc/c-api/module.rst +Doc/c-api/object.rst +Doc/c-api/set.rst +Doc/c-api/stable.rst +Doc/c-api/structures.rst +Doc/c-api/sys.rst +Doc/c-api/type.rst +Doc/c-api/typeobj.rst +Doc/c-api/unicode.rst +Doc/extending/extending.rst +Doc/extending/newtypes.rst +Doc/faq/design.rst +Doc/faq/extending.rst +Doc/faq/gui.rst +Doc/faq/library.rst +Doc/faq/programming.rst +Doc/glossary.rst +Doc/howto/curses.rst +Doc/howto/descriptor.rst +Doc/howto/enum.rst +Doc/howto/isolating-extensions.rst +Doc/howto/logging-cookbook.rst +Doc/howto/logging.rst +Doc/howto/urllib2.rst +Doc/install/index.rst +Doc/library/__future__.rst +Doc/library/abc.rst +Doc/library/ast.rst +Doc/library/asyncio-dev.rst +Doc/library/asyncio-eventloop.rst +Doc/library/asyncio-extending.rst +Doc/library/asyncio-future.rst +Doc/library/asyncio-policy.rst +Doc/library/asyncio-stream.rst +Doc/library/asyncio-subprocess.rst +Doc/library/asyncio-task.rst +Doc/library/bdb.rst +Doc/library/bisect.rst +Doc/library/bz2.rst +Doc/library/calendar.rst +Doc/library/cmd.rst +Doc/library/code.rst +Doc/library/codecs.rst +Doc/library/collections.abc.rst +Doc/library/collections.rst +Doc/library/concurrent.futures.rst +Doc/library/concurrent.rst +Doc/library/configparser.rst +Doc/library/contextlib.rst +Doc/library/copy.rst +Doc/library/csv.rst +Doc/library/ctypes.rst +Doc/library/datetime.rst +Doc/library/dbm.rst +Doc/library/decimal.rst +Doc/library/devmode.rst +Doc/library/difflib.rst +Doc/library/dis.rst +Doc/library/doctest.rst +Doc/library/email.charset.rst +Doc/library/email.compat32-message.rst +Doc/library/email.encoders.rst +Doc/library/email.errors.rst +Doc/library/email.generator.rst +Doc/library/email.headerregistry.rst +Doc/library/email.message.rst +Doc/library/email.mime.rst +Doc/library/email.parser.rst +Doc/library/email.policy.rst +Doc/library/enum.rst +Doc/library/exceptions.rst +Doc/library/faulthandler.rst +Doc/library/fcntl.rst +Doc/library/filecmp.rst +Doc/library/fileinput.rst +Doc/library/fractions.rst +Doc/library/ftplib.rst +Doc/library/functions.rst +Doc/library/functools.rst +Doc/library/getpass.rst +Doc/library/gettext.rst +Doc/library/graphlib.rst +Doc/library/gzip.rst +Doc/library/hashlib.rst +Doc/library/http.client.rst +Doc/library/http.cookiejar.rst +Doc/library/http.cookies.rst +Doc/library/http.server.rst +Doc/library/importlib.resources.abc.rst +Doc/library/importlib.resources.rst +Doc/library/importlib.rst +Doc/library/inspect.rst +Doc/library/io.rst +Doc/library/json.rst +Doc/library/locale.rst +Doc/library/logging.config.rst +Doc/library/logging.handlers.rst +Doc/library/logging.rst +Doc/library/lzma.rst +Doc/library/mailbox.rst +Doc/library/mmap.rst +Doc/library/msvcrt.rst +Doc/library/multiprocessing.rst +Doc/library/multiprocessing.shared_memory.rst +Doc/library/netrc.rst +Doc/library/numbers.rst +Doc/library/operator.rst +Doc/library/optparse.rst +Doc/library/os.path.rst +Doc/library/os.rst +Doc/library/pickle.rst +Doc/library/pickletools.rst +Doc/library/platform.rst +Doc/library/plistlib.rst +Doc/library/poplib.rst +Doc/library/pprint.rst +Doc/library/profile.rst +Doc/library/pty.rst +Doc/library/pyclbr.rst +Doc/library/pydoc.rst +Doc/library/pyexpat.rst +Doc/library/random.rst +Doc/library/readline.rst +Doc/library/reprlib.rst +Doc/library/resource.rst +Doc/library/rlcompleter.rst +Doc/library/sched.rst +Doc/library/select.rst +Doc/library/selectors.rst +Doc/library/shelve.rst +Doc/library/shutil.rst +Doc/library/signal.rst +Doc/library/site.rst +Doc/library/smtplib.rst +Doc/library/socket.rst +Doc/library/socketserver.rst +Doc/library/ssl.rst +Doc/library/stat.rst +Doc/library/stdtypes.rst +Doc/library/string.rst +Doc/library/subprocess.rst +Doc/library/sys.rst +Doc/library/sys_path_init.rst +Doc/library/sysconfig.rst +Doc/library/syslog.rst +Doc/library/tarfile.rst +Doc/library/tempfile.rst +Doc/library/termios.rst +Doc/library/test.rst +Doc/library/textwrap.rst +Doc/library/threading.rst +Doc/library/time.rst +Doc/library/tkinter.rst +Doc/library/tkinter.scrolledtext.rst +Doc/library/tkinter.ttk.rst +Doc/library/traceback.rst +Doc/library/tty.rst +Doc/library/turtle.rst +Doc/library/unittest.mock-examples.rst +Doc/library/unittest.mock.rst +Doc/library/unittest.rst +Doc/library/urllib.error.rst +Doc/library/urllib.parse.rst +Doc/library/urllib.request.rst +Doc/library/uuid.rst +Doc/library/weakref.rst +Doc/library/winreg.rst +Doc/library/winsound.rst +Doc/library/wsgiref.rst +Doc/library/xml.dom.minidom.rst +Doc/library/xml.dom.pulldom.rst +Doc/library/xml.dom.rst +Doc/library/xml.etree.elementtree.rst +Doc/library/xml.rst +Doc/library/xml.sax.handler.rst +Doc/library/xml.sax.reader.rst +Doc/library/xml.sax.rst +Doc/library/xml.sax.utils.rst +Doc/library/xmlrpc.client.rst +Doc/library/xmlrpc.rst +Doc/library/xmlrpc.server.rst +Doc/library/zlib.rst +Doc/license.rst +Doc/reference/compound_stmts.rst +Doc/reference/datamodel.rst +Doc/reference/expressions.rst +Doc/reference/import.rst +Doc/reference/lexical_analysis.rst +Doc/reference/simple_stmts.rst +Doc/tutorial/appendix.rst +Doc/tutorial/classes.rst +Doc/tutorial/controlflow.rst +Doc/tutorial/datastructures.rst +Doc/tutorial/errors.rst +Doc/tutorial/inputoutput.rst +Doc/tutorial/interactive.rst +Doc/tutorial/introduction.rst +Doc/tutorial/modules.rst +Doc/tutorial/stdlib2.rst +Doc/using/cmdline.rst +Doc/using/configure.rst +Doc/using/windows.rst +Doc/whatsnew/2.0.rst +Doc/whatsnew/2.1.rst +Doc/whatsnew/2.2.rst +Doc/whatsnew/2.3.rst +Doc/whatsnew/2.4.rst +Doc/whatsnew/2.5.rst +Doc/whatsnew/2.6.rst +Doc/whatsnew/2.7.rst +Doc/whatsnew/3.0.rst +Doc/whatsnew/3.1.rst +Doc/whatsnew/3.2.rst +Doc/whatsnew/3.3.rst +Doc/whatsnew/3.4.rst +Doc/whatsnew/3.5.rst +Doc/whatsnew/3.6.rst +Doc/whatsnew/3.7.rst +Doc/whatsnew/3.8.rst +Doc/whatsnew/3.9.rst +Doc/whatsnew/3.10.rst +Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index 162d2c342f50f..a15eefc2953f4 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -2151,8 +2151,8 @@ Changes to Python's build process and to the C API include: Previously these different families all reduced to the platform's :c:func:`malloc` and :c:func:`free` functions. This meant it didn't matter if - you got things wrong and allocated memory with the :c:func:`PyMem` function but - freed it with the :c:func:`PyObject` function. With 2.5's changes to obmalloc, + you got things wrong and allocated memory with the ``PyMem`` function but + freed it with the ``PyObject`` function. With 2.5's changes to obmalloc, these families now do different things and mismatches will probably result in a segfault. You should carefully test your C extension modules with Python 2.5. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 7bfff94c423f8..b3a6f114dd444 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1847,7 +1847,7 @@ Changes in Python behavior finalizing, making them consistent with :c:func:`PyEval_RestoreThread`, :c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`. If this behavior is not desired, guard the call by checking :c:func:`_Py_IsFinalizing` - or :c:func:`sys.is_finalizing`. + or :func:`sys.is_finalizing`. (Contributed by Joannah Nanjekye in :issue:`36475`.) From webhook-mailer at python.org Fri Jul 28 04:48:40 2023 From: webhook-mailer at python.org (markshannon) Date: Fri, 28 Jul 2023 08:48:40 -0000 Subject: [Python-checkins] [3.12] GH-106897: Add `RERAISE` event to `sys.monitoring`. (GH-107291) (GH-107346) Message-ID: https://github.com/python/cpython/commit/3b1a4c18426c78a2fda0d59728bfe9eb92889722 commit: 3b1a4c18426c78a2fda0d59728bfe9eb92889722 branch: 3.12 author: Mark Shannon committer: markshannon date: 2023-07-28T09:48:35+01:00 summary: [3.12] GH-106897: Add `RERAISE` event to `sys.monitoring`. (GH-107291) (GH-107346) * Ensures that exception handling events are balanced. Each [re]raise event has a matching unwind/handled event. files: A Misc/NEWS.d/next/Core and Builtins/2023-07-26-12-18-10.gh-issue-106897.EsGurc.rst M Doc/data/python3.12.abi M Include/cpython/code.h M Include/internal/pycore_instruments.h M Lib/test/test_monitoring.py M Python/bytecodes.c M Python/ceval.c M Python/generated_cases.c.h M Python/instrumentation.c diff --git a/Doc/data/python3.12.abi b/Doc/data/python3.12.abi index 4cd130f86e2f7..4c92e1d43c965 100644 --- a/Doc/data/python3.12.abi +++ b/Doc/data/python3.12.abi @@ -739,6 +739,7 @@ + @@ -1165,6 +1166,7 @@ + @@ -1703,7 +1705,7 @@ - + @@ -1721,7 +1723,7 @@ - + @@ -1731,10 +1733,10 @@ - + - + @@ -1822,11 +1824,11 @@ - + - + @@ -1838,14 +1840,14 @@ - + - + @@ -1864,7 +1866,7 @@ - + @@ -1874,7 +1876,7 @@ - + @@ -1916,13 +1918,13 @@ - + - + @@ -1930,7 +1932,7 @@ - + @@ -2010,7 +2012,7 @@ - + @@ -2041,7 +2043,7 @@ - + @@ -2069,7 +2071,7 @@ - + @@ -3366,7 +3368,7 @@ - + @@ -3420,7 +3422,7 @@ - + @@ -3526,7 +3528,7 @@ - + @@ -3535,7 +3537,7 @@ - + @@ -3591,7 +3593,7 @@ - + @@ -3613,12 +3615,12 @@ - + - + @@ -3635,7 +3637,7 @@ - + @@ -3718,58 +3720,58 @@ - - - - + + + + - + - - + + - - + + - + - - + + - + - + - - + + - - - - + + + + - - - + + + - - + + - - + + @@ -3777,7 +3779,7 @@ - + @@ -3848,7 +3850,7 @@ - + @@ -3907,7 +3909,7 @@ - + @@ -3930,7 +3932,7 @@ - + @@ -3984,13 +3986,13 @@ - + - + @@ -4085,18 +4087,18 @@ - + - + - + @@ -4659,7 +4661,7 @@ - + @@ -4687,13 +4689,13 @@ - + - + @@ -4900,27 +4902,27 @@ - + - + - + - + - + @@ -5034,7 +5036,7 @@ - + @@ -5131,7 +5133,7 @@ - + @@ -5189,7 +5191,7 @@ - + @@ -5249,77 +5251,77 @@ - - - - - - + + + + + + - - - + + + - - - + + + - - + + - - - + + + - - - + + + - - + + - - + + - - - - + + + + - - - - + + + + - - - + + + - - - + + + - - - - - + + + + + - + @@ -5346,12 +5348,12 @@ - + - + @@ -5411,12 +5413,12 @@ - + - + @@ -5573,7 +5575,7 @@ - + @@ -5638,9 +5640,9 @@ - + - + @@ -5660,7 +5662,7 @@ - + @@ -5672,21 +5674,21 @@ - + - + - + - + @@ -5716,7 +5718,7 @@ - + @@ -5970,12 +5972,12 @@ - - + + - + @@ -6092,7 +6094,7 @@ - + @@ -6114,24 +6116,24 @@ - + - + - + - + - + @@ -6169,11 +6171,11 @@ - + - + @@ -6229,8 +6231,8 @@ - - + + @@ -6300,6 +6302,11 @@ + + + + + @@ -6374,22 +6381,12 @@ - - - - - - - - - - @@ -6529,7 +6526,7 @@ - + @@ -6538,7 +6535,7 @@ - + @@ -6600,11 +6597,11 @@ - + - + @@ -6891,7 +6888,7 @@ - + @@ -6980,7 +6977,7 @@ - + @@ -7022,7 +7019,7 @@ - + @@ -7173,7 +7170,7 @@ - + @@ -7213,6 +7210,16 @@ + + + + + + + + + + @@ -7248,71 +7255,66 @@ - - - + + + - - - + + + - - + + - - + + - - - + + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - + - - - - - @@ -7326,80 +7328,80 @@ - - - - + + + + - - - + + + - - + + - - + + - - + + - - + + - - - + + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - - + + + - - + + - - + + - + @@ -7419,7 +7421,7 @@ - + @@ -7428,7 +7430,7 @@ - + @@ -7474,10 +7476,6 @@ - - - - @@ -7485,10 +7483,14 @@ - + + + + + @@ -7569,7 +7571,7 @@ - + @@ -7625,7 +7627,7 @@ - + @@ -7639,7 +7641,7 @@ - + @@ -7653,15 +7655,15 @@ - + - + - + @@ -7696,7 +7698,7 @@ - + @@ -7721,8 +7723,8 @@ - - + + @@ -7851,7 +7853,7 @@ - + @@ -7879,7 +7881,7 @@ - + @@ -7906,7 +7908,7 @@ - + @@ -7967,7 +7969,7 @@ - + @@ -7998,7 +8000,7 @@ - + @@ -8088,7 +8090,7 @@ - + @@ -8100,7 +8102,7 @@ - + @@ -8138,12 +8140,9 @@ - - - - - - + + + @@ -8162,6 +8161,9 @@ + + + @@ -8226,8 +8228,8 @@ - - + + @@ -8369,41 +8371,41 @@ - - + + - - + + - - + + - - + + - - - + + + - - + + - - + + - - + + - + @@ -8538,7 +8540,7 @@ - + @@ -8583,7 +8585,7 @@ - + @@ -8598,7 +8600,7 @@ - + @@ -8611,7 +8613,7 @@ - + @@ -8639,7 +8641,7 @@ - + @@ -8667,41 +8669,41 @@ - - + + - + - + - - + + - + - + - + - + - + - + - + @@ -8738,7 +8740,7 @@ - + @@ -8757,7 +8759,7 @@ - + @@ -8849,9 +8851,9 @@ - - - + + + @@ -8869,116 +8871,120 @@ - + + + + + - - + + - - + + - - - + + + - - - + + + - - + + - - + + - - - + + + - - + + - - + + - - - + + + - - - - - + + + + + - - - - + + + + - - + + - - + + - - - + + + - - + + - - + + - - - + + + - - - + + + - - + + - - + + - - - + + + - + - + @@ -8994,7 +9000,7 @@ - + @@ -9016,7 +9022,7 @@ - + @@ -9345,10 +9351,6 @@ - - - - @@ -9681,11 +9683,11 @@ - + - + @@ -9699,7 +9701,7 @@ - + @@ -9717,7 +9719,7 @@ - + @@ -10803,6 +10805,10 @@ + + + + @@ -10837,7 +10843,7 @@ - + @@ -10868,61 +10874,61 @@ - + - + - - + + - + - + - + - + - + - + - + - - + + - - + + - - + + - + - + - - - + + + - - + + - + - - + + @@ -10954,69 +10960,69 @@ - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - + - - + + @@ -11028,7 +11034,7 @@ - + @@ -11046,7 +11052,7 @@ - + @@ -11058,7 +11064,7 @@ - + @@ -11099,9 +11105,9 @@ - + - + @@ -11150,7 +11156,7 @@ - + @@ -11659,7 +11665,7 @@ - + @@ -11703,7 +11709,7 @@ - + @@ -11743,7 +11749,7 @@ - + @@ -12759,13 +12765,13 @@ - + - + @@ -12830,7 +12836,7 @@ - + @@ -12839,7 +12845,7 @@ - + @@ -12987,7 +12993,7 @@ - + @@ -13005,7 +13011,7 @@ - + @@ -13016,7 +13022,7 @@ - + @@ -13247,7 +13253,7 @@ - + @@ -13258,7 +13264,7 @@ - + @@ -13283,16 +13289,16 @@ - + - + - + @@ -13312,7 +13318,7 @@ - + @@ -13363,7 +13369,7 @@ - + @@ -13384,7 +13390,7 @@ - + @@ -13406,7 +13412,7 @@ - + @@ -13454,10 +13460,10 @@ - + - + @@ -13486,7 +13492,7 @@ - + @@ -13501,7 +13507,7 @@ - + @@ -13522,13 +13528,13 @@ - + - + - + @@ -13545,7 +13551,7 @@ - + @@ -13553,7 +13559,7 @@ - + @@ -13564,7 +13570,7 @@ - + @@ -13593,7 +13599,7 @@ - + @@ -13601,7 +13607,7 @@ - + @@ -13609,7 +13615,7 @@ - + @@ -13665,7 +13671,7 @@ - + @@ -14265,7 +14271,7 @@ - + @@ -14325,7 +14331,7 @@ - + @@ -14379,7 +14385,7 @@ - + @@ -14538,7 +14544,7 @@ - + @@ -14622,7 +14628,7 @@ - + @@ -15105,7 +15111,7 @@ - + @@ -15240,7 +15246,7 @@ - + @@ -15324,7 +15330,7 @@ - + @@ -15396,7 +15402,7 @@ - + @@ -15702,7 +15708,7 @@ - + @@ -15716,7 +15722,7 @@ - + @@ -15740,7 +15746,7 @@ - + @@ -15780,7 +15786,7 @@ - + @@ -15839,7 +15845,7 @@ - + @@ -16031,7 +16037,7 @@ - + @@ -16126,7 +16132,7 @@ - + @@ -16150,7 +16156,7 @@ - + @@ -16212,14 +16218,14 @@ - - - - + - + + + + @@ -16230,16 +16236,16 @@ - + - + - + - + @@ -16325,7 +16331,7 @@ - + @@ -16343,32 +16349,32 @@ - + - + - + - + - + - + - + @@ -16383,7 +16389,7 @@ - + @@ -16391,20 +16397,20 @@ - + - + - + - + @@ -16551,7 +16557,7 @@ - + @@ -16755,7 +16761,7 @@ - + @@ -16774,7 +16780,7 @@ - + @@ -16828,10 +16834,10 @@ - + - + @@ -16839,7 +16845,7 @@ - + @@ -16852,7 +16858,7 @@ - + @@ -16887,7 +16893,7 @@ - + @@ -17004,7 +17010,7 @@ - + @@ -17017,7 +17023,7 @@ - + @@ -17029,7 +17035,7 @@ - + @@ -17169,7 +17175,7 @@ - + @@ -17304,7 +17310,7 @@ - + @@ -17325,10 +17331,10 @@ - + - + @@ -17352,15 +17358,15 @@ - - - + + + - + @@ -17380,7 +17386,7 @@ - + @@ -17396,13 +17402,13 @@ - + - - - + + + @@ -17628,7 +17634,7 @@ - + @@ -17754,7 +17760,7 @@ - + @@ -17900,16 +17906,13 @@ - + - + - - - - + @@ -18929,7 +18932,7 @@ - + @@ -18967,7 +18970,7 @@ - + @@ -18975,10 +18978,10 @@ - + - + @@ -19012,7 +19015,7 @@ - + @@ -19941,61 +19944,61 @@ - + - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + @@ -20374,7 +20377,7 @@ - + @@ -20520,16 +20523,13 @@ - + - + - - - - + @@ -20702,7 +20702,7 @@ - + @@ -20835,38 +20835,38 @@ - + - + - + - + - + - + - + @@ -20981,7 +20981,7 @@ - + @@ -20993,7 +20993,7 @@ - + @@ -21049,7 +21049,7 @@ - + @@ -21104,7 +21104,7 @@ - + @@ -21129,7 +21129,7 @@ - + @@ -21189,7 +21189,7 @@ - + @@ -21273,30 +21273,30 @@ - - + + - + - + - + - + @@ -21307,7 +21307,7 @@ - + @@ -21335,7 +21335,7 @@ - + @@ -21375,7 +21375,7 @@ - + @@ -21395,7 +21395,7 @@ - + @@ -21408,7 +21408,7 @@ - + @@ -21509,13 +21509,13 @@ - + - + @@ -21564,7 +21564,7 @@ - + @@ -21734,16 +21734,21 @@ - + + + + + + - + - + @@ -21757,27 +21762,27 @@ - + - + - + - + @@ -21785,7 +21790,7 @@ - + @@ -21793,7 +21798,7 @@ - + @@ -21802,14 +21807,14 @@ - + - + @@ -21885,81 +21890,84 @@ - + - - + + - - + + - - - + + + - - + + - - + + - - - + + + - - - + + + - - - + + + - - - + + + - + - - + + + + + - - + + - - + + - - + + - - + + - + - + - + @@ -22124,7 +22132,7 @@ - + @@ -22215,7 +22223,7 @@ - + @@ -22771,17 +22779,17 @@ - + - + - + @@ -22870,7 +22878,7 @@ - + @@ -22881,7 +22889,7 @@ - + @@ -23033,7 +23041,7 @@ - + @@ -23047,7 +23055,7 @@ - + @@ -23171,7 +23179,7 @@ - + @@ -23181,7 +23189,7 @@ - + @@ -23267,7 +23275,7 @@ - + @@ -23281,7 +23289,7 @@ - + @@ -23386,7 +23394,7 @@ - + @@ -23458,7 +23466,7 @@ - + @@ -23563,21 +23571,21 @@ - + - + - + - + @@ -23586,7 +23594,7 @@ - + @@ -23660,7 +23668,7 @@ - + @@ -23876,7 +23884,7 @@ - + @@ -24217,13 +24225,13 @@ - - - + + + - + - + @@ -24240,7 +24248,7 @@ - + @@ -24296,28 +24304,28 @@ - - - + + + - + - + - + - + @@ -24397,32 +24405,32 @@ - - + + - - + + - - + + - - + + - - - + + + - + - + @@ -24438,20 +24446,26 @@ - - - - + + + + + + + + + + - - - - + + + + - + @@ -24467,7 +24481,7 @@ - + @@ -24539,7 +24553,7 @@ - + @@ -24548,7 +24562,7 @@ - + @@ -24584,7 +24598,7 @@ - + @@ -24596,7 +24610,7 @@ - + @@ -24681,31 +24695,31 @@ - - + + - + - + - + - + - + - + - + - + @@ -25405,7 +25419,7 @@ - + @@ -25612,14 +25626,14 @@ - + - + @@ -25638,7 +25652,7 @@ - + @@ -25875,7 +25889,7 @@ - + @@ -26132,7 +26146,7 @@ - + @@ -26163,7 +26177,7 @@ - + @@ -26175,10 +26189,10 @@ - + - + @@ -26358,33 +26372,33 @@ - - - - + + + + - + - - - - - + - + - + + + + + - - + + - + @@ -26392,7 +26406,7 @@ - + @@ -26418,7 +26432,7 @@ - + diff --git a/Include/cpython/code.h b/Include/cpython/code.h index 7449a98e4326e..565b3942ad2d4 100644 --- a/Include/cpython/code.h +++ b/Include/cpython/code.h @@ -10,9 +10,9 @@ extern "C" { /* Count of all "real" monitoring events (not derived from other events) */ -#define _PY_MONITORING_UNGROUPED_EVENTS 14 +#define _PY_MONITORING_UNGROUPED_EVENTS 15 /* Count of all monitoring events */ -#define _PY_MONITORING_EVENTS 16 +#define _PY_MONITORING_EVENTS 17 /* Table of which tools are active for each monitored event. */ typedef struct _Py_Monitors { diff --git a/Include/internal/pycore_instruments.h b/Include/internal/pycore_instruments.h index 9fb3952227af1..cfa5d09a4704f 100644 --- a/Include/internal/pycore_instruments.h +++ b/Include/internal/pycore_instruments.h @@ -36,12 +36,13 @@ extern "C" { #define PY_MONITORING_EVENT_EXCEPTION_HANDLED 11 #define PY_MONITORING_EVENT_PY_UNWIND 12 #define PY_MONITORING_EVENT_PY_THROW 13 +#define PY_MONITORING_EVENT_RERAISE 14 /* Ancilliary events */ -#define PY_MONITORING_EVENT_C_RETURN 14 -#define PY_MONITORING_EVENT_C_RAISE 15 +#define PY_MONITORING_EVENT_C_RETURN 15 +#define PY_MONITORING_EVENT_C_RAISE 16 typedef uint32_t _PyMonitoringEventSet; diff --git a/Lib/test/test_monitoring.py b/Lib/test/test_monitoring.py index ef4a79aefd646..8bf95a73d7db9 100644 --- a/Lib/test/test_monitoring.py +++ b/Lib/test/test_monitoring.py @@ -8,7 +8,7 @@ import textwrap import types import unittest - +import asyncio PAIR = (0,1) @@ -243,7 +243,6 @@ def check_events(self, func, expected=None): expected = func.events self.assertEqual(events, expected) - class MonitoringEventsTest(MonitoringEventsBase, unittest.TestCase): def test_just_pass(self): @@ -632,7 +631,7 @@ def __call__(self, code, offset, exc): class CheckEvents(MonitoringTestBase, unittest.TestCase): - def check_events(self, func, expected, tool=TEST_TOOL, recorders=(ExceptionRecorder,)): + def get_events(self, func, tool, recorders): try: self.assertEqual(sys.monitoring._all_events(), {}) event_list = [] @@ -646,19 +645,63 @@ def check_events(self, func, expected, tool=TEST_TOOL, recorders=(ExceptionRecor sys.monitoring.set_events(tool, 0) for recorder in recorders: sys.monitoring.register_callback(tool, recorder.event_type, None) - self.assertEqual(event_list, expected) + return event_list finally: sys.monitoring.set_events(tool, 0) for recorder in recorders: sys.monitoring.register_callback(tool, recorder.event_type, None) + def check_events(self, func, expected, tool=TEST_TOOL, recorders=(ExceptionRecorder,)): + events = self.get_events(func, tool, recorders) + if events != expected: + print(events, file = sys.stderr) + self.assertEqual(events, expected) + + def check_balanced(self, func, recorders): + events = self.get_events(func, TEST_TOOL, recorders) + self.assertEqual(len(events)%2, 0) + for r, h in zip(events[::2],events[1::2]): + r0 = r[0] + self.assertIn(r0, ("raise", "reraise")) + h0 = h[0] + self.assertIn(h0, ("handled", "unwind")) + + + class StopiterationRecorder(ExceptionRecorder): event_type = E.STOP_ITERATION -class ExceptionMontoringTest(CheckEvents): +class ReraiseRecorder(ExceptionRecorder): + + event_type = E.RERAISE + + def __call__(self, code, offset, exc): + self.events.append(("reraise", type(exc))) + +class UnwindRecorder(ExceptionRecorder): + + event_type = E.PY_UNWIND + + def __call__(self, *args): + self.events.append(("unwind", None)) + +class ExceptionHandledRecorder(ExceptionRecorder): + + event_type = E.EXCEPTION_HANDLED + + def __call__(self, code, offset, exc): + self.events.append(("handled", type(exc))) - recorder = ExceptionRecorder +class ExceptionMonitoringTest(CheckEvents): + + + exception_recorders = ( + ExceptionRecorder, + ReraiseRecorder, + ExceptionHandledRecorder, + UnwindRecorder + ) def test_simple_try_except(self): @@ -672,6 +715,8 @@ def func1(): self.check_events(func1, [("raise", KeyError)]) + def test_implicit_stop_iteration(self): + def gen(): yield 1 return 2 @@ -682,6 +727,117 @@ def implicit_stop_iteration(): self.check_events(implicit_stop_iteration, [("raise", StopIteration)], recorders=(StopiterationRecorder,)) + initial = [ + ("raise", ZeroDivisionError), + ("handled", ZeroDivisionError) + ] + + reraise = [ + ("reraise", ZeroDivisionError), + ("handled", ZeroDivisionError) + ] + + def test_explicit_reraise(self): + + def func(): + try: + try: + 1/0 + except: + raise + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + def test_explicit_reraise_named(self): + + def func(): + try: + try: + 1/0 + except Exception as ex: + raise + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + def test_implicit_reraise(self): + + def func(): + try: + try: + 1/0 + except ValueError: + pass + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + + def test_implicit_reraise_named(self): + + def func(): + try: + try: + 1/0 + except ValueError as ex: + pass + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + def test_try_finally(self): + + def func(): + try: + try: + 1/0 + finally: + pass + except: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + + def test_async_for(self): + + def func(): + + async def async_generator(): + for i in range(1): + raise ZeroDivisionError + yield i + + async def async_loop(): + try: + async for item in async_generator(): + pass + except Exception: + pass + + try: + async_loop().send(None) + except StopIteration: + pass + + self.check_balanced( + func, + recorders = self.exception_recorders) + class LineRecorder: event_type = E.LINE @@ -733,12 +889,12 @@ def func1(): line3 = 3 self.check_events(func1, recorders = MANY_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'func1', sys.monitoring.MISSING), ('line', 'func1', 1), ('line', 'func1', 2), ('line', 'func1', 3), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2)]) def test_c_call(self): @@ -749,14 +905,14 @@ def func2(): line3 = 3 self.check_events(func2, recorders = MANY_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'func2', sys.monitoring.MISSING), ('line', 'func2', 1), ('line', 'func2', 2), ('call', 'append', [2]), ('C return', 'append', [2]), ('line', 'func2', 3), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2)]) def test_try_except(self): @@ -770,7 +926,7 @@ def func3(): line = 6 self.check_events(func3, recorders = MANY_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'func3', sys.monitoring.MISSING), ('line', 'func3', 1), ('line', 'func3', 2), @@ -779,7 +935,7 @@ def func3(): ('line', 'func3', 4), ('line', 'func3', 5), ('line', 'func3', 6), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2)]) class InstructionRecorder: @@ -791,7 +947,7 @@ def __init__(self, events): def __call__(self, code, offset): # Filter out instructions in check_events to lower noise - if code.co_name != "check_events": + if code.co_name != "get_events": self.events.append(("instruction", code.co_name, offset)) @@ -808,7 +964,7 @@ def func1(): line3 = 3 self.check_events(func1, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func1', 1), ('instruction', 'func1', 2), ('instruction', 'func1', 4), @@ -819,7 +975,7 @@ def func1(): ('instruction', 'func1', 10), ('instruction', 'func1', 12), ('instruction', 'func1', 14), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) def test_c_call(self): @@ -829,7 +985,7 @@ def func2(): line3 = 3 self.check_events(func2, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func2', 1), ('instruction', 'func2', 2), ('instruction', 'func2', 4), @@ -843,7 +999,7 @@ def func2(): ('instruction', 'func2', 40), ('instruction', 'func2', 42), ('instruction', 'func2', 44), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) def test_try_except(self): @@ -856,7 +1012,7 @@ def func3(): line = 6 self.check_events(func3, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func3', 1), ('instruction', 'func3', 2), ('line', 'func3', 2), @@ -876,7 +1032,7 @@ def func3(): ('instruction', 'func3', 30), ('instruction', 'func3', 32), ('instruction', 'func3', 34), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) def test_with_restart(self): def func1(): @@ -885,7 +1041,7 @@ def func1(): line3 = 3 self.check_events(func1, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func1', 1), ('instruction', 'func1', 2), ('instruction', 'func1', 4), @@ -896,12 +1052,12 @@ def func1(): ('instruction', 'func1', 10), ('instruction', 'func1', 12), ('instruction', 'func1', 14), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) sys.monitoring.restart_events() self.check_events(func1, recorders = LINE_AND_INSTRUCTION_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func1', 1), ('instruction', 'func1', 2), ('instruction', 'func1', 4), @@ -912,7 +1068,7 @@ def func1(): ('instruction', 'func1', 10), ('instruction', 'func1', 12), ('instruction', 'func1', 14), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) class TestInstallIncrementallly(MonitoringTestBase, unittest.TestCase): @@ -1114,7 +1270,7 @@ def func(): ('branch', 'func', 2, 2)]) self.check_events(func, recorders = JUMP_BRANCH_AND_LINE_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func', 1), ('line', 'func', 2), ('branch', 'func', 2, 2), @@ -1130,7 +1286,7 @@ def func(): ('jump', 'func', 4, 2), ('line', 'func', 2), ('branch', 'func', 2, 2), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) def test_except_star(self): @@ -1149,7 +1305,7 @@ def func(): self.check_events(func, recorders = JUMP_BRANCH_AND_LINE_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func', 1), ('line', 'func', 2), ('line', 'func', 3), @@ -1160,10 +1316,10 @@ def func(): ('jump', 'func', 5, 5), ('jump', 'func', 5, '[offset=114]'), ('branch', 'func', '[offset=120]', '[offset=122]'), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) self.check_events(func, recorders = FLOW_AND_LINE_RECORDERS, expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('line', 'func', 1), ('line', 'func', 2), ('line', 'func', 3), @@ -1177,7 +1333,7 @@ def func(): ('jump', 'func', 5, '[offset=114]'), ('branch', 'func', '[offset=120]', '[offset=122]'), ('return', None), - ('line', 'check_events', 11)]) + ('line', 'get_events', 11)]) class TestLoadSuperAttr(CheckEvents): RECORDERS = CallRecorder, LineRecorder, CRaiseRecorder, CReturnRecorder @@ -1229,7 +1385,7 @@ def f(): """ d = self._exec_super(codestr, optimized) expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'f', sys.monitoring.MISSING), ('line', 'f', 1), ('call', 'method', d["b"]), @@ -1242,7 +1398,7 @@ def f(): ('call', 'method', 1), ('line', 'method', 1), ('line', 'method', 1), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2), ] return d["f"], expected @@ -1280,7 +1436,7 @@ def f(): """ d = self._exec_super(codestr, optimized) expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'f', sys.monitoring.MISSING), ('line', 'f', 1), ('line', 'f', 2), @@ -1293,7 +1449,7 @@ def f(): ('C raise', 'super', 1), ('line', 'f', 3), ('line', 'f', 4), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2), ] return d["f"], expected @@ -1321,7 +1477,7 @@ def f(): """ d = self._exec_super(codestr, optimized) expected = [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'f', sys.monitoring.MISSING), ('line', 'f', 1), ('call', 'method', d["b"]), @@ -1330,7 +1486,7 @@ def f(): ('C return', 'super', sys.monitoring.MISSING), ('line', 'method', 2), ('line', 'method', 1), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2) ] return d["f"], expected @@ -1355,7 +1511,7 @@ def f(): def get_expected(name, call_method, ns): repr_arg = 0 if name == "int" else sys.monitoring.MISSING return [ - ('line', 'check_events', 10), + ('line', 'get_events', 10), ('call', 'f', sys.monitoring.MISSING), ('line', 'f', 1), ('call', 'method', ns["c"]), @@ -1368,7 +1524,7 @@ def get_expected(name, call_method, ns): ('C return', '__repr__', repr_arg), ] if call_method else [] ), - ('line', 'check_events', 11), + ('line', 'get_events', 11), ('call', 'set_events', 2), ] diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-26-12-18-10.gh-issue-106897.EsGurc.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-12-18-10.gh-issue-106897.EsGurc.rst new file mode 100644 index 0000000000000..52c49c3c0f5e7 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-12-18-10.gh-issue-106897.EsGurc.rst @@ -0,0 +1,3 @@ +Add a ``RERAISE`` event to ``sys.monitoring``, which occurs when an +exception is reraised, either explicitly by a plain ``raise`` statement, or +implicitly in an ``except`` or ``finally`` block. diff --git a/Python/bytecodes.c b/Python/bytecodes.c index b914afa07fba4..adfa5ce00b6a0 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -612,7 +612,11 @@ dummy_func( exc = args[0]; /* fall through */ case 0: - ERROR_IF(do_raise(tstate, exc, cause), exception_unwind); + if (do_raise(tstate, exc, cause)) { + assert(oparg == 0); + monitor_reraise(tstate, frame, next_instr-1); + goto exception_unwind; + } break; default: _PyErr_SetString(tstate, PyExc_SystemError, @@ -944,6 +948,7 @@ dummy_func( assert(exc && PyExceptionInstance_Check(exc)); Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } @@ -955,6 +960,7 @@ dummy_func( else { Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } } @@ -969,6 +975,7 @@ dummy_func( } else { _PyErr_SetRaisedException(tstate, Py_NewRef(exc_value)); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } } diff --git a/Python/ceval.c b/Python/ceval.c index 72d679a4e3f53..899f135bdd635 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -185,6 +185,9 @@ lltrace_resume_frame(_PyInterpreterFrame *frame) static void monitor_raise(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr); +static void monitor_reraise(PyThreadState *tstate, + _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr); static int monitor_stop_iteration(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr); @@ -923,7 +926,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int } } monitor_raise(tstate, frame, next_instr-1); - exception_unwind: { /* We can't use frame->f_lasti here, as RERAISE may have set it */ @@ -2037,6 +2039,16 @@ monitor_raise(PyThreadState *tstate, _PyInterpreterFrame *frame, do_monitor_exc(tstate, frame, instr, PY_MONITORING_EVENT_RAISE); } +static void +monitor_reraise(PyThreadState *tstate, _PyInterpreterFrame *frame, + _Py_CODEUNIT *instr) +{ + if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_RERAISE)) { + return; + } + do_monitor_exc(tstate, frame, instr, PY_MONITORING_EVENT_RERAISE); +} + static int monitor_stop_iteration(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr) diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 281c8b5e5245b..d5c048a881235 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -896,7 +896,11 @@ exc = args[0]; /* fall through */ case 0: - if (do_raise(tstate, exc, cause)) { STACK_SHRINK(oparg); goto exception_unwind; } + if (do_raise(tstate, exc, cause)) { + assert(oparg == 0); + monitor_reraise(tstate, frame, next_instr-1); + goto exception_unwind; + } break; default: _PyErr_SetString(tstate, PyExc_SystemError, @@ -904,12 +908,12 @@ break; } if (true) { STACK_SHRINK(oparg); goto error; } - #line 908 "Python/generated_cases.c.h" + #line 912 "Python/generated_cases.c.h" } TARGET(INTERPRETER_EXIT) { PyObject *retval = stack_pointer[-1]; - #line 626 "Python/bytecodes.c" + #line 630 "Python/bytecodes.c" assert(frame == &entry_frame); assert(_PyFrame_IsIncomplete(frame)); STACK_SHRINK(1); // Since we're not going to DISPATCH() @@ -920,12 +924,12 @@ assert(!_PyErr_Occurred(tstate)); _Py_LeaveRecursiveCallTstate(tstate); return retval; - #line 924 "Python/generated_cases.c.h" + #line 928 "Python/generated_cases.c.h" } TARGET(RETURN_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 639 "Python/bytecodes.c" + #line 643 "Python/bytecodes.c" STACK_SHRINK(1); assert(EMPTY()); _PyFrame_SetStackPointer(frame, stack_pointer); @@ -938,12 +942,12 @@ frame->prev_instr += frame->return_offset; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 942 "Python/generated_cases.c.h" + #line 946 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_RETURN_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 654 "Python/bytecodes.c" + #line 658 "Python/bytecodes.c" int err = _Py_call_instrumentation_arg( tstate, PY_MONITORING_EVENT_PY_RETURN, frame, next_instr-1, retval); @@ -960,11 +964,11 @@ frame->prev_instr += frame->return_offset; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 964 "Python/generated_cases.c.h" + #line 968 "Python/generated_cases.c.h" } TARGET(RETURN_CONST) { - #line 673 "Python/bytecodes.c" + #line 677 "Python/bytecodes.c" PyObject *retval = GETITEM(frame->f_code->co_consts, oparg); Py_INCREF(retval); assert(EMPTY()); @@ -978,11 +982,11 @@ frame->prev_instr += frame->return_offset; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 982 "Python/generated_cases.c.h" + #line 986 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_RETURN_CONST) { - #line 689 "Python/bytecodes.c" + #line 693 "Python/bytecodes.c" PyObject *retval = GETITEM(frame->f_code->co_consts, oparg); int err = _Py_call_instrumentation_arg( tstate, PY_MONITORING_EVENT_PY_RETURN, @@ -1000,13 +1004,13 @@ frame->prev_instr += frame->return_offset; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1004 "Python/generated_cases.c.h" + #line 1008 "Python/generated_cases.c.h" } TARGET(GET_AITER) { PyObject *obj = stack_pointer[-1]; PyObject *iter; - #line 709 "Python/bytecodes.c" + #line 713 "Python/bytecodes.c" unaryfunc getter = NULL; PyTypeObject *type = Py_TYPE(obj); @@ -1019,16 +1023,16 @@ "'async for' requires an object with " "__aiter__ method, got %.100s", type->tp_name); - #line 1023 "Python/generated_cases.c.h" + #line 1027 "Python/generated_cases.c.h" Py_DECREF(obj); - #line 722 "Python/bytecodes.c" + #line 726 "Python/bytecodes.c" if (true) goto pop_1_error; } iter = (*getter)(obj); - #line 1030 "Python/generated_cases.c.h" + #line 1034 "Python/generated_cases.c.h" Py_DECREF(obj); - #line 727 "Python/bytecodes.c" + #line 731 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; if (Py_TYPE(iter)->tp_as_async == NULL || @@ -1041,7 +1045,7 @@ Py_DECREF(iter); if (true) goto pop_1_error; } - #line 1045 "Python/generated_cases.c.h" + #line 1049 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -1049,7 +1053,7 @@ TARGET(GET_ANEXT) { PyObject *aiter = stack_pointer[-1]; PyObject *awaitable; - #line 742 "Python/bytecodes.c" + #line 746 "Python/bytecodes.c" unaryfunc getter = NULL; PyObject *next_iter = NULL; PyTypeObject *type = Py_TYPE(aiter); @@ -1093,7 +1097,7 @@ } } - #line 1097 "Python/generated_cases.c.h" + #line 1101 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = awaitable; PREDICT(LOAD_CONST); @@ -1104,16 +1108,16 @@ PREDICTED(GET_AWAITABLE); PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 789 "Python/bytecodes.c" + #line 793 "Python/bytecodes.c" iter = _PyCoro_GetAwaitableIter(iterable); if (iter == NULL) { format_awaitable_error(tstate, Py_TYPE(iterable), oparg); } - #line 1115 "Python/generated_cases.c.h" + #line 1119 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 796 "Python/bytecodes.c" + #line 800 "Python/bytecodes.c" if (iter != NULL && PyCoro_CheckExact(iter)) { PyObject *yf = _PyGen_yf((PyGenObject*)iter); @@ -1131,7 +1135,7 @@ if (iter == NULL) goto pop_1_error; - #line 1135 "Python/generated_cases.c.h" + #line 1139 "Python/generated_cases.c.h" stack_pointer[-1] = iter; PREDICT(LOAD_CONST); DISPATCH(); @@ -1143,7 +1147,7 @@ PyObject *v = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; PyObject *retval; - #line 822 "Python/bytecodes.c" + #line 826 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PySendCache *cache = (_PySendCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1190,7 +1194,7 @@ } } Py_DECREF(v); - #line 1194 "Python/generated_cases.c.h" + #line 1198 "Python/generated_cases.c.h" stack_pointer[-1] = retval; next_instr += 1; DISPATCH(); @@ -1199,7 +1203,7 @@ TARGET(SEND_GEN) { PyObject *v = stack_pointer[-1]; PyObject *receiver = stack_pointer[-2]; - #line 871 "Python/bytecodes.c" + #line 875 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, SEND); PyGenObject *gen = (PyGenObject *)receiver; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type && @@ -1215,12 +1219,12 @@ tstate->exc_info = &gen->gi_exc_state; JUMPBY(INLINE_CACHE_ENTRIES_SEND); DISPATCH_INLINED(gen_frame); - #line 1219 "Python/generated_cases.c.h" + #line 1223 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_YIELD_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 889 "Python/bytecodes.c" + #line 893 "Python/bytecodes.c" assert(frame != &entry_frame); PyGenObject *gen = _PyFrame_GetGenerator(frame); gen->gi_frame_state = FRAME_SUSPENDED; @@ -1237,12 +1241,12 @@ gen_frame->previous = NULL; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1241 "Python/generated_cases.c.h" + #line 1245 "Python/generated_cases.c.h" } TARGET(YIELD_VALUE) { PyObject *retval = stack_pointer[-1]; - #line 908 "Python/bytecodes.c" + #line 912 "Python/bytecodes.c" // NOTE: It's important that YIELD_VALUE never raises an exception! // The compiler treats any exception raised here as a failed close() // or throw() call. @@ -1258,15 +1262,15 @@ gen_frame->previous = NULL; _PyFrame_StackPush(frame, retval); goto resume_frame; - #line 1262 "Python/generated_cases.c.h" + #line 1266 "Python/generated_cases.c.h" } TARGET(POP_EXCEPT) { PyObject *exc_value = stack_pointer[-1]; - #line 926 "Python/bytecodes.c" + #line 930 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; Py_XSETREF(exc_info->exc_value, exc_value); - #line 1270 "Python/generated_cases.c.h" + #line 1274 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -1274,7 +1278,7 @@ TARGET(RERAISE) { PyObject *exc = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); - #line 931 "Python/bytecodes.c" + #line 935 "Python/bytecodes.c" assert(oparg >= 0 && oparg <= 2); if (oparg) { PyObject *lasti = values[0]; @@ -1291,27 +1295,29 @@ assert(exc && PyExceptionInstance_Check(exc)); Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; - #line 1296 "Python/generated_cases.c.h" + #line 1301 "Python/generated_cases.c.h" } TARGET(END_ASYNC_FOR) { PyObject *exc = stack_pointer[-1]; PyObject *awaitable = stack_pointer[-2]; - #line 951 "Python/bytecodes.c" + #line 956 "Python/bytecodes.c" assert(exc && PyExceptionInstance_Check(exc)); if (PyErr_GivenExceptionMatches(exc, PyExc_StopAsyncIteration)) { - #line 1305 "Python/generated_cases.c.h" + #line 1310 "Python/generated_cases.c.h" Py_DECREF(awaitable); Py_DECREF(exc); - #line 954 "Python/bytecodes.c" + #line 959 "Python/bytecodes.c" } else { Py_INCREF(exc); _PyErr_SetRaisedException(tstate, exc); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } - #line 1315 "Python/generated_cases.c.h" + #line 1321 "Python/generated_cases.c.h" STACK_SHRINK(2); DISPATCH(); } @@ -1322,23 +1328,24 @@ PyObject *sub_iter = stack_pointer[-3]; PyObject *none; PyObject *value; - #line 963 "Python/bytecodes.c" + #line 969 "Python/bytecodes.c" assert(throwflag); assert(exc_value && PyExceptionInstance_Check(exc_value)); if (PyErr_GivenExceptionMatches(exc_value, PyExc_StopIteration)) { value = Py_NewRef(((PyStopIterationObject *)exc_value)->value); - #line 1331 "Python/generated_cases.c.h" + #line 1337 "Python/generated_cases.c.h" Py_DECREF(sub_iter); Py_DECREF(last_sent_val); Py_DECREF(exc_value); - #line 968 "Python/bytecodes.c" + #line 974 "Python/bytecodes.c" none = Py_None; } else { _PyErr_SetRaisedException(tstate, Py_NewRef(exc_value)); + monitor_reraise(tstate, frame, next_instr-1); goto exception_unwind; } - #line 1342 "Python/generated_cases.c.h" + #line 1349 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = value; stack_pointer[-2] = none; @@ -1347,9 +1354,9 @@ TARGET(LOAD_ASSERTION_ERROR) { PyObject *value; - #line 977 "Python/bytecodes.c" + #line 984 "Python/bytecodes.c" value = Py_NewRef(PyExc_AssertionError); - #line 1353 "Python/generated_cases.c.h" + #line 1360 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -1357,7 +1364,7 @@ TARGET(LOAD_BUILD_CLASS) { PyObject *bc; - #line 981 "Python/bytecodes.c" + #line 988 "Python/bytecodes.c" if (PyDict_CheckExact(BUILTINS())) { bc = _PyDict_GetItemWithError(BUILTINS(), &_Py_ID(__build_class__)); @@ -1379,7 +1386,7 @@ if (true) goto error; } } - #line 1383 "Python/generated_cases.c.h" + #line 1390 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = bc; DISPATCH(); @@ -1387,33 +1394,33 @@ TARGET(STORE_NAME) { PyObject *v = stack_pointer[-1]; - #line 1006 "Python/bytecodes.c" + #line 1013 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); PyObject *ns = LOCALS(); int err; if (ns == NULL) { _PyErr_Format(tstate, PyExc_SystemError, "no locals found when storing %R", name); - #line 1398 "Python/generated_cases.c.h" + #line 1405 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1013 "Python/bytecodes.c" + #line 1020 "Python/bytecodes.c" if (true) goto pop_1_error; } if (PyDict_CheckExact(ns)) err = PyDict_SetItem(ns, name, v); else err = PyObject_SetItem(ns, name, v); - #line 1407 "Python/generated_cases.c.h" + #line 1414 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1020 "Python/bytecodes.c" + #line 1027 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1411 "Python/generated_cases.c.h" + #line 1418 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(DELETE_NAME) { - #line 1024 "Python/bytecodes.c" + #line 1031 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); PyObject *ns = LOCALS(); int err; @@ -1430,7 +1437,7 @@ name); goto error; } - #line 1434 "Python/generated_cases.c.h" + #line 1441 "Python/generated_cases.c.h" DISPATCH(); } @@ -1438,7 +1445,7 @@ PREDICTED(UNPACK_SEQUENCE); static_assert(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE == 1, "incorrect cache size"); PyObject *seq = stack_pointer[-1]; - #line 1050 "Python/bytecodes.c" + #line 1057 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1451,11 +1458,11 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject **top = stack_pointer + oparg - 1; int res = unpack_iterable(tstate, seq, oparg, -1, top); - #line 1455 "Python/generated_cases.c.h" + #line 1462 "Python/generated_cases.c.h" Py_DECREF(seq); - #line 1063 "Python/bytecodes.c" + #line 1070 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 1459 "Python/generated_cases.c.h" + #line 1466 "Python/generated_cases.c.h" STACK_SHRINK(1); STACK_GROW(oparg); next_instr += 1; @@ -1465,14 +1472,14 @@ TARGET(UNPACK_SEQUENCE_TWO_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1067 "Python/bytecodes.c" + #line 1074 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != 2, UNPACK_SEQUENCE); assert(oparg == 2); STAT_INC(UNPACK_SEQUENCE, hit); values[0] = Py_NewRef(PyTuple_GET_ITEM(seq, 1)); values[1] = Py_NewRef(PyTuple_GET_ITEM(seq, 0)); - #line 1476 "Python/generated_cases.c.h" + #line 1483 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1483,7 +1490,7 @@ TARGET(UNPACK_SEQUENCE_TUPLE) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1077 "Python/bytecodes.c" + #line 1084 "Python/bytecodes.c" DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyTuple_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -1491,7 +1498,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 1495 "Python/generated_cases.c.h" + #line 1502 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1502,7 +1509,7 @@ TARGET(UNPACK_SEQUENCE_LIST) { PyObject *seq = stack_pointer[-1]; PyObject **values = stack_pointer - (1); - #line 1088 "Python/bytecodes.c" + #line 1095 "Python/bytecodes.c" DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE); DEOPT_IF(PyList_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); @@ -1510,7 +1517,7 @@ for (int i = oparg; --i >= 0; ) { *values++ = Py_NewRef(items[i]); } - #line 1514 "Python/generated_cases.c.h" + #line 1521 "Python/generated_cases.c.h" Py_DECREF(seq); STACK_SHRINK(1); STACK_GROW(oparg); @@ -1520,15 +1527,15 @@ TARGET(UNPACK_EX) { PyObject *seq = stack_pointer[-1]; - #line 1099 "Python/bytecodes.c" + #line 1106 "Python/bytecodes.c" int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject **top = stack_pointer + totalargs - 1; int res = unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top); - #line 1528 "Python/generated_cases.c.h" + #line 1535 "Python/generated_cases.c.h" Py_DECREF(seq); - #line 1103 "Python/bytecodes.c" + #line 1110 "Python/bytecodes.c" if (res == 0) goto pop_1_error; - #line 1532 "Python/generated_cases.c.h" + #line 1539 "Python/generated_cases.c.h" STACK_GROW((oparg & 0xFF) + (oparg >> 8)); DISPATCH(); } @@ -1539,7 +1546,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *v = stack_pointer[-2]; uint16_t counter = read_u16(&next_instr[0].cache); - #line 1114 "Python/bytecodes.c" + #line 1121 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION if (ADAPTIVE_COUNTER_IS_ZERO(counter)) { PyObject *name = GETITEM(frame->f_code->co_names, oparg); @@ -1555,12 +1562,12 @@ #endif /* ENABLE_SPECIALIZATION */ PyObject *name = GETITEM(frame->f_code->co_names, oparg); int err = PyObject_SetAttr(owner, name, v); - #line 1559 "Python/generated_cases.c.h" + #line 1566 "Python/generated_cases.c.h" Py_DECREF(v); Py_DECREF(owner); - #line 1130 "Python/bytecodes.c" + #line 1137 "Python/bytecodes.c" if (err) goto pop_2_error; - #line 1564 "Python/generated_cases.c.h" + #line 1571 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -1568,34 +1575,34 @@ TARGET(DELETE_ATTR) { PyObject *owner = stack_pointer[-1]; - #line 1134 "Python/bytecodes.c" + #line 1141 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); int err = PyObject_SetAttr(owner, name, (PyObject *)NULL); - #line 1575 "Python/generated_cases.c.h" + #line 1582 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1137 "Python/bytecodes.c" + #line 1144 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1579 "Python/generated_cases.c.h" + #line 1586 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(STORE_GLOBAL) { PyObject *v = stack_pointer[-1]; - #line 1141 "Python/bytecodes.c" + #line 1148 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); int err = PyDict_SetItem(GLOBALS(), name, v); - #line 1589 "Python/generated_cases.c.h" + #line 1596 "Python/generated_cases.c.h" Py_DECREF(v); - #line 1144 "Python/bytecodes.c" + #line 1151 "Python/bytecodes.c" if (err) goto pop_1_error; - #line 1593 "Python/generated_cases.c.h" + #line 1600 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(DELETE_GLOBAL) { - #line 1148 "Python/bytecodes.c" + #line 1155 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); int err; err = PyDict_DelItem(GLOBALS(), name); @@ -1607,7 +1614,7 @@ } goto error; } - #line 1611 "Python/generated_cases.c.h" + #line 1618 "Python/generated_cases.c.h" DISPATCH(); } @@ -1615,7 +1622,7 @@ PyObject *_tmp_1; { PyObject *locals; - #line 1162 "Python/bytecodes.c" + #line 1169 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1623,7 +1630,7 @@ if (true) goto error; } Py_INCREF(locals); - #line 1627 "Python/generated_cases.c.h" + #line 1634 "Python/generated_cases.c.h" _tmp_1 = locals; } STACK_GROW(1); @@ -1635,7 +1642,7 @@ PyObject *_tmp_1; { PyObject *locals; - #line 1162 "Python/bytecodes.c" + #line 1169 "Python/bytecodes.c" locals = LOCALS(); if (locals == NULL) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -1643,13 +1650,13 @@ if (true) goto error; } Py_INCREF(locals); - #line 1647 "Python/generated_cases.c.h" + #line 1654 "Python/generated_cases.c.h" _tmp_1 = locals; } { PyObject *mod_or_class_dict = _tmp_1; PyObject *v; - #line 1174 "Python/bytecodes.c" + #line 1181 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); if (PyDict_CheckExact(mod_or_class_dict)) { v = PyDict_GetItemWithError(mod_or_class_dict, name); @@ -1706,7 +1713,7 @@ } } } - #line 1710 "Python/generated_cases.c.h" + #line 1717 "Python/generated_cases.c.h" _tmp_1 = v; } STACK_GROW(1); @@ -1719,7 +1726,7 @@ { PyObject *mod_or_class_dict = _tmp_1; PyObject *v; - #line 1174 "Python/bytecodes.c" + #line 1181 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); if (PyDict_CheckExact(mod_or_class_dict)) { v = PyDict_GetItemWithError(mod_or_class_dict, name); @@ -1776,7 +1783,7 @@ } } } - #line 1780 "Python/generated_cases.c.h" + #line 1787 "Python/generated_cases.c.h" _tmp_1 = v; } stack_pointer[-1] = _tmp_1; @@ -1788,7 +1795,7 @@ static_assert(INLINE_CACHE_ENTRIES_LOAD_GLOBAL == 4, "incorrect cache size"); PyObject *null = NULL; PyObject *v; - #line 1243 "Python/bytecodes.c" + #line 1250 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -1840,7 +1847,7 @@ } } null = NULL; - #line 1844 "Python/generated_cases.c.h" + #line 1851 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = v; @@ -1854,7 +1861,7 @@ PyObject *res; uint16_t index = read_u16(&next_instr[1].cache); uint16_t version = read_u16(&next_instr[2].cache); - #line 1297 "Python/bytecodes.c" + #line 1304 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); PyDictObject *dict = (PyDictObject *)GLOBALS(); DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); @@ -1865,7 +1872,7 @@ Py_INCREF(res); STAT_INC(LOAD_GLOBAL, hit); null = NULL; - #line 1869 "Python/generated_cases.c.h" + #line 1876 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -1880,7 +1887,7 @@ uint16_t index = read_u16(&next_instr[1].cache); uint16_t mod_version = read_u16(&next_instr[2].cache); uint16_t bltn_version = read_u16(&next_instr[3].cache); - #line 1310 "Python/bytecodes.c" + #line 1317 "Python/bytecodes.c" DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); DEOPT_IF(!PyDict_CheckExact(BUILTINS()), LOAD_GLOBAL); PyDictObject *mdict = (PyDictObject *)GLOBALS(); @@ -1895,7 +1902,7 @@ Py_INCREF(res); STAT_INC(LOAD_GLOBAL, hit); null = NULL; - #line 1899 "Python/generated_cases.c.h" + #line 1906 "Python/generated_cases.c.h" STACK_GROW(1); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -1905,16 +1912,16 @@ } TARGET(DELETE_FAST) { - #line 1327 "Python/bytecodes.c" + #line 1334 "Python/bytecodes.c" PyObject *v = GETLOCAL(oparg); if (v == NULL) goto unbound_local_error; SETLOCAL(oparg, NULL); - #line 1913 "Python/generated_cases.c.h" + #line 1920 "Python/generated_cases.c.h" DISPATCH(); } TARGET(MAKE_CELL) { - #line 1333 "Python/bytecodes.c" + #line 1340 "Python/bytecodes.c" // "initial" is probably NULL but not if it's an arg (or set // via PyFrame_LocalsToFast() before MAKE_CELL has run). PyObject *initial = GETLOCAL(oparg); @@ -1923,12 +1930,12 @@ goto resume_with_error; } SETLOCAL(oparg, cell); - #line 1927 "Python/generated_cases.c.h" + #line 1934 "Python/generated_cases.c.h" DISPATCH(); } TARGET(DELETE_DEREF) { - #line 1344 "Python/bytecodes.c" + #line 1351 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); // Can't use ERROR_IF here. @@ -1939,14 +1946,14 @@ } PyCell_SET(cell, NULL); Py_DECREF(oldobj); - #line 1943 "Python/generated_cases.c.h" + #line 1950 "Python/generated_cases.c.h" DISPATCH(); } TARGET(LOAD_FROM_DICT_OR_DEREF) { PyObject *class_dict = stack_pointer[-1]; PyObject *value; - #line 1357 "Python/bytecodes.c" + #line 1364 "Python/bytecodes.c" PyObject *name; assert(class_dict); assert(oparg >= 0 && oparg < frame->f_code->co_nlocalsplus); @@ -1981,14 +1988,14 @@ } Py_INCREF(value); } - #line 1985 "Python/generated_cases.c.h" + #line 1992 "Python/generated_cases.c.h" stack_pointer[-1] = value; DISPATCH(); } TARGET(LOAD_DEREF) { PyObject *value; - #line 1394 "Python/bytecodes.c" + #line 1401 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); value = PyCell_GET(cell); if (value == NULL) { @@ -1996,7 +2003,7 @@ if (true) goto error; } Py_INCREF(value); - #line 2000 "Python/generated_cases.c.h" + #line 2007 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = value; DISPATCH(); @@ -2004,18 +2011,18 @@ TARGET(STORE_DEREF) { PyObject *v = stack_pointer[-1]; - #line 1404 "Python/bytecodes.c" + #line 1411 "Python/bytecodes.c" PyObject *cell = GETLOCAL(oparg); PyObject *oldobj = PyCell_GET(cell); PyCell_SET(cell, v); Py_XDECREF(oldobj); - #line 2013 "Python/generated_cases.c.h" + #line 2020 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(COPY_FREE_VARS) { - #line 1411 "Python/bytecodes.c" + #line 1418 "Python/bytecodes.c" /* Copy closure variables to free variables */ PyCodeObject *co = frame->f_code; assert(PyFunction_Check(frame->f_funcobj)); @@ -2026,22 +2033,22 @@ PyObject *o = PyTuple_GET_ITEM(closure, i); frame->localsplus[offset + i] = Py_NewRef(o); } - #line 2030 "Python/generated_cases.c.h" + #line 2037 "Python/generated_cases.c.h" DISPATCH(); } TARGET(BUILD_STRING) { PyObject **pieces = (stack_pointer - oparg); PyObject *str; - #line 1424 "Python/bytecodes.c" + #line 1431 "Python/bytecodes.c" str = _PyUnicode_JoinArray(&_Py_STR(empty), pieces, oparg); - #line 2039 "Python/generated_cases.c.h" + #line 2046 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(pieces[_i]); } - #line 1426 "Python/bytecodes.c" + #line 1433 "Python/bytecodes.c" if (str == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2045 "Python/generated_cases.c.h" + #line 2052 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = str; @@ -2051,10 +2058,10 @@ TARGET(BUILD_TUPLE) { PyObject **values = (stack_pointer - oparg); PyObject *tup; - #line 1430 "Python/bytecodes.c" + #line 1437 "Python/bytecodes.c" tup = _PyTuple_FromArraySteal(values, oparg); if (tup == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2058 "Python/generated_cases.c.h" + #line 2065 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = tup; @@ -2064,10 +2071,10 @@ TARGET(BUILD_LIST) { PyObject **values = (stack_pointer - oparg); PyObject *list; - #line 1435 "Python/bytecodes.c" + #line 1442 "Python/bytecodes.c" list = _PyList_FromArraySteal(values, oparg); if (list == NULL) { STACK_SHRINK(oparg); goto error; } - #line 2071 "Python/generated_cases.c.h" + #line 2078 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = list; @@ -2077,7 +2084,7 @@ TARGET(LIST_EXTEND) { PyObject *iterable = stack_pointer[-1]; PyObject *list = stack_pointer[-(2 + (oparg-1))]; - #line 1440 "Python/bytecodes.c" + #line 1447 "Python/bytecodes.c" PyObject *none_val = _PyList_Extend((PyListObject *)list, iterable); if (none_val == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError) && @@ -2088,13 +2095,13 @@ "Value after * must be an iterable, not %.200s", Py_TYPE(iterable)->tp_name); } - #line 2092 "Python/generated_cases.c.h" + #line 2099 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1451 "Python/bytecodes.c" + #line 1458 "Python/bytecodes.c" if (true) goto pop_1_error; } assert(Py_IsNone(none_val)); - #line 2098 "Python/generated_cases.c.h" + #line 2105 "Python/generated_cases.c.h" Py_DECREF(iterable); STACK_SHRINK(1); DISPATCH(); @@ -2103,13 +2110,13 @@ TARGET(SET_UPDATE) { PyObject *iterable = stack_pointer[-1]; PyObject *set = stack_pointer[-(2 + (oparg-1))]; - #line 1458 "Python/bytecodes.c" + #line 1465 "Python/bytecodes.c" int err = _PySet_Update(set, iterable); - #line 2109 "Python/generated_cases.c.h" + #line 2116 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 1460 "Python/bytecodes.c" + #line 1467 "Python/bytecodes.c" if (err < 0) goto pop_1_error; - #line 2113 "Python/generated_cases.c.h" + #line 2120 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } @@ -2117,7 +2124,7 @@ TARGET(BUILD_SET) { PyObject **values = (stack_pointer - oparg); PyObject *set; - #line 1464 "Python/bytecodes.c" + #line 1471 "Python/bytecodes.c" set = PySet_New(NULL); if (set == NULL) goto error; @@ -2132,7 +2139,7 @@ Py_DECREF(set); if (true) { STACK_SHRINK(oparg); goto error; } } - #line 2136 "Python/generated_cases.c.h" + #line 2143 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_GROW(1); stack_pointer[-1] = set; @@ -2142,7 +2149,7 @@ TARGET(BUILD_MAP) { PyObject **values = (stack_pointer - oparg*2); PyObject *map; - #line 1481 "Python/bytecodes.c" + #line 1488 "Python/bytecodes.c" map = _PyDict_FromItems( values, 2, values+1, 2, @@ -2150,13 +2157,13 @@ if (map == NULL) goto error; - #line 2154 "Python/generated_cases.c.h" + #line 2161 "Python/generated_cases.c.h" for (int _i = oparg*2; --_i >= 0;) { Py_DECREF(values[_i]); } - #line 1489 "Python/bytecodes.c" + #line 1496 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg*2); goto error; } - #line 2160 "Python/generated_cases.c.h" + #line 2167 "Python/generated_cases.c.h" STACK_SHRINK(oparg*2); STACK_GROW(1); stack_pointer[-1] = map; @@ -2164,7 +2171,7 @@ } TARGET(SETUP_ANNOTATIONS) { - #line 1493 "Python/bytecodes.c" + #line 1500 "Python/bytecodes.c" int err; PyObject *ann_dict; if (LOCALS() == NULL) { @@ -2204,7 +2211,7 @@ Py_DECREF(ann_dict); } } - #line 2208 "Python/generated_cases.c.h" + #line 2215 "Python/generated_cases.c.h" DISPATCH(); } @@ -2212,7 +2219,7 @@ PyObject *keys = stack_pointer[-1]; PyObject **values = (stack_pointer - (1 + oparg)); PyObject *map; - #line 1535 "Python/bytecodes.c" + #line 1542 "Python/bytecodes.c" if (!PyTuple_CheckExact(keys) || PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) { _PyErr_SetString(tstate, PyExc_SystemError, @@ -2222,14 +2229,14 @@ map = _PyDict_FromItems( &PyTuple_GET_ITEM(keys, 0), 1, values, 1, oparg); - #line 2226 "Python/generated_cases.c.h" + #line 2233 "Python/generated_cases.c.h" for (int _i = oparg; --_i >= 0;) { Py_DECREF(values[_i]); } Py_DECREF(keys); - #line 1545 "Python/bytecodes.c" + #line 1552 "Python/bytecodes.c" if (map == NULL) { STACK_SHRINK(oparg); goto pop_1_error; } - #line 2233 "Python/generated_cases.c.h" + #line 2240 "Python/generated_cases.c.h" STACK_SHRINK(oparg); stack_pointer[-1] = map; DISPATCH(); @@ -2237,7 +2244,7 @@ TARGET(DICT_UPDATE) { PyObject *update = stack_pointer[-1]; - #line 1549 "Python/bytecodes.c" + #line 1556 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (PyDict_Update(dict, update) < 0) { if (_PyErr_ExceptionMatches(tstate, PyExc_AttributeError)) { @@ -2245,12 +2252,12 @@ "'%.200s' object is not a mapping", Py_TYPE(update)->tp_name); } - #line 2249 "Python/generated_cases.c.h" + #line 2256 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1557 "Python/bytecodes.c" + #line 1564 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 2254 "Python/generated_cases.c.h" + #line 2261 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); DISPATCH(); @@ -2258,17 +2265,17 @@ TARGET(DICT_MERGE) { PyObject *update = stack_pointer[-1]; - #line 1563 "Python/bytecodes.c" + #line 1570 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 1); // update is still on the stack if (_PyDict_MergeEx(dict, update, 2) < 0) { format_kwargs_error(tstate, PEEK(3 + oparg), update); - #line 2267 "Python/generated_cases.c.h" + #line 2274 "Python/generated_cases.c.h" Py_DECREF(update); - #line 1568 "Python/bytecodes.c" + #line 1575 "Python/bytecodes.c" if (true) goto pop_1_error; } - #line 2272 "Python/generated_cases.c.h" + #line 2279 "Python/generated_cases.c.h" Py_DECREF(update); STACK_SHRINK(1); PREDICT(CALL_FUNCTION_EX); @@ -2278,26 +2285,26 @@ TARGET(MAP_ADD) { PyObject *value = stack_pointer[-1]; PyObject *key = stack_pointer[-2]; - #line 1575 "Python/bytecodes.c" + #line 1582 "Python/bytecodes.c" PyObject *dict = PEEK(oparg + 2); // key, value are still on the stack assert(PyDict_CheckExact(dict)); /* dict[key] = value */ // Do not DECREF INPUTS because the function steals the references if (_PyDict_SetItem_Take2((PyDictObject *)dict, key, value) != 0) goto pop_2_error; - #line 2288 "Python/generated_cases.c.h" + #line 2295 "Python/generated_cases.c.h" STACK_SHRINK(2); PREDICT(JUMP_BACKWARD); DISPATCH(); } TARGET(INSTRUMENTED_LOAD_SUPER_ATTR) { - #line 1584 "Python/bytecodes.c" + #line 1591 "Python/bytecodes.c" _PySuperAttrCache *cache = (_PySuperAttrCache *)next_instr; // cancel out the decrement that will happen in LOAD_SUPER_ATTR; we // don't want to specialize instrumented instructions INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(LOAD_SUPER_ATTR); - #line 2301 "Python/generated_cases.c.h" + #line 2308 "Python/generated_cases.c.h" } TARGET(LOAD_SUPER_ATTR) { @@ -2308,7 +2315,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1598 "Python/bytecodes.c" + #line 1605 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg >> 2); int load_method = oparg & 1; #if ENABLE_SPECIALIZATION @@ -2350,16 +2357,16 @@ } } } - #line 2354 "Python/generated_cases.c.h" + #line 2361 "Python/generated_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1640 "Python/bytecodes.c" + #line 1647 "Python/bytecodes.c" if (super == NULL) goto pop_3_error; res = PyObject_GetAttr(super, name); Py_DECREF(super); if (res == NULL) goto pop_3_error; - #line 2363 "Python/generated_cases.c.h" + #line 2370 "Python/generated_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2374,20 +2381,20 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2 = NULL; PyObject *res; - #line 1647 "Python/bytecodes.c" + #line 1654 "Python/bytecodes.c" assert(!(oparg & 1)); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); STAT_INC(LOAD_SUPER_ATTR, hit); PyObject *name = GETITEM(frame->f_code->co_names, oparg >> 2); res = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL); - #line 2385 "Python/generated_cases.c.h" + #line 2392 "Python/generated_cases.c.h" Py_DECREF(global_super); Py_DECREF(class); Py_DECREF(self); - #line 1654 "Python/bytecodes.c" + #line 1661 "Python/bytecodes.c" if (res == NULL) goto pop_3_error; - #line 2391 "Python/generated_cases.c.h" + #line 2398 "Python/generated_cases.c.h" STACK_SHRINK(2); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2402,7 +2409,7 @@ PyObject *global_super = stack_pointer[-3]; PyObject *res2; PyObject *res; - #line 1658 "Python/bytecodes.c" + #line 1665 "Python/bytecodes.c" assert(oparg & 1); DEOPT_IF(global_super != (PyObject *)&PySuper_Type, LOAD_SUPER_ATTR); DEOPT_IF(!PyType_Check(class), LOAD_SUPER_ATTR); @@ -2425,7 +2432,7 @@ res = res2; res2 = NULL; } - #line 2429 "Python/generated_cases.c.h" + #line 2436 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; stack_pointer[-2] = res2; @@ -2439,7 +2446,7 @@ PyObject *owner = stack_pointer[-1]; PyObject *res2 = NULL; PyObject *res; - #line 1697 "Python/bytecodes.c" + #line 1704 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyAttrCache *cache = (_PyAttrCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2473,9 +2480,9 @@ NULL | meth | arg1 | ... | argN */ - #line 2477 "Python/generated_cases.c.h" + #line 2484 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1731 "Python/bytecodes.c" + #line 1738 "Python/bytecodes.c" if (meth == NULL) goto pop_1_error; res2 = NULL; res = meth; @@ -2484,12 +2491,12 @@ else { /* Classic, pushes one value. */ res = PyObject_GetAttr(owner, name); - #line 2488 "Python/generated_cases.c.h" + #line 2495 "Python/generated_cases.c.h" Py_DECREF(owner); - #line 1740 "Python/bytecodes.c" + #line 1747 "Python/bytecodes.c" if (res == NULL) goto pop_1_error; } - #line 2493 "Python/generated_cases.c.h" + #line 2500 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -2503,7 +2510,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1745 "Python/bytecodes.c" + #line 1752 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2516,7 +2523,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2520 "Python/generated_cases.c.h" + #line 2527 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2531,7 +2538,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1761 "Python/bytecodes.c" + #line 1768 "Python/bytecodes.c" DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; assert(dict != NULL); @@ -2544,7 +2551,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2548 "Python/generated_cases.c.h" + #line 2555 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2559,7 +2566,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1777 "Python/bytecodes.c" + #line 1784 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2586,7 +2593,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2590 "Python/generated_cases.c.h" + #line 2597 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2601,7 +2608,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1807 "Python/bytecodes.c" + #line 1814 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, LOAD_ATTR); @@ -2611,7 +2618,7 @@ STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); res2 = NULL; - #line 2615 "Python/generated_cases.c.h" + #line 2622 "Python/generated_cases.c.h" Py_DECREF(owner); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2626,7 +2633,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 1820 "Python/bytecodes.c" + #line 1827 "Python/bytecodes.c" DEOPT_IF(!PyType_Check(cls), LOAD_ATTR); DEOPT_IF(((PyTypeObject *)cls)->tp_version_tag != type_version, @@ -2638,7 +2645,7 @@ res = descr; assert(res != NULL); Py_INCREF(res); - #line 2642 "Python/generated_cases.c.h" + #line 2649 "Python/generated_cases.c.h" Py_DECREF(cls); STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; @@ -2652,7 +2659,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *fget = read_obj(&next_instr[5].cache); - #line 1835 "Python/bytecodes.c" + #line 1842 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); @@ -2676,7 +2683,7 @@ JUMPBY(INLINE_CACHE_ENTRIES_LOAD_ATTR); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 2680 "Python/generated_cases.c.h" + #line 2687 "Python/generated_cases.c.h" } TARGET(LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN) { @@ -2684,7 +2691,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t func_version = read_u32(&next_instr[3].cache); PyObject *getattribute = read_obj(&next_instr[5].cache); - #line 1861 "Python/bytecodes.c" + #line 1868 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, LOAD_ATTR); PyTypeObject *cls = Py_TYPE(owner); DEOPT_IF(cls->tp_version_tag != type_version, LOAD_ATTR); @@ -2710,7 +2717,7 @@ JUMPBY(INLINE_CACHE_ENTRIES_LOAD_ATTR); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 2714 "Python/generated_cases.c.h" + #line 2721 "Python/generated_cases.c.h" } TARGET(STORE_ATTR_INSTANCE_VALUE) { @@ -2718,7 +2725,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1889 "Python/bytecodes.c" + #line 1896 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2736,7 +2743,7 @@ Py_DECREF(old_value); } Py_DECREF(owner); - #line 2740 "Python/generated_cases.c.h" + #line 2747 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2747,7 +2754,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t hint = read_u16(&next_instr[3].cache); - #line 1909 "Python/bytecodes.c" + #line 1916 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2786,7 +2793,7 @@ /* PEP 509 */ dict->ma_version_tag = new_version; Py_DECREF(owner); - #line 2790 "Python/generated_cases.c.h" + #line 2797 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2797,7 +2804,7 @@ PyObject *value = stack_pointer[-2]; uint32_t type_version = read_u32(&next_instr[1].cache); uint16_t index = read_u16(&next_instr[3].cache); - #line 1950 "Python/bytecodes.c" + #line 1957 "Python/bytecodes.c" PyTypeObject *tp = Py_TYPE(owner); assert(type_version != 0); DEOPT_IF(tp->tp_version_tag != type_version, STORE_ATTR); @@ -2807,7 +2814,7 @@ *(PyObject **)addr = value; Py_XDECREF(old_value); Py_DECREF(owner); - #line 2811 "Python/generated_cases.c.h" + #line 2818 "Python/generated_cases.c.h" STACK_SHRINK(2); next_instr += 4; DISPATCH(); @@ -2819,7 +2826,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 1969 "Python/bytecodes.c" + #line 1976 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyCompareOpCache *cache = (_PyCompareOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -2832,12 +2839,12 @@ #endif /* ENABLE_SPECIALIZATION */ assert((oparg >> 4) <= Py_GE); res = PyObject_RichCompare(left, right, oparg>>4); - #line 2836 "Python/generated_cases.c.h" + #line 2843 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 1982 "Python/bytecodes.c" + #line 1989 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 2841 "Python/generated_cases.c.h" + #line 2848 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2848,7 +2855,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 1986 "Python/bytecodes.c" + #line 1993 "Python/bytecodes.c" DEOPT_IF(!PyFloat_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyFloat_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2859,7 +2866,7 @@ _Py_DECREF_SPECIALIZED(left, _PyFloat_ExactDealloc); _Py_DECREF_SPECIALIZED(right, _PyFloat_ExactDealloc); res = (sign_ish & oparg) ? Py_True : Py_False; - #line 2863 "Python/generated_cases.c.h" + #line 2870 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2870,7 +2877,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2000 "Python/bytecodes.c" + #line 2007 "Python/bytecodes.c" DEOPT_IF(!PyLong_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyLong_CheckExact(right), COMPARE_OP); DEOPT_IF(!_PyLong_IsCompact((PyLongObject *)left), COMPARE_OP); @@ -2885,7 +2892,7 @@ _Py_DECREF_SPECIALIZED(left, (destructor)PyObject_Free); _Py_DECREF_SPECIALIZED(right, (destructor)PyObject_Free); res = (sign_ish & oparg) ? Py_True : Py_False; - #line 2889 "Python/generated_cases.c.h" + #line 2896 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2896,7 +2903,7 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *res; - #line 2018 "Python/bytecodes.c" + #line 2025 "Python/bytecodes.c" DEOPT_IF(!PyUnicode_CheckExact(left), COMPARE_OP); DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); @@ -2908,7 +2915,7 @@ assert((oparg & 0xf) == COMPARISON_NOT_EQUALS || (oparg & 0xf) == COMPARISON_EQUALS); assert(COMPARISON_NOT_EQUALS + 1 == COMPARISON_EQUALS); res = ((COMPARISON_NOT_EQUALS + eq) & oparg) ? Py_True : Py_False; - #line 2912 "Python/generated_cases.c.h" + #line 2919 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -2919,14 +2926,14 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2032 "Python/bytecodes.c" + #line 2039 "Python/bytecodes.c" int res = Py_Is(left, right) ^ oparg; - #line 2925 "Python/generated_cases.c.h" + #line 2932 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2034 "Python/bytecodes.c" + #line 2041 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 2930 "Python/generated_cases.c.h" + #line 2937 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -2936,15 +2943,15 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2038 "Python/bytecodes.c" + #line 2045 "Python/bytecodes.c" int res = PySequence_Contains(right, left); - #line 2942 "Python/generated_cases.c.h" + #line 2949 "Python/generated_cases.c.h" Py_DECREF(left); Py_DECREF(right); - #line 2040 "Python/bytecodes.c" + #line 2047 "Python/bytecodes.c" if (res < 0) goto pop_2_error; b = (res ^ oparg) ? Py_True : Py_False; - #line 2948 "Python/generated_cases.c.h" + #line 2955 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = b; DISPATCH(); @@ -2955,12 +2962,12 @@ PyObject *exc_value = stack_pointer[-2]; PyObject *rest; PyObject *match; - #line 2045 "Python/bytecodes.c" + #line 2052 "Python/bytecodes.c" if (check_except_star_type_valid(tstate, match_type) < 0) { - #line 2961 "Python/generated_cases.c.h" + #line 2968 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2047 "Python/bytecodes.c" + #line 2054 "Python/bytecodes.c" if (true) goto pop_2_error; } @@ -2968,10 +2975,10 @@ rest = NULL; int res = exception_group_match(exc_value, match_type, &match, &rest); - #line 2972 "Python/generated_cases.c.h" + #line 2979 "Python/generated_cases.c.h" Py_DECREF(exc_value); Py_DECREF(match_type); - #line 2055 "Python/bytecodes.c" + #line 2062 "Python/bytecodes.c" if (res < 0) goto pop_2_error; assert((match == NULL) == (rest == NULL)); @@ -2980,7 +2987,7 @@ if (!Py_IsNone(match)) { PyErr_SetHandledException(match); } - #line 2984 "Python/generated_cases.c.h" + #line 2991 "Python/generated_cases.c.h" stack_pointer[-1] = match; stack_pointer[-2] = rest; DISPATCH(); @@ -2990,21 +2997,21 @@ PyObject *right = stack_pointer[-1]; PyObject *left = stack_pointer[-2]; PyObject *b; - #line 2066 "Python/bytecodes.c" + #line 2073 "Python/bytecodes.c" assert(PyExceptionInstance_Check(left)); if (check_except_type_valid(tstate, right) < 0) { - #line 2997 "Python/generated_cases.c.h" + #line 3004 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2069 "Python/bytecodes.c" + #line 2076 "Python/bytecodes.c" if (true) goto pop_1_error; } int res = PyErr_GivenExceptionMatches(left, right); - #line 3004 "Python/generated_cases.c.h" + #line 3011 "Python/generated_cases.c.h" Py_DECREF(right); - #line 2074 "Python/bytecodes.c" + #line 2081 "Python/bytecodes.c" b = res ? Py_True : Py_False; - #line 3008 "Python/generated_cases.c.h" + #line 3015 "Python/generated_cases.c.h" stack_pointer[-1] = b; DISPATCH(); } @@ -3013,15 +3020,15 @@ PyObject *fromlist = stack_pointer[-1]; PyObject *level = stack_pointer[-2]; PyObject *res; - #line 2078 "Python/bytecodes.c" + #line 2085 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); res = import_name(tstate, frame, name, fromlist, level); - #line 3020 "Python/generated_cases.c.h" + #line 3027 "Python/generated_cases.c.h" Py_DECREF(level); Py_DECREF(fromlist); - #line 2081 "Python/bytecodes.c" + #line 2088 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 3025 "Python/generated_cases.c.h" + #line 3032 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; DISPATCH(); @@ -3030,29 +3037,29 @@ TARGET(IMPORT_FROM) { PyObject *from = stack_pointer[-1]; PyObject *res; - #line 2085 "Python/bytecodes.c" + #line 2092 "Python/bytecodes.c" PyObject *name = GETITEM(frame->f_code->co_names, oparg); res = import_from(tstate, from, name); if (res == NULL) goto error; - #line 3038 "Python/generated_cases.c.h" + #line 3045 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); } TARGET(JUMP_FORWARD) { - #line 2091 "Python/bytecodes.c" + #line 2098 "Python/bytecodes.c" JUMPBY(oparg); - #line 3047 "Python/generated_cases.c.h" + #line 3054 "Python/generated_cases.c.h" DISPATCH(); } TARGET(JUMP_BACKWARD) { PREDICTED(JUMP_BACKWARD); - #line 2095 "Python/bytecodes.c" + #line 2102 "Python/bytecodes.c" assert(oparg < INSTR_OFFSET()); JUMPBY(-oparg); - #line 3056 "Python/generated_cases.c.h" + #line 3063 "Python/generated_cases.c.h" CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -3060,15 +3067,15 @@ TARGET(POP_JUMP_IF_FALSE) { PREDICTED(POP_JUMP_IF_FALSE); PyObject *cond = stack_pointer[-1]; - #line 2101 "Python/bytecodes.c" + #line 2108 "Python/bytecodes.c" if (Py_IsFalse(cond)) { JUMPBY(oparg); } else if (!Py_IsTrue(cond)) { int err = PyObject_IsTrue(cond); - #line 3070 "Python/generated_cases.c.h" + #line 3077 "Python/generated_cases.c.h" Py_DECREF(cond); - #line 2107 "Python/bytecodes.c" + #line 2114 "Python/bytecodes.c" if (err == 0) { JUMPBY(oparg); } @@ -3076,22 +3083,22 @@ if (err < 0) goto pop_1_error; } } - #line 3080 "Python/generated_cases.c.h" + #line 3087 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_TRUE) { PyObject *cond = stack_pointer[-1]; - #line 2117 "Python/bytecodes.c" + #line 2124 "Python/bytecodes.c" if (Py_IsTrue(cond)) { JUMPBY(oparg); } else if (!Py_IsFalse(cond)) { int err = PyObject_IsTrue(cond); - #line 3093 "Python/generated_cases.c.h" + #line 3100 "Python/generated_cases.c.h" Py_DECREF(cond); - #line 2123 "Python/bytecodes.c" + #line 2130 "Python/bytecodes.c" if (err > 0) { JUMPBY(oparg); } @@ -3099,63 +3106,63 @@ if (err < 0) goto pop_1_error; } } - #line 3103 "Python/generated_cases.c.h" + #line 3110 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_NOT_NONE) { PyObject *value = stack_pointer[-1]; - #line 2133 "Python/bytecodes.c" + #line 2140 "Python/bytecodes.c" if (!Py_IsNone(value)) { - #line 3112 "Python/generated_cases.c.h" + #line 3119 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2135 "Python/bytecodes.c" + #line 2142 "Python/bytecodes.c" JUMPBY(oparg); } - #line 3117 "Python/generated_cases.c.h" + #line 3124 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(POP_JUMP_IF_NONE) { PyObject *value = stack_pointer[-1]; - #line 2140 "Python/bytecodes.c" + #line 2147 "Python/bytecodes.c" if (Py_IsNone(value)) { JUMPBY(oparg); } else { - #line 3129 "Python/generated_cases.c.h" + #line 3136 "Python/generated_cases.c.h" Py_DECREF(value); - #line 2145 "Python/bytecodes.c" + #line 2152 "Python/bytecodes.c" } - #line 3133 "Python/generated_cases.c.h" + #line 3140 "Python/generated_cases.c.h" STACK_SHRINK(1); DISPATCH(); } TARGET(JUMP_BACKWARD_NO_INTERRUPT) { - #line 2149 "Python/bytecodes.c" + #line 2156 "Python/bytecodes.c" /* This bytecode is used in the `yield from` or `await` loop. * If there is an interrupt, we want it handled in the innermost * generator or coroutine, so we deliberately do not check it here. * (see bpo-30039). */ JUMPBY(-oparg); - #line 3146 "Python/generated_cases.c.h" + #line 3153 "Python/generated_cases.c.h" DISPATCH(); } TARGET(GET_LEN) { PyObject *obj = stack_pointer[-1]; PyObject *len_o; - #line 2158 "Python/bytecodes.c" + #line 2165 "Python/bytecodes.c" // PUSH(len(TOS)) Py_ssize_t len_i = PyObject_Length(obj); if (len_i < 0) goto error; len_o = PyLong_FromSsize_t(len_i); if (len_o == NULL) goto error; - #line 3159 "Python/generated_cases.c.h" + #line 3166 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = len_o; DISPATCH(); @@ -3166,16 +3173,16 @@ PyObject *type = stack_pointer[-2]; PyObject *subject = stack_pointer[-3]; PyObject *attrs; - #line 2166 "Python/bytecodes.c" + #line 2173 "Python/bytecodes.c" // Pop TOS and TOS1. Set TOS to a tuple of attributes on success, or // None on failure. assert(PyTuple_CheckExact(names)); attrs = match_class(tstate, subject, type, oparg, names); - #line 3175 "Python/generated_cases.c.h" + #line 3182 "Python/generated_cases.c.h" Py_DECREF(subject); Py_DECREF(type); Py_DECREF(names); - #line 2171 "Python/bytecodes.c" + #line 2178 "Python/bytecodes.c" if (attrs) { assert(PyTuple_CheckExact(attrs)); // Success! } @@ -3183,7 +3190,7 @@ if (_PyErr_Occurred(tstate)) goto pop_3_error; attrs = Py_None; // Failure! } - #line 3187 "Python/generated_cases.c.h" + #line 3194 "Python/generated_cases.c.h" STACK_SHRINK(2); stack_pointer[-1] = attrs; DISPATCH(); @@ -3192,10 +3199,10 @@ TARGET(MATCH_MAPPING) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2181 "Python/bytecodes.c" + #line 2188 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_MAPPING; res = match ? Py_True : Py_False; - #line 3199 "Python/generated_cases.c.h" + #line 3206 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; PREDICT(POP_JUMP_IF_FALSE); @@ -3205,10 +3212,10 @@ TARGET(MATCH_SEQUENCE) { PyObject *subject = stack_pointer[-1]; PyObject *res; - #line 2187 "Python/bytecodes.c" + #line 2194 "Python/bytecodes.c" int match = Py_TYPE(subject)->tp_flags & Py_TPFLAGS_SEQUENCE; res = match ? Py_True : Py_False; - #line 3212 "Python/generated_cases.c.h" + #line 3219 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; PREDICT(POP_JUMP_IF_FALSE); @@ -3219,11 +3226,11 @@ PyObject *keys = stack_pointer[-1]; PyObject *subject = stack_pointer[-2]; PyObject *values_or_none; - #line 2193 "Python/bytecodes.c" + #line 2200 "Python/bytecodes.c" // On successful match, PUSH(values). Otherwise, PUSH(None). values_or_none = match_keys(tstate, subject, keys); if (values_or_none == NULL) goto error; - #line 3227 "Python/generated_cases.c.h" + #line 3234 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = values_or_none; DISPATCH(); @@ -3232,14 +3239,14 @@ TARGET(GET_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2199 "Python/bytecodes.c" + #line 2206 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ iter = PyObject_GetIter(iterable); - #line 3239 "Python/generated_cases.c.h" + #line 3246 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2202 "Python/bytecodes.c" + #line 2209 "Python/bytecodes.c" if (iter == NULL) goto pop_1_error; - #line 3243 "Python/generated_cases.c.h" + #line 3250 "Python/generated_cases.c.h" stack_pointer[-1] = iter; DISPATCH(); } @@ -3247,7 +3254,7 @@ TARGET(GET_YIELD_FROM_ITER) { PyObject *iterable = stack_pointer[-1]; PyObject *iter; - #line 2206 "Python/bytecodes.c" + #line 2213 "Python/bytecodes.c" /* before: [obj]; after [getiter(obj)] */ if (PyCoro_CheckExact(iterable)) { /* `iterable` is a coroutine */ @@ -3270,11 +3277,11 @@ if (iter == NULL) { goto error; } - #line 3274 "Python/generated_cases.c.h" + #line 3281 "Python/generated_cases.c.h" Py_DECREF(iterable); - #line 2229 "Python/bytecodes.c" + #line 2236 "Python/bytecodes.c" } - #line 3278 "Python/generated_cases.c.h" + #line 3285 "Python/generated_cases.c.h" stack_pointer[-1] = iter; PREDICT(LOAD_CONST); DISPATCH(); @@ -3285,7 +3292,7 @@ static_assert(INLINE_CACHE_ENTRIES_FOR_ITER == 1, "incorrect cache size"); PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2248 "Python/bytecodes.c" + #line 2255 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyForIterCache *cache = (_PyForIterCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -3316,7 +3323,7 @@ DISPATCH(); } // Common case: no jump, leave it to the code generator - #line 3320 "Python/generated_cases.c.h" + #line 3327 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3324,7 +3331,7 @@ } TARGET(INSTRUMENTED_FOR_ITER) { - #line 2281 "Python/bytecodes.c" + #line 2288 "Python/bytecodes.c" _Py_CODEUNIT *here = next_instr-1; _Py_CODEUNIT *target; PyObject *iter = TOP(); @@ -3350,14 +3357,14 @@ target = next_instr + INLINE_CACHE_ENTRIES_FOR_ITER + oparg + 1; } INSTRUMENTED_JUMP(here, target, PY_MONITORING_EVENT_BRANCH); - #line 3354 "Python/generated_cases.c.h" + #line 3361 "Python/generated_cases.c.h" DISPATCH(); } TARGET(FOR_ITER_LIST) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2309 "Python/bytecodes.c" + #line 2316 "Python/bytecodes.c" DEOPT_IF(Py_TYPE(iter) != &PyListIter_Type, FOR_ITER); _PyListIterObject *it = (_PyListIterObject *)iter; STAT_INC(FOR_ITER, hit); @@ -3377,7 +3384,7 @@ DISPATCH(); end_for_iter_list: // Common case: no jump, leave it to the code generator - #line 3381 "Python/generated_cases.c.h" + #line 3388 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3387,7 +3394,7 @@ TARGET(FOR_ITER_TUPLE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2331 "Python/bytecodes.c" + #line 2338 "Python/bytecodes.c" _PyTupleIterObject *it = (_PyTupleIterObject *)iter; DEOPT_IF(Py_TYPE(it) != &PyTupleIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3407,7 +3414,7 @@ DISPATCH(); end_for_iter_tuple: // Common case: no jump, leave it to the code generator - #line 3411 "Python/generated_cases.c.h" + #line 3418 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3417,7 +3424,7 @@ TARGET(FOR_ITER_RANGE) { PyObject *iter = stack_pointer[-1]; PyObject *next; - #line 2353 "Python/bytecodes.c" + #line 2360 "Python/bytecodes.c" _PyRangeIterObject *r = (_PyRangeIterObject *)iter; DEOPT_IF(Py_TYPE(r) != &PyRangeIter_Type, FOR_ITER); STAT_INC(FOR_ITER, hit); @@ -3435,7 +3442,7 @@ if (next == NULL) { goto error; } - #line 3439 "Python/generated_cases.c.h" + #line 3446 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = next; next_instr += 1; @@ -3444,7 +3451,7 @@ TARGET(FOR_ITER_GEN) { PyObject *iter = stack_pointer[-1]; - #line 2373 "Python/bytecodes.c" + #line 2380 "Python/bytecodes.c" DEOPT_IF(tstate->interp->eval_frame, FOR_ITER); PyGenObject *gen = (PyGenObject *)iter; DEOPT_IF(Py_TYPE(gen) != &PyGen_Type, FOR_ITER); @@ -3460,14 +3467,14 @@ assert(next_instr[oparg].op.code == END_FOR || next_instr[oparg].op.code == INSTRUMENTED_END_FOR); DISPATCH_INLINED(gen_frame); - #line 3464 "Python/generated_cases.c.h" + #line 3471 "Python/generated_cases.c.h" } TARGET(BEFORE_ASYNC_WITH) { PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2391 "Python/bytecodes.c" + #line 2398 "Python/bytecodes.c" PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { @@ -3490,16 +3497,16 @@ Py_DECREF(enter); goto error; } - #line 3494 "Python/generated_cases.c.h" + #line 3501 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2414 "Python/bytecodes.c" + #line 2421 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3503 "Python/generated_cases.c.h" + #line 3510 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3511,7 +3518,7 @@ PyObject *mgr = stack_pointer[-1]; PyObject *exit; PyObject *res; - #line 2424 "Python/bytecodes.c" + #line 2431 "Python/bytecodes.c" /* pop the context manager, push its __exit__ and the * value returned from calling its __enter__ */ @@ -3537,16 +3544,16 @@ Py_DECREF(enter); goto error; } - #line 3541 "Python/generated_cases.c.h" + #line 3548 "Python/generated_cases.c.h" Py_DECREF(mgr); - #line 2450 "Python/bytecodes.c" + #line 2457 "Python/bytecodes.c" res = _PyObject_CallNoArgs(enter); Py_DECREF(enter); if (res == NULL) { Py_DECREF(exit); if (true) goto pop_1_error; } - #line 3550 "Python/generated_cases.c.h" + #line 3557 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; stack_pointer[-2] = exit; @@ -3558,7 +3565,7 @@ PyObject *lasti = stack_pointer[-3]; PyObject *exit_func = stack_pointer[-4]; PyObject *res; - #line 2459 "Python/bytecodes.c" + #line 2466 "Python/bytecodes.c" /* At the top of the stack are 4 values: - val: TOP = exc_info() - unused: SECOND = previous exception @@ -3579,7 +3586,7 @@ res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 3583 "Python/generated_cases.c.h" + #line 3590 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3588,7 +3595,7 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2482 "Python/bytecodes.c" + #line 2489 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -3598,7 +3605,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 3602 "Python/generated_cases.c.h" + #line 3609 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -3612,7 +3619,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2494 "Python/bytecodes.c" + #line 2501 "Python/bytecodes.c" /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3629,7 +3636,7 @@ assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; assert(oparg & 1); - #line 3633 "Python/generated_cases.c.h" + #line 3640 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3643,7 +3650,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2513 "Python/bytecodes.c" + #line 2520 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); assert(self_cls->tp_dictoffset == 0); @@ -3653,7 +3660,7 @@ res2 = Py_NewRef(descr); res = self; assert(oparg & 1); - #line 3657 "Python/generated_cases.c.h" + #line 3664 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3667,7 +3674,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2525 "Python/bytecodes.c" + #line 2532 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); Py_ssize_t dictoffset = self_cls->tp_dictoffset; @@ -3681,7 +3688,7 @@ res2 = Py_NewRef(descr); res = self; assert(oparg & 1); - #line 3685 "Python/generated_cases.c.h" + #line 3692 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3690,16 +3697,16 @@ } TARGET(KW_NAMES) { - #line 2541 "Python/bytecodes.c" + #line 2548 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(frame->f_code->co_consts)); kwnames = GETITEM(frame->f_code->co_consts, oparg); - #line 3698 "Python/generated_cases.c.h" + #line 3705 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_CALL) { - #line 2547 "Python/bytecodes.c" + #line 2554 "Python/bytecodes.c" int is_meth = PEEK(oparg+2) != NULL; int total_args = oparg + is_meth; PyObject *function = PEEK(total_args + 1); @@ -3712,7 +3719,7 @@ _PyCallCache *cache = (_PyCallCache *)next_instr; INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(CALL); - #line 3716 "Python/generated_cases.c.h" + #line 3723 "Python/generated_cases.c.h" } TARGET(CALL) { @@ -3722,7 +3729,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2592 "Python/bytecodes.c" + #line 2599 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3804,7 +3811,7 @@ Py_DECREF(args[i]); } if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3808 "Python/generated_cases.c.h" + #line 3815 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3816,7 +3823,7 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2680 "Python/bytecodes.c" + #line 2687 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit); @@ -3826,7 +3833,7 @@ PEEK(oparg + 2) = Py_NewRef(meth); // method Py_DECREF(callable); GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS); - #line 3830 "Python/generated_cases.c.h" + #line 3837 "Python/generated_cases.c.h" } TARGET(CALL_PY_EXACT_ARGS) { @@ -3835,7 +3842,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2692 "Python/bytecodes.c" + #line 2699 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -3861,7 +3868,7 @@ JUMPBY(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 3865 "Python/generated_cases.c.h" + #line 3872 "Python/generated_cases.c.h" } TARGET(CALL_PY_WITH_DEFAULTS) { @@ -3869,7 +3876,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2720 "Python/bytecodes.c" + #line 2727 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -3905,7 +3912,7 @@ JUMPBY(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 3909 "Python/generated_cases.c.h" + #line 3916 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_TYPE_1) { @@ -3913,7 +3920,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2758 "Python/bytecodes.c" + #line 2765 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -3923,7 +3930,7 @@ res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(obj); Py_DECREF(&PyType_Type); // I.e., callable - #line 3927 "Python/generated_cases.c.h" + #line 3934 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3936,7 +3943,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2770 "Python/bytecodes.c" + #line 2777 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -3947,7 +3954,7 @@ Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); // I.e., callable if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3951 "Python/generated_cases.c.h" + #line 3958 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3961,7 +3968,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2784 "Python/bytecodes.c" + #line 2791 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -3972,7 +3979,7 @@ Py_DECREF(arg); Py_DECREF(&PyTuple_Type); // I.e., tuple if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3976 "Python/generated_cases.c.h" + #line 3983 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3986,7 +3993,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2798 "Python/bytecodes.c" + #line 2805 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4008,7 +4015,7 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4012 "Python/generated_cases.c.h" + #line 4019 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4022,7 +4029,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2823 "Python/bytecodes.c" + #line 2830 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4050,7 +4057,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4054 "Python/generated_cases.c.h" + #line 4061 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4064,7 +4071,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2854 "Python/bytecodes.c" + #line 2861 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4096,7 +4103,7 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 4100 "Python/generated_cases.c.h" + #line 4107 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4110,7 +4117,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2889 "Python/bytecodes.c" + #line 2896 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -4142,7 +4149,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4146 "Python/generated_cases.c.h" + #line 4153 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4156,7 +4163,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2924 "Python/bytecodes.c" + #line 2931 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -4181,7 +4188,7 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4185 "Python/generated_cases.c.h" + #line 4192 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4194,7 +4201,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2951 "Python/bytecodes.c" + #line 2958 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -4221,7 +4228,7 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4225 "Python/generated_cases.c.h" + #line 4232 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4233,7 +4240,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2981 "Python/bytecodes.c" + #line 2988 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -4251,14 +4258,14 @@ JUMPBY(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 4255 "Python/generated_cases.c.h" + #line 4262 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3001 "Python/bytecodes.c" + #line 3008 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4289,7 +4296,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4293 "Python/generated_cases.c.h" + #line 4300 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4302,7 +4309,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3035 "Python/bytecodes.c" + #line 3042 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4331,7 +4338,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4335 "Python/generated_cases.c.h" + #line 4342 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4344,7 +4351,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3067 "Python/bytecodes.c" + #line 3074 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -4373,7 +4380,7 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4377 "Python/generated_cases.c.h" + #line 4384 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4386,7 +4393,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3099 "Python/bytecodes.c" + #line 3106 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4414,7 +4421,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4418 "Python/generated_cases.c.h" + #line 4425 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4424,9 +4431,9 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { - #line 3130 "Python/bytecodes.c" + #line 3137 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); - #line 4430 "Python/generated_cases.c.h" + #line 4437 "Python/generated_cases.c.h" } TARGET(CALL_FUNCTION_EX) { @@ -4435,7 +4442,7 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 3134 "Python/bytecodes.c" + #line 3141 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs)); @@ -4497,14 +4504,14 @@ } result = PyObject_Call(func, callargs, kwargs); } - #line 4501 "Python/generated_cases.c.h" + #line 4508 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 3196 "Python/bytecodes.c" + #line 3203 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4508 "Python/generated_cases.c.h" + #line 4515 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4519,7 +4526,7 @@ PyObject *kwdefaults = (oparg & 0x02) ? stack_pointer[-(1 + ((oparg & 0x08) ? 1 : 0) + ((oparg & 0x04) ? 1 : 0) + ((oparg & 0x02) ? 1 : 0))] : NULL; PyObject *defaults = (oparg & 0x01) ? stack_pointer[-(1 + ((oparg & 0x08) ? 1 : 0) + ((oparg & 0x04) ? 1 : 0) + ((oparg & 0x02) ? 1 : 0) + ((oparg & 0x01) ? 1 : 0))] : NULL; PyObject *func; - #line 3206 "Python/bytecodes.c" + #line 3213 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4548,14 +4555,14 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4552 "Python/generated_cases.c.h" + #line 4559 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 0x01) ? 1 : 0) + ((oparg & 0x02) ? 1 : 0) + ((oparg & 0x04) ? 1 : 0) + ((oparg & 0x08) ? 1 : 0)); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 3237 "Python/bytecodes.c" + #line 3244 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4576,7 +4583,7 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4580 "Python/generated_cases.c.h" + #line 4587 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4584,15 +4591,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3260 "Python/bytecodes.c" + #line 3267 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4590 "Python/generated_cases.c.h" + #line 4597 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3262 "Python/bytecodes.c" + #line 3269 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4596 "Python/generated_cases.c.h" + #line 4603 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4603,7 +4610,7 @@ PyObject *fmt_spec = ((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? stack_pointer[-((((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0))] : NULL; PyObject *value = stack_pointer[-(1 + (((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0))]; PyObject *result; - #line 3266 "Python/bytecodes.c" + #line 3273 "Python/bytecodes.c" /* Handles f-string value formatting. */ PyObject *(*conv_fn)(PyObject *); int which_conversion = oparg & FVC_MASK; @@ -4638,7 +4645,7 @@ Py_DECREF(value); Py_XDECREF(fmt_spec); if (result == NULL) { STACK_SHRINK((((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0)); goto pop_1_error; } - #line 4642 "Python/generated_cases.c.h" + #line 4649 "Python/generated_cases.c.h" STACK_SHRINK((((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0)); stack_pointer[-1] = result; DISPATCH(); @@ -4647,10 +4654,10 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3303 "Python/bytecodes.c" + #line 3310 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4654 "Python/generated_cases.c.h" + #line 4661 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4662,7 +4669,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3308 "Python/bytecodes.c" + #line 3315 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -4677,12 +4684,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4681 "Python/generated_cases.c.h" + #line 4688 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3323 "Python/bytecodes.c" + #line 3330 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 4686 "Python/generated_cases.c.h" + #line 4693 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -4692,16 +4699,16 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3328 "Python/bytecodes.c" + #line 3335 "Python/bytecodes.c" assert(oparg >= 2); - #line 4698 "Python/generated_cases.c.h" + #line 4705 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } TARGET(INSTRUMENTED_INSTRUCTION) { - #line 3332 "Python/bytecodes.c" + #line 3339 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error; @@ -4713,26 +4720,26 @@ assert(next_opcode > 0 && next_opcode < 256); opcode = next_opcode; DISPATCH_GOTO(); - #line 4717 "Python/generated_cases.c.h" + #line 4724 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_JUMP_FORWARD) { - #line 3346 "Python/bytecodes.c" + #line 3353 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); - #line 4723 "Python/generated_cases.c.h" + #line 4730 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) { - #line 3350 "Python/bytecodes.c" + #line 3357 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr-oparg, PY_MONITORING_EVENT_JUMP); - #line 4730 "Python/generated_cases.c.h" + #line 4737 "Python/generated_cases.c.h" CHECK_EVAL_BREAKER(); DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { - #line 3355 "Python/bytecodes.c" + #line 3362 "Python/bytecodes.c" PyObject *cond = POP(); int err = PyObject_IsTrue(cond); Py_DECREF(cond); @@ -4741,12 +4748,12 @@ assert(err == 0 || err == 1); int offset = err*oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4745 "Python/generated_cases.c.h" + #line 4752 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { - #line 3366 "Python/bytecodes.c" + #line 3373 "Python/bytecodes.c" PyObject *cond = POP(); int err = PyObject_IsTrue(cond); Py_DECREF(cond); @@ -4755,12 +4762,12 @@ assert(err == 0 || err == 1); int offset = (1-err)*oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4759 "Python/generated_cases.c.h" + #line 4766 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { - #line 3377 "Python/bytecodes.c" + #line 3384 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -4772,12 +4779,12 @@ offset = 0; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4776 "Python/generated_cases.c.h" + #line 4783 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { - #line 3391 "Python/bytecodes.c" + #line 3398 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -4789,30 +4796,30 @@ offset = oparg; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4793 "Python/generated_cases.c.h" + #line 4800 "Python/generated_cases.c.h" DISPATCH(); } TARGET(EXTENDED_ARG) { - #line 3405 "Python/bytecodes.c" + #line 3412 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 4804 "Python/generated_cases.c.h" + #line 4811 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3413 "Python/bytecodes.c" + #line 3420 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); - #line 4811 "Python/generated_cases.c.h" + #line 4818 "Python/generated_cases.c.h" } TARGET(RESERVED) { - #line 3418 "Python/bytecodes.c" + #line 3425 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 4818 "Python/generated_cases.c.h" + #line 4825 "Python/generated_cases.c.h" } diff --git a/Python/instrumentation.c b/Python/instrumentation.c index 2fd3344b555a9..37933ffbd89b5 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -2037,6 +2037,7 @@ static const char *const event_names [] = { [PY_MONITORING_EVENT_C_RETURN] = "C_RETURN", [PY_MONITORING_EVENT_PY_THROW] = "PY_THROW", [PY_MONITORING_EVENT_RAISE] = "RAISE", + [PY_MONITORING_EVENT_RERAISE] = "RERAISE", [PY_MONITORING_EVENT_EXCEPTION_HANDLED] = "EXCEPTION_HANDLED", [PY_MONITORING_EVENT_C_RAISE] = "C_RAISE", [PY_MONITORING_EVENT_PY_UNWIND] = "PY_UNWIND", From webhook-mailer at python.org Fri Jul 28 05:53:37 2023 From: webhook-mailer at python.org (markshannon) Date: Fri, 28 Jul 2023 09:53:37 -0000 Subject: [Python-checkins] [3.12] GH-106895: Raise a `ValueError` when attempting to disable events that cannot be disabled. (GH-107337) (GH-107351) Message-ID: https://github.com/python/cpython/commit/0902afbae29ef88bf9d212a7e11f9f17b6cbdeb5 commit: 0902afbae29ef88bf9d212a7e11f9f17b6cbdeb5 branch: 3.12 author: Mark Shannon committer: markshannon date: 2023-07-28T10:53:33+01:00 summary: [3.12] GH-106895: Raise a `ValueError` when attempting to disable events that cannot be disabled. (GH-107337) (GH-107351) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-26-18-53-34.gh-issue-106895.DdEwV8.rst M Include/internal/pycore_instruments.h M Lib/test/test_monitoring.py M Objects/classobject.c M Python/bytecodes.c M Python/ceval.c M Python/generated_cases.c.h M Python/instrumentation.c diff --git a/Include/internal/pycore_instruments.h b/Include/internal/pycore_instruments.h index cfa5d09a4704f..ccccd54a2f70a 100644 --- a/Include/internal/pycore_instruments.h +++ b/Include/internal/pycore_instruments.h @@ -28,7 +28,8 @@ extern "C" { #define PY_MONITORING_EVENT_BRANCH 8 #define PY_MONITORING_EVENT_STOP_ITERATION 9 -#define PY_MONITORING_INSTRUMENTED_EVENTS 10 +#define PY_MONITORING_IS_INSTRUMENTED_EVENT(ev) \ + ((ev) <= PY_MONITORING_EVENT_STOP_ITERATION) /* Other events, mainly exceptions */ diff --git a/Lib/test/test_monitoring.py b/Lib/test/test_monitoring.py index 8bf95a73d7db9..9685a43810136 100644 --- a/Lib/test/test_monitoring.py +++ b/Lib/test/test_monitoring.py @@ -136,20 +136,27 @@ def test_c_return_count(self): E = sys.monitoring.events -SIMPLE_EVENTS = [ +INSTRUMENTED_EVENTS = [ (E.PY_START, "start"), (E.PY_RESUME, "resume"), (E.PY_RETURN, "return"), (E.PY_YIELD, "yield"), (E.JUMP, "jump"), (E.BRANCH, "branch"), +] + +EXCEPT_EVENTS = [ (E.RAISE, "raise"), (E.PY_UNWIND, "unwind"), (E.EXCEPTION_HANDLED, "exception_handled"), +] + +SIMPLE_EVENTS = INSTRUMENTED_EVENTS + EXCEPT_EVENTS + [ (E.C_RAISE, "c_raise"), (E.C_RETURN, "c_return"), ] + SIMPLE_EVENT_SET = functools.reduce(operator.or_, [ev for (ev, _) in SIMPLE_EVENTS], 0) | E.CALL @@ -618,6 +625,49 @@ def func2(): self.check_lines(func2, [1,2,3,4,5,6]) +class TestDisable(MonitoringTestBase, unittest.TestCase): + + def gen(self, cond): + for i in range(10): + if cond: + yield 1 + else: + yield 2 + + def raise_handle_reraise(self): + try: + 1/0 + except: + raise + + def test_disable_legal_events(self): + for event, name in INSTRUMENTED_EVENTS: + try: + counter = CounterWithDisable() + counter.disable = True + sys.monitoring.register_callback(TEST_TOOL, event, counter) + sys.monitoring.set_events(TEST_TOOL, event) + for _ in self.gen(1): + pass + self.assertLess(counter.count, 4) + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.register_callback(TEST_TOOL, event, None) + + + def test_disable_illegal_events(self): + for event, name in EXCEPT_EVENTS: + try: + counter = CounterWithDisable() + counter.disable = True + sys.monitoring.register_callback(TEST_TOOL, event, counter) + sys.monitoring.set_events(TEST_TOOL, event) + with self.assertRaises(ValueError): + self.raise_handle_reraise() + finally: + sys.monitoring.set_events(TEST_TOOL, 0) + sys.monitoring.register_callback(TEST_TOOL, event, None) + class ExceptionRecorder: diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-26-18-53-34.gh-issue-106895.DdEwV8.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-18-53-34.gh-issue-106895.DdEwV8.rst new file mode 100644 index 0000000000000..370a29d34c860 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-18-53-34.gh-issue-106895.DdEwV8.rst @@ -0,0 +1,2 @@ +Raise a ``ValueError`` when a monitoring callback funtion returns +``DISABLE`` for events that cannot be disabled locally. diff --git a/Objects/classobject.c b/Objects/classobject.c index 71c4a4e5d0f8a..12dc276f28976 100644 --- a/Objects/classobject.c +++ b/Objects/classobject.c @@ -48,6 +48,7 @@ method_vectorcall(PyObject *method, PyObject *const *args, PyObject *self = PyMethod_GET_SELF(method); PyObject *func = PyMethod_GET_FUNCTION(method); Py_ssize_t nargs = PyVectorcall_NARGS(nargsf); + assert(nargs == 0 || args[nargs-1]); PyObject *result; if (nargsf & PY_VECTORCALL_ARGUMENTS_OFFSET) { @@ -56,6 +57,7 @@ method_vectorcall(PyObject *method, PyObject *const *args, nargs += 1; PyObject *tmp = newargs[0]; newargs[0] = self; + assert(newargs[nargs-1]); result = _PyObject_VectorcallTstate(tstate, func, newargs, nargs, kwnames); newargs[0] = tmp; diff --git a/Python/bytecodes.c b/Python/bytecodes.c index adfa5ce00b6a0..7ee48c70b0005 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -2476,7 +2476,12 @@ dummy_func( assert(val && PyExceptionInstance_Check(val)); exc = PyExceptionInstance_Class(val); tb = PyException_GetTraceback(val); - Py_XDECREF(tb); + if (tb == NULL) { + tb = Py_None; + } + else { + Py_DECREF(tb); + } assert(PyLong_Check(lasti)); (void)lasti; // Shut up compiler warning if asserts are off PyObject *stack[4] = {NULL, exc, val, tb}; diff --git a/Python/ceval.c b/Python/ceval.c index 899f135bdd635..27dea27e81200 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -194,7 +194,7 @@ static int monitor_stop_iteration(PyThreadState *tstate, static void monitor_unwind(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr); -static void monitor_handled(PyThreadState *tstate, +static int monitor_handled(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *exc); static void monitor_throw(PyThreadState *tstate, @@ -969,7 +969,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int PyObject *exc = _PyErr_GetRaisedException(tstate); PUSH(exc); JUMPTO(handler); - monitor_handled(tstate, frame, next_instr, exc); + if (monitor_handled(tstate, frame, next_instr, exc) < 0) { + goto exception_unwind; + } /* Resume normal execution */ DISPATCH(); } @@ -2007,6 +2009,7 @@ do_monitor_exc(PyThreadState *tstate, _PyInterpreterFrame *frame, PyErr_SetRaisedException(exc); } else { + assert(PyErr_Occurred()); Py_DECREF(exc); } return err; @@ -2071,15 +2074,15 @@ monitor_unwind(PyThreadState *tstate, } -static void +static int monitor_handled(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr, PyObject *exc) { if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_EXCEPTION_HANDLED)) { - return; + return 0; } - _Py_call_instrumentation_arg(tstate, PY_MONITORING_EVENT_EXCEPTION_HANDLED, frame, instr, exc); + return _Py_call_instrumentation_arg(tstate, PY_MONITORING_EVENT_EXCEPTION_HANDLED, frame, instr, exc); } static void diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index d5c048a881235..eabb9fe5fac4d 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3579,14 +3579,19 @@ assert(val && PyExceptionInstance_Check(val)); exc = PyExceptionInstance_Class(val); tb = PyException_GetTraceback(val); - Py_XDECREF(tb); + if (tb == NULL) { + tb = Py_None; + } + else { + Py_DECREF(tb); + } assert(PyLong_Check(lasti)); (void)lasti; // Shut up compiler warning if asserts are off PyObject *stack[4] = {NULL, exc, val, tb}; res = PyObject_Vectorcall(exit_func, stack + 1, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); if (res == NULL) goto error; - #line 3590 "Python/generated_cases.c.h" + #line 3595 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = res; DISPATCH(); @@ -3595,7 +3600,7 @@ TARGET(PUSH_EXC_INFO) { PyObject *new_exc = stack_pointer[-1]; PyObject *prev_exc; - #line 2489 "Python/bytecodes.c" + #line 2494 "Python/bytecodes.c" _PyErr_StackItem *exc_info = tstate->exc_info; if (exc_info->exc_value != NULL) { prev_exc = exc_info->exc_value; @@ -3605,7 +3610,7 @@ } assert(PyExceptionInstance_Check(new_exc)); exc_info->exc_value = Py_NewRef(new_exc); - #line 3609 "Python/generated_cases.c.h" + #line 3614 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = new_exc; stack_pointer[-2] = prev_exc; @@ -3619,7 +3624,7 @@ uint32_t type_version = read_u32(&next_instr[1].cache); uint32_t keys_version = read_u32(&next_instr[3].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2501 "Python/bytecodes.c" + #line 2506 "Python/bytecodes.c" /* Cached method object */ PyTypeObject *self_cls = Py_TYPE(self); assert(type_version != 0); @@ -3636,7 +3641,7 @@ assert(_PyType_HasFeature(Py_TYPE(res2), Py_TPFLAGS_METHOD_DESCRIPTOR)); res = self; assert(oparg & 1); - #line 3640 "Python/generated_cases.c.h" + #line 3645 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3650,7 +3655,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2520 "Python/bytecodes.c" + #line 2525 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); assert(self_cls->tp_dictoffset == 0); @@ -3660,7 +3665,7 @@ res2 = Py_NewRef(descr); res = self; assert(oparg & 1); - #line 3664 "Python/generated_cases.c.h" + #line 3669 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3674,7 +3679,7 @@ PyObject *res; uint32_t type_version = read_u32(&next_instr[1].cache); PyObject *descr = read_obj(&next_instr[5].cache); - #line 2532 "Python/bytecodes.c" + #line 2537 "Python/bytecodes.c" PyTypeObject *self_cls = Py_TYPE(self); DEOPT_IF(self_cls->tp_version_tag != type_version, LOAD_ATTR); Py_ssize_t dictoffset = self_cls->tp_dictoffset; @@ -3688,7 +3693,7 @@ res2 = Py_NewRef(descr); res = self; assert(oparg & 1); - #line 3692 "Python/generated_cases.c.h" + #line 3697 "Python/generated_cases.c.h" STACK_GROW(((oparg & 1) ? 1 : 0)); stack_pointer[-1] = res; if (oparg & 1) { stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))] = res2; } @@ -3697,16 +3702,16 @@ } TARGET(KW_NAMES) { - #line 2548 "Python/bytecodes.c" + #line 2553 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(frame->f_code->co_consts)); kwnames = GETITEM(frame->f_code->co_consts, oparg); - #line 3705 "Python/generated_cases.c.h" + #line 3710 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_CALL) { - #line 2554 "Python/bytecodes.c" + #line 2559 "Python/bytecodes.c" int is_meth = PEEK(oparg+2) != NULL; int total_args = oparg + is_meth; PyObject *function = PEEK(total_args + 1); @@ -3719,7 +3724,7 @@ _PyCallCache *cache = (_PyCallCache *)next_instr; INCREMENT_ADAPTIVE_COUNTER(cache->counter); GO_TO_INSTRUCTION(CALL); - #line 3723 "Python/generated_cases.c.h" + #line 3728 "Python/generated_cases.c.h" } TARGET(CALL) { @@ -3729,7 +3734,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2599 "Python/bytecodes.c" + #line 2604 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -3811,7 +3816,7 @@ Py_DECREF(args[i]); } if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3815 "Python/generated_cases.c.h" + #line 3820 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3823,7 +3828,7 @@ TARGET(CALL_BOUND_METHOD_EXACT_ARGS) { PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2687 "Python/bytecodes.c" + #line 2692 "Python/bytecodes.c" DEOPT_IF(method != NULL, CALL); DEOPT_IF(Py_TYPE(callable) != &PyMethod_Type, CALL); STAT_INC(CALL, hit); @@ -3833,7 +3838,7 @@ PEEK(oparg + 2) = Py_NewRef(meth); // method Py_DECREF(callable); GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS); - #line 3837 "Python/generated_cases.c.h" + #line 3842 "Python/generated_cases.c.h" } TARGET(CALL_PY_EXACT_ARGS) { @@ -3842,7 +3847,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2699 "Python/bytecodes.c" + #line 2704 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -3868,7 +3873,7 @@ JUMPBY(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 3872 "Python/generated_cases.c.h" + #line 3877 "Python/generated_cases.c.h" } TARGET(CALL_PY_WITH_DEFAULTS) { @@ -3876,7 +3881,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; uint32_t func_version = read_u32(&next_instr[1].cache); - #line 2727 "Python/bytecodes.c" + #line 2732 "Python/bytecodes.c" assert(kwnames == NULL); DEOPT_IF(tstate->interp->eval_frame, CALL); int is_meth = method != NULL; @@ -3912,7 +3917,7 @@ JUMPBY(INLINE_CACHE_ENTRIES_CALL); frame->return_offset = 0; DISPATCH_INLINED(new_frame); - #line 3916 "Python/generated_cases.c.h" + #line 3921 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_TYPE_1) { @@ -3920,7 +3925,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2765 "Python/bytecodes.c" + #line 2770 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -3930,7 +3935,7 @@ res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(obj); Py_DECREF(&PyType_Type); // I.e., callable - #line 3934 "Python/generated_cases.c.h" + #line 3939 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3943,7 +3948,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2777 "Python/bytecodes.c" + #line 2782 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -3954,7 +3959,7 @@ Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); // I.e., callable if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3958 "Python/generated_cases.c.h" + #line 3963 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3968,7 +3973,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *null = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2791 "Python/bytecodes.c" + #line 2796 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); DEOPT_IF(null != NULL, CALL); @@ -3979,7 +3984,7 @@ Py_DECREF(arg); Py_DECREF(&PyTuple_Type); // I.e., tuple if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 3983 "Python/generated_cases.c.h" + #line 3988 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -3993,7 +3998,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2805 "Python/bytecodes.c" + #line 2810 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4015,7 +4020,7 @@ } Py_DECREF(tp); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4019 "Python/generated_cases.c.h" + #line 4024 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4029,7 +4034,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2830 "Python/bytecodes.c" + #line 2835 "Python/bytecodes.c" /* Builtin METH_O functions */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4057,7 +4062,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4061 "Python/generated_cases.c.h" + #line 4066 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4071,7 +4076,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2861 "Python/bytecodes.c" + #line 2866 "Python/bytecodes.c" /* Builtin METH_FASTCALL functions, without keywords */ assert(kwnames == NULL); int is_meth = method != NULL; @@ -4103,7 +4108,7 @@ 'invalid'). In those cases an exception is set, so we must handle it. */ - #line 4107 "Python/generated_cases.c.h" + #line 4112 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4117,7 +4122,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2896 "Python/bytecodes.c" + #line 2901 "Python/bytecodes.c" /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ int is_meth = method != NULL; int total_args = oparg; @@ -4149,7 +4154,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4153 "Python/generated_cases.c.h" + #line 4158 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4163,7 +4168,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2931 "Python/bytecodes.c" + #line 2936 "Python/bytecodes.c" assert(kwnames == NULL); /* len(o) */ int is_meth = method != NULL; @@ -4188,7 +4193,7 @@ Py_DECREF(callable); Py_DECREF(arg); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4192 "Python/generated_cases.c.h" + #line 4197 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4201,7 +4206,7 @@ PyObject *callable = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 2958 "Python/bytecodes.c" + #line 2963 "Python/bytecodes.c" assert(kwnames == NULL); /* isinstance(o, o2) */ int is_meth = method != NULL; @@ -4228,7 +4233,7 @@ Py_DECREF(cls); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4232 "Python/generated_cases.c.h" + #line 4237 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4240,7 +4245,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *self = stack_pointer[-(1 + oparg)]; PyObject *method = stack_pointer[-(2 + oparg)]; - #line 2988 "Python/bytecodes.c" + #line 2993 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 1); assert(method != NULL); @@ -4258,14 +4263,14 @@ JUMPBY(INLINE_CACHE_ENTRIES_CALL + 1); assert(next_instr[-1].op.code == POP_TOP); DISPATCH(); - #line 4262 "Python/generated_cases.c.h" + #line 4267 "Python/generated_cases.c.h" } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3008 "Python/bytecodes.c" + #line 3013 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4296,7 +4301,7 @@ Py_DECREF(arg); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4300 "Python/generated_cases.c.h" + #line 4305 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4309,7 +4314,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3042 "Python/bytecodes.c" + #line 3047 "Python/bytecodes.c" int is_meth = method != NULL; int total_args = oparg; if (is_meth) { @@ -4338,7 +4343,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4342 "Python/generated_cases.c.h" + #line 4347 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4351,7 +4356,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3074 "Python/bytecodes.c" + #line 3079 "Python/bytecodes.c" assert(kwnames == NULL); assert(oparg == 0 || oparg == 1); int is_meth = method != NULL; @@ -4380,7 +4385,7 @@ Py_DECREF(self); Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4384 "Python/generated_cases.c.h" + #line 4389 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4393,7 +4398,7 @@ PyObject **args = (stack_pointer - oparg); PyObject *method = stack_pointer[-(2 + oparg)]; PyObject *res; - #line 3106 "Python/bytecodes.c" + #line 3111 "Python/bytecodes.c" assert(kwnames == NULL); int is_meth = method != NULL; int total_args = oparg; @@ -4421,7 +4426,7 @@ } Py_DECREF(callable); if (res == NULL) { STACK_SHRINK(oparg); goto pop_2_error; } - #line 4425 "Python/generated_cases.c.h" + #line 4430 "Python/generated_cases.c.h" STACK_SHRINK(oparg); STACK_SHRINK(1); stack_pointer[-1] = res; @@ -4431,9 +4436,9 @@ } TARGET(INSTRUMENTED_CALL_FUNCTION_EX) { - #line 3137 "Python/bytecodes.c" + #line 3142 "Python/bytecodes.c" GO_TO_INSTRUCTION(CALL_FUNCTION_EX); - #line 4437 "Python/generated_cases.c.h" + #line 4442 "Python/generated_cases.c.h" } TARGET(CALL_FUNCTION_EX) { @@ -4442,7 +4447,7 @@ PyObject *callargs = stack_pointer[-(1 + ((oparg & 1) ? 1 : 0))]; PyObject *func = stack_pointer[-(2 + ((oparg & 1) ? 1 : 0))]; PyObject *result; - #line 3141 "Python/bytecodes.c" + #line 3146 "Python/bytecodes.c" // DICT_MERGE is called before this opcode if there are kwargs. // It converts all dict subtypes in kwargs into regular dicts. assert(kwargs == NULL || PyDict_CheckExact(kwargs)); @@ -4504,14 +4509,14 @@ } result = PyObject_Call(func, callargs, kwargs); } - #line 4508 "Python/generated_cases.c.h" + #line 4513 "Python/generated_cases.c.h" Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); - #line 3203 "Python/bytecodes.c" + #line 3208 "Python/bytecodes.c" assert(PEEK(3 + (oparg & 1)) == NULL); if (result == NULL) { STACK_SHRINK(((oparg & 1) ? 1 : 0)); goto pop_3_error; } - #line 4515 "Python/generated_cases.c.h" + #line 4520 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 1) ? 1 : 0)); STACK_SHRINK(2); stack_pointer[-1] = result; @@ -4526,7 +4531,7 @@ PyObject *kwdefaults = (oparg & 0x02) ? stack_pointer[-(1 + ((oparg & 0x08) ? 1 : 0) + ((oparg & 0x04) ? 1 : 0) + ((oparg & 0x02) ? 1 : 0))] : NULL; PyObject *defaults = (oparg & 0x01) ? stack_pointer[-(1 + ((oparg & 0x08) ? 1 : 0) + ((oparg & 0x04) ? 1 : 0) + ((oparg & 0x02) ? 1 : 0) + ((oparg & 0x01) ? 1 : 0))] : NULL; PyObject *func; - #line 3213 "Python/bytecodes.c" + #line 3218 "Python/bytecodes.c" PyFunctionObject *func_obj = (PyFunctionObject *) PyFunction_New(codeobj, GLOBALS()); @@ -4555,14 +4560,14 @@ func_obj->func_version = ((PyCodeObject *)codeobj)->co_version; func = (PyObject *)func_obj; - #line 4559 "Python/generated_cases.c.h" + #line 4564 "Python/generated_cases.c.h" STACK_SHRINK(((oparg & 0x01) ? 1 : 0) + ((oparg & 0x02) ? 1 : 0) + ((oparg & 0x04) ? 1 : 0) + ((oparg & 0x08) ? 1 : 0)); stack_pointer[-1] = func; DISPATCH(); } TARGET(RETURN_GENERATOR) { - #line 3244 "Python/bytecodes.c" + #line 3249 "Python/bytecodes.c" assert(PyFunction_Check(frame->f_funcobj)); PyFunctionObject *func = (PyFunctionObject *)frame->f_funcobj; PyGenObject *gen = (PyGenObject *)_Py_MakeCoro(func); @@ -4583,7 +4588,7 @@ frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); goto resume_frame; - #line 4587 "Python/generated_cases.c.h" + #line 4592 "Python/generated_cases.c.h" } TARGET(BUILD_SLICE) { @@ -4591,15 +4596,15 @@ PyObject *stop = stack_pointer[-(1 + ((oparg == 3) ? 1 : 0))]; PyObject *start = stack_pointer[-(2 + ((oparg == 3) ? 1 : 0))]; PyObject *slice; - #line 3267 "Python/bytecodes.c" + #line 3272 "Python/bytecodes.c" slice = PySlice_New(start, stop, step); - #line 4597 "Python/generated_cases.c.h" + #line 4602 "Python/generated_cases.c.h" Py_DECREF(start); Py_DECREF(stop); Py_XDECREF(step); - #line 3269 "Python/bytecodes.c" + #line 3274 "Python/bytecodes.c" if (slice == NULL) { STACK_SHRINK(((oparg == 3) ? 1 : 0)); goto pop_2_error; } - #line 4603 "Python/generated_cases.c.h" + #line 4608 "Python/generated_cases.c.h" STACK_SHRINK(((oparg == 3) ? 1 : 0)); STACK_SHRINK(1); stack_pointer[-1] = slice; @@ -4610,7 +4615,7 @@ PyObject *fmt_spec = ((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? stack_pointer[-((((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0))] : NULL; PyObject *value = stack_pointer[-(1 + (((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0))]; PyObject *result; - #line 3273 "Python/bytecodes.c" + #line 3278 "Python/bytecodes.c" /* Handles f-string value formatting. */ PyObject *(*conv_fn)(PyObject *); int which_conversion = oparg & FVC_MASK; @@ -4645,7 +4650,7 @@ Py_DECREF(value); Py_XDECREF(fmt_spec); if (result == NULL) { STACK_SHRINK((((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0)); goto pop_1_error; } - #line 4649 "Python/generated_cases.c.h" + #line 4654 "Python/generated_cases.c.h" STACK_SHRINK((((oparg & FVS_MASK) == FVS_HAVE_SPEC) ? 1 : 0)); stack_pointer[-1] = result; DISPATCH(); @@ -4654,10 +4659,10 @@ TARGET(COPY) { PyObject *bottom = stack_pointer[-(1 + (oparg-1))]; PyObject *top; - #line 3310 "Python/bytecodes.c" + #line 3315 "Python/bytecodes.c" assert(oparg > 0); top = Py_NewRef(bottom); - #line 4661 "Python/generated_cases.c.h" + #line 4666 "Python/generated_cases.c.h" STACK_GROW(1); stack_pointer[-1] = top; DISPATCH(); @@ -4669,7 +4674,7 @@ PyObject *rhs = stack_pointer[-1]; PyObject *lhs = stack_pointer[-2]; PyObject *res; - #line 3315 "Python/bytecodes.c" + #line 3320 "Python/bytecodes.c" #if ENABLE_SPECIALIZATION _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; if (ADAPTIVE_COUNTER_IS_ZERO(cache->counter)) { @@ -4684,12 +4689,12 @@ assert((unsigned)oparg < Py_ARRAY_LENGTH(binary_ops)); assert(binary_ops[oparg]); res = binary_ops[oparg](lhs, rhs); - #line 4688 "Python/generated_cases.c.h" + #line 4693 "Python/generated_cases.c.h" Py_DECREF(lhs); Py_DECREF(rhs); - #line 3330 "Python/bytecodes.c" + #line 3335 "Python/bytecodes.c" if (res == NULL) goto pop_2_error; - #line 4693 "Python/generated_cases.c.h" + #line 4698 "Python/generated_cases.c.h" STACK_SHRINK(1); stack_pointer[-1] = res; next_instr += 1; @@ -4699,16 +4704,16 @@ TARGET(SWAP) { PyObject *top = stack_pointer[-1]; PyObject *bottom = stack_pointer[-(2 + (oparg-2))]; - #line 3335 "Python/bytecodes.c" + #line 3340 "Python/bytecodes.c" assert(oparg >= 2); - #line 4705 "Python/generated_cases.c.h" + #line 4710 "Python/generated_cases.c.h" stack_pointer[-1] = bottom; stack_pointer[-(2 + (oparg-2))] = top; DISPATCH(); } TARGET(INSTRUMENTED_INSTRUCTION) { - #line 3339 "Python/bytecodes.c" + #line 3344 "Python/bytecodes.c" int next_opcode = _Py_call_instrumentation_instruction( tstate, frame, next_instr-1); if (next_opcode < 0) goto error; @@ -4720,26 +4725,26 @@ assert(next_opcode > 0 && next_opcode < 256); opcode = next_opcode; DISPATCH_GOTO(); - #line 4724 "Python/generated_cases.c.h" + #line 4729 "Python/generated_cases.c.h" } TARGET(INSTRUMENTED_JUMP_FORWARD) { - #line 3353 "Python/bytecodes.c" + #line 3358 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr+oparg, PY_MONITORING_EVENT_JUMP); - #line 4730 "Python/generated_cases.c.h" + #line 4735 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_JUMP_BACKWARD) { - #line 3357 "Python/bytecodes.c" + #line 3362 "Python/bytecodes.c" INSTRUMENTED_JUMP(next_instr-1, next_instr-oparg, PY_MONITORING_EVENT_JUMP); - #line 4737 "Python/generated_cases.c.h" + #line 4742 "Python/generated_cases.c.h" CHECK_EVAL_BREAKER(); DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_TRUE) { - #line 3362 "Python/bytecodes.c" + #line 3367 "Python/bytecodes.c" PyObject *cond = POP(); int err = PyObject_IsTrue(cond); Py_DECREF(cond); @@ -4748,12 +4753,12 @@ assert(err == 0 || err == 1); int offset = err*oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4752 "Python/generated_cases.c.h" + #line 4757 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_FALSE) { - #line 3373 "Python/bytecodes.c" + #line 3378 "Python/bytecodes.c" PyObject *cond = POP(); int err = PyObject_IsTrue(cond); Py_DECREF(cond); @@ -4762,12 +4767,12 @@ assert(err == 0 || err == 1); int offset = (1-err)*oparg; INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4766 "Python/generated_cases.c.h" + #line 4771 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NONE) { - #line 3384 "Python/bytecodes.c" + #line 3389 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -4779,12 +4784,12 @@ offset = 0; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4783 "Python/generated_cases.c.h" + #line 4788 "Python/generated_cases.c.h" DISPATCH(); } TARGET(INSTRUMENTED_POP_JUMP_IF_NOT_NONE) { - #line 3398 "Python/bytecodes.c" + #line 3403 "Python/bytecodes.c" PyObject *value = POP(); _Py_CODEUNIT *here = next_instr-1; int offset; @@ -4796,30 +4801,30 @@ offset = oparg; } INSTRUMENTED_JUMP(here, next_instr + offset, PY_MONITORING_EVENT_BRANCH); - #line 4800 "Python/generated_cases.c.h" + #line 4805 "Python/generated_cases.c.h" DISPATCH(); } TARGET(EXTENDED_ARG) { - #line 3412 "Python/bytecodes.c" + #line 3417 "Python/bytecodes.c" assert(oparg); opcode = next_instr->op.code; oparg = oparg << 8 | next_instr->op.arg; PRE_DISPATCH_GOTO(); DISPATCH_GOTO(); - #line 4811 "Python/generated_cases.c.h" + #line 4816 "Python/generated_cases.c.h" } TARGET(CACHE) { - #line 3420 "Python/bytecodes.c" + #line 3425 "Python/bytecodes.c" assert(0 && "Executing a cache."); Py_UNREACHABLE(); - #line 4818 "Python/generated_cases.c.h" + #line 4823 "Python/generated_cases.c.h" } TARGET(RESERVED) { - #line 3425 "Python/bytecodes.c" + #line 3430 "Python/bytecodes.c" assert(0 && "Executing RESERVED instruction."); Py_UNREACHABLE(); - #line 4825 "Python/generated_cases.c.h" + #line 4830 "Python/generated_cases.c.h" } diff --git a/Python/instrumentation.c b/Python/instrumentation.c index 37933ffbd89b5..05a53d04bf3cc 100644 --- a/Python/instrumentation.c +++ b/Python/instrumentation.c @@ -696,29 +696,13 @@ instrument_per_instruction(PyCodeObject *code, int i) *opcode_ptr = INSTRUMENTED_INSTRUCTION; } -#ifndef NDEBUG -static bool -instruction_has_event(PyCodeObject *code, int offset) -{ - _Py_CODEUNIT instr = _PyCode_CODE(code)[offset]; - int opcode = instr.op.code; - if (opcode == INSTRUMENTED_LINE) { - opcode = code->_co_monitoring->lines[offset].original_opcode; - } - if (opcode == INSTRUMENTED_INSTRUCTION) { - opcode = code->_co_monitoring->per_instruction_opcodes[offset]; - } - return opcode_has_event(opcode); -} -#endif - static void remove_tools(PyCodeObject * code, int offset, int event, int tools) { assert(event != PY_MONITORING_EVENT_LINE); assert(event != PY_MONITORING_EVENT_INSTRUCTION); - assert(event < PY_MONITORING_INSTRUMENTED_EVENTS); - assert(instruction_has_event(code, offset)); + assert(PY_MONITORING_IS_INSTRUMENTED_EVENT(event)); + assert(opcode_has_event(_Py_GetBaseOpcode(code, offset))); _PyCoMonitoringData *monitoring = code->_co_monitoring; if (monitoring && monitoring->tools) { monitoring->tools[offset] &= ~tools; @@ -773,7 +757,7 @@ add_tools(PyCodeObject * code, int offset, int event, int tools) { assert(event != PY_MONITORING_EVENT_LINE); assert(event != PY_MONITORING_EVENT_INSTRUCTION); - assert(event < PY_MONITORING_INSTRUMENTED_EVENTS); + assert(PY_MONITORING_IS_INSTRUMENTED_EVENT(event)); assert(code->_co_monitoring); if (code->_co_monitoring && code->_co_monitoring->tools @@ -915,7 +899,7 @@ get_tools_for_instruction(PyCodeObject * code, int i, int event) event == PY_MONITORING_EVENT_C_RETURN); event = PY_MONITORING_EVENT_CALL; } - if (event < PY_MONITORING_INSTRUMENTED_EVENTS && monitoring->tools) { + if (PY_MONITORING_IS_INSTRUMENTED_EVENT(event) && monitoring->tools) { tools = monitoring->tools[i]; } else { @@ -926,6 +910,26 @@ get_tools_for_instruction(PyCodeObject * code, int i, int event) return tools; } +static const char *const event_names [] = { + [PY_MONITORING_EVENT_PY_START] = "PY_START", + [PY_MONITORING_EVENT_PY_RESUME] = "PY_RESUME", + [PY_MONITORING_EVENT_PY_RETURN] = "PY_RETURN", + [PY_MONITORING_EVENT_PY_YIELD] = "PY_YIELD", + [PY_MONITORING_EVENT_CALL] = "CALL", + [PY_MONITORING_EVENT_LINE] = "LINE", + [PY_MONITORING_EVENT_INSTRUCTION] = "INSTRUCTION", + [PY_MONITORING_EVENT_JUMP] = "JUMP", + [PY_MONITORING_EVENT_BRANCH] = "BRANCH", + [PY_MONITORING_EVENT_C_RETURN] = "C_RETURN", + [PY_MONITORING_EVENT_PY_THROW] = "PY_THROW", + [PY_MONITORING_EVENT_RAISE] = "RAISE", + [PY_MONITORING_EVENT_RERAISE] = "RERAISE", + [PY_MONITORING_EVENT_EXCEPTION_HANDLED] = "EXCEPTION_HANDLED", + [PY_MONITORING_EVENT_C_RAISE] = "C_RAISE", + [PY_MONITORING_EVENT_PY_UNWIND] = "PY_UNWIND", + [PY_MONITORING_EVENT_STOP_ITERATION] = "STOP_ITERATION", +}; + static int call_instrumentation_vector( PyThreadState *tstate, int event, @@ -973,7 +977,18 @@ call_instrumentation_vector( } else { /* DISABLE */ - remove_tools(code, offset, event, 1 << tool); + if (!PY_MONITORING_IS_INSTRUMENTED_EVENT(event)) { + PyErr_Format(PyExc_ValueError, + "Cannot disable %s events. Callback removed.", + event_names[event]); + /* Clear tool to prevent infinite loop */ + Py_CLEAR(interp->monitoring_callables[tool][event]); + err = -1; + break; + } + else { + remove_tools(code, offset, event, 1 << tool); + } } } Py_DECREF(offset_obj); @@ -1251,7 +1266,7 @@ initialize_tools(PyCodeObject *code) assert(event > 0); } assert(event >= 0); - assert(event < PY_MONITORING_INSTRUMENTED_EVENTS); + assert(PY_MONITORING_IS_INSTRUMENTED_EVENT(event)); tools[i] = code->_co_monitoring->active_monitors.tools[event]; CHECK(tools[i] != 0); } @@ -2024,26 +2039,6 @@ add_power2_constant(PyObject *obj, const char *name, int i) return err; } -static const char *const event_names [] = { - [PY_MONITORING_EVENT_PY_START] = "PY_START", - [PY_MONITORING_EVENT_PY_RESUME] = "PY_RESUME", - [PY_MONITORING_EVENT_PY_RETURN] = "PY_RETURN", - [PY_MONITORING_EVENT_PY_YIELD] = "PY_YIELD", - [PY_MONITORING_EVENT_CALL] = "CALL", - [PY_MONITORING_EVENT_LINE] = "LINE", - [PY_MONITORING_EVENT_INSTRUCTION] = "INSTRUCTION", - [PY_MONITORING_EVENT_JUMP] = "JUMP", - [PY_MONITORING_EVENT_BRANCH] = "BRANCH", - [PY_MONITORING_EVENT_C_RETURN] = "C_RETURN", - [PY_MONITORING_EVENT_PY_THROW] = "PY_THROW", - [PY_MONITORING_EVENT_RAISE] = "RAISE", - [PY_MONITORING_EVENT_RERAISE] = "RERAISE", - [PY_MONITORING_EVENT_EXCEPTION_HANDLED] = "EXCEPTION_HANDLED", - [PY_MONITORING_EVENT_C_RAISE] = "C_RAISE", - [PY_MONITORING_EVENT_PY_UNWIND] = "PY_UNWIND", - [PY_MONITORING_EVENT_STOP_ITERATION] = "STOP_ITERATION", -}; - /*[clinic input] monitoring._all_events [clinic start generated code]*/ From webhook-mailer at python.org Fri Jul 28 06:26:32 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 10:26:32 -0000 Subject: [Python-checkins] gh-107091: Fix some uses of :const: role (GH-107379) Message-ID: https://github.com/python/cpython/commit/0aa58fa7a62cd0ee7ec27fa87122425aeff0467d commit: 0aa58fa7a62cd0ee7ec27fa87122425aeff0467d branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-28T13:26:28+03:00 summary: gh-107091: Fix some uses of :const: role (GH-107379) It is for references, not for literals. files: M Doc/library/fcntl.rst M Doc/library/fractions.rst M Doc/library/logging.handlers.rst M Doc/library/os.rst M Doc/library/signal.rst M Doc/library/ssl.rst M Doc/library/subprocess.rst M Doc/library/sys.rst M Doc/tools/.nitignore M Doc/whatsnew/3.3.rst diff --git a/Doc/library/fcntl.rst b/Doc/library/fcntl.rst index 5a27646a96f59..969a79fa87339 100644 --- a/Doc/library/fcntl.rst +++ b/Doc/library/fcntl.rst @@ -172,9 +172,9 @@ The module defines the following functions: which the lock starts, relative to *whence*, and *whence* is as with :func:`io.IOBase.seek`, specifically: - * :const:`0` -- relative to the start of the file (:const:`os.SEEK_SET`) - * :const:`1` -- relative to the current buffer position (:const:`os.SEEK_CUR`) - * :const:`2` -- relative to the end of the file (:const:`os.SEEK_END`) + * ``0`` -- relative to the start of the file (:const:`os.SEEK_SET`) + * ``1`` -- relative to the current buffer position (:const:`os.SEEK_CUR`) + * ``2`` -- relative to the end of the file (:const:`os.SEEK_END`) The default for *start* is 0, which means to start at the beginning of the file. The default for *len* is 0 which means to lock to the end of the file. The diff --git a/Doc/library/fractions.rst b/Doc/library/fractions.rst index fe2e8ab655edf..509c63686f5a7 100644 --- a/Doc/library/fractions.rst +++ b/Doc/library/fractions.rst @@ -25,7 +25,7 @@ another rational number, or from a string. The first version requires that *numerator* and *denominator* are instances of :class:`numbers.Rational` and returns a new :class:`Fraction` instance - with value ``numerator/denominator``. If *denominator* is :const:`0`, it + with value ``numerator/denominator``. If *denominator* is ``0``, it raises a :exc:`ZeroDivisionError`. The second version requires that *other_fraction* is an instance of :class:`numbers.Rational` and returns a :class:`Fraction` instance with the same value. The next two versions accept diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index 72e5ffb3a1218..2a825db54aed5 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -97,7 +97,7 @@ sends logging output to a disk file. It inherits the output functionality from Returns a new instance of the :class:`FileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, - :const:`'a'` is used. If *encoding* is not ``None``, it is used to open the file + ``'a'`` is used. If *encoding* is not ``None``, it is used to open the file with that encoding. If *delay* is true, then file opening is deferred until the first call to :meth:`emit`. By default, the file grows indefinitely. If *errors* is specified, it's used to determine how encoding errors are handled. @@ -182,7 +182,7 @@ for this value. Returns a new instance of the :class:`WatchedFileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, - :const:`'a'` is used. If *encoding* is not ``None``, it is used to open the file + ``'a'`` is used. If *encoding* is not ``None``, it is used to open the file with that encoding. If *delay* is true, then file opening is deferred until the first call to :meth:`emit`. By default, the file grows indefinitely. If *errors* is provided, it determines how encoding errors are handled. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 956eeffa1cd45..9735baa5bc0f3 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -401,11 +401,11 @@ process and user. On macOS, :func:`getgroups` behavior differs somewhat from other Unix platforms. If the Python interpreter was built with a - deployment target of :const:`10.5` or earlier, :func:`getgroups` returns + deployment target of ``10.5`` or earlier, :func:`getgroups` returns the list of effective group ids associated with the current user process; this list is limited to a system-defined number of entries, typically 16, and may be modified by calls to :func:`setgroups` if suitably privileged. - If built with a deployment target greater than :const:`10.5`, + If built with a deployment target greater than ``10.5``, :func:`getgroups` returns the current group access list for the user associated with the effective user id of the process; the group access list may change over the lifetime of the process, it is not affected by diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index e53315bee3ea3..7ee5ece885982 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -656,7 +656,7 @@ The :mod:`signal` module defines the following functions: .. function:: sigtimedwait(sigset, timeout) Like :func:`sigwaitinfo`, but takes an additional *timeout* argument - specifying a timeout. If *timeout* is specified as :const:`0`, a poll is + specifying a timeout. If *timeout* is specified as ``0``, a poll is performed. Returns :const:`None` if a timeout occurs. .. availability:: Unix. diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 1f8bbe1e3de3e..5d6bc829d6887 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -320,7 +320,7 @@ Random generation Mix the given *bytes* into the SSL pseudo-random number generator. The parameter *entropy* (a float) is a lower bound on the entropy contained in - string (so you can always use :const:`0.0`). See :rfc:`1750` for more + string (so you can always use ``0.0``). See :rfc:`1750` for more information on sources of entropy. .. versionchanged:: 3.5 diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index db05cb294d93f..04340cca9e4a5 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -465,9 +465,9 @@ functions. :func:`open` function when creating the stdin/stdout/stderr pipe file objects: - - :const:`0` means unbuffered (read and write are one + - ``0`` means unbuffered (read and write are one system call and can return short) - - :const:`1` means line buffered + - ``1`` means line buffered (only usable if ``text=True`` or ``universal_newlines=True``) - any other positive value means use a buffer of approximately that size @@ -477,7 +477,7 @@ functions. .. versionchanged:: 3.3.1 *bufsize* now defaults to -1 to enable buffering by default to match the behavior that most code expects. In versions prior to Python 3.2.4 and - 3.3.1 it incorrectly defaulted to :const:`0` which was unbuffered + 3.3.1 it incorrectly defaulted to ``0`` which was unbuffered and allowed short reads. This was unintentional and did not match the behavior of Python 2 as most code expected. @@ -541,8 +541,8 @@ functions. :exc:`RuntimeError`. The new restriction may affect applications that are deployed in mod_wsgi, uWSGI, and other embedded environments. - If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and - :const:`2` will be closed before the child process is executed. Otherwise + If *close_fds* is true, all file descriptors except ``0``, ``1`` and + ``2`` will be closed before the child process is executed. Otherwise when *close_fds* is false, file descriptors obey their inheritable flag as described in :ref:`fd_inheritance`. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index f6b67dc57c187..a61737383e393 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -877,19 +877,19 @@ always available. ``sys.getwindowsversion().major``. For compatibility with prior versions, only the first 5 elements are retrievable by indexing. - *platform* will be :const:`2 (VER_PLATFORM_WIN32_NT)`. + *platform* will be ``2`` (VER_PLATFORM_WIN32_NT). *product_type* may be one of the following values: +---------------------------------------+---------------------------------+ | Constant | Meaning | +=======================================+=================================+ - | :const:`1 (VER_NT_WORKSTATION)` | The system is a workstation. | + | ``1`` (VER_NT_WORKSTATION) | The system is a workstation. | +---------------------------------------+---------------------------------+ - | :const:`2 (VER_NT_DOMAIN_CONTROLLER)` | The system is a domain | + | ``2`` (VER_NT_DOMAIN_CONTROLLER) | The system is a domain | | | controller. | +---------------------------------------+---------------------------------+ - | :const:`3 (VER_NT_SERVER)` | The system is a server, but not | + | ``3`` (VER_NT_SERVER) | The system is a server, but not | | | a domain controller. | +---------------------------------------+---------------------------------+ diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 27f483b74184b..22eb47856f571 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -90,7 +90,6 @@ Doc/library/faulthandler.rst Doc/library/fcntl.rst Doc/library/filecmp.rst Doc/library/fileinput.rst -Doc/library/fractions.rst Doc/library/ftplib.rst Doc/library/functions.rst Doc/library/functools.rst diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 2b87f9eb9196a..bcdc0222be6de 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -1135,20 +1135,20 @@ API changes * The C module has the following context limits, depending on the machine architecture: - +-------------------+---------------------+------------------------------+ - | | 32-bit | 64-bit | - +===================+=====================+==============================+ - | :const:`MAX_PREC` | :const:`425000000` | :const:`999999999999999999` | - +-------------------+---------------------+------------------------------+ - | :const:`MAX_EMAX` | :const:`425000000` | :const:`999999999999999999` | - +-------------------+---------------------+------------------------------+ - | :const:`MIN_EMIN` | :const:`-425000000` | :const:`-999999999999999999` | - +-------------------+---------------------+------------------------------+ + +-------------------+----------------+-------------------------+ + | | 32-bit | 64-bit | + +===================+================+=========================+ + | :const:`MAX_PREC` | ``425000000`` | ``999999999999999999`` | + +-------------------+----------------+-------------------------+ + | :const:`MAX_EMAX` | ``425000000`` | ``999999999999999999`` | + +-------------------+----------------+-------------------------+ + | :const:`MIN_EMIN` | ``-425000000`` | ``-999999999999999999`` | + +-------------------+----------------+-------------------------+ * In the context templates (:class:`~decimal.DefaultContext`, :class:`~decimal.BasicContext` and :class:`~decimal.ExtendedContext`) the magnitude of :attr:`~decimal.Context.Emax` and - :attr:`~decimal.Context.Emin` has changed to :const:`999999`. + :attr:`~decimal.Context.Emin` has changed to ``999999``. * The :class:`~decimal.Decimal` constructor in decimal.py does not observe the context limits and converts values with arbitrary exponents or precision From webhook-mailer at python.org Fri Jul 28 06:30:20 2023 From: webhook-mailer at python.org (markshannon) Date: Fri, 28 Jul 2023 10:30:20 -0000 Subject: [Python-checkins] [3.12] GH-106898: Add the exception as an argument to the `PY_UNWIND` event callback function. (GH-107347) (GH-107382) Message-ID: https://github.com/python/cpython/commit/3f167de440f65eb03f795e33b98cf3e84627e213 commit: 3f167de440f65eb03f795e33b98cf3e84627e213 branch: 3.12 author: Mark Shannon committer: markshannon date: 2023-07-28T11:30:16+01:00 summary: [3.12] GH-106898: Add the exception as an argument to the `PY_UNWIND` event callback function. (GH-107347) (GH-107382) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-26-21-28-06.gh-issue-106898.8Wjuiv.rst M Lib/test/test_monitoring.py M Python/ceval.c M Python/legacy_tracing.c diff --git a/Lib/test/test_monitoring.py b/Lib/test/test_monitoring.py index 9685a43810136..9d0ad6fa834bc 100644 --- a/Lib/test/test_monitoring.py +++ b/Lib/test/test_monitoring.py @@ -715,7 +715,7 @@ def check_balanced(self, func, recorders): self.assertIn(r0, ("raise", "reraise")) h0 = h[0] self.assertIn(h0, ("handled", "unwind")) - + self.assertEqual(r[1], h[1]) class StopiterationRecorder(ExceptionRecorder): @@ -733,8 +733,8 @@ class UnwindRecorder(ExceptionRecorder): event_type = E.PY_UNWIND - def __call__(self, *args): - self.events.append(("unwind", None)) + def __call__(self, code, offset, exc): + self.events.append(("unwind", type(exc))) class ExceptionHandledRecorder(ExceptionRecorder): diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-26-21-28-06.gh-issue-106898.8Wjuiv.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-21-28-06.gh-issue-106898.8Wjuiv.rst new file mode 100644 index 0000000000000..f1b1c4c64b4ac --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-26-21-28-06.gh-issue-106898.8Wjuiv.rst @@ -0,0 +1,3 @@ +Add the exception as the third argument to ``PY_UNIND`` callbacks in +``sys.monitoring``. This makes the ``PY_UNWIND`` callback consistent with +the other exception hanlding callbacks. diff --git a/Python/ceval.c b/Python/ceval.c index 27dea27e81200..4947420616213 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2070,7 +2070,7 @@ monitor_unwind(PyThreadState *tstate, if (no_tools_for_event(tstate, frame, PY_MONITORING_EVENT_PY_UNWIND)) { return; } - _Py_call_instrumentation_exc0(tstate, PY_MONITORING_EVENT_PY_UNWIND, frame, instr); + do_monitor_exc(tstate, frame, instr, PY_MONITORING_EVENT_PY_UNWIND); } diff --git a/Python/legacy_tracing.c b/Python/legacy_tracing.c index 5143b79b0864d..c1c70f667ccd2 100644 --- a/Python/legacy_tracing.c +++ b/Python/legacy_tracing.c @@ -64,6 +64,16 @@ sys_profile_func3( return call_profile_func(self, args[2]); } +static PyObject * +sys_profile_unwind( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 3); + return call_profile_func(self, Py_None); +} + static PyObject * sys_profile_call_or_return( _PyLegacyEventHandler *self, PyObject *const *args, @@ -152,6 +162,16 @@ sys_trace_func2( return call_trace_func(self, Py_None); } +static PyObject * +sys_trace_unwind( + _PyLegacyEventHandler *self, PyObject *const *args, + size_t nargsf, PyObject *kwnames +) { + assert(kwnames == NULL); + assert(PyVectorcall_NARGS(nargsf) == 3); + return call_trace_func(self, Py_None); +} + static PyObject * sys_trace_return( _PyLegacyEventHandler *self, PyObject *const *args, @@ -363,7 +383,7 @@ _PyEval_SetProfile(PyThreadState *tstate, Py_tracefunc func, PyObject *arg) return -1; } if (set_callbacks(PY_MONITORING_SYS_PROFILE_ID, - (vectorcallfunc)sys_profile_func2, PyTrace_RETURN, + (vectorcallfunc)sys_profile_unwind, PyTrace_RETURN, PY_MONITORING_EVENT_PY_UNWIND, -1)) { return -1; } @@ -451,7 +471,7 @@ _PyEval_SetTrace(PyThreadState *tstate, Py_tracefunc func, PyObject *arg) return -1; } if (set_callbacks(PY_MONITORING_SYS_TRACE_ID, - (vectorcallfunc)sys_trace_func2, PyTrace_RETURN, + (vectorcallfunc)sys_trace_unwind, PyTrace_RETURN, PY_MONITORING_EVENT_PY_UNWIND, -1)) { return -1; } From webhook-mailer at python.org Fri Jul 28 06:35:24 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 10:35:24 -0000 Subject: [Python-checkins] [3.12] gh-107091: Fix some uses of :const: role (GH-107379) (GH-107384) Message-ID: https://github.com/python/cpython/commit/0f42f41ea804e76bdb72c012d13b45a048e46c36 commit: 0f42f41ea804e76bdb72c012d13b45a048e46c36 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-28T10:35:20Z summary: [3.12] gh-107091: Fix some uses of :const: role (GH-107379) (GH-107384) It is for references, not for literals. (cherry picked from commit 0aa58fa7a62cd0ee7ec27fa87122425aeff0467d) Co-authored-by: Serhiy Storchaka files: M Doc/library/fcntl.rst M Doc/library/fractions.rst M Doc/library/logging.handlers.rst M Doc/library/os.rst M Doc/library/signal.rst M Doc/library/ssl.rst M Doc/library/subprocess.rst M Doc/library/sys.rst M Doc/tools/.nitignore M Doc/whatsnew/3.3.rst diff --git a/Doc/library/fcntl.rst b/Doc/library/fcntl.rst index 5a27646a96f59..969a79fa87339 100644 --- a/Doc/library/fcntl.rst +++ b/Doc/library/fcntl.rst @@ -172,9 +172,9 @@ The module defines the following functions: which the lock starts, relative to *whence*, and *whence* is as with :func:`io.IOBase.seek`, specifically: - * :const:`0` -- relative to the start of the file (:const:`os.SEEK_SET`) - * :const:`1` -- relative to the current buffer position (:const:`os.SEEK_CUR`) - * :const:`2` -- relative to the end of the file (:const:`os.SEEK_END`) + * ``0`` -- relative to the start of the file (:const:`os.SEEK_SET`) + * ``1`` -- relative to the current buffer position (:const:`os.SEEK_CUR`) + * ``2`` -- relative to the end of the file (:const:`os.SEEK_END`) The default for *start* is 0, which means to start at the beginning of the file. The default for *len* is 0 which means to lock to the end of the file. The diff --git a/Doc/library/fractions.rst b/Doc/library/fractions.rst index fe2e8ab655edf..509c63686f5a7 100644 --- a/Doc/library/fractions.rst +++ b/Doc/library/fractions.rst @@ -25,7 +25,7 @@ another rational number, or from a string. The first version requires that *numerator* and *denominator* are instances of :class:`numbers.Rational` and returns a new :class:`Fraction` instance - with value ``numerator/denominator``. If *denominator* is :const:`0`, it + with value ``numerator/denominator``. If *denominator* is ``0``, it raises a :exc:`ZeroDivisionError`. The second version requires that *other_fraction* is an instance of :class:`numbers.Rational` and returns a :class:`Fraction` instance with the same value. The next two versions accept diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index 72e5ffb3a1218..2a825db54aed5 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -97,7 +97,7 @@ sends logging output to a disk file. It inherits the output functionality from Returns a new instance of the :class:`FileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, - :const:`'a'` is used. If *encoding* is not ``None``, it is used to open the file + ``'a'`` is used. If *encoding* is not ``None``, it is used to open the file with that encoding. If *delay* is true, then file opening is deferred until the first call to :meth:`emit`. By default, the file grows indefinitely. If *errors* is specified, it's used to determine how encoding errors are handled. @@ -182,7 +182,7 @@ for this value. Returns a new instance of the :class:`WatchedFileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, - :const:`'a'` is used. If *encoding* is not ``None``, it is used to open the file + ``'a'`` is used. If *encoding* is not ``None``, it is used to open the file with that encoding. If *delay* is true, then file opening is deferred until the first call to :meth:`emit`. By default, the file grows indefinitely. If *errors* is provided, it determines how encoding errors are handled. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 956eeffa1cd45..9735baa5bc0f3 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -401,11 +401,11 @@ process and user. On macOS, :func:`getgroups` behavior differs somewhat from other Unix platforms. If the Python interpreter was built with a - deployment target of :const:`10.5` or earlier, :func:`getgroups` returns + deployment target of ``10.5`` or earlier, :func:`getgroups` returns the list of effective group ids associated with the current user process; this list is limited to a system-defined number of entries, typically 16, and may be modified by calls to :func:`setgroups` if suitably privileged. - If built with a deployment target greater than :const:`10.5`, + If built with a deployment target greater than ``10.5``, :func:`getgroups` returns the current group access list for the user associated with the effective user id of the process; the group access list may change over the lifetime of the process, it is not affected by diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index e53315bee3ea3..7ee5ece885982 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -656,7 +656,7 @@ The :mod:`signal` module defines the following functions: .. function:: sigtimedwait(sigset, timeout) Like :func:`sigwaitinfo`, but takes an additional *timeout* argument - specifying a timeout. If *timeout* is specified as :const:`0`, a poll is + specifying a timeout. If *timeout* is specified as ``0``, a poll is performed. Returns :const:`None` if a timeout occurs. .. availability:: Unix. diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 1f8bbe1e3de3e..5d6bc829d6887 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -320,7 +320,7 @@ Random generation Mix the given *bytes* into the SSL pseudo-random number generator. The parameter *entropy* (a float) is a lower bound on the entropy contained in - string (so you can always use :const:`0.0`). See :rfc:`1750` for more + string (so you can always use ``0.0``). See :rfc:`1750` for more information on sources of entropy. .. versionchanged:: 3.5 diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index db05cb294d93f..04340cca9e4a5 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -465,9 +465,9 @@ functions. :func:`open` function when creating the stdin/stdout/stderr pipe file objects: - - :const:`0` means unbuffered (read and write are one + - ``0`` means unbuffered (read and write are one system call and can return short) - - :const:`1` means line buffered + - ``1`` means line buffered (only usable if ``text=True`` or ``universal_newlines=True``) - any other positive value means use a buffer of approximately that size @@ -477,7 +477,7 @@ functions. .. versionchanged:: 3.3.1 *bufsize* now defaults to -1 to enable buffering by default to match the behavior that most code expects. In versions prior to Python 3.2.4 and - 3.3.1 it incorrectly defaulted to :const:`0` which was unbuffered + 3.3.1 it incorrectly defaulted to ``0`` which was unbuffered and allowed short reads. This was unintentional and did not match the behavior of Python 2 as most code expected. @@ -541,8 +541,8 @@ functions. :exc:`RuntimeError`. The new restriction may affect applications that are deployed in mod_wsgi, uWSGI, and other embedded environments. - If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and - :const:`2` will be closed before the child process is executed. Otherwise + If *close_fds* is true, all file descriptors except ``0``, ``1`` and + ``2`` will be closed before the child process is executed. Otherwise when *close_fds* is false, file descriptors obey their inheritable flag as described in :ref:`fd_inheritance`. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index f6b67dc57c187..a61737383e393 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -877,19 +877,19 @@ always available. ``sys.getwindowsversion().major``. For compatibility with prior versions, only the first 5 elements are retrievable by indexing. - *platform* will be :const:`2 (VER_PLATFORM_WIN32_NT)`. + *platform* will be ``2`` (VER_PLATFORM_WIN32_NT). *product_type* may be one of the following values: +---------------------------------------+---------------------------------+ | Constant | Meaning | +=======================================+=================================+ - | :const:`1 (VER_NT_WORKSTATION)` | The system is a workstation. | + | ``1`` (VER_NT_WORKSTATION) | The system is a workstation. | +---------------------------------------+---------------------------------+ - | :const:`2 (VER_NT_DOMAIN_CONTROLLER)` | The system is a domain | + | ``2`` (VER_NT_DOMAIN_CONTROLLER) | The system is a domain | | | controller. | +---------------------------------------+---------------------------------+ - | :const:`3 (VER_NT_SERVER)` | The system is a server, but not | + | ``3`` (VER_NT_SERVER) | The system is a server, but not | | | a domain controller. | +---------------------------------------+---------------------------------+ diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 63223de613ff7..899e98acec8ad 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -100,7 +100,6 @@ Doc/library/faulthandler.rst Doc/library/fcntl.rst Doc/library/filecmp.rst Doc/library/fileinput.rst -Doc/library/fractions.rst Doc/library/ftplib.rst Doc/library/functions.rst Doc/library/functools.rst diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 6eb5818c317d8..1414b2f79a67d 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -1135,20 +1135,20 @@ API changes * The C module has the following context limits, depending on the machine architecture: - +-------------------+---------------------+------------------------------+ - | | 32-bit | 64-bit | - +===================+=====================+==============================+ - | :const:`MAX_PREC` | :const:`425000000` | :const:`999999999999999999` | - +-------------------+---------------------+------------------------------+ - | :const:`MAX_EMAX` | :const:`425000000` | :const:`999999999999999999` | - +-------------------+---------------------+------------------------------+ - | :const:`MIN_EMIN` | :const:`-425000000` | :const:`-999999999999999999` | - +-------------------+---------------------+------------------------------+ + +-------------------+----------------+-------------------------+ + | | 32-bit | 64-bit | + +===================+================+=========================+ + | :const:`MAX_PREC` | ``425000000`` | ``999999999999999999`` | + +-------------------+----------------+-------------------------+ + | :const:`MAX_EMAX` | ``425000000`` | ``999999999999999999`` | + +-------------------+----------------+-------------------------+ + | :const:`MIN_EMIN` | ``-425000000`` | ``-999999999999999999`` | + +-------------------+----------------+-------------------------+ * In the context templates (:class:`~decimal.DefaultContext`, :class:`~decimal.BasicContext` and :class:`~decimal.ExtendedContext`) the magnitude of :attr:`~decimal.Context.Emax` and - :attr:`~decimal.Context.Emin` has changed to :const:`999999`. + :attr:`~decimal.Context.Emin` has changed to ``999999``. * The :class:`~decimal.Decimal` constructor in decimal.py does not observe the context limits and converts values with arbitrary exponents or precision From webhook-mailer at python.org Fri Jul 28 06:36:23 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 28 Jul 2023 10:36:23 -0000 Subject: [Python-checkins] [3.11] gh-107091: Fix some uses of :const: role (GH-107379) (GH-107385) Message-ID: https://github.com/python/cpython/commit/c3432523d13dbbabe58bd0a13747b801d52e4f5a commit: c3432523d13dbbabe58bd0a13747b801d52e4f5a branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-28T10:36:19Z summary: [3.11] gh-107091: Fix some uses of :const: role (GH-107379) (GH-107385) It is for references, not for literals. (cherry picked from commit 0aa58fa7a62cd0ee7ec27fa87122425aeff0467d) Co-authored-by: Serhiy Storchaka files: M Doc/library/fcntl.rst M Doc/library/fractions.rst M Doc/library/logging.handlers.rst M Doc/library/os.rst M Doc/library/signal.rst M Doc/library/ssl.rst M Doc/library/subprocess.rst M Doc/library/sys.rst M Doc/tools/.nitignore M Doc/whatsnew/3.3.rst diff --git a/Doc/library/fcntl.rst b/Doc/library/fcntl.rst index 13de0c7bc33bf..1951f80f91e66 100644 --- a/Doc/library/fcntl.rst +++ b/Doc/library/fcntl.rst @@ -166,9 +166,9 @@ The module defines the following functions: which the lock starts, relative to *whence*, and *whence* is as with :func:`io.IOBase.seek`, specifically: - * :const:`0` -- relative to the start of the file (:const:`os.SEEK_SET`) - * :const:`1` -- relative to the current buffer position (:const:`os.SEEK_CUR`) - * :const:`2` -- relative to the end of the file (:const:`os.SEEK_END`) + * ``0`` -- relative to the start of the file (:const:`os.SEEK_SET`) + * ``1`` -- relative to the current buffer position (:const:`os.SEEK_CUR`) + * ``2`` -- relative to the end of the file (:const:`os.SEEK_END`) The default for *start* is 0, which means to start at the beginning of the file. The default for *len* is 0 which means to lock to the end of the file. The diff --git a/Doc/library/fractions.rst b/Doc/library/fractions.rst index 85bec73eafc9f..c751253a51c86 100644 --- a/Doc/library/fractions.rst +++ b/Doc/library/fractions.rst @@ -25,7 +25,7 @@ another rational number, or from a string. The first version requires that *numerator* and *denominator* are instances of :class:`numbers.Rational` and returns a new :class:`Fraction` instance - with value ``numerator/denominator``. If *denominator* is :const:`0`, it + with value ``numerator/denominator``. If *denominator* is ``0``, it raises a :exc:`ZeroDivisionError`. The second version requires that *other_fraction* is an instance of :class:`numbers.Rational` and returns a :class:`Fraction` instance with the same value. The next two versions accept diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst index d1566957e7ed1..f9b3b9b9abe65 100644 --- a/Doc/library/logging.handlers.rst +++ b/Doc/library/logging.handlers.rst @@ -97,7 +97,7 @@ sends logging output to a disk file. It inherits the output functionality from Returns a new instance of the :class:`FileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, - :const:`'a'` is used. If *encoding* is not ``None``, it is used to open the file + ``'a'`` is used. If *encoding* is not ``None``, it is used to open the file with that encoding. If *delay* is true, then file opening is deferred until the first call to :meth:`emit`. By default, the file grows indefinitely. If *errors* is specified, it's used to determine how encoding errors are handled. @@ -182,7 +182,7 @@ for this value. Returns a new instance of the :class:`WatchedFileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, - :const:`'a'` is used. If *encoding* is not ``None``, it is used to open the file + ``'a'`` is used. If *encoding* is not ``None``, it is used to open the file with that encoding. If *delay* is true, then file opening is deferred until the first call to :meth:`emit`. By default, the file grows indefinitely. If *errors* is provided, it determines how encoding errors are handled. diff --git a/Doc/library/os.rst b/Doc/library/os.rst index bb9cbae2b1345..5577921478452 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -401,11 +401,11 @@ process and user. On macOS, :func:`getgroups` behavior differs somewhat from other Unix platforms. If the Python interpreter was built with a - deployment target of :const:`10.5` or earlier, :func:`getgroups` returns + deployment target of ``10.5`` or earlier, :func:`getgroups` returns the list of effective group ids associated with the current user process; this list is limited to a system-defined number of entries, typically 16, and may be modified by calls to :func:`setgroups` if suitably privileged. - If built with a deployment target greater than :const:`10.5`, + If built with a deployment target greater than ``10.5``, :func:`getgroups` returns the current group access list for the user associated with the effective user id of the process; the group access list may change over the lifetime of the process, it is not affected by diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index e53315bee3ea3..7ee5ece885982 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -656,7 +656,7 @@ The :mod:`signal` module defines the following functions: .. function:: sigtimedwait(sigset, timeout) Like :func:`sigwaitinfo`, but takes an additional *timeout* argument - specifying a timeout. If *timeout* is specified as :const:`0`, a poll is + specifying a timeout. If *timeout* is specified as ``0``, a poll is performed. Returns :const:`None` if a timeout occurs. .. availability:: Unix. diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index fe896be2faeb5..25369969f544b 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -344,7 +344,7 @@ Random generation Mix the given *bytes* into the SSL pseudo-random number generator. The parameter *entropy* (a float) is a lower bound on the entropy contained in - string (so you can always use :const:`0.0`). See :rfc:`1750` for more + string (so you can always use ``0.0``). See :rfc:`1750` for more information on sources of entropy. .. versionchanged:: 3.5 diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 78718f7983e04..6ef3c34b4fb81 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -463,9 +463,9 @@ functions. :func:`open` function when creating the stdin/stdout/stderr pipe file objects: - - :const:`0` means unbuffered (read and write are one + - ``0`` means unbuffered (read and write are one system call and can return short) - - :const:`1` means line buffered + - ``1`` means line buffered (only usable if ``text=True`` or ``universal_newlines=True``) - any other positive value means use a buffer of approximately that size @@ -475,7 +475,7 @@ functions. .. versionchanged:: 3.3.1 *bufsize* now defaults to -1 to enable buffering by default to match the behavior that most code expects. In versions prior to Python 3.2.4 and - 3.3.1 it incorrectly defaulted to :const:`0` which was unbuffered + 3.3.1 it incorrectly defaulted to ``0`` which was unbuffered and allowed short reads. This was unintentional and did not match the behavior of Python 2 as most code expected. @@ -540,8 +540,8 @@ functions. :exc:`RuntimeError`. The new restriction may affect applications that are deployed in mod_wsgi, uWSGI, and other embedded environments. - If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and - :const:`2` will be closed before the child process is executed. Otherwise + If *close_fds* is true, all file descriptors except ``0``, ``1`` and + ``2`` will be closed before the child process is executed. Otherwise when *close_fds* is false, file descriptors obey their inheritable flag as described in :ref:`fd_inheritance`. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index dca35cc731b69..b03860603c286 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -850,19 +850,19 @@ always available. ``sys.getwindowsversion().major``. For compatibility with prior versions, only the first 5 elements are retrievable by indexing. - *platform* will be :const:`2 (VER_PLATFORM_WIN32_NT)`. + *platform* will be ``2`` (VER_PLATFORM_WIN32_NT). *product_type* may be one of the following values: +---------------------------------------+---------------------------------+ | Constant | Meaning | +=======================================+=================================+ - | :const:`1 (VER_NT_WORKSTATION)` | The system is a workstation. | + | ``1`` (VER_NT_WORKSTATION) | The system is a workstation. | +---------------------------------------+---------------------------------+ - | :const:`2 (VER_NT_DOMAIN_CONTROLLER)` | The system is a domain | + | ``2`` (VER_NT_DOMAIN_CONTROLLER) | The system is a domain | | | controller. | +---------------------------------------+---------------------------------+ - | :const:`3 (VER_NT_SERVER)` | The system is a server, but not | + | ``3`` (VER_NT_SERVER) | The system is a server, but not | | | a domain controller. | +---------------------------------------+---------------------------------+ diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 27f483b74184b..22eb47856f571 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -90,7 +90,6 @@ Doc/library/faulthandler.rst Doc/library/fcntl.rst Doc/library/filecmp.rst Doc/library/fileinput.rst -Doc/library/fractions.rst Doc/library/ftplib.rst Doc/library/functions.rst Doc/library/functools.rst diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 1fda66cf9f0ea..9fb06c1d1f807 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -1134,20 +1134,20 @@ API changes * The C module has the following context limits, depending on the machine architecture: - +-------------------+---------------------+------------------------------+ - | | 32-bit | 64-bit | - +===================+=====================+==============================+ - | :const:`MAX_PREC` | :const:`425000000` | :const:`999999999999999999` | - +-------------------+---------------------+------------------------------+ - | :const:`MAX_EMAX` | :const:`425000000` | :const:`999999999999999999` | - +-------------------+---------------------+------------------------------+ - | :const:`MIN_EMIN` | :const:`-425000000` | :const:`-999999999999999999` | - +-------------------+---------------------+------------------------------+ + +-------------------+----------------+-------------------------+ + | | 32-bit | 64-bit | + +===================+================+=========================+ + | :const:`MAX_PREC` | ``425000000`` | ``999999999999999999`` | + +-------------------+----------------+-------------------------+ + | :const:`MAX_EMAX` | ``425000000`` | ``999999999999999999`` | + +-------------------+----------------+-------------------------+ + | :const:`MIN_EMIN` | ``-425000000`` | ``-999999999999999999`` | + +-------------------+----------------+-------------------------+ * In the context templates (:class:`~decimal.DefaultContext`, :class:`~decimal.BasicContext` and :class:`~decimal.ExtendedContext`) the magnitude of :attr:`~decimal.Context.Emax` and - :attr:`~decimal.Context.Emin` has changed to :const:`999999`. + :attr:`~decimal.Context.Emin` has changed to ``999999``. * The :class:`~decimal.Decimal` constructor in decimal.py does not observe the context limits and converts values with arbitrary exponents or precision From webhook-mailer at python.org Fri Jul 28 08:55:29 2023 From: webhook-mailer at python.org (markshannon) Date: Fri, 28 Jul 2023 12:55:29 -0000 Subject: [Python-checkins] GH-104580: Put `eval_breaker` back at the start of the interpreter state. (GH-107383) Message-ID: https://github.com/python/cpython/commit/a1b679572e7156bc4299819e73235dc608d4a570 commit: a1b679572e7156bc4299819e73235dc608d4a570 branch: main author: Mark Shannon committer: markshannon date: 2023-07-28T13:55:25+01:00 summary: GH-104580: Put `eval_breaker` back at the start of the interpreter state. (GH-107383) files: M Include/internal/pycore_ceval_state.h M Include/internal/pycore_interp.h diff --git a/Include/internal/pycore_ceval_state.h b/Include/internal/pycore_ceval_state.h index e56e43c6e0c6a..1ebfcc9bebd0a 100644 --- a/Include/internal/pycore_ceval_state.h +++ b/Include/internal/pycore_ceval_state.h @@ -84,7 +84,9 @@ struct _ceval_runtime_state { struct _ceval_state { /* This single variable consolidates all requests to break out of - the fast path in the eval loop. */ + * the fast path in the eval loop. + * It is by far the hottest field in this struct and + * should be placed at the beginning. */ _Py_atomic_int eval_breaker; /* Request for dropping the GIL */ _Py_atomic_int gil_drop_request; diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index bd6a9f28f468a..91c473e58eaba 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -48,6 +48,11 @@ struct _Py_long_state { */ struct _is { + /* This struct countains the eval_breaker, + * which is by far the hottest field in this struct + * and should be placed at the beginning. */ + struct _ceval_state ceval; + PyInterpreterState *next; int64_t id; @@ -109,8 +114,6 @@ struct _is { // Dictionary of the builtins module PyObject *builtins; - struct _ceval_state ceval; - struct _import_state imports; /* The per-interpreter GIL, which might not be used. */ From webhook-mailer at python.org Fri Jul 28 09:39:58 2023 From: webhook-mailer at python.org (markshannon) Date: Fri, 28 Jul 2023 13:39:58 -0000 Subject: [Python-checkins] gh-102509: Start initializing `ob_digit` of `_PyLongValue` (GH-102510) Message-ID: https://github.com/python/cpython/commit/fc130c47daa715d60d8925c478a96d5083e47b6a commit: fc130c47daa715d60d8925c478a96d5083e47b6a branch: main author: Illia Volochii committer: markshannon date: 2023-07-28T14:39:54+01:00 summary: gh-102509: Start initializing `ob_digit` of `_PyLongValue` (GH-102510) files: A Misc/NEWS.d/next/Security/2023-03-07-21-46-29.gh-issue-102509.5ouaH_.rst M Objects/longobject.c diff --git a/Misc/NEWS.d/next/Security/2023-03-07-21-46-29.gh-issue-102509.5ouaH_.rst b/Misc/NEWS.d/next/Security/2023-03-07-21-46-29.gh-issue-102509.5ouaH_.rst new file mode 100644 index 0000000000000..d1a8e8b5a8d3c --- /dev/null +++ b/Misc/NEWS.d/next/Security/2023-03-07-21-46-29.gh-issue-102509.5ouaH_.rst @@ -0,0 +1,2 @@ +Start initializing ``ob_digit`` during creation of :c:type:`PyLongObject` +objects. Patch by Illia Volochii. diff --git a/Objects/longobject.c b/Objects/longobject.c index 5fca55e5c3a2b..5d9b413861478 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -163,6 +163,9 @@ _PyLong_New(Py_ssize_t size) } _PyLong_SetSignAndDigitCount(result, size != 0, size); _PyObject_Init((PyObject*)result, &PyLong_Type); + /* The digit has to be initialized explicitly to avoid + * use-of-uninitialized-value. */ + result->long_value.ob_digit[0] = 0; return result; } From webhook-mailer at python.org Fri Jul 28 10:53:56 2023 From: webhook-mailer at python.org (ericvsmith) Date: Fri, 28 Jul 2023 14:53:56 -0000 Subject: [Python-checkins] Fix typo in comment (gh-107389) Message-ID: https://github.com/python/cpython/commit/2aaa83d5f5c7b025f4bf2e04836139eb01a33bd8 commit: 2aaa83d5f5c7b025f4bf2e04836139eb01a33bd8 branch: main author: Tom Niget committer: ericvsmith date: 2023-07-28T10:52:29-04:00 summary: Fix typo in comment (gh-107389) files: M Lib/dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index e766a7b554afe..3c72c289e4388 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -1036,7 +1036,7 @@ def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen, # Was this class defined with an explicit __hash__? Note that if # __eq__ is defined in this class, then python will automatically # set __hash__ to None. This is a heuristic, as it's possible - # that such a __hash__ == None was not auto-generated, but it + # that such a __hash__ == None was not auto-generated, but it's # close enough. class_hash = cls.__dict__.get('__hash__', MISSING) has_explicit_hash = not (class_hash is MISSING or From webhook-mailer at python.org Fri Jul 28 12:07:37 2023 From: webhook-mailer at python.org (markshannon) Date: Fri, 28 Jul 2023 16:07:37 -0000 Subject: [Python-checkins] GH-101291: Add warning to "what's new" that `PyLongObject` internals have changed. (GH-107388) Message-ID: https://github.com/python/cpython/commit/1ee605c5888fbc3d51b3e7610bac38ea6bc25e31 commit: 1ee605c5888fbc3d51b3e7610bac38ea6bc25e31 branch: main author: Mark Shannon committer: markshannon date: 2023-07-28T17:07:33+01:00 summary: GH-101291: Add warning to "what's new" that `PyLongObject` internals have changed. (GH-107388) files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 68e5b0691f00e..99500e1f32146 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1865,6 +1865,17 @@ Porting to Python 3.12 subinterpreter that they don't support (or haven't yet been loaded in). See :gh:`104668` for more info. +* :c:struct:`PyLongObject` has had its internals changed for better performance. + Although the internals of :c:struct:`PyLongObject` are private, they are used + by some extension modules. + The internal fields should no longer be accessed directly, instead the API + functions beginning ``PyLong_...`` should be used instead. + Two new *unstable* API functions are provided for efficient access to the + value of :c:struct:`PyLongObject`\s which fit into a single machine word: + + * :c:func:`PyUnstable_Long_IsCompact` + * :c:func:`PyUnstable_Long_CompactValue` + Deprecated ---------- From webhook-mailer at python.org Fri Jul 28 12:31:35 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Fri, 28 Jul 2023 16:31:35 -0000 Subject: [Python-checkins] gh-104629: Build _testclinic extension module on Windows (#104723) Message-ID: https://github.com/python/cpython/commit/3a1d819ebc36189e086198212822c9b29384f242 commit: 3a1d819ebc36189e086198212822c9b29384f242 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-28T18:31:30+02:00 summary: gh-104629: Build _testclinic extension module on Windows (#104723) files: A PCbuild/_testclinic.vcxproj A PCbuild/_testclinic.vcxproj.filters M PCbuild/pcbuild.proj M PCbuild/pcbuild.sln M PCbuild/readme.txt M Tools/msi/test/test_files.wxs diff --git a/PCbuild/_testclinic.vcxproj b/PCbuild/_testclinic.vcxproj new file mode 100644 index 0000000000000..e319b3c0f42e0 --- /dev/null +++ b/PCbuild/_testclinic.vcxproj @@ -0,0 +1,110 @@ +? + + + + Debug + ARM + + + Debug + ARM64 + + + Debug + Win32 + + + Debug + x64 + + + PGInstrument + ARM + + + PGInstrument + ARM64 + + + PGInstrument + Win32 + + + PGInstrument + x64 + + + PGUpdate + ARM + + + PGUpdate + ARM64 + + + PGUpdate + Win32 + + + PGUpdate + x64 + + + Release + ARM + + + Release + ARM64 + + + Release + Win32 + + + Release + x64 + + + + {A840DDFB-ED50-484B-B527-B32E7CF90FD5} + _testclinic + Win32Proj + false + + + + + DynamicLibrary + NotSet + + + + .pyd + + + + + + + + + + <_ProjectFileVersion>10.0.30319.1 + + + + + + + + + + {cf7ac3d1-e2df-41d2-bea6-1e2556cdea26} + false + + + + + + \ No newline at end of file diff --git a/PCbuild/_testclinic.vcxproj.filters b/PCbuild/_testclinic.vcxproj.filters new file mode 100644 index 0000000000000..4a2987eb27b22 --- /dev/null +++ b/PCbuild/_testclinic.vcxproj.filters @@ -0,0 +1,21 @@ +? + + + + {5b0a9282-a01c-4b83-9fd4-6deb6c558f9c} + + + {6a89c8a9-5b51-4525-ac5c-7d0a22f9657e} + + + + + Source Files + + + + + Resource Files + + + \ No newline at end of file diff --git a/PCbuild/pcbuild.proj b/PCbuild/pcbuild.proj index 46d6961eea0ac..7735c22f8ae1b 100644 --- a/PCbuild/pcbuild.proj +++ b/PCbuild/pcbuild.proj @@ -77,7 +77,7 @@ - + diff --git a/PCbuild/pcbuild.sln b/PCbuild/pcbuild.sln index f8f1b83db97e8..ca70213755310 100644 --- a/PCbuild/pcbuild.sln +++ b/PCbuild/pcbuild.sln @@ -46,6 +46,7 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "python", "python.vcxproj", {EB6E69DD-04BF-4543-9B92-49FAABCEAC2E} = {EB6E69DD-04BF-4543-9B92-49FAABCEAC2E} {16BFE6F0-22EF-40B5-B831-7E937119EF10} = {16BFE6F0-22EF-40B5-B831-7E937119EF10} {FCBE1EF2-E0F0-40B1-88B5-00A35D378742} = {FCBE1EF2-E0F0-40B1-88B5-00A35D378742} + {A840DDFB-ED50-484B-B527-B32E7CF90FD5} = {A840DDFB-ED50-484B-B527-B32E7CF90FD5} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pythoncore", "pythoncore.vcxproj", "{CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26}" @@ -76,6 +77,8 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_ssl", "_ssl.vcxproj", "{C6 EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_testcapi", "_testcapi.vcxproj", "{6901D91C-6E48-4BB7-9FEC-700C8131DF1D}" EndProject +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_testclinic", "_testclinic.vcxproj", "{A840DDFB-ED50-484B-B527-B32E7CF90FD5}" +EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_testinternalcapi", "_testinternalcapi.vcxproj", "{900342D7-516A-4469-B1AD-59A66E49A25F}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_testimportmultiple", "_testimportmultiple.vcxproj", "{36D0C52C-DF4E-45D0-8BC7-E294C3ABC781}" @@ -590,6 +593,38 @@ Global {6901D91C-6E48-4BB7-9FEC-700C8131DF1D}.Release|Win32.Build.0 = Release|Win32 {6901D91C-6E48-4BB7-9FEC-700C8131DF1D}.Release|x64.ActiveCfg = Release|x64 {6901D91C-6E48-4BB7-9FEC-700C8131DF1D}.Release|x64.Build.0 = Release|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|ARM.ActiveCfg = Debug|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|ARM.Build.0 = Debug|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|ARM64.ActiveCfg = Debug|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|ARM64.Build.0 = Debug|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|Win32.ActiveCfg = Debug|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|Win32.Build.0 = Debug|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|x64.ActiveCfg = Debug|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|x64.Build.0 = Debug|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|ARM.ActiveCfg = PGInstrument|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|ARM.Build.0 = PGInstrument|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|ARM64.ActiveCfg = PGInstrument|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|ARM64.Build.0 = PGInstrument|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|Win32.ActiveCfg = PGInstrument|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|Win32.Build.0 = PGInstrument|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|x64.ActiveCfg = PGInstrument|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|x64.Build.0 = PGInstrument|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|ARM.ActiveCfg = PGUpdate|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|ARM.Build.0 = PGUpdate|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|ARM64.ActiveCfg = PGUpdate|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|ARM64.Build.0 = PGUpdate|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|Win32.ActiveCfg = PGUpdate|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|Win32.Build.0 = PGUpdate|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|x64.ActiveCfg = PGUpdate|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|x64.Build.0 = PGUpdate|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|ARM.ActiveCfg = Release|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|ARM.Build.0 = Release|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|ARM64.ActiveCfg = Release|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|ARM64.Build.0 = Release|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|Win32.ActiveCfg = Release|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|Win32.Build.0 = Release|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|x64.ActiveCfg = Release|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|x64.Build.0 = Release|x64 {900342D7-516A-4469-B1AD-59A66E49A25F}.Debug|ARM.ActiveCfg = Debug|ARM {900342D7-516A-4469-B1AD-59A66E49A25F}.Debug|ARM.Build.0 = Debug|ARM {900342D7-516A-4469-B1AD-59A66E49A25F}.Debug|ARM64.ActiveCfg = Debug|ARM64 diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt index f0de142f0573b..86ad3ab1a40d9 100644 --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -143,6 +143,7 @@ _overlapped _socket _testbuffer _testcapi +_testclinic _testconsole _testimportmultiple _testmultiphase diff --git a/Tools/msi/test/test_files.wxs b/Tools/msi/test/test_files.wxs index b5f68faef30e0..87e164cb6759f 100644 --- a/Tools/msi/test/test_files.wxs +++ b/Tools/msi/test/test_files.wxs @@ -1,6 +1,6 @@ - + From webhook-mailer at python.org Fri Jul 28 14:10:49 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Fri, 28 Jul 2023 18:10:49 -0000 Subject: [Python-checkins] gh-106368: Argument clinic: Fix minor bug in `state_modulename_name` (#107387) Message-ID: https://github.com/python/cpython/commit/ecc05e23a1f4086645cfb5362c4c1351f70a4eb1 commit: ecc05e23a1f4086645cfb5362c4c1351f70a4eb1 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-28T19:10:45+01:00 summary: gh-106368: Argument clinic: Fix minor bug in `state_modulename_name` (#107387) files: M Lib/test/test_clinic.py M Tools/clinic/clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 2f74ee29b204d..3ce27d1dd6b48 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -84,6 +84,7 @@ def __init__(self): ('parser_definition', d('block')), ('impl_definition', d('block')), )) + self.functions = [] def get_destination(self, name): d = self.destinations.get(name) @@ -104,6 +105,9 @@ def directive(self, name, args): _module_and_class = clinic.Clinic._module_and_class + def __repr__(self): + return "" + class ClinicWholeFileTest(_ParserBase): def setUp(self): @@ -672,6 +676,19 @@ def test_c_name(self): """) self.assertEqual("os_stat_fn", function.c_basename) + def test_cloning_nonexistent_function_correctly_fails(self): + stdout = self.parse_function_should_fail(""" + cloned = fooooooooooooooooooooooo + This is trying to clone a nonexistent function!! + """) + expected_error = """\ +cls=None, module=, existing='fooooooooooooooooooooooo' +(cls or module).functions=[] +Error on line 0: +Couldn't find existing function 'fooooooooooooooooooooooo'! +""" + self.assertEqual(expected_error, stdout) + def test_return_converter(self): function = self.parse_function(""" module os diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index a343dc5c7fc08..39c6de3527301 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4707,11 +4707,9 @@ def state_modulename_name(self, line: str | None) -> None: if existing_function.name == function_name: break else: - existing_function = None - if not existing_function: - print("class", cls, "module", module, "existing", existing) - print("cls. functions", cls.functions) - fail("Couldn't find existing function " + repr(existing) + "!") + print(f"{cls=}, {module=}, {existing=}") + print(f"{(cls or module).functions=}") + fail(f"Couldn't find existing function {existing!r}!") fields = [x.strip() for x in full_name.split('.')] function_name = fields.pop() From webhook-mailer at python.org Fri Jul 28 15:46:59 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Fri, 28 Jul 2023 19:46:59 -0000 Subject: [Python-checkins] gh-107305: Update the C-API Docs for PEP 684 (gh-107324) Message-ID: https://github.com/python/cpython/commit/c0b81c4b5438a3565fadd9d6f5bc69f989a3fdee commit: c0b81c4b5438a3565fadd9d6f5bc69f989a3fdee branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-28T13:46:55-06:00 summary: gh-107305: Update the C-API Docs for PEP 684 (gh-107324) files: A Misc/NEWS.d/next/Documentation/2023-07-26-16-33-04.gh-issue-107305.qB2LS4.rst M Doc/c-api/init.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index a447eefc55e44..60912a1ecaede 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1226,7 +1226,95 @@ You can switch between sub-interpreters using the :c:func:`PyThreadState_Swap` function. You can create and destroy them using the following functions: -.. c:function:: PyThreadState* Py_NewInterpreter() +.. c:type:: PyInterpreterConfig + + Structure containing most parameters to configure a sub-interpreter. + Its values are used only in :c:func:`Py_NewInterpreterFromConfig` and + never modified by the runtime. + + .. versionadded:: 3.12 + + Structure fields: + + .. c:member:: int use_main_obmalloc + + If this is ``0`` then the sub-interpreter will use its own + "object" allocator state. + Otherwise it will use (share) the main interpreter's. + + If this is ``0`` then + :c:member:`~PyInterpreterConfig.check_multi_interp_extensions` + must be ``1`` (non-zero). + If this is ``1`` then :c:member:`~PyInterpreterConfig.gil` + must not be :c:macro:`PyInterpreterConfig_OWN_GIL`. + + .. c:member:: int allow_fork + + If this is ``0`` then the runtime will not support forking the + process in any thread where the sub-interpreter is currently active. + Otherwise fork is unrestricted. + + Note that the :mod:`subprocess` module still works + when fork is disallowed. + + .. c:member:: int allow_exec + + If this is ``0`` then the runtime will not support replacing the + current process via exec (e.g. :func:`os.execv`) in any thread + where the sub-interpreter is currently active. + Otherwise exec is unrestricted. + + Note that the :mod:`subprocess` module still works + when exec is disallowed. + + .. c:member:: int allow_threads + + If this is ``0`` then the sub-interpreter's :mod:`threading` module + won't create threads. + Otherwise threads are allowed. + + .. c:member:: int allow_daemon_threads + + If this is ``0`` then the sub-interpreter's :mod:`threading` module + won't create daemon threads. + Otherwise daemon threads are allowed (as long as + :c:member:`~PyInterpreterConfig.allow_threads` is non-zero). + + .. c:member:: int check_multi_interp_extensions + + If this is ``0`` then all extension modules may be imported, + including legacy (single-phase init) modules, + in any thread where the sub-interpreter is currently active. + Otherwise only multi-phase init extension modules + (see :pep:`489`) may be imported. + + This must be ``1`` (non-zero) if + :c:member:`~PyInterpreterConfig.use_main_obmalloc` is ``0``. + + .. c:member:: int gil + + This determines the operation of the GIL for the sub-interpreter. + It may be one of the following: + + .. c:namespace:: NULL + + .. c:macro:: PyInterpreterConfig_DEFAULT_GIL + + Use the default selection (:c:macro:`PyInterpreterConfig_SHARED_GIL`). + + .. c:macro:: PyInterpreterConfig_SHARED_GIL + + Use (share) the main interpreter's GIL. + + .. c:macro:: PyInterpreterConfig_OWN_GIL + + Use the sub-interpreter's own GIL. + + If this is :c:macro:`PyInterpreterConfig_OWN_GIL` then + :c:member:`PyInterpreterConfig.use_main_obmalloc` must be ``0``. + + +.. c:function:: PyStatus Py_NewInterpreterFromConfig(PyThreadState **tstate_p, const PyInterpreterConfig *config) .. index:: pair: module; builtins @@ -1246,16 +1334,47 @@ function. You can create and destroy them using the following functions: ``sys.stdout`` and ``sys.stderr`` (however these refer to the same underlying file descriptors). - The return value points to the first thread state created in the new + The given *config* controls the options with which the interpreter + is initialized. + + Upon success, *tstate_p* will be set to the first thread state + created in the new sub-interpreter. This thread state is made in the current thread state. Note that no actual thread is created; see the discussion of thread states - below. If creation of the new interpreter is unsuccessful, ``NULL`` is - returned; no exception is set since the exception state is stored in the - current thread state and there may not be a current thread state. (Like all - other Python/C API functions, the global interpreter lock must be held before - calling this function and is still held when it returns; however, unlike most - other Python/C API functions, there needn't be a current thread state on - entry.) + below. If creation of the new interpreter is unsuccessful, + *tstate_p* is set to ``NULL``; + no exception is set since the exception state is stored in the + current thread state and there may not be a current thread state. + + Like all other Python/C API functions, the global interpreter lock + must be held before calling this function and is still held when it + returns. Likewise a current thread state must be set on entry. On + success, the returned thread state will be set as current. If the + sub-interpreter is created with its own GIL then the GIL of the + calling interpreter will be released. When the function returns, + the new interpreter's GIL will be held by the current thread and + the previously interpreter's GIL will remain released here. + + .. versionadded:: 3.12 + + Sub-interpreters are most effective when isolated from each other, + with certain functionality restricted:: + + PyInterpreterConfig config = { + .use_main_obmalloc = 0, + .allow_fork = 0, + .allow_exec = 0, + .allow_threads = 1, + .allow_daemon_threads = 0, + .check_multi_interp_extensions = 1, + .gil = PyInterpreterConfig_OWN_GIL, + }; + PyThreadState *tstate = Py_NewInterpreterFromConfig(&config); + + Note that the config is used only briefly and does not get modified. + During initialization the config's values are converted into various + :c:type:`PyInterpreterState` values. A read-only copy of the config + may be stored internally on the :c:type:`PyInterpreterState`. .. index:: single: Py_FinalizeEx() @@ -1290,19 +1409,79 @@ function. You can create and destroy them using the following functions: .. index:: single: close() (in module os) +.. c:function:: PyThreadState* Py_NewInterpreter(void) + + .. index:: + pair: module; builtins + pair: module; __main__ + pair: module; sys + single: stdout (in module sys) + single: stderr (in module sys) + single: stdin (in module sys) + + Create a new sub-interpreter. This is essentially just a wrapper + around :c:func:`Py_NewInterpreterFromConfig` with a config that + preserves the existing behavior. The result is an unisolated + sub-interpreter that shares the main interpreter's GIL, allows + fork/exec, allows daemon threads, and allows single-phase init + modules. + + .. c:function:: void Py_EndInterpreter(PyThreadState *tstate) .. index:: single: Py_FinalizeEx() - Destroy the (sub-)interpreter represented by the given thread state. The given - thread state must be the current thread state. See the discussion of thread - states below. When the call returns, the current thread state is ``NULL``. All - thread states associated with this interpreter are destroyed. (The global - interpreter lock must be held before calling this function and is still held - when it returns.) :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that + Destroy the (sub-)interpreter represented by the given thread state. + The given thread state must be the current thread state. See the + discussion of thread states below. When the call returns, + the current thread state is ``NULL``. All thread states associated + with this interpreter are destroyed. The global interpreter lock + used by the target interpreter must be held before calling this + function. No GIL is held when it returns. + + :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that haven't been explicitly destroyed at that point. +A Per-Interpreter GIL +--------------------- + +Using :c:func:`Py_NewInterpreterFromConfig` you can create +a sub-interpreter that is completely isolated from other interpreters, +including having its own GIL. The most important benefit of this +isolation is that such an interpreter can execute Python code without +being blocked by other interpreters or blocking any others. Thus a +single Python process can truly take advantage of multiple CPU cores +when running Python code. The isolation also encourages a different +approach to concurrency than that of just using threads. +(See :pep:`554`.) + +Using an isolated interpreter requires vigilance in preserving that +isolation. That especially means not sharing any objects or mutable +state without guarantees about thread-safety. Even objects that are +otherwise immutable (e.g. ``None``, ``(1, 5)``) can't normally be shared +because of the refcount. One simple but less-efficient approach around +this is to use a global lock around all use of some state (or object). +Alternately, effectively immutable objects (like integers or strings) +can be made safe in spite of their refcounts by making them "immortal". +In fact, this has been done for the builtin singletons, small integers, +and a number of other builtin objects. + +If you preserve isolation then you will have access to proper multi-core +computing without the complications that come with free-threading. +Failure to preserve isolation will expose you to the full consequences +of free-threading, including races and hard-to-debug crashes. + +Aside from that, one of the main challenges of using multiple isolated +interpreters is how to communicate between them safely (not break +isolation) and efficiently. The runtime and stdlib do not provide +any standard approach to this yet. A future stdlib module would help +mitigate the effort of preserving isolation and expose effective tools +for communicating (and sharing) data between interpreters. + +.. versionadded:: 3.12 + + Bugs and caveats ---------------- diff --git a/Misc/NEWS.d/next/Documentation/2023-07-26-16-33-04.gh-issue-107305.qB2LS4.rst b/Misc/NEWS.d/next/Documentation/2023-07-26-16-33-04.gh-issue-107305.qB2LS4.rst new file mode 100644 index 0000000000000..038f9e68a5422 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-07-26-16-33-04.gh-issue-107305.qB2LS4.rst @@ -0,0 +1,3 @@ +Add documentation for :c:type:`PyInterpreterConfig` and +:c:func:`Py_NewInterpreterFromConfig`. Also clarify some of the nearby docs +relative to per-interpreter GIL. From webhook-mailer at python.org Fri Jul 28 15:55:01 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Fri, 28 Jul 2023 19:55:01 -0000 Subject: [Python-checkins] [3.12] gh-107305: Update the C-API Docs for PEP 684 (gh-107324) (gh-107402) Message-ID: https://github.com/python/cpython/commit/32502da987549a6bfe52c80a21244ab18f663e2b commit: 32502da987549a6bfe52c80a21244ab18f663e2b branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ericsnowcurrently date: 2023-07-28T19:54:57Z summary: [3.12] gh-107305: Update the C-API Docs for PEP 684 (gh-107324) (gh-107402) gh-107305: Update the C-API Docs for PEP 684 (gh-107324) (cherry picked from commit c0b81c4b5438a3565fadd9d6f5bc69f989a3fdee) Co-authored-by: Eric Snow files: A Misc/NEWS.d/next/Documentation/2023-07-26-16-33-04.gh-issue-107305.qB2LS4.rst M Doc/c-api/init.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 8a2543affc4d1..52824b6d88afd 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1494,7 +1494,95 @@ You can switch between sub-interpreters using the :c:func:`PyThreadState_Swap` function. You can create and destroy them using the following functions: -.. c:function:: PyThreadState* Py_NewInterpreter() +.. c:type:: PyInterpreterConfig + + Structure containing most parameters to configure a sub-interpreter. + Its values are used only in :c:func:`Py_NewInterpreterFromConfig` and + never modified by the runtime. + + .. versionadded:: 3.12 + + Structure fields: + + .. c:member:: int use_main_obmalloc + + If this is ``0`` then the sub-interpreter will use its own + "object" allocator state. + Otherwise it will use (share) the main interpreter's. + + If this is ``0`` then + :c:member:`~PyInterpreterConfig.check_multi_interp_extensions` + must be ``1`` (non-zero). + If this is ``1`` then :c:member:`~PyInterpreterConfig.gil` + must not be :c:macro:`PyInterpreterConfig_OWN_GIL`. + + .. c:member:: int allow_fork + + If this is ``0`` then the runtime will not support forking the + process in any thread where the sub-interpreter is currently active. + Otherwise fork is unrestricted. + + Note that the :mod:`subprocess` module still works + when fork is disallowed. + + .. c:member:: int allow_exec + + If this is ``0`` then the runtime will not support replacing the + current process via exec (e.g. :func:`os.execv`) in any thread + where the sub-interpreter is currently active. + Otherwise exec is unrestricted. + + Note that the :mod:`subprocess` module still works + when exec is disallowed. + + .. c:member:: int allow_threads + + If this is ``0`` then the sub-interpreter's :mod:`threading` module + won't create threads. + Otherwise threads are allowed. + + .. c:member:: int allow_daemon_threads + + If this is ``0`` then the sub-interpreter's :mod:`threading` module + won't create daemon threads. + Otherwise daemon threads are allowed (as long as + :c:member:`~PyInterpreterConfig.allow_threads` is non-zero). + + .. c:member:: int check_multi_interp_extensions + + If this is ``0`` then all extension modules may be imported, + including legacy (single-phase init) modules, + in any thread where the sub-interpreter is currently active. + Otherwise only multi-phase init extension modules + (see :pep:`489`) may be imported. + + This must be ``1`` (non-zero) if + :c:member:`~PyInterpreterConfig.use_main_obmalloc` is ``0``. + + .. c:member:: int gil + + This determines the operation of the GIL for the sub-interpreter. + It may be one of the following: + + .. c:namespace:: NULL + + .. c:macro:: PyInterpreterConfig_DEFAULT_GIL + + Use the default selection (:c:macro:`PyInterpreterConfig_SHARED_GIL`). + + .. c:macro:: PyInterpreterConfig_SHARED_GIL + + Use (share) the main interpreter's GIL. + + .. c:macro:: PyInterpreterConfig_OWN_GIL + + Use the sub-interpreter's own GIL. + + If this is :c:macro:`PyInterpreterConfig_OWN_GIL` then + :c:member:`PyInterpreterConfig.use_main_obmalloc` must be ``0``. + + +.. c:function:: PyStatus Py_NewInterpreterFromConfig(PyThreadState **tstate_p, const PyInterpreterConfig *config) .. index:: pair: module; builtins @@ -1514,16 +1602,47 @@ function. You can create and destroy them using the following functions: ``sys.stdout`` and ``sys.stderr`` (however these refer to the same underlying file descriptors). - The return value points to the first thread state created in the new + The given *config* controls the options with which the interpreter + is initialized. + + Upon success, *tstate_p* will be set to the first thread state + created in the new sub-interpreter. This thread state is made in the current thread state. Note that no actual thread is created; see the discussion of thread states - below. If creation of the new interpreter is unsuccessful, ``NULL`` is - returned; no exception is set since the exception state is stored in the - current thread state and there may not be a current thread state. (Like all - other Python/C API functions, the global interpreter lock must be held before - calling this function and is still held when it returns; however, unlike most - other Python/C API functions, there needn't be a current thread state on - entry.) + below. If creation of the new interpreter is unsuccessful, + *tstate_p* is set to ``NULL``; + no exception is set since the exception state is stored in the + current thread state and there may not be a current thread state. + + Like all other Python/C API functions, the global interpreter lock + must be held before calling this function and is still held when it + returns. Likewise a current thread state must be set on entry. On + success, the returned thread state will be set as current. If the + sub-interpreter is created with its own GIL then the GIL of the + calling interpreter will be released. When the function returns, + the new interpreter's GIL will be held by the current thread and + the previously interpreter's GIL will remain released here. + + .. versionadded:: 3.12 + + Sub-interpreters are most effective when isolated from each other, + with certain functionality restricted:: + + PyInterpreterConfig config = { + .use_main_obmalloc = 0, + .allow_fork = 0, + .allow_exec = 0, + .allow_threads = 1, + .allow_daemon_threads = 0, + .check_multi_interp_extensions = 1, + .gil = PyInterpreterConfig_OWN_GIL, + }; + PyThreadState *tstate = Py_NewInterpreterFromConfig(&config); + + Note that the config is used only briefly and does not get modified. + During initialization the config's values are converted into various + :c:type:`PyInterpreterState` values. A read-only copy of the config + may be stored internally on the :c:type:`PyInterpreterState`. .. index:: single: Py_FinalizeEx() @@ -1558,19 +1677,79 @@ function. You can create and destroy them using the following functions: .. index:: single: close() (in module os) +.. c:function:: PyThreadState* Py_NewInterpreter(void) + + .. index:: + pair: module; builtins + pair: module; __main__ + pair: module; sys + single: stdout (in module sys) + single: stderr (in module sys) + single: stdin (in module sys) + + Create a new sub-interpreter. This is essentially just a wrapper + around :c:func:`Py_NewInterpreterFromConfig` with a config that + preserves the existing behavior. The result is an unisolated + sub-interpreter that shares the main interpreter's GIL, allows + fork/exec, allows daemon threads, and allows single-phase init + modules. + + .. c:function:: void Py_EndInterpreter(PyThreadState *tstate) .. index:: single: Py_FinalizeEx() - Destroy the (sub-)interpreter represented by the given thread state. The given - thread state must be the current thread state. See the discussion of thread - states below. When the call returns, the current thread state is ``NULL``. All - thread states associated with this interpreter are destroyed. (The global - interpreter lock must be held before calling this function and is still held - when it returns.) :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that + Destroy the (sub-)interpreter represented by the given thread state. + The given thread state must be the current thread state. See the + discussion of thread states below. When the call returns, + the current thread state is ``NULL``. All thread states associated + with this interpreter are destroyed. The global interpreter lock + used by the target interpreter must be held before calling this + function. No GIL is held when it returns. + + :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that haven't been explicitly destroyed at that point. +A Per-Interpreter GIL +--------------------- + +Using :c:func:`Py_NewInterpreterFromConfig` you can create +a sub-interpreter that is completely isolated from other interpreters, +including having its own GIL. The most important benefit of this +isolation is that such an interpreter can execute Python code without +being blocked by other interpreters or blocking any others. Thus a +single Python process can truly take advantage of multiple CPU cores +when running Python code. The isolation also encourages a different +approach to concurrency than that of just using threads. +(See :pep:`554`.) + +Using an isolated interpreter requires vigilance in preserving that +isolation. That especially means not sharing any objects or mutable +state without guarantees about thread-safety. Even objects that are +otherwise immutable (e.g. ``None``, ``(1, 5)``) can't normally be shared +because of the refcount. One simple but less-efficient approach around +this is to use a global lock around all use of some state (or object). +Alternately, effectively immutable objects (like integers or strings) +can be made safe in spite of their refcounts by making them "immortal". +In fact, this has been done for the builtin singletons, small integers, +and a number of other builtin objects. + +If you preserve isolation then you will have access to proper multi-core +computing without the complications that come with free-threading. +Failure to preserve isolation will expose you to the full consequences +of free-threading, including races and hard-to-debug crashes. + +Aside from that, one of the main challenges of using multiple isolated +interpreters is how to communicate between them safely (not break +isolation) and efficiently. The runtime and stdlib do not provide +any standard approach to this yet. A future stdlib module would help +mitigate the effort of preserving isolation and expose effective tools +for communicating (and sharing) data between interpreters. + +.. versionadded:: 3.12 + + Bugs and caveats ---------------- diff --git a/Misc/NEWS.d/next/Documentation/2023-07-26-16-33-04.gh-issue-107305.qB2LS4.rst b/Misc/NEWS.d/next/Documentation/2023-07-26-16-33-04.gh-issue-107305.qB2LS4.rst new file mode 100644 index 0000000000000..038f9e68a5422 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2023-07-26-16-33-04.gh-issue-107305.qB2LS4.rst @@ -0,0 +1,3 @@ +Add documentation for :c:type:`PyInterpreterConfig` and +:c:func:`Py_NewInterpreterFromConfig`. Also clarify some of the nearby docs +relative to per-interpreter GIL. From webhook-mailer at python.org Fri Jul 28 16:27:25 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Fri, 28 Jul 2023 20:27:25 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: complete type annotations (#107399) Message-ID: https://github.com/python/cpython/commit/55ed85e49c16b1dc5470a8a893869261f6356c99 commit: 55ed85e49c16b1dc5470a8a893869261f6356c99 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-28T21:27:21+01:00 summary: gh-104050: Argument clinic: complete type annotations (#107399) files: M Tools/clinic/clinic.py M Tools/clinic/mypy.ini diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 39c6de3527301..6bfa6d0d56814 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2290,8 +2290,9 @@ def parse(self, input: str) -> str: return printer.f.getvalue() - - def _module_and_class(self, fields): + def _module_and_class( + self, fields: Iterable[str] + ) -> tuple[Module | Clinic, Class | None]: """ fields should be an iterable of field names. returns a tuple of (module, class). @@ -2299,19 +2300,21 @@ def _module_and_class(self, fields): this function is only ever used to find the parent of where a new class/module should go. """ - in_classes = False + parent: Clinic | Module | Class + child: Module | Class | None + module: Clinic | Module + cls: Class | None = None + so_far: list[str] = [] + parent = module = self - cls = None - so_far = [] for field in fields: so_far.append(field) - if not in_classes: + if not isinstance(parent, Class): child = parent.modules.get(field) if child: parent = module = child continue - in_classes = True if not hasattr(parent, 'classes'): return module, cls child = parent.classes.get(field) @@ -2379,7 +2382,7 @@ def parse(self, block: Block) -> None: @dc.dataclass(repr=False) class Module: name: str - module: Module | None = None + module: Module | Clinic def __post_init__(self) -> None: self.parent = self.module @@ -2394,7 +2397,7 @@ def __repr__(self) -> str: @dc.dataclass(repr=False) class Class: name: str - module: Module + module: Module | Clinic cls: Class | None typedef: str type_object: str @@ -2522,7 +2525,7 @@ class Function: parameters: ParamDict = dc.field(default_factory=dict) _: dc.KW_ONLY name: str - module: Module + module: Module | Clinic cls: Class | None c_basename: str | None full_name: str @@ -2538,7 +2541,7 @@ class Function: docstring_only: bool = False def __post_init__(self) -> None: - self.parent: Class | Module = self.cls or self.module + self.parent = self.cls or self.module self.self_converter: self_converter | None = None self.__render_parameters__: list[Parameter] | None = None diff --git a/Tools/clinic/mypy.ini b/Tools/clinic/mypy.ini index 672911dc19abd..4aa65a5c14489 100644 --- a/Tools/clinic/mypy.ini +++ b/Tools/clinic/mypy.ini @@ -1,12 +1,15 @@ [mypy] +files = Tools/clinic/ +pretty = True + # make sure clinic can still be run on Python 3.10 python_version = 3.10 -pretty = True -enable_error_code = ignore-without-code -disallow_any_generics = True + +# be strict... +strict = True strict_concatenate = True -warn_redundant_casts = True -warn_unused_ignores = True -warn_unused_configs = True +enable_error_code = ignore-without-code,redundant-expr warn_unreachable = True -files = Tools/clinic/ + +# ...except for one extra rule we can't enable just yet +warn_return_any = False From webhook-mailer at python.org Fri Jul 28 16:39:12 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Fri, 28 Jul 2023 20:39:12 -0000 Subject: [Python-checkins] gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974) Message-ID: https://github.com/python/cpython/commit/8ba4df91ae60833723d8d3b9afeb2b642f7176d5 commit: 8ba4df91ae60833723d8d3b9afeb2b642f7176d5 branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-28T14:39:08-06:00 summary: gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974) This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules. The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL. It turns out it's tricky to use the same thread state for different threads. Who could have guessed? We solve the problem by eliminating the one object we were still sharing between interpreters. We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter. We also remove the accommodations for "detached" thread states, which were a dubious idea to start with. files: M Include/internal/pycore_import.h M Include/internal/pycore_pystate.h M Include/internal/pycore_runtime_init.h M Python/import.c M Python/pystate.c diff --git a/Include/internal/pycore_import.h b/Include/internal/pycore_import.h index f61f0af6bb263..077508e6c58f6 100644 --- a/Include/internal/pycore_import.h +++ b/Include/internal/pycore_import.h @@ -5,6 +5,7 @@ extern "C" { #endif +#include "pycore_hashtable.h" // _Py_hashtable_t #include "pycore_time.h" // _PyTime_t extern int _PyImport_IsInitialized(PyInterpreterState *); @@ -36,19 +37,15 @@ struct _import_runtime_state { See PyInterpreterState.modules_by_index for more info. */ Py_ssize_t last_module_index; struct { - /* A thread state tied to the main interpreter, - used exclusively for when the extensions dict is access/modified - from an arbitrary thread. */ - PyThreadState main_tstate; - /* A lock to guard the dict. */ + /* A lock to guard the cache. */ PyThread_type_lock mutex; - /* A dict mapping (filename, name) to PyModuleDef for modules. + /* The actual cache of (filename, name, PyModuleDef) for modules. Only legacy (single-phase init) extension modules are added and only if they support multiple initialization (m_size >- 0) or are imported in the main interpreter. This is initialized lazily in _PyImport_FixupExtensionObject(). Modules are added there and looked up in _imp.find_extension(). */ - PyObject *dict; + _Py_hashtable_t *hashtable; } extensions; /* Package context -- the full module name for package imports */ const char * pkgcontext; diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h index f0db757c77c58..acc6cf953343f 100644 --- a/Include/internal/pycore_pystate.h +++ b/Include/internal/pycore_pystate.h @@ -125,11 +125,6 @@ extern PyThreadState * _PyThreadState_New(PyInterpreterState *interp); extern void _PyThreadState_Bind(PyThreadState *tstate); extern void _PyThreadState_DeleteExcept(PyThreadState *tstate); -extern void _PyThreadState_InitDetached(PyThreadState *, PyInterpreterState *); -extern void _PyThreadState_ClearDetached(PyThreadState *); -extern void _PyThreadState_BindDetached(PyThreadState *); -extern void _PyThreadState_UnbindDetached(PyThreadState *); - // Export for '_testinternalcapi' shared extension PyAPI_FUNC(PyObject*) _PyThreadState_GetDict(PyThreadState *tstate); diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h index 840e41c59ca73..dcfceef4f175a 100644 --- a/Include/internal/pycore_runtime_init.h +++ b/Include/internal/pycore_runtime_init.h @@ -97,11 +97,6 @@ extern PyTypeObject _PyExc_MemoryError; in accordance with the specification. */ \ .autoTSSkey = Py_tss_NEEDS_INIT, \ .parser = _parser_runtime_state_INIT, \ - .imports = { \ - .extensions = { \ - .main_tstate = _PyThreadState_INIT, \ - }, \ - }, \ .ceval = { \ .perf = _PyEval_RUNTIME_PERF_INIT, \ }, \ diff --git a/Python/import.c b/Python/import.c index 711ed5d32ca97..e381d1fd7d433 100644 --- a/Python/import.c +++ b/Python/import.c @@ -2,10 +2,12 @@ #include "Python.h" +#include "pycore_hashtable.h" // _Py_hashtable_new_full() #include "pycore_import.h" // _PyImport_BootstrapImp() #include "pycore_initconfig.h" // _PyStatus_OK() #include "pycore_interp.h" // struct _import_runtime_state #include "pycore_namespace.h" // _PyNamespace_Type +#include "pycore_object.h" // _Py_SetImmortal() #include "pycore_pyerrors.h" // _PyErr_SetString() #include "pycore_pyhash.h" // _Py_KeyedHash() #include "pycore_pylifecycle.h" @@ -905,35 +907,79 @@ extensions_lock_release(void) dictionary, to avoid loading shared libraries twice. */ +static void * +hashtable_key_from_2_strings(PyObject *str1, PyObject *str2, const char sep) +{ + Py_ssize_t str1_len, str2_len; + const char *str1_data = PyUnicode_AsUTF8AndSize(str1, &str1_len); + const char *str2_data = PyUnicode_AsUTF8AndSize(str2, &str2_len); + if (str1_data == NULL || str2_data == NULL) { + return NULL; + } + /* Make sure sep and the NULL byte won't cause an overflow. */ + assert(SIZE_MAX - str1_len - str2_len > 2); + size_t size = str1_len + 1 + str2_len + 1; + + char *key = PyMem_RawMalloc(size); + if (key == NULL) { + PyErr_NoMemory(); + return NULL; + } + + strncpy(key, str1_data, str1_len); + key[str1_len] = sep; + strncpy(key + str1_len + 1, str2_data, str2_len + 1); + assert(strlen(key) == size - 1); + return key; +} + +static Py_uhash_t +hashtable_hash_str(const void *key) +{ + return _Py_HashBytes(key, strlen((const char *)key)); +} + +static int +hashtable_compare_str(const void *key1, const void *key2) +{ + return strcmp((const char *)key1, (const char *)key2) == 0; +} + static void -_extensions_cache_init(void) +hashtable_destroy_str(void *ptr) { - /* The runtime (i.e. main interpreter) must be initializing, - so we don't need to worry about the lock. */ - _PyThreadState_InitDetached(&EXTENSIONS.main_tstate, - _PyInterpreterState_Main()); + PyMem_RawFree(ptr); } +#define HTSEP ':' + static PyModuleDef * _extensions_cache_get(PyObject *filename, PyObject *name) { PyModuleDef *def = NULL; + void *key = NULL; extensions_lock_acquire(); - PyObject *key = PyTuple_Pack(2, filename, name); - if (key == NULL) { + if (EXTENSIONS.hashtable == NULL) { goto finally; } - PyObject *extensions = EXTENSIONS.dict; - if (extensions == NULL) { + key = hashtable_key_from_2_strings(filename, name, HTSEP); + if (key == NULL) { + goto finally; + } + _Py_hashtable_entry_t *entry = _Py_hashtable_get_entry( + EXTENSIONS.hashtable, key); + if (entry == NULL) { goto finally; } - def = (PyModuleDef *)PyDict_GetItemWithError(extensions, key); + def = (PyModuleDef *)entry->value; finally: - Py_XDECREF(key); extensions_lock_release(); + if (key != NULL) { + PyMem_RawFree(key); + } return def; } @@ -941,124 +987,99 @@ static int _extensions_cache_set(PyObject *filename, PyObject *name, PyModuleDef *def) { int res = -1; - PyThreadState *oldts = NULL; extensions_lock_acquire(); - /* Swap to the main interpreter, if necessary. This matters if - the dict hasn't been created yet or if the item isn't in the - dict yet. In both cases we must ensure the relevant objects - are created using the main interpreter. */ - PyThreadState *main_tstate = &EXTENSIONS.main_tstate; - PyInterpreterState *interp = _PyInterpreterState_GET(); - if (!_Py_IsMainInterpreter(interp)) { - _PyThreadState_BindDetached(main_tstate); - oldts = _PyThreadState_Swap(interp->runtime, main_tstate); - assert(!_Py_IsMainInterpreter(oldts->interp)); - - /* Make sure the name and filename objects are owned - by the main interpreter. */ - name = PyUnicode_InternFromString(PyUnicode_AsUTF8(name)); - assert(name != NULL); - filename = PyUnicode_InternFromString(PyUnicode_AsUTF8(filename)); - assert(filename != NULL); + if (EXTENSIONS.hashtable == NULL) { + _Py_hashtable_allocator_t alloc = {PyMem_RawMalloc, PyMem_RawFree}; + EXTENSIONS.hashtable = _Py_hashtable_new_full( + hashtable_hash_str, + hashtable_compare_str, + hashtable_destroy_str, // key + /* There's no need to decref the def since it's immortal. */ + NULL, // value + &alloc + ); + if (EXTENSIONS.hashtable == NULL) { + PyErr_NoMemory(); + goto finally; + } } - PyObject *key = PyTuple_Pack(2, filename, name); + void *key = hashtable_key_from_2_strings(filename, name, HTSEP); if (key == NULL) { goto finally; } - PyObject *extensions = EXTENSIONS.dict; - if (extensions == NULL) { - extensions = PyDict_New(); - if (extensions == NULL) { + int already_set = 0; + _Py_hashtable_entry_t *entry = _Py_hashtable_get_entry( + EXTENSIONS.hashtable, key); + if (entry == NULL) { + if (_Py_hashtable_set(EXTENSIONS.hashtable, key, def) < 0) { + PyMem_RawFree(key); + PyErr_NoMemory(); goto finally; } - EXTENSIONS.dict = extensions; - } - - PyModuleDef *actual = (PyModuleDef *)PyDict_GetItemWithError(extensions, key); - if (PyErr_Occurred()) { - goto finally; } - else if (actual != NULL) { - /* We expect it to be static, so it must be the same pointer. */ - assert(def == actual); - res = 0; - goto finally; + else { + if (entry->value == NULL) { + entry->value = def; + } + else { + /* We expect it to be static, so it must be the same pointer. */ + assert((PyModuleDef *)entry->value == def); + already_set = 1; + } + PyMem_RawFree(key); } - - /* This might trigger a resize, which is why we must switch - to the main interpreter. */ - res = PyDict_SetItem(extensions, key, (PyObject *)def); - if (res < 0) { - res = -1; - goto finally; + if (!already_set) { + /* We assume that all module defs are statically allocated + and will never be freed. Otherwise, we would incref here. */ + _Py_SetImmortal(def); } res = 0; finally: - Py_XDECREF(key); - if (oldts != NULL) { - _PyThreadState_Swap(interp->runtime, oldts); - _PyThreadState_UnbindDetached(main_tstate); - Py_DECREF(name); - Py_DECREF(filename); - } extensions_lock_release(); return res; } -static int +static void _extensions_cache_delete(PyObject *filename, PyObject *name) { - int res = -1; - PyThreadState *oldts = NULL; + void *key = NULL; extensions_lock_acquire(); - PyObject *key = PyTuple_Pack(2, filename, name); - if (key == NULL) { + if (EXTENSIONS.hashtable == NULL) { + /* It was never added. */ goto finally; } - PyObject *extensions = EXTENSIONS.dict; - if (extensions == NULL) { - res = 0; + key = hashtable_key_from_2_strings(filename, name, HTSEP); + if (key == NULL) { goto finally; } - PyModuleDef *actual = (PyModuleDef *)PyDict_GetItemWithError(extensions, key); - if (PyErr_Occurred()) { + _Py_hashtable_entry_t *entry = _Py_hashtable_get_entry( + EXTENSIONS.hashtable, key); + if (entry == NULL) { + /* It was never added. */ goto finally; } - else if (actual == NULL) { - /* It was already removed or never added. */ - res = 0; + if (entry->value == NULL) { + /* It was already removed. */ goto finally; } - - /* Swap to the main interpreter, if necessary. */ - PyThreadState *main_tstate = &EXTENSIONS.main_tstate; - PyInterpreterState *interp = _PyInterpreterState_GET(); - if (!_Py_IsMainInterpreter(interp)) { - _PyThreadState_BindDetached(main_tstate); - oldts = _PyThreadState_Swap(interp->runtime, main_tstate); - assert(!_Py_IsMainInterpreter(oldts->interp)); - } - - if (PyDict_DelItem(extensions, key) < 0) { - goto finally; - } - res = 0; + /* If we hadn't made the stored defs immortal, we would decref here. + However, this decref would be problematic if the module def were + dynamically allocated, it were the last ref, and this function + were called with an interpreter other than the def's owner. */ + entry->value = NULL; finally: - if (oldts != NULL) { - _PyThreadState_Swap(interp->runtime, oldts); - _PyThreadState_UnbindDetached(main_tstate); - } - Py_XDECREF(key); extensions_lock_release(); - return res; + if (key != NULL) { + PyMem_RawFree(key); + } } static void @@ -1066,11 +1087,12 @@ _extensions_cache_clear_all(void) { /* The runtime (i.e. main interpreter) must be finalizing, so we don't need to worry about the lock. */ - // XXX assert(_Py_IsMainInterpreter(_PyInterpreterState_GET())); - Py_CLEAR(EXTENSIONS.dict); - _PyThreadState_ClearDetached(&EXTENSIONS.main_tstate); + _Py_hashtable_destroy(EXTENSIONS.hashtable); + EXTENSIONS.hashtable = NULL; } +#undef HTSEP + static bool check_multi_interp_extensions(PyInterpreterState *interp) @@ -1231,6 +1253,8 @@ import_find_extension(PyThreadState *tstate, PyObject *name, PyObject *m_copy = def->m_base.m_copy; /* Module does not support repeated initialization */ if (m_copy == NULL) { + /* It might be a core module (e.g. sys & builtins), + for which we don't set m_copy. */ m_copy = get_core_module_dict(tstate->interp, name, filename); if (m_copy == NULL) { return NULL; @@ -1300,9 +1324,7 @@ clear_singlephase_extension(PyInterpreterState *interp, } /* Clear the cached module def. */ - if (_extensions_cache_delete(filename, name) < 0) { - return -1; - } + _extensions_cache_delete(filename, name); return 0; } @@ -3051,6 +3073,8 @@ void _PyImport_Fini(void) { /* Destroy the database used by _PyImport_{Fixup,Find}Extension */ + // XXX Should we actually leave them (mostly) intact, since we don't + // ever dlclose() the module files? _extensions_cache_clear_all(); /* Use the same memory allocator as _PyImport_Init(). */ @@ -3088,10 +3112,6 @@ _PyImport_Fini2(void) PyStatus _PyImport_InitCore(PyThreadState *tstate, PyObject *sysmod, int importlib) { - if (_Py_IsMainInterpreter(tstate->interp)) { - _extensions_cache_init(); - } - // XXX Initialize here: interp->modules and interp->import_func. // XXX Initialize here: sys.modules and sys.meta_path. diff --git a/Python/pystate.c b/Python/pystate.c index 3ec131b6cba99..a1864cc3249e2 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -1640,75 +1640,6 @@ _PyThreadState_DeleteExcept(PyThreadState *tstate) } -//------------------------- -// "detached" thread states -//------------------------- - -void -_PyThreadState_InitDetached(PyThreadState *tstate, PyInterpreterState *interp) -{ - _PyRuntimeState *runtime = interp->runtime; - - HEAD_LOCK(runtime); - interp->threads.next_unique_id += 1; - uint64_t id = interp->threads.next_unique_id; - HEAD_UNLOCK(runtime); - - init_threadstate(tstate, interp, id); - // We do not call add_threadstate(). -} - -void -_PyThreadState_ClearDetached(PyThreadState *tstate) -{ - assert(!tstate->_status.bound); - assert(!tstate->_status.bound_gilstate); - assert(tstate->datastack_chunk == NULL); - assert(tstate->thread_id == 0); - assert(tstate->native_thread_id == 0); - assert(tstate->next == NULL); - assert(tstate->prev == NULL); - - PyThreadState_Clear(tstate); - clear_datastack(tstate); -} - -void -_PyThreadState_BindDetached(PyThreadState *tstate) -{ - assert(!_Py_IsMainInterpreter( - current_fast_get(tstate->interp->runtime)->interp)); - assert(_Py_IsMainInterpreter(tstate->interp)); - bind_tstate(tstate); - /* Unlike _PyThreadState_Bind(), we do not modify gilstate TSS. */ -} - -void -_PyThreadState_UnbindDetached(PyThreadState *tstate) -{ - assert(!_Py_IsMainInterpreter( - current_fast_get(tstate->interp->runtime)->interp)); - assert(_Py_IsMainInterpreter(tstate->interp)); - assert(tstate_is_alive(tstate)); - assert(!tstate->_status.active); - assert(gilstate_tss_get(tstate->interp->runtime) != tstate); - - unbind_tstate(tstate); - - /* This thread state may be bound/unbound repeatedly, - so we must erase evidence that it was ever bound (or unbound). */ - tstate->_status.bound = 0; - tstate->_status.unbound = 0; - - /* We must fully unlink the thread state from any OS thread, - to allow it to be bound more than once. */ - tstate->thread_id = 0; -#ifdef PY_HAVE_THREAD_NATIVE_ID - tstate->native_thread_id = 0; -#endif -} - - //---------- // accessors //---------- From webhook-mailer at python.org Fri Jul 28 16:57:30 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Fri, 28 Jul 2023 20:57:30 -0000 Subject: [Python-checkins] [3.12] gh-104629: Build _testclinic extension module on Windows (GH-104723) (#107393) Message-ID: https://github.com/python/cpython/commit/b4355de94705362a0b83b6ce92572f45273f64b9 commit: b4355de94705362a0b83b6ce92572f45273f64b9 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-28T20:57:26Z summary: [3.12] gh-104629: Build _testclinic extension module on Windows (GH-104723) (#107393) (cherry picked from commit 3a1d819ebc36189e086198212822c9b29384f242) Co-authored-by: Erlend E. Aasland files: A PCbuild/_testclinic.vcxproj A PCbuild/_testclinic.vcxproj.filters M PCbuild/pcbuild.proj M PCbuild/pcbuild.sln M PCbuild/readme.txt M Tools/msi/test/test_files.wxs diff --git a/PCbuild/_testclinic.vcxproj b/PCbuild/_testclinic.vcxproj new file mode 100644 index 0000000000000..e319b3c0f42e0 --- /dev/null +++ b/PCbuild/_testclinic.vcxproj @@ -0,0 +1,110 @@ +? + + + + Debug + ARM + + + Debug + ARM64 + + + Debug + Win32 + + + Debug + x64 + + + PGInstrument + ARM + + + PGInstrument + ARM64 + + + PGInstrument + Win32 + + + PGInstrument + x64 + + + PGUpdate + ARM + + + PGUpdate + ARM64 + + + PGUpdate + Win32 + + + PGUpdate + x64 + + + Release + ARM + + + Release + ARM64 + + + Release + Win32 + + + Release + x64 + + + + {A840DDFB-ED50-484B-B527-B32E7CF90FD5} + _testclinic + Win32Proj + false + + + + + DynamicLibrary + NotSet + + + + .pyd + + + + + + + + + + <_ProjectFileVersion>10.0.30319.1 + + + + + + + + + + {cf7ac3d1-e2df-41d2-bea6-1e2556cdea26} + false + + + + + + \ No newline at end of file diff --git a/PCbuild/_testclinic.vcxproj.filters b/PCbuild/_testclinic.vcxproj.filters new file mode 100644 index 0000000000000..4a2987eb27b22 --- /dev/null +++ b/PCbuild/_testclinic.vcxproj.filters @@ -0,0 +1,21 @@ +? + + + + {5b0a9282-a01c-4b83-9fd4-6deb6c558f9c} + + + {6a89c8a9-5b51-4525-ac5c-7d0a22f9657e} + + + + + Source Files + + + + + Resource Files + + + \ No newline at end of file diff --git a/PCbuild/pcbuild.proj b/PCbuild/pcbuild.proj index e13a0d409293f..28269f08b4244 100644 --- a/PCbuild/pcbuild.proj +++ b/PCbuild/pcbuild.proj @@ -77,7 +77,7 @@ - + diff --git a/PCbuild/pcbuild.sln b/PCbuild/pcbuild.sln index 848d59504381c..bdddec60daa82 100644 --- a/PCbuild/pcbuild.sln +++ b/PCbuild/pcbuild.sln @@ -46,6 +46,7 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "python", "python.vcxproj", {EB6E69DD-04BF-4543-9B92-49FAABCEAC2E} = {EB6E69DD-04BF-4543-9B92-49FAABCEAC2E} {16BFE6F0-22EF-40B5-B831-7E937119EF10} = {16BFE6F0-22EF-40B5-B831-7E937119EF10} {FCBE1EF2-E0F0-40B1-88B5-00A35D378742} = {FCBE1EF2-E0F0-40B1-88B5-00A35D378742} + {A840DDFB-ED50-484B-B527-B32E7CF90FD5} = {A840DDFB-ED50-484B-B527-B32E7CF90FD5} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pythoncore", "pythoncore.vcxproj", "{CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26}" @@ -78,6 +79,8 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_ssl", "_ssl.vcxproj", "{C6 EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_testcapi", "_testcapi.vcxproj", "{6901D91C-6E48-4BB7-9FEC-700C8131DF1D}" EndProject +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_testclinic", "_testclinic.vcxproj", "{A840DDFB-ED50-484B-B527-B32E7CF90FD5}" +EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_testinternalcapi", "_testinternalcapi.vcxproj", "{900342D7-516A-4469-B1AD-59A66E49A25F}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_testimportmultiple", "_testimportmultiple.vcxproj", "{36D0C52C-DF4E-45D0-8BC7-E294C3ABC781}" @@ -592,6 +595,38 @@ Global {6901D91C-6E48-4BB7-9FEC-700C8131DF1D}.Release|Win32.Build.0 = Release|Win32 {6901D91C-6E48-4BB7-9FEC-700C8131DF1D}.Release|x64.ActiveCfg = Release|x64 {6901D91C-6E48-4BB7-9FEC-700C8131DF1D}.Release|x64.Build.0 = Release|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|ARM.ActiveCfg = Debug|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|ARM.Build.0 = Debug|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|ARM64.ActiveCfg = Debug|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|ARM64.Build.0 = Debug|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|Win32.ActiveCfg = Debug|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|Win32.Build.0 = Debug|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|x64.ActiveCfg = Debug|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Debug|x64.Build.0 = Debug|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|ARM.ActiveCfg = PGInstrument|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|ARM.Build.0 = PGInstrument|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|ARM64.ActiveCfg = PGInstrument|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|ARM64.Build.0 = PGInstrument|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|Win32.ActiveCfg = PGInstrument|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|Win32.Build.0 = PGInstrument|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|x64.ActiveCfg = PGInstrument|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGInstrument|x64.Build.0 = PGInstrument|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|ARM.ActiveCfg = PGUpdate|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|ARM.Build.0 = PGUpdate|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|ARM64.ActiveCfg = PGUpdate|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|ARM64.Build.0 = PGUpdate|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|Win32.ActiveCfg = PGUpdate|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|Win32.Build.0 = PGUpdate|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|x64.ActiveCfg = PGUpdate|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.PGUpdate|x64.Build.0 = PGUpdate|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|ARM.ActiveCfg = Release|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|ARM.Build.0 = Release|ARM + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|ARM64.ActiveCfg = Release|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|ARM64.Build.0 = Release|ARM64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|Win32.ActiveCfg = Release|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|Win32.Build.0 = Release|Win32 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|x64.ActiveCfg = Release|x64 + {A840DDFB-ED50-484B-B527-B32E7CF90FD5}.Release|x64.Build.0 = Release|x64 {900342D7-516A-4469-B1AD-59A66E49A25F}.Debug|ARM.ActiveCfg = Debug|ARM {900342D7-516A-4469-B1AD-59A66E49A25F}.Debug|ARM.Build.0 = Debug|ARM {900342D7-516A-4469-B1AD-59A66E49A25F}.Debug|ARM64.ActiveCfg = Debug|ARM64 diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt index 2ab4537b99882..fe44ca40f4a0e 100644 --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -144,6 +144,7 @@ _overlapped _socket _testbuffer _testcapi +_testclinic _testconsole _testimportmultiple _testmultiphase diff --git a/Tools/msi/test/test_files.wxs b/Tools/msi/test/test_files.wxs index b5f68faef30e0..87e164cb6759f 100644 --- a/Tools/msi/test/test_files.wxs +++ b/Tools/msi/test/test_files.wxs @@ -1,6 +1,6 @@ - + From webhook-mailer at python.org Fri Jul 28 18:00:07 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Fri, 28 Jul 2023 22:00:07 -0000 Subject: [Python-checkins] gh-107307: Update the importlib Docs for PEP 684 (gh-107400) Message-ID: https://github.com/python/cpython/commit/cf63df88d38ec3e6ebd44ed184312df9f07f9782 commit: cf63df88d38ec3e6ebd44ed184312df9f07f9782 branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-28T16:00:03-06:00 summary: gh-107307: Update the importlib Docs for PEP 684 (gh-107400) files: M Doc/library/importlib.rst diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index a14e5a1a1a350..1d378dbbdace5 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -941,8 +941,15 @@ find and load modules. The *fullname* argument specifies the name of the module the loader is to support. The *path* argument is the path to the extension module's file. + Note that, by default, importing an extension module will fail + in subinterpreters if it doesn't implement multi-phase init + (see :pep:`489`), even if it would otherwise import successfully. + .. versionadded:: 3.3 + .. versionchanged:: 3.12 + Multi-phase init is now required for use in subinterpreters. + .. attribute:: name Name of the module the loader supports. @@ -1248,6 +1255,30 @@ an :term:`importer`. .. versionadded:: 3.7 +.. function:: _incompatible_extension_module_restrictions(*, disable_check) + + A context manager that can temporarily skip the compatibility check + for extension modules. By default the check is enabled and will fail + when a single-phase init module is imported in a subinterpreter. + It will also fail for a multi-phase init module that doesn't + explicitly support a per-interpreter GIL, when imported + in an interpreter with its own GIL. + + Note that this function is meant to accommodate an unusual case; + one which is likely to eventually go away. There's is a pretty good + chance this is not what you were looking for. + + You can get the same effect as this function by implementing the + basic interface of multi-phase init (:pep:`489`) and lying about + support for mulitple interpreters (or per-interpreter GIL). + + .. warning:: + Using this function to disable the check can lead to + unexpected behavior and even crashes. It should only be used during + extension module development. + + .. versionadded:: 3.12 + .. class:: LazyLoader(loader) A class which postpones the execution of the loader of a module until the From webhook-mailer at python.org Fri Jul 28 18:08:47 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 28 Jul 2023 22:08:47 -0000 Subject: [Python-checkins] gh-106881: Check for linux/limits.h before including it (#107397) Message-ID: https://github.com/python/cpython/commit/11c055f5ff1a353de6d2e77f2af24aaa782878ba commit: 11c055f5ff1a353de6d2e77f2af24aaa782878ba branch: main author: justdan6 <134341009+justdan6 at users.noreply.github.com> committer: gpshead date: 2023-07-28T15:08:43-07:00 summary: gh-106881: Check for linux/limits.h before including it (#107397) * Check for linux/limits.h before including it Co-authored-by: Erlend E. Aasland files: A Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst M Modules/posixmodule.c M configure M configure.ac M pyconfig.h.in diff --git a/Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst b/Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst new file mode 100644 index 0000000000000..40b2609e95c7e --- /dev/null +++ b/Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst @@ -0,0 +1 @@ +Check for `linux/limits.h` before including it in `Modules/posixmodule.c`. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index bab2ac2ccf59f..a42e41c081e5b 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -285,7 +285,7 @@ corresponding Unix manual entries for more information on calls."); # undef HAVE_SCHED_SETAFFINITY #endif -#if defined(HAVE_SYS_XATTR_H) && defined(__linux__) && !defined(__FreeBSD_kernel__) && !defined(__GNU__) +#if defined(HAVE_SYS_XATTR_H) && defined(HAVE_LINUX_LIMITS_H) && !defined(__FreeBSD_kernel__) && !defined(__GNU__) # define USE_XATTRS # include // Needed for XATTR_SIZE_MAX on musl libc. #endif diff --git a/configure b/configure index 18d404c8dc060..80b4a001c6d6d 100755 --- a/configure +++ b/configure @@ -10549,6 +10549,12 @@ if test "x$ac_cv_header_linux_fs_h" = xyes then : printf "%s\n" "#define HAVE_LINUX_FS_H 1" >>confdefs.h +fi +ac_fn_c_check_header_compile "$LINENO" "linux/limits.h" "ac_cv_header_linux_limits_h" "$ac_includes_default" +if test "x$ac_cv_header_linux_limits_h" = xyes +then : + printf "%s\n" "#define HAVE_LINUX_LIMITS_H 1" >>confdefs.h + fi ac_fn_c_check_header_compile "$LINENO" "linux/memfd.h" "ac_cv_header_linux_memfd_h" "$ac_includes_default" if test "x$ac_cv_header_linux_memfd_h" = xyes diff --git a/configure.ac b/configure.ac index cdb88a3071976..8a84eaf6b370a 100644 --- a/configure.ac +++ b/configure.ac @@ -2848,7 +2848,7 @@ AC_DEFINE([STDC_HEADERS], [1], # checks for header files AC_CHECK_HEADERS([ \ alloca.h asm/types.h bluetooth.h conio.h direct.h dlfcn.h endian.h errno.h fcntl.h grp.h \ - ieeefp.h io.h langinfo.h libintl.h libutil.h linux/auxvec.h sys/auxv.h linux/fs.h linux/memfd.h \ + ieeefp.h io.h langinfo.h libintl.h libutil.h linux/auxvec.h sys/auxv.h linux/fs.h linux/limits.h linux/memfd.h \ linux/random.h linux/soundcard.h \ linux/tipc.h linux/wait.h netdb.h net/ethernet.h netinet/in.h netpacket/packet.h poll.h process.h pthread.h pty.h \ sched.h setjmp.h shadow.h signal.h spawn.h stropts.h sys/audioio.h sys/bsdtty.h sys/devpoll.h \ diff --git a/pyconfig.h.in b/pyconfig.h.in index 0828dc8d4b58c..dab8ebf64371f 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -721,6 +721,9 @@ /* Define to 1 if you have the header file. */ #undef HAVE_LINUX_FS_H +/* Define to 1 if you have the header file. */ +#undef HAVE_LINUX_LIMITS_H + /* Define to 1 if you have the header file. */ #undef HAVE_LINUX_MEMFD_H From webhook-mailer at python.org Fri Jul 28 18:09:37 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Fri, 28 Jul 2023 22:09:37 -0000 Subject: [Python-checkins] [3.12] gh-107307: Update the importlib Docs for PEP 684 (gh-107400) (gh-107413) Message-ID: https://github.com/python/cpython/commit/e5ca2aa2c6d86842183565606afcd0f85ed3eac9 commit: e5ca2aa2c6d86842183565606afcd0f85ed3eac9 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ericsnowcurrently date: 2023-07-28T22:09:34Z summary: [3.12] gh-107307: Update the importlib Docs for PEP 684 (gh-107400) (gh-107413) gh-107307: Update the importlib Docs for PEP 684 (gh-107400) (cherry picked from commit cf63df88d38ec3e6ebd44ed184312df9f07f9782) Co-authored-by: Eric Snow files: M Doc/library/importlib.rst diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index a14e5a1a1a350..1d378dbbdace5 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -941,8 +941,15 @@ find and load modules. The *fullname* argument specifies the name of the module the loader is to support. The *path* argument is the path to the extension module's file. + Note that, by default, importing an extension module will fail + in subinterpreters if it doesn't implement multi-phase init + (see :pep:`489`), even if it would otherwise import successfully. + .. versionadded:: 3.3 + .. versionchanged:: 3.12 + Multi-phase init is now required for use in subinterpreters. + .. attribute:: name Name of the module the loader supports. @@ -1248,6 +1255,30 @@ an :term:`importer`. .. versionadded:: 3.7 +.. function:: _incompatible_extension_module_restrictions(*, disable_check) + + A context manager that can temporarily skip the compatibility check + for extension modules. By default the check is enabled and will fail + when a single-phase init module is imported in a subinterpreter. + It will also fail for a multi-phase init module that doesn't + explicitly support a per-interpreter GIL, when imported + in an interpreter with its own GIL. + + Note that this function is meant to accommodate an unusual case; + one which is likely to eventually go away. There's is a pretty good + chance this is not what you were looking for. + + You can get the same effect as this function by implementing the + basic interface of multi-phase init (:pep:`489`) and lying about + support for mulitple interpreters (or per-interpreter GIL). + + .. warning:: + Using this function to disable the check can lead to + unexpected behavior and even crashes. It should only be used during + extension module development. + + .. versionadded:: 3.12 + .. class:: LazyLoader(loader) A class which postpones the execution of the loader of a module until the From webhook-mailer at python.org Fri Jul 28 19:16:24 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Fri, 28 Jul 2023 23:16:24 -0000 Subject: [Python-checkins] [3.12] gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974) (gh-107412) Message-ID: https://github.com/python/cpython/commit/da151fdc7ac4a6d30740e4ef18071936a136790d commit: da151fdc7ac4a6d30740e4ef18071936a136790d branch: 3.12 author: Eric Snow committer: ericsnowcurrently date: 2023-07-28T23:16:12Z summary: [3.12] gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974) (gh-107412) gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974) This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules. The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL. It turns out it's tricky to use the same thread state for different threads. Who could have guessed? We solve the problem by eliminating the one object we were still sharing between interpreters. We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter. We also remove the accommodations for "detached" thread states, which were a dubious idea to start with. (cherry picked from commit 8ba4df91ae60833723d8d3b9afeb2b642f7176d5) files: M Doc/data/python3.12.abi M Include/internal/pycore_import.h M Include/internal/pycore_pystate.h M Include/internal/pycore_runtime_init.h M Python/import.c M Python/pystate.c diff --git a/Doc/data/python3.12.abi b/Doc/data/python3.12.abi index 4c92e1d43c965..5eec83faf1679 100644 --- a/Doc/data/python3.12.abi +++ b/Doc/data/python3.12.abi @@ -1,26484 +1,26464 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Include/internal/pycore_import.h b/Include/internal/pycore_import.h index 0a9f24efbdb90..376957bdc9987 100644 --- a/Include/internal/pycore_import.h +++ b/Include/internal/pycore_import.h @@ -5,6 +5,9 @@ extern "C" { #endif +#include "pycore_hashtable.h" // _Py_hashtable_t +#include "pycore_time.h" // _PyTime_t + struct _import_runtime_state { /* The builtin modules (defined in config.c). */ @@ -15,19 +18,15 @@ struct _import_runtime_state { See PyInterpreterState.modules_by_index for more info. */ Py_ssize_t last_module_index; struct { - /* A thread state tied to the main interpreter, - used exclusively for when the extensions dict is access/modified - from an arbitrary thread. */ - PyThreadState main_tstate; - /* A lock to guard the dict. */ + /* A lock to guard the cache. */ PyThread_type_lock mutex; - /* A dict mapping (filename, name) to PyModuleDef for modules. + /* The actual cache of (filename, name, PyModuleDef) for modules. Only legacy (single-phase init) extension modules are added and only if they support multiple initialization (m_size >- 0) or are imported in the main interpreter. This is initialized lazily in _PyImport_FixupExtensionObject(). Modules are added there and looked up in _imp.find_extension(). */ - PyObject *dict; + _Py_hashtable_t *hashtable; } extensions; /* Package context -- the full module name for package imports */ const char * pkgcontext; diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h index 43652c4405ec1..ccfc2586f0f23 100644 --- a/Include/internal/pycore_pystate.h +++ b/Include/internal/pycore_pystate.h @@ -128,11 +128,6 @@ PyAPI_FUNC(void) _PyThreadState_Init( PyThreadState *tstate); PyAPI_FUNC(void) _PyThreadState_DeleteExcept(PyThreadState *tstate); -extern void _PyThreadState_InitDetached(PyThreadState *, PyInterpreterState *); -extern void _PyThreadState_ClearDetached(PyThreadState *); -extern void _PyThreadState_BindDetached(PyThreadState *); -extern void _PyThreadState_UnbindDetached(PyThreadState *); - /* Other */ diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h index b507de0437d9a..4130188079cff 100644 --- a/Include/internal/pycore_runtime_init.h +++ b/Include/internal/pycore_runtime_init.h @@ -41,11 +41,6 @@ extern PyTypeObject _PyExc_MemoryError; in accordance with the specification. */ \ .autoTSSkey = Py_tss_NEEDS_INIT, \ .parser = _parser_runtime_state_INIT, \ - .imports = { \ - .extensions = { \ - .main_tstate = _PyThreadState_INIT, \ - }, \ - }, \ .ceval = { \ .perf = _PyEval_RUNTIME_PERF_INIT, \ }, \ diff --git a/Python/import.c b/Python/import.c index a93a6450285cc..f8f01f1bcd8c6 100644 --- a/Python/import.c +++ b/Python/import.c @@ -2,10 +2,12 @@ #include "Python.h" +#include "pycore_hashtable.h" // _Py_hashtable_new_full() #include "pycore_import.h" // _PyImport_BootstrapImp() #include "pycore_initconfig.h" // _PyStatus_OK() #include "pycore_interp.h" // struct _import_runtime_state #include "pycore_namespace.h" // _PyNamespace_Type +#include "pycore_object.h" // _Py_SetImmortal() #include "pycore_pyerrors.h" // _PyErr_SetString() #include "pycore_pyhash.h" // _Py_KeyedHash() #include "pycore_pylifecycle.h" @@ -912,35 +914,79 @@ extensions_lock_release(void) dictionary, to avoid loading shared libraries twice. */ +static void * +hashtable_key_from_2_strings(PyObject *str1, PyObject *str2, const char sep) +{ + Py_ssize_t str1_len, str2_len; + const char *str1_data = PyUnicode_AsUTF8AndSize(str1, &str1_len); + const char *str2_data = PyUnicode_AsUTF8AndSize(str2, &str2_len); + if (str1_data == NULL || str2_data == NULL) { + return NULL; + } + /* Make sure sep and the NULL byte won't cause an overflow. */ + assert(SIZE_MAX - str1_len - str2_len > 2); + size_t size = str1_len + 1 + str2_len + 1; + + char *key = PyMem_RawMalloc(size); + if (key == NULL) { + PyErr_NoMemory(); + return NULL; + } + + strncpy(key, str1_data, str1_len); + key[str1_len] = sep; + strncpy(key + str1_len + 1, str2_data, str2_len + 1); + assert(strlen(key) == size - 1); + return key; +} + +static Py_uhash_t +hashtable_hash_str(const void *key) +{ + return _Py_HashBytes(key, strlen((const char *)key)); +} + +static int +hashtable_compare_str(const void *key1, const void *key2) +{ + return strcmp((const char *)key1, (const char *)key2) == 0; +} + static void -_extensions_cache_init(void) +hashtable_destroy_str(void *ptr) { - /* The runtime (i.e. main interpreter) must be initializing, - so we don't need to worry about the lock. */ - _PyThreadState_InitDetached(&EXTENSIONS.main_tstate, - _PyInterpreterState_Main()); + PyMem_RawFree(ptr); } +#define HTSEP ':' + static PyModuleDef * _extensions_cache_get(PyObject *filename, PyObject *name) { PyModuleDef *def = NULL; + void *key = NULL; extensions_lock_acquire(); - PyObject *key = PyTuple_Pack(2, filename, name); - if (key == NULL) { + if (EXTENSIONS.hashtable == NULL) { goto finally; } - PyObject *extensions = EXTENSIONS.dict; - if (extensions == NULL) { + key = hashtable_key_from_2_strings(filename, name, HTSEP); + if (key == NULL) { + goto finally; + } + _Py_hashtable_entry_t *entry = _Py_hashtable_get_entry( + EXTENSIONS.hashtable, key); + if (entry == NULL) { goto finally; } - def = (PyModuleDef *)PyDict_GetItemWithError(extensions, key); + def = (PyModuleDef *)entry->value; finally: - Py_XDECREF(key); extensions_lock_release(); + if (key != NULL) { + PyMem_RawFree(key); + } return def; } @@ -948,124 +994,99 @@ static int _extensions_cache_set(PyObject *filename, PyObject *name, PyModuleDef *def) { int res = -1; - PyThreadState *oldts = NULL; extensions_lock_acquire(); - /* Swap to the main interpreter, if necessary. This matters if - the dict hasn't been created yet or if the item isn't in the - dict yet. In both cases we must ensure the relevant objects - are created using the main interpreter. */ - PyThreadState *main_tstate = &EXTENSIONS.main_tstate; - PyInterpreterState *interp = _PyInterpreterState_GET(); - if (!_Py_IsMainInterpreter(interp)) { - _PyThreadState_BindDetached(main_tstate); - oldts = _PyThreadState_Swap(interp->runtime, main_tstate); - assert(!_Py_IsMainInterpreter(oldts->interp)); - - /* Make sure the name and filename objects are owned - by the main interpreter. */ - name = PyUnicode_InternFromString(PyUnicode_AsUTF8(name)); - assert(name != NULL); - filename = PyUnicode_InternFromString(PyUnicode_AsUTF8(filename)); - assert(filename != NULL); + if (EXTENSIONS.hashtable == NULL) { + _Py_hashtable_allocator_t alloc = {PyMem_RawMalloc, PyMem_RawFree}; + EXTENSIONS.hashtable = _Py_hashtable_new_full( + hashtable_hash_str, + hashtable_compare_str, + hashtable_destroy_str, // key + /* There's no need to decref the def since it's immortal. */ + NULL, // value + &alloc + ); + if (EXTENSIONS.hashtable == NULL) { + PyErr_NoMemory(); + goto finally; + } } - PyObject *key = PyTuple_Pack(2, filename, name); + void *key = hashtable_key_from_2_strings(filename, name, HTSEP); if (key == NULL) { goto finally; } - PyObject *extensions = EXTENSIONS.dict; - if (extensions == NULL) { - extensions = PyDict_New(); - if (extensions == NULL) { + int already_set = 0; + _Py_hashtable_entry_t *entry = _Py_hashtable_get_entry( + EXTENSIONS.hashtable, key); + if (entry == NULL) { + if (_Py_hashtable_set(EXTENSIONS.hashtable, key, def) < 0) { + PyMem_RawFree(key); + PyErr_NoMemory(); goto finally; } - EXTENSIONS.dict = extensions; - } - - PyModuleDef *actual = (PyModuleDef *)PyDict_GetItemWithError(extensions, key); - if (PyErr_Occurred()) { - goto finally; } - else if (actual != NULL) { - /* We expect it to be static, so it must be the same pointer. */ - assert(def == actual); - res = 0; - goto finally; + else { + if (entry->value == NULL) { + entry->value = def; + } + else { + /* We expect it to be static, so it must be the same pointer. */ + assert((PyModuleDef *)entry->value == def); + already_set = 1; + } + PyMem_RawFree(key); } - - /* This might trigger a resize, which is why we must switch - to the main interpreter. */ - res = PyDict_SetItem(extensions, key, (PyObject *)def); - if (res < 0) { - res = -1; - goto finally; + if (!already_set) { + /* We assume that all module defs are statically allocated + and will never be freed. Otherwise, we would incref here. */ + _Py_SetImmortal(def); } res = 0; finally: - Py_XDECREF(key); - if (oldts != NULL) { - _PyThreadState_Swap(interp->runtime, oldts); - _PyThreadState_UnbindDetached(main_tstate); - Py_DECREF(name); - Py_DECREF(filename); - } extensions_lock_release(); return res; } -static int +static void _extensions_cache_delete(PyObject *filename, PyObject *name) { - int res = -1; - PyThreadState *oldts = NULL; + void *key = NULL; extensions_lock_acquire(); - PyObject *key = PyTuple_Pack(2, filename, name); - if (key == NULL) { + if (EXTENSIONS.hashtable == NULL) { + /* It was never added. */ goto finally; } - PyObject *extensions = EXTENSIONS.dict; - if (extensions == NULL) { - res = 0; + key = hashtable_key_from_2_strings(filename, name, HTSEP); + if (key == NULL) { goto finally; } - PyModuleDef *actual = (PyModuleDef *)PyDict_GetItemWithError(extensions, key); - if (PyErr_Occurred()) { + _Py_hashtable_entry_t *entry = _Py_hashtable_get_entry( + EXTENSIONS.hashtable, key); + if (entry == NULL) { + /* It was never added. */ goto finally; } - else if (actual == NULL) { - /* It was already removed or never added. */ - res = 0; + if (entry->value == NULL) { + /* It was already removed. */ goto finally; } - - /* Swap to the main interpreter, if necessary. */ - PyThreadState *main_tstate = &EXTENSIONS.main_tstate; - PyInterpreterState *interp = _PyInterpreterState_GET(); - if (!_Py_IsMainInterpreter(interp)) { - _PyThreadState_BindDetached(main_tstate); - oldts = _PyThreadState_Swap(interp->runtime, main_tstate); - assert(!_Py_IsMainInterpreter(oldts->interp)); - } - - if (PyDict_DelItem(extensions, key) < 0) { - goto finally; - } - res = 0; + /* If we hadn't made the stored defs immortal, we would decref here. + However, this decref would be problematic if the module def were + dynamically allocated, it were the last ref, and this function + were called with an interpreter other than the def's owner. */ + entry->value = NULL; finally: - if (oldts != NULL) { - _PyThreadState_Swap(interp->runtime, oldts); - _PyThreadState_UnbindDetached(main_tstate); - } - Py_XDECREF(key); extensions_lock_release(); - return res; + if (key != NULL) { + PyMem_RawFree(key); + } } static void @@ -1073,11 +1094,12 @@ _extensions_cache_clear_all(void) { /* The runtime (i.e. main interpreter) must be finalizing, so we don't need to worry about the lock. */ - // XXX assert(_Py_IsMainInterpreter(_PyInterpreterState_GET())); - Py_CLEAR(EXTENSIONS.dict); - _PyThreadState_ClearDetached(&EXTENSIONS.main_tstate); + _Py_hashtable_destroy(EXTENSIONS.hashtable); + EXTENSIONS.hashtable = NULL; } +#undef HTSEP + static bool check_multi_interp_extensions(PyInterpreterState *interp) @@ -1238,6 +1260,8 @@ import_find_extension(PyThreadState *tstate, PyObject *name, PyObject *m_copy = def->m_base.m_copy; /* Module does not support repeated initialization */ if (m_copy == NULL) { + /* It might be a core module (e.g. sys & builtins), + for which we don't set m_copy. */ m_copy = get_core_module_dict(tstate->interp, name, filename); if (m_copy == NULL) { return NULL; @@ -1307,9 +1331,7 @@ clear_singlephase_extension(PyInterpreterState *interp, } /* Clear the cached module def. */ - if (_extensions_cache_delete(filename, name) < 0) { - return -1; - } + _extensions_cache_delete(filename, name); return 0; } @@ -3059,6 +3081,8 @@ void _PyImport_Fini(void) { /* Destroy the database used by _PyImport_{Fixup,Find}Extension */ + // XXX Should we actually leave them (mostly) intact, since we don't + // ever dlclose() the module files? _extensions_cache_clear_all(); /* Use the same memory allocator as _PyImport_Init(). */ @@ -3096,10 +3120,6 @@ _PyImport_Fini2(void) PyStatus _PyImport_InitCore(PyThreadState *tstate, PyObject *sysmod, int importlib) { - if (_Py_IsMainInterpreter(tstate->interp)) { - _extensions_cache_init(); - } - // XXX Initialize here: interp->modules and interp->import_func. // XXX Initialize here: sys.modules and sys.meta_path. diff --git a/Python/pystate.c b/Python/pystate.c index e1261bf2acf95..8097124965cb7 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -1632,75 +1632,6 @@ _PyThreadState_DeleteExcept(PyThreadState *tstate) } -//------------------------- -// "detached" thread states -//------------------------- - -void -_PyThreadState_InitDetached(PyThreadState *tstate, PyInterpreterState *interp) -{ - _PyRuntimeState *runtime = interp->runtime; - - HEAD_LOCK(runtime); - interp->threads.next_unique_id += 1; - uint64_t id = interp->threads.next_unique_id; - HEAD_UNLOCK(runtime); - - init_threadstate(tstate, interp, id); - // We do not call add_threadstate(). -} - -void -_PyThreadState_ClearDetached(PyThreadState *tstate) -{ - assert(!tstate->_status.bound); - assert(!tstate->_status.bound_gilstate); - assert(tstate->datastack_chunk == NULL); - assert(tstate->thread_id == 0); - assert(tstate->native_thread_id == 0); - assert(tstate->next == NULL); - assert(tstate->prev == NULL); - - PyThreadState_Clear(tstate); - clear_datastack(tstate); -} - -void -_PyThreadState_BindDetached(PyThreadState *tstate) -{ - assert(!_Py_IsMainInterpreter( - current_fast_get(tstate->interp->runtime)->interp)); - assert(_Py_IsMainInterpreter(tstate->interp)); - bind_tstate(tstate); - /* Unlike _PyThreadState_Bind(), we do not modify gilstate TSS. */ -} - -void -_PyThreadState_UnbindDetached(PyThreadState *tstate) -{ - assert(!_Py_IsMainInterpreter( - current_fast_get(tstate->interp->runtime)->interp)); - assert(_Py_IsMainInterpreter(tstate->interp)); - assert(tstate_is_alive(tstate)); - assert(!tstate->_status.active); - assert(gilstate_tss_get(tstate->interp->runtime) != tstate); - - unbind_tstate(tstate); - - /* This thread state may be bound/unbound repeatedly, - so we must erase evidence that it was ever bound (or unbound). */ - tstate->_status.bound = 0; - tstate->_status.unbound = 0; - - /* We must fully unlink the thread state from any OS thread, - to allow it to be bound more than once. */ - tstate->thread_id = 0; -#ifdef PY_HAVE_THREAD_NATIVE_ID - tstate->native_thread_id = 0; -#endif -} - - //---------- // accessors //---------- From webhook-mailer at python.org Fri Jul 28 19:29:35 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 28 Jul 2023 23:29:35 -0000 Subject: [Python-checkins] [3.12] gh-106881: Check for linux/limits.h before including it (#107397) (#107414) Message-ID: https://github.com/python/cpython/commit/80aebd54c8c562d3d318c124ec735c5bf7f81d65 commit: 80aebd54c8c562d3d318c124ec735c5bf7f81d65 branch: 3.12 author: justdan6 <134341009+justdan6 at users.noreply.github.com> committer: gpshead date: 2023-07-28T16:29:31-07:00 summary: [3.12] gh-106881: Check for linux/limits.h before including it (#107397) (#107414) * Check for linux/limits.h before including it Co-authored-by: Erlend E. Aasland (cherry picked from commit 11c055f5ff1a353de6d2e77f2af24aaa782878ba) files: A Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst M Modules/posixmodule.c M configure M configure.ac M pyconfig.h.in diff --git a/Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst b/Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst new file mode 100644 index 0000000000000..40b2609e95c7e --- /dev/null +++ b/Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst @@ -0,0 +1 @@ +Check for `linux/limits.h` before including it in `Modules/posixmodule.c`. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index fde1e4f461bc9..342f393b1f0f9 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -286,7 +286,7 @@ corresponding Unix manual entries for more information on calls."); # undef HAVE_SCHED_SETAFFINITY #endif -#if defined(HAVE_SYS_XATTR_H) && defined(__linux__) && !defined(__FreeBSD_kernel__) && !defined(__GNU__) +#if defined(HAVE_SYS_XATTR_H) && defined(HAVE_LINUX_LIMITS_H) && !defined(__FreeBSD_kernel__) && !defined(__GNU__) # define USE_XATTRS # include // Needed for XATTR_SIZE_MAX on musl libc. #endif diff --git a/configure b/configure index 3806da7723fda..b6f90bcd8c730 100755 --- a/configure +++ b/configure @@ -10580,6 +10580,12 @@ if test "x$ac_cv_header_linux_fs_h" = xyes then : printf "%s\n" "#define HAVE_LINUX_FS_H 1" >>confdefs.h +fi +ac_fn_c_check_header_compile "$LINENO" "linux/limits.h" "ac_cv_header_linux_limits_h" "$ac_includes_default" +if test "x$ac_cv_header_linux_limits_h" = xyes +then : + printf "%s\n" "#define HAVE_LINUX_LIMITS_H 1" >>confdefs.h + fi ac_fn_c_check_header_compile "$LINENO" "linux/memfd.h" "ac_cv_header_linux_memfd_h" "$ac_includes_default" if test "x$ac_cv_header_linux_memfd_h" = xyes diff --git a/configure.ac b/configure.ac index 630db4bfbd53c..ba768aea93071 100644 --- a/configure.ac +++ b/configure.ac @@ -2848,7 +2848,7 @@ AC_DEFINE([STDC_HEADERS], [1], # checks for header files AC_CHECK_HEADERS([ \ alloca.h asm/types.h bluetooth.h conio.h crypt.h direct.h dlfcn.h endian.h errno.h fcntl.h grp.h \ - ieeefp.h io.h langinfo.h libintl.h libutil.h linux/auxvec.h sys/auxv.h linux/fs.h linux/memfd.h \ + ieeefp.h io.h langinfo.h libintl.h libutil.h linux/auxvec.h sys/auxv.h linux/fs.h linux/limits.h linux/memfd.h \ linux/random.h linux/soundcard.h \ linux/tipc.h linux/wait.h netdb.h net/ethernet.h netinet/in.h netpacket/packet.h poll.h process.h pthread.h pty.h \ sched.h setjmp.h shadow.h signal.h spawn.h stropts.h sys/audioio.h sys/bsdtty.h sys/devpoll.h \ diff --git a/pyconfig.h.in b/pyconfig.h.in index 7c87cd7d5ddd1..ada9dccfef108 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -727,6 +727,9 @@ /* Define to 1 if you have the header file. */ #undef HAVE_LINUX_FS_H +/* Define to 1 if you have the header file. */ +#undef HAVE_LINUX_LIMITS_H + /* Define to 1 if you have the header file. */ #undef HAVE_LINUX_MEMFD_H From webhook-mailer at python.org Fri Jul 28 19:36:58 2023 From: webhook-mailer at python.org (gpshead) Date: Fri, 28 Jul 2023 23:36:58 -0000 Subject: [Python-checkins] [3.11] gh-106881: Check for linux/limits.h before including it (#107397) (#107415) Message-ID: https://github.com/python/cpython/commit/4049c5d6f85627f117e9be6a1729a17fce899645 commit: 4049c5d6f85627f117e9be6a1729a17fce899645 branch: 3.11 author: justdan6 <134341009+justdan6 at users.noreply.github.com> committer: gpshead date: 2023-07-28T23:36:54Z summary: [3.11] gh-106881: Check for linux/limits.h before including it (#107397) (#107415) * [3.11] gh-106881: Check for linux/limits.h before including it (#107397) * Check for linux/limits.h before including it Co-authored-by: Erlend E. Aasland (cherry picked from commit 11c055f5ff1a353de6d2e77f2af24aaa782878ba) * Fix sphinx-lint error in NEWS entry files: A Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst M Modules/posixmodule.c M configure M configure.ac M pyconfig.h.in diff --git a/Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst b/Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst new file mode 100644 index 0000000000000..7febf99c48a79 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2023-07-28-18-17-33.gh-issue-106881.U3Ezdq.rst @@ -0,0 +1 @@ +Check for ``linux/limits.h`` before including it in ``Modules/posixmodule.c``. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index c000a32032287..9d4228896d230 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -280,7 +280,7 @@ corresponding Unix manual entries for more information on calls."); # undef HAVE_SCHED_SETAFFINITY #endif -#if defined(HAVE_SYS_XATTR_H) && defined(__linux__) && !defined(__FreeBSD_kernel__) && !defined(__GNU__) +#if defined(HAVE_SYS_XATTR_H) && defined(HAVE_LINUX_LIMITS_H) && !defined(__FreeBSD_kernel__) && !defined(__GNU__) # define USE_XATTRS # include // Needed for XATTR_SIZE_MAX on musl libc. #endif diff --git a/configure b/configure index d686e09d81c2d..af4a5bbfdfa1a 100755 --- a/configure +++ b/configure @@ -9191,7 +9191,7 @@ $as_echo "#define STDC_HEADERS 1" >>confdefs.h # checks for header files for ac_header in \ alloca.h asm/types.h bluetooth.h conio.h crypt.h direct.h dlfcn.h endian.h errno.h fcntl.h grp.h \ - ieeefp.h io.h langinfo.h libintl.h libutil.h linux/auxvec.h sys/auxv.h linux/memfd.h \ + ieeefp.h io.h langinfo.h libintl.h libutil.h linux/auxvec.h sys/auxv.h linux/limits.h linux/memfd.h \ linux/random.h linux/soundcard.h \ linux/tipc.h linux/wait.h netdb.h netinet/in.h netpacket/packet.h poll.h process.h pthread.h pty.h \ sched.h setjmp.h shadow.h signal.h spawn.h stropts.h sys/audioio.h sys/bsdtty.h sys/devpoll.h \ diff --git a/configure.ac b/configure.ac index 35026f7a2369a..e1cbb7c7fbe9d 100644 --- a/configure.ac +++ b/configure.ac @@ -2668,7 +2668,7 @@ AC_DEFINE(STDC_HEADERS, 1, [Define to 1 if you have the ANSI C header files.]) # checks for header files AC_CHECK_HEADERS([ \ alloca.h asm/types.h bluetooth.h conio.h crypt.h direct.h dlfcn.h endian.h errno.h fcntl.h grp.h \ - ieeefp.h io.h langinfo.h libintl.h libutil.h linux/auxvec.h sys/auxv.h linux/memfd.h \ + ieeefp.h io.h langinfo.h libintl.h libutil.h linux/auxvec.h sys/auxv.h linux/limits.h linux/memfd.h \ linux/random.h linux/soundcard.h \ linux/tipc.h linux/wait.h netdb.h netinet/in.h netpacket/packet.h poll.h process.h pthread.h pty.h \ sched.h setjmp.h shadow.h signal.h spawn.h stropts.h sys/audioio.h sys/bsdtty.h sys/devpoll.h \ diff --git a/pyconfig.h.in b/pyconfig.h.in index 75f1d90e9bd37..0536047f573ce 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -718,6 +718,9 @@ /* Define if compiling using Linux 4.1 or later. */ #undef HAVE_LINUX_CAN_RAW_JOIN_FILTERS +/* Define to 1 if you have the header file. */ +#undef HAVE_LINUX_LIMITS_H + /* Define to 1 if you have the header file. */ #undef HAVE_LINUX_MEMFD_H From webhook-mailer at python.org Fri Jul 28 20:08:14 2023 From: webhook-mailer at python.org (corona10) Date: Sat, 29 Jul 2023 00:08:14 -0000 Subject: [Python-checkins] gh-107089: Improve Shelf.clear method performance (gh-107090) Message-ID: https://github.com/python/cpython/commit/810d5d87d9fe8d86aad99e48cef4f78a72e16ccf commit: 810d5d87d9fe8d86aad99e48cef4f78a72e16ccf branch: main author: James Cave committer: corona10 date: 2023-07-29T09:08:11+09:00 summary: gh-107089: Improve Shelf.clear method performance (gh-107090) files: A Misc/NEWS.d/next/Library/2023-07-22-21-57-34.gh-issue-107089.Dnget2.rst M Lib/shelve.py M Misc/ACKS diff --git a/Lib/shelve.py b/Lib/shelve.py index e053c397345a0..50584716e9ea6 100644 --- a/Lib/shelve.py +++ b/Lib/shelve.py @@ -226,6 +226,13 @@ def __init__(self, filename, flag='c', protocol=None, writeback=False): import dbm Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback) + def clear(self): + """Remove all items from the shelf.""" + # Call through to the clear method on dbm-backed shelves. + # see https://github.com/python/cpython/issues/107089 + self.cache.clear() + self.dict.clear() + def open(filename, flag='c', protocol=None, writeback=False): """Open a persistent dictionary for reading and writing. diff --git a/Misc/ACKS b/Misc/ACKS index fadf488888aa8..8b8c5ad8434bd 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -289,6 +289,7 @@ Edward Catmur Lorenzo M. Catucci Bruno Cauet Donn Cave +James Cave Charles Cazabon Jes?s Cea Avi?n Per Cederqvist diff --git a/Misc/NEWS.d/next/Library/2023-07-22-21-57-34.gh-issue-107089.Dnget2.rst b/Misc/NEWS.d/next/Library/2023-07-22-21-57-34.gh-issue-107089.Dnget2.rst new file mode 100644 index 0000000000000..9d5ba1a2d7ccb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-22-21-57-34.gh-issue-107089.Dnget2.rst @@ -0,0 +1,2 @@ +Shelves opened with :func:`shelve.open` have a much faster :meth:`clear` +method. Patch by James Cave. \ No newline at end of file From webhook-mailer at python.org Sat Jul 29 01:43:13 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 29 Jul 2023 05:43:13 -0000 Subject: [Python-checkins] gh-107091: Fix some uses of :func: role (GH-107378) Message-ID: https://github.com/python/cpython/commit/413ba8943e2f1d896a0568eb571a041b88589440 commit: 413ba8943e2f1d896a0568eb571a041b88589440 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-29T08:43:10+03:00 summary: gh-107091: Fix some uses of :func: role (GH-107378) :c:func: or :c:macro: should be used instead. files: M Doc/whatsnew/2.0.rst M Doc/whatsnew/2.1.rst M Misc/NEWS.d/3.11.0b1.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0b1.rst M Misc/NEWS.d/3.9.0a1.rst M Misc/NEWS.d/next/C API/2020-11-11-22-36-29.bpo-42327.ODSZBM.rst diff --git a/Doc/whatsnew/2.0.rst b/Doc/whatsnew/2.0.rst index 489268ced4c86..71f681881f446 100644 --- a/Doc/whatsnew/2.0.rst +++ b/Doc/whatsnew/2.0.rst @@ -664,7 +664,7 @@ extra set of parentheses to pass both values as a tuple: ``L.append( (1,2) )``. The earlier versions of these methods were more forgiving because they used an old function in Python's C interface to parse their arguments; 2.0 modernizes -them to use :func:`PyArg_ParseTuple`, the current argument parsing function, +them to use :c:func:`PyArg_ParseTuple`, the current argument parsing function, which provides more helpful error messages and treats multi-argument calls as errors. If you absolutely must use 2.0 but can't fix your code, you can edit :file:`Objects/listobject.c` and define the preprocessor symbol @@ -766,7 +766,7 @@ file, :file:`Include/pyport.h`. Vladimir Marangozov's long-awaited malloc restructuring was completed, to make it easy to have the Python interpreter use a custom allocator instead of C's -standard :func:`malloc`. For documentation, read the comments in +standard :c:func:`malloc`. For documentation, read the comments in :file:`Include/pymem.h` and :file:`Include/objimpl.h`. For the lengthy discussions during which the interface was hammered out, see the web archives of the 'patches' and 'python-dev' lists at python.org. @@ -794,15 +794,15 @@ are generating Python code would run into this limit. A patch by Charles G. Waldman raises the limit from ``2**16`` to ``2**32``. Three new convenience functions intended for adding constants to a module's -dictionary at module initialization time were added: :func:`PyModule_AddObject`, -:func:`PyModule_AddIntConstant`, and :func:`PyModule_AddStringConstant`. Each +dictionary at module initialization time were added: :c:func:`PyModule_AddObject`, +:c:func:`PyModule_AddIntConstant`, and :c:func:`PyModule_AddStringConstant`. Each of these functions takes a module object, a null-terminated C string containing the name to be added, and a third argument for the value to be assigned to the name. This third argument is, respectively, a Python object, a C long, or a C string. -A wrapper API was added for Unix-style signal handlers. :func:`PyOS_getsig` gets -a signal handler and :func:`PyOS_setsig` will set a new handler. +A wrapper API was added for Unix-style signal handlers. :c:func:`PyOS_getsig` gets +a signal handler and :c:func:`PyOS_setsig` will set a new handler. .. ====================================================================== diff --git a/Doc/whatsnew/2.1.rst b/Doc/whatsnew/2.1.rst index 676da702b3969..f0e1ded75a9d2 100644 --- a/Doc/whatsnew/2.1.rst +++ b/Doc/whatsnew/2.1.rst @@ -692,8 +692,8 @@ applied, and 136 bugs fixed; both figures are likely to be underestimates. Some of the more notable changes are: * A specialized object allocator is now optionally available, that should be - faster than the system :func:`malloc` and have less memory overhead. The - allocator uses C's :func:`malloc` function to get large pools of memory, and + faster than the system :c:func:`malloc` and have less memory overhead. The + allocator uses C's :c:func:`!malloc` function to get large pools of memory, and then fulfills smaller memory requests from these pools. It can be enabled by providing the :option:`!--with-pymalloc` option to the :program:`configure` script; see :file:`Objects/obmalloc.c` for the implementation details. @@ -701,13 +701,13 @@ of the more notable changes are: Authors of C extension modules should test their code with the object allocator enabled, because some incorrect code may break, causing core dumps at runtime. There are a bunch of memory allocation functions in Python's C API that have - previously been just aliases for the C library's :func:`malloc` and - :func:`free`, meaning that if you accidentally called mismatched functions, the + previously been just aliases for the C library's :c:func:`malloc` and + :c:func:`free`, meaning that if you accidentally called mismatched functions, the error wouldn't be noticeable. When the object allocator is enabled, these - functions aren't aliases of :func:`malloc` and :func:`free` any more, and + functions aren't aliases of :c:func:`!malloc` and :c:func:`!free` any more, and calling the wrong function to free memory will get you a core dump. For - example, if memory was allocated using :func:`PyMem_New`, it has to be freed - using :func:`PyMem_Del`, not :func:`free`. A few modules included with Python + example, if memory was allocated using :c:macro:`PyMem_New`, it has to be freed + using :c:func:`PyMem_Del`, not :c:func:`!free`. A few modules included with Python fell afoul of this and had to be fixed; doubtless there are more third-party modules that will have the same problem. @@ -717,7 +717,7 @@ of the more notable changes are: complain about its lack of speed, and because it's often been used as a na?ve benchmark. The :meth:`readline` method of file objects has therefore been rewritten to be much faster. The exact amount of the speedup will vary from - platform to platform depending on how slow the C library's :func:`getc` was, but + platform to platform depending on how slow the C library's :c:func:`!getc` was, but is around 66%, and potentially much faster on some particular operating systems. Tim Peters did much of the benchmarking and coding for this change, motivated by a discussion in comp.lang.python. @@ -770,7 +770,7 @@ of the more notable changes are: reorganization done by Jeremy Hylton. * C extensions which import other modules have been changed to use - :func:`PyImport_ImportModule`, which means that they will use any import hooks + :c:func:`PyImport_ImportModule`, which means that they will use any import hooks that have been installed. This is also encouraged for third-party extensions that need to import some other module from C code. diff --git a/Misc/NEWS.d/3.11.0b1.rst b/Misc/NEWS.d/3.11.0b1.rst index a4f113cddb0f8..71efc21cbc4a6 100644 --- a/Misc/NEWS.d/3.11.0b1.rst +++ b/Misc/NEWS.d/3.11.0b1.rst @@ -1799,7 +1799,7 @@ The documentation now lists which members of C structs are part of the .. nonce: FIVe9I .. section: Documentation -All docstrings in code snippets are now wrapped into :func:`PyDoc_STR` to +All docstrings in code snippets are now wrapped into :c:macro:`PyDoc_STR` to follow the guideline of `PEP 7's Documentation Strings paragraph `_. Patch by Oleg Iarygin. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index 8ae78ebb1b629..40f6a6cb430d3 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -203,7 +203,7 @@ the interpreter. .. nonce: LYAWlE .. section: Core and Builtins -Bugfix: :func:`PyFunction_GetAnnotations` should return a borrowed +Bugfix: :c:func:`PyFunction_GetAnnotations` should return a borrowed reference. It was returning a new reference. .. diff --git a/Misc/NEWS.d/3.12.0b1.rst b/Misc/NEWS.d/3.12.0b1.rst index 89af6efdae048..652b706880fb9 100644 --- a/Misc/NEWS.d/3.12.0b1.rst +++ b/Misc/NEWS.d/3.12.0b1.rst @@ -1205,7 +1205,7 @@ the future, it will raise a ``KeyError``. Fixed a bug where :mod:`pdb` crashes when reading source file with different encoding by replacing :func:`io.open` with :func:`io.open_code`. The new -method would also call into the hook set by :func:`PyFile_SetOpenCodeHook`. +method would also call into the hook set by :c:func:`PyFile_SetOpenCodeHook`. .. diff --git a/Misc/NEWS.d/3.9.0a1.rst b/Misc/NEWS.d/3.9.0a1.rst index 3b8197c5f039e..43676836478c9 100644 --- a/Misc/NEWS.d/3.9.0a1.rst +++ b/Misc/NEWS.d/3.9.0a1.rst @@ -5686,7 +5686,7 @@ positional argument. .. nonce: zrmgki .. section: C API -Add :func:`PyConfig_SetWideStringList` function. +Add :c:func:`PyConfig_SetWideStringList` function. .. diff --git a/Misc/NEWS.d/next/C API/2020-11-11-22-36-29.bpo-42327.ODSZBM.rst b/Misc/NEWS.d/next/C API/2020-11-11-22-36-29.bpo-42327.ODSZBM.rst index 3d935aceb57a7..bcea7a1f9825b 100644 --- a/Misc/NEWS.d/next/C API/2020-11-11-22-36-29.bpo-42327.ODSZBM.rst +++ b/Misc/NEWS.d/next/C API/2020-11-11-22-36-29.bpo-42327.ODSZBM.rst @@ -1 +1 @@ -Add :func:`PyModule_Add` function: similar to :c:func:`PyModule_AddObjectRef` and :c:func:`PyModule_AddObject`, but always steals a reference to the value. +Add :c:func:`PyModule_Add` function: similar to :c:func:`PyModule_AddObjectRef` and :c:func:`PyModule_AddObject`, but always steals a reference to the value. From webhook-mailer at python.org Sat Jul 29 01:48:14 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 29 Jul 2023 05:48:14 -0000 Subject: [Python-checkins] gh-101100: Sphinx warnings: pick the low hanging fruits (GH-107386) Message-ID: https://github.com/python/cpython/commit/f2d07d3289947d10b065b2bb7670c8fb6b6582f2 commit: f2d07d3289947d10b065b2bb7670c8fb6b6582f2 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-29T08:48:10+03:00 summary: gh-101100: Sphinx warnings: pick the low hanging fruits (GH-107386) files: M Doc/c-api/bool.rst M Doc/faq/extending.rst M Doc/howto/curses.rst M Doc/install/index.rst M Doc/library/asyncio-extending.rst M Doc/library/asyncio-future.rst M Doc/library/bz2.rst M Doc/library/code.rst M Doc/library/concurrent.rst M Doc/library/curses.rst M Doc/library/email.charset.rst M Doc/library/email.encoders.rst M Doc/library/email.generator.rst M Doc/library/email.message.rst M Doc/library/email.parser.rst M Doc/library/filecmp.rst M Doc/library/fileinput.rst M Doc/library/graphlib.rst M Doc/library/gzip.rst M Doc/library/hashlib.rst M Doc/library/importlib.resources.abc.rst M Doc/library/json.rst M Doc/library/logging.rst M Doc/library/lzma.rst M Doc/library/msvcrt.rst M Doc/library/netrc.rst M Doc/library/operator.rst M Doc/library/poplib.rst M Doc/library/pprint.rst M Doc/library/pty.rst M Doc/library/sched.rst M Doc/library/selectors.rst M Doc/library/shutil.rst M Doc/library/stat.rst M Doc/library/stdtypes.rst M Doc/library/sysconfig.rst M Doc/library/textwrap.rst M Doc/library/urllib.error.rst M Doc/library/winreg.rst M Doc/library/winsound.rst M Doc/library/xml.rst M Doc/library/xml.sax.handler.rst M Doc/library/xml.sax.utils.rst M Doc/library/xmlrpc.rst M Doc/reference/datamodel.rst M Doc/reference/expressions.rst M Doc/reference/lexical_analysis.rst M Doc/tools/.nitignore M Doc/tutorial/errors.rst M Doc/tutorial/interactive.rst M Doc/whatsnew/3.0.rst M Misc/NEWS.d/3.11.0a6.rst diff --git a/Doc/c-api/bool.rst b/Doc/c-api/bool.rst index b2d8f2124fc20..b14fa6a0a982e 100644 --- a/Doc/c-api/bool.rst +++ b/Doc/c-api/bool.rst @@ -11,6 +11,12 @@ creation and deletion functions don't apply to booleans. The following macros are available, however. +.. c:var:: PyTypeObject PyBool_Type + + This instance of :c:type:`PyTypeObject` represents the Python boolean type; it + is the same object as :class:`bool` in the Python layer. + + .. c:function:: int PyBool_Check(PyObject *o) Return true if *o* is of type :c:data:`PyBool_Type`. This function always diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst index bc3080f60ee23..2a8b976925d04 100644 --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -81,13 +81,13 @@ How do I extract C values from a Python object? That depends on the object's type. If it's a tuple, :c:func:`PyTuple_Size` returns its length and :c:func:`PyTuple_GetItem` returns the item at a specified -index. Lists have similar functions, :c:func:`PyListSize` and +index. Lists have similar functions, :c:func:`PyList_Size` and :c:func:`PyList_GetItem`. For bytes, :c:func:`PyBytes_Size` returns its length and :c:func:`PyBytes_AsStringAndSize` provides a pointer to its value and its length. Note that Python bytes objects may contain null bytes so C's -:c:func:`strlen` should not be used. +:c:func:`!strlen` should not be used. To test the type of an object, first make sure it isn't ``NULL``, and then use :c:func:`PyBytes_Check`, :c:func:`PyTuple_Check`, :c:func:`PyList_Check`, etc. diff --git a/Doc/howto/curses.rst b/Doc/howto/curses.rst index a3068d86d85bc..4828e2fa29bd2 100644 --- a/Doc/howto/curses.rst +++ b/Doc/howto/curses.rst @@ -527,7 +527,7 @@ If you're in doubt about the detailed behavior of the curses functions, consult the manual pages for your curses implementation, whether it's ncurses or a proprietary Unix vendor's. The manual pages will document any quirks, and provide complete lists of all the -functions, attributes, and :const:`ACS_\*` characters available to +functions, attributes, and :ref:`ACS_\* ` characters available to you. Because the curses API is so large, some functions aren't supported in diff --git a/Doc/install/index.rst b/Doc/install/index.rst index 443c75b7684fb..ffb4a202fe89f 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -778,7 +778,7 @@ Notes: (2) On Unix, if the :envvar:`HOME` environment variable is not defined, the user's - home directory will be determined with the :func:`getpwuid` function from the + home directory will be determined with the :func:`~pwd.getpwuid` function from the standard :mod:`pwd` module. This is done by the :func:`os.path.expanduser` function used by Distutils. diff --git a/Doc/library/asyncio-extending.rst b/Doc/library/asyncio-extending.rst index 8ffd356f2d1cc..e7b293f484f8d 100644 --- a/Doc/library/asyncio-extending.rst +++ b/Doc/library/asyncio-extending.rst @@ -69,7 +69,7 @@ Task lifetime support ===================== A third party task implementation should call the following functions to keep a task -visible by :func:`asyncio.get_tasks` and :func:`asyncio.current_task`: +visible by :func:`asyncio.all_tasks` and :func:`asyncio.current_task`: .. function:: _register_task(task) diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index 70cec9b2f9024..893ae5518f757 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -276,4 +276,4 @@ the Future has a result:: :func:`concurrent.futures.as_completed` functions. - :meth:`asyncio.Future.cancel` accepts an optional ``msg`` argument, - but :func:`concurrent.futures.cancel` does not. + but :meth:`concurrent.futures.Future.cancel` does not. diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst index 32df99869eb53..ec4aeaa04395a 100644 --- a/Doc/library/bz2.rst +++ b/Doc/library/bz2.rst @@ -87,7 +87,8 @@ The :mod:`bz2` module contains: compressed streams. :class:`BZ2File` provides all of the members specified by the - :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`. + :class:`io.BufferedIOBase`, except for :meth:`~io.BufferedIOBase.detach` + and :meth:`~io.IOBase.truncate`. Iteration and the :keyword:`with` statement are supported. :class:`BZ2File` also provides the following method: diff --git a/Doc/library/code.rst b/Doc/library/code.rst index 538e5afc7822a..3d7f43c86a055 100644 --- a/Doc/library/code.rst +++ b/Doc/library/code.rst @@ -163,12 +163,12 @@ interpreter objects as well as the following additions. Push a line of source text to the interpreter. The line should not have a trailing newline; it may have internal newlines. The line is appended to a - buffer and the interpreter's :meth:`runsource` method is called with the + buffer and the interpreter's :meth:`~InteractiveInterpreter.runsource` method is called with the concatenated contents of the buffer as source. If this indicates that the command was executed or invalid, the buffer is reset; otherwise, the command is incomplete, and the buffer is left as it was after the line was appended. The return value is ``True`` if more input is required, ``False`` if the line was - dealt with in some way (this is the same as :meth:`runsource`). + dealt with in some way (this is the same as :meth:`!runsource`). .. method:: InteractiveConsole.resetbuffer() diff --git a/Doc/library/concurrent.rst b/Doc/library/concurrent.rst index 2eba536512580..8caea78bbb57e 100644 --- a/Doc/library/concurrent.rst +++ b/Doc/library/concurrent.rst @@ -1,5 +1,5 @@ -The :mod:`concurrent` package -============================= +The :mod:`!concurrent` package +============================== Currently, there is only one module in this package: diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst index 391c81a844d3e..9ab67c2197539 100644 --- a/Doc/library/curses.rst +++ b/Doc/library/curses.rst @@ -1648,6 +1648,8 @@ keys); also, the following keypad mappings are standard: | :kbd:`Page Down` | KEY_NPAGE | +------------------+-----------+ +.. _curses-acs-codes: + The following table lists characters from the alternate character set. These are inherited from the VT100 terminal, and will generally be available on software emulations such as X terminals. When there is no graphic available, curses diff --git a/Doc/library/email.charset.rst b/Doc/library/email.charset.rst index adbe6c1c7d29b..aa0134412f3a6 100644 --- a/Doc/library/email.charset.rst +++ b/Doc/library/email.charset.rst @@ -150,7 +150,7 @@ Import this class from the :mod:`email.charset` module. .. method:: __str__() Returns *input_charset* as a string coerced to lower - case. :meth:`__repr__` is an alias for :meth:`__str__`. + case. :meth:`!__repr__` is an alias for :meth:`!__str__`. .. method:: __eq__(other) diff --git a/Doc/library/email.encoders.rst b/Doc/library/email.encoders.rst index 5d68b104f3a45..3bd377e33f6c1 100644 --- a/Doc/library/email.encoders.rst +++ b/Doc/library/email.encoders.rst @@ -25,7 +25,7 @@ is especially true for :mimetype:`image/\*` and :mimetype:`text/\*` type message containing binary data. The :mod:`email` package provides some convenient encoders in its -:mod:`encoders` module. These encoders are actually used by the +:mod:`~email.encoders` module. These encoders are actually used by the :class:`~email.mime.audio.MIMEAudio` and :class:`~email.mime.image.MIMEImage` class constructors to provide default encodings. All encoder functions take exactly one argument, the message object to encode. They usually extract the diff --git a/Doc/library/email.generator.rst b/Doc/library/email.generator.rst index eb775b68362c7..afa0038ea2d6c 100644 --- a/Doc/library/email.generator.rst +++ b/Doc/library/email.generator.rst @@ -274,9 +274,9 @@ in with information about the part. .. rubric:: Footnotes .. [#] This statement assumes that you use the appropriate setting for - ``unixfrom``, and that there are no :mod:`policy` settings calling for + ``unixfrom``, and that there are no :mod:`email.policy` settings calling for automatic adjustments (for example, - :attr:`~email.policy.Policy.refold_source` must be ``none``, which is + :attr:`~email.policy.EmailPolicy.refold_source` must be ``none``, which is *not* the default). It is also not 100% true, since if the message does not conform to the RFC standards occasionally information about the exact original text is lost during parsing error recovery. It is a goal diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst index 5e0509f418119..225f498781fa8 100644 --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -67,7 +67,7 @@ message objects. with the base :class:`~email.message.Message` class *maxheaderlen* is accepted, but defaults to ``None``, which means that by default the line length is controlled by the - :attr:`~email.policy.EmailPolicy.max_line_length` of the policy. The + :attr:`~email.policy.Policy.max_line_length` of the policy. The *policy* argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified *policy* will be @@ -213,7 +213,7 @@ message objects. del msg['subject'] msg['subject'] = 'Python roolz!' - If the :mod:`policy` defines certain headers to be unique (as the standard + If the :mod:`policy ` defines certain headers to be unique (as the standard policies do), this method may raise a :exc:`ValueError` when an attempt is made to assign a value to such a header when one already exists. This behavior is intentional for consistency's sake, but do not depend on it @@ -378,7 +378,7 @@ message objects. deprecated. Note that existing parameter values of headers may be accessed through - the :attr:`~email.headerregistry.BaseHeader.params` attribute of the + the :attr:`~email.headerregistry.ParameterizedMIMEHeader.params` attribute of the header value (for example, ``msg['Content-Type'].params['charset']``). .. versionchanged:: 3.4 ``replace`` keyword was added. @@ -691,7 +691,7 @@ message objects. .. method:: clear_content() - Remove the payload and all of the :exc:`Content-` headers, leaving + Remove the payload and all of the :mailheader:`!Content-` headers, leaving all other headers intact and in their original order. diff --git a/Doc/library/email.parser.rst b/Doc/library/email.parser.rst index d9a61616bbbdf..dda0466a6afa7 100644 --- a/Doc/library/email.parser.rst +++ b/Doc/library/email.parser.rst @@ -39,9 +39,9 @@ returns the root object when you close the parser. Note that the parser can be extended in limited ways, and of course you can implement your own parser completely from scratch. All of the logic that connects the :mod:`email` package's bundled parser and the -:class:`~email.message.EmailMessage` class is embodied in the :mod:`policy` +:class:`~email.message.EmailMessage` class is embodied in the :class:`~email.policy.Policy` class, so a custom parser can create message object trees any way it finds -necessary by implementing custom versions of the appropriate :mod:`policy` +necessary by implementing custom versions of the appropriate :class:`!Policy` methods. diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst index 0efb4897a1eb8..dfe4b7c59fd57 100644 --- a/Doc/library/filecmp.rst +++ b/Doc/library/filecmp.rst @@ -100,7 +100,7 @@ The :class:`dircmp` class used to get various bits of information about the directory trees being compared. - Note that via :meth:`__getattr__` hooks, all attributes are computed lazily, + Note that via :meth:`~object.__getattr__` hooks, all attributes are computed lazily, so there is no speed penalty if only those attributes which are lightweight to compute are used. diff --git a/Doc/library/fileinput.rst b/Doc/library/fileinput.rst index 4bc868759f202..f93e9a58791ee 100644 --- a/Doc/library/fileinput.rst +++ b/Doc/library/fileinput.rst @@ -177,7 +177,7 @@ available for subclassing as well: The keyword-only parameter *encoding* and *errors* are added. .. versionchanged:: 3.11 - The ``'rU'`` and ``'U'`` modes and the :meth:`__getitem__` method have + The ``'rU'`` and ``'U'`` modes and the :meth:`!__getitem__` method have been removed. diff --git a/Doc/library/graphlib.rst b/Doc/library/graphlib.rst index fe7932e7a61cb..fdd8f39ef4e1c 100644 --- a/Doc/library/graphlib.rst +++ b/Doc/library/graphlib.rst @@ -115,7 +115,7 @@ :meth:`TopologicalSorter.done` is less than the number that have been returned by :meth:`TopologicalSorter.get_ready`. - The :meth:`~TopologicalSorter.__bool__` method of this class defers to + The :meth:`~object.__bool__` method of this class defers to this function, so instead of:: if ts.is_active(): @@ -204,7 +204,7 @@ The :mod:`graphlib` module defines the following exception classes: in the working graph. If multiple cycles exist, only one undefined choice among them will be reported and included in the exception. - The detected cycle can be accessed via the second element in the :attr:`~CycleError.args` + The detected cycle can be accessed via the second element in the :attr:`~BaseException.args` attribute of the exception instance and consists in a list of nodes, such that each node is, in the graph, an immediate predecessor of the next node in the list. In the reported list, the first and the last node will be the same, to make it clear that it is cyclic. diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst index 979b39a3a5abb..60236a1190e42 100644 --- a/Doc/library/gzip.rst +++ b/Doc/library/gzip.rst @@ -70,7 +70,7 @@ The module defines the following items: .. class:: GzipFile(filename=None, mode=None, compresslevel=9, fileobj=None, mtime=None) Constructor for the :class:`GzipFile` class, which simulates most of the - methods of a :term:`file object`, with the exception of the :meth:`truncate` + methods of a :term:`file object`, with the exception of the :meth:`~io.IOBase.truncate` method. At least one of *fileobj* and *filename* must be given a non-trivial value. @@ -113,7 +113,7 @@ The module defines the following items: :class:`GzipFile` supports the :class:`io.BufferedIOBase` interface, including iteration and the :keyword:`with` statement. Only the - :meth:`truncate` method isn't implemented. + :meth:`~io.IOBase.truncate` method isn't implemented. :class:`GzipFile` also provides the following method and attribute: diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst index 8102767a43d6d..69fb79b49ca2a 100644 --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -244,7 +244,7 @@ by the SHAKE algorithm. .. method:: shake.digest(length) - Return the digest of the data passed to the :meth:`update` method so far. + Return the digest of the data passed to the :meth:`~hash.update` method so far. This is a bytes object of size *length* which may contain bytes in the whole range from 0 to 255. @@ -507,9 +507,9 @@ Simple hashing To calculate hash of some data, you should first construct a hash object by calling the appropriate constructor function (:func:`blake2b` or -:func:`blake2s`), then update it with the data by calling :meth:`update` on the +:func:`blake2s`), then update it with the data by calling :meth:`~hash.update` on the object, and, finally, get the digest out of the object by calling -:meth:`digest` (or :meth:`hexdigest` for hex-encoded string). +:meth:`~hash.digest` (or :meth:`~hash.hexdigest` for hex-encoded string). >>> from hashlib import blake2b >>> h = blake2b() diff --git a/Doc/library/importlib.resources.abc.rst b/Doc/library/importlib.resources.abc.rst index b0e75737137f2..65c42858bbbb7 100644 --- a/Doc/library/importlib.resources.abc.rst +++ b/Doc/library/importlib.resources.abc.rst @@ -145,10 +145,10 @@ An abstract base class for resource readers capable of serving the :meth:`importlib.resources.files` interface. Subclasses - :class:`importlib.resources.abc.ResourceReader` and provides - concrete implementations of the :class:`importlib.resources.abc.ResourceReader`'s + :class:`ResourceReader` and provides + concrete implementations of the :class:`!ResourceReader`'s abstract methods. Therefore, any loader supplying - :class:`importlib.abc.TraversableResources` also supplies ResourceReader. + :class:`!TraversableResources` also supplies :class:`!ResourceReader`. Loaders that wish to support resource reading are expected to implement this interface. diff --git a/Doc/library/json.rst b/Doc/library/json.rst index 35a08995487c1..6c3059381776c 100644 --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -192,7 +192,7 @@ Basic Usage dictionaries will be sorted by key. To use a custom :class:`JSONEncoder` subclass (e.g. one that overrides the - :meth:`default` method to serialize additional types), specify it with the + :meth:`~JSONEncoder.default` method to serialize additional types), specify it with the *cls* kwarg; otherwise :class:`JSONEncoder` is used. .. versionchanged:: 3.6 @@ -422,7 +422,7 @@ Encoders and Decoders Added support for int- and float-derived Enum classes. To extend this to recognize other objects, subclass and implement a - :meth:`default` method with another method that returns a serializable object + :meth:`~JSONEncoder.default` method with another method that returns a serializable object for ``o`` if possible, otherwise it should call the superclass implementation (to raise :exc:`TypeError`). @@ -483,7 +483,7 @@ Encoders and Decoders :exc:`TypeError`). For example, to support arbitrary iterators, you could implement - :meth:`default` like this:: + :meth:`~JSONEncoder.default` like this:: def default(self, o): try: diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index 4e07eabd57f5e..4c6e74ff66a11 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -397,21 +397,21 @@ have specific values relative to the predefined levels. If you define a level with the same numeric value, it overwrites the predefined value; the predefined name is lost. -+--------------+---------------+ -| Level | Numeric value | -+==============+===============+ -| ``CRITICAL`` | 50 | -+--------------+---------------+ -| ``ERROR`` | 40 | -+--------------+---------------+ -| ``WARNING`` | 30 | -+--------------+---------------+ -| ``INFO`` | 20 | -+--------------+---------------+ -| ``DEBUG`` | 10 | -+--------------+---------------+ -| ``NOTSET`` | 0 | -+--------------+---------------+ ++-----------------------+---------------+ +| Level | Numeric value | ++=======================+===============+ +| .. py:data:: CRITICAL | 50 | ++-----------------------+---------------+ +| .. py:data:: ERROR | 40 | ++-----------------------+---------------+ +| .. py:data:: WARNING | 30 | ++-----------------------+---------------+ +| .. py:data:: INFO | 20 | ++-----------------------+---------------+ +| .. py:data:: DEBUG | 10 | ++-----------------------+---------------+ +| .. py:data:: NOTSET | 0 | ++-----------------------+---------------+ .. _handler: diff --git a/Doc/library/lzma.rst b/Doc/library/lzma.rst index 868d4dcfb6c99..434e7ac906118 100644 --- a/Doc/library/lzma.rst +++ b/Doc/library/lzma.rst @@ -100,7 +100,8 @@ Reading and writing compressed files *filters* arguments have the same meanings as for :class:`LZMACompressor`. :class:`LZMAFile` supports all the members specified by - :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`. + :class:`io.BufferedIOBase`, except for :meth:`~io.BufferedIOBase.detach` + and :meth:`~io.IOBase.truncate`. Iteration and the :keyword:`with` statement are supported. The following method is also provided: diff --git a/Doc/library/msvcrt.rst b/Doc/library/msvcrt.rst index 42fffee6a0f44..32693e3d007c0 100644 --- a/Doc/library/msvcrt.rst +++ b/Doc/library/msvcrt.rst @@ -38,7 +38,7 @@ File Operations Lock part of a file based on file descriptor *fd* from the C runtime. Raises :exc:`OSError` on failure. The locked region of the file extends from the current file position for *nbytes* bytes, and may continue beyond the end of the - file. *mode* must be one of the :const:`LK_\*` constants listed below. Multiple + file. *mode* must be one of the :const:`!LK_\*` constants listed below. Multiple regions in a file may be locked at the same time, but may not overlap. Adjacent regions are not merged; they must be unlocked individually. diff --git a/Doc/library/netrc.rst b/Doc/library/netrc.rst index 88265d9b9e9e9..c36e5cfecfc6a 100644 --- a/Doc/library/netrc.rst +++ b/Doc/library/netrc.rst @@ -51,9 +51,19 @@ the Unix :program:`ftp` program and other FTP clients. Exception raised by the :class:`~netrc.netrc` class when syntactical errors are encountered in source text. Instances of this exception provide three - interesting attributes: :attr:`msg` is a textual explanation of the error, - :attr:`filename` is the name of the source file, and :attr:`lineno` gives the - line number on which the error was found. + interesting attributes: + + .. attribute:: msg + + Textual explanation of the error. + + .. attribute:: filename + + The name of the source file. + + .. attribute:: lineno + + The line number on which the error was found. .. _netrc-objects: diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst index dab4de9eb6abb..57c67bcf3aa12 100644 --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -59,9 +59,9 @@ truth tests, identity tests, and boolean operations: __not__(obj) Return the outcome of :keyword:`not` *obj*. (Note that there is no - :meth:`__not__` method for object instances; only the interpreter core defines - this operation. The result is affected by the :meth:`__bool__` and - :meth:`__len__` methods.) + :meth:`!__not__` method for object instances; only the interpreter core defines + this operation. The result is affected by the :meth:`~object.__bool__` and + :meth:`~object.__len__` methods.) .. function:: truth(obj) diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index fbd5e150b4cd5..943eb21f6eec0 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -148,7 +148,7 @@ A :class:`POP3` instance has the following methods: .. method:: POP3.pass_(password) Send password, response includes message count and mailbox size. Note: the - mailbox on the server is locked until :meth:`~poplib.quit` is called. + mailbox on the server is locked until :meth:`~POP3.quit` is called. .. method:: POP3.apop(user, secret) diff --git a/Doc/library/pprint.rst b/Doc/library/pprint.rst index d8269ef48cb36..e883acd67d6c7 100644 --- a/Doc/library/pprint.rst +++ b/Doc/library/pprint.rst @@ -45,7 +45,7 @@ The :mod:`pprint` module defines one class: several keyword parameters. *stream* (default ``sys.stdout``) is a :term:`file-like object` to - which the output will be written by calling its :meth:`write` method. + which the output will be written by calling its :meth:`!write` method. If both *stream* and ``sys.stdout`` are ``None``, then :meth:`~PrettyPrinter.pprint` silently returns. diff --git a/Doc/library/pty.rst b/Doc/library/pty.rst index 7f4da41e93802..ad4981c97119f 100644 --- a/Doc/library/pty.rst +++ b/Doc/library/pty.rst @@ -71,7 +71,7 @@ The :mod:`pty` module defines the following functions: Return the exit status value from :func:`os.waitpid` on the child process. - :func:`waitstatus_to_exitcode` can be used to convert the exit status into + :func:`os.waitstatus_to_exitcode` can be used to convert the exit status into an exit code. .. audit-event:: pty.spawn argv pty.spawn diff --git a/Doc/library/sched.rst b/Doc/library/sched.rst index 04215d31ba10c..01bac5afd0b9b 100644 --- a/Doc/library/sched.rst +++ b/Doc/library/sched.rst @@ -115,7 +115,7 @@ Scheduler Objects .. method:: scheduler.run(blocking=True) - Run all scheduled events. This method will wait (using the :func:`delayfunc` + Run all scheduled events. This method will wait (using the *delayfunc* function passed to the constructor) for the next event, then execute it and so on until there are no more scheduled events. diff --git a/Doc/library/selectors.rst b/Doc/library/selectors.rst index 0deb15cf4c503..dd50bac37e49b 100644 --- a/Doc/library/selectors.rst +++ b/Doc/library/selectors.rst @@ -60,9 +60,9 @@ constants below: +-----------------------+-----------------------------------------------+ | Constant | Meaning | +=======================+===============================================+ - | :const:`EVENT_READ` | Available for read | + | .. data:: EVENT_READ | Available for read | +-----------------------+-----------------------------------------------+ - | :const:`EVENT_WRITE` | Available for write | + | .. data:: EVENT_WRITE | Available for write | +-----------------------+-----------------------------------------------+ @@ -132,8 +132,8 @@ constants below: Change a registered file object's monitored events or attached data. - This is equivalent to :meth:`BaseSelector.unregister(fileobj)` followed - by :meth:`BaseSelector.register(fileobj, events, data)`, except that it + This is equivalent to ``BaseSelector.unregister(fileobj)`` followed + by ``BaseSelector.register(fileobj, events, data)``, except that it can be implemented more efficiently. This returns a new :class:`SelectorKey` instance, or raises a diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 7699d22a72aaf..4390a8e22306f 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -369,7 +369,7 @@ Directory and files operations If *copy_function* is given, it must be a callable that takes two arguments *src* and *dst*, and will be used to copy *src* to *dst* if :func:`os.rename` cannot be used. If the source is a directory, - :func:`copytree` is called, passing it the :func:`copy_function`. The + :func:`copytree` is called, passing it the *copy_function*. The default *copy_function* is :func:`copy2`. Using :func:`~shutil.copy` as the *copy_function* allows the move to succeed when it is not possible to also copy the metadata, at the expense of not copying any of the metadata. diff --git a/Doc/library/stat.rst b/Doc/library/stat.rst index 083dc5e3bcfd6..77538514598a5 100644 --- a/Doc/library/stat.rst +++ b/Doc/library/stat.rst @@ -13,8 +13,8 @@ The :mod:`stat` module defines constants and functions for interpreting the results of :func:`os.stat`, :func:`os.fstat` and :func:`os.lstat` (if they -exist). For complete details about the :c:func:`stat`, :c:func:`fstat` and -:c:func:`lstat` calls, consult the documentation for your system. +exist). For complete details about the :c:func:`stat`, :c:func:`!fstat` and +:c:func:`!lstat` calls, consult the documentation for your system. .. versionchanged:: 3.4 The stat module is backed by a C implementation. @@ -89,9 +89,9 @@ mode: .. function:: S_IFMT(mode) Return the portion of the file's mode that describes the file type (used by the - :func:`S_IS\*` functions above). + :func:`!S_IS\*` functions above). -Normally, you would use the :func:`os.path.is\*` functions for testing the type +Normally, you would use the :func:`!os.path.is\*` functions for testing the type of a file; the functions here are useful when you are doing multiple tests of the same file and wish to avoid the overhead of the :c:func:`stat` system call for each test. These are also useful when checking for information about a file diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 26266b9eb28b8..6c07ee585480e 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -44,7 +44,7 @@ Any object can be tested for truth value, for use in an :keyword:`if` or .. index:: single: true By default, an object is considered true unless its class defines either a -:meth:`__bool__` method that returns ``False`` or a :meth:`__len__` method that +:meth:`~object.__bool__` method that returns ``False`` or a :meth:`__len__` method that returns zero, when called with the object. [1]_ Here are most of the built-in objects considered false: diff --git a/Doc/library/sysconfig.rst b/Doc/library/sysconfig.rst index 839c2c015b49a..c805c50ffc689 100644 --- a/Doc/library/sysconfig.rst +++ b/Doc/library/sysconfig.rst @@ -69,7 +69,7 @@ Python uses an installation scheme that differs depending on the platform and on the installation options. These schemes are stored in :mod:`sysconfig` under unique identifiers based on the value returned by :const:`os.name`. -Every new component that is installed using :mod:`distutils` or a +Every new component that is installed using :mod:`!distutils` or a Distutils-based system will follow the same scheme to copy its file in the right places. diff --git a/Doc/library/textwrap.rst b/Doc/library/textwrap.rst index 1a9d5f98f78a7..a150eefbf932e 100644 --- a/Doc/library/textwrap.rst +++ b/Doc/library/textwrap.rst @@ -60,7 +60,7 @@ functions should be good enough; otherwise, you should use an instance of First the whitespace in *text* is collapsed (all whitespace is replaced by single spaces). If the result fits in the *width*, it is returned. Otherwise, enough words are dropped from the end so that the remaining words - plus the :attr:`placeholder` fit within :attr:`width`:: + plus the :attr:`.placeholder` fit within :attr:`.width`:: >>> textwrap.shorten("Hello world!", width=12) 'Hello world!' @@ -173,7 +173,7 @@ hyphenated words; only then will long words be broken if necessary, unless .. attribute:: expand_tabs (default: ``True``) If true, then all tab characters in *text* will be - expanded to spaces using the :meth:`expandtabs` method of *text*. + expanded to spaces using the :meth:`~str.expandtabs` method of *text*. .. attribute:: tabsize diff --git a/Doc/library/urllib.error.rst b/Doc/library/urllib.error.rst index 3adbdd2613227..a5bcb5b1e643b 100644 --- a/Doc/library/urllib.error.rst +++ b/Doc/library/urllib.error.rst @@ -72,6 +72,8 @@ The following exceptions are raised by :mod:`urllib.error` as appropriate: This exception is raised when the :func:`~urllib.request.urlretrieve` function detects that the amount of the downloaded data is less than the expected amount (given by - the *Content-Length* header). The :attr:`content` attribute stores the - downloaded (and supposedly truncated) data. + the *Content-Length* header). + .. attribute:: content + + The downloaded (and supposedly truncated) data. diff --git a/Doc/library/winreg.rst b/Doc/library/winreg.rst index 4ab671817710d..06bd4d87eb03c 100644 --- a/Doc/library/winreg.rst +++ b/Doc/library/winreg.rst @@ -288,7 +288,7 @@ This module offers the following functions: table (FAT) file system, the filename may not have an extension. A call to :func:`LoadKey` fails if the calling process does not have the - :const:`SE_RESTORE_PRIVILEGE` privilege. Note that privileges are different + :c:data:`!SE_RESTORE_PRIVILEGE` privilege. Note that privileges are different from permissions -- see the `RegLoadKey documentation `__ for more details. @@ -414,7 +414,7 @@ This module offers the following functions: If *key* represents a key on a remote computer, the path described by *file_name* is relative to the remote computer. The caller of this method must - possess the :const:`SeBackupPrivilege` security privilege. Note that + possess the **SeBackupPrivilege** security privilege. Note that privileges are different than permissions -- see the `Conflicts Between User Rights and Permissions documentation `__ @@ -536,7 +536,7 @@ This module offers the following functions: Constants ------------------ -The following constants are defined for use in many :mod:`_winreg` functions. +The following constants are defined for use in many :mod:`winreg` functions. .. _hkey-constants: @@ -745,7 +745,7 @@ All registry functions in this module return one of these objects. All registry functions in this module which accept a handle object also accept an integer, however, use of the handle object is encouraged. -Handle objects provide semantics for :meth:`__bool__` -- thus :: +Handle objects provide semantics for :meth:`~object.__bool__` -- thus :: if handle: print("Yes") diff --git a/Doc/library/winsound.rst b/Doc/library/winsound.rst index 372f792a0f938..370c5216652ba 100644 --- a/Doc/library/winsound.rst +++ b/Doc/library/winsound.rst @@ -24,7 +24,7 @@ provided by Windows platforms. It includes functions and several constants. .. function:: PlaySound(sound, flags) - Call the underlying :c:func:`PlaySound` function from the Platform API. The + Call the underlying :c:func:`!PlaySound` function from the Platform API. The *sound* parameter may be a filename, a system sound alias, audio data as a :term:`bytes-like object`, or ``None``. Its interpretation depends on the value of *flags*, which can be a bitwise ORed @@ -35,7 +35,7 @@ provided by Windows platforms. It includes functions and several constants. .. function:: MessageBeep(type=MB_OK) - Call the underlying :c:func:`MessageBeep` function from the Platform API. This + Call the underlying :c:func:`!MessageBeep` function from the Platform API. This plays a sound as specified in the registry. The *type* argument specifies which sound to play; possible values are ``-1``, ``MB_ICONASTERISK``, ``MB_ICONEXCLAMATION``, ``MB_ICONHAND``, ``MB_ICONQUESTION``, and ``MB_OK``, all diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index cf30de67719b9..1e49b6568dfc2 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -75,10 +75,10 @@ decompression bomb Safe Safe Safe potential reliance on system-provided libraries. Check :const:`pyexpat.EXPAT_VERSION`. 2. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a - :exc:`ParserError` when an entity occurs. + :exc:`~xml.etree.ElementTree.ParseError` when an entity occurs. 3. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns the unexpanded entity verbatim. -4. :mod:`xmlrpclib` doesn't expand external entities and omits them. +4. :mod:`xmlrpc.client` doesn't expand external entities and omits them. 5. Since Python 3.7.1, external general entities are no longer processed by default. @@ -119,8 +119,8 @@ all known attack vectors with examples and references. .. _defusedxml-package: -The :mod:`defusedxml` Package ------------------------------------------------------- +The :mod:`!defusedxml` Package +------------------------------ `defusedxml`_ is a pure Python package with modified subclasses of all stdlib XML parsers that prevent any potentially malicious operation. Use of this diff --git a/Doc/library/xml.sax.handler.rst b/Doc/library/xml.sax.handler.rst index 719ce5ab1bcf6..e2f28e3244cb0 100644 --- a/Doc/library/xml.sax.handler.rst +++ b/Doc/library/xml.sax.handler.rst @@ -393,7 +393,7 @@ implements this interface, then register the object with your :class:`~xml.sax.xmlreader.XMLReader`, the parser will call the methods in your object to report all warnings and errors. There are three levels of errors available: warnings, (possibly) recoverable errors, -and unrecoverable errors. All methods take a :exc:`SAXParseException` as the +and unrecoverable errors. All methods take a :exc:`~xml.sax.SAXParseException` as the only parameter. Errors and warnings may be converted to an exception by raising the passed-in exception object. diff --git a/Doc/library/xml.sax.utils.rst b/Doc/library/xml.sax.utils.rst index ab4606bcf9fe6..e57e76dcac782 100644 --- a/Doc/library/xml.sax.utils.rst +++ b/Doc/library/xml.sax.utils.rst @@ -92,5 +92,5 @@ or as base classes. reading. The input source can be given as a string, a file-like object, or an :class:`~xml.sax.xmlreader.InputSource` object; parsers will use this function to implement the polymorphic *source* argument to their - :meth:`parse` method. + :meth:`~xml.sax.xmlreader.XMLReader.parse` method. diff --git a/Doc/library/xmlrpc.rst b/Doc/library/xmlrpc.rst index ae68157b0f63c..5f0a2cf68d01f 100644 --- a/Doc/library/xmlrpc.rst +++ b/Doc/library/xmlrpc.rst @@ -1,5 +1,5 @@ -:mod:`xmlrpc` --- XMLRPC server and client modules -================================================== +:mod:`!xmlrpc` --- XMLRPC server and client modules +=================================================== XML-RPC is a Remote Procedure Call method that uses XML passed via HTTP as a transport. With it, a client can call methods with parameters on a remote diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index cbf854126bd5b..229fa696c9142 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1594,9 +1594,9 @@ Basic customization Called to implement truth value testing and the built-in operation ``bool()``; should return ``False`` or ``True``. When this method is not - defined, :meth:`__len__` is called, if it is defined, and the object is + defined, :meth:`~object.__len__` is called, if it is defined, and the object is considered true if its result is nonzero. If a class defines neither - :meth:`__len__` nor :meth:`__bool__`, all its instances are considered + :meth:`!__len__` nor :meth:`!__bool__`, all its instances are considered true. @@ -2494,7 +2494,7 @@ through the object's keys; for sequences, it should iterate through the values. Called to implement the built-in function :func:`len`. Should return the length of the object, an integer ``>=`` 0. Also, an object that doesn't define a - :meth:`__bool__` method and whose :meth:`__len__` method returns zero is + :meth:`~object.__bool__` method and whose :meth:`!__len__` method returns zero is considered to be false in a Boolean context. .. impl-detail:: @@ -2503,7 +2503,7 @@ through the object's keys; for sequences, it should iterate through the values. If the length is larger than :data:`!sys.maxsize` some features (such as :func:`len`) may raise :exc:`OverflowError`. To prevent raising :exc:`!OverflowError` by truth value testing, an object must define a - :meth:`__bool__` method. + :meth:`~object.__bool__` method. .. method:: object.__length_hint__(self) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index ce1c9a59d5835..8e0346ccc718d 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1724,7 +1724,7 @@ control flow statements, the following values are interpreted as false: ``False``, ``None``, numeric zero of all types, and empty strings and containers (including strings, tuples, lists, dictionaries, sets and frozensets). All other values are interpreted as true. User-defined objects can customize their -truth value by providing a :meth:`__bool__` method. +truth value by providing a :meth:`~object.__bool__` method. .. index:: pair: operator; not diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index dde7ba1d941dc..83cd4402a36cf 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -787,7 +787,7 @@ is converted before formatting. Conversion ``'!s'`` calls :func:`str` on the result, ``'!r'`` calls :func:`repr`, and ``'!a'`` calls :func:`ascii`. The result is then formatted using the :func:`format` protocol. The -format specifier is passed to the :meth:`__format__` method of the +format specifier is passed to the :meth:`~object.__format__` method of the expression or conversion result. An empty string is passed when the format specifier is omitted. The formatted result is then included in the final value of the whole string. diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 22eb47856f571..df7dcd607489c 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -2,7 +2,6 @@ # as tested on the CI via check-warnings.py in reusable-docs.yml. # Keep lines sorted lexicographically to help avoid merge conflicts. -Doc/c-api/bool.rst Doc/c-api/buffer.rst Doc/c-api/datetime.rst Doc/c-api/descriptor.rst @@ -27,26 +26,22 @@ Doc/c-api/unicode.rst Doc/extending/extending.rst Doc/extending/newtypes.rst Doc/faq/design.rst -Doc/faq/extending.rst Doc/faq/gui.rst Doc/faq/library.rst Doc/faq/programming.rst Doc/glossary.rst -Doc/howto/curses.rst Doc/howto/descriptor.rst Doc/howto/enum.rst Doc/howto/isolating-extensions.rst Doc/howto/logging-cookbook.rst Doc/howto/logging.rst Doc/howto/urllib2.rst -Doc/install/index.rst Doc/library/__future__.rst Doc/library/abc.rst Doc/library/ast.rst Doc/library/asyncio-dev.rst Doc/library/asyncio-eventloop.rst Doc/library/asyncio-extending.rst -Doc/library/asyncio-future.rst Doc/library/asyncio-policy.rst Doc/library/asyncio-stream.rst Doc/library/asyncio-subprocess.rst @@ -56,12 +51,10 @@ Doc/library/bisect.rst Doc/library/bz2.rst Doc/library/calendar.rst Doc/library/cmd.rst -Doc/library/code.rst Doc/library/codecs.rst Doc/library/collections.abc.rst Doc/library/collections.rst Doc/library/concurrent.futures.rst -Doc/library/concurrent.rst Doc/library/configparser.rst Doc/library/contextlib.rst Doc/library/copy.rst @@ -76,11 +69,8 @@ Doc/library/dis.rst Doc/library/doctest.rst Doc/library/email.charset.rst Doc/library/email.compat32-message.rst -Doc/library/email.encoders.rst Doc/library/email.errors.rst -Doc/library/email.generator.rst Doc/library/email.headerregistry.rst -Doc/library/email.message.rst Doc/library/email.mime.rst Doc/library/email.parser.rst Doc/library/email.policy.rst @@ -88,26 +78,20 @@ Doc/library/enum.rst Doc/library/exceptions.rst Doc/library/faulthandler.rst Doc/library/fcntl.rst -Doc/library/filecmp.rst -Doc/library/fileinput.rst Doc/library/ftplib.rst Doc/library/functions.rst Doc/library/functools.rst Doc/library/getpass.rst Doc/library/gettext.rst -Doc/library/graphlib.rst Doc/library/gzip.rst -Doc/library/hashlib.rst Doc/library/http.client.rst Doc/library/http.cookiejar.rst Doc/library/http.cookies.rst Doc/library/http.server.rst -Doc/library/importlib.resources.abc.rst Doc/library/importlib.resources.rst Doc/library/importlib.rst Doc/library/inspect.rst Doc/library/io.rst -Doc/library/json.rst Doc/library/locale.rst Doc/library/logging.config.rst Doc/library/logging.handlers.rst @@ -115,12 +99,9 @@ Doc/library/logging.rst Doc/library/lzma.rst Doc/library/mailbox.rst Doc/library/mmap.rst -Doc/library/msvcrt.rst Doc/library/multiprocessing.rst Doc/library/multiprocessing.shared_memory.rst -Doc/library/netrc.rst Doc/library/numbers.rst -Doc/library/operator.rst Doc/library/optparse.rst Doc/library/os.path.rst Doc/library/os.rst @@ -128,10 +109,7 @@ Doc/library/pickle.rst Doc/library/pickletools.rst Doc/library/platform.rst Doc/library/plistlib.rst -Doc/library/poplib.rst -Doc/library/pprint.rst Doc/library/profile.rst -Doc/library/pty.rst Doc/library/pyclbr.rst Doc/library/pydoc.rst Doc/library/pyexpat.rst @@ -140,30 +118,25 @@ Doc/library/readline.rst Doc/library/reprlib.rst Doc/library/resource.rst Doc/library/rlcompleter.rst -Doc/library/sched.rst Doc/library/select.rst Doc/library/selectors.rst Doc/library/shelve.rst -Doc/library/shutil.rst Doc/library/signal.rst Doc/library/site.rst Doc/library/smtplib.rst Doc/library/socket.rst Doc/library/socketserver.rst Doc/library/ssl.rst -Doc/library/stat.rst Doc/library/stdtypes.rst Doc/library/string.rst Doc/library/subprocess.rst Doc/library/sys.rst Doc/library/sys_path_init.rst -Doc/library/sysconfig.rst Doc/library/syslog.rst Doc/library/tarfile.rst Doc/library/tempfile.rst Doc/library/termios.rst Doc/library/test.rst -Doc/library/textwrap.rst Doc/library/threading.rst Doc/library/time.rst Doc/library/tkinter.rst @@ -175,13 +148,10 @@ Doc/library/turtle.rst Doc/library/unittest.mock-examples.rst Doc/library/unittest.mock.rst Doc/library/unittest.rst -Doc/library/urllib.error.rst Doc/library/urllib.parse.rst Doc/library/urllib.request.rst Doc/library/uuid.rst Doc/library/weakref.rst -Doc/library/winreg.rst -Doc/library/winsound.rst Doc/library/wsgiref.rst Doc/library/xml.dom.minidom.rst Doc/library/xml.dom.pulldom.rst @@ -191,9 +161,7 @@ Doc/library/xml.rst Doc/library/xml.sax.handler.rst Doc/library/xml.sax.reader.rst Doc/library/xml.sax.rst -Doc/library/xml.sax.utils.rst Doc/library/xmlrpc.client.rst -Doc/library/xmlrpc.rst Doc/library/xmlrpc.server.rst Doc/library/zlib.rst Doc/license.rst @@ -201,18 +169,14 @@ Doc/reference/compound_stmts.rst Doc/reference/datamodel.rst Doc/reference/expressions.rst Doc/reference/import.rst -Doc/reference/lexical_analysis.rst Doc/reference/simple_stmts.rst Doc/tutorial/appendix.rst Doc/tutorial/classes.rst Doc/tutorial/controlflow.rst Doc/tutorial/datastructures.rst -Doc/tutorial/errors.rst Doc/tutorial/inputoutput.rst -Doc/tutorial/interactive.rst Doc/tutorial/introduction.rst Doc/tutorial/modules.rst -Doc/tutorial/stdlib2.rst Doc/using/cmdline.rst Doc/using/configure.rst Doc/using/windows.rst diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index 8a207c385c6ab..1ec59767e9ce1 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -154,7 +154,7 @@ exception type. The *except clause* may specify a variable after the exception name. The variable is bound to the exception instance which typically has an ``args`` attribute that stores the arguments. For convenience, builtin exception -types define :meth:`__str__` to print all the arguments without explicitly +types define :meth:`~object.__str__` to print all the arguments without explicitly accessing ``.args``. :: >>> try: @@ -174,7 +174,7 @@ accessing ``.args``. :: x = spam y = eggs -The exception's :meth:`__str__` output is printed as the last part ('detail') +The exception's :meth:`~object.__str__` output is printed as the last part ('detail') of the message for unhandled exceptions. :exc:`BaseException` is the common base class of all exceptions. One of its diff --git a/Doc/tutorial/interactive.rst b/Doc/tutorial/interactive.rst index c0eb1feec4eb4..0d3896a4832b5 100644 --- a/Doc/tutorial/interactive.rst +++ b/Doc/tutorial/interactive.rst @@ -23,7 +23,7 @@ Python statement names, the current local variables, and the available module names. For dotted expressions such as ``string.a``, it will evaluate the expression up to the final ``'.'`` and then suggest completions from the attributes of the resulting object. Note that this may execute -application-defined code if an object with a :meth:`__getattr__` method +application-defined code if an object with a :meth:`~object.__getattr__` method is part of the expression. The default configuration also saves your history into a file named :file:`.python_history` in your user directory. The history will be available again during the next interactive interpreter diff --git a/Doc/whatsnew/3.0.rst b/Doc/whatsnew/3.0.rst index b8cd7c48b359b..b767bbe177abe 100644 --- a/Doc/whatsnew/3.0.rst +++ b/Doc/whatsnew/3.0.rst @@ -789,7 +789,7 @@ Operators And Special Methods :attr:`__doc__`, :attr:`__globals__`, :attr:`~definition.__name__`, respectively. -* :meth:`__nonzero__` is now :meth:`__bool__`. +* :meth:`!__nonzero__` is now :meth:`~object.__bool__`. Builtins -------- diff --git a/Misc/NEWS.d/3.11.0a6.rst b/Misc/NEWS.d/3.11.0a6.rst index 8621edcfb04bb..fcec71c6f59da 100644 --- a/Misc/NEWS.d/3.11.0a6.rst +++ b/Misc/NEWS.d/3.11.0a6.rst @@ -352,7 +352,7 @@ rather than ``JUMP_FORWARD`` with an argument of ``(2**32)+offset``. .. nonce: 3Z_qxd .. section: Core and Builtins -Correct the docstring for the :meth:`__bool__` method. Patch by Jelle +Correct the docstring for the :meth:`~object.__bool__` method. Patch by Jelle Zijlstra. .. From webhook-mailer at python.org Sat Jul 29 01:52:29 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 29 Jul 2023 05:52:29 -0000 Subject: [Python-checkins] [3.12] gh-107091: Fix some uses of :func: role (GH-107378) (GH-107416) Message-ID: https://github.com/python/cpython/commit/34e6e14602ae5d03040f8fa38f4a841c9fe66e10 commit: 34e6e14602ae5d03040f8fa38f4a841c9fe66e10 branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-29T05:52:25Z summary: [3.12] gh-107091: Fix some uses of :func: role (GH-107378) (GH-107416) :c:func: or :c:macro: should be used instead. (cherry picked from commit 413ba8943e2f1d896a0568eb571a041b88589440) files: M Doc/whatsnew/2.0.rst M Doc/whatsnew/2.1.rst M Misc/NEWS.d/3.11.0b1.rst M Misc/NEWS.d/3.12.0a1.rst M Misc/NEWS.d/3.12.0b1.rst M Misc/NEWS.d/3.9.0a1.rst diff --git a/Doc/whatsnew/2.0.rst b/Doc/whatsnew/2.0.rst index 0eefefd863a68..5e9d03e48d97c 100644 --- a/Doc/whatsnew/2.0.rst +++ b/Doc/whatsnew/2.0.rst @@ -664,7 +664,7 @@ extra set of parentheses to pass both values as a tuple: ``L.append( (1,2) )``. The earlier versions of these methods were more forgiving because they used an old function in Python's C interface to parse their arguments; 2.0 modernizes -them to use :func:`PyArg_ParseTuple`, the current argument parsing function, +them to use :c:func:`PyArg_ParseTuple`, the current argument parsing function, which provides more helpful error messages and treats multi-argument calls as errors. If you absolutely must use 2.0 but can't fix your code, you can edit :file:`Objects/listobject.c` and define the preprocessor symbol @@ -766,7 +766,7 @@ file, :file:`Include/pyport.h`. Vladimir Marangozov's long-awaited malloc restructuring was completed, to make it easy to have the Python interpreter use a custom allocator instead of C's -standard :func:`malloc`. For documentation, read the comments in +standard :c:func:`malloc`. For documentation, read the comments in :file:`Include/pymem.h` and :file:`Include/objimpl.h`. For the lengthy discussions during which the interface was hammered out, see the web archives of the 'patches' and 'python-dev' lists at python.org. @@ -794,15 +794,15 @@ are generating Python code would run into this limit. A patch by Charles G. Waldman raises the limit from ``2**16`` to ``2**32``. Three new convenience functions intended for adding constants to a module's -dictionary at module initialization time were added: :func:`PyModule_AddObject`, -:func:`PyModule_AddIntConstant`, and :func:`PyModule_AddStringConstant`. Each +dictionary at module initialization time were added: :c:func:`PyModule_AddObject`, +:c:func:`PyModule_AddIntConstant`, and :c:func:`PyModule_AddStringConstant`. Each of these functions takes a module object, a null-terminated C string containing the name to be added, and a third argument for the value to be assigned to the name. This third argument is, respectively, a Python object, a C long, or a C string. -A wrapper API was added for Unix-style signal handlers. :func:`PyOS_getsig` gets -a signal handler and :func:`PyOS_setsig` will set a new handler. +A wrapper API was added for Unix-style signal handlers. :c:func:`PyOS_getsig` gets +a signal handler and :c:func:`PyOS_setsig` will set a new handler. .. ====================================================================== diff --git a/Doc/whatsnew/2.1.rst b/Doc/whatsnew/2.1.rst index 676da702b3969..f0e1ded75a9d2 100644 --- a/Doc/whatsnew/2.1.rst +++ b/Doc/whatsnew/2.1.rst @@ -692,8 +692,8 @@ applied, and 136 bugs fixed; both figures are likely to be underestimates. Some of the more notable changes are: * A specialized object allocator is now optionally available, that should be - faster than the system :func:`malloc` and have less memory overhead. The - allocator uses C's :func:`malloc` function to get large pools of memory, and + faster than the system :c:func:`malloc` and have less memory overhead. The + allocator uses C's :c:func:`!malloc` function to get large pools of memory, and then fulfills smaller memory requests from these pools. It can be enabled by providing the :option:`!--with-pymalloc` option to the :program:`configure` script; see :file:`Objects/obmalloc.c` for the implementation details. @@ -701,13 +701,13 @@ of the more notable changes are: Authors of C extension modules should test their code with the object allocator enabled, because some incorrect code may break, causing core dumps at runtime. There are a bunch of memory allocation functions in Python's C API that have - previously been just aliases for the C library's :func:`malloc` and - :func:`free`, meaning that if you accidentally called mismatched functions, the + previously been just aliases for the C library's :c:func:`malloc` and + :c:func:`free`, meaning that if you accidentally called mismatched functions, the error wouldn't be noticeable. When the object allocator is enabled, these - functions aren't aliases of :func:`malloc` and :func:`free` any more, and + functions aren't aliases of :c:func:`!malloc` and :c:func:`!free` any more, and calling the wrong function to free memory will get you a core dump. For - example, if memory was allocated using :func:`PyMem_New`, it has to be freed - using :func:`PyMem_Del`, not :func:`free`. A few modules included with Python + example, if memory was allocated using :c:macro:`PyMem_New`, it has to be freed + using :c:func:`PyMem_Del`, not :c:func:`!free`. A few modules included with Python fell afoul of this and had to be fixed; doubtless there are more third-party modules that will have the same problem. @@ -717,7 +717,7 @@ of the more notable changes are: complain about its lack of speed, and because it's often been used as a na?ve benchmark. The :meth:`readline` method of file objects has therefore been rewritten to be much faster. The exact amount of the speedup will vary from - platform to platform depending on how slow the C library's :func:`getc` was, but + platform to platform depending on how slow the C library's :c:func:`!getc` was, but is around 66%, and potentially much faster on some particular operating systems. Tim Peters did much of the benchmarking and coding for this change, motivated by a discussion in comp.lang.python. @@ -770,7 +770,7 @@ of the more notable changes are: reorganization done by Jeremy Hylton. * C extensions which import other modules have been changed to use - :func:`PyImport_ImportModule`, which means that they will use any import hooks + :c:func:`PyImport_ImportModule`, which means that they will use any import hooks that have been installed. This is also encouraged for third-party extensions that need to import some other module from C code. diff --git a/Misc/NEWS.d/3.11.0b1.rst b/Misc/NEWS.d/3.11.0b1.rst index ad52bd14dbfd1..6b601489a7728 100644 --- a/Misc/NEWS.d/3.11.0b1.rst +++ b/Misc/NEWS.d/3.11.0b1.rst @@ -1799,7 +1799,7 @@ The documentation now lists which members of C structs are part of the .. nonce: FIVe9I .. section: Documentation -All docstrings in code snippets are now wrapped into :func:`PyDoc_STR` to +All docstrings in code snippets are now wrapped into :c:macro:`PyDoc_STR` to follow the guideline of `PEP 7's Documentation Strings paragraph `_. Patch by Oleg Iarygin. diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index 51b5c6e35a4f2..1888cae0ba720 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -203,7 +203,7 @@ the interpreter. .. nonce: LYAWlE .. section: Core and Builtins -Bugfix: :func:`PyFunction_GetAnnotations` should return a borrowed +Bugfix: :c:func:`PyFunction_GetAnnotations` should return a borrowed reference. It was returning a new reference. .. diff --git a/Misc/NEWS.d/3.12.0b1.rst b/Misc/NEWS.d/3.12.0b1.rst index acb68db2db912..23540f71a347c 100644 --- a/Misc/NEWS.d/3.12.0b1.rst +++ b/Misc/NEWS.d/3.12.0b1.rst @@ -1205,7 +1205,7 @@ the future, it will raise a ``KeyError``. Fixed a bug where :mod:`pdb` crashes when reading source file with different encoding by replacing :func:`io.open` with :func:`io.open_code`. The new -method would also call into the hook set by :func:`PyFile_SetOpenCodeHook`. +method would also call into the hook set by :c:func:`PyFile_SetOpenCodeHook`. .. diff --git a/Misc/NEWS.d/3.9.0a1.rst b/Misc/NEWS.d/3.9.0a1.rst index 5e6fa6759d577..85473da78bd43 100644 --- a/Misc/NEWS.d/3.9.0a1.rst +++ b/Misc/NEWS.d/3.9.0a1.rst @@ -5686,7 +5686,7 @@ positional argument. .. nonce: zrmgki .. section: C API -Add :func:`PyConfig_SetWideStringList` function. +Add :c:func:`PyConfig_SetWideStringList` function. .. From webhook-mailer at python.org Sat Jul 29 01:56:29 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 29 Jul 2023 05:56:29 -0000 Subject: [Python-checkins] [3.11] gh-107091: Fix some uses of :func: role (GH-107378) (GH-107417) Message-ID: https://github.com/python/cpython/commit/733a2a90466e879fa4b87fcbb4f0a61ac70003dc commit: 733a2a90466e879fa4b87fcbb4f0a61ac70003dc branch: 3.11 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-29T05:56:26Z summary: [3.11] gh-107091: Fix some uses of :func: role (GH-107378) (GH-107417) :c:func: or :c:macro: should be used instead. (cherry picked from commit 413ba8943e2f1d896a0568eb571a041b88589440) files: M Doc/whatsnew/2.0.rst M Doc/whatsnew/2.1.rst M Misc/NEWS.d/3.11.0b1.rst M Misc/NEWS.d/3.9.0a1.rst diff --git a/Doc/whatsnew/2.0.rst b/Doc/whatsnew/2.0.rst index cd9fa89ca8a90..096248e41b849 100644 --- a/Doc/whatsnew/2.0.rst +++ b/Doc/whatsnew/2.0.rst @@ -664,7 +664,7 @@ extra set of parentheses to pass both values as a tuple: ``L.append( (1,2) )``. The earlier versions of these methods were more forgiving because they used an old function in Python's C interface to parse their arguments; 2.0 modernizes -them to use :func:`PyArg_ParseTuple`, the current argument parsing function, +them to use :c:func:`PyArg_ParseTuple`, the current argument parsing function, which provides more helpful error messages and treats multi-argument calls as errors. If you absolutely must use 2.0 but can't fix your code, you can edit :file:`Objects/listobject.c` and define the preprocessor symbol @@ -766,7 +766,7 @@ file, :file:`Include/pyport.h`. Vladimir Marangozov's long-awaited malloc restructuring was completed, to make it easy to have the Python interpreter use a custom allocator instead of C's -standard :func:`malloc`. For documentation, read the comments in +standard :c:func:`malloc`. For documentation, read the comments in :file:`Include/pymem.h` and :file:`Include/objimpl.h`. For the lengthy discussions during which the interface was hammered out, see the web archives of the 'patches' and 'python-dev' lists at python.org. @@ -794,15 +794,15 @@ are generating Python code would run into this limit. A patch by Charles G. Waldman raises the limit from ``2**16`` to ``2**32``. Three new convenience functions intended for adding constants to a module's -dictionary at module initialization time were added: :func:`PyModule_AddObject`, -:func:`PyModule_AddIntConstant`, and :func:`PyModule_AddStringConstant`. Each +dictionary at module initialization time were added: :c:func:`PyModule_AddObject`, +:c:func:`PyModule_AddIntConstant`, and :c:func:`PyModule_AddStringConstant`. Each of these functions takes a module object, a null-terminated C string containing the name to be added, and a third argument for the value to be assigned to the name. This third argument is, respectively, a Python object, a C long, or a C string. -A wrapper API was added for Unix-style signal handlers. :func:`PyOS_getsig` gets -a signal handler and :func:`PyOS_setsig` will set a new handler. +A wrapper API was added for Unix-style signal handlers. :c:func:`PyOS_getsig` gets +a signal handler and :c:func:`PyOS_setsig` will set a new handler. .. ====================================================================== diff --git a/Doc/whatsnew/2.1.rst b/Doc/whatsnew/2.1.rst index 676da702b3969..f0e1ded75a9d2 100644 --- a/Doc/whatsnew/2.1.rst +++ b/Doc/whatsnew/2.1.rst @@ -692,8 +692,8 @@ applied, and 136 bugs fixed; both figures are likely to be underestimates. Some of the more notable changes are: * A specialized object allocator is now optionally available, that should be - faster than the system :func:`malloc` and have less memory overhead. The - allocator uses C's :func:`malloc` function to get large pools of memory, and + faster than the system :c:func:`malloc` and have less memory overhead. The + allocator uses C's :c:func:`!malloc` function to get large pools of memory, and then fulfills smaller memory requests from these pools. It can be enabled by providing the :option:`!--with-pymalloc` option to the :program:`configure` script; see :file:`Objects/obmalloc.c` for the implementation details. @@ -701,13 +701,13 @@ of the more notable changes are: Authors of C extension modules should test their code with the object allocator enabled, because some incorrect code may break, causing core dumps at runtime. There are a bunch of memory allocation functions in Python's C API that have - previously been just aliases for the C library's :func:`malloc` and - :func:`free`, meaning that if you accidentally called mismatched functions, the + previously been just aliases for the C library's :c:func:`malloc` and + :c:func:`free`, meaning that if you accidentally called mismatched functions, the error wouldn't be noticeable. When the object allocator is enabled, these - functions aren't aliases of :func:`malloc` and :func:`free` any more, and + functions aren't aliases of :c:func:`!malloc` and :c:func:`!free` any more, and calling the wrong function to free memory will get you a core dump. For - example, if memory was allocated using :func:`PyMem_New`, it has to be freed - using :func:`PyMem_Del`, not :func:`free`. A few modules included with Python + example, if memory was allocated using :c:macro:`PyMem_New`, it has to be freed + using :c:func:`PyMem_Del`, not :c:func:`!free`. A few modules included with Python fell afoul of this and had to be fixed; doubtless there are more third-party modules that will have the same problem. @@ -717,7 +717,7 @@ of the more notable changes are: complain about its lack of speed, and because it's often been used as a na?ve benchmark. The :meth:`readline` method of file objects has therefore been rewritten to be much faster. The exact amount of the speedup will vary from - platform to platform depending on how slow the C library's :func:`getc` was, but + platform to platform depending on how slow the C library's :c:func:`!getc` was, but is around 66%, and potentially much faster on some particular operating systems. Tim Peters did much of the benchmarking and coding for this change, motivated by a discussion in comp.lang.python. @@ -770,7 +770,7 @@ of the more notable changes are: reorganization done by Jeremy Hylton. * C extensions which import other modules have been changed to use - :func:`PyImport_ImportModule`, which means that they will use any import hooks + :c:func:`PyImport_ImportModule`, which means that they will use any import hooks that have been installed. This is also encouraged for third-party extensions that need to import some other module from C code. diff --git a/Misc/NEWS.d/3.11.0b1.rst b/Misc/NEWS.d/3.11.0b1.rst index ad52bd14dbfd1..6b601489a7728 100644 --- a/Misc/NEWS.d/3.11.0b1.rst +++ b/Misc/NEWS.d/3.11.0b1.rst @@ -1799,7 +1799,7 @@ The documentation now lists which members of C structs are part of the .. nonce: FIVe9I .. section: Documentation -All docstrings in code snippets are now wrapped into :func:`PyDoc_STR` to +All docstrings in code snippets are now wrapped into :c:macro:`PyDoc_STR` to follow the guideline of `PEP 7's Documentation Strings paragraph `_. Patch by Oleg Iarygin. diff --git a/Misc/NEWS.d/3.9.0a1.rst b/Misc/NEWS.d/3.9.0a1.rst index 97e07a643e59f..64410a1d34dfe 100644 --- a/Misc/NEWS.d/3.9.0a1.rst +++ b/Misc/NEWS.d/3.9.0a1.rst @@ -5686,7 +5686,7 @@ positional argument. .. nonce: zrmgki .. section: C API -Add :func:`PyConfig_SetWideStringList` function. +Add :c:func:`PyConfig_SetWideStringList` function. .. From webhook-mailer at python.org Sat Jul 29 02:16:31 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 29 Jul 2023 06:16:31 -0000 Subject: [Python-checkins] [3.12] gh-101100: Sphinx warnings: pick the low hanging fruits (GH-107386) (GH-107419) Message-ID: https://github.com/python/cpython/commit/d514e1439f396d984f4f886e19c47f56e565a7d1 commit: d514e1439f396d984f4f886e19c47f56e565a7d1 branch: 3.12 author: Serhiy Storchaka committer: serhiy-storchaka date: 2023-07-29T09:16:27+03:00 summary: [3.12] gh-101100: Sphinx warnings: pick the low hanging fruits (GH-107386) (GH-107419) (cherry picked from commit f2d07d3289947d10b065b2bb7670c8fb6b6582f2) files: M Doc/c-api/bool.rst M Doc/faq/extending.rst M Doc/howto/curses.rst M Doc/install/index.rst M Doc/library/asyncio-extending.rst M Doc/library/asyncio-future.rst M Doc/library/bz2.rst M Doc/library/code.rst M Doc/library/concurrent.rst M Doc/library/curses.rst M Doc/library/email.charset.rst M Doc/library/email.encoders.rst M Doc/library/email.generator.rst M Doc/library/email.message.rst M Doc/library/email.parser.rst M Doc/library/filecmp.rst M Doc/library/fileinput.rst M Doc/library/graphlib.rst M Doc/library/gzip.rst M Doc/library/hashlib.rst M Doc/library/importlib.resources.abc.rst M Doc/library/json.rst M Doc/library/logging.rst M Doc/library/lzma.rst M Doc/library/msvcrt.rst M Doc/library/netrc.rst M Doc/library/operator.rst M Doc/library/poplib.rst M Doc/library/pprint.rst M Doc/library/pty.rst M Doc/library/sched.rst M Doc/library/selectors.rst M Doc/library/shutil.rst M Doc/library/stat.rst M Doc/library/stdtypes.rst M Doc/library/sysconfig.rst M Doc/library/textwrap.rst M Doc/library/urllib.error.rst M Doc/library/winreg.rst M Doc/library/winsound.rst M Doc/library/xml.rst M Doc/library/xml.sax.handler.rst M Doc/library/xml.sax.utils.rst M Doc/library/xmlrpc.rst M Doc/reference/datamodel.rst M Doc/reference/expressions.rst M Doc/reference/lexical_analysis.rst M Doc/tools/.nitignore M Doc/tutorial/errors.rst M Doc/tutorial/interactive.rst M Doc/whatsnew/3.0.rst M Misc/NEWS.d/3.11.0a6.rst diff --git a/Doc/c-api/bool.rst b/Doc/c-api/bool.rst index b2d8f2124fc20..b14fa6a0a982e 100644 --- a/Doc/c-api/bool.rst +++ b/Doc/c-api/bool.rst @@ -11,6 +11,12 @@ creation and deletion functions don't apply to booleans. The following macros are available, however. +.. c:var:: PyTypeObject PyBool_Type + + This instance of :c:type:`PyTypeObject` represents the Python boolean type; it + is the same object as :class:`bool` in the Python layer. + + .. c:function:: int PyBool_Check(PyObject *o) Return true if *o* is of type :c:data:`PyBool_Type`. This function always diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst index bc3080f60ee23..2a8b976925d04 100644 --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -81,13 +81,13 @@ How do I extract C values from a Python object? That depends on the object's type. If it's a tuple, :c:func:`PyTuple_Size` returns its length and :c:func:`PyTuple_GetItem` returns the item at a specified -index. Lists have similar functions, :c:func:`PyListSize` and +index. Lists have similar functions, :c:func:`PyList_Size` and :c:func:`PyList_GetItem`. For bytes, :c:func:`PyBytes_Size` returns its length and :c:func:`PyBytes_AsStringAndSize` provides a pointer to its value and its length. Note that Python bytes objects may contain null bytes so C's -:c:func:`strlen` should not be used. +:c:func:`!strlen` should not be used. To test the type of an object, first make sure it isn't ``NULL``, and then use :c:func:`PyBytes_Check`, :c:func:`PyTuple_Check`, :c:func:`PyList_Check`, etc. diff --git a/Doc/howto/curses.rst b/Doc/howto/curses.rst index a3068d86d85bc..4828e2fa29bd2 100644 --- a/Doc/howto/curses.rst +++ b/Doc/howto/curses.rst @@ -527,7 +527,7 @@ If you're in doubt about the detailed behavior of the curses functions, consult the manual pages for your curses implementation, whether it's ncurses or a proprietary Unix vendor's. The manual pages will document any quirks, and provide complete lists of all the -functions, attributes, and :const:`ACS_\*` characters available to +functions, attributes, and :ref:`ACS_\* ` characters available to you. Because the curses API is so large, some functions aren't supported in diff --git a/Doc/install/index.rst b/Doc/install/index.rst index 443c75b7684fb..ffb4a202fe89f 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -778,7 +778,7 @@ Notes: (2) On Unix, if the :envvar:`HOME` environment variable is not defined, the user's - home directory will be determined with the :func:`getpwuid` function from the + home directory will be determined with the :func:`~pwd.getpwuid` function from the standard :mod:`pwd` module. This is done by the :func:`os.path.expanduser` function used by Distutils. diff --git a/Doc/library/asyncio-extending.rst b/Doc/library/asyncio-extending.rst index 8ffd356f2d1cc..e7b293f484f8d 100644 --- a/Doc/library/asyncio-extending.rst +++ b/Doc/library/asyncio-extending.rst @@ -69,7 +69,7 @@ Task lifetime support ===================== A third party task implementation should call the following functions to keep a task -visible by :func:`asyncio.get_tasks` and :func:`asyncio.current_task`: +visible by :func:`asyncio.all_tasks` and :func:`asyncio.current_task`: .. function:: _register_task(task) diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index 70cec9b2f9024..893ae5518f757 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -276,4 +276,4 @@ the Future has a result:: :func:`concurrent.futures.as_completed` functions. - :meth:`asyncio.Future.cancel` accepts an optional ``msg`` argument, - but :func:`concurrent.futures.cancel` does not. + but :meth:`concurrent.futures.Future.cancel` does not. diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst index 32df99869eb53..ec4aeaa04395a 100644 --- a/Doc/library/bz2.rst +++ b/Doc/library/bz2.rst @@ -87,7 +87,8 @@ The :mod:`bz2` module contains: compressed streams. :class:`BZ2File` provides all of the members specified by the - :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`. + :class:`io.BufferedIOBase`, except for :meth:`~io.BufferedIOBase.detach` + and :meth:`~io.IOBase.truncate`. Iteration and the :keyword:`with` statement are supported. :class:`BZ2File` also provides the following method: diff --git a/Doc/library/code.rst b/Doc/library/code.rst index 538e5afc7822a..3d7f43c86a055 100644 --- a/Doc/library/code.rst +++ b/Doc/library/code.rst @@ -163,12 +163,12 @@ interpreter objects as well as the following additions. Push a line of source text to the interpreter. The line should not have a trailing newline; it may have internal newlines. The line is appended to a - buffer and the interpreter's :meth:`runsource` method is called with the + buffer and the interpreter's :meth:`~InteractiveInterpreter.runsource` method is called with the concatenated contents of the buffer as source. If this indicates that the command was executed or invalid, the buffer is reset; otherwise, the command is incomplete, and the buffer is left as it was after the line was appended. The return value is ``True`` if more input is required, ``False`` if the line was - dealt with in some way (this is the same as :meth:`runsource`). + dealt with in some way (this is the same as :meth:`!runsource`). .. method:: InteractiveConsole.resetbuffer() diff --git a/Doc/library/concurrent.rst b/Doc/library/concurrent.rst index 2eba536512580..8caea78bbb57e 100644 --- a/Doc/library/concurrent.rst +++ b/Doc/library/concurrent.rst @@ -1,5 +1,5 @@ -The :mod:`concurrent` package -============================= +The :mod:`!concurrent` package +============================== Currently, there is only one module in this package: diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst index 391c81a844d3e..9ab67c2197539 100644 --- a/Doc/library/curses.rst +++ b/Doc/library/curses.rst @@ -1648,6 +1648,8 @@ keys); also, the following keypad mappings are standard: | :kbd:`Page Down` | KEY_NPAGE | +------------------+-----------+ +.. _curses-acs-codes: + The following table lists characters from the alternate character set. These are inherited from the VT100 terminal, and will generally be available on software emulations such as X terminals. When there is no graphic available, curses diff --git a/Doc/library/email.charset.rst b/Doc/library/email.charset.rst index adbe6c1c7d29b..aa0134412f3a6 100644 --- a/Doc/library/email.charset.rst +++ b/Doc/library/email.charset.rst @@ -150,7 +150,7 @@ Import this class from the :mod:`email.charset` module. .. method:: __str__() Returns *input_charset* as a string coerced to lower - case. :meth:`__repr__` is an alias for :meth:`__str__`. + case. :meth:`!__repr__` is an alias for :meth:`!__str__`. .. method:: __eq__(other) diff --git a/Doc/library/email.encoders.rst b/Doc/library/email.encoders.rst index 5d68b104f3a45..3bd377e33f6c1 100644 --- a/Doc/library/email.encoders.rst +++ b/Doc/library/email.encoders.rst @@ -25,7 +25,7 @@ is especially true for :mimetype:`image/\*` and :mimetype:`text/\*` type message containing binary data. The :mod:`email` package provides some convenient encoders in its -:mod:`encoders` module. These encoders are actually used by the +:mod:`~email.encoders` module. These encoders are actually used by the :class:`~email.mime.audio.MIMEAudio` and :class:`~email.mime.image.MIMEImage` class constructors to provide default encodings. All encoder functions take exactly one argument, the message object to encode. They usually extract the diff --git a/Doc/library/email.generator.rst b/Doc/library/email.generator.rst index 34ad7b7f200af..91d9d69a63d73 100644 --- a/Doc/library/email.generator.rst +++ b/Doc/library/email.generator.rst @@ -274,9 +274,9 @@ in with information about the part. .. rubric:: Footnotes .. [#] This statement assumes that you use the appropriate setting for - ``unixfrom``, and that there are no :mod:`policy` settings calling for + ``unixfrom``, and that there are no :mod:`email.policy` settings calling for automatic adjustments (for example, - :attr:`~email.policy.Policy.refold_source` must be ``none``, which is + :attr:`~email.policy.EmailPolicy.refold_source` must be ``none``, which is *not* the default). It is also not 100% true, since if the message does not conform to the RFC standards occasionally information about the exact original text is lost during parsing error recovery. It is a goal diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst index 5e0509f418119..225f498781fa8 100644 --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -67,7 +67,7 @@ message objects. with the base :class:`~email.message.Message` class *maxheaderlen* is accepted, but defaults to ``None``, which means that by default the line length is controlled by the - :attr:`~email.policy.EmailPolicy.max_line_length` of the policy. The + :attr:`~email.policy.Policy.max_line_length` of the policy. The *policy* argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified *policy* will be @@ -213,7 +213,7 @@ message objects. del msg['subject'] msg['subject'] = 'Python roolz!' - If the :mod:`policy` defines certain headers to be unique (as the standard + If the :mod:`policy ` defines certain headers to be unique (as the standard policies do), this method may raise a :exc:`ValueError` when an attempt is made to assign a value to such a header when one already exists. This behavior is intentional for consistency's sake, but do not depend on it @@ -378,7 +378,7 @@ message objects. deprecated. Note that existing parameter values of headers may be accessed through - the :attr:`~email.headerregistry.BaseHeader.params` attribute of the + the :attr:`~email.headerregistry.ParameterizedMIMEHeader.params` attribute of the header value (for example, ``msg['Content-Type'].params['charset']``). .. versionchanged:: 3.4 ``replace`` keyword was added. @@ -691,7 +691,7 @@ message objects. .. method:: clear_content() - Remove the payload and all of the :exc:`Content-` headers, leaving + Remove the payload and all of the :mailheader:`!Content-` headers, leaving all other headers intact and in their original order. diff --git a/Doc/library/email.parser.rst b/Doc/library/email.parser.rst index d9a61616bbbdf..dda0466a6afa7 100644 --- a/Doc/library/email.parser.rst +++ b/Doc/library/email.parser.rst @@ -39,9 +39,9 @@ returns the root object when you close the parser. Note that the parser can be extended in limited ways, and of course you can implement your own parser completely from scratch. All of the logic that connects the :mod:`email` package's bundled parser and the -:class:`~email.message.EmailMessage` class is embodied in the :mod:`policy` +:class:`~email.message.EmailMessage` class is embodied in the :class:`~email.policy.Policy` class, so a custom parser can create message object trees any way it finds -necessary by implementing custom versions of the appropriate :mod:`policy` +necessary by implementing custom versions of the appropriate :class:`!Policy` methods. diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst index 0efb4897a1eb8..dfe4b7c59fd57 100644 --- a/Doc/library/filecmp.rst +++ b/Doc/library/filecmp.rst @@ -100,7 +100,7 @@ The :class:`dircmp` class used to get various bits of information about the directory trees being compared. - Note that via :meth:`__getattr__` hooks, all attributes are computed lazily, + Note that via :meth:`~object.__getattr__` hooks, all attributes are computed lazily, so there is no speed penalty if only those attributes which are lightweight to compute are used. diff --git a/Doc/library/fileinput.rst b/Doc/library/fileinput.rst index 4bc868759f202..f93e9a58791ee 100644 --- a/Doc/library/fileinput.rst +++ b/Doc/library/fileinput.rst @@ -177,7 +177,7 @@ available for subclassing as well: The keyword-only parameter *encoding* and *errors* are added. .. versionchanged:: 3.11 - The ``'rU'`` and ``'U'`` modes and the :meth:`__getitem__` method have + The ``'rU'`` and ``'U'`` modes and the :meth:`!__getitem__` method have been removed. diff --git a/Doc/library/graphlib.rst b/Doc/library/graphlib.rst index fe7932e7a61cb..fdd8f39ef4e1c 100644 --- a/Doc/library/graphlib.rst +++ b/Doc/library/graphlib.rst @@ -115,7 +115,7 @@ :meth:`TopologicalSorter.done` is less than the number that have been returned by :meth:`TopologicalSorter.get_ready`. - The :meth:`~TopologicalSorter.__bool__` method of this class defers to + The :meth:`~object.__bool__` method of this class defers to this function, so instead of:: if ts.is_active(): @@ -204,7 +204,7 @@ The :mod:`graphlib` module defines the following exception classes: in the working graph. If multiple cycles exist, only one undefined choice among them will be reported and included in the exception. - The detected cycle can be accessed via the second element in the :attr:`~CycleError.args` + The detected cycle can be accessed via the second element in the :attr:`~BaseException.args` attribute of the exception instance and consists in a list of nodes, such that each node is, in the graph, an immediate predecessor of the next node in the list. In the reported list, the first and the last node will be the same, to make it clear that it is cyclic. diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst index 979b39a3a5abb..60236a1190e42 100644 --- a/Doc/library/gzip.rst +++ b/Doc/library/gzip.rst @@ -70,7 +70,7 @@ The module defines the following items: .. class:: GzipFile(filename=None, mode=None, compresslevel=9, fileobj=None, mtime=None) Constructor for the :class:`GzipFile` class, which simulates most of the - methods of a :term:`file object`, with the exception of the :meth:`truncate` + methods of a :term:`file object`, with the exception of the :meth:`~io.IOBase.truncate` method. At least one of *fileobj* and *filename* must be given a non-trivial value. @@ -113,7 +113,7 @@ The module defines the following items: :class:`GzipFile` supports the :class:`io.BufferedIOBase` interface, including iteration and the :keyword:`with` statement. Only the - :meth:`truncate` method isn't implemented. + :meth:`~io.IOBase.truncate` method isn't implemented. :class:`GzipFile` also provides the following method and attribute: diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst index 8102767a43d6d..69fb79b49ca2a 100644 --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -244,7 +244,7 @@ by the SHAKE algorithm. .. method:: shake.digest(length) - Return the digest of the data passed to the :meth:`update` method so far. + Return the digest of the data passed to the :meth:`~hash.update` method so far. This is a bytes object of size *length* which may contain bytes in the whole range from 0 to 255. @@ -507,9 +507,9 @@ Simple hashing To calculate hash of some data, you should first construct a hash object by calling the appropriate constructor function (:func:`blake2b` or -:func:`blake2s`), then update it with the data by calling :meth:`update` on the +:func:`blake2s`), then update it with the data by calling :meth:`~hash.update` on the object, and, finally, get the digest out of the object by calling -:meth:`digest` (or :meth:`hexdigest` for hex-encoded string). +:meth:`~hash.digest` (or :meth:`~hash.hexdigest` for hex-encoded string). >>> from hashlib import blake2b >>> h = blake2b() diff --git a/Doc/library/importlib.resources.abc.rst b/Doc/library/importlib.resources.abc.rst index b0e75737137f2..65c42858bbbb7 100644 --- a/Doc/library/importlib.resources.abc.rst +++ b/Doc/library/importlib.resources.abc.rst @@ -145,10 +145,10 @@ An abstract base class for resource readers capable of serving the :meth:`importlib.resources.files` interface. Subclasses - :class:`importlib.resources.abc.ResourceReader` and provides - concrete implementations of the :class:`importlib.resources.abc.ResourceReader`'s + :class:`ResourceReader` and provides + concrete implementations of the :class:`!ResourceReader`'s abstract methods. Therefore, any loader supplying - :class:`importlib.abc.TraversableResources` also supplies ResourceReader. + :class:`!TraversableResources` also supplies :class:`!ResourceReader`. Loaders that wish to support resource reading are expected to implement this interface. diff --git a/Doc/library/json.rst b/Doc/library/json.rst index 35a08995487c1..6c3059381776c 100644 --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -192,7 +192,7 @@ Basic Usage dictionaries will be sorted by key. To use a custom :class:`JSONEncoder` subclass (e.g. one that overrides the - :meth:`default` method to serialize additional types), specify it with the + :meth:`~JSONEncoder.default` method to serialize additional types), specify it with the *cls* kwarg; otherwise :class:`JSONEncoder` is used. .. versionchanged:: 3.6 @@ -422,7 +422,7 @@ Encoders and Decoders Added support for int- and float-derived Enum classes. To extend this to recognize other objects, subclass and implement a - :meth:`default` method with another method that returns a serializable object + :meth:`~JSONEncoder.default` method with another method that returns a serializable object for ``o`` if possible, otherwise it should call the superclass implementation (to raise :exc:`TypeError`). @@ -483,7 +483,7 @@ Encoders and Decoders :exc:`TypeError`). For example, to support arbitrary iterators, you could implement - :meth:`default` like this:: + :meth:`~JSONEncoder.default` like this:: def default(self, o): try: diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index 22412e1a2113b..70a9c8b9aa86e 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -397,21 +397,21 @@ have specific values relative to the predefined levels. If you define a level with the same numeric value, it overwrites the predefined value; the predefined name is lost. -+--------------+---------------+ -| Level | Numeric value | -+==============+===============+ -| ``CRITICAL`` | 50 | -+--------------+---------------+ -| ``ERROR`` | 40 | -+--------------+---------------+ -| ``WARNING`` | 30 | -+--------------+---------------+ -| ``INFO`` | 20 | -+--------------+---------------+ -| ``DEBUG`` | 10 | -+--------------+---------------+ -| ``NOTSET`` | 0 | -+--------------+---------------+ ++-----------------------+---------------+ +| Level | Numeric value | ++=======================+===============+ +| .. py:data:: CRITICAL | 50 | ++-----------------------+---------------+ +| .. py:data:: ERROR | 40 | ++-----------------------+---------------+ +| .. py:data:: WARNING | 30 | ++-----------------------+---------------+ +| .. py:data:: INFO | 20 | ++-----------------------+---------------+ +| .. py:data:: DEBUG | 10 | ++-----------------------+---------------+ +| .. py:data:: NOTSET | 0 | ++-----------------------+---------------+ .. _handler: diff --git a/Doc/library/lzma.rst b/Doc/library/lzma.rst index 868d4dcfb6c99..434e7ac906118 100644 --- a/Doc/library/lzma.rst +++ b/Doc/library/lzma.rst @@ -100,7 +100,8 @@ Reading and writing compressed files *filters* arguments have the same meanings as for :class:`LZMACompressor`. :class:`LZMAFile` supports all the members specified by - :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`. + :class:`io.BufferedIOBase`, except for :meth:`~io.BufferedIOBase.detach` + and :meth:`~io.IOBase.truncate`. Iteration and the :keyword:`with` statement are supported. The following method is also provided: diff --git a/Doc/library/msvcrt.rst b/Doc/library/msvcrt.rst index 42fffee6a0f44..32693e3d007c0 100644 --- a/Doc/library/msvcrt.rst +++ b/Doc/library/msvcrt.rst @@ -38,7 +38,7 @@ File Operations Lock part of a file based on file descriptor *fd* from the C runtime. Raises :exc:`OSError` on failure. The locked region of the file extends from the current file position for *nbytes* bytes, and may continue beyond the end of the - file. *mode* must be one of the :const:`LK_\*` constants listed below. Multiple + file. *mode* must be one of the :const:`!LK_\*` constants listed below. Multiple regions in a file may be locked at the same time, but may not overlap. Adjacent regions are not merged; they must be unlocked individually. diff --git a/Doc/library/netrc.rst b/Doc/library/netrc.rst index 88265d9b9e9e9..c36e5cfecfc6a 100644 --- a/Doc/library/netrc.rst +++ b/Doc/library/netrc.rst @@ -51,9 +51,19 @@ the Unix :program:`ftp` program and other FTP clients. Exception raised by the :class:`~netrc.netrc` class when syntactical errors are encountered in source text. Instances of this exception provide three - interesting attributes: :attr:`msg` is a textual explanation of the error, - :attr:`filename` is the name of the source file, and :attr:`lineno` gives the - line number on which the error was found. + interesting attributes: + + .. attribute:: msg + + Textual explanation of the error. + + .. attribute:: filename + + The name of the source file. + + .. attribute:: lineno + + The line number on which the error was found. .. _netrc-objects: diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst index dab4de9eb6abb..57c67bcf3aa12 100644 --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -59,9 +59,9 @@ truth tests, identity tests, and boolean operations: __not__(obj) Return the outcome of :keyword:`not` *obj*. (Note that there is no - :meth:`__not__` method for object instances; only the interpreter core defines - this operation. The result is affected by the :meth:`__bool__` and - :meth:`__len__` methods.) + :meth:`!__not__` method for object instances; only the interpreter core defines + this operation. The result is affected by the :meth:`~object.__bool__` and + :meth:`~object.__len__` methods.) .. function:: truth(obj) diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index fbd5e150b4cd5..943eb21f6eec0 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -148,7 +148,7 @@ A :class:`POP3` instance has the following methods: .. method:: POP3.pass_(password) Send password, response includes message count and mailbox size. Note: the - mailbox on the server is locked until :meth:`~poplib.quit` is called. + mailbox on the server is locked until :meth:`~POP3.quit` is called. .. method:: POP3.apop(user, secret) diff --git a/Doc/library/pprint.rst b/Doc/library/pprint.rst index d8269ef48cb36..e883acd67d6c7 100644 --- a/Doc/library/pprint.rst +++ b/Doc/library/pprint.rst @@ -45,7 +45,7 @@ The :mod:`pprint` module defines one class: several keyword parameters. *stream* (default ``sys.stdout``) is a :term:`file-like object` to - which the output will be written by calling its :meth:`write` method. + which the output will be written by calling its :meth:`!write` method. If both *stream* and ``sys.stdout`` are ``None``, then :meth:`~PrettyPrinter.pprint` silently returns. diff --git a/Doc/library/pty.rst b/Doc/library/pty.rst index 7f4da41e93802..ad4981c97119f 100644 --- a/Doc/library/pty.rst +++ b/Doc/library/pty.rst @@ -71,7 +71,7 @@ The :mod:`pty` module defines the following functions: Return the exit status value from :func:`os.waitpid` on the child process. - :func:`waitstatus_to_exitcode` can be used to convert the exit status into + :func:`os.waitstatus_to_exitcode` can be used to convert the exit status into an exit code. .. audit-event:: pty.spawn argv pty.spawn diff --git a/Doc/library/sched.rst b/Doc/library/sched.rst index 04215d31ba10c..01bac5afd0b9b 100644 --- a/Doc/library/sched.rst +++ b/Doc/library/sched.rst @@ -115,7 +115,7 @@ Scheduler Objects .. method:: scheduler.run(blocking=True) - Run all scheduled events. This method will wait (using the :func:`delayfunc` + Run all scheduled events. This method will wait (using the *delayfunc* function passed to the constructor) for the next event, then execute it and so on until there are no more scheduled events. diff --git a/Doc/library/selectors.rst b/Doc/library/selectors.rst index 0deb15cf4c503..dd50bac37e49b 100644 --- a/Doc/library/selectors.rst +++ b/Doc/library/selectors.rst @@ -60,9 +60,9 @@ constants below: +-----------------------+-----------------------------------------------+ | Constant | Meaning | +=======================+===============================================+ - | :const:`EVENT_READ` | Available for read | + | .. data:: EVENT_READ | Available for read | +-----------------------+-----------------------------------------------+ - | :const:`EVENT_WRITE` | Available for write | + | .. data:: EVENT_WRITE | Available for write | +-----------------------+-----------------------------------------------+ @@ -132,8 +132,8 @@ constants below: Change a registered file object's monitored events or attached data. - This is equivalent to :meth:`BaseSelector.unregister(fileobj)` followed - by :meth:`BaseSelector.register(fileobj, events, data)`, except that it + This is equivalent to ``BaseSelector.unregister(fileobj)`` followed + by ``BaseSelector.register(fileobj, events, data)``, except that it can be implemented more efficiently. This returns a new :class:`SelectorKey` instance, or raises a diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 7699d22a72aaf..4390a8e22306f 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -369,7 +369,7 @@ Directory and files operations If *copy_function* is given, it must be a callable that takes two arguments *src* and *dst*, and will be used to copy *src* to *dst* if :func:`os.rename` cannot be used. If the source is a directory, - :func:`copytree` is called, passing it the :func:`copy_function`. The + :func:`copytree` is called, passing it the *copy_function*. The default *copy_function* is :func:`copy2`. Using :func:`~shutil.copy` as the *copy_function* allows the move to succeed when it is not possible to also copy the metadata, at the expense of not copying any of the metadata. diff --git a/Doc/library/stat.rst b/Doc/library/stat.rst index 083dc5e3bcfd6..77538514598a5 100644 --- a/Doc/library/stat.rst +++ b/Doc/library/stat.rst @@ -13,8 +13,8 @@ The :mod:`stat` module defines constants and functions for interpreting the results of :func:`os.stat`, :func:`os.fstat` and :func:`os.lstat` (if they -exist). For complete details about the :c:func:`stat`, :c:func:`fstat` and -:c:func:`lstat` calls, consult the documentation for your system. +exist). For complete details about the :c:func:`stat`, :c:func:`!fstat` and +:c:func:`!lstat` calls, consult the documentation for your system. .. versionchanged:: 3.4 The stat module is backed by a C implementation. @@ -89,9 +89,9 @@ mode: .. function:: S_IFMT(mode) Return the portion of the file's mode that describes the file type (used by the - :func:`S_IS\*` functions above). + :func:`!S_IS\*` functions above). -Normally, you would use the :func:`os.path.is\*` functions for testing the type +Normally, you would use the :func:`!os.path.is\*` functions for testing the type of a file; the functions here are useful when you are doing multiple tests of the same file and wish to avoid the overhead of the :c:func:`stat` system call for each test. These are also useful when checking for information about a file diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index d0233e3f35394..804342386abd7 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -44,7 +44,7 @@ Any object can be tested for truth value, for use in an :keyword:`if` or .. index:: single: true By default, an object is considered true unless its class defines either a -:meth:`__bool__` method that returns ``False`` or a :meth:`__len__` method that +:meth:`~object.__bool__` method that returns ``False`` or a :meth:`__len__` method that returns zero, when called with the object. [1]_ Here are most of the built-in objects considered false: diff --git a/Doc/library/sysconfig.rst b/Doc/library/sysconfig.rst index 839c2c015b49a..c805c50ffc689 100644 --- a/Doc/library/sysconfig.rst +++ b/Doc/library/sysconfig.rst @@ -69,7 +69,7 @@ Python uses an installation scheme that differs depending on the platform and on the installation options. These schemes are stored in :mod:`sysconfig` under unique identifiers based on the value returned by :const:`os.name`. -Every new component that is installed using :mod:`distutils` or a +Every new component that is installed using :mod:`!distutils` or a Distutils-based system will follow the same scheme to copy its file in the right places. diff --git a/Doc/library/textwrap.rst b/Doc/library/textwrap.rst index 1a9d5f98f78a7..a150eefbf932e 100644 --- a/Doc/library/textwrap.rst +++ b/Doc/library/textwrap.rst @@ -60,7 +60,7 @@ functions should be good enough; otherwise, you should use an instance of First the whitespace in *text* is collapsed (all whitespace is replaced by single spaces). If the result fits in the *width*, it is returned. Otherwise, enough words are dropped from the end so that the remaining words - plus the :attr:`placeholder` fit within :attr:`width`:: + plus the :attr:`.placeholder` fit within :attr:`.width`:: >>> textwrap.shorten("Hello world!", width=12) 'Hello world!' @@ -173,7 +173,7 @@ hyphenated words; only then will long words be broken if necessary, unless .. attribute:: expand_tabs (default: ``True``) If true, then all tab characters in *text* will be - expanded to spaces using the :meth:`expandtabs` method of *text*. + expanded to spaces using the :meth:`~str.expandtabs` method of *text*. .. attribute:: tabsize diff --git a/Doc/library/urllib.error.rst b/Doc/library/urllib.error.rst index 3adbdd2613227..a5bcb5b1e643b 100644 --- a/Doc/library/urllib.error.rst +++ b/Doc/library/urllib.error.rst @@ -72,6 +72,8 @@ The following exceptions are raised by :mod:`urllib.error` as appropriate: This exception is raised when the :func:`~urllib.request.urlretrieve` function detects that the amount of the downloaded data is less than the expected amount (given by - the *Content-Length* header). The :attr:`content` attribute stores the - downloaded (and supposedly truncated) data. + the *Content-Length* header). + .. attribute:: content + + The downloaded (and supposedly truncated) data. diff --git a/Doc/library/winreg.rst b/Doc/library/winreg.rst index 4ab671817710d..06bd4d87eb03c 100644 --- a/Doc/library/winreg.rst +++ b/Doc/library/winreg.rst @@ -288,7 +288,7 @@ This module offers the following functions: table (FAT) file system, the filename may not have an extension. A call to :func:`LoadKey` fails if the calling process does not have the - :const:`SE_RESTORE_PRIVILEGE` privilege. Note that privileges are different + :c:data:`!SE_RESTORE_PRIVILEGE` privilege. Note that privileges are different from permissions -- see the `RegLoadKey documentation `__ for more details. @@ -414,7 +414,7 @@ This module offers the following functions: If *key* represents a key on a remote computer, the path described by *file_name* is relative to the remote computer. The caller of this method must - possess the :const:`SeBackupPrivilege` security privilege. Note that + possess the **SeBackupPrivilege** security privilege. Note that privileges are different than permissions -- see the `Conflicts Between User Rights and Permissions documentation `__ @@ -536,7 +536,7 @@ This module offers the following functions: Constants ------------------ -The following constants are defined for use in many :mod:`_winreg` functions. +The following constants are defined for use in many :mod:`winreg` functions. .. _hkey-constants: @@ -745,7 +745,7 @@ All registry functions in this module return one of these objects. All registry functions in this module which accept a handle object also accept an integer, however, use of the handle object is encouraged. -Handle objects provide semantics for :meth:`__bool__` -- thus :: +Handle objects provide semantics for :meth:`~object.__bool__` -- thus :: if handle: print("Yes") diff --git a/Doc/library/winsound.rst b/Doc/library/winsound.rst index 372f792a0f938..370c5216652ba 100644 --- a/Doc/library/winsound.rst +++ b/Doc/library/winsound.rst @@ -24,7 +24,7 @@ provided by Windows platforms. It includes functions and several constants. .. function:: PlaySound(sound, flags) - Call the underlying :c:func:`PlaySound` function from the Platform API. The + Call the underlying :c:func:`!PlaySound` function from the Platform API. The *sound* parameter may be a filename, a system sound alias, audio data as a :term:`bytes-like object`, or ``None``. Its interpretation depends on the value of *flags*, which can be a bitwise ORed @@ -35,7 +35,7 @@ provided by Windows platforms. It includes functions and several constants. .. function:: MessageBeep(type=MB_OK) - Call the underlying :c:func:`MessageBeep` function from the Platform API. This + Call the underlying :c:func:`!MessageBeep` function from the Platform API. This plays a sound as specified in the registry. The *type* argument specifies which sound to play; possible values are ``-1``, ``MB_ICONASTERISK``, ``MB_ICONEXCLAMATION``, ``MB_ICONHAND``, ``MB_ICONQUESTION``, and ``MB_OK``, all diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index cf30de67719b9..1e49b6568dfc2 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -75,10 +75,10 @@ decompression bomb Safe Safe Safe potential reliance on system-provided libraries. Check :const:`pyexpat.EXPAT_VERSION`. 2. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a - :exc:`ParserError` when an entity occurs. + :exc:`~xml.etree.ElementTree.ParseError` when an entity occurs. 3. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns the unexpanded entity verbatim. -4. :mod:`xmlrpclib` doesn't expand external entities and omits them. +4. :mod:`xmlrpc.client` doesn't expand external entities and omits them. 5. Since Python 3.7.1, external general entities are no longer processed by default. @@ -119,8 +119,8 @@ all known attack vectors with examples and references. .. _defusedxml-package: -The :mod:`defusedxml` Package ------------------------------------------------------- +The :mod:`!defusedxml` Package +------------------------------ `defusedxml`_ is a pure Python package with modified subclasses of all stdlib XML parsers that prevent any potentially malicious operation. Use of this diff --git a/Doc/library/xml.sax.handler.rst b/Doc/library/xml.sax.handler.rst index 719ce5ab1bcf6..e2f28e3244cb0 100644 --- a/Doc/library/xml.sax.handler.rst +++ b/Doc/library/xml.sax.handler.rst @@ -393,7 +393,7 @@ implements this interface, then register the object with your :class:`~xml.sax.xmlreader.XMLReader`, the parser will call the methods in your object to report all warnings and errors. There are three levels of errors available: warnings, (possibly) recoverable errors, -and unrecoverable errors. All methods take a :exc:`SAXParseException` as the +and unrecoverable errors. All methods take a :exc:`~xml.sax.SAXParseException` as the only parameter. Errors and warnings may be converted to an exception by raising the passed-in exception object. diff --git a/Doc/library/xml.sax.utils.rst b/Doc/library/xml.sax.utils.rst index ab4606bcf9fe6..e57e76dcac782 100644 --- a/Doc/library/xml.sax.utils.rst +++ b/Doc/library/xml.sax.utils.rst @@ -92,5 +92,5 @@ or as base classes. reading. The input source can be given as a string, a file-like object, or an :class:`~xml.sax.xmlreader.InputSource` object; parsers will use this function to implement the polymorphic *source* argument to their - :meth:`parse` method. + :meth:`~xml.sax.xmlreader.XMLReader.parse` method. diff --git a/Doc/library/xmlrpc.rst b/Doc/library/xmlrpc.rst index ae68157b0f63c..5f0a2cf68d01f 100644 --- a/Doc/library/xmlrpc.rst +++ b/Doc/library/xmlrpc.rst @@ -1,5 +1,5 @@ -:mod:`xmlrpc` --- XMLRPC server and client modules -================================================== +:mod:`!xmlrpc` --- XMLRPC server and client modules +=================================================== XML-RPC is a Remote Procedure Call method that uses XML passed via HTTP as a transport. With it, a client can call methods with parameters on a remote diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index ce36bff89424c..4bc49a5a83d7a 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1594,9 +1594,9 @@ Basic customization Called to implement truth value testing and the built-in operation ``bool()``; should return ``False`` or ``True``. When this method is not - defined, :meth:`__len__` is called, if it is defined, and the object is + defined, :meth:`~object.__len__` is called, if it is defined, and the object is considered true if its result is nonzero. If a class defines neither - :meth:`__len__` nor :meth:`__bool__`, all its instances are considered + :meth:`!__len__` nor :meth:`!__bool__`, all its instances are considered true. @@ -2494,7 +2494,7 @@ through the object's keys; for sequences, it should iterate through the values. Called to implement the built-in function :func:`len`. Should return the length of the object, an integer ``>=`` 0. Also, an object that doesn't define a - :meth:`__bool__` method and whose :meth:`__len__` method returns zero is + :meth:`~object.__bool__` method and whose :meth:`!__len__` method returns zero is considered to be false in a Boolean context. .. impl-detail:: @@ -2503,7 +2503,7 @@ through the object's keys; for sequences, it should iterate through the values. If the length is larger than :data:`!sys.maxsize` some features (such as :func:`len`) may raise :exc:`OverflowError`. To prevent raising :exc:`!OverflowError` by truth value testing, an object must define a - :meth:`__bool__` method. + :meth:`~object.__bool__` method. .. method:: object.__length_hint__(self) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 08dcc8095bee8..5d7a36aa8ff52 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1717,7 +1717,7 @@ control flow statements, the following values are interpreted as false: ``False``, ``None``, numeric zero of all types, and empty strings and containers (including strings, tuples, lists, dictionaries, sets and frozensets). All other values are interpreted as true. User-defined objects can customize their -truth value by providing a :meth:`__bool__` method. +truth value by providing a :meth:`~object.__bool__` method. .. index:: pair: operator; not diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index dde7ba1d941dc..83cd4402a36cf 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -787,7 +787,7 @@ is converted before formatting. Conversion ``'!s'`` calls :func:`str` on the result, ``'!r'`` calls :func:`repr`, and ``'!a'`` calls :func:`ascii`. The result is then formatted using the :func:`format` protocol. The -format specifier is passed to the :meth:`__format__` method of the +format specifier is passed to the :meth:`~object.__format__` method of the expression or conversion result. An empty string is passed when the format specifier is omitted. The formatted result is then included in the final value of the whole string. diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 899e98acec8ad..b3ef4a6373195 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -4,7 +4,6 @@ # to help avoid merge conflicts. Doc/c-api/arg.rst -Doc/c-api/bool.rst Doc/c-api/buffer.rst Doc/c-api/datetime.rst Doc/c-api/descriptor.rst @@ -31,19 +30,16 @@ Doc/c-api/unicode.rst Doc/extending/extending.rst Doc/extending/newtypes.rst Doc/faq/design.rst -Doc/faq/extending.rst Doc/faq/gui.rst Doc/faq/library.rst Doc/faq/programming.rst Doc/glossary.rst -Doc/howto/curses.rst Doc/howto/descriptor.rst Doc/howto/enum.rst Doc/howto/isolating-extensions.rst Doc/howto/logging-cookbook.rst Doc/howto/logging.rst Doc/howto/urllib2.rst -Doc/install/index.rst Doc/library/2to3.rst Doc/library/__future__.rst Doc/library/abc.rst @@ -52,7 +48,6 @@ Doc/library/ast.rst Doc/library/asyncio-dev.rst Doc/library/asyncio-eventloop.rst Doc/library/asyncio-extending.rst -Doc/library/asyncio-future.rst Doc/library/asyncio-policy.rst Doc/library/asyncio-stream.rst Doc/library/asyncio-subprocess.rst @@ -66,12 +61,10 @@ Doc/library/cgi.rst Doc/library/chunk.rst Doc/library/cmath.rst Doc/library/cmd.rst -Doc/library/code.rst Doc/library/codecs.rst Doc/library/collections.abc.rst Doc/library/collections.rst Doc/library/concurrent.futures.rst -Doc/library/concurrent.rst Doc/library/configparser.rst Doc/library/contextlib.rst Doc/library/copy.rst @@ -86,11 +79,8 @@ Doc/library/dis.rst Doc/library/doctest.rst Doc/library/email.charset.rst Doc/library/email.compat32-message.rst -Doc/library/email.encoders.rst Doc/library/email.errors.rst -Doc/library/email.generator.rst Doc/library/email.headerregistry.rst -Doc/library/email.message.rst Doc/library/email.mime.rst Doc/library/email.parser.rst Doc/library/email.policy.rst @@ -98,27 +88,21 @@ Doc/library/enum.rst Doc/library/exceptions.rst Doc/library/faulthandler.rst Doc/library/fcntl.rst -Doc/library/filecmp.rst -Doc/library/fileinput.rst Doc/library/ftplib.rst Doc/library/functions.rst Doc/library/functools.rst Doc/library/getopt.rst Doc/library/getpass.rst Doc/library/gettext.rst -Doc/library/graphlib.rst Doc/library/gzip.rst -Doc/library/hashlib.rst Doc/library/http.client.rst Doc/library/http.cookiejar.rst Doc/library/http.cookies.rst Doc/library/http.server.rst -Doc/library/importlib.resources.abc.rst Doc/library/importlib.resources.rst Doc/library/importlib.rst Doc/library/inspect.rst Doc/library/io.rst -Doc/library/json.rst Doc/library/locale.rst Doc/library/logging.config.rst Doc/library/logging.handlers.rst @@ -127,13 +111,10 @@ Doc/library/lzma.rst Doc/library/mailbox.rst Doc/library/mmap.rst Doc/library/msilib.rst -Doc/library/msvcrt.rst Doc/library/multiprocessing.rst Doc/library/multiprocessing.shared_memory.rst -Doc/library/netrc.rst Doc/library/nntplib.rst Doc/library/numbers.rst -Doc/library/operator.rst Doc/library/optparse.rst Doc/library/os.path.rst Doc/library/os.rst @@ -142,10 +123,7 @@ Doc/library/pickle.rst Doc/library/pickletools.rst Doc/library/platform.rst Doc/library/plistlib.rst -Doc/library/poplib.rst -Doc/library/pprint.rst Doc/library/profile.rst -Doc/library/pty.rst Doc/library/pyclbr.rst Doc/library/pydoc.rst Doc/library/pyexpat.rst @@ -154,32 +132,27 @@ Doc/library/readline.rst Doc/library/reprlib.rst Doc/library/resource.rst Doc/library/rlcompleter.rst -Doc/library/sched.rst Doc/library/select.rst Doc/library/selectors.rst Doc/library/shelve.rst -Doc/library/shutil.rst Doc/library/signal.rst Doc/library/site.rst Doc/library/smtplib.rst Doc/library/socket.rst Doc/library/socketserver.rst Doc/library/ssl.rst -Doc/library/stat.rst Doc/library/stdtypes.rst Doc/library/string.rst Doc/library/subprocess.rst Doc/library/sunau.rst Doc/library/sys.rst Doc/library/sys_path_init.rst -Doc/library/sysconfig.rst Doc/library/syslog.rst Doc/library/tarfile.rst Doc/library/telnetlib.rst Doc/library/tempfile.rst Doc/library/termios.rst Doc/library/test.rst -Doc/library/textwrap.rst Doc/library/threading.rst Doc/library/time.rst Doc/library/tkinter.rst @@ -192,14 +165,11 @@ Doc/library/turtle.rst Doc/library/unittest.mock-examples.rst Doc/library/unittest.mock.rst Doc/library/unittest.rst -Doc/library/urllib.error.rst Doc/library/urllib.parse.rst Doc/library/urllib.request.rst Doc/library/uuid.rst Doc/library/weakref.rst Doc/library/webbrowser.rst -Doc/library/winreg.rst -Doc/library/winsound.rst Doc/library/wsgiref.rst Doc/library/xdrlib.rst Doc/library/xml.dom.minidom.rst @@ -210,9 +180,7 @@ Doc/library/xml.rst Doc/library/xml.sax.handler.rst Doc/library/xml.sax.reader.rst Doc/library/xml.sax.rst -Doc/library/xml.sax.utils.rst Doc/library/xmlrpc.client.rst -Doc/library/xmlrpc.rst Doc/library/xmlrpc.server.rst Doc/library/zlib.rst Doc/license.rst @@ -220,18 +188,14 @@ Doc/reference/compound_stmts.rst Doc/reference/datamodel.rst Doc/reference/expressions.rst Doc/reference/import.rst -Doc/reference/lexical_analysis.rst Doc/reference/simple_stmts.rst Doc/tutorial/appendix.rst Doc/tutorial/classes.rst Doc/tutorial/controlflow.rst Doc/tutorial/datastructures.rst -Doc/tutorial/errors.rst Doc/tutorial/inputoutput.rst -Doc/tutorial/interactive.rst Doc/tutorial/introduction.rst Doc/tutorial/modules.rst -Doc/tutorial/stdlib2.rst Doc/using/cmdline.rst Doc/using/configure.rst Doc/using/windows.rst diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index 8a207c385c6ab..1ec59767e9ce1 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -154,7 +154,7 @@ exception type. The *except clause* may specify a variable after the exception name. The variable is bound to the exception instance which typically has an ``args`` attribute that stores the arguments. For convenience, builtin exception -types define :meth:`__str__` to print all the arguments without explicitly +types define :meth:`~object.__str__` to print all the arguments without explicitly accessing ``.args``. :: >>> try: @@ -174,7 +174,7 @@ accessing ``.args``. :: x = spam y = eggs -The exception's :meth:`__str__` output is printed as the last part ('detail') +The exception's :meth:`~object.__str__` output is printed as the last part ('detail') of the message for unhandled exceptions. :exc:`BaseException` is the common base class of all exceptions. One of its diff --git a/Doc/tutorial/interactive.rst b/Doc/tutorial/interactive.rst index c0eb1feec4eb4..0d3896a4832b5 100644 --- a/Doc/tutorial/interactive.rst +++ b/Doc/tutorial/interactive.rst @@ -23,7 +23,7 @@ Python statement names, the current local variables, and the available module names. For dotted expressions such as ``string.a``, it will evaluate the expression up to the final ``'.'`` and then suggest completions from the attributes of the resulting object. Note that this may execute -application-defined code if an object with a :meth:`__getattr__` method +application-defined code if an object with a :meth:`~object.__getattr__` method is part of the expression. The default configuration also saves your history into a file named :file:`.python_history` in your user directory. The history will be available again during the next interactive interpreter diff --git a/Doc/whatsnew/3.0.rst b/Doc/whatsnew/3.0.rst index f9ac13036cbc8..379c74981fbf0 100644 --- a/Doc/whatsnew/3.0.rst +++ b/Doc/whatsnew/3.0.rst @@ -789,7 +789,7 @@ Operators And Special Methods :attr:`__doc__`, :attr:`__globals__`, :attr:`~definition.__name__`, respectively. -* :meth:`__nonzero__` is now :meth:`__bool__`. +* :meth:`!__nonzero__` is now :meth:`~object.__bool__`. Builtins -------- diff --git a/Misc/NEWS.d/3.11.0a6.rst b/Misc/NEWS.d/3.11.0a6.rst index 8621edcfb04bb..fcec71c6f59da 100644 --- a/Misc/NEWS.d/3.11.0a6.rst +++ b/Misc/NEWS.d/3.11.0a6.rst @@ -352,7 +352,7 @@ rather than ``JUMP_FORWARD`` with an argument of ``(2**32)+offset``. .. nonce: 3Z_qxd .. section: Core and Builtins -Correct the docstring for the :meth:`__bool__` method. Patch by Jelle +Correct the docstring for the :meth:`~object.__bool__` method. Patch by Jelle Zijlstra. .. From webhook-mailer at python.org Sat Jul 29 02:17:24 2023 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 29 Jul 2023 06:17:24 -0000 Subject: [Python-checkins] [3.11] gh-101100: Sphinx warnings: pick the low hanging fruits (GH-107386) (GH-107418) Message-ID: https://github.com/python/cpython/commit/3c1bcae0753736d79e6c994d1c20144cad430008 commit: 3c1bcae0753736d79e6c994d1c20144cad430008 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2023-07-29T09:17:20+03:00 summary: [3.11] gh-101100: Sphinx warnings: pick the low hanging fruits (GH-107386) (GH-107418) (cherry picked from commit f2d07d3289947d10b065b2bb7670c8fb6b6582f2) Co-authored-by: Serhiy Storchaka files: M Doc/c-api/bool.rst M Doc/faq/extending.rst M Doc/howto/curses.rst M Doc/install/index.rst M Doc/library/asyncio-extending.rst M Doc/library/asyncio-future.rst M Doc/library/bz2.rst M Doc/library/code.rst M Doc/library/concurrent.rst M Doc/library/curses.rst M Doc/library/email.charset.rst M Doc/library/email.encoders.rst M Doc/library/email.generator.rst M Doc/library/email.message.rst M Doc/library/email.parser.rst M Doc/library/filecmp.rst M Doc/library/fileinput.rst M Doc/library/graphlib.rst M Doc/library/gzip.rst M Doc/library/hashlib.rst M Doc/library/importlib.resources.abc.rst M Doc/library/json.rst M Doc/library/logging.rst M Doc/library/lzma.rst M Doc/library/msvcrt.rst M Doc/library/netrc.rst M Doc/library/operator.rst M Doc/library/poplib.rst M Doc/library/pprint.rst M Doc/library/pty.rst M Doc/library/sched.rst M Doc/library/selectors.rst M Doc/library/shutil.rst M Doc/library/stat.rst M Doc/library/stdtypes.rst M Doc/library/sysconfig.rst M Doc/library/textwrap.rst M Doc/library/urllib.error.rst M Doc/library/winreg.rst M Doc/library/winsound.rst M Doc/library/xml.rst M Doc/library/xml.sax.handler.rst M Doc/library/xml.sax.utils.rst M Doc/library/xmlrpc.rst M Doc/reference/datamodel.rst M Doc/reference/expressions.rst M Doc/reference/lexical_analysis.rst M Doc/tools/.nitignore M Doc/tutorial/errors.rst M Doc/tutorial/interactive.rst M Doc/whatsnew/3.0.rst M Misc/NEWS.d/3.11.0a6.rst diff --git a/Doc/c-api/bool.rst b/Doc/c-api/bool.rst index c197d447e9618..b15ee156c52c2 100644 --- a/Doc/c-api/bool.rst +++ b/Doc/c-api/bool.rst @@ -11,6 +11,12 @@ creation and deletion functions don't apply to booleans. The following macros are available, however. +.. c:var:: PyTypeObject PyBool_Type + + This instance of :c:type:`PyTypeObject` represents the Python boolean type; it + is the same object as :class:`bool` in the Python layer. + + .. c:function:: int PyBool_Check(PyObject *o) Return true if *o* is of type :c:data:`PyBool_Type`. This function always diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst index bc3080f60ee23..2a8b976925d04 100644 --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -81,13 +81,13 @@ How do I extract C values from a Python object? That depends on the object's type. If it's a tuple, :c:func:`PyTuple_Size` returns its length and :c:func:`PyTuple_GetItem` returns the item at a specified -index. Lists have similar functions, :c:func:`PyListSize` and +index. Lists have similar functions, :c:func:`PyList_Size` and :c:func:`PyList_GetItem`. For bytes, :c:func:`PyBytes_Size` returns its length and :c:func:`PyBytes_AsStringAndSize` provides a pointer to its value and its length. Note that Python bytes objects may contain null bytes so C's -:c:func:`strlen` should not be used. +:c:func:`!strlen` should not be used. To test the type of an object, first make sure it isn't ``NULL``, and then use :c:func:`PyBytes_Check`, :c:func:`PyTuple_Check`, :c:func:`PyList_Check`, etc. diff --git a/Doc/howto/curses.rst b/Doc/howto/curses.rst index a3068d86d85bc..4828e2fa29bd2 100644 --- a/Doc/howto/curses.rst +++ b/Doc/howto/curses.rst @@ -527,7 +527,7 @@ If you're in doubt about the detailed behavior of the curses functions, consult the manual pages for your curses implementation, whether it's ncurses or a proprietary Unix vendor's. The manual pages will document any quirks, and provide complete lists of all the -functions, attributes, and :const:`ACS_\*` characters available to +functions, attributes, and :ref:`ACS_\* ` characters available to you. Because the curses API is so large, some functions aren't supported in diff --git a/Doc/install/index.rst b/Doc/install/index.rst index fc4377e0d2b82..283070d36badf 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -774,7 +774,7 @@ Notes: (2) On Unix, if the :envvar:`HOME` environment variable is not defined, the user's - home directory will be determined with the :func:`getpwuid` function from the + home directory will be determined with the :func:`~pwd.getpwuid` function from the standard :mod:`pwd` module. This is done by the :func:`os.path.expanduser` function used by Distutils. diff --git a/Doc/library/asyncio-extending.rst b/Doc/library/asyncio-extending.rst index 8ffd356f2d1cc..e7b293f484f8d 100644 --- a/Doc/library/asyncio-extending.rst +++ b/Doc/library/asyncio-extending.rst @@ -69,7 +69,7 @@ Task lifetime support ===================== A third party task implementation should call the following functions to keep a task -visible by :func:`asyncio.get_tasks` and :func:`asyncio.current_task`: +visible by :func:`asyncio.all_tasks` and :func:`asyncio.current_task`: .. function:: _register_task(task) diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index 70cec9b2f9024..893ae5518f757 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -276,4 +276,4 @@ the Future has a result:: :func:`concurrent.futures.as_completed` functions. - :meth:`asyncio.Future.cancel` accepts an optional ``msg`` argument, - but :func:`concurrent.futures.cancel` does not. + but :meth:`concurrent.futures.Future.cancel` does not. diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst index ae5a1598f84b4..d03f2c4f0ca97 100644 --- a/Doc/library/bz2.rst +++ b/Doc/library/bz2.rst @@ -87,7 +87,8 @@ The :mod:`bz2` module contains: compressed streams. :class:`BZ2File` provides all of the members specified by the - :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`. + :class:`io.BufferedIOBase`, except for :meth:`~io.BufferedIOBase.detach` + and :meth:`~io.IOBase.truncate`. Iteration and the :keyword:`with` statement are supported. :class:`BZ2File` also provides the following method: diff --git a/Doc/library/code.rst b/Doc/library/code.rst index 538e5afc7822a..3d7f43c86a055 100644 --- a/Doc/library/code.rst +++ b/Doc/library/code.rst @@ -163,12 +163,12 @@ interpreter objects as well as the following additions. Push a line of source text to the interpreter. The line should not have a trailing newline; it may have internal newlines. The line is appended to a - buffer and the interpreter's :meth:`runsource` method is called with the + buffer and the interpreter's :meth:`~InteractiveInterpreter.runsource` method is called with the concatenated contents of the buffer as source. If this indicates that the command was executed or invalid, the buffer is reset; otherwise, the command is incomplete, and the buffer is left as it was after the line was appended. The return value is ``True`` if more input is required, ``False`` if the line was - dealt with in some way (this is the same as :meth:`runsource`). + dealt with in some way (this is the same as :meth:`!runsource`). .. method:: InteractiveConsole.resetbuffer() diff --git a/Doc/library/concurrent.rst b/Doc/library/concurrent.rst index 2eba536512580..8caea78bbb57e 100644 --- a/Doc/library/concurrent.rst +++ b/Doc/library/concurrent.rst @@ -1,5 +1,5 @@ -The :mod:`concurrent` package -============================= +The :mod:`!concurrent` package +============================== Currently, there is only one module in this package: diff --git a/Doc/library/curses.rst b/Doc/library/curses.rst index e0bc462fb1c95..a17d528793eff 100644 --- a/Doc/library/curses.rst +++ b/Doc/library/curses.rst @@ -1651,6 +1651,8 @@ keys); also, the following keypad mappings are standard: | :kbd:`Page Down` | KEY_NPAGE | +------------------+-----------+ +.. _curses-acs-codes: + The following table lists characters from the alternate character set. These are inherited from the VT100 terminal, and will generally be available on software emulations such as X terminals. When there is no graphic available, curses diff --git a/Doc/library/email.charset.rst b/Doc/library/email.charset.rst index 38fda23bd8237..1b1cd17bc81ff 100644 --- a/Doc/library/email.charset.rst +++ b/Doc/library/email.charset.rst @@ -150,7 +150,7 @@ Import this class from the :mod:`email.charset` module. .. method:: __str__() Returns *input_charset* as a string coerced to lower - case. :meth:`__repr__` is an alias for :meth:`__str__`. + case. :meth:`!__repr__` is an alias for :meth:`!__str__`. .. method:: __eq__(other) diff --git a/Doc/library/email.encoders.rst b/Doc/library/email.encoders.rst index 5d68b104f3a45..3bd377e33f6c1 100644 --- a/Doc/library/email.encoders.rst +++ b/Doc/library/email.encoders.rst @@ -25,7 +25,7 @@ is especially true for :mimetype:`image/\*` and :mimetype:`text/\*` type message containing binary data. The :mod:`email` package provides some convenient encoders in its -:mod:`encoders` module. These encoders are actually used by the +:mod:`~email.encoders` module. These encoders are actually used by the :class:`~email.mime.audio.MIMEAudio` and :class:`~email.mime.image.MIMEImage` class constructors to provide default encodings. All encoder functions take exactly one argument, the message object to encode. They usually extract the diff --git a/Doc/library/email.generator.rst b/Doc/library/email.generator.rst index 34ad7b7f200af..91d9d69a63d73 100644 --- a/Doc/library/email.generator.rst +++ b/Doc/library/email.generator.rst @@ -274,9 +274,9 @@ in with information about the part. .. rubric:: Footnotes .. [#] This statement assumes that you use the appropriate setting for - ``unixfrom``, and that there are no :mod:`policy` settings calling for + ``unixfrom``, and that there are no :mod:`email.policy` settings calling for automatic adjustments (for example, - :attr:`~email.policy.Policy.refold_source` must be ``none``, which is + :attr:`~email.policy.EmailPolicy.refold_source` must be ``none``, which is *not* the default). It is also not 100% true, since if the message does not conform to the RFC standards occasionally information about the exact original text is lost during parsing error recovery. It is a goal diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst index 5e0509f418119..225f498781fa8 100644 --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -67,7 +67,7 @@ message objects. with the base :class:`~email.message.Message` class *maxheaderlen* is accepted, but defaults to ``None``, which means that by default the line length is controlled by the - :attr:`~email.policy.EmailPolicy.max_line_length` of the policy. The + :attr:`~email.policy.Policy.max_line_length` of the policy. The *policy* argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified *policy* will be @@ -213,7 +213,7 @@ message objects. del msg['subject'] msg['subject'] = 'Python roolz!' - If the :mod:`policy` defines certain headers to be unique (as the standard + If the :mod:`policy ` defines certain headers to be unique (as the standard policies do), this method may raise a :exc:`ValueError` when an attempt is made to assign a value to such a header when one already exists. This behavior is intentional for consistency's sake, but do not depend on it @@ -378,7 +378,7 @@ message objects. deprecated. Note that existing parameter values of headers may be accessed through - the :attr:`~email.headerregistry.BaseHeader.params` attribute of the + the :attr:`~email.headerregistry.ParameterizedMIMEHeader.params` attribute of the header value (for example, ``msg['Content-Type'].params['charset']``). .. versionchanged:: 3.4 ``replace`` keyword was added. @@ -691,7 +691,7 @@ message objects. .. method:: clear_content() - Remove the payload and all of the :exc:`Content-` headers, leaving + Remove the payload and all of the :mailheader:`!Content-` headers, leaving all other headers intact and in their original order. diff --git a/Doc/library/email.parser.rst b/Doc/library/email.parser.rst index d9a61616bbbdf..dda0466a6afa7 100644 --- a/Doc/library/email.parser.rst +++ b/Doc/library/email.parser.rst @@ -39,9 +39,9 @@ returns the root object when you close the parser. Note that the parser can be extended in limited ways, and of course you can implement your own parser completely from scratch. All of the logic that connects the :mod:`email` package's bundled parser and the -:class:`~email.message.EmailMessage` class is embodied in the :mod:`policy` +:class:`~email.message.EmailMessage` class is embodied in the :class:`~email.policy.Policy` class, so a custom parser can create message object trees any way it finds -necessary by implementing custom versions of the appropriate :mod:`policy` +necessary by implementing custom versions of the appropriate :class:`!Policy` methods. diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst index 0efb4897a1eb8..dfe4b7c59fd57 100644 --- a/Doc/library/filecmp.rst +++ b/Doc/library/filecmp.rst @@ -100,7 +100,7 @@ The :class:`dircmp` class used to get various bits of information about the directory trees being compared. - Note that via :meth:`__getattr__` hooks, all attributes are computed lazily, + Note that via :meth:`~object.__getattr__` hooks, all attributes are computed lazily, so there is no speed penalty if only those attributes which are lightweight to compute are used. diff --git a/Doc/library/fileinput.rst b/Doc/library/fileinput.rst index 4bc868759f202..f93e9a58791ee 100644 --- a/Doc/library/fileinput.rst +++ b/Doc/library/fileinput.rst @@ -177,7 +177,7 @@ available for subclassing as well: The keyword-only parameter *encoding* and *errors* are added. .. versionchanged:: 3.11 - The ``'rU'`` and ``'U'`` modes and the :meth:`__getitem__` method have + The ``'rU'`` and ``'U'`` modes and the :meth:`!__getitem__` method have been removed. diff --git a/Doc/library/graphlib.rst b/Doc/library/graphlib.rst index fe7932e7a61cb..fdd8f39ef4e1c 100644 --- a/Doc/library/graphlib.rst +++ b/Doc/library/graphlib.rst @@ -115,7 +115,7 @@ :meth:`TopologicalSorter.done` is less than the number that have been returned by :meth:`TopologicalSorter.get_ready`. - The :meth:`~TopologicalSorter.__bool__` method of this class defers to + The :meth:`~object.__bool__` method of this class defers to this function, so instead of:: if ts.is_active(): @@ -204,7 +204,7 @@ The :mod:`graphlib` module defines the following exception classes: in the working graph. If multiple cycles exist, only one undefined choice among them will be reported and included in the exception. - The detected cycle can be accessed via the second element in the :attr:`~CycleError.args` + The detected cycle can be accessed via the second element in the :attr:`~BaseException.args` attribute of the exception instance and consists in a list of nodes, such that each node is, in the graph, an immediate predecessor of the next node in the list. In the reported list, the first and the last node will be the same, to make it clear that it is cyclic. diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst index 2f26295d057f8..c28cfdc50e398 100644 --- a/Doc/library/gzip.rst +++ b/Doc/library/gzip.rst @@ -70,7 +70,7 @@ The module defines the following items: .. class:: GzipFile(filename=None, mode=None, compresslevel=9, fileobj=None, mtime=None) Constructor for the :class:`GzipFile` class, which simulates most of the - methods of a :term:`file object`, with the exception of the :meth:`truncate` + methods of a :term:`file object`, with the exception of the :meth:`~io.IOBase.truncate` method. At least one of *fileobj* and *filename* must be given a non-trivial value. @@ -113,7 +113,7 @@ The module defines the following items: :class:`GzipFile` supports the :class:`io.BufferedIOBase` interface, including iteration and the :keyword:`with` statement. Only the - :meth:`truncate` method isn't implemented. + :meth:`~io.IOBase.truncate` method isn't implemented. :class:`GzipFile` also provides the following method and attribute: diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst index 8d7b3413d1972..c77b728dbf305 100644 --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -244,7 +244,7 @@ by the SHAKE algorithm. .. method:: shake.digest(length) - Return the digest of the data passed to the :meth:`update` method so far. + Return the digest of the data passed to the :meth:`~hash.update` method so far. This is a bytes object of size *length* which may contain bytes in the whole range from 0 to 255. @@ -513,9 +513,9 @@ Simple hashing To calculate hash of some data, you should first construct a hash object by calling the appropriate constructor function (:func:`blake2b` or -:func:`blake2s`), then update it with the data by calling :meth:`update` on the +:func:`blake2s`), then update it with the data by calling :meth:`~hash.update` on the object, and, finally, get the digest out of the object by calling -:meth:`digest` (or :meth:`hexdigest` for hex-encoded string). +:meth:`~hash.digest` (or :meth:`~hash.hexdigest` for hex-encoded string). >>> from hashlib import blake2b >>> h = blake2b() diff --git a/Doc/library/importlib.resources.abc.rst b/Doc/library/importlib.resources.abc.rst index b91e37510fe11..8851396c16b37 100644 --- a/Doc/library/importlib.resources.abc.rst +++ b/Doc/library/importlib.resources.abc.rst @@ -139,10 +139,10 @@ An abstract base class for resource readers capable of serving the :meth:`importlib.resources.files` interface. Subclasses - :class:`importlib.resources.abc.ResourceReader` and provides - concrete implementations of the :class:`importlib.resources.abc.ResourceReader`'s + :class:`ResourceReader` and provides + concrete implementations of the :class:`!ResourceReader`'s abstract methods. Therefore, any loader supplying - :class:`importlib.abc.TraversableResources` also supplies ResourceReader. + :class:`!TraversableResources` also supplies :class:`!ResourceReader`. Loaders that wish to support resource reading are expected to implement this interface. diff --git a/Doc/library/json.rst b/Doc/library/json.rst index 35a08995487c1..6c3059381776c 100644 --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -192,7 +192,7 @@ Basic Usage dictionaries will be sorted by key. To use a custom :class:`JSONEncoder` subclass (e.g. one that overrides the - :meth:`default` method to serialize additional types), specify it with the + :meth:`~JSONEncoder.default` method to serialize additional types), specify it with the *cls* kwarg; otherwise :class:`JSONEncoder` is used. .. versionchanged:: 3.6 @@ -422,7 +422,7 @@ Encoders and Decoders Added support for int- and float-derived Enum classes. To extend this to recognize other objects, subclass and implement a - :meth:`default` method with another method that returns a serializable object + :meth:`~JSONEncoder.default` method with another method that returns a serializable object for ``o`` if possible, otherwise it should call the superclass implementation (to raise :exc:`TypeError`). @@ -483,7 +483,7 @@ Encoders and Decoders :exc:`TypeError`). For example, to support arbitrary iterators, you could implement - :meth:`default` like this:: + :meth:`~JSONEncoder.default` like this:: def default(self, o): try: diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index 6c47b9e206c4f..954681efb4436 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -385,21 +385,21 @@ have specific values relative to the predefined levels. If you define a level with the same numeric value, it overwrites the predefined value; the predefined name is lost. -+--------------+---------------+ -| Level | Numeric value | -+==============+===============+ -| ``CRITICAL`` | 50 | -+--------------+---------------+ -| ``ERROR`` | 40 | -+--------------+---------------+ -| ``WARNING`` | 30 | -+--------------+---------------+ -| ``INFO`` | 20 | -+--------------+---------------+ -| ``DEBUG`` | 10 | -+--------------+---------------+ -| ``NOTSET`` | 0 | -+--------------+---------------+ ++-----------------------+---------------+ +| Level | Numeric value | ++=======================+===============+ +| .. py:data:: CRITICAL | 50 | ++-----------------------+---------------+ +| .. py:data:: ERROR | 40 | ++-----------------------+---------------+ +| .. py:data:: WARNING | 30 | ++-----------------------+---------------+ +| .. py:data:: INFO | 20 | ++-----------------------+---------------+ +| .. py:data:: DEBUG | 10 | ++-----------------------+---------------+ +| .. py:data:: NOTSET | 0 | ++-----------------------+---------------+ .. _handler: diff --git a/Doc/library/lzma.rst b/Doc/library/lzma.rst index 868d4dcfb6c99..434e7ac906118 100644 --- a/Doc/library/lzma.rst +++ b/Doc/library/lzma.rst @@ -100,7 +100,8 @@ Reading and writing compressed files *filters* arguments have the same meanings as for :class:`LZMACompressor`. :class:`LZMAFile` supports all the members specified by - :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`. + :class:`io.BufferedIOBase`, except for :meth:`~io.BufferedIOBase.detach` + and :meth:`~io.IOBase.truncate`. Iteration and the :keyword:`with` statement are supported. The following method is also provided: diff --git a/Doc/library/msvcrt.rst b/Doc/library/msvcrt.rst index 42fffee6a0f44..32693e3d007c0 100644 --- a/Doc/library/msvcrt.rst +++ b/Doc/library/msvcrt.rst @@ -38,7 +38,7 @@ File Operations Lock part of a file based on file descriptor *fd* from the C runtime. Raises :exc:`OSError` on failure. The locked region of the file extends from the current file position for *nbytes* bytes, and may continue beyond the end of the - file. *mode* must be one of the :const:`LK_\*` constants listed below. Multiple + file. *mode* must be one of the :const:`!LK_\*` constants listed below. Multiple regions in a file may be locked at the same time, but may not overlap. Adjacent regions are not merged; they must be unlocked individually. diff --git a/Doc/library/netrc.rst b/Doc/library/netrc.rst index 88265d9b9e9e9..c36e5cfecfc6a 100644 --- a/Doc/library/netrc.rst +++ b/Doc/library/netrc.rst @@ -51,9 +51,19 @@ the Unix :program:`ftp` program and other FTP clients. Exception raised by the :class:`~netrc.netrc` class when syntactical errors are encountered in source text. Instances of this exception provide three - interesting attributes: :attr:`msg` is a textual explanation of the error, - :attr:`filename` is the name of the source file, and :attr:`lineno` gives the - line number on which the error was found. + interesting attributes: + + .. attribute:: msg + + Textual explanation of the error. + + .. attribute:: filename + + The name of the source file. + + .. attribute:: lineno + + The line number on which the error was found. .. _netrc-objects: diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst index dab4de9eb6abb..57c67bcf3aa12 100644 --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -59,9 +59,9 @@ truth tests, identity tests, and boolean operations: __not__(obj) Return the outcome of :keyword:`not` *obj*. (Note that there is no - :meth:`__not__` method for object instances; only the interpreter core defines - this operation. The result is affected by the :meth:`__bool__` and - :meth:`__len__` methods.) + :meth:`!__not__` method for object instances; only the interpreter core defines + this operation. The result is affected by the :meth:`~object.__bool__` and + :meth:`~object.__len__` methods.) .. function:: truth(obj) diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index c9ada42447e8a..5e0371592ecb9 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -156,7 +156,7 @@ A :class:`POP3` instance has the following methods: .. method:: POP3.pass_(password) Send password, response includes message count and mailbox size. Note: the - mailbox on the server is locked until :meth:`~poplib.quit` is called. + mailbox on the server is locked until :meth:`~POP3.quit` is called. .. method:: POP3.apop(user, secret) diff --git a/Doc/library/pprint.rst b/Doc/library/pprint.rst index 2a3bc92e1733a..fa5153284a2aa 100644 --- a/Doc/library/pprint.rst +++ b/Doc/library/pprint.rst @@ -45,7 +45,7 @@ The :mod:`pprint` module defines one class: several keyword parameters. *stream* (default ``sys.stdout``) is a :term:`file-like object` to - which the output will be written by calling its :meth:`write` method. + which the output will be written by calling its :meth:`!write` method. If both *stream* and ``sys.stdout`` are ``None``, then :meth:`~PrettyPrinter.pprint` silently returns. diff --git a/Doc/library/pty.rst b/Doc/library/pty.rst index 7f4da41e93802..ad4981c97119f 100644 --- a/Doc/library/pty.rst +++ b/Doc/library/pty.rst @@ -71,7 +71,7 @@ The :mod:`pty` module defines the following functions: Return the exit status value from :func:`os.waitpid` on the child process. - :func:`waitstatus_to_exitcode` can be used to convert the exit status into + :func:`os.waitstatus_to_exitcode` can be used to convert the exit status into an exit code. .. audit-event:: pty.spawn argv pty.spawn diff --git a/Doc/library/sched.rst b/Doc/library/sched.rst index 04215d31ba10c..01bac5afd0b9b 100644 --- a/Doc/library/sched.rst +++ b/Doc/library/sched.rst @@ -115,7 +115,7 @@ Scheduler Objects .. method:: scheduler.run(blocking=True) - Run all scheduled events. This method will wait (using the :func:`delayfunc` + Run all scheduled events. This method will wait (using the *delayfunc* function passed to the constructor) for the next event, then execute it and so on until there are no more scheduled events. diff --git a/Doc/library/selectors.rst b/Doc/library/selectors.rst index 0deb15cf4c503..dd50bac37e49b 100644 --- a/Doc/library/selectors.rst +++ b/Doc/library/selectors.rst @@ -60,9 +60,9 @@ constants below: +-----------------------+-----------------------------------------------+ | Constant | Meaning | +=======================+===============================================+ - | :const:`EVENT_READ` | Available for read | + | .. data:: EVENT_READ | Available for read | +-----------------------+-----------------------------------------------+ - | :const:`EVENT_WRITE` | Available for write | + | .. data:: EVENT_WRITE | Available for write | +-----------------------+-----------------------------------------------+ @@ -132,8 +132,8 @@ constants below: Change a registered file object's monitored events or attached data. - This is equivalent to :meth:`BaseSelector.unregister(fileobj)` followed - by :meth:`BaseSelector.register(fileobj, events, data)`, except that it + This is equivalent to ``BaseSelector.unregister(fileobj)`` followed + by ``BaseSelector.register(fileobj, events, data)``, except that it can be implemented more efficiently. This returns a new :class:`SelectorKey` instance, or raises a diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index dcf421f162b15..4ba2856fce265 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -363,7 +363,7 @@ Directory and files operations If *copy_function* is given, it must be a callable that takes two arguments *src* and *dst*, and will be used to copy *src* to *dst* if :func:`os.rename` cannot be used. If the source is a directory, - :func:`copytree` is called, passing it the :func:`copy_function`. The + :func:`copytree` is called, passing it the *copy_function*. The default *copy_function* is :func:`copy2`. Using :func:`~shutil.copy` as the *copy_function* allows the move to succeed when it is not possible to also copy the metadata, at the expense of not copying any of the metadata. diff --git a/Doc/library/stat.rst b/Doc/library/stat.rst index 083dc5e3bcfd6..77538514598a5 100644 --- a/Doc/library/stat.rst +++ b/Doc/library/stat.rst @@ -13,8 +13,8 @@ The :mod:`stat` module defines constants and functions for interpreting the results of :func:`os.stat`, :func:`os.fstat` and :func:`os.lstat` (if they -exist). For complete details about the :c:func:`stat`, :c:func:`fstat` and -:c:func:`lstat` calls, consult the documentation for your system. +exist). For complete details about the :c:func:`stat`, :c:func:`!fstat` and +:c:func:`!lstat` calls, consult the documentation for your system. .. versionchanged:: 3.4 The stat module is backed by a C implementation. @@ -89,9 +89,9 @@ mode: .. function:: S_IFMT(mode) Return the portion of the file's mode that describes the file type (used by the - :func:`S_IS\*` functions above). + :func:`!S_IS\*` functions above). -Normally, you would use the :func:`os.path.is\*` functions for testing the type +Normally, you would use the :func:`!os.path.is\*` functions for testing the type of a file; the functions here are useful when you are doing multiple tests of the same file and wish to avoid the overhead of the :c:func:`stat` system call for each test. These are also useful when checking for information about a file diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 4437bc496ab9b..6158911fb28a7 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -44,7 +44,7 @@ Any object can be tested for truth value, for use in an :keyword:`if` or .. index:: single: true By default, an object is considered true unless its class defines either a -:meth:`__bool__` method that returns ``False`` or a :meth:`__len__` method that +:meth:`~object.__bool__` method that returns ``False`` or a :meth:`__len__` method that returns zero, when called with the object. [1]_ Here are most of the built-in objects considered false: diff --git a/Doc/library/sysconfig.rst b/Doc/library/sysconfig.rst index 0049b663973f4..dfe6934e6da8c 100644 --- a/Doc/library/sysconfig.rst +++ b/Doc/library/sysconfig.rst @@ -69,7 +69,7 @@ Python uses an installation scheme that differs depending on the platform and on the installation options. These schemes are stored in :mod:`sysconfig` under unique identifiers based on the value returned by :const:`os.name`. -Every new component that is installed using :mod:`distutils` or a +Every new component that is installed using :mod:`!distutils` or a Distutils-based system will follow the same scheme to copy its file in the right places. diff --git a/Doc/library/textwrap.rst b/Doc/library/textwrap.rst index 1a9d5f98f78a7..a150eefbf932e 100644 --- a/Doc/library/textwrap.rst +++ b/Doc/library/textwrap.rst @@ -60,7 +60,7 @@ functions should be good enough; otherwise, you should use an instance of First the whitespace in *text* is collapsed (all whitespace is replaced by single spaces). If the result fits in the *width*, it is returned. Otherwise, enough words are dropped from the end so that the remaining words - plus the :attr:`placeholder` fit within :attr:`width`:: + plus the :attr:`.placeholder` fit within :attr:`.width`:: >>> textwrap.shorten("Hello world!", width=12) 'Hello world!' @@ -173,7 +173,7 @@ hyphenated words; only then will long words be broken if necessary, unless .. attribute:: expand_tabs (default: ``True``) If true, then all tab characters in *text* will be - expanded to spaces using the :meth:`expandtabs` method of *text*. + expanded to spaces using the :meth:`~str.expandtabs` method of *text*. .. attribute:: tabsize diff --git a/Doc/library/urllib.error.rst b/Doc/library/urllib.error.rst index f7d47ed76aca1..8772e8600795e 100644 --- a/Doc/library/urllib.error.rst +++ b/Doc/library/urllib.error.rst @@ -61,6 +61,8 @@ The following exceptions are raised by :mod:`urllib.error` as appropriate: This exception is raised when the :func:`~urllib.request.urlretrieve` function detects that the amount of the downloaded data is less than the expected amount (given by - the *Content-Length* header). The :attr:`content` attribute stores the - downloaded (and supposedly truncated) data. + the *Content-Length* header). + .. attribute:: content + + The downloaded (and supposedly truncated) data. diff --git a/Doc/library/winreg.rst b/Doc/library/winreg.rst index 4ab671817710d..06bd4d87eb03c 100644 --- a/Doc/library/winreg.rst +++ b/Doc/library/winreg.rst @@ -288,7 +288,7 @@ This module offers the following functions: table (FAT) file system, the filename may not have an extension. A call to :func:`LoadKey` fails if the calling process does not have the - :const:`SE_RESTORE_PRIVILEGE` privilege. Note that privileges are different + :c:data:`!SE_RESTORE_PRIVILEGE` privilege. Note that privileges are different from permissions -- see the `RegLoadKey documentation `__ for more details. @@ -414,7 +414,7 @@ This module offers the following functions: If *key* represents a key on a remote computer, the path described by *file_name* is relative to the remote computer. The caller of this method must - possess the :const:`SeBackupPrivilege` security privilege. Note that + possess the **SeBackupPrivilege** security privilege. Note that privileges are different than permissions -- see the `Conflicts Between User Rights and Permissions documentation `__ @@ -536,7 +536,7 @@ This module offers the following functions: Constants ------------------ -The following constants are defined for use in many :mod:`_winreg` functions. +The following constants are defined for use in many :mod:`winreg` functions. .. _hkey-constants: @@ -745,7 +745,7 @@ All registry functions in this module return one of these objects. All registry functions in this module which accept a handle object also accept an integer, however, use of the handle object is encouraged. -Handle objects provide semantics for :meth:`__bool__` -- thus :: +Handle objects provide semantics for :meth:`~object.__bool__` -- thus :: if handle: print("Yes") diff --git a/Doc/library/winsound.rst b/Doc/library/winsound.rst index 372f792a0f938..370c5216652ba 100644 --- a/Doc/library/winsound.rst +++ b/Doc/library/winsound.rst @@ -24,7 +24,7 @@ provided by Windows platforms. It includes functions and several constants. .. function:: PlaySound(sound, flags) - Call the underlying :c:func:`PlaySound` function from the Platform API. The + Call the underlying :c:func:`!PlaySound` function from the Platform API. The *sound* parameter may be a filename, a system sound alias, audio data as a :term:`bytes-like object`, or ``None``. Its interpretation depends on the value of *flags*, which can be a bitwise ORed @@ -35,7 +35,7 @@ provided by Windows platforms. It includes functions and several constants. .. function:: MessageBeep(type=MB_OK) - Call the underlying :c:func:`MessageBeep` function from the Platform API. This + Call the underlying :c:func:`!MessageBeep` function from the Platform API. This plays a sound as specified in the registry. The *type* argument specifies which sound to play; possible values are ``-1``, ``MB_ICONASTERISK``, ``MB_ICONEXCLAMATION``, ``MB_ICONHAND``, ``MB_ICONQUESTION``, and ``MB_OK``, all diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index cf30de67719b9..1e49b6568dfc2 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -75,10 +75,10 @@ decompression bomb Safe Safe Safe potential reliance on system-provided libraries. Check :const:`pyexpat.EXPAT_VERSION`. 2. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a - :exc:`ParserError` when an entity occurs. + :exc:`~xml.etree.ElementTree.ParseError` when an entity occurs. 3. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns the unexpanded entity verbatim. -4. :mod:`xmlrpclib` doesn't expand external entities and omits them. +4. :mod:`xmlrpc.client` doesn't expand external entities and omits them. 5. Since Python 3.7.1, external general entities are no longer processed by default. @@ -119,8 +119,8 @@ all known attack vectors with examples and references. .. _defusedxml-package: -The :mod:`defusedxml` Package ------------------------------------------------------- +The :mod:`!defusedxml` Package +------------------------------ `defusedxml`_ is a pure Python package with modified subclasses of all stdlib XML parsers that prevent any potentially malicious operation. Use of this diff --git a/Doc/library/xml.sax.handler.rst b/Doc/library/xml.sax.handler.rst index 719ce5ab1bcf6..e2f28e3244cb0 100644 --- a/Doc/library/xml.sax.handler.rst +++ b/Doc/library/xml.sax.handler.rst @@ -393,7 +393,7 @@ implements this interface, then register the object with your :class:`~xml.sax.xmlreader.XMLReader`, the parser will call the methods in your object to report all warnings and errors. There are three levels of errors available: warnings, (possibly) recoverable errors, -and unrecoverable errors. All methods take a :exc:`SAXParseException` as the +and unrecoverable errors. All methods take a :exc:`~xml.sax.SAXParseException` as the only parameter. Errors and warnings may be converted to an exception by raising the passed-in exception object. diff --git a/Doc/library/xml.sax.utils.rst b/Doc/library/xml.sax.utils.rst index e46fefdf99751..8c1be43d1f277 100644 --- a/Doc/library/xml.sax.utils.rst +++ b/Doc/library/xml.sax.utils.rst @@ -87,5 +87,5 @@ or as base classes. reading. The input source can be given as a string, a file-like object, or an :class:`~xml.sax.xmlreader.InputSource` object; parsers will use this function to implement the polymorphic *source* argument to their - :meth:`parse` method. + :meth:`~xml.sax.xmlreader.XMLReader.parse` method. diff --git a/Doc/library/xmlrpc.rst b/Doc/library/xmlrpc.rst index ae68157b0f63c..5f0a2cf68d01f 100644 --- a/Doc/library/xmlrpc.rst +++ b/Doc/library/xmlrpc.rst @@ -1,5 +1,5 @@ -:mod:`xmlrpc` --- XMLRPC server and client modules -================================================== +:mod:`!xmlrpc` --- XMLRPC server and client modules +=================================================== XML-RPC is a Remote Procedure Call method that uses XML passed via HTTP as a transport. With it, a client can call methods with parameters on a remote diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index d2a39c57adfc5..65d1ca1b8ddcb 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1581,9 +1581,9 @@ Basic customization Called to implement truth value testing and the built-in operation ``bool()``; should return ``False`` or ``True``. When this method is not - defined, :meth:`__len__` is called, if it is defined, and the object is + defined, :meth:`~object.__len__` is called, if it is defined, and the object is considered true if its result is nonzero. If a class defines neither - :meth:`__len__` nor :meth:`__bool__`, all its instances are considered + :meth:`!__len__` nor :meth:`!__bool__`, all its instances are considered true. @@ -2477,7 +2477,7 @@ through the object's keys; for sequences, it should iterate through the values. Called to implement the built-in function :func:`len`. Should return the length of the object, an integer ``>=`` 0. Also, an object that doesn't define a - :meth:`__bool__` method and whose :meth:`__len__` method returns zero is + :meth:`~object.__bool__` method and whose :meth:`!__len__` method returns zero is considered to be false in a Boolean context. .. impl-detail:: @@ -2486,7 +2486,7 @@ through the object's keys; for sequences, it should iterate through the values. If the length is larger than :data:`!sys.maxsize` some features (such as :func:`len`) may raise :exc:`OverflowError`. To prevent raising :exc:`!OverflowError` by truth value testing, an object must define a - :meth:`__bool__` method. + :meth:`~object.__bool__` method. .. method:: object.__length_hint__(self) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 2a16025ad6212..8320cd0ae7519 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1707,7 +1707,7 @@ control flow statements, the following values are interpreted as false: ``False``, ``None``, numeric zero of all types, and empty strings and containers (including strings, tuples, lists, dictionaries, sets and frozensets). All other values are interpreted as true. User-defined objects can customize their -truth value by providing a :meth:`__bool__` method. +truth value by providing a :meth:`~object.__bool__` method. .. index:: pair: operator; not diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 44726de3e0373..8d07e071f71be 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -757,7 +757,7 @@ is converted before formatting. Conversion ``'!s'`` calls :func:`str` on the result, ``'!r'`` calls :func:`repr`, and ``'!a'`` calls :func:`ascii`. The result is then formatted using the :func:`format` protocol. The -format specifier is passed to the :meth:`__format__` method of the +format specifier is passed to the :meth:`~object.__format__` method of the expression or conversion result. An empty string is passed when the format specifier is omitted. The formatted result is then included in the final value of the whole string. diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 22eb47856f571..df7dcd607489c 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -2,7 +2,6 @@ # as tested on the CI via check-warnings.py in reusable-docs.yml. # Keep lines sorted lexicographically to help avoid merge conflicts. -Doc/c-api/bool.rst Doc/c-api/buffer.rst Doc/c-api/datetime.rst Doc/c-api/descriptor.rst @@ -27,26 +26,22 @@ Doc/c-api/unicode.rst Doc/extending/extending.rst Doc/extending/newtypes.rst Doc/faq/design.rst -Doc/faq/extending.rst Doc/faq/gui.rst Doc/faq/library.rst Doc/faq/programming.rst Doc/glossary.rst -Doc/howto/curses.rst Doc/howto/descriptor.rst Doc/howto/enum.rst Doc/howto/isolating-extensions.rst Doc/howto/logging-cookbook.rst Doc/howto/logging.rst Doc/howto/urllib2.rst -Doc/install/index.rst Doc/library/__future__.rst Doc/library/abc.rst Doc/library/ast.rst Doc/library/asyncio-dev.rst Doc/library/asyncio-eventloop.rst Doc/library/asyncio-extending.rst -Doc/library/asyncio-future.rst Doc/library/asyncio-policy.rst Doc/library/asyncio-stream.rst Doc/library/asyncio-subprocess.rst @@ -56,12 +51,10 @@ Doc/library/bisect.rst Doc/library/bz2.rst Doc/library/calendar.rst Doc/library/cmd.rst -Doc/library/code.rst Doc/library/codecs.rst Doc/library/collections.abc.rst Doc/library/collections.rst Doc/library/concurrent.futures.rst -Doc/library/concurrent.rst Doc/library/configparser.rst Doc/library/contextlib.rst Doc/library/copy.rst @@ -76,11 +69,8 @@ Doc/library/dis.rst Doc/library/doctest.rst Doc/library/email.charset.rst Doc/library/email.compat32-message.rst -Doc/library/email.encoders.rst Doc/library/email.errors.rst -Doc/library/email.generator.rst Doc/library/email.headerregistry.rst -Doc/library/email.message.rst Doc/library/email.mime.rst Doc/library/email.parser.rst Doc/library/email.policy.rst @@ -88,26 +78,20 @@ Doc/library/enum.rst Doc/library/exceptions.rst Doc/library/faulthandler.rst Doc/library/fcntl.rst -Doc/library/filecmp.rst -Doc/library/fileinput.rst Doc/library/ftplib.rst Doc/library/functions.rst Doc/library/functools.rst Doc/library/getpass.rst Doc/library/gettext.rst -Doc/library/graphlib.rst Doc/library/gzip.rst -Doc/library/hashlib.rst Doc/library/http.client.rst Doc/library/http.cookiejar.rst Doc/library/http.cookies.rst Doc/library/http.server.rst -Doc/library/importlib.resources.abc.rst Doc/library/importlib.resources.rst Doc/library/importlib.rst Doc/library/inspect.rst Doc/library/io.rst -Doc/library/json.rst Doc/library/locale.rst Doc/library/logging.config.rst Doc/library/logging.handlers.rst @@ -115,12 +99,9 @@ Doc/library/logging.rst Doc/library/lzma.rst Doc/library/mailbox.rst Doc/library/mmap.rst -Doc/library/msvcrt.rst Doc/library/multiprocessing.rst Doc/library/multiprocessing.shared_memory.rst -Doc/library/netrc.rst Doc/library/numbers.rst -Doc/library/operator.rst Doc/library/optparse.rst Doc/library/os.path.rst Doc/library/os.rst @@ -128,10 +109,7 @@ Doc/library/pickle.rst Doc/library/pickletools.rst Doc/library/platform.rst Doc/library/plistlib.rst -Doc/library/poplib.rst -Doc/library/pprint.rst Doc/library/profile.rst -Doc/library/pty.rst Doc/library/pyclbr.rst Doc/library/pydoc.rst Doc/library/pyexpat.rst @@ -140,30 +118,25 @@ Doc/library/readline.rst Doc/library/reprlib.rst Doc/library/resource.rst Doc/library/rlcompleter.rst -Doc/library/sched.rst Doc/library/select.rst Doc/library/selectors.rst Doc/library/shelve.rst -Doc/library/shutil.rst Doc/library/signal.rst Doc/library/site.rst Doc/library/smtplib.rst Doc/library/socket.rst Doc/library/socketserver.rst Doc/library/ssl.rst -Doc/library/stat.rst Doc/library/stdtypes.rst Doc/library/string.rst Doc/library/subprocess.rst Doc/library/sys.rst Doc/library/sys_path_init.rst -Doc/library/sysconfig.rst Doc/library/syslog.rst Doc/library/tarfile.rst Doc/library/tempfile.rst Doc/library/termios.rst Doc/library/test.rst -Doc/library/textwrap.rst Doc/library/threading.rst Doc/library/time.rst Doc/library/tkinter.rst @@ -175,13 +148,10 @@ Doc/library/turtle.rst Doc/library/unittest.mock-examples.rst Doc/library/unittest.mock.rst Doc/library/unittest.rst -Doc/library/urllib.error.rst Doc/library/urllib.parse.rst Doc/library/urllib.request.rst Doc/library/uuid.rst Doc/library/weakref.rst -Doc/library/winreg.rst -Doc/library/winsound.rst Doc/library/wsgiref.rst Doc/library/xml.dom.minidom.rst Doc/library/xml.dom.pulldom.rst @@ -191,9 +161,7 @@ Doc/library/xml.rst Doc/library/xml.sax.handler.rst Doc/library/xml.sax.reader.rst Doc/library/xml.sax.rst -Doc/library/xml.sax.utils.rst Doc/library/xmlrpc.client.rst -Doc/library/xmlrpc.rst Doc/library/xmlrpc.server.rst Doc/library/zlib.rst Doc/license.rst @@ -201,18 +169,14 @@ Doc/reference/compound_stmts.rst Doc/reference/datamodel.rst Doc/reference/expressions.rst Doc/reference/import.rst -Doc/reference/lexical_analysis.rst Doc/reference/simple_stmts.rst Doc/tutorial/appendix.rst Doc/tutorial/classes.rst Doc/tutorial/controlflow.rst Doc/tutorial/datastructures.rst -Doc/tutorial/errors.rst Doc/tutorial/inputoutput.rst -Doc/tutorial/interactive.rst Doc/tutorial/introduction.rst Doc/tutorial/modules.rst -Doc/tutorial/stdlib2.rst Doc/using/cmdline.rst Doc/using/configure.rst Doc/using/windows.rst diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index 8a207c385c6ab..1ec59767e9ce1 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -154,7 +154,7 @@ exception type. The *except clause* may specify a variable after the exception name. The variable is bound to the exception instance which typically has an ``args`` attribute that stores the arguments. For convenience, builtin exception -types define :meth:`__str__` to print all the arguments without explicitly +types define :meth:`~object.__str__` to print all the arguments without explicitly accessing ``.args``. :: >>> try: @@ -174,7 +174,7 @@ accessing ``.args``. :: x = spam y = eggs -The exception's :meth:`__str__` output is printed as the last part ('detail') +The exception's :meth:`~object.__str__` output is printed as the last part ('detail') of the message for unhandled exceptions. :exc:`BaseException` is the common base class of all exceptions. One of its diff --git a/Doc/tutorial/interactive.rst b/Doc/tutorial/interactive.rst index c0eb1feec4eb4..0d3896a4832b5 100644 --- a/Doc/tutorial/interactive.rst +++ b/Doc/tutorial/interactive.rst @@ -23,7 +23,7 @@ Python statement names, the current local variables, and the available module names. For dotted expressions such as ``string.a``, it will evaluate the expression up to the final ``'.'`` and then suggest completions from the attributes of the resulting object. Note that this may execute -application-defined code if an object with a :meth:`__getattr__` method +application-defined code if an object with a :meth:`~object.__getattr__` method is part of the expression. The default configuration also saves your history into a file named :file:`.python_history` in your user directory. The history will be available again during the next interactive interpreter diff --git a/Doc/whatsnew/3.0.rst b/Doc/whatsnew/3.0.rst index 63b24748d8aab..6f2a155abe10b 100644 --- a/Doc/whatsnew/3.0.rst +++ b/Doc/whatsnew/3.0.rst @@ -789,7 +789,7 @@ Operators And Special Methods :attr:`__doc__`, :attr:`__globals__`, :attr:`~definition.__name__`, respectively. -* :meth:`__nonzero__` is now :meth:`__bool__`. +* :meth:`!__nonzero__` is now :meth:`~object.__bool__`. Builtins -------- diff --git a/Misc/NEWS.d/3.11.0a6.rst b/Misc/NEWS.d/3.11.0a6.rst index 8621edcfb04bb..fcec71c6f59da 100644 --- a/Misc/NEWS.d/3.11.0a6.rst +++ b/Misc/NEWS.d/3.11.0a6.rst @@ -352,7 +352,7 @@ rather than ``JUMP_FORWARD`` with an argument of ``(2**32)+offset``. .. nonce: 3Z_qxd .. section: Core and Builtins -Correct the docstring for the :meth:`__bool__` method. Patch by Jelle +Correct the docstring for the :meth:`~object.__bool__` method. Patch by Jelle Zijlstra. .. From webhook-mailer at python.org Sat Jul 29 02:37:26 2023 From: webhook-mailer at python.org (methane) Date: Sat, 29 Jul 2023 06:37:26 -0000 Subject: [Python-checkins] gh-107369: optimize textwrap.indent() (#107374) Message-ID: https://github.com/python/cpython/commit/37551c9cef307ca3172c60b28c7de9db921808ad commit: 37551c9cef307ca3172c60b28c7de9db921808ad branch: main author: Inada Naoki committer: methane date: 2023-07-29T06:37:23Z summary: gh-107369: optimize textwrap.indent() (#107374) files: A Misc/NEWS.d/next/Library/2023-07-28-14-56-35.gh-issue-107369.bvTq8F.rst M Doc/whatsnew/3.13.rst M Lib/textwrap.py diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 7cad5d1ec02aa..e20832a3a4c30 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -150,7 +150,8 @@ typing Optimizations ============= - +* :func:`textwrap.indent` is now ~30% faster than before for large input. + (Contributed by Inada Naoki in :gh:`107369`.) Deprecated diff --git a/Lib/textwrap.py b/Lib/textwrap.py index 98bedd27ea3a1..7ca393d1c371a 100644 --- a/Lib/textwrap.py +++ b/Lib/textwrap.py @@ -476,13 +476,19 @@ def indent(text, prefix, predicate=None): consist solely of whitespace characters. """ if predicate is None: - def predicate(line): - return line.strip() - - def prefixed_lines(): - for line in text.splitlines(True): - yield (prefix + line if predicate(line) else line) - return ''.join(prefixed_lines()) + # str.splitlines(True) doesn't produce empty string. + # ''.splitlines(True) => [] + # 'foo\n'.splitlines(True) => ['foo\n'] + # So we can use just `not s.isspace()` here. + predicate = lambda s: not s.isspace() + + prefixed_lines = [] + for line in text.splitlines(True): + if predicate(line): + prefixed_lines.append(prefix) + prefixed_lines.append(line) + + return ''.join(prefixed_lines) if __name__ == "__main__": diff --git a/Misc/NEWS.d/next/Library/2023-07-28-14-56-35.gh-issue-107369.bvTq8F.rst b/Misc/NEWS.d/next/Library/2023-07-28-14-56-35.gh-issue-107369.bvTq8F.rst new file mode 100644 index 0000000000000..76aeab65e90a2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-28-14-56-35.gh-issue-107369.bvTq8F.rst @@ -0,0 +1,2 @@ +Optimize :func:`textwrap.indent`. It is ~30% faster for large input. Patch +by Inada Naoki. From webhook-mailer at python.org Sat Jul 29 08:39:25 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 29 Jul 2023 12:39:25 -0000 Subject: [Python-checkins] gh-104050: Argument clinic: enable mypy's `--warn-return-any` setting (#107405) Message-ID: https://github.com/python/cpython/commit/87de2fbb1e9b94f25ae8c0c233fb4690a13e4b02 commit: 87de2fbb1e9b94f25ae8c0c233fb4690a13e4b02 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-29T13:39:21+01:00 summary: gh-104050: Argument clinic: enable mypy's `--warn-return-any` setting (#107405) files: M Tools/clinic/clinic.py M Tools/clinic/mypy.ini diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 6bfa6d0d56814..cb999c156ee28 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -4288,10 +4288,11 @@ def eval_ast_expr( globals: dict[str, Any], *, filename: str = '-' -) -> FunctionType: +) -> Any: """ - Takes an ast.Expr node. Compiles and evaluates it. - Returns the result of the expression. + Takes an ast.Expr node. Compiles it into a function object, + then calls the function object with 0 arguments. + Returns the result of that function call. globals represents the globals dict the expression should see. (There's no equivalent for "locals" here.) diff --git a/Tools/clinic/mypy.ini b/Tools/clinic/mypy.ini index 4aa65a5c14489..4cfc05bec0160 100644 --- a/Tools/clinic/mypy.ini +++ b/Tools/clinic/mypy.ini @@ -5,11 +5,8 @@ pretty = True # make sure clinic can still be run on Python 3.10 python_version = 3.10 -# be strict... +# and be strict! strict = True strict_concatenate = True enable_error_code = ignore-without-code,redundant-expr warn_unreachable = True - -# ...except for one extra rule we can't enable just yet -warn_return_any = False From webhook-mailer at python.org Sat Jul 29 10:42:47 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 29 Jul 2023 14:42:47 -0000 Subject: [Python-checkins] Improve the GitHub issue templates (#107150) Message-ID: https://github.com/python/cpython/commit/d0dcd27d3a8b8680803510fee59836cd732feb7a commit: d0dcd27d3a8b8680803510fee59836cd732feb7a branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-29T15:42:44+01:00 summary: Improve the GitHub issue templates (#107150) Co-authored-by: Ezio Melotti files: M .github/ISSUE_TEMPLATE/bug.md M .github/ISSUE_TEMPLATE/crash.md M .github/ISSUE_TEMPLATE/feature.md diff --git a/.github/ISSUE_TEMPLATE/bug.md b/.github/ISSUE_TEMPLATE/bug.md index 1d93e0735e50f..737eb6dfebb77 100644 --- a/.github/ISSUE_TEMPLATE/bug.md +++ b/.github/ISSUE_TEMPLATE/bug.md @@ -5,28 +5,48 @@ labels: "type-bug" --- # Bug report -A clear and concise description of what the bug is. -Include a minimal, reproducible example (https://stackoverflow.com/help/minimal-reproducible-example), if possible. +## Checklist + + + +- [ ] I am confident this is a bug in CPython, + not a bug in a third-party project +- [ ] I have searched the CPython issue tracker, + and am confident this bug has not been reported before + +## A clear and concise description of the bug + + + + # Your environment - + - CPython versions tested on: - Operating system and architecture: diff --git a/.github/ISSUE_TEMPLATE/crash.md b/.github/ISSUE_TEMPLATE/crash.md index dad3423db0341..a268249d1c1e6 100644 --- a/.github/ISSUE_TEMPLATE/crash.md +++ b/.github/ISSUE_TEMPLATE/crash.md @@ -5,29 +5,44 @@ labels: "type-crash" --- # Crash report -Tell us what happened, ideally including a minimal, reproducible example (https://stackoverflow.com/help/minimal-reproducible-example). + + + # Error messages -Enter any relevant error message caused by the crash, including a core dump if there is one. + + + # Your environment - + - CPython versions tested on: - Operating system and architecture: diff --git a/.github/ISSUE_TEMPLATE/feature.md b/.github/ISSUE_TEMPLATE/feature.md index ed051e945f812..7e96bc9df665c 100644 --- a/.github/ISSUE_TEMPLATE/feature.md +++ b/.github/ISSUE_TEMPLATE/feature.md @@ -4,25 +4,47 @@ about: Submit a proposal for a new CPython feature or enhancement labels: "type-feature" --- + + # Feature or enhancement -(A clear and concise description of your proposal.) + + + # Pitch -(Explain why this feature or enhancement should be implemented and how it would be used. - Add examples, if applicable.) + + + # Previous discussion + From webhook-mailer at python.org Sat Jul 29 13:04:49 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 29 Jul 2023 17:04:49 -0000 Subject: [Python-checkins] gh-107422: Remove outdated `TypedDict` example from typing docs (#107436) Message-ID: https://github.com/python/cpython/commit/89fd4f4a3fc5fb8076ec064c22a30108480e946b commit: 89fd4f4a3fc5fb8076ec064c22a30108480e946b branch: main author: Rakesh Sabale <102187286+ghubrakesh at users.noreply.github.com> committer: AlexWaygood date: 2023-07-29T17:04:46Z summary: gh-107422: Remove outdated `TypedDict` example from typing docs (#107436) files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 0265a39ce646f..e2791bbc97b00 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -2329,9 +2329,6 @@ types. class XZ(X, Z): pass # raises TypeError - T = TypeVar('T') - class XT(X, Generic[T]): pass # raises TypeError - A ``TypedDict`` can be generic:: class Group[T](TypedDict): From webhook-mailer at python.org Sat Jul 29 13:12:53 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 29 Jul 2023 17:12:53 -0000 Subject: [Python-checkins] [3.12] gh-107422: Remove outdated `TypedDict` example from typing docs (GH-107436) (#107437) Message-ID: https://github.com/python/cpython/commit/2c0a99d322037d653063dc8466b886110e0fc447 commit: 2c0a99d322037d653063dc8466b886110e0fc447 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-29T17:12:50Z summary: [3.12] gh-107422: Remove outdated `TypedDict` example from typing docs (GH-107436) (#107437) gh-107422: Remove outdated `TypedDict` example from typing docs (GH-107436) (cherry picked from commit 89fd4f4a3fc5fb8076ec064c22a30108480e946b) Co-authored-by: Rakesh Sabale <102187286+ghubrakesh at users.noreply.github.com> files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index e1acbd83a4173..11c39a444ef65 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -2327,9 +2327,6 @@ types. class XZ(X, Z): pass # raises TypeError - T = TypeVar('T') - class XT(X, Generic[T]): pass # raises TypeError - A ``TypedDict`` can be generic:: class Group[T](TypedDict): From webhook-mailer at python.org Sat Jul 29 13:21:49 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 29 Jul 2023 17:21:49 -0000 Subject: [Python-checkins] [3.11] gh-107422: Remove outdated `TypedDict` example from typing docs (#107436) (#107438) Message-ID: https://github.com/python/cpython/commit/357708eee0a665240905f77a4a6832b642be6d52 commit: 357708eee0a665240905f77a4a6832b642be6d52 branch: 3.11 author: Alex Waygood committer: AlexWaygood date: 2023-07-29T17:21:45Z summary: [3.11] gh-107422: Remove outdated `TypedDict` example from typing docs (#107436) (#107438) gh-107422: Remove outdated `TypedDict` example from typing docs (#107436) Co-authored-by: Rakesh Sabale <102187286+ghubrakesh at users.noreply.github.com> files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index d03f0af1bc6b6..a0766690bad28 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -2073,9 +2073,6 @@ types. class XZ(X, Z): pass # raises TypeError - T = TypeVar('T') - class XT(X, Generic[T]): pass # raises TypeError - A ``TypedDict`` can be generic: .. testcode:: From webhook-mailer at python.org Sat Jul 29 13:41:43 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sat, 29 Jul 2023 17:41:43 -0000 Subject: [Python-checkins] Fix the documentation for PyCode_New add `qualname` parameter (#107186) Message-ID: https://github.com/python/cpython/commit/f2abeb590dae5918388f91b60b31f040d44218f0 commit: f2abeb590dae5918388f91b60b31f040d44218f0 branch: main author: da-woods committer: kumaraditya303 date: 2023-07-29T23:11:40+05:30 summary: Fix the documentation for PyCode_New add `qualname` parameter (#107186) files: M Doc/c-api/code.rst diff --git a/Doc/c-api/code.rst b/Doc/c-api/code.rst index 89fe42d1ff05f..5082b0cb6ad3f 100644 --- a/Doc/c-api/code.rst +++ b/Doc/c-api/code.rst @@ -33,7 +33,7 @@ bound into a function. Return the number of free variables in *co*. -.. c:function:: PyCodeObject* PyUnstable_Code_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable) +.. c:function:: PyCodeObject* PyUnstable_Code_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, PyObject *qualname, int firstlineno, PyObject *linetable, PyObject *exceptiontable) Return a new code object. If you need a dummy code object to create a frame, use :c:func:`PyCode_NewEmpty` instead. @@ -46,7 +46,7 @@ bound into a function. execution or VM crashes. Use this function only with extreme care. .. versionchanged:: 3.11 - Added ``exceptiontable`` parameter. + Added ``qualname`` and ``exceptiontable`` parameters. .. index:: single: PyCode_New @@ -56,7 +56,7 @@ bound into a function. The old name is deprecated, but will remain available until the signature changes again. -.. c:function:: PyCodeObject* PyUnstable_Code_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable) +.. c:function:: PyCodeObject* PyUnstable_Code_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, PyObject *qualname, int firstlineno, PyObject *linetable, PyObject *exceptiontable) Similar to :c:func:`PyUnstable_Code_New`, but with an extra "posonlyargcount" for positional-only arguments. The same caveats that apply to ``PyUnstable_Code_New`` also apply to this function. @@ -66,7 +66,7 @@ bound into a function. .. versionadded:: 3.8 as ``PyCode_NewWithPosOnlyArgs`` .. versionchanged:: 3.11 - Added ``exceptiontable`` parameter. + Added ``qualname`` and ``exceptiontable`` parameters. .. versionchanged:: 3.12 From webhook-mailer at python.org Sat Jul 29 13:52:21 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sat, 29 Jul 2023 17:52:21 -0000 Subject: [Python-checkins] [3.12] Fix the documentation for PyCode_New add `qualname` parameter (GH-107186) (#107440) Message-ID: https://github.com/python/cpython/commit/e3b5ed7b1cb8ad12aa285cf78303c6dda0953181 commit: e3b5ed7b1cb8ad12aa285cf78303c6dda0953181 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-29T17:52:17Z summary: [3.12] Fix the documentation for PyCode_New add `qualname` parameter (GH-107186) (#107440) Fix the documentation for PyCode_New add `qualname` parameter (GH-107186) (cherry picked from commit f2abeb590dae5918388f91b60b31f040d44218f0) Co-authored-by: da-woods files: M Doc/c-api/code.rst diff --git a/Doc/c-api/code.rst b/Doc/c-api/code.rst index 89fe42d1ff05f..5082b0cb6ad3f 100644 --- a/Doc/c-api/code.rst +++ b/Doc/c-api/code.rst @@ -33,7 +33,7 @@ bound into a function. Return the number of free variables in *co*. -.. c:function:: PyCodeObject* PyUnstable_Code_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable) +.. c:function:: PyCodeObject* PyUnstable_Code_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, PyObject *qualname, int firstlineno, PyObject *linetable, PyObject *exceptiontable) Return a new code object. If you need a dummy code object to create a frame, use :c:func:`PyCode_NewEmpty` instead. @@ -46,7 +46,7 @@ bound into a function. execution or VM crashes. Use this function only with extreme care. .. versionchanged:: 3.11 - Added ``exceptiontable`` parameter. + Added ``qualname`` and ``exceptiontable`` parameters. .. index:: single: PyCode_New @@ -56,7 +56,7 @@ bound into a function. The old name is deprecated, but will remain available until the signature changes again. -.. c:function:: PyCodeObject* PyUnstable_Code_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable) +.. c:function:: PyCodeObject* PyUnstable_Code_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, PyObject *qualname, int firstlineno, PyObject *linetable, PyObject *exceptiontable) Similar to :c:func:`PyUnstable_Code_New`, but with an extra "posonlyargcount" for positional-only arguments. The same caveats that apply to ``PyUnstable_Code_New`` also apply to this function. @@ -66,7 +66,7 @@ bound into a function. .. versionadded:: 3.8 as ``PyCode_NewWithPosOnlyArgs`` .. versionchanged:: 3.11 - Added ``exceptiontable`` parameter. + Added ``qualname`` and ``exceptiontable`` parameters. .. versionchanged:: 3.12 From webhook-mailer at python.org Sat Jul 29 14:07:53 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sat, 29 Jul 2023 18:07:53 -0000 Subject: [Python-checkins] [3.11] gh-106634: Corrected minor asyncio doc issues (GH-106671) (#106711) Message-ID: https://github.com/python/cpython/commit/ddccdbfc403b3e45693cb307fc05ea15b0c642cb commit: ddccdbfc403b3e45693cb307fc05ea15b0c642cb branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: kumaraditya303 date: 2023-07-29T18:07:49Z summary: [3.11] gh-106634: Corrected minor asyncio doc issues (GH-106671) (#106711) gh-106634: Corrected minor asyncio doc issues (GH-106671) (cherry picked from commit 4b4a5b70aa8d47b1e2a0582b741c31b786da762a) Co-authored-by: Chris Brett files: M Lib/asyncio/base_events.py M Lib/asyncio/events.py M Misc/ACKS diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index c5c7bd46ff8b1..d4c8238ba5217 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -716,7 +716,7 @@ def call_later(self, delay, callback, *args, context=None): always relative to the current time. Each callback will be called exactly once. If two callbacks - are scheduled for exactly the same time, it undefined which + are scheduled for exactly the same time, it is undefined which will be called first. Any positional arguments after the callback will be passed to diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py index b1799320eaa08..5fe4074faf3a0 100644 --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -613,7 +613,7 @@ class AbstractEventLoopPolicy: def get_event_loop(self): """Get the event loop for the current context. - Returns an event loop object implementing the BaseEventLoop interface, + Returns an event loop object implementing the AbstractEventLoop interface, or raises an exception in case no event loop has been set for the current context and the current policy does not specify to create one. diff --git a/Misc/ACKS b/Misc/ACKS index eba0ffba9a633..5fe93d638082d 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -224,6 +224,7 @@ Erik Bray Brian Brazil Demian Brecht Dave Brennan +Christopher Richard James Brett Tom Bridgman Anthony Briggs Keith Briggs From webhook-mailer at python.org Sat Jul 29 14:26:27 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sat, 29 Jul 2023 18:26:27 -0000 Subject: [Python-checkins] add Kumar Aditya as CODEOWNER for import.c (#107441) Message-ID: https://github.com/python/cpython/commit/2e9ddb943c2b61216f473c92a434a9adeaf483d7 commit: 2e9ddb943c2b61216f473c92a434a9adeaf483d7 branch: main author: Kumar Aditya committer: kumaraditya303 date: 2023-07-29T18:26:24Z summary: add Kumar Aditya as CODEOWNER for import.c (#107441) files: M .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 882ba9e9c9ebe..578cd71a7bd21 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -69,6 +69,7 @@ Python/traceback.c @iritkatriel # Import (including importlib). **/*import* @brettcannon @ericsnowcurrently @ncoghlan @warsaw +/Python/import.c @kumaraditya303 **/*importlib/resources/* @jaraco @warsaw @FFY00 **/importlib/metadata/* @jaraco @warsaw From webhook-mailer at python.org Sat Jul 29 14:46:55 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 29 Jul 2023 18:46:55 -0000 Subject: [Python-checkins] gh-104683: Argument clinic: Make the `filename` parameter to `Clinic` required (#107439) Message-ID: https://github.com/python/cpython/commit/6c74b2f66957912d4202869939499864ca268e23 commit: 6c74b2f66957912d4202869939499864ca268e23 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-29T19:46:52+01:00 summary: gh-104683: Argument clinic: Make the `filename` parameter to `Clinic` required (#107439) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index cb999c156ee28..3c16dc389e3eb 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2015,7 +2015,6 @@ def __post_init__(self, args: tuple[str, ...]) -> None: if self.type =='file': d = {} filename = self.clinic.filename - assert filename is not None d['path'] = filename dirname, basename = os.path.split(filename) if not dirname: @@ -2133,8 +2132,8 @@ def __init__( language: CLanguage, printer: BlockPrinter | None = None, *, + filename: str, verify: bool = True, - filename: str | None = None ) -> None: # maps strings to Parser objects. # (instantiated from the "parsers" global.) From webhook-mailer at python.org Sat Jul 29 14:47:45 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Sat, 29 Jul 2023 18:47:45 -0000 Subject: [Python-checkins] gh-104683: Argument clinic: pass `clinic` as a parameter where possible (#107435) Message-ID: https://github.com/python/cpython/commit/5113ed7a2b92e8beabebe5fe2f6e856c52fbe1a0 commit: 5113ed7a2b92e8beabebe5fe2f6e856c52fbe1a0 branch: main author: Alex Waygood committer: AlexWaygood date: 2023-07-29T19:47:42+01:00 summary: gh-104683: Argument clinic: pass `clinic` as a parameter where possible (#107435) files: M Tools/clinic/clinic.py diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 3c16dc389e3eb..defe703b825b1 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -805,7 +805,8 @@ def docstring_for_c_string( def output_templates( self, - f: Function + f: Function, + clinic: Clinic ) -> dict[str, str]: parameters = list(f.parameters.values()) assert parameters @@ -1324,7 +1325,6 @@ def parser_body( cpp_if = "#if " + conditional cpp_endif = "#endif /* " + conditional + " */" - assert clinic is not None if methoddef_define and f.full_name not in clinic.ifndef_symbols: clinic.ifndef_symbols.add(f.full_name) methoddef_ifndef = normalize_snippet(""" @@ -1490,7 +1490,7 @@ def render_function( parameters = f.render_parameters converters = [p.converter for p in parameters] - templates = self.output_templates(f) + templates = self.output_templates(f, clinic) f_self = parameters[0] selfless = parameters[1:] @@ -4613,7 +4613,7 @@ def parse(self, block: Block) -> None: self.next(self.state_terminal) self.state(None) - block.output.extend(self.clinic.language.render(clinic, block.signatures)) + block.output.extend(self.clinic.language.render(self.clinic, block.signatures)) if self.preserve_output: if block.output: From webhook-mailer at python.org Sun Jul 30 04:28:58 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sun, 30 Jul 2023 08:28:58 -0000 Subject: [Python-checkins] gh-106263: Fix segfault in `signaldict_repr` in `_decimal` module (#106270) Message-ID: https://github.com/python/cpython/commit/3979150a0d406707f6d253d7c15fb32c1e005a77 commit: 3979150a0d406707f6d253d7c15fb32c1e005a77 branch: main author: Charlie Zhao committer: kumaraditya303 date: 2023-07-30T13:58:54+05:30 summary: gh-106263: Fix segfault in `signaldict_repr` in `_decimal` module (#106270) Co-authored-by: sunmy2019 <59365878+sunmy2019 at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst M Lib/test/test_decimal.py M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/ignored.tsv diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py index db67f37608f1f..fc66a309788ac 100644 --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -5701,6 +5701,36 @@ def test_c_disallow_instantiation(self): ContextManager = type(C.localcontext()) check_disallow_instantiation(self, ContextManager) + def test_c_signaldict_segfault(self): + # See gh-106263 for details. + SignalDict = type(C.Context().flags) + sd = SignalDict() + err_msg = "invalid signal dict" + + with self.assertRaisesRegex(ValueError, err_msg): + len(sd) + + with self.assertRaisesRegex(ValueError, err_msg): + iter(sd) + + with self.assertRaisesRegex(ValueError, err_msg): + repr(sd) + + with self.assertRaisesRegex(ValueError, err_msg): + sd[C.InvalidOperation] = True + + with self.assertRaisesRegex(ValueError, err_msg): + sd[C.InvalidOperation] + + with self.assertRaisesRegex(ValueError, err_msg): + sd == C.Context().flags + + with self.assertRaisesRegex(ValueError, err_msg): + C.Context().flags == sd + + with self.assertRaisesRegex(ValueError, err_msg): + sd.copy() + @requires_docstrings @requires_cdecimal class SignatureTest(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst b/Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst new file mode 100644 index 0000000000000..23763818d84ba --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst @@ -0,0 +1,2 @@ +Fix crash when calling ``repr`` with a manually constructed SignalDict object. +Patch by Charlie Zhao. \ No newline at end of file diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 7a206973975cc..585214cc45d6c 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -314,14 +314,12 @@ value_error_int(const char *mesg) return -1; } -#ifdef CONFIG_32 static PyObject * value_error_ptr(const char *mesg) { PyErr_SetString(PyExc_ValueError, mesg); return NULL; } -#endif static int type_error_int(const char *mesg) @@ -608,6 +606,8 @@ getround(decimal_state *state, PyObject *v) initialized to new SignalDicts. Once a SignalDict is tied to a context, it cannot be deleted. */ +static const char *INVALID_SIGNALDICT_ERROR_MSG = "invalid signal dict"; + static int signaldict_init(PyObject *self, PyObject *args UNUSED, PyObject *kwds UNUSED) { @@ -616,14 +616,20 @@ signaldict_init(PyObject *self, PyObject *args UNUSED, PyObject *kwds UNUSED) } static Py_ssize_t -signaldict_len(PyObject *self UNUSED) +signaldict_len(PyObject *self) { + if (SdFlagAddr(self) == NULL) { + return value_error_int(INVALID_SIGNALDICT_ERROR_MSG); + } return SIGNAL_MAP_LEN; } static PyObject * signaldict_iter(PyObject *self) { + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } decimal_state *state = get_module_state_by_def(Py_TYPE(self)); return PyTuple_Type.tp_iter(state->SignalTuple); } @@ -632,6 +638,9 @@ static PyObject * signaldict_getitem(PyObject *self, PyObject *key) { uint32_t flag; + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } decimal_state *state = get_module_state_by_def(Py_TYPE(self)); flag = exception_as_flag(state, key); @@ -648,11 +657,15 @@ signaldict_setitem(PyObject *self, PyObject *key, PyObject *value) uint32_t flag; int x; - decimal_state *state = get_module_state_by_def(Py_TYPE(self)); + if (SdFlagAddr(self) == NULL) { + return value_error_int(INVALID_SIGNALDICT_ERROR_MSG); + } + if (value == NULL) { return value_error_int("signal keys cannot be deleted"); } + decimal_state *state = get_module_state_by_def(Py_TYPE(self)); flag = exception_as_flag(state, key); if (flag & DEC_ERRORS) { return -1; @@ -697,6 +710,10 @@ signaldict_repr(PyObject *self) const char *b[SIGNAL_MAP_LEN]; /* bool */ int i; + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } + assert(SIGNAL_MAP_LEN == 9); decimal_state *state = get_module_state_by_def(Py_TYPE(self)); @@ -721,6 +738,10 @@ signaldict_richcompare(PyObject *v, PyObject *w, int op) decimal_state *state = find_state_left_or_right(v, w); assert(PyDecSignalDict_Check(state, v)); + if ((SdFlagAddr(v) == NULL) || (SdFlagAddr(w) == NULL)) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } + if (op == Py_EQ || op == Py_NE) { if (PyDecSignalDict_Check(state, w)) { res = (SdFlags(v)==SdFlags(w)) ^ (op==Py_NE) ? Py_True : Py_False; @@ -748,6 +769,9 @@ signaldict_richcompare(PyObject *v, PyObject *w, int op) static PyObject * signaldict_copy(PyObject *self, PyObject *args UNUSED) { + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } decimal_state *state = get_module_state_by_def(Py_TYPE(self)); return flags_as_dict(state, SdFlags(self)); } diff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsv index cf6d3b2443523..099f20b5a1b43 100644 --- a/Tools/c-analyzer/cpython/ignored.tsv +++ b/Tools/c-analyzer/cpython/ignored.tsv @@ -213,6 +213,7 @@ Modules/_decimal/_decimal.c - invalid_rounding_err - Modules/_decimal/_decimal.c - invalid_signals_err - Modules/_decimal/_decimal.c - signal_map_template - Modules/_decimal/_decimal.c - ssize_constants - +Modules/_decimal/_decimal.c - INVALID_SIGNALDICT_ERROR_MSG - Modules/_elementtree.c - ExpatMemoryHandler - Modules/_hashopenssl.c - py_hashes - Modules/_hacl/Hacl_Hash_SHA1.c - _h0 - From webhook-mailer at python.org Sun Jul 30 04:48:05 2023 From: webhook-mailer at python.org (kumaraditya303) Date: Sun, 30 Jul 2023 08:48:05 -0000 Subject: [Python-checkins] [3.11] Fix the documentation for PyCode_New add qualname parameter (GH-107186) (#107454) Message-ID: https://github.com/python/cpython/commit/ff5dd9d9db0d036dbc65b7c5de49c6f3117a2355 commit: ff5dd9d9db0d036dbc65b7c5de49c6f3117a2355 branch: 3.11 author: da-woods committer: kumaraditya303 date: 2023-07-30T08:48:01Z summary: [3.11] Fix the documentation for PyCode_New add qualname parameter (GH-107186) (#107454) [3.11] Fix the documentation for PyCode_New add `qualname` parameter (GH-107186). (cherry picked from commit f2abeb590dae5918388f91b60b31f040d44218f0) files: M Doc/c-api/code.rst diff --git a/Doc/c-api/code.rst b/Doc/c-api/code.rst index ee39f2aea85b0..33682d30dff33 100644 --- a/Doc/c-api/code.rst +++ b/Doc/c-api/code.rst @@ -33,7 +33,7 @@ bound into a function. Return the number of free variables in *co*. -.. c:function:: PyCodeObject* PyCode_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable) +.. c:function:: PyCodeObject* PyCode_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, PyObject *qualname, int firstlineno, PyObject *linetable, PyObject *exceptiontable) Return a new code object. If you need a dummy code object to create a frame, use :c:func:`PyCode_NewEmpty` instead. Calling :c:func:`PyCode_New` directly @@ -43,9 +43,9 @@ bound into a function. execution or VM crashes. Use this function only with extreme care. .. versionchanged:: 3.11 - Added ``exceptiontable`` parameter. + Added ``qualname`` and ``exceptiontable`` parameters. -.. c:function:: PyCodeObject* PyCode_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable) +.. c:function:: PyCodeObject* PyCode_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, PyObject *qualname, int firstlineno, PyObject *linetable, PyObject *exceptiontable) Similar to :c:func:`PyCode_New`, but with an extra "posonlyargcount" for positional-only arguments. The same caveats that apply to ``PyCode_New`` also apply to this function. @@ -53,7 +53,7 @@ bound into a function. .. versionadded:: 3.8 .. versionchanged:: 3.11 - Added ``exceptiontable`` parameter. + Added ``qualname`` and ``exceptiontable`` parameters. .. c:function:: PyCodeObject* PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno) From webhook-mailer at python.org Sun Jul 30 11:40:41 2023 From: webhook-mailer at python.org (corona10) Date: Sun, 30 Jul 2023 15:40:41 -0000 Subject: [Python-checkins] gh-107427: Update the description of UNPACK_SEQUENCE (gh-107429) Message-ID: https://github.com/python/cpython/commit/a24e25d74bb5d30775a61853d34e0bb5a7e12c64 commit: a24e25d74bb5d30775a61853d34e0bb5a7e12c64 branch: main author: Dong-hee Na committer: corona10 date: 2023-07-31T00:40:37+09:00 summary: gh-107427: Update the description of UNPACK_SEQUENCE (gh-107429) files: M Doc/library/dis.rst diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index accc652a90649..2217c3a803545 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -922,9 +922,10 @@ iterations of the loop. .. opcode:: UNPACK_SEQUENCE (count) Unpacks ``STACK[-1]`` into *count* individual values, which are put onto the stack - right-to-left:: + right-to-left. Require there to be exactly *count* values.:: - STACK.extend(STACK.pop()[:count:-1]) + assert(len(STACK[-1]) == count) + STACK.extend(STACK.pop()[:-count-1:-1]) .. opcode:: UNPACK_EX (counts) From webhook-mailer at python.org Sun Jul 30 11:49:30 2023 From: webhook-mailer at python.org (corona10) Date: Sun, 30 Jul 2023 15:49:30 -0000 Subject: [Python-checkins] [3.12] gh-107427: Update the description of UNPACK_SEQUENCE (gh-107429) (gh-107459) Message-ID: https://github.com/python/cpython/commit/0b3d6381757a6616fdf093a2ba99cc68c018b845 commit: 0b3d6381757a6616fdf093a2ba99cc68c018b845 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: corona10 date: 2023-07-30T15:49:26Z summary: [3.12] gh-107427: Update the description of UNPACK_SEQUENCE (gh-107429) (gh-107459) files: M Doc/library/dis.rst diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 97d5121c4e8a5..085fd2baba55c 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -864,9 +864,10 @@ iterations of the loop. .. opcode:: UNPACK_SEQUENCE (count) Unpacks ``STACK[-1]`` into *count* individual values, which are put onto the stack - right-to-left:: + right-to-left. Require there to be exactly *count* values.:: - STACK.extend(STACK.pop()[:count:-1]) + assert(len(STACK[-1]) == count) + STACK.extend(STACK.pop()[:-count-1:-1]) .. opcode:: UNPACK_EX (counts) From webhook-mailer at python.org Mon Jul 31 01:21:47 2023 From: webhook-mailer at python.org (ned-deily) Date: Mon, 31 Jul 2023 05:21:47 -0000 Subject: [Python-checkins] [3.12] gh-99079: Update macOS installer to use OpenSSL 3.0.9. (gh-107472) Message-ID: https://github.com/python/cpython/commit/23655fc62714aeb374d77bd895e318fb55689fe4 commit: 23655fc62714aeb374d77bd895e318fb55689fe4 branch: 3.12 author: Ned Deily committer: ned-deily date: 2023-07-31T01:21:43-04:00 summary: [3.12] gh-99079: Update macOS installer to use OpenSSL 3.0.9. (gh-107472) files: A Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst M Doc/license.rst M Mac/BuildScript/build-installer.py diff --git a/Doc/license.rst b/Doc/license.rst index 005d048b6eb20..812a0adffe1a4 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -659,134 +659,186 @@ The modules :mod:`hashlib`, :mod:`posix`, :mod:`ssl`, :mod:`crypt` use the OpenSSL library for added performance if made available by the operating system. Additionally, the Windows and macOS installers for Python may include a copy of the OpenSSL libraries, so we include a copy -of the OpenSSL license here:: - - - LICENSE ISSUES - ============== - - The OpenSSL toolkit stays under a dual license, i.e. both the conditions of - the OpenSSL License and the original SSLeay license apply to the toolkit. - See below for the actual license texts. Actually both licenses are BSD-style - Open Source licenses. In case of any license issues related to OpenSSL - please contact openssl-core at openssl.org. - - OpenSSL License - --------------- - - /* ==================================================================== - * Copyright (c) 1998-2008 The OpenSSL Project. All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * - * 3. All advertising materials mentioning features or use of this - * software must display the following acknowledgment: - * "This product includes software developed by the OpenSSL Project - * for use in the OpenSSL Toolkit. (http://www.openssl.org/)" - * - * 4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to - * endorse or promote products derived from this software without - * prior written permission. For written permission, please contact - * openssl-core at openssl.org. - * - * 5. Products derived from this software may not be called "OpenSSL" - * nor may "OpenSSL" appear in their names without prior written - * permission of the OpenSSL Project. - * - * 6. Redistributions of any form whatsoever must retain the following - * acknowledgment: - * "This product includes software developed by the OpenSSL Project - * for use in the OpenSSL Toolkit (http://www.openssl.org/)" - * - * THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY - * EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR - * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT - * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - * ==================================================================== - * - * This product includes cryptographic software written by Eric Young - * (eay at cryptsoft.com). This product includes software written by Tim - * Hudson (tjh at cryptsoft.com). - * - */ - - Original SSLeay License - ----------------------- - - /* Copyright (C) 1995-1998 Eric Young (eay at cryptsoft.com) - * All rights reserved. - * - * This package is an SSL implementation written - * by Eric Young (eay at cryptsoft.com). - * The implementation was written so as to conform with Netscapes SSL. - * - * This library is free for commercial and non-commercial use as long as - * the following conditions are aheared to. The following conditions - * apply to all code found in this distribution, be it the RC4, RSA, - * lhash, DES, etc., code; not just the SSL code. The SSL documentation - * included with this distribution is covered by the same copyright terms - * except that the holder is Tim Hudson (tjh at cryptsoft.com). - * - * Copyright remains Eric Young's, and as such any Copyright notices in - * the code are not to be removed. - * If this package is used in a product, Eric Young should be given attribution - * as the author of the parts of the library used. - * This can be in the form of a textual message at program startup or - * in documentation (online or textual) provided with the package. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * 3. All advertising materials mentioning features or use of this software - * must display the following acknowledgement: - * "This product includes cryptographic software written by - * Eric Young (eay at cryptsoft.com)" - * The word 'cryptographic' can be left out if the rouines from the library - * being used are not cryptographic related :-). - * 4. If you include any Windows specific code (or a derivative thereof) from - * the apps directory (application code) you must include an acknowledgement: - * "This product includes software written by Tim Hudson (tjh at cryptsoft.com)" - * - * THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - * The licence and distribution terms for any publically available version or - * derivative of this code cannot be changed. i.e. this code cannot simply be - * copied and put under another distribution licence - * [including the GNU Public Licence.] - */ +of the OpenSSL license here. For the OpenSSL 3.0 release, +and later releases derived from that, the Apache License v2 applies:: + + + Apache License + Version 2.0, January 2004 + https://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS expat diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index 9729759434a9f..0fc25ec74565d 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -246,9 +246,9 @@ def library_recipes(): result.extend([ dict( - name="OpenSSL 1.1.1u", - url="https://www.openssl.org/source/openssl-1.1.1u.tar.gz", - checksum='e2f8d84b523eecd06c7be7626830370300fbcc15386bf5142d72758f6963ebc6', + name="OpenSSL 3.0.9", + url="https://www.openssl.org/source/openssl-3.0.9.tar.gz", + checksum='eb1ab04781474360f77c318ab89d8c5a03abc38e63d65a603cabbf1b00a1dc90', buildrecipe=build_universal_openssl, configure=None, install=None, diff --git a/Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst b/Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst new file mode 100644 index 0000000000000..d0eef4ec1003c --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst @@ -0,0 +1 @@ +Update macOS installer to use OpenSSL 3.0.9. From webhook-mailer at python.org Mon Jul 31 01:22:37 2023 From: webhook-mailer at python.org (ned-deily) Date: Mon, 31 Jul 2023 05:22:37 -0000 Subject: [Python-checkins] [3.12] Update macOS installer screens for 3.12rc/final. (GH-107473) Message-ID: https://github.com/python/cpython/commit/08f8165b132390b27cce8d78203383bf3ca364bc commit: 08f8165b132390b27cce8d78203383bf3ca364bc branch: 3.12 author: Ned Deily committer: ned-deily date: 2023-07-31T01:22:34-04:00 summary: [3.12] Update macOS installer screens for 3.12rc/final. (GH-107473) files: M Mac/BuildScript/resources/ReadMe.rtf M Mac/BuildScript/resources/Welcome.rtf diff --git a/Mac/BuildScript/resources/ReadMe.rtf b/Mac/BuildScript/resources/ReadMe.rtf index 5bc356d526704..384840cd92dc5 100644 --- a/Mac/BuildScript/resources/ReadMe.rtf +++ b/Mac/BuildScript/resources/ReadMe.rtf @@ -8,25 +8,19 @@ \f0\fs24 \cf0 This package will install Python $FULL_VERSION for macOS $MACOSX_DEPLOYMENT_TARGET for the following architecture(s): $ARCHITECTURES.\ \ -\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0 - -\f1\b \cf0 NOTE: -\f0\b0 This is a beta preview of Python 3.12.0, the next feature release of Python 3. It is not intended for production use.\ -\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 -\cf0 \ \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 \f1\b \cf0 \ul \ulc0 Certificate verification and OpenSSL\ \f0\b0 \ulnone \ -This package includes its own private copy of OpenSSL 1.1.1. The trust certificates in system and user keychains managed by the +This package includes its own private copy of OpenSSL 3.0. The trust certificates in system and user keychains managed by the \f2\i Keychain Access \f0\i0 application and the \f2\i security \f0\i0 command line utility are not used as defaults by the Python \f3 ssl \f0 module. A sample command script is included in -\f3 /Applications/Python 3.11 +\f3 /Applications/Python 3.12 \f0 to install a curated bundle of default root certificates from the third-party \f3 certifi \f0 package ({\field{\*\fldinst{HYPERLINK "https://pypi.org/project/certifi/"}}{\fldrslt https://pypi.org/project/certifi/}}). Double-click on diff --git a/Mac/BuildScript/resources/Welcome.rtf b/Mac/BuildScript/resources/Welcome.rtf index dfb02a0b314b7..8ae9b01b6ddbc 100644 --- a/Mac/BuildScript/resources/Welcome.rtf +++ b/Mac/BuildScript/resources/Welcome.rtf @@ -12,9 +12,8 @@ \f1\b macOS $MACOSX_DEPLOYMENT_TARGET \f0\b0 .\ \ -\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0 -\f1\b \cf0 Python for macOS +\f1\b Python for macOS \f0\b0 consists of the {\field{\*\fldinst{HYPERLINK "https://www.python.org"}}{\fldrslt Python}} programming language interpreter and its batteries-included standard library to allow easy access to macOS features. It also includes the Python integrated development environment, \f1\b IDLE \f0\b0 . You can also use the included @@ -24,30 +23,4 @@ At the end of this install, click on \f2 Install Certificates \f0 to install a set of current SSL root certificates.\ -\ - -\f1\b [UPDATE: fixed in macOS 13.4] macOS 13 Ventura users -\f0\b0 : Due to an issue with the macOS -\f1\b Installer -\f0\b0 app in macOS 13 Ventura updates prior to macOS 13.4, installation of some third-party packages including this Python package may fail with a vague -\f1\b "The installer encountered an error" -\f0\b0 message if the -\f1\b Installer -\f0\b0 app does not have permission to access the folder containing the downloaded installer file, typically in the -\f1\b Downloads -\f0\b0 folder. Go to -\f1\b System Settings -\f0\b0 -> -\f1\b Privacy & Security -\f0\b0 -> -\f1\b Files and Folders -\f0\b0 , then click the mark in front of -\f1\b Installer -\f0\b0 to expand, and enable -\f1\b Downloads Folder -\f0\b0 by moving the toggle to the right. See {\field{\*\fldinst{HYPERLINK "https://github.com/python/cpython/issues/103207"}}{\fldrslt https://github.com/python/cpython/issues/103207}} for up-to-date information on this issue. This problem has been resolved in macOS 13.4.\ -\ - -\f1\b NOTE: -\f0\b0 This is a beta test preview of Python 3.12.0, the next feature release of Python 3. It is not intended for production use.\ } \ No newline at end of file From webhook-mailer at python.org Mon Jul 31 01:58:39 2023 From: webhook-mailer at python.org (ned-deily) Date: Mon, 31 Jul 2023 05:58:39 -0000 Subject: [Python-checkins] gh-99079: Update macOS installer to use OpenSSL 3.0.9. (GH-107474) Message-ID: https://github.com/python/cpython/commit/83e0976f897168fcbeff3dbe8912e80b51e444ed commit: 83e0976f897168fcbeff3dbe8912e80b51e444ed branch: main author: Ned Deily committer: ned-deily date: 2023-07-31T05:58:35Z summary: gh-99079: Update macOS installer to use OpenSSL 3.0.9. (GH-107474) files: A Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst M Doc/license.rst M Mac/BuildScript/build-installer.py diff --git a/Doc/license.rst b/Doc/license.rst index 1dadb6264a005..be8efa7061137 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -659,134 +659,186 @@ The modules :mod:`hashlib`, :mod:`posix` and :mod:`ssl` use the OpenSSL library for added performance if made available by the operating system. Additionally, the Windows and macOS installers for Python may include a copy of the OpenSSL libraries, so we include a copy -of the OpenSSL license here:: - - - LICENSE ISSUES - ============== - - The OpenSSL toolkit stays under a dual license, i.e. both the conditions of - the OpenSSL License and the original SSLeay license apply to the toolkit. - See below for the actual license texts. Actually both licenses are BSD-style - Open Source licenses. In case of any license issues related to OpenSSL - please contact openssl-core at openssl.org. - - OpenSSL License - --------------- - - /* ==================================================================== - * Copyright (c) 1998-2008 The OpenSSL Project. All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * - * 3. All advertising materials mentioning features or use of this - * software must display the following acknowledgment: - * "This product includes software developed by the OpenSSL Project - * for use in the OpenSSL Toolkit. (http://www.openssl.org/)" - * - * 4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to - * endorse or promote products derived from this software without - * prior written permission. For written permission, please contact - * openssl-core at openssl.org. - * - * 5. Products derived from this software may not be called "OpenSSL" - * nor may "OpenSSL" appear in their names without prior written - * permission of the OpenSSL Project. - * - * 6. Redistributions of any form whatsoever must retain the following - * acknowledgment: - * "This product includes software developed by the OpenSSL Project - * for use in the OpenSSL Toolkit (http://www.openssl.org/)" - * - * THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY - * EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR - * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT - * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - * ==================================================================== - * - * This product includes cryptographic software written by Eric Young - * (eay at cryptsoft.com). This product includes software written by Tim - * Hudson (tjh at cryptsoft.com). - * - */ - - Original SSLeay License - ----------------------- - - /* Copyright (C) 1995-1998 Eric Young (eay at cryptsoft.com) - * All rights reserved. - * - * This package is an SSL implementation written - * by Eric Young (eay at cryptsoft.com). - * The implementation was written so as to conform with Netscapes SSL. - * - * This library is free for commercial and non-commercial use as long as - * the following conditions are aheared to. The following conditions - * apply to all code found in this distribution, be it the RC4, RSA, - * lhash, DES, etc., code; not just the SSL code. The SSL documentation - * included with this distribution is covered by the same copyright terms - * except that the holder is Tim Hudson (tjh at cryptsoft.com). - * - * Copyright remains Eric Young's, and as such any Copyright notices in - * the code are not to be removed. - * If this package is used in a product, Eric Young should be given attribution - * as the author of the parts of the library used. - * This can be in the form of a textual message at program startup or - * in documentation (online or textual) provided with the package. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * 3. All advertising materials mentioning features or use of this software - * must display the following acknowledgement: - * "This product includes cryptographic software written by - * Eric Young (eay at cryptsoft.com)" - * The word 'cryptographic' can be left out if the rouines from the library - * being used are not cryptographic related :-). - * 4. If you include any Windows specific code (or a derivative thereof) from - * the apps directory (application code) you must include an acknowledgement: - * "This product includes software written by Tim Hudson (tjh at cryptsoft.com)" - * - * THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - * The licence and distribution terms for any publically available version or - * derivative of this code cannot be changed. i.e. this code cannot simply be - * copied and put under another distribution licence - * [including the GNU Public Licence.] - */ +of the OpenSSL license here. For the OpenSSL 3.0 release, +and later releases derived from that, the Apache License v2 applies:: + + + Apache License + Version 2.0, January 2004 + https://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS expat diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index 9729759434a9f..0fc25ec74565d 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -246,9 +246,9 @@ def library_recipes(): result.extend([ dict( - name="OpenSSL 1.1.1u", - url="https://www.openssl.org/source/openssl-1.1.1u.tar.gz", - checksum='e2f8d84b523eecd06c7be7626830370300fbcc15386bf5142d72758f6963ebc6', + name="OpenSSL 3.0.9", + url="https://www.openssl.org/source/openssl-3.0.9.tar.gz", + checksum='eb1ab04781474360f77c318ab89d8c5a03abc38e63d65a603cabbf1b00a1dc90', buildrecipe=build_universal_openssl, configure=None, install=None, diff --git a/Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst b/Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst new file mode 100644 index 0000000000000..d0eef4ec1003c --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst @@ -0,0 +1 @@ +Update macOS installer to use OpenSSL 3.0.9. From webhook-mailer at python.org Mon Jul 31 02:05:51 2023 From: webhook-mailer at python.org (ned-deily) Date: Mon, 31 Jul 2023 06:05:51 -0000 Subject: [Python-checkins] Update macOS installer screens to 3.13. (GH-107475) Message-ID: https://github.com/python/cpython/commit/68f94715bb4fce6a06105ce4090581612ed0d176 commit: 68f94715bb4fce6a06105ce4090581612ed0d176 branch: main author: Ned Deily committer: ned-deily date: 2023-07-31T06:05:47Z summary: Update macOS installer screens to 3.13. (GH-107475) files: M Mac/BuildScript/resources/ReadMe.rtf M Mac/BuildScript/resources/Welcome.rtf diff --git a/Mac/BuildScript/resources/ReadMe.rtf b/Mac/BuildScript/resources/ReadMe.rtf index 5bc356d526704..efd76b9b1ae64 100644 --- a/Mac/BuildScript/resources/ReadMe.rtf +++ b/Mac/BuildScript/resources/ReadMe.rtf @@ -11,7 +11,7 @@ \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0 \f1\b \cf0 NOTE: -\f0\b0 This is a beta preview of Python 3.12.0, the next feature release of Python 3. It is not intended for production use.\ +\f0\b0 This is an alpha preview of Python 3.13.0, the next feature release of Python 3. It is not intended for production use.\ \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 \cf0 \ \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 @@ -19,14 +19,14 @@ \f1\b \cf0 \ul \ulc0 Certificate verification and OpenSSL\ \f0\b0 \ulnone \ -This package includes its own private copy of OpenSSL 1.1.1. The trust certificates in system and user keychains managed by the +This package includes its own private copy of OpenSSL 3.0. The trust certificates in system and user keychains managed by the \f2\i Keychain Access \f0\i0 application and the \f2\i security \f0\i0 command line utility are not used as defaults by the Python \f3 ssl \f0 module. A sample command script is included in -\f3 /Applications/Python 3.11 +\f3 /Applications/Python 3.13 \f0 to install a curated bundle of default root certificates from the third-party \f3 certifi \f0 package ({\field{\*\fldinst{HYPERLINK "https://pypi.org/project/certifi/"}}{\fldrslt https://pypi.org/project/certifi/}}). Double-click on diff --git a/Mac/BuildScript/resources/Welcome.rtf b/Mac/BuildScript/resources/Welcome.rtf index 83b7aa9d883a1..79851e1f4a69c 100644 --- a/Mac/BuildScript/resources/Welcome.rtf +++ b/Mac/BuildScript/resources/Welcome.rtf @@ -12,9 +12,8 @@ \f1\b macOS $MACOSX_DEPLOYMENT_TARGET \f0\b0 .\ \ -\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0 -\f1\b \cf0 Python for macOS +\f1\b Python for macOS \f0\b0 consists of the {\field{\*\fldinst{HYPERLINK "https://www.python.org"}}{\fldrslt Python}} programming language interpreter and its batteries-included standard library to allow easy access to macOS features. It also includes the Python integrated development environment, \f1\b IDLE \f0\b0 . You can also use the included @@ -27,5 +26,5 @@ At the end of this install, click on \ \f1\b NOTE: -\f0\b0 This is a beta test preview of Python 3.12.0, the next feature release of Python 3. It is not intended for production use.\ +\f0\b0 This is an alpha test preview of Python 3.13.0, the next feature release of Python 3. It is not intended for production use.\ } \ No newline at end of file From webhook-mailer at python.org Mon Jul 31 02:15:32 2023 From: webhook-mailer at python.org (ned-deily) Date: Mon, 31 Jul 2023 06:15:32 -0000 Subject: [Python-checkins] [3.11] Update macOS installer screens for 3.11.5. (GH-107477) Message-ID: https://github.com/python/cpython/commit/9690efc5552153cb7276b7d481827f26a4f3ae2b commit: 9690efc5552153cb7276b7d481827f26a4f3ae2b branch: 3.11 author: Ned Deily committer: ned-deily date: 2023-07-31T06:15:27Z summary: [3.11] Update macOS installer screens for 3.11.5. (GH-107477) files: M Mac/BuildScript/resources/ReadMe.rtf M Mac/BuildScript/resources/Welcome.rtf diff --git a/Mac/BuildScript/resources/ReadMe.rtf b/Mac/BuildScript/resources/ReadMe.rtf index 2d6ca8f261898..94e61ac5a4df7 100644 --- a/Mac/BuildScript/resources/ReadMe.rtf +++ b/Mac/BuildScript/resources/ReadMe.rtf @@ -1,4 +1,4 @@ -{\rtf1\ansi\ansicpg1252\cocoartf2639 +{\rtf1\ansi\ansicpg1252\cocoartf2709 \cocoatextscaling0\cocoaplatform0{\fonttbl\f0\fswiss\fcharset0 Helvetica;\f1\fswiss\fcharset0 Helvetica-Bold;\f2\fswiss\fcharset0 Helvetica-Oblique; \f3\fmodern\fcharset0 CourierNewPSMT;\f4\fmodern\fcharset0 Courier;} {\colortbl;\red255\green255\blue255;} @@ -13,7 +13,7 @@ \f1\b \cf0 \ul \ulc0 Certificate verification and OpenSSL\ \f0\b0 \ulnone \ -This package includes its own private copy of OpenSSL 1.1.1. The trust certificates in system and user keychains managed by the +This package includes its own private copy of OpenSSL 3.0. The trust certificates in system and user keychains managed by the \f2\i Keychain Access \f0\i0 application and the \f2\i security diff --git a/Mac/BuildScript/resources/Welcome.rtf b/Mac/BuildScript/resources/Welcome.rtf index 93500bc0e1e2e..86f4dfdbe9c0b 100644 --- a/Mac/BuildScript/resources/Welcome.rtf +++ b/Mac/BuildScript/resources/Welcome.rtf @@ -25,27 +25,4 @@ At the end of this install, click on \f2 Install Certificates \f0 to install a set of current SSL root certificates.\ \ - -\f1\b [UPDATE: fixed in macOS 13.4] macOS 13 Ventura users -\f0\b0 : Due to an issue with the macOS -\f1\b Installer -\f0\b0 app in macOS 13 Ventura updates prior to macOS 13.4, installation of some third-party packages including this Python package may fail with a vague -\f1\b "The installer encountered an error" -\f0\b0 message if the -\f1\b Installer -\f0\b0 app does not have permission to access the folder containing the downloaded installer file, typically in the -\f1\b Downloads -\f0\b0 folder. Go to -\f1\b System Settings -\f0\b0 -> -\f1\b Privacy & Security -\f0\b0 -> -\f1\b Files and Folders -\f0\b0 , then click the mark in front of -\f1\b Installer -\f0\b0 to expand, and enable -\f1\b Downloads Folder -\f0\b0 by moving the toggle to the right. See {\field{\*\fldinst{HYPERLINK "https://github.com/python/cpython/issues/103207"}}{\fldrslt https://github.com/python/cpython/issues/103207}} for up-to-date information on this issue. This problem has been resolved in macOS 13.4.\ -\ -\ } \ No newline at end of file From webhook-mailer at python.org Mon Jul 31 02:16:43 2023 From: webhook-mailer at python.org (ned-deily) Date: Mon, 31 Jul 2023 06:16:43 -0000 Subject: [Python-checkins] [3.11] gh-99079: Update macOS installer to use OpenSSL 3.0.9. (GH-107476) Message-ID: https://github.com/python/cpython/commit/b8beb0762024b70fd176310fd5d21fd14b5df387 commit: b8beb0762024b70fd176310fd5d21fd14b5df387 branch: 3.11 author: Ned Deily committer: ned-deily date: 2023-07-31T06:16:39Z summary: [3.11] gh-99079: Update macOS installer to use OpenSSL 3.0.9. (GH-107476) files: A Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst M Doc/license.rst M Mac/BuildScript/build-installer.py diff --git a/Doc/license.rst b/Doc/license.rst index 37a3d4fa9e5f2..4e8a25ed5ecc1 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -657,134 +657,186 @@ The modules :mod:`hashlib`, :mod:`posix`, :mod:`ssl`, :mod:`crypt` use the OpenSSL library for added performance if made available by the operating system. Additionally, the Windows and macOS installers for Python may include a copy of the OpenSSL libraries, so we include a copy -of the OpenSSL license here:: - - - LICENSE ISSUES - ============== - - The OpenSSL toolkit stays under a dual license, i.e. both the conditions of - the OpenSSL License and the original SSLeay license apply to the toolkit. - See below for the actual license texts. Actually both licenses are BSD-style - Open Source licenses. In case of any license issues related to OpenSSL - please contact openssl-core at openssl.org. - - OpenSSL License - --------------- - - /* ==================================================================== - * Copyright (c) 1998-2008 The OpenSSL Project. All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * - * 3. All advertising materials mentioning features or use of this - * software must display the following acknowledgment: - * "This product includes software developed by the OpenSSL Project - * for use in the OpenSSL Toolkit. (http://www.openssl.org/)" - * - * 4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to - * endorse or promote products derived from this software without - * prior written permission. For written permission, please contact - * openssl-core at openssl.org. - * - * 5. Products derived from this software may not be called "OpenSSL" - * nor may "OpenSSL" appear in their names without prior written - * permission of the OpenSSL Project. - * - * 6. Redistributions of any form whatsoever must retain the following - * acknowledgment: - * "This product includes software developed by the OpenSSL Project - * for use in the OpenSSL Toolkit (http://www.openssl.org/)" - * - * THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY - * EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR - * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT - * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - * ==================================================================== - * - * This product includes cryptographic software written by Eric Young - * (eay at cryptsoft.com). This product includes software written by Tim - * Hudson (tjh at cryptsoft.com). - * - */ - - Original SSLeay License - ----------------------- - - /* Copyright (C) 1995-1998 Eric Young (eay at cryptsoft.com) - * All rights reserved. - * - * This package is an SSL implementation written - * by Eric Young (eay at cryptsoft.com). - * The implementation was written so as to conform with Netscapes SSL. - * - * This library is free for commercial and non-commercial use as long as - * the following conditions are aheared to. The following conditions - * apply to all code found in this distribution, be it the RC4, RSA, - * lhash, DES, etc., code; not just the SSL code. The SSL documentation - * included with this distribution is covered by the same copyright terms - * except that the holder is Tim Hudson (tjh at cryptsoft.com). - * - * Copyright remains Eric Young's, and as such any Copyright notices in - * the code are not to be removed. - * If this package is used in a product, Eric Young should be given attribution - * as the author of the parts of the library used. - * This can be in the form of a textual message at program startup or - * in documentation (online or textual) provided with the package. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * 3. All advertising materials mentioning features or use of this software - * must display the following acknowledgement: - * "This product includes cryptographic software written by - * Eric Young (eay at cryptsoft.com)" - * The word 'cryptographic' can be left out if the rouines from the library - * being used are not cryptographic related :-). - * 4. If you include any Windows specific code (or a derivative thereof) from - * the apps directory (application code) you must include an acknowledgement: - * "This product includes software written by Tim Hudson (tjh at cryptsoft.com)" - * - * THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - * The licence and distribution terms for any publically available version or - * derivative of this code cannot be changed. i.e. this code cannot simply be - * copied and put under another distribution licence - * [including the GNU Public Licence.] - */ +of the OpenSSL license here. For the OpenSSL 3.0 release, +and later releases derived from that, the Apache License v2 applies:: + + + Apache License + Version 2.0, January 2004 + https://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS expat diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index 436276aee38bc..71fbe3c0777e1 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -246,9 +246,9 @@ def library_recipes(): result.extend([ dict( - name="OpenSSL 1.1.1u", - url="https://www.openssl.org/source/openssl-1.1.1u.tar.gz", - checksum='e2f8d84b523eecd06c7be7626830370300fbcc15386bf5142d72758f6963ebc6', + name="OpenSSL 3.0.9", + url="https://www.openssl.org/source/openssl-3.0.9.tar.gz", + checksum='eb1ab04781474360f77c318ab89d8c5a03abc38e63d65a603cabbf1b00a1dc90', buildrecipe=build_universal_openssl, configure=None, install=None, diff --git a/Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst b/Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst new file mode 100644 index 0000000000000..d0eef4ec1003c --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2023-07-30-23-42-20.gh-issue-99079.JAtoh1.rst @@ -0,0 +1 @@ +Update macOS installer to use OpenSSL 3.0.9. From webhook-mailer at python.org Mon Jul 31 03:10:56 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 07:10:56 -0000 Subject: [Python-checkins] gh-46376: Return existing pointer when possible in ctypes (#107131) Message-ID: https://github.com/python/cpython/commit/08447b5deb47e2a0df87fa0a0576d300e5c909b4 commit: 08447b5deb47e2a0df87fa0a0576d300e5c909b4 branch: main author: Konstantin committer: ambv date: 2023-07-31T09:10:53+02:00 summary: gh-46376: Return existing pointer when possible in ctypes (#107131) files: A Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst M Lib/test/test_ctypes/test_keeprefs.py M Modules/_ctypes/_ctypes.c diff --git a/Lib/test/test_ctypes/test_keeprefs.py b/Lib/test/test_ctypes/test_keeprefs.py index 23b03b64b4a71..c6fe1de62eae7 100644 --- a/Lib/test/test_ctypes/test_keeprefs.py +++ b/Lib/test/test_ctypes/test_keeprefs.py @@ -98,6 +98,33 @@ def test_p_cint(self): x = pointer(i) self.assertEqual(x._objects, {'1': i}) + def test_pp_ownership(self): + d = c_int(123) + n = c_int(456) + + p = pointer(d) + pp = pointer(p) + + self.assertIs(pp._objects['1'], p) + self.assertIs(pp._objects['0']['1'], d) + + pp.contents.contents = n + + self.assertIs(pp._objects['1'], p) + self.assertIs(pp._objects['0']['1'], n) + + self.assertIs(p._objects['1'], n) + self.assertEqual(len(p._objects), 1) + + del d + del p + + self.assertIs(pp._objects['0']['1'], n) + self.assertEqual(len(pp._objects), 2) + + del n + + self.assertEqual(len(pp._objects), 2) class PointerToStructure(unittest.TestCase): def test(self): diff --git a/Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst b/Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst new file mode 100644 index 0000000000000..8e8f0245b4539 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst @@ -0,0 +1 @@ +Prevent memory leak and use-after-free when using pointers to pointers with ctypes diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index c20d6ae55a06e..8d4fd30150f19 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -5129,6 +5129,8 @@ static PyObject * Pointer_get_contents(CDataObject *self, void *closure) { StgDictObject *stgdict; + PyObject *keep, *ptr_probe; + CDataObject *ptr2ptr; if (*(void **)self->b_ptr == NULL) { PyErr_SetString(PyExc_ValueError, @@ -5138,6 +5140,33 @@ Pointer_get_contents(CDataObject *self, void *closure) stgdict = PyObject_stgdict((PyObject *)self); assert(stgdict); /* Cannot be NULL for pointer instances */ + + keep = GetKeepedObjects(self); + if (keep != NULL) { + // check if it's a pointer to a pointer: + // pointers will have '0' key in the _objects + ptr_probe = PyDict_GetItemString(keep, "0"); + + if (ptr_probe != NULL) { + ptr2ptr = (CDataObject*) PyDict_GetItemString(keep, "1"); + if (ptr2ptr == NULL) { + PyErr_SetString(PyExc_ValueError, + "Unexpected NULL pointer in _objects"); + return NULL; + } + // don't construct a new object, + // return existing one instead to preserve refcount + assert( + *(void**) self->b_ptr == ptr2ptr->b_ptr || + *(void**) self->b_value.c == ptr2ptr->b_ptr || + *(void**) self->b_ptr == ptr2ptr->b_value.c || + *(void**) self->b_value.c == ptr2ptr->b_value.c + ); // double-check that we are returning the same thing + Py_INCREF(ptr2ptr); + return (PyObject *) ptr2ptr; + } + } + return PyCData_FromBaseObj(stgdict->proto, (PyObject *)self, 0, *(void **)self->b_ptr); From webhook-mailer at python.org Mon Jul 31 03:12:11 2023 From: webhook-mailer at python.org (ned-deily) Date: Mon, 31 Jul 2023 07:12:11 -0000 Subject: [Python-checkins] [3.12] gh-99079: add What's New item (GH-107481) Message-ID: https://github.com/python/cpython/commit/f9f9bc9bb74ae58155428527dc1b67d5e1b8c80f commit: f9f9bc9bb74ae58155428527dc1b67d5e1b8c80f branch: 3.12 author: Ned Deily committer: ned-deily date: 2023-07-31T07:12:07Z summary: [3.12] gh-99079: add What's New item (GH-107481) files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 71b9358323e3d..2ec25eb25ecb4 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1586,6 +1586,8 @@ Build Changes :file:`!configure`. (Contributed by Christian Heimes in :gh:`89886`.) +* Windows builds and macOS installers from python.org now use OpenSSL 3.0. + C API Changes ============= From webhook-mailer at python.org Mon Jul 31 03:25:33 2023 From: webhook-mailer at python.org (ned-deily) Date: Mon, 31 Jul 2023 07:25:33 -0000 Subject: [Python-checkins] [3.11] gh-99079: add What's New item (GH-107482) Message-ID: https://github.com/python/cpython/commit/ed5be956ee20b6a7993bf8477e11d35498794887 commit: ed5be956ee20b6a7993bf8477e11d35498794887 branch: 3.11 author: Ned Deily committer: ned-deily date: 2023-07-31T07:25:29Z summary: [3.11] gh-99079: add What's New item (GH-107482) files: M Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 91f54f38428a7..f102aa599069c 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -2712,4 +2712,13 @@ tarfile In Python 3.14, the default will switch to ``'data'``. (Contributed by Petr Viktorin in :pep:`706`.) + +Notable Changes in 3.11.5 +========================= + +OpenSSL +------- + +* Windows builds and macOS installers from python.org now use OpenSSL 3.0. + .. _libb2: https://www.blake2.net/ From webhook-mailer at python.org Mon Jul 31 04:28:40 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 08:28:40 -0000 Subject: [Python-checkins] [3.12] gh-105751, test_ctypes: Remove disabled tests (GH-105826) (#107483) Message-ID: https://github.com/python/cpython/commit/1d21c99e1d234041cf0593df3dc5591c10223d5d commit: 1d21c99e1d234041cf0593df3dc5591c10223d5d branch: 3.12 author: ?ukasz Langa committer: ambv date: 2023-07-31T10:28:36+02:00 summary: [3.12] gh-105751, test_ctypes: Remove disabled tests (GH-105826) (#107483) * The following tests were disabled since the initial ctypes commit in 2006, commit babddfca758abe34ff12023f63b18d745fae7ca9: * Callbacks.test_char_p() * DeletePointerTestCase.test_X() * NumberTestCase.test_perf() * StructureTestCase.test_subclass_creation() * Tests.test_X() of test_byteswap * NumberTestCase.test_bool_from_address() was disabled in 2007 by commit 5dc4fe09b7648f9801558e766b21a3d3b2dcad3b. * Remove check_perf() and run_test() of test_numbers. (cherry picked from commit 8f10140e74d141a0a894702044e213e6f0690d9c) Co-authored-by: Victor Stinner files: M Lib/test/test_ctypes/test_byteswap.py M Lib/test/test_ctypes/test_callbacks.py M Lib/test/test_ctypes/test_keeprefs.py M Lib/test/test_ctypes/test_numbers.py M Lib/test/test_ctypes/test_structures.py diff --git a/Lib/test/test_ctypes/test_byteswap.py b/Lib/test/test_ctypes/test_byteswap.py index 7e98559dfbccb..caefb774cc537 100644 --- a/Lib/test/test_ctypes/test_byteswap.py +++ b/Lib/test/test_ctypes/test_byteswap.py @@ -14,14 +14,6 @@ def bin(s): # For Structures and Unions, these types are created on demand. class Test(unittest.TestCase): - @unittest.skip('test disabled') - def test_X(self): - print(sys.byteorder, file=sys.stderr) - for i in range(32): - bits = BITS() - setattr(bits, "i%s" % i, 1) - dump(bits) - def test_slots(self): class BigPoint(BigEndianStructure): __slots__ = () diff --git a/Lib/test/test_ctypes/test_callbacks.py b/Lib/test/test_ctypes/test_callbacks.py index b185e388ab152..0069c640d7927 100644 --- a/Lib/test/test_ctypes/test_callbacks.py +++ b/Lib/test/test_ctypes/test_callbacks.py @@ -93,14 +93,6 @@ def test_char(self): self.check_type(c_char, b"x") self.check_type(c_char, b"a") - # disabled: would now (correctly) raise a RuntimeWarning about - # a memory leak. A callback function cannot return a non-integral - # C type without causing a memory leak. - @unittest.skip('test disabled') - def test_char_p(self): - self.check_type(c_char_p, "abc") - self.check_type(c_char_p, "def") - def test_pyobject(self): o = () from sys import getrefcount as grc diff --git a/Lib/test/test_ctypes/test_keeprefs.py b/Lib/test/test_ctypes/test_keeprefs.py index 94c02573fa19d..e20adc7696f50 100644 --- a/Lib/test/test_ctypes/test_keeprefs.py +++ b/Lib/test/test_ctypes/test_keeprefs.py @@ -93,37 +93,6 @@ def test_p_cint(self): x = pointer(i) self.assertEqual(x._objects, {'1': i}) -class DeletePointerTestCase(unittest.TestCase): - @unittest.skip('test disabled') - def test_X(self): - class X(Structure): - _fields_ = [("p", POINTER(c_char_p))] - x = X() - i = c_char_p("abc def") - from sys import getrefcount as grc - print("2?", grc(i)) - x.p = pointer(i) - print("3?", grc(i)) - for i in range(320): - c_int(99) - x.p[0] - print(x.p[0]) -## del x -## print "2?", grc(i) -## del i - import gc - gc.collect() - for i in range(320): - c_int(99) - x.p[0] - print(x.p[0]) - print(x.p.contents) -## print x._objects - - x.p[0] = "spam spam" -## print x.p[0] - print("+" * 42) - print(x._objects) class PointerToStructure(unittest.TestCase): def test(self): diff --git a/Lib/test/test_ctypes/test_numbers.py b/Lib/test/test_ctypes/test_numbers.py index db500e812beb1..dc5d4a713c5b0 100644 --- a/Lib/test/test_ctypes/test_numbers.py +++ b/Lib/test/test_ctypes/test_numbers.py @@ -82,14 +82,6 @@ def test_typeerror(self): self.assertRaises(TypeError, t, "") self.assertRaises(TypeError, t, None) - @unittest.skip('test disabled') - def test_valid_ranges(self): - # invalid values of the correct type - # raise ValueError (not OverflowError) - for t, (l, h) in zip(unsigned_types, unsigned_ranges): - self.assertRaises(ValueError, t, l-1) - self.assertRaises(ValueError, t, h+1) - def test_from_param(self): # the from_param class method attribute always # returns PyCArgObject instances @@ -205,19 +197,6 @@ def test_char_from_address(self): a[0] = ord('?') self.assertEqual(v.value, b'?') - # array does not support c_bool / 't' - @unittest.skip('test disabled') - def test_bool_from_address(self): - from ctypes import c_bool - from array import array - a = array(c_bool._type_, [True]) - v = t.from_address(a.buffer_info()[0]) - self.assertEqual(v.value, a[0]) - self.assertEqual(type(v) is t) - a[0] = False - self.assertEqual(v.value, a[0]) - self.assertEqual(type(v) is t) - def test_init(self): # c_int() can be initialized from Python's int, and c_int. # Not from c_long or so, which seems strange, abc should @@ -234,62 +213,6 @@ def test_float_overflow(self): if (hasattr(t, "__ctype_le__")): self.assertRaises(OverflowError, t.__ctype_le__, big_int) - @unittest.skip('test disabled') - def test_perf(self): - check_perf() - -from ctypes import _SimpleCData -class c_int_S(_SimpleCData): - _type_ = "i" - __slots__ = [] - -def run_test(rep, msg, func, arg=None): -## items = [None] * rep - items = range(rep) - from time import perf_counter as clock - if arg is not None: - start = clock() - for i in items: - func(arg); func(arg); func(arg); func(arg); func(arg) - stop = clock() - else: - start = clock() - for i in items: - func(); func(); func(); func(); func() - stop = clock() - print("%15s: %.2f us" % (msg, ((stop-start)*1e6/5/rep))) - -def check_perf(): - # Construct 5 objects - from ctypes import c_int - - REP = 200000 - - run_test(REP, "int()", int) - run_test(REP, "int(999)", int) - run_test(REP, "c_int()", c_int) - run_test(REP, "c_int(999)", c_int) - run_test(REP, "c_int_S()", c_int_S) - run_test(REP, "c_int_S(999)", c_int_S) - -# Python 2.3 -OO, win2k, P4 700 MHz: -# -# int(): 0.87 us -# int(999): 0.87 us -# c_int(): 3.35 us -# c_int(999): 3.34 us -# c_int_S(): 3.23 us -# c_int_S(999): 3.24 us - -# Python 2.2 -OO, win2k, P4 700 MHz: -# -# int(): 0.89 us -# int(999): 0.89 us -# c_int(): 9.99 us -# c_int(999): 10.02 us -# c_int_S(): 9.87 us -# c_int_S(999): 9.85 us if __name__ == '__main__': -## check_perf() unittest.main() diff --git a/Lib/test/test_ctypes/test_structures.py b/Lib/test/test_ctypes/test_structures.py index df39dc7f50d3f..a40ce3e86695a 100644 --- a/Lib/test/test_ctypes/test_structures.py +++ b/Lib/test/test_ctypes/test_structures.py @@ -359,15 +359,6 @@ def get_except(self, func, *args): except Exception as detail: return detail.__class__, str(detail) - @unittest.skip('test disabled') - def test_subclass_creation(self): - meta = type(Structure) - # same as 'class X(Structure): pass' - # fails, since we need either a _fields_ or a _abstract_ attribute - cls, msg = self.get_except(meta, "X", (Structure,), {}) - self.assertEqual((cls, msg), - (AttributeError, "class must define a '_fields_' attribute")) - def test_abstract_class(self): class X(Structure): _abstract_ = "something" From webhook-mailer at python.org Mon Jul 31 04:28:49 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 08:28:49 -0000 Subject: [Python-checkins] [3.11] gh-105751, test_ctypes: Remove disabled tests (GH-105826) (#107484) Message-ID: https://github.com/python/cpython/commit/aa5f2b1f3cdb5f71f5210d4578d5c68b6dd5e255 commit: aa5f2b1f3cdb5f71f5210d4578d5c68b6dd5e255 branch: 3.11 author: ?ukasz Langa committer: ambv date: 2023-07-31T10:28:45+02:00 summary: [3.11] gh-105751, test_ctypes: Remove disabled tests (GH-105826) (#107484) * The following tests were disabled since the initial ctypes commit in 2006, commit babddfca758abe34ff12023f63b18d745fae7ca9: * Callbacks.test_char_p() * DeletePointerTestCase.test_X() * NumberTestCase.test_perf() * StructureTestCase.test_subclass_creation() * Tests.test_X() of test_byteswap * NumberTestCase.test_bool_from_address() was disabled in 2007 by commit 5dc4fe09b7648f9801558e766b21a3d3b2dcad3b. * Remove check_perf() and run_test() of test_numbers. (cherry picked from commit 8f10140e74d141a0a894702044e213e6f0690d9c) Co-authored-by: Victor Stinner files: M Lib/ctypes/test/test_byteswap.py M Lib/ctypes/test/test_callbacks.py M Lib/ctypes/test/test_keeprefs.py M Lib/ctypes/test/test_numbers.py M Lib/ctypes/test/test_structures.py diff --git a/Lib/ctypes/test/test_byteswap.py b/Lib/ctypes/test/test_byteswap.py index 7e98559dfbccb..caefb774cc537 100644 --- a/Lib/ctypes/test/test_byteswap.py +++ b/Lib/ctypes/test/test_byteswap.py @@ -14,14 +14,6 @@ def bin(s): # For Structures and Unions, these types are created on demand. class Test(unittest.TestCase): - @unittest.skip('test disabled') - def test_X(self): - print(sys.byteorder, file=sys.stderr) - for i in range(32): - bits = BITS() - setattr(bits, "i%s" % i, 1) - dump(bits) - def test_slots(self): class BigPoint(BigEndianStructure): __slots__ = () diff --git a/Lib/ctypes/test/test_callbacks.py b/Lib/ctypes/test/test_callbacks.py index 8f95a244439cd..8e27359e4a8bf 100644 --- a/Lib/ctypes/test/test_callbacks.py +++ b/Lib/ctypes/test/test_callbacks.py @@ -93,14 +93,6 @@ def test_char(self): self.check_type(c_char, b"x") self.check_type(c_char, b"a") - # disabled: would now (correctly) raise a RuntimeWarning about - # a memory leak. A callback function cannot return a non-integral - # C type without causing a memory leak. - @unittest.skip('test disabled') - def test_char_p(self): - self.check_type(c_char_p, "abc") - self.check_type(c_char_p, "def") - def test_pyobject(self): o = () from sys import getrefcount as grc diff --git a/Lib/ctypes/test/test_keeprefs.py b/Lib/ctypes/test/test_keeprefs.py index 94c02573fa19d..e20adc7696f50 100644 --- a/Lib/ctypes/test/test_keeprefs.py +++ b/Lib/ctypes/test/test_keeprefs.py @@ -93,37 +93,6 @@ def test_p_cint(self): x = pointer(i) self.assertEqual(x._objects, {'1': i}) -class DeletePointerTestCase(unittest.TestCase): - @unittest.skip('test disabled') - def test_X(self): - class X(Structure): - _fields_ = [("p", POINTER(c_char_p))] - x = X() - i = c_char_p("abc def") - from sys import getrefcount as grc - print("2?", grc(i)) - x.p = pointer(i) - print("3?", grc(i)) - for i in range(320): - c_int(99) - x.p[0] - print(x.p[0]) -## del x -## print "2?", grc(i) -## del i - import gc - gc.collect() - for i in range(320): - c_int(99) - x.p[0] - print(x.p[0]) - print(x.p.contents) -## print x._objects - - x.p[0] = "spam spam" -## print x.p[0] - print("+" * 42) - print(x._objects) class PointerToStructure(unittest.TestCase): def test(self): diff --git a/Lib/ctypes/test/test_numbers.py b/Lib/ctypes/test/test_numbers.py index db500e812beb1..dc5d4a713c5b0 100644 --- a/Lib/ctypes/test/test_numbers.py +++ b/Lib/ctypes/test/test_numbers.py @@ -82,14 +82,6 @@ def test_typeerror(self): self.assertRaises(TypeError, t, "") self.assertRaises(TypeError, t, None) - @unittest.skip('test disabled') - def test_valid_ranges(self): - # invalid values of the correct type - # raise ValueError (not OverflowError) - for t, (l, h) in zip(unsigned_types, unsigned_ranges): - self.assertRaises(ValueError, t, l-1) - self.assertRaises(ValueError, t, h+1) - def test_from_param(self): # the from_param class method attribute always # returns PyCArgObject instances @@ -205,19 +197,6 @@ def test_char_from_address(self): a[0] = ord('?') self.assertEqual(v.value, b'?') - # array does not support c_bool / 't' - @unittest.skip('test disabled') - def test_bool_from_address(self): - from ctypes import c_bool - from array import array - a = array(c_bool._type_, [True]) - v = t.from_address(a.buffer_info()[0]) - self.assertEqual(v.value, a[0]) - self.assertEqual(type(v) is t) - a[0] = False - self.assertEqual(v.value, a[0]) - self.assertEqual(type(v) is t) - def test_init(self): # c_int() can be initialized from Python's int, and c_int. # Not from c_long or so, which seems strange, abc should @@ -234,62 +213,6 @@ def test_float_overflow(self): if (hasattr(t, "__ctype_le__")): self.assertRaises(OverflowError, t.__ctype_le__, big_int) - @unittest.skip('test disabled') - def test_perf(self): - check_perf() - -from ctypes import _SimpleCData -class c_int_S(_SimpleCData): - _type_ = "i" - __slots__ = [] - -def run_test(rep, msg, func, arg=None): -## items = [None] * rep - items = range(rep) - from time import perf_counter as clock - if arg is not None: - start = clock() - for i in items: - func(arg); func(arg); func(arg); func(arg); func(arg) - stop = clock() - else: - start = clock() - for i in items: - func(); func(); func(); func(); func() - stop = clock() - print("%15s: %.2f us" % (msg, ((stop-start)*1e6/5/rep))) - -def check_perf(): - # Construct 5 objects - from ctypes import c_int - - REP = 200000 - - run_test(REP, "int()", int) - run_test(REP, "int(999)", int) - run_test(REP, "c_int()", c_int) - run_test(REP, "c_int(999)", c_int) - run_test(REP, "c_int_S()", c_int_S) - run_test(REP, "c_int_S(999)", c_int_S) - -# Python 2.3 -OO, win2k, P4 700 MHz: -# -# int(): 0.87 us -# int(999): 0.87 us -# c_int(): 3.35 us -# c_int(999): 3.34 us -# c_int_S(): 3.23 us -# c_int_S(999): 3.24 us - -# Python 2.2 -OO, win2k, P4 700 MHz: -# -# int(): 0.89 us -# int(999): 0.89 us -# c_int(): 9.99 us -# c_int(999): 10.02 us -# c_int_S(): 9.87 us -# c_int_S(999): 9.85 us if __name__ == '__main__': -## check_perf() unittest.main() diff --git a/Lib/ctypes/test/test_structures.py b/Lib/ctypes/test/test_structures.py index f95d5a99a3a15..2168aa7df7668 100644 --- a/Lib/ctypes/test/test_structures.py +++ b/Lib/ctypes/test/test_structures.py @@ -359,15 +359,6 @@ def get_except(self, func, *args): except Exception as detail: return detail.__class__, str(detail) - @unittest.skip('test disabled') - def test_subclass_creation(self): - meta = type(Structure) - # same as 'class X(Structure): pass' - # fails, since we need either a _fields_ or a _abstract_ attribute - cls, msg = self.get_except(meta, "X", (Structure,), {}) - self.assertEqual((cls, msg), - (AttributeError, "class must define a '_fields_' attribute")) - def test_abstract_class(self): class X(Structure): _abstract_ = "something" From webhook-mailer at python.org Mon Jul 31 04:33:41 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 08:33:41 -0000 Subject: [Python-checkins] gh-104280: Add test cases for DTrace probes (#107125) Message-ID: https://github.com/python/cpython/commit/a1c737b73d3658be0e1d072a340d42e3d96373c6 commit: a1c737b73d3658be0e1d072a340d42e3d96373c6 branch: main author: Furkan Onder committer: ambv date: 2023-07-31T10:33:37+02:00 summary: gh-104280: Add test cases for DTrace probes (#107125) files: M Lib/test/test_dtrace.py diff --git a/Lib/test/test_dtrace.py b/Lib/test/test_dtrace.py index 4b971deacc1a5..e1adf8e974850 100644 --- a/Lib/test/test_dtrace.py +++ b/Lib/test/test_dtrace.py @@ -3,6 +3,7 @@ import re import subprocess import sys +import sysconfig import types import unittest @@ -173,6 +174,87 @@ class SystemTapOptimizedTests(TraceTests, unittest.TestCase): backend = SystemTapBackend() optimize_python = 2 +class CheckDtraceProbes(unittest.TestCase): + @classmethod + def setUpClass(cls): + if sysconfig.get_config_var('WITH_DTRACE'): + readelf_major_version, readelf_minor_version = cls.get_readelf_version() + if support.verbose: + print(f"readelf version: {readelf_major_version}.{readelf_minor_version}") + else: + raise unittest.SkipTest("CPython must be configured with the --with-dtrace option.") + + + @staticmethod + def get_readelf_version(): + try: + cmd = ["readelf", "--version"] + proc = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + universal_newlines=True, + ) + with proc: + version, stderr = proc.communicate() + + if proc.returncode: + raise Exception( + f"Command {' '.join(cmd)!r} failed " + f"with exit code {proc.returncode}: " + f"stdout={version!r} stderr={stderr!r}" + ) + except OSError: + raise unittest.SkipTest("Couldn't find readelf on the path") + + # Regex to parse: + # 'GNU readelf (GNU Binutils) 2.40.0\n' -> 2.40 + match = re.search(r"^(?:GNU) readelf.*?\b(\d+)\.(\d+)", version) + if match is None: + raise unittest.SkipTest(f"Unable to parse readelf version: {version}") + + return int(match.group(1)), int(match.group(2)) + + def get_readelf_output(self): + command = ["readelf", "-n", sys.executable] + stdout, _ = subprocess.Popen( + command, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + universal_newlines=True, + ).communicate() + return stdout + + def test_check_probes(self): + readelf_output = self.get_readelf_output() + + available_probe_names = [ + "Name: import__find__load__done", + "Name: import__find__load__start", + "Name: audit", + "Name: gc__start", + "Name: gc__done", + ] + + for probe_name in available_probe_names: + with self.subTest(probe_name=probe_name): + self.assertIn(probe_name, readelf_output) + + @unittest.expectedFailure + def test_missing_probes(self): + readelf_output = self.get_readelf_output() + + # Missing probes will be added in the future. + missing_probe_names = [ + "Name: function__entry", + "Name: function__return", + "Name: line", + ] + + for probe_name in missing_probe_names: + with self.subTest(probe_name=probe_name): + self.assertIn(probe_name, readelf_output) + if __name__ == '__main__': unittest.main() From webhook-mailer at python.org Mon Jul 31 05:16:34 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 09:16:34 -0000 Subject: [Python-checkins] [3.12] gh-104280: Add test cases for DTrace probes (GH-107125) (#107489) Message-ID: https://github.com/python/cpython/commit/04bd8c76b2578fb6715d6766c637f2b4d3e6a52f commit: 04bd8c76b2578fb6715d6766c637f2b4d3e6a52f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2023-07-31T11:16:30+02:00 summary: [3.12] gh-104280: Add test cases for DTrace probes (GH-107125) (#107489) gh-104280: Add test cases for DTrace probes (GH-107125) (cherry picked from commit a1c737b73d3658be0e1d072a340d42e3d96373c6) Co-authored-by: Furkan Onder files: M Lib/test/test_dtrace.py diff --git a/Lib/test/test_dtrace.py b/Lib/test/test_dtrace.py index 4b971deacc1a5..e1adf8e974850 100644 --- a/Lib/test/test_dtrace.py +++ b/Lib/test/test_dtrace.py @@ -3,6 +3,7 @@ import re import subprocess import sys +import sysconfig import types import unittest @@ -173,6 +174,87 @@ class SystemTapOptimizedTests(TraceTests, unittest.TestCase): backend = SystemTapBackend() optimize_python = 2 +class CheckDtraceProbes(unittest.TestCase): + @classmethod + def setUpClass(cls): + if sysconfig.get_config_var('WITH_DTRACE'): + readelf_major_version, readelf_minor_version = cls.get_readelf_version() + if support.verbose: + print(f"readelf version: {readelf_major_version}.{readelf_minor_version}") + else: + raise unittest.SkipTest("CPython must be configured with the --with-dtrace option.") + + + @staticmethod + def get_readelf_version(): + try: + cmd = ["readelf", "--version"] + proc = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + universal_newlines=True, + ) + with proc: + version, stderr = proc.communicate() + + if proc.returncode: + raise Exception( + f"Command {' '.join(cmd)!r} failed " + f"with exit code {proc.returncode}: " + f"stdout={version!r} stderr={stderr!r}" + ) + except OSError: + raise unittest.SkipTest("Couldn't find readelf on the path") + + # Regex to parse: + # 'GNU readelf (GNU Binutils) 2.40.0\n' -> 2.40 + match = re.search(r"^(?:GNU) readelf.*?\b(\d+)\.(\d+)", version) + if match is None: + raise unittest.SkipTest(f"Unable to parse readelf version: {version}") + + return int(match.group(1)), int(match.group(2)) + + def get_readelf_output(self): + command = ["readelf", "-n", sys.executable] + stdout, _ = subprocess.Popen( + command, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + universal_newlines=True, + ).communicate() + return stdout + + def test_check_probes(self): + readelf_output = self.get_readelf_output() + + available_probe_names = [ + "Name: import__find__load__done", + "Name: import__find__load__start", + "Name: audit", + "Name: gc__start", + "Name: gc__done", + ] + + for probe_name in available_probe_names: + with self.subTest(probe_name=probe_name): + self.assertIn(probe_name, readelf_output) + + @unittest.expectedFailure + def test_missing_probes(self): + readelf_output = self.get_readelf_output() + + # Missing probes will be added in the future. + missing_probe_names = [ + "Name: function__entry", + "Name: function__return", + "Name: line", + ] + + for probe_name in missing_probe_names: + with self.subTest(probe_name=probe_name): + self.assertIn(probe_name, readelf_output) + if __name__ == '__main__': unittest.main() From webhook-mailer at python.org Mon Jul 31 05:16:49 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 09:16:49 -0000 Subject: [Python-checkins] [3.12] gh-46376: Return existing pointer when possible in ctypes (GH-107131) (#107487) Message-ID: https://github.com/python/cpython/commit/54aaaadef8a44324f6be674707c67a3516470ff6 commit: 54aaaadef8a44324f6be674707c67a3516470ff6 branch: 3.12 author: ?ukasz Langa committer: ambv date: 2023-07-31T11:16:45+02:00 summary: [3.12] gh-46376: Return existing pointer when possible in ctypes (GH-107131) (#107487) (cherry picked from commit 08447b5deb47e2a0df87fa0a0576d300e5c909b4) Co-authored-by: Konstantin files: A Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst M Lib/test/test_ctypes/test_keeprefs.py M Modules/_ctypes/_ctypes.c diff --git a/Lib/test/test_ctypes/test_keeprefs.py b/Lib/test/test_ctypes/test_keeprefs.py index e20adc7696f50..61650ad170475 100644 --- a/Lib/test/test_ctypes/test_keeprefs.py +++ b/Lib/test/test_ctypes/test_keeprefs.py @@ -93,6 +93,33 @@ def test_p_cint(self): x = pointer(i) self.assertEqual(x._objects, {'1': i}) + def test_pp_ownership(self): + d = c_int(123) + n = c_int(456) + + p = pointer(d) + pp = pointer(p) + + self.assertIs(pp._objects['1'], p) + self.assertIs(pp._objects['0']['1'], d) + + pp.contents.contents = n + + self.assertIs(pp._objects['1'], p) + self.assertIs(pp._objects['0']['1'], n) + + self.assertIs(p._objects['1'], n) + self.assertEqual(len(p._objects), 1) + + del d + del p + + self.assertIs(pp._objects['0']['1'], n) + self.assertEqual(len(pp._objects), 2) + + del n + + self.assertEqual(len(pp._objects), 2) class PointerToStructure(unittest.TestCase): def test(self): diff --git a/Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst b/Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst new file mode 100644 index 0000000000000..8e8f0245b4539 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst @@ -0,0 +1 @@ +Prevent memory leak and use-after-free when using pointers to pointers with ctypes diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index 534ef8c1d6cf8..8496077322a66 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -5122,6 +5122,8 @@ static PyObject * Pointer_get_contents(CDataObject *self, void *closure) { StgDictObject *stgdict; + PyObject *keep, *ptr_probe; + CDataObject *ptr2ptr; if (*(void **)self->b_ptr == NULL) { PyErr_SetString(PyExc_ValueError, @@ -5131,6 +5133,33 @@ Pointer_get_contents(CDataObject *self, void *closure) stgdict = PyObject_stgdict((PyObject *)self); assert(stgdict); /* Cannot be NULL for pointer instances */ + + keep = GetKeepedObjects(self); + if (keep != NULL) { + // check if it's a pointer to a pointer: + // pointers will have '0' key in the _objects + ptr_probe = PyDict_GetItemString(keep, "0"); + + if (ptr_probe != NULL) { + ptr2ptr = (CDataObject*) PyDict_GetItemString(keep, "1"); + if (ptr2ptr == NULL) { + PyErr_SetString(PyExc_ValueError, + "Unexpected NULL pointer in _objects"); + return NULL; + } + // don't construct a new object, + // return existing one instead to preserve refcount + assert( + *(void**) self->b_ptr == ptr2ptr->b_ptr || + *(void**) self->b_value.c == ptr2ptr->b_ptr || + *(void**) self->b_ptr == ptr2ptr->b_value.c || + *(void**) self->b_value.c == ptr2ptr->b_value.c + ); // double-check that we are returning the same thing + Py_INCREF(ptr2ptr); + return (PyObject *) ptr2ptr; + } + } + return PyCData_FromBaseObj(stgdict->proto, (PyObject *)self, 0, *(void **)self->b_ptr); From webhook-mailer at python.org Mon Jul 31 05:17:03 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 09:17:03 -0000 Subject: [Python-checkins] [3.11] gh-46376: Return existing pointer when possible in ctypes (GH-107131) (#107488) Message-ID: https://github.com/python/cpython/commit/57f27e444175a8a5ffcd86971e06de61c1c38628 commit: 57f27e444175a8a5ffcd86971e06de61c1c38628 branch: 3.11 author: ?ukasz Langa committer: ambv date: 2023-07-31T11:16:59+02:00 summary: [3.11] gh-46376: Return existing pointer when possible in ctypes (GH-107131) (#107488) (cherry picked from commit 08447b5deb47e2a0df87fa0a0576d300e5c909b4) Co-authored-by: Konstantin files: A Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst M Lib/ctypes/test/test_keeprefs.py M Modules/_ctypes/_ctypes.c diff --git a/Lib/ctypes/test/test_keeprefs.py b/Lib/ctypes/test/test_keeprefs.py index e20adc7696f50..61650ad170475 100644 --- a/Lib/ctypes/test/test_keeprefs.py +++ b/Lib/ctypes/test/test_keeprefs.py @@ -93,6 +93,33 @@ def test_p_cint(self): x = pointer(i) self.assertEqual(x._objects, {'1': i}) + def test_pp_ownership(self): + d = c_int(123) + n = c_int(456) + + p = pointer(d) + pp = pointer(p) + + self.assertIs(pp._objects['1'], p) + self.assertIs(pp._objects['0']['1'], d) + + pp.contents.contents = n + + self.assertIs(pp._objects['1'], p) + self.assertIs(pp._objects['0']['1'], n) + + self.assertIs(p._objects['1'], n) + self.assertEqual(len(p._objects), 1) + + del d + del p + + self.assertIs(pp._objects['0']['1'], n) + self.assertEqual(len(pp._objects), 2) + + del n + + self.assertEqual(len(pp._objects), 2) class PointerToStructure(unittest.TestCase): def test(self): diff --git a/Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst b/Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst new file mode 100644 index 0000000000000..8e8f0245b4539 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-24-01-21-16.gh-issue-46376.w-xuDL.rst @@ -0,0 +1 @@ +Prevent memory leak and use-after-free when using pointers to pointers with ctypes diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index fc73264ff5a32..9cc518132f92f 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -5158,6 +5158,8 @@ static PyObject * Pointer_get_contents(CDataObject *self, void *closure) { StgDictObject *stgdict; + PyObject *keep, *ptr_probe; + CDataObject *ptr2ptr; if (*(void **)self->b_ptr == NULL) { PyErr_SetString(PyExc_ValueError, @@ -5167,6 +5169,33 @@ Pointer_get_contents(CDataObject *self, void *closure) stgdict = PyObject_stgdict((PyObject *)self); assert(stgdict); /* Cannot be NULL for pointer instances */ + + keep = GetKeepedObjects(self); + if (keep != NULL) { + // check if it's a pointer to a pointer: + // pointers will have '0' key in the _objects + ptr_probe = PyDict_GetItemString(keep, "0"); + + if (ptr_probe != NULL) { + ptr2ptr = (CDataObject*) PyDict_GetItemString(keep, "1"); + if (ptr2ptr == NULL) { + PyErr_SetString(PyExc_ValueError, + "Unexpected NULL pointer in _objects"); + return NULL; + } + // don't construct a new object, + // return existing one instead to preserve refcount + assert( + *(void**) self->b_ptr == ptr2ptr->b_ptr || + *(void**) self->b_value.c == ptr2ptr->b_ptr || + *(void**) self->b_ptr == ptr2ptr->b_value.c || + *(void**) self->b_value.c == ptr2ptr->b_value.c + ); // double-check that we are returning the same thing + Py_INCREF(ptr2ptr); + return (PyObject *) ptr2ptr; + } + } + return PyCData_FromBaseObj(stgdict->proto, (PyObject *)self, 0, *(void **)self->b_ptr); From webhook-mailer at python.org Mon Jul 31 05:40:24 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 09:40:24 -0000 Subject: [Python-checkins] [3.11] gh-104280: Add test cases for DTrace probes (GH-107125) (#107492) Message-ID: https://github.com/python/cpython/commit/3abcdc71b6820e3f46f29cdcc7c91ccaf3145bdf commit: 3abcdc71b6820e3f46f29cdcc7c91ccaf3145bdf branch: 3.11 author: ?ukasz Langa committer: ambv date: 2023-07-31T11:40:20+02:00 summary: [3.11] gh-104280: Add test cases for DTrace probes (GH-107125) (#107492) (cherry picked from commit a1c737b73d3658be0e1d072a340d42e3d96373c6) Co-authored-by: Furkan Onder files: M Lib/test/test_dtrace.py diff --git a/Lib/test/test_dtrace.py b/Lib/test/test_dtrace.py index 4b971deacc1a5..092a050fd1d97 100644 --- a/Lib/test/test_dtrace.py +++ b/Lib/test/test_dtrace.py @@ -3,6 +3,7 @@ import re import subprocess import sys +import sysconfig import types import unittest @@ -173,6 +174,75 @@ class SystemTapOptimizedTests(TraceTests, unittest.TestCase): backend = SystemTapBackend() optimize_python = 2 +class CheckDtraceProbes(unittest.TestCase): + @classmethod + def setUpClass(cls): + if sysconfig.get_config_var('WITH_DTRACE'): + readelf_major_version, readelf_minor_version = cls.get_readelf_version() + if support.verbose: + print(f"readelf version: {readelf_major_version}.{readelf_minor_version}") + else: + raise unittest.SkipTest("CPython must be configured with the --with-dtrace option.") + + + @staticmethod + def get_readelf_version(): + try: + cmd = ["readelf", "--version"] + proc = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + universal_newlines=True, + ) + with proc: + version, stderr = proc.communicate() + + if proc.returncode: + raise Exception( + f"Command {' '.join(cmd)!r} failed " + f"with exit code {proc.returncode}: " + f"stdout={version!r} stderr={stderr!r}" + ) + except OSError: + raise unittest.SkipTest("Couldn't find readelf on the path") + + # Regex to parse: + # 'GNU readelf (GNU Binutils) 2.40.0\n' -> 2.40 + match = re.search(r"^(?:GNU) readelf.*?\b(\d+)\.(\d+)", version) + if match is None: + raise unittest.SkipTest(f"Unable to parse readelf version: {version}") + + return int(match.group(1)), int(match.group(2)) + + def get_readelf_output(self): + command = ["readelf", "-n", sys.executable] + stdout, _ = subprocess.Popen( + command, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + universal_newlines=True, + ).communicate() + return stdout + + def test_check_probes(self): + readelf_output = self.get_readelf_output() + + available_probe_names = [ + "Name: import__find__load__done", + "Name: import__find__load__start", + "Name: audit", + "Name: gc__start", + "Name: gc__done", + "Name: function__entry", + "Name: function__return", + "Name: line", + ] + + for probe_name in available_probe_names: + with self.subTest(probe_name=probe_name): + self.assertIn(probe_name, readelf_output) + if __name__ == '__main__': unittest.main() From webhook-mailer at python.org Mon Jul 31 07:50:07 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 31 Jul 2023 11:50:07 -0000 Subject: [Python-checkins] gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (#107496) Message-ID: https://github.com/python/cpython/commit/2c5d206b33e4cdcafaaaf1eeaa189c10de332dc5 commit: 2c5d206b33e4cdcafaaaf1eeaa189c10de332dc5 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-31T11:50:03Z summary: gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (#107496) files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 3ce27d1dd6b48..2f94f0168c916 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -318,6 +318,26 @@ def test_unknown_destination_command(self): msg = "unknown destination command 'nosuchcommand'" self.assertIn(msg, out) + def test_no_access_to_members_in_converter_init(self): + out = self.expect_failure(""" + /*[python input] + class Custom_converter(CConverter): + converter = "some_c_function" + def converter_init(self): + self.function.noaccess + [python start generated code]*/ + /*[clinic input] + module test + test.fn + a: Custom + [clinic start generated code]*/ + """) + msg = ( + "Stepped on a land mine, trying to access attribute 'noaccess':\n" + "Don't access members of self.function inside converter_init!" + ) + self.assertIn(msg, out) + class ClinicGroupPermuterTest(TestCase): def _test(self, l, m, r, output): From webhook-mailer at python.org Mon Jul 31 08:07:21 2023 From: webhook-mailer at python.org (corona10) Date: Mon, 31 Jul 2023 12:07:21 -0000 Subject: [Python-checkins] no-issue: Fix typo in import.c (gh-107498) Message-ID: https://github.com/python/cpython/commit/f57b9fd2b6b403e59dcd7f86a0943d61d79e5768 commit: f57b9fd2b6b403e59dcd7f86a0943d61d79e5768 branch: main author: Georg Brandl committer: corona10 date: 2023-07-31T12:07:17Z summary: no-issue: Fix typo in import.c (gh-107498) files: M Python/import.c diff --git a/Python/import.c b/Python/import.c index e381d1fd7d433..3be2f76c9eca7 100644 --- a/Python/import.c +++ b/Python/import.c @@ -601,7 +601,7 @@ _PyImport_ClearModulesByIndex(PyInterpreterState *interp) when an extension is loaded. This includes when it is imported for the first time. - Here's a summary, using importlib._boostrap._load() as a starting point. + Here's a summary, using importlib._bootstrap._load() as a starting point. 1. importlib._bootstrap._load() 2. _load(): acquire import lock From webhook-mailer at python.org Mon Jul 31 08:28:03 2023 From: webhook-mailer at python.org (Yhg1s) Date: Mon, 31 Jul 2023 12:28:03 -0000 Subject: [Python-checkins] [3.12] gh-102509: Start initializing `ob_digit` of `_PyLongValue` (GH-102510) (#107464) Message-ID: https://github.com/python/cpython/commit/8f080a290bd45f4664d3a457256310cc02883d7d commit: 8f080a290bd45f4664d3a457256310cc02883d7d branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Yhg1s date: 2023-07-31T14:27:59+02:00 summary: [3.12] gh-102509: Start initializing `ob_digit` of `_PyLongValue` (GH-102510) (#107464) gh-102509: Start initializing `ob_digit` of `_PyLongValue` (GH-102510) (cherry picked from commit fc130c47daa715d60d8925c478a96d5083e47b6a) Co-authored-by: Illia Volochii files: A Misc/NEWS.d/next/Security/2023-03-07-21-46-29.gh-issue-102509.5ouaH_.rst M Objects/longobject.c diff --git a/Misc/NEWS.d/next/Security/2023-03-07-21-46-29.gh-issue-102509.5ouaH_.rst b/Misc/NEWS.d/next/Security/2023-03-07-21-46-29.gh-issue-102509.5ouaH_.rst new file mode 100644 index 0000000000000..d1a8e8b5a8d3c --- /dev/null +++ b/Misc/NEWS.d/next/Security/2023-03-07-21-46-29.gh-issue-102509.5ouaH_.rst @@ -0,0 +1,2 @@ +Start initializing ``ob_digit`` during creation of :c:type:`PyLongObject` +objects. Patch by Illia Volochii. diff --git a/Objects/longobject.c b/Objects/longobject.c index 5fca55e5c3a2b..5d9b413861478 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -163,6 +163,9 @@ _PyLong_New(Py_ssize_t size) } _PyLong_SetSignAndDigitCount(result, size != 0, size); _PyObject_Init((PyObject*)result, &PyLong_Type); + /* The digit has to be initialized explicitly to avoid + * use-of-uninitialized-value. */ + result->long_value.ob_digit[0] = 0; return result; } From webhook-mailer at python.org Mon Jul 31 08:32:22 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 31 Jul 2023 12:32:22 -0000 Subject: [Python-checkins] [3.11] gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (GH-107496) (#107500) Message-ID: https://github.com/python/cpython/commit/81d0c7ca5aa6b2f86eb9ae5b66946c54dd483548 commit: 81d0c7ca5aa6b2f86eb9ae5b66946c54dd483548 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-31T12:32:18Z summary: [3.11] gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (GH-107496) (#107500) (cherry picked from commit 2c5d206b33e4cdcafaaaf1eeaa189c10de332dc5) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 1c01fb0ed3688..bc0eee1552e46 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -314,6 +314,26 @@ def test_unknown_destination_command(self): msg = "unknown destination command 'nosuchcommand'" self.assertIn(msg, out) + def test_no_access_to_members_in_converter_init(self): + out = self.expect_failure(""" + /*[python input] + class Custom_converter(CConverter): + converter = "some_c_function" + def converter_init(self): + self.function.noaccess + [python start generated code]*/ + /*[clinic input] + module test + test.fn + a: Custom + [clinic start generated code]*/ + """) + msg = ( + "Stepped on a land mine, trying to access attribute 'noaccess':\n" + "Don't access members of self.function inside converter_init!" + ) + self.assertIn(msg, out) + class ClinicGroupPermuterTest(TestCase): def _test(self, l, m, r, output): From webhook-mailer at python.org Mon Jul 31 08:34:20 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 12:34:20 -0000 Subject: [Python-checkins] =?utf-8?q?=5B3=2E12=5D_gh-106263=3A_Fix_segfau?= =?utf-8?q?lt_in_=60signaldict=5Frepr=60__in_=60=5Fdecimal=60_module_=28?= =?utf-8?b?I+KApiAoIzEwNzQ5MSk=?= Message-ID: https://github.com/python/cpython/commit/99518bbbf47a198f8c51e82e7357ac7efdafdcd8 commit: 99518bbbf47a198f8c51e82e7357ac7efdafdcd8 branch: 3.12 author: Charlie Zhao committer: ambv date: 2023-07-31T14:34:16+02:00 summary: [3.12] gh-106263: Fix segfault in `signaldict_repr` in `_decimal` module (#? (#107491) Co-authored-by: sunmy2019 <59365878+sunmy2019 at users.noreply.github.com> (cherry picked from commit 3979150a0d406707f6d253d7c15fb32c1e005a77) files: A Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst M Lib/test/test_decimal.py M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/ignored.tsv diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py index 749496e3e9455..d0ba34803c21e 100644 --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -5682,6 +5682,36 @@ def test_maxcontext_exact_arith(self): self.assertEqual(Decimal(400) ** -1, Decimal('0.0025')) + def test_c_signaldict_segfault(self): + # See gh-106263 for details. + SignalDict = type(C.Context().flags) + sd = SignalDict() + err_msg = "invalid signal dict" + + with self.assertRaisesRegex(ValueError, err_msg): + len(sd) + + with self.assertRaisesRegex(ValueError, err_msg): + iter(sd) + + with self.assertRaisesRegex(ValueError, err_msg): + repr(sd) + + with self.assertRaisesRegex(ValueError, err_msg): + sd[C.InvalidOperation] = True + + with self.assertRaisesRegex(ValueError, err_msg): + sd[C.InvalidOperation] + + with self.assertRaisesRegex(ValueError, err_msg): + sd == C.Context().flags + + with self.assertRaisesRegex(ValueError, err_msg): + C.Context().flags == sd + + with self.assertRaisesRegex(ValueError, err_msg): + sd.copy() + @requires_docstrings @requires_cdecimal class SignatureTest(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst b/Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst new file mode 100644 index 0000000000000..23763818d84ba --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst @@ -0,0 +1,2 @@ +Fix crash when calling ``repr`` with a manually constructed SignalDict object. +Patch by Charlie Zhao. \ No newline at end of file diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 70e18edb46453..70b13982bb0cc 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -246,14 +246,12 @@ value_error_int(const char *mesg) return -1; } -#ifdef CONFIG_32 static PyObject * value_error_ptr(const char *mesg) { PyErr_SetString(PyExc_ValueError, mesg); return NULL; } -#endif static int type_error_int(const char *mesg) @@ -540,6 +538,8 @@ getround(PyObject *v) initialized to new SignalDicts. Once a SignalDict is tied to a context, it cannot be deleted. */ +static const char *INVALID_SIGNALDICT_ERROR_MSG = "invalid signal dict"; + static int signaldict_init(PyObject *self, PyObject *args UNUSED, PyObject *kwds UNUSED) { @@ -548,8 +548,11 @@ signaldict_init(PyObject *self, PyObject *args UNUSED, PyObject *kwds UNUSED) } static Py_ssize_t -signaldict_len(PyObject *self UNUSED) +signaldict_len(PyObject *self) { + if (SdFlagAddr(self) == NULL) { + return value_error_int(INVALID_SIGNALDICT_ERROR_MSG); + } return SIGNAL_MAP_LEN; } @@ -557,6 +560,9 @@ static PyObject *SignalTuple; static PyObject * signaldict_iter(PyObject *self UNUSED) { + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } return PyTuple_Type.tp_iter(SignalTuple); } @@ -564,6 +570,9 @@ static PyObject * signaldict_getitem(PyObject *self, PyObject *key) { uint32_t flag; + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } flag = exception_as_flag(key); if (flag & DEC_ERRORS) { @@ -579,6 +588,10 @@ signaldict_setitem(PyObject *self, PyObject *key, PyObject *value) uint32_t flag; int x; + if (SdFlagAddr(self) == NULL) { + return value_error_int(INVALID_SIGNALDICT_ERROR_MSG); + } + if (value == NULL) { return value_error_int("signal keys cannot be deleted"); } @@ -611,6 +624,10 @@ signaldict_repr(PyObject *self) const char *b[SIGNAL_MAP_LEN]; /* bool */ int i; + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } + assert(SIGNAL_MAP_LEN == 9); for (cm=signal_map, i=0; cm->name != NULL; cm++, i++) { @@ -632,6 +649,9 @@ signaldict_richcompare(PyObject *v, PyObject *w, int op) PyObject *res = Py_NotImplemented; assert(PyDecSignalDict_Check(v)); + if ((SdFlagAddr(v) == NULL) || (SdFlagAddr(w) == NULL)) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } if (op == Py_EQ || op == Py_NE) { if (PyDecSignalDict_Check(w)) { @@ -660,6 +680,9 @@ signaldict_richcompare(PyObject *v, PyObject *w, int op) static PyObject * signaldict_copy(PyObject *self, PyObject *args UNUSED) { + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } return flags_as_dict(SdFlags(self)); } diff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsv index afc28e551813b..78be97da41b50 100644 --- a/Tools/c-analyzer/cpython/ignored.tsv +++ b/Tools/c-analyzer/cpython/ignored.tsv @@ -213,6 +213,7 @@ Modules/_decimal/_decimal.c - invalid_rounding_err - Modules/_decimal/_decimal.c - invalid_signals_err - Modules/_decimal/_decimal.c - signal_map - Modules/_decimal/_decimal.c - ssize_constants - +Modules/_decimal/_decimal.c - INVALID_SIGNALDICT_ERROR_MSG - Modules/_elementtree.c - ExpatMemoryHandler - Modules/_hashopenssl.c - py_hashes - Modules/_hacl/Hacl_Hash_SHA1.c - _h0 - From webhook-mailer at python.org Mon Jul 31 08:48:31 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 12:48:31 -0000 Subject: [Python-checkins] gh-105751: Remove obsolete `object` base class in some ctypes tests (#107460) Message-ID: https://github.com/python/cpython/commit/520efecfc3aed34d3a44545c7cd872d1aea8c7dc commit: 520efecfc3aed34d3a44545c7cd872d1aea8c7dc branch: main author: Tomas R committer: ambv date: 2023-07-31T14:48:27+02:00 summary: gh-105751: Remove obsolete `object` base class in some ctypes tests (#107460) files: M Lib/test/test_ctypes/test_as_parameter.py M Lib/test/test_ctypes/test_callbacks.py M Lib/test/test_ctypes/test_numbers.py M Lib/test/test_ctypes/test_parameters.py diff --git a/Lib/test/test_ctypes/test_as_parameter.py b/Lib/test/test_ctypes/test_as_parameter.py index 39f70e864757d..a1a8745e737fa 100644 --- a/Lib/test/test_ctypes/test_as_parameter.py +++ b/Lib/test/test_ctypes/test_as_parameter.py @@ -192,7 +192,7 @@ class S8I(Structure): (9*2, 8*3, 7*4, 6*5, 5*6, 4*7, 3*8, 2*9)) def test_recursive_as_param(self): - class A(object): + class A: pass a = A() @@ -201,7 +201,7 @@ class A(object): c_int.from_param(a) -class AsParamWrapper(object): +class AsParamWrapper: def __init__(self, param): self._as_parameter_ = param @@ -209,7 +209,7 @@ class AsParamWrapperTestCase(BasicWrapTestCase): wrap = AsParamWrapper -class AsParamPropertyWrapper(object): +class AsParamPropertyWrapper: def __init__(self, param): self._param = param diff --git a/Lib/test/test_ctypes/test_callbacks.py b/Lib/test/test_ctypes/test_callbacks.py index 037677e37ab34..6fe3e11967240 100644 --- a/Lib/test/test_ctypes/test_callbacks.py +++ b/Lib/test/test_ctypes/test_callbacks.py @@ -120,7 +120,7 @@ def test_unsupported_restype_2(self): def test_issue_7959(self): proto = self.functype.__func__(None) - class X(object): + class X: def func(self): pass def __init__(self): self.v = proto(self.func) diff --git a/Lib/test/test_ctypes/test_numbers.py b/Lib/test/test_ctypes/test_numbers.py index fd318f9a18e53..29108a28ec16e 100644 --- a/Lib/test/test_ctypes/test_numbers.py +++ b/Lib/test/test_ctypes/test_numbers.py @@ -86,7 +86,7 @@ def test_byref(self): def test_floats(self): # c_float and c_double can be created from # Python int and float - class FloatLike(object): + class FloatLike: def __float__(self): return 2.0 f = FloatLike() @@ -97,15 +97,15 @@ def __float__(self): self.assertEqual(t(f).value, 2.0) def test_integers(self): - class FloatLike(object): + class FloatLike: def __float__(self): return 2.0 f = FloatLike() - class IntLike(object): + class IntLike: def __int__(self): return 2 d = IntLike() - class IndexLike(object): + class IndexLike: def __index__(self): return 2 i = IndexLike() diff --git a/Lib/test/test_ctypes/test_parameters.py b/Lib/test/test_ctypes/test_parameters.py index 40979212a627d..d1eeee6b0306f 100644 --- a/Lib/test/test_ctypes/test_parameters.py +++ b/Lib/test/test_ctypes/test_parameters.py @@ -157,7 +157,7 @@ def test_noctypes_argtype(self): # TypeError: has no from_param method self.assertRaises(TypeError, setattr, func, "argtypes", (object,)) - class Adapter(object): + class Adapter: def from_param(cls, obj): return None @@ -165,7 +165,7 @@ def from_param(cls, obj): self.assertEqual(func(None), None) self.assertEqual(func(object()), None) - class Adapter(object): + class Adapter: def from_param(cls, obj): return obj @@ -174,7 +174,7 @@ def from_param(cls, obj): self.assertRaises(ArgumentError, func, object()) self.assertEqual(func(c_void_p(42)), 42) - class Adapter(object): + class Adapter: def from_param(cls, obj): raise ValueError(obj) From webhook-mailer at python.org Mon Jul 31 09:52:27 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 13:52:27 -0000 Subject: [Python-checkins] =?utf-8?q?=5B3=2E11=5D_gh-106263=3A_Fix_segfau?= =?utf-8?q?lt_in_=60signaldict=5Frepr=60__in_=60=5Fdecimal=60_module_=28?= =?utf-8?b?I+KApiAoIzEwNzQ5MCk=?= Message-ID: https://github.com/python/cpython/commit/a15d06c2305844fe0ac3723336f3815edbef98bc commit: a15d06c2305844fe0ac3723336f3815edbef98bc branch: 3.11 author: Charlie Zhao committer: ambv date: 2023-07-31T15:52:23+02:00 summary: [3.11] gh-106263: Fix segfault in `signaldict_repr` in `_decimal` module (#? (#107490) Co-authored-by: sunmy2019 <59365878+sunmy2019 at users.noreply.github.com> (cherry picked from commit 3979150a0d406707f6d253d7c15fb32c1e005a77) files: A Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst M Lib/test/test_decimal.py M Modules/_decimal/_decimal.c M Tools/c-analyzer/cpython/ignored.tsv diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py index 749496e3e9455..d0ba34803c21e 100644 --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -5682,6 +5682,36 @@ def test_maxcontext_exact_arith(self): self.assertEqual(Decimal(400) ** -1, Decimal('0.0025')) + def test_c_signaldict_segfault(self): + # See gh-106263 for details. + SignalDict = type(C.Context().flags) + sd = SignalDict() + err_msg = "invalid signal dict" + + with self.assertRaisesRegex(ValueError, err_msg): + len(sd) + + with self.assertRaisesRegex(ValueError, err_msg): + iter(sd) + + with self.assertRaisesRegex(ValueError, err_msg): + repr(sd) + + with self.assertRaisesRegex(ValueError, err_msg): + sd[C.InvalidOperation] = True + + with self.assertRaisesRegex(ValueError, err_msg): + sd[C.InvalidOperation] + + with self.assertRaisesRegex(ValueError, err_msg): + sd == C.Context().flags + + with self.assertRaisesRegex(ValueError, err_msg): + C.Context().flags == sd + + with self.assertRaisesRegex(ValueError, err_msg): + sd.copy() + @requires_docstrings @requires_cdecimal class SignatureTest(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst b/Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst new file mode 100644 index 0000000000000..d86a6bfdabbfb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-06-30-16-42-44.gh-issue-106263.tk-t93.rst @@ -0,0 +1,2 @@ +Fix crash when calling ``repr`` with a manually constructed SignalDict object. +Patch by Charlie Zhao. diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c index 5cacdca3f44ba..a97490d48c90a 100644 --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -247,14 +247,12 @@ value_error_int(const char *mesg) return -1; } -#ifdef CONFIG_32 static PyObject * value_error_ptr(const char *mesg) { PyErr_SetString(PyExc_ValueError, mesg); return NULL; } -#endif static int type_error_int(const char *mesg) @@ -541,6 +539,8 @@ getround(PyObject *v) initialized to new SignalDicts. Once a SignalDict is tied to a context, it cannot be deleted. */ +static const char *INVALID_SIGNALDICT_ERROR_MSG = "invalid signal dict"; + static int signaldict_init(PyObject *self, PyObject *args UNUSED, PyObject *kwds UNUSED) { @@ -549,8 +549,11 @@ signaldict_init(PyObject *self, PyObject *args UNUSED, PyObject *kwds UNUSED) } static Py_ssize_t -signaldict_len(PyObject *self UNUSED) +signaldict_len(PyObject *self) { + if (SdFlagAddr(self) == NULL) { + return value_error_int(INVALID_SIGNALDICT_ERROR_MSG); + } return SIGNAL_MAP_LEN; } @@ -558,6 +561,9 @@ static PyObject *SignalTuple; static PyObject * signaldict_iter(PyObject *self UNUSED) { + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } return PyTuple_Type.tp_iter(SignalTuple); } @@ -565,6 +571,9 @@ static PyObject * signaldict_getitem(PyObject *self, PyObject *key) { uint32_t flag; + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } flag = exception_as_flag(key); if (flag & DEC_ERRORS) { @@ -580,6 +589,10 @@ signaldict_setitem(PyObject *self, PyObject *key, PyObject *value) uint32_t flag; int x; + if (SdFlagAddr(self) == NULL) { + return value_error_int(INVALID_SIGNALDICT_ERROR_MSG); + } + if (value == NULL) { return value_error_int("signal keys cannot be deleted"); } @@ -612,6 +625,10 @@ signaldict_repr(PyObject *self) const char *b[SIGNAL_MAP_LEN]; /* bool */ int i; + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } + assert(SIGNAL_MAP_LEN == 9); for (cm=signal_map, i=0; cm->name != NULL; cm++, i++) { @@ -634,6 +651,10 @@ signaldict_richcompare(PyObject *v, PyObject *w, int op) assert(PyDecSignalDict_Check(v)); + if ((SdFlagAddr(v) == NULL) || (SdFlagAddr(w) == NULL)) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } + if (op == Py_EQ || op == Py_NE) { if (PyDecSignalDict_Check(w)) { res = (SdFlags(v)==SdFlags(w)) ^ (op==Py_NE) ? Py_True : Py_False; @@ -662,6 +683,9 @@ signaldict_richcompare(PyObject *v, PyObject *w, int op) static PyObject * signaldict_copy(PyObject *self, PyObject *args UNUSED) { + if (SdFlagAddr(self) == NULL) { + return value_error_ptr(INVALID_SIGNALDICT_ERROR_MSG); + } return flags_as_dict(SdFlags(self)); } diff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsv index 63d0695bfb4a6..6e4f1c4046398 100644 --- a/Tools/c-analyzer/cpython/ignored.tsv +++ b/Tools/c-analyzer/cpython/ignored.tsv @@ -1432,6 +1432,7 @@ Modules/_decimal/_decimal.c - invalid_rounding_err - Modules/_decimal/_decimal.c - invalid_signals_err - Modules/_decimal/_decimal.c - signal_map - Modules/_decimal/_decimal.c - ssize_constants - +Modules/_decimal/_decimal.c - INVALID_SIGNALDICT_ERROR_MSG - Modules/_elementtree.c - ExpatMemoryHandler - Modules/_io/_iomodule.c - static_types - Modules/_io/textio.c - encodefuncs - From webhook-mailer at python.org Mon Jul 31 10:33:31 2023 From: webhook-mailer at python.org (asvetlov) Date: Mon, 31 Jul 2023 14:33:31 -0000 Subject: [Python-checkins] gh-87799: Improve the textual representation of IPv4-mapped IPv6 addresses (#29345) Message-ID: https://github.com/python/cpython/commit/f22bf8e3cf899896cf587099d29290cb43aa9724 commit: f22bf8e3cf899896cf587099d29290cb43aa9724 branch: main author: opavliuk <40970635+opavlyuk at users.noreply.github.com> committer: asvetlov date: 2023-07-31T14:33:26Z summary: gh-87799: Improve the textual representation of IPv4-mapped IPv6 addresses (#29345) Represent IPv4-mapped IPv6 address as x:x:x:x:x:x:d.d.d.d, where the 'x's are the hexadecimal values of the six high-order 16-bit pieces of the address, and the 'd's are the decimal values of the four low-order 8-bit pieces of the address (standard IPv4 representation). --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Andrew Svetlov files: A Misc/NEWS.d/next/Library/2021-10-31-16-06-28.bpo-43633.vflwXv.rst M Lib/ipaddress.py M Lib/test/test_ipaddress.py diff --git a/Lib/ipaddress.py b/Lib/ipaddress.py index af1d5c4800cce..f5aba434fd425 100644 --- a/Lib/ipaddress.py +++ b/Lib/ipaddress.py @@ -1923,8 +1923,40 @@ def __init__(self, address): self._ip = self._ip_int_from_string(addr_str) + def _explode_shorthand_ip_string(self): + ipv4_mapped = self.ipv4_mapped + if ipv4_mapped is None: + long_form = super()._explode_shorthand_ip_string() + else: + prefix_len = 30 + raw_exploded_str = super()._explode_shorthand_ip_string() + long_form = "%s%s" % (raw_exploded_str[:prefix_len], str(ipv4_mapped)) + return long_form + + def _ipv4_mapped_ipv6_to_str(self): + """Return convenient text representation of IPv4-mapped IPv6 address + + See RFC 4291 2.5.5.2, 2.2 p.3 for details. + + Returns: + A string, 'x:x:x:x:x:x:d.d.d.d', where the 'x's are the hexadecimal values of + the six high-order 16-bit pieces of the address, and the 'd's are + the decimal values of the four low-order 8-bit pieces of the + address (standard IPv4 representation) as defined in RFC 4291 2.2 p.3. + + """ + ipv4_mapped = self.ipv4_mapped + if ipv4_mapped is None: + raise AddressValueError("Can not apply to non-IPv4-mapped IPv6 address %s" % str(self)) + high_order_bits = self._ip >> 32 + return "%s:%s" % (self._string_from_ip_int(high_order_bits), str(ipv4_mapped)) + def __str__(self): - ip_str = super().__str__() + ipv4_mapped = self.ipv4_mapped + if ipv4_mapped is None: + ip_str = super().__str__() + else: + ip_str = self._ipv4_mapped_ipv6_to_str() return ip_str + '%' + self._scope_id if self._scope_id else ip_str def __hash__(self): diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py index a5388b2e5debd..6f204948c9fc4 100644 --- a/Lib/test/test_ipaddress.py +++ b/Lib/test/test_ipaddress.py @@ -1321,6 +1321,17 @@ def testGetIp(self): self.assertEqual(str(self.ipv6_scoped_interface.ip), '2001:658:22a:cafe:200::1') + def testIPv6IPv4MappedStringRepresentation(self): + long_prefix = '0000:0000:0000:0000:0000:ffff:' + short_prefix = '::ffff:' + ipv4 = '1.2.3.4' + ipv6_ipv4_str = short_prefix + ipv4 + ipv6_ipv4_addr = ipaddress.IPv6Address(ipv6_ipv4_str) + ipv6_ipv4_iface = ipaddress.IPv6Interface(ipv6_ipv4_str) + self.assertEqual(str(ipv6_ipv4_addr), ipv6_ipv4_str) + self.assertEqual(ipv6_ipv4_addr.exploded, long_prefix + ipv4) + self.assertEqual(str(ipv6_ipv4_iface.ip), ipv6_ipv4_str) + def testGetScopeId(self): self.assertEqual(self.ipv6_address.scope_id, None) diff --git a/Misc/NEWS.d/next/Library/2021-10-31-16-06-28.bpo-43633.vflwXv.rst b/Misc/NEWS.d/next/Library/2021-10-31-16-06-28.bpo-43633.vflwXv.rst new file mode 100644 index 0000000000000..025de1e1a7d6e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-10-31-16-06-28.bpo-43633.vflwXv.rst @@ -0,0 +1 @@ +Improve the textual representation of IPv4-mapped IPv6 addresses (:rfc:`4291` Sections 2.2, 2.5.5.2) in :mod:`ipaddress`. Patch by Oleksandr Pavliuk. From webhook-mailer at python.org Mon Jul 31 10:40:52 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 14:40:52 -0000 Subject: [Python-checkins] [3.11] gh-105751: Remove obsolete `object` base class in some ctypes tests (GH-107460) (#107502) Message-ID: https://github.com/python/cpython/commit/1b40431189b2fe5da8020d28b60d6d2854b780ba commit: 1b40431189b2fe5da8020d28b60d6d2854b780ba branch: 3.11 author: ?ukasz Langa committer: ambv date: 2023-07-31T16:40:47+02:00 summary: [3.11] gh-105751: Remove obsolete `object` base class in some ctypes tests (GH-107460) (#107502) (cherry picked from commit 520efecfc3aed34d3a44545c7cd872d1aea8c7dc) Co-authored-by: Tomas R files: M Lib/ctypes/test/test_as_parameter.py M Lib/ctypes/test/test_callbacks.py M Lib/ctypes/test/test_numbers.py M Lib/ctypes/test/test_parameters.py diff --git a/Lib/ctypes/test/test_as_parameter.py b/Lib/ctypes/test/test_as_parameter.py index 9c39179d2a446..aaaf6e2ceb0e8 100644 --- a/Lib/ctypes/test/test_as_parameter.py +++ b/Lib/ctypes/test/test_as_parameter.py @@ -194,7 +194,7 @@ class S8I(Structure): def test_recursive_as_param(self): from ctypes import c_int - class A(object): + class A: pass a = A() @@ -205,7 +205,7 @@ class A(object): #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -class AsParamWrapper(object): +class AsParamWrapper: def __init__(self, param): self._as_parameter_ = param @@ -214,7 +214,7 @@ class AsParamWrapperTestCase(BasicWrapTestCase): #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -class AsParamPropertyWrapper(object): +class AsParamPropertyWrapper: def __init__(self, param): self._param = param diff --git a/Lib/ctypes/test/test_callbacks.py b/Lib/ctypes/test/test_callbacks.py index 8e27359e4a8bf..5c514db5114e9 100644 --- a/Lib/ctypes/test/test_callbacks.py +++ b/Lib/ctypes/test/test_callbacks.py @@ -122,7 +122,7 @@ def test_unsupported_restype_2(self): def test_issue_7959(self): proto = self.functype.__func__(None) - class X(object): + class X: def func(self): pass def __init__(self): self.v = proto(self.func) diff --git a/Lib/ctypes/test/test_numbers.py b/Lib/ctypes/test/test_numbers.py index dc5d4a713c5b0..a7696376a5ab0 100644 --- a/Lib/ctypes/test/test_numbers.py +++ b/Lib/ctypes/test/test_numbers.py @@ -98,7 +98,7 @@ def test_byref(self): def test_floats(self): # c_float and c_double can be created from # Python int and float - class FloatLike(object): + class FloatLike: def __float__(self): return 2.0 f = FloatLike() @@ -109,15 +109,15 @@ def __float__(self): self.assertEqual(t(f).value, 2.0) def test_integers(self): - class FloatLike(object): + class FloatLike: def __float__(self): return 2.0 f = FloatLike() - class IntLike(object): + class IntLike: def __int__(self): return 2 d = IntLike() - class IndexLike(object): + class IndexLike: def __index__(self): return 2 i = IndexLike() diff --git a/Lib/ctypes/test/test_parameters.py b/Lib/ctypes/test/test_parameters.py index 3fdc994e9078c..59c94e3cc23cb 100644 --- a/Lib/ctypes/test/test_parameters.py +++ b/Lib/ctypes/test/test_parameters.py @@ -145,7 +145,7 @@ def test_noctypes_argtype(self): # TypeError: has no from_param method self.assertRaises(TypeError, setattr, func, "argtypes", (object,)) - class Adapter(object): + class Adapter: def from_param(cls, obj): return None @@ -153,7 +153,7 @@ def from_param(cls, obj): self.assertEqual(func(None), None) self.assertEqual(func(object()), None) - class Adapter(object): + class Adapter: def from_param(cls, obj): return obj @@ -162,7 +162,7 @@ def from_param(cls, obj): self.assertRaises(ArgumentError, func, object()) self.assertEqual(func(c_void_p(42)), 42) - class Adapter(object): + class Adapter: def from_param(cls, obj): raise ValueError(obj) From webhook-mailer at python.org Mon Jul 31 11:01:29 2023 From: webhook-mailer at python.org (ambv) Date: Mon, 31 Jul 2023 15:01:29 -0000 Subject: [Python-checkins] [3.12] gh-105751: Remove obsolete `object` base class in some ctypes tests (GH-107460) (#107501) Message-ID: https://github.com/python/cpython/commit/831fd19d30a4051d96117f42b625642ee8a250fb commit: 831fd19d30a4051d96117f42b625642ee8a250fb branch: 3.12 author: ?ukasz Langa committer: ambv date: 2023-07-31T17:01:25+02:00 summary: [3.12] gh-105751: Remove obsolete `object` base class in some ctypes tests (GH-107460) (#107501) (cherry picked from commit 520efecfc3aed34d3a44545c7cd872d1aea8c7dc) Co-authored-by: Tomas R files: M Lib/test/test_ctypes/test_as_parameter.py M Lib/test/test_ctypes/test_callbacks.py M Lib/test/test_ctypes/test_numbers.py M Lib/test/test_ctypes/test_parameters.py diff --git a/Lib/test/test_ctypes/test_as_parameter.py b/Lib/test/test_ctypes/test_as_parameter.py index e9ec9ad847b48..36fec572b1674 100644 --- a/Lib/test/test_ctypes/test_as_parameter.py +++ b/Lib/test/test_ctypes/test_as_parameter.py @@ -194,7 +194,7 @@ class S8I(Structure): def test_recursive_as_param(self): from ctypes import c_int - class A(object): + class A: pass a = A() @@ -205,7 +205,7 @@ class A(object): #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -class AsParamWrapper(object): +class AsParamWrapper: def __init__(self, param): self._as_parameter_ = param @@ -214,7 +214,7 @@ class AsParamWrapperTestCase(BasicWrapTestCase): #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -class AsParamPropertyWrapper(object): +class AsParamPropertyWrapper: def __init__(self, param): self._param = param diff --git a/Lib/test/test_ctypes/test_callbacks.py b/Lib/test/test_ctypes/test_callbacks.py index 0069c640d7927..a9c6524b4d4ee 100644 --- a/Lib/test/test_ctypes/test_callbacks.py +++ b/Lib/test/test_ctypes/test_callbacks.py @@ -122,7 +122,7 @@ def test_unsupported_restype_2(self): def test_issue_7959(self): proto = self.functype.__func__(None) - class X(object): + class X: def func(self): pass def __init__(self): self.v = proto(self.func) diff --git a/Lib/test/test_ctypes/test_numbers.py b/Lib/test/test_ctypes/test_numbers.py index dc5d4a713c5b0..a7696376a5ab0 100644 --- a/Lib/test/test_ctypes/test_numbers.py +++ b/Lib/test/test_ctypes/test_numbers.py @@ -98,7 +98,7 @@ def test_byref(self): def test_floats(self): # c_float and c_double can be created from # Python int and float - class FloatLike(object): + class FloatLike: def __float__(self): return 2.0 f = FloatLike() @@ -109,15 +109,15 @@ def __float__(self): self.assertEqual(t(f).value, 2.0) def test_integers(self): - class FloatLike(object): + class FloatLike: def __float__(self): return 2.0 f = FloatLike() - class IntLike(object): + class IntLike: def __int__(self): return 2 d = IntLike() - class IndexLike(object): + class IndexLike: def __index__(self): return 2 i = IndexLike() diff --git a/Lib/test/test_ctypes/test_parameters.py b/Lib/test/test_ctypes/test_parameters.py index 06cc95107b79f..f5afa76fd92a3 100644 --- a/Lib/test/test_ctypes/test_parameters.py +++ b/Lib/test/test_ctypes/test_parameters.py @@ -167,7 +167,7 @@ def test_noctypes_argtype(self): # TypeError: has no from_param method self.assertRaises(TypeError, setattr, func, "argtypes", (object,)) - class Adapter(object): + class Adapter: def from_param(cls, obj): return None @@ -175,7 +175,7 @@ def from_param(cls, obj): self.assertEqual(func(None), None) self.assertEqual(func(object()), None) - class Adapter(object): + class Adapter: def from_param(cls, obj): return obj @@ -184,7 +184,7 @@ def from_param(cls, obj): self.assertRaises(ArgumentError, func, object()) self.assertEqual(func(c_void_p(42)), 42) - class Adapter(object): + class Adapter: def from_param(cls, obj): raise ValueError(obj) From webhook-mailer at python.org Mon Jul 31 11:23:13 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 31 Jul 2023 15:23:13 -0000 Subject: [Python-checkins] gh-105578: Add more usage examples to `typing.AnyStr` docs (#107045) Message-ID: https://github.com/python/cpython/commit/f877b32b879f2076bb1c52826af0c28ebf1aaeed commit: f877b32b879f2076bb1c52826af0c28ebf1aaeed branch: main author: Michael The committer: AlexWaygood date: 2023-07-31T16:23:08+01:00 summary: gh-105578: Add more usage examples to `typing.AnyStr` docs (#107045) ``typing.AnyStr`` has different semantics to ``str | bytes``, which often leads to user confusion files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index e2791bbc97b00..539203f1d7136 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -849,6 +849,21 @@ using ``[]``. concat(b"foo", b"bar") # OK, output has type 'bytes' concat("foo", b"bar") # Error, cannot mix str and bytes + Note that, despite its name, ``AnyStr`` has nothing to do with the + :class:`Any` type, nor does it mean "any string". In particular, ``AnyStr`` + and ``str | bytes`` are different from each other and have different use + cases:: + + # Invalid use of AnyStr: + # The type variable is used only once in the function signature, + # so cannot be "solved" by the type checker + def greet_bad(cond: bool) -> AnyStr: + return "hi there!" if cond else b"greetings!" + + # The better way of annotating this function: + def greet_proper(cond: bool) -> str | bytes: + return "hi there!" if cond else b"greetings!" + .. data:: LiteralString Special type that includes only literal strings. From webhook-mailer at python.org Mon Jul 31 11:33:26 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 31 Jul 2023 15:33:26 -0000 Subject: [Python-checkins] [3.12] gh-105578: Add more usage examples to `typing.AnyStr` docs (GH-107045) (#107503) Message-ID: https://github.com/python/cpython/commit/9f58d9ec9061f1e6ec37432b6031b7c9a580fb6c commit: 9f58d9ec9061f1e6ec37432b6031b7c9a580fb6c branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-31T15:33:00Z summary: [3.12] gh-105578: Add more usage examples to `typing.AnyStr` docs (GH-107045) (#107503) gh-105578: Add more usage examples to `typing.AnyStr` docs (GH-107045) ``typing.AnyStr`` has different semantics to ``str | bytes``, which often leads to user confusion (cherry picked from commit f877b32b879f2076bb1c52826af0c28ebf1aaeed) Co-authored-by: Michael The files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 11c39a444ef65..60bf06e471cb2 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -849,6 +849,21 @@ using ``[]``. concat(b"foo", b"bar") # OK, output has type 'bytes' concat("foo", b"bar") # Error, cannot mix str and bytes + Note that, despite its name, ``AnyStr`` has nothing to do with the + :class:`Any` type, nor does it mean "any string". In particular, ``AnyStr`` + and ``str | bytes`` are different from each other and have different use + cases:: + + # Invalid use of AnyStr: + # The type variable is used only once in the function signature, + # so cannot be "solved" by the type checker + def greet_bad(cond: bool) -> AnyStr: + return "hi there!" if cond else b"greetings!" + + # The better way of annotating this function: + def greet_proper(cond: bool) -> str | bytes: + return "hi there!" if cond else b"greetings!" + .. data:: LiteralString Special type that includes only literal strings. From webhook-mailer at python.org Mon Jul 31 11:33:39 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 31 Jul 2023 15:33:39 -0000 Subject: [Python-checkins] [3.11] gh-105578: Add more usage examples to `typing.AnyStr` docs (GH-107045) (#107504) Message-ID: https://github.com/python/cpython/commit/3de42bbf8cf372b2e6a59bb505b1c2c1544018db commit: 3de42bbf8cf372b2e6a59bb505b1c2c1544018db branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: AlexWaygood date: 2023-07-31T15:33:07Z summary: [3.11] gh-105578: Add more usage examples to `typing.AnyStr` docs (GH-107045) (#107504) gh-105578: Add more usage examples to `typing.AnyStr` docs (GH-107045) ``typing.AnyStr`` has different semantics to ``str | bytes``, which often leads to user confusion (cherry picked from commit f877b32b879f2076bb1c52826af0c28ebf1aaeed) Co-authored-by: Michael The files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index a0766690bad28..b6bb50b65db62 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -801,6 +801,21 @@ using ``[]``. concat(b"foo", b"bar") # OK, output has type 'bytes' concat("foo", b"bar") # Error, cannot mix str and bytes + Note that, despite its name, ``AnyStr`` has nothing to do with the + :class:`Any` type, nor does it mean "any string". In particular, ``AnyStr`` + and ``str | bytes`` are different from each other and have different use + cases:: + + # Invalid use of AnyStr: + # The type variable is used only once in the function signature, + # so cannot be "solved" by the type checker + def greet_bad(cond: bool) -> AnyStr: + return "hi there!" if cond else b"greetings!" + + # The better way of annotating this function: + def greet_proper(cond: bool) -> str | bytes: + return "hi there!" if cond else b"greetings!" + .. data:: LiteralString Special type that includes only literal strings. From webhook-mailer at python.org Mon Jul 31 11:43:57 2023 From: webhook-mailer at python.org (AlexWaygood) Date: Mon, 31 Jul 2023 15:43:57 -0000 Subject: [Python-checkins] gh-105578: Document that `AnyStr` is deprecated in py313 (#107116) Message-ID: https://github.com/python/cpython/commit/a22ff8e11c114e0c4c54c57b9e7f7520db9d5163 commit: a22ff8e11c114e0c4c54c57b9e7f7520db9d5163 branch: main author: Michael The committer: AlexWaygood date: 2023-07-31T15:43:53Z summary: gh-105578: Document that `AnyStr` is deprecated in py313 (#107116) It will not be removed until Python 3.18. files: A Misc/NEWS.d/next/Library/2023-07-23-13-05-32.gh-issue-105578.XAQtyR.rst M Doc/library/typing.rst M Doc/whatsnew/3.13.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 539203f1d7136..f96a805cef7b2 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -864,6 +864,16 @@ using ``[]``. def greet_proper(cond: bool) -> str | bytes: return "hi there!" if cond else b"greetings!" + .. deprecated-removed:: 3.13 3.18 + Deprecated in favor of the new :ref:`type parameter syntax `. + Use ``class A[T: (str, bytes)]: ...`` instead of importing ``AnyStr``. See + :pep:`695` for more details. + + In Python 3.16, ``AnyStr`` will be removed from ``typing.__all__``, and + deprecation warnings will be emitted at runtime when it is accessed or + imported from ``typing``. ``AnyStr`` will be removed from ``typing`` + in Python 3.18. + .. data:: LiteralString Special type that includes only literal strings. @@ -3700,3 +3710,7 @@ convenience. This is subject to change, and not all deprecations are listed. - 3.13 - 3.15 - :gh:`106309` + * - :data:`typing.AnyStr` + - 3.13 + - 3.18 + - :gh:`105578` diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index e20832a3a4c30..8fb4e6cfdf14c 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -196,6 +196,12 @@ Deprecated has yet to be supported by any major type checkers. (Contributed by Alex Waygood in :gh:`106309`.) + * :data:`typing.AnyStr` is deprecated. In Python 3.16, it will be removed from + ``typing.__all__``, and a :exc:`DeprecationWarning` will be emitted when it + is imported or accessed. It will be removed entirely in Python 3.18. Use + the new :ref:`type parameter syntax ` instead. + (Contributed by Michael The in :gh:`107116`.) + * :mod:`wave`: Deprecate the ``getmark()``, ``setmark()`` and ``getmarkers()`` methods of the :class:`wave.Wave_read` and :class:`wave.Wave_write` classes. They will be removed in Python 3.15. diff --git a/Misc/NEWS.d/next/Library/2023-07-23-13-05-32.gh-issue-105578.XAQtyR.rst b/Misc/NEWS.d/next/Library/2023-07-23-13-05-32.gh-issue-105578.XAQtyR.rst new file mode 100644 index 0000000000000..4a03f5c35ff6c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-07-23-13-05-32.gh-issue-105578.XAQtyR.rst @@ -0,0 +1,2 @@ +Deprecate :class:`typing.AnyStr` in favor of the new Type Parameter syntax. +See PEP 695. From webhook-mailer at python.org Mon Jul 31 11:55:37 2023 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 31 Jul 2023 15:55:37 -0000 Subject: [Python-checkins] GH-104909: Move unused cache entries from uops to macros (#107444) Message-ID: https://github.com/python/cpython/commit/5eb80a61f582802c3f1caa3bf4dc754847bf1e75 commit: 5eb80a61f582802c3f1caa3bf4dc754847bf1e75 branch: main author: Guido van Rossum committer: gvanrossum date: 2023-07-31T08:55:33-07:00 summary: GH-104909: Move unused cache entries from uops to macros (#107444) There's no need to use a dummy uop to skip unused cache entries. The macro syntax lets you write `unused/1` instead. Similarly, move `unused/5` from op `_LOAD_ATTR_INSTANCE_VALUE` to macro `LOAD_ATTR_INSTANCE_VALUE`. files: M Include/internal/pycore_opcode_metadata.h M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index d525913f8a7ab..64a084618e13b 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -35,27 +35,26 @@ #define _BINARY_OP_ADD_UNICODE 311 #define _LOAD_LOCALS 312 #define _LOAD_FROM_DICT_OR_GLOBALS 313 -#define _SKIP_CACHE 314 -#define _GUARD_GLOBALS_VERSION 315 -#define _GUARD_BUILTINS_VERSION 316 -#define _LOAD_GLOBAL_MODULE 317 -#define _LOAD_GLOBAL_BUILTINS 318 -#define _GUARD_TYPE_VERSION 319 -#define _CHECK_MANAGED_OBJECT_HAS_VALUES 320 -#define _LOAD_ATTR_INSTANCE_VALUE 321 -#define IS_NONE 322 -#define _ITER_CHECK_LIST 323 -#define _IS_ITER_EXHAUSTED_LIST 324 -#define _ITER_NEXT_LIST 325 -#define _ITER_CHECK_TUPLE 326 -#define _IS_ITER_EXHAUSTED_TUPLE 327 -#define _ITER_NEXT_TUPLE 328 -#define _ITER_CHECK_RANGE 329 -#define _IS_ITER_EXHAUSTED_RANGE 330 -#define _ITER_NEXT_RANGE 331 -#define _POP_JUMP_IF_FALSE 332 -#define _POP_JUMP_IF_TRUE 333 -#define JUMP_TO_TOP 334 +#define _GUARD_GLOBALS_VERSION 314 +#define _GUARD_BUILTINS_VERSION 315 +#define _LOAD_GLOBAL_MODULE 316 +#define _LOAD_GLOBAL_BUILTINS 317 +#define _GUARD_TYPE_VERSION 318 +#define _CHECK_MANAGED_OBJECT_HAS_VALUES 319 +#define _LOAD_ATTR_INSTANCE_VALUE 320 +#define IS_NONE 321 +#define _ITER_CHECK_LIST 322 +#define _IS_ITER_EXHAUSTED_LIST 323 +#define _ITER_NEXT_LIST 324 +#define _ITER_CHECK_TUPLE 325 +#define _IS_ITER_EXHAUSTED_TUPLE 326 +#define _ITER_NEXT_TUPLE 327 +#define _ITER_CHECK_RANGE 328 +#define _IS_ITER_EXHAUSTED_RANGE 329 +#define _ITER_NEXT_RANGE 330 +#define _POP_JUMP_IF_FALSE 331 +#define _POP_JUMP_IF_TRUE 332 +#define JUMP_TO_TOP 333 #ifndef NEED_OPCODE_METADATA extern int _PyOpcode_num_popped(int opcode, int oparg, bool jump); @@ -945,7 +944,7 @@ _PyOpcode_num_pushed(int opcode, int oparg, bool jump) { } #endif -enum InstructionFormat { INSTR_FMT_IB, INSTR_FMT_IBC, INSTR_FMT_IBC00, INSTR_FMT_IBC000, INSTR_FMT_IBC00000, INSTR_FMT_IBC00000000, INSTR_FMT_IX, INSTR_FMT_IXC, INSTR_FMT_IXC0, INSTR_FMT_IXC00, INSTR_FMT_IXC000 }; +enum InstructionFormat { INSTR_FMT_IB, INSTR_FMT_IBC, INSTR_FMT_IBC00, INSTR_FMT_IBC000, INSTR_FMT_IBC00000000, INSTR_FMT_IX, INSTR_FMT_IXC, INSTR_FMT_IXC0, INSTR_FMT_IXC00, INSTR_FMT_IXC000 }; #define IS_VALID_OPCODE(OP) \ (((OP) >= 0) && ((OP) < OPCODE_METADATA_SIZE) && \ @@ -1279,8 +1278,8 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[OPCODE_MACRO_EXPAN [LOAD_NAME] = { .nuops = 2, .uops = { { _LOAD_LOCALS, 0, 0 }, { _LOAD_FROM_DICT_OR_GLOBALS, 0, 0 } } }, [LOAD_FROM_DICT_OR_GLOBALS] = { .nuops = 1, .uops = { { _LOAD_FROM_DICT_OR_GLOBALS, 0, 0 } } }, [LOAD_GLOBAL] = { .nuops = 1, .uops = { { LOAD_GLOBAL, 0, 0 } } }, - [LOAD_GLOBAL_MODULE] = { .nuops = 4, .uops = { { _SKIP_CACHE, 0, 0 }, { _GUARD_GLOBALS_VERSION, 1, 1 }, { _SKIP_CACHE, 0, 0 }, { _LOAD_GLOBAL_MODULE, 1, 3 } } }, - [LOAD_GLOBAL_BUILTIN] = { .nuops = 4, .uops = { { _SKIP_CACHE, 0, 0 }, { _GUARD_GLOBALS_VERSION, 1, 1 }, { _GUARD_BUILTINS_VERSION, 1, 2 }, { _LOAD_GLOBAL_BUILTINS, 1, 3 } } }, + [LOAD_GLOBAL_MODULE] = { .nuops = 2, .uops = { { _GUARD_GLOBALS_VERSION, 1, 1 }, { _LOAD_GLOBAL_MODULE, 1, 3 } } }, + [LOAD_GLOBAL_BUILTIN] = { .nuops = 3, .uops = { { _GUARD_GLOBALS_VERSION, 1, 1 }, { _GUARD_BUILTINS_VERSION, 1, 2 }, { _LOAD_GLOBAL_BUILTINS, 1, 3 } } }, [DELETE_FAST] = { .nuops = 1, .uops = { { DELETE_FAST, 0, 0 } } }, [DELETE_DEREF] = { .nuops = 1, .uops = { { DELETE_DEREF, 0, 0 } } }, [LOAD_FROM_DICT_OR_DEREF] = { .nuops = 1, .uops = { { LOAD_FROM_DICT_OR_DEREF, 0, 0 } } }, @@ -1302,7 +1301,7 @@ const struct opcode_macro_expansion _PyOpcode_macro_expansion[OPCODE_MACRO_EXPAN [LOAD_SUPER_ATTR_ATTR] = { .nuops = 1, .uops = { { LOAD_SUPER_ATTR_ATTR, 0, 0 } } }, [LOAD_SUPER_ATTR_METHOD] = { .nuops = 1, .uops = { { LOAD_SUPER_ATTR_METHOD, 0, 0 } } }, [LOAD_ATTR] = { .nuops = 1, .uops = { { LOAD_ATTR, 0, 0 } } }, - [LOAD_ATTR_INSTANCE_VALUE] = { .nuops = 4, .uops = { { _SKIP_CACHE, 0, 0 }, { _GUARD_TYPE_VERSION, 2, 1 }, { _CHECK_MANAGED_OBJECT_HAS_VALUES, 0, 0 }, { _LOAD_ATTR_INSTANCE_VALUE, 1, 3 } } }, + [LOAD_ATTR_INSTANCE_VALUE] = { .nuops = 3, .uops = { { _GUARD_TYPE_VERSION, 2, 1 }, { _CHECK_MANAGED_OBJECT_HAS_VALUES, 0, 0 }, { _LOAD_ATTR_INSTANCE_VALUE, 1, 3 } } }, [COMPARE_OP] = { .nuops = 1, .uops = { { COMPARE_OP, 0, 0 } } }, [COMPARE_OP_FLOAT] = { .nuops = 1, .uops = { { COMPARE_OP_FLOAT, 0, 0 } } }, [COMPARE_OP_INT] = { .nuops = 1, .uops = { { COMPARE_OP_INT, 0, 0 } } }, @@ -1356,7 +1355,6 @@ const char * const _PyOpcode_uop_name[OPCODE_UOP_NAME_SIZE] = { [_BINARY_OP_ADD_UNICODE] = "_BINARY_OP_ADD_UNICODE", [_LOAD_LOCALS] = "_LOAD_LOCALS", [_LOAD_FROM_DICT_OR_GLOBALS] = "_LOAD_FROM_DICT_OR_GLOBALS", - [_SKIP_CACHE] = "_SKIP_CACHE", [_GUARD_GLOBALS_VERSION] = "_GUARD_GLOBALS_VERSION", [_GUARD_BUILTINS_VERSION] = "_GUARD_BUILTINS_VERSION", [_LOAD_GLOBAL_MODULE] = "_LOAD_GLOBAL_MODULE", diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 2b871e8546efb..7320c309884ac 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1348,9 +1348,6 @@ dummy_func( null = NULL; } - op(_SKIP_CACHE, (unused/1 -- )) { - } - op(_GUARD_GLOBALS_VERSION, (version/1 --)) { PyDictObject *dict = (PyDictObject *)GLOBALS(); DEOPT_IF(!PyDict_CheckExact(dict), LOAD_GLOBAL); @@ -1386,13 +1383,13 @@ dummy_func( } macro(LOAD_GLOBAL_MODULE) = - _SKIP_CACHE + // Skip over the counter + unused/1 + // Skip over the counter _GUARD_GLOBALS_VERSION + - _SKIP_CACHE + // Skip over the builtins version + unused/1 + // Skip over the builtins version _LOAD_GLOBAL_MODULE; macro(LOAD_GLOBAL_BUILTIN) = - _SKIP_CACHE + // Skip over the counter + unused/1 + // Skip over the counter _GUARD_GLOBALS_VERSION + _GUARD_BUILTINS_VERSION + _LOAD_GLOBAL_BUILTINS; @@ -1824,7 +1821,7 @@ dummy_func( DEOPT_IF(!_PyDictOrValues_IsValues(dorv), LOAD_ATTR); } - op(_LOAD_ATTR_INSTANCE_VALUE, (index/1, unused/5, owner -- res2 if (oparg & 1), res)) { + op(_LOAD_ATTR_INSTANCE_VALUE, (index/1, owner -- res2 if (oparg & 1), res)) { PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(owner); res = _PyDictOrValues_GetValues(dorv)->values[index]; DEOPT_IF(res == NULL, LOAD_ATTR); @@ -1835,10 +1832,11 @@ dummy_func( } macro(LOAD_ATTR_INSTANCE_VALUE) = - _SKIP_CACHE + // Skip over the counter + unused/1 + // Skip over the counter _GUARD_TYPE_VERSION + _CHECK_MANAGED_OBJECT_HAS_VALUES + - _LOAD_ATTR_INSTANCE_VALUE; + _LOAD_ATTR_INSTANCE_VALUE + + unused/5; // Skip over rest of cache inst(LOAD_ATTR_MODULE, (unused/1, type_version/2, index/1, unused/5, owner -- res2 if (oparg & 1), res)) { DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR); diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index f3e24bc0479ec..c2f0d0a77b16d 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1009,10 +1009,6 @@ break; } - case _SKIP_CACHE: { - break; - } - case _GUARD_GLOBALS_VERSION: { uint16_t version = (uint16_t)operand; PyDictObject *dict = (PyDictObject *)GLOBALS(); diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 6ac622b11acac..dccd6da912376 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -1663,8 +1663,6 @@ TARGET(LOAD_GLOBAL_MODULE) { PyObject *_tmp_1; PyObject *_tmp_2; - { - } { uint16_t version = read_u16(&next_instr[1].cache); PyDictObject *dict = (PyDictObject *)GLOBALS(); @@ -1672,8 +1670,6 @@ DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); assert(DK_IS_UNICODE(dict->ma_keys)); } - { - } { PyObject *null = NULL; PyObject *res; @@ -1699,8 +1695,6 @@ TARGET(LOAD_GLOBAL_BUILTIN) { PyObject *_tmp_1; PyObject *_tmp_2; - { - } { uint16_t version = read_u16(&next_instr[1].cache); PyDictObject *dict = (PyDictObject *)GLOBALS(); @@ -2243,8 +2237,6 @@ TARGET(LOAD_ATTR_INSTANCE_VALUE) { PyObject *_tmp_1; PyObject *_tmp_2 = stack_pointer[-1]; - { - } { PyObject *owner = _tmp_2; uint32_t type_version = read_u32(&next_instr[1].cache); From webhook-mailer at python.org Mon Jul 31 11:57:29 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 31 Jul 2023 15:57:29 -0000 Subject: [Python-checkins] [3.12] gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (GH-107496) (#107499) Message-ID: https://github.com/python/cpython/commit/f66b992e27e7af534606132a48837b190db2d81c commit: f66b992e27e7af534606132a48837b190db2d81c branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-31T15:57:25Z summary: [3.12] gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (GH-107496) (#107499) (cherry picked from commit 2c5d206b33e4cdcafaaaf1eeaa189c10de332dc5) Co-authored-by: Erlend E. Aasland files: M Lib/test/test_clinic.py diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 3d740910a57ae..520cc51302212 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -314,6 +314,26 @@ def test_unknown_destination_command(self): msg = "unknown destination command 'nosuchcommand'" self.assertIn(msg, out) + def test_no_access_to_members_in_converter_init(self): + out = self.expect_failure(""" + /*[python input] + class Custom_converter(CConverter): + converter = "some_c_function" + def converter_init(self): + self.function.noaccess + [python start generated code]*/ + /*[clinic input] + module test + test.fn + a: Custom + [clinic start generated code]*/ + """) + msg = ( + "Stepped on a land mine, trying to access attribute 'noaccess':\n" + "Don't access members of self.function inside converter_init!" + ) + self.assertIn(msg, out) + class ClinicGroupPermuterTest(TestCase): def _test(self, l, m, r, output): From webhook-mailer at python.org Mon Jul 31 13:16:04 2023 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 31 Jul 2023 17:16:04 -0000 Subject: [Python-checkins] [3.12] GH-101291: Add warning to "what's new" that `PyLongObject` internals have changed. (GH-107388) (#107392) Message-ID: https://github.com/python/cpython/commit/5f7862dc1f0ba7e47fc01acb7f7a2e0f68176924 commit: 5f7862dc1f0ba7e47fc01acb7f7a2e0f68176924 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: gvanrossum date: 2023-07-31T10:16:00-07:00 summary: [3.12] GH-101291: Add warning to "what's new" that `PyLongObject` internals have changed. (GH-107388) (#107392) (cherry picked from commit 1ee605c5888fbc3d51b3e7610bac38ea6bc25e31) Co-authored-by: Mark Shannon files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 2ec25eb25ecb4..fe912e8422e9e 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1867,6 +1867,17 @@ Porting to Python 3.12 subinterpreter that they don't support (or haven't yet been loaded in). See :gh:`104668` for more info. +* :c:struct:`PyLongObject` has had its internals changed for better performance. + Although the internals of :c:struct:`PyLongObject` are private, they are used + by some extension modules. + The internal fields should no longer be accessed directly, instead the API + functions beginning ``PyLong_...`` should be used instead. + Two new *unstable* API functions are provided for efficient access to the + value of :c:struct:`PyLongObject`\s which fit into a single machine word: + + * :c:func:`PyUnstable_Long_IsCompact` + * :c:func:`PyUnstable_Long_CompactValue` + Deprecated ---------- From webhook-mailer at python.org Mon Jul 31 13:24:48 2023 From: webhook-mailer at python.org (ethanfurman) Date: Mon, 31 Jul 2023 17:24:48 -0000 Subject: [Python-checkins] gh-106762: Add news for `EnumMeta.__getattr__` removal (GH-107466) Message-ID: https://github.com/python/cpython/commit/de51dede5b48ef23d7d33d92f3616824e23fd205 commit: de51dede5b48ef23d7d33d92f3616824e23fd205 branch: main author: Jacob Walls committer: ethanfurman date: 2023-07-31T10:24:44-07:00 summary: gh-106762: Add news for `EnumMeta.__getattr__` removal (GH-107466) files: M Doc/whatsnew/3.12.rst M Misc/NEWS.d/3.12.0a1.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 99500e1f32146..0ac4a3df8fefa 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1234,6 +1234,10 @@ Removed (Contributed by Pradyun Gedam in :gh:`95299`.) +* :mod:`enum`: Remove ``EnumMeta.__getattr__``, which is no longer needed for + enum attribute access. + (Contributed by Ethan Furman in :gh:`95083`.) + * :mod:`ftplib`: Remove the ``FTP_TLS.ssl_version`` class attribute: use the *context* parameter instead. (Contributed by Victor Stinner in :gh:`94172`.) diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index 40f6a6cb430d3..5178f4055e7b8 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -2752,7 +2752,7 @@ by Shin-myoung-serp. .. section: Library Add deprecation warning for enum ``member.member`` access (e.g. -``Color.RED.BLUE``). +``Color.RED.BLUE``). Remove ``EnumMeta.__getattr__``. .. From webhook-mailer at python.org Mon Jul 31 14:33:36 2023 From: webhook-mailer at python.org (ethanfurman) Date: Mon, 31 Jul 2023 18:33:36 -0000 Subject: [Python-checkins] [3.12] gh-106762: Add news for `EnumMeta.__getattr__` removal (GH-107466) (GH-107509) Message-ID: https://github.com/python/cpython/commit/0fbe69fc41dd2e0bcbe4ab68fc82b727a4cc607b commit: 0fbe69fc41dd2e0bcbe4ab68fc82b727a4cc607b branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ethanfurman date: 2023-07-31T11:33:32-07:00 summary: [3.12] gh-106762: Add news for `EnumMeta.__getattr__` removal (GH-107466) (GH-107509) gh-106762: Add news for `EnumMeta.__getattr__` removal (GH-107466) (cherry picked from commit de51dede5b48ef23d7d33d92f3616824e23fd205) Co-authored-by: Jacob Walls files: M Doc/whatsnew/3.12.rst M Misc/NEWS.d/3.12.0a1.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index fe912e8422e9e..d7389f696ea2e 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1234,6 +1234,10 @@ Removed (Contributed by Pradyun Gedam in :gh:`95299`.) +* :mod:`enum`: Remove ``EnumMeta.__getattr__``, which is no longer needed for + enum attribute access. + (Contributed by Ethan Furman in :gh:`95083`.) + * :mod:`ftplib`: Remove the ``FTP_TLS.ssl_version`` class attribute: use the *context* parameter instead. (Contributed by Victor Stinner in :gh:`94172`.) diff --git a/Misc/NEWS.d/3.12.0a1.rst b/Misc/NEWS.d/3.12.0a1.rst index 1888cae0ba720..84036788774f7 100644 --- a/Misc/NEWS.d/3.12.0a1.rst +++ b/Misc/NEWS.d/3.12.0a1.rst @@ -2752,7 +2752,7 @@ by Shin-myoung-serp. .. section: Library Add deprecation warning for enum ``member.member`` access (e.g. -``Color.RED.BLUE``). +``Color.RED.BLUE``). Remove ``EnumMeta.__getattr__``. .. From webhook-mailer at python.org Mon Jul 31 17:17:01 2023 From: webhook-mailer at python.org (brandtbucher) Date: Mon, 31 Jul 2023 21:17:01 -0000 Subject: [Python-checkins] GH-104584: Fix incorrect uoperands (GH-107513) Message-ID: https://github.com/python/cpython/commit/5e584eb704d656c9a6a648dadda63237706287db commit: 5e584eb704d656c9a6a648dadda63237706287db branch: main author: Brandt Bucher committer: brandtbucher date: 2023-07-31T21:16:57Z summary: GH-104584: Fix incorrect uoperands (GH-107513) files: A Misc/NEWS.d/next/Core and Builtins/2023-07-29-22-01-30.gh-issue-104584.tINuoA.rst M Python/optimizer.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-07-29-22-01-30.gh-issue-104584.tINuoA.rst b/Misc/NEWS.d/next/Core and Builtins/2023-07-29-22-01-30.gh-issue-104584.tINuoA.rst new file mode 100644 index 0000000000000..059524831597b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-07-29-22-01-30.gh-issue-104584.tINuoA.rst @@ -0,0 +1,2 @@ +Fix an issue which caused incorrect inline caches to be read when running +with :envvar:`PYTHONUOPS` or :option:`-X uops <-X>` enabled. diff --git a/Python/optimizer.c b/Python/optimizer.c index 09120c33d130c..238ab02d09faa 100644 --- a/Python/optimizer.c +++ b/Python/optimizer.c @@ -579,7 +579,8 @@ translate_bytecode_to_trace( for (int i = 0; i < nuops; i++) { oparg = orig_oparg; uint64_t operand = 0; - int offset = expansion->uops[i].offset; + // Add one to account for the actual opcode/oparg pair: + int offset = expansion->uops[i].offset + 1; switch (expansion->uops[i].size) { case OPARG_FULL: if (extras && OPCODE_HAS_JUMP(opcode)) { From webhook-mailer at python.org Mon Jul 31 17:18:43 2023 From: webhook-mailer at python.org (brandtbucher) Date: Mon, 31 Jul 2023 21:18:43 -0000 Subject: [Python-checkins] Use tstate->interp to get the interpreter state in bytecodes.c (GH-107506) Message-ID: https://github.com/python/cpython/commit/dfb55d9d7f9349582a2077f87e95c8f9d5e2ebd4 commit: dfb55d9d7f9349582a2077f87e95c8f9d5e2ebd4 branch: main author: Brandt Bucher committer: brandtbucher date: 2023-07-31T14:18:38-07:00 summary: Use tstate->interp to get the interpreter state in bytecodes.c (GH-107506) files: M Python/bytecodes.c M Python/executor_cases.c.h M Python/generated_cases.c.h diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 7320c309884ac..0bea7b59597a2 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -3245,7 +3245,7 @@ dummy_func( total_args++; } DEOPT_IF(total_args != 1, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.len, CALL); STAT_INC(CALL, hit); PyObject *arg = args[0]; @@ -3272,7 +3272,7 @@ dummy_func( total_args++; } DEOPT_IF(total_args != 2, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.isinstance, CALL); STAT_INC(CALL, hit); PyObject *cls = args[1]; @@ -3295,7 +3295,7 @@ dummy_func( ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); assert(method != NULL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(method != interp->callable_cache.list_append, CALL); DEOPT_IF(!PyList_Check(self), CALL); STAT_INC(CALL, hit); diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index c2f0d0a77b16d..ed7cf4c15ec4a 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -2154,7 +2154,7 @@ total_args++; } DEOPT_IF(total_args != 1, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.len, CALL); STAT_INC(CALL, hit); PyObject *arg = args[0]; @@ -2189,7 +2189,7 @@ total_args++; } DEOPT_IF(total_args != 2, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.isinstance, CALL); STAT_INC(CALL, hit); PyObject *cls = args[1]; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index dccd6da912376..e43b3dedf14f5 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3984,7 +3984,7 @@ total_args++; } DEOPT_IF(total_args != 1, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.len, CALL); STAT_INC(CALL, hit); PyObject *arg = args[0]; @@ -4020,7 +4020,7 @@ total_args++; } DEOPT_IF(total_args != 2, CALL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(callable != interp->callable_cache.isinstance, CALL); STAT_INC(CALL, hit); PyObject *cls = args[1]; @@ -4050,7 +4050,7 @@ ASSERT_KWNAMES_IS_NULL(); assert(oparg == 1); assert(method != NULL); - PyInterpreterState *interp = _PyInterpreterState_GET(); + PyInterpreterState *interp = tstate->interp; DEOPT_IF(method != interp->callable_cache.list_append, CALL); DEOPT_IF(!PyList_Check(self), CALL); STAT_INC(CALL, hit); From webhook-mailer at python.org Mon Jul 31 17:33:17 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 31 Jul 2023 21:33:17 -0000 Subject: [Python-checkins] gh-107507: Replace 'The goals of Argument Clinic' with a summary (#107508) Message-ID: https://github.com/python/cpython/commit/abb71c6a8f73482c910ffdf050a86089a48e0e60 commit: abb71c6a8f73482c910ffdf050a86089a48e0e60 branch: main author: Erlend E. Aasland committer: erlend-aasland date: 2023-07-31T23:33:13+02:00 summary: gh-107507: Replace 'The goals of Argument Clinic' with a summary (#107508) Summarise the goals of Argument Clinic in a single sentence. Mention that Argument Clinic was introduced with PEP-436. files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 7aafd48711b58..9c9a4f45dd0f5 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -13,8 +13,10 @@ Argument Clinic How-To .. topic:: Abstract Argument Clinic is a preprocessor for CPython C files. - Its purpose is to automate all the boilerplate involved - with writing argument parsing code for "builtins", + It was introduced in Python 3.4 with :pep:`436`, + in order to provide introspection signatures, + and to generate performant and tailor-made boilerplate code + for argument parsing in CPython builtins, module level functions, and class methods. This document is divided in four major sections: @@ -44,58 +46,6 @@ Argument Clinic How-To Background ========== - -The goals of Argument Clinic ----------------------------- - -Argument Clinic's primary goal -is to take over responsibility for all argument parsing code -inside CPython. This means that, when you convert a function -to work with Argument Clinic, that function should no longer -do any of its own argument parsing?the code generated by -Argument Clinic should be a "black box" to you, where CPython -calls in at the top, and your code gets called at the bottom, -with ``PyObject *args`` (and maybe ``PyObject *kwargs``) -magically converted into the C variables and types you need. - -In order for Argument Clinic to accomplish its primary goal, -it must be easy to use. Currently, working with CPython's -argument parsing library is a chore, requiring maintaining -redundant information in a surprising number of places. -When you use Argument Clinic, you don't have to repeat yourself. - -Obviously, no one would want to use Argument Clinic unless -it's solving their problem?and without creating new problems of -its own. -So it's paramount that Argument Clinic generate correct code. -It'd be nice if the code was faster, too, but at the very least -it should not introduce a major speed regression. (Eventually Argument -Clinic *should* make a major speedup possible?we could -rewrite its code generator to produce tailor-made argument -parsing code, rather than calling the general-purpose CPython -argument parsing library. That would make for the fastest -argument parsing possible!) - -Additionally, Argument Clinic must be flexible enough to -work with any approach to argument parsing. Python has -some functions with some very strange parsing behaviors; -Argument Clinic's goal is to support all of them. - -Finally, the original motivation for Argument Clinic was -to provide introspection "signatures" for CPython builtins. -It used to be, the introspection query functions would throw -an exception if you passed in a builtin. With Argument -Clinic, that's a thing of the past! - -One idea you should keep in mind, as you work with -Argument Clinic: the more information you give it, the -better job it'll be able to do. -Argument Clinic is admittedly relatively simple right -now. But as it evolves it will get more sophisticated, -and it should be able to do many interesting and smart -things with all the information you give it. - - Basic concepts -------------- From webhook-mailer at python.org Mon Jul 31 17:43:02 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 31 Jul 2023 21:43:02 -0000 Subject: [Python-checkins] [3.12] gh-107507: Replace 'The goals of Argument Clinic' with a summary (GH-107508) (#107516) Message-ID: https://github.com/python/cpython/commit/fd6085529935afec085cea0729507ba6d4b34ab3 commit: fd6085529935afec085cea0729507ba6d4b34ab3 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-31T21:42:58Z summary: [3.12] gh-107507: Replace 'The goals of Argument Clinic' with a summary (GH-107508) (#107516) Summarise the goals of Argument Clinic in a single sentence. Mention that Argument Clinic was introduced with PEP-436. (cherry picked from commit abb71c6a8f73482c910ffdf050a86089a48e0e60) Co-authored-by: Erlend E. Aasland files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 7aafd48711b58..9c9a4f45dd0f5 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -13,8 +13,10 @@ Argument Clinic How-To .. topic:: Abstract Argument Clinic is a preprocessor for CPython C files. - Its purpose is to automate all the boilerplate involved - with writing argument parsing code for "builtins", + It was introduced in Python 3.4 with :pep:`436`, + in order to provide introspection signatures, + and to generate performant and tailor-made boilerplate code + for argument parsing in CPython builtins, module level functions, and class methods. This document is divided in four major sections: @@ -44,58 +46,6 @@ Argument Clinic How-To Background ========== - -The goals of Argument Clinic ----------------------------- - -Argument Clinic's primary goal -is to take over responsibility for all argument parsing code -inside CPython. This means that, when you convert a function -to work with Argument Clinic, that function should no longer -do any of its own argument parsing?the code generated by -Argument Clinic should be a "black box" to you, where CPython -calls in at the top, and your code gets called at the bottom, -with ``PyObject *args`` (and maybe ``PyObject *kwargs``) -magically converted into the C variables and types you need. - -In order for Argument Clinic to accomplish its primary goal, -it must be easy to use. Currently, working with CPython's -argument parsing library is a chore, requiring maintaining -redundant information in a surprising number of places. -When you use Argument Clinic, you don't have to repeat yourself. - -Obviously, no one would want to use Argument Clinic unless -it's solving their problem?and without creating new problems of -its own. -So it's paramount that Argument Clinic generate correct code. -It'd be nice if the code was faster, too, but at the very least -it should not introduce a major speed regression. (Eventually Argument -Clinic *should* make a major speedup possible?we could -rewrite its code generator to produce tailor-made argument -parsing code, rather than calling the general-purpose CPython -argument parsing library. That would make for the fastest -argument parsing possible!) - -Additionally, Argument Clinic must be flexible enough to -work with any approach to argument parsing. Python has -some functions with some very strange parsing behaviors; -Argument Clinic's goal is to support all of them. - -Finally, the original motivation for Argument Clinic was -to provide introspection "signatures" for CPython builtins. -It used to be, the introspection query functions would throw -an exception if you passed in a builtin. With Argument -Clinic, that's a thing of the past! - -One idea you should keep in mind, as you work with -Argument Clinic: the more information you give it, the -better job it'll be able to do. -Argument Clinic is admittedly relatively simple right -now. But as it evolves it will get more sophisticated, -and it should be able to do many interesting and smart -things with all the information you give it. - - Basic concepts -------------- From webhook-mailer at python.org Mon Jul 31 17:43:20 2023 From: webhook-mailer at python.org (erlend-aasland) Date: Mon, 31 Jul 2023 21:43:20 -0000 Subject: [Python-checkins] [3.11] gh-107507: Replace 'The goals of Argument Clinic' with a summary (GH-107508) (#107517) Message-ID: https://github.com/python/cpython/commit/3be07c98b3e22601a5837ce25c04803ba63a1aa6 commit: 3be07c98b3e22601a5837ce25c04803ba63a1aa6 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: erlend-aasland date: 2023-07-31T21:43:16Z summary: [3.11] gh-107507: Replace 'The goals of Argument Clinic' with a summary (GH-107508) (#107517) Summarise the goals of Argument Clinic in a single sentence. Mention that Argument Clinic was introduced with PEP-436. (cherry picked from commit abb71c6a8f73482c910ffdf050a86089a48e0e60) Co-authored-by: Erlend E. Aasland files: M Doc/howto/clinic.rst diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index 1ec8984391bbe..913a3df7e3c48 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -13,8 +13,10 @@ Argument Clinic How-To .. topic:: Abstract Argument Clinic is a preprocessor for CPython C files. - Its purpose is to automate all the boilerplate involved - with writing argument parsing code for "builtins", + It was introduced in Python 3.4 with :pep:`436`, + in order to provide introspection signatures, + and to generate performant and tailor-made boilerplate code + for argument parsing in CPython builtins, module level functions, and class methods. This document is divided in four major sections: @@ -44,58 +46,6 @@ Argument Clinic How-To Background ========== - -The goals of Argument Clinic ----------------------------- - -Argument Clinic's primary goal -is to take over responsibility for all argument parsing code -inside CPython. This means that, when you convert a function -to work with Argument Clinic, that function should no longer -do any of its own argument parsing?the code generated by -Argument Clinic should be a "black box" to you, where CPython -calls in at the top, and your code gets called at the bottom, -with ``PyObject *args`` (and maybe ``PyObject *kwargs``) -magically converted into the C variables and types you need. - -In order for Argument Clinic to accomplish its primary goal, -it must be easy to use. Currently, working with CPython's -argument parsing library is a chore, requiring maintaining -redundant information in a surprising number of places. -When you use Argument Clinic, you don't have to repeat yourself. - -Obviously, no one would want to use Argument Clinic unless -it's solving their problem?and without creating new problems of -its own. -So it's paramount that Argument Clinic generate correct code. -It'd be nice if the code was faster, too, but at the very least -it should not introduce a major speed regression. (Eventually Argument -Clinic *should* make a major speedup possible?we could -rewrite its code generator to produce tailor-made argument -parsing code, rather than calling the general-purpose CPython -argument parsing library. That would make for the fastest -argument parsing possible!) - -Additionally, Argument Clinic must be flexible enough to -work with any approach to argument parsing. Python has -some functions with some very strange parsing behaviors; -Argument Clinic's goal is to support all of them. - -Finally, the original motivation for Argument Clinic was -to provide introspection "signatures" for CPython builtins. -It used to be, the introspection query functions would throw -an exception if you passed in a builtin. With Argument -Clinic, that's a thing of the past! - -One idea you should keep in mind, as you work with -Argument Clinic: the more information you give it, the -better job it'll be able to do. -Argument Clinic is admittedly relatively simple right -now. But as it evolves it will get more sophisticated, -and it should be able to do many interesting and smart -things with all the information you give it. - - Basic concepts -------------- From webhook-mailer at python.org Mon Jul 31 19:03:06 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Mon, 31 Jul 2023 23:03:06 -0000 Subject: [Python-checkins] gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (#107403) Message-ID: https://github.com/python/cpython/commit/fb344e99aa0da5bef9318684ade69978585fe060 commit: fb344e99aa0da5bef9318684ade69978585fe060 branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-31T17:03:01-06:00 summary: gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (#107403) It was added in 3.12 for PEP 684 (per-interpreter GIL). files: M Doc/c-api/init.rst M Doc/c-api/module.rst M Doc/howto/isolating-extensions.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 60912a1ecaede..dd53fe2bc9569 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1287,6 +1287,7 @@ function. You can create and destroy them using the following functions: in any thread where the sub-interpreter is currently active. Otherwise only multi-phase init extension modules (see :pep:`489`) may be imported. + (Also see :c:macro:`Py_mod_multiple_interpreters`.) This must be ``1`` (non-zero) if :c:member:`~PyInterpreterConfig.use_main_obmalloc` is ``0``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index fa5e97846e7bc..e37505f6450a3 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -376,6 +376,37 @@ The available slot types are: If multiple ``Py_mod_exec`` slots are specified, they are processed in the order they appear in the *m_slots* array. +.. c:macro:: Py_mod_multiple_interpreters + + Specifies one of the following values: + + .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED + + The module does not support being imported in subinterpreters. + + .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_SUPPORTED + + The module supports being imported in subinterpreters, + but only when they share the main interpreter's GIL. + (See :ref:`isolating-extensions-howto`.) + + .. c:macro:: Py_MOD_PER_INTERPRETER_GIL_SUPPORTED + + The module supports being imported in subinterpreters, + even when they have their own GIL. + (See :ref:`isolating-extensions-howto`.) + + This slot determines whether or not importing this module + in a subinterpreter will fail. + + Multiple ``Py_mod_multiple_interpreters`` slots may not be specified + in one module definition. + + If ``Py_mod_multiple_interpreters`` is not specified, the import + machinery defaults to ``Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED``. + + .. versionadded:: 3.12 + See :PEP:`489` for more details on multi-phase initialization. Low-level module creation functions diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index 60854c3c034d7..2551fbe87b5c2 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -1,5 +1,7 @@ .. highlight:: c +.. _isolating-extensions-howto: + *************************** Isolating Extension Modules *************************** From webhook-mailer at python.org Mon Jul 31 19:11:19 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Mon, 31 Jul 2023 23:11:19 -0000 Subject: [Python-checkins] gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) Message-ID: https://github.com/python/cpython/commit/db361a340af3970c279908c8746a6b9ed45f47b8 commit: db361a340af3970c279908c8746a6b9ed45f47b8 branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-31T23:11:15Z summary: gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) files: M Doc/c-api/memory.rst M Doc/whatsnew/3.12.rst diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 8968b26b64320..1df8c2b911ca8 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -476,6 +476,10 @@ Customize Memory Allocators thread-safe: the :term:`GIL ` is not held when the allocator is called. + For the remaining domains, the allocator must also be thread-safe: + the allocator may be called in different interpreters that do not + share a ``GIL``. + If the new allocator is not a hook (does not call the previous allocator), the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the debug hooks on top on the new allocator. @@ -500,6 +504,8 @@ Customize Memory Allocators **must** wrap the existing allocator. Substituting the current allocator for some other arbitrary one is **not supported**. + .. versionchanged:: 3.12 + All allocators must be thread-safe. .. c:function:: void PyMem_SetupDebugHooks(void) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 0ac4a3df8fefa..a5cb325cedb22 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1880,6 +1880,13 @@ Porting to Python 3.12 * :c:func:`PyUnstable_Long_IsCompact` * :c:func:`PyUnstable_Long_CompactValue` +* Custom allocators, set via :c:func:`PyMem_SetAllocator`, are now + required to be thread-safe, regardless of memory domain. Allocators + that don't have their own state, including "hooks", are not affected. + If your custom allocator is not already thread-safe and you need + guidance then please create a new GitHub issue + and CC ``@ericsnowcurrently``. + Deprecated ---------- From webhook-mailer at python.org Mon Jul 31 19:13:59 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Mon, 31 Jul 2023 23:13:59 -0000 Subject: [Python-checkins] gh-99113: Add a What's New Entry for PEP 684 (gh-107520) Message-ID: https://github.com/python/cpython/commit/79e479c45fc63b6001b206fec832064c31fc1f11 commit: 79e479c45fc63b6001b206fec832064c31fc1f11 branch: main author: Eric Snow committer: ericsnowcurrently date: 2023-07-31T23:13:55Z summary: gh-99113: Add a What's New Entry for PEP 684 (gh-107520) files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index a5cb325cedb22..3c39476ced9f4 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -69,6 +69,10 @@ New grammar features: * :pep:`701`: Syntactic formalization of f-strings +Interpreter improvements: + +* :ref:`whatsnew312-pep684` + New typing features: * :pep:`688`: Making the buffer protocol accessible in Python @@ -276,6 +280,36 @@ The new :class:`inspect.BufferFlags` enum represents the flags that can be used to customize buffer creation. (Contributed by Jelle Zijlstra in :gh:`102500`.) +.. _whatsnew312-pep684: + +PEP 684: A Per-Interpreter GIL +------------------------------ + +Sub-interpreters may now be created with a unique GIL per interpreter. +This allows Python programs to take full advantage of multiple CPU +cores. + +Use the new :c:func:`Py_NewInterpreterFromConfig` function to +create an interpreter with its own GIL:: + + PyInterpreterConfig config = { + .check_multi_interp_extensions = 1, + .gil = PyInterpreterConfig_OWN_GIL, + }; + PyThreadState *tstate = NULL; + PyStatus status = Py_NewInterpreterFromConfig(&tstate, &config); + if (PyStatus_Exception(status)) { + return -1; + } + /* The new interpeter is now active in the current thread. */ + +For further examples how to use the C-API for sub-interpreters with a +per-interpreter GIL, see :source:`Modules/_xxsubinterpretersmodule.c`. + +A Python API is anticipated for 3.13. (See :pep:`554`.) + +(Contributed by Eric Snow in :gh:`104210`, etc.) + New Features Related to Type Hints ================================== @@ -1742,6 +1776,12 @@ New Features (Contributed by Eddie Elizondo in :gh:`84436`.) +* :pep:`684`: Added the new :c:func:`Py_NewInterpreterFromConfig` + function and :c:type:`PyInterpreterConfig`, which may be used + to create sub-interpreters with their own GILs. + (See :ref:`whatsnew312-pep684` for more info.) + (Contributed by Eric Snow in :gh:`104110`.) + * In the limited C API version 3.12, :c:func:`Py_INCREF` and :c:func:`Py_DECREF` functions are now implemented as opaque function calls to hide implementation details. From webhook-mailer at python.org Mon Jul 31 19:15:17 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Mon, 31 Jul 2023 23:15:17 -0000 Subject: [Python-checkins] [3.12] gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (GH-107403) (gh-107521) Message-ID: https://github.com/python/cpython/commit/3f6afafa7e7294786913dbecf79fe42c8b510f85 commit: 3f6afafa7e7294786913dbecf79fe42c8b510f85 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ericsnowcurrently date: 2023-07-31T23:15:13Z summary: [3.12] gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (GH-107403) (gh-107521) gh-107306: Add a Doc Entry for Py_mod_multiple_interpreters (GH-107403) It was added in 3.12 for PEP 684 (per-interpreter GIL). (cherry picked from commit fb344e99aa0da5bef9318684ade69978585fe060) Co-authored-by: Eric Snow files: M Doc/c-api/init.rst M Doc/c-api/module.rst M Doc/howto/isolating-extensions.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 52824b6d88afd..8b960b06871b1 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1555,6 +1555,7 @@ function. You can create and destroy them using the following functions: in any thread where the sub-interpreter is currently active. Otherwise only multi-phase init extension modules (see :pep:`489`) may be imported. + (Also see :c:macro:`Py_mod_multiple_interpreters`.) This must be ``1`` (non-zero) if :c:member:`~PyInterpreterConfig.use_main_obmalloc` is ``0``. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index 8ca48c852d4e6..2b6875d533c82 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -376,6 +376,37 @@ The available slot types are: If multiple ``Py_mod_exec`` slots are specified, they are processed in the order they appear in the *m_slots* array. +.. c:macro:: Py_mod_multiple_interpreters + + Specifies one of the following values: + + .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED + + The module does not support being imported in subinterpreters. + + .. c:macro:: Py_MOD_MULTIPLE_INTERPRETERS_SUPPORTED + + The module supports being imported in subinterpreters, + but only when they share the main interpreter's GIL. + (See :ref:`isolating-extensions-howto`.) + + .. c:macro:: Py_MOD_PER_INTERPRETER_GIL_SUPPORTED + + The module supports being imported in subinterpreters, + even when they have their own GIL. + (See :ref:`isolating-extensions-howto`.) + + This slot determines whether or not importing this module + in a subinterpreter will fail. + + Multiple ``Py_mod_multiple_interpreters`` slots may not be specified + in one module definition. + + If ``Py_mod_multiple_interpreters`` is not specified, the import + machinery defaults to ``Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED``. + + .. versionadded:: 3.12 + See :PEP:`489` for more details on multi-phase initialization. Low-level module creation functions diff --git a/Doc/howto/isolating-extensions.rst b/Doc/howto/isolating-extensions.rst index 60854c3c034d7..2551fbe87b5c2 100644 --- a/Doc/howto/isolating-extensions.rst +++ b/Doc/howto/isolating-extensions.rst @@ -1,5 +1,7 @@ .. highlight:: c +.. _isolating-extensions-howto: + *************************** Isolating Extension Modules *************************** From webhook-mailer at python.org Mon Jul 31 19:24:50 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Mon, 31 Jul 2023 23:24:50 -0000 Subject: [Python-checkins] [3.12] gh-99113: Add a What's New Entry for PEP 684 (gh-107520) (gh-107523) Message-ID: https://github.com/python/cpython/commit/31cd12ab21b305f19732372634cb50940f353e6e commit: 31cd12ab21b305f19732372634cb50940f353e6e branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ericsnowcurrently date: 2023-07-31T17:24:45-06:00 summary: [3.12] gh-99113: Add a What's New Entry for PEP 684 (gh-107520) (gh-107523) gh-99113: Add a What's New Entry for PEP 684 (gh-107520) (cherry picked from commit 79e479c45fc63b6001b206fec832064c31fc1f11) Co-authored-by: Eric Snow files: M Doc/whatsnew/3.12.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index d7389f696ea2e..cbf47e4ad835d 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -70,6 +70,10 @@ New grammar features: * :pep:`701`: Syntactic formalization of f-strings +Interpreter improvements: + +* :ref:`whatsnew312-pep684` + New typing features: * :pep:`688`: Making the buffer protocol accessible in Python @@ -277,6 +281,36 @@ The new :class:`inspect.BufferFlags` enum represents the flags that can be used to customize buffer creation. (Contributed by Jelle Zijlstra in :gh:`102500`.) +.. _whatsnew312-pep684: + +PEP 684: A Per-Interpreter GIL +------------------------------ + +Sub-interpreters may now be created with a unique GIL per interpreter. +This allows Python programs to take full advantage of multiple CPU +cores. + +Use the new :c:func:`Py_NewInterpreterFromConfig` function to +create an interpreter with its own GIL:: + + PyInterpreterConfig config = { + .check_multi_interp_extensions = 1, + .gil = PyInterpreterConfig_OWN_GIL, + }; + PyThreadState *tstate = NULL; + PyStatus status = Py_NewInterpreterFromConfig(&tstate, &config); + if (PyStatus_Exception(status)) { + return -1; + } + /* The new interpeter is now active in the current thread. */ + +For further examples how to use the C-API for sub-interpreters with a +per-interpreter GIL, see :source:`Modules/_xxsubinterpretersmodule.c`. + +A Python API is anticipated for 3.13. (See :pep:`554`.) + +(Contributed by Eric Snow in :gh:`104210`, etc.) + New Features Related to Type Hints ================================== @@ -1744,6 +1778,12 @@ New Features (Contributed by Eddie Elizondo in :gh:`84436`.) +* :pep:`684`: Added the new :c:func:`Py_NewInterpreterFromConfig` + function and :c:type:`PyInterpreterConfig`, which may be used + to create sub-interpreters with their own GILs. + (See :ref:`whatsnew312-pep684` for more info.) + (Contributed by Eric Snow in :gh:`104110`.) + * In the limited C API version 3.12, :c:func:`Py_INCREF` and :c:func:`Py_DECREF` functions are now implemented as opaque function calls to hide implementation details. From webhook-mailer at python.org Mon Jul 31 19:25:21 2023 From: webhook-mailer at python.org (ericsnowcurrently) Date: Mon, 31 Jul 2023 23:25:21 -0000 Subject: [Python-checkins] [3.12] gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) (gh-107522) Message-ID: https://github.com/python/cpython/commit/fc4532a55d23887bae49350d2f939c597d6b5b98 commit: fc4532a55d23887bae49350d2f939c597d6b5b98 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ericsnowcurrently date: 2023-07-31T17:25:18-06:00 summary: [3.12] gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) (gh-107522) gh-105766: Document that Custom Allocators Must Be Thread-Safe (gh-107519) (cherry picked from commit db361a340af3970c279908c8746a6b9ed45f47b8) Co-authored-by: Eric Snow files: M Doc/c-api/memory.rst M Doc/whatsnew/3.12.rst diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 8968b26b64320..1df8c2b911ca8 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -476,6 +476,10 @@ Customize Memory Allocators thread-safe: the :term:`GIL ` is not held when the allocator is called. + For the remaining domains, the allocator must also be thread-safe: + the allocator may be called in different interpreters that do not + share a ``GIL``. + If the new allocator is not a hook (does not call the previous allocator), the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the debug hooks on top on the new allocator. @@ -500,6 +504,8 @@ Customize Memory Allocators **must** wrap the existing allocator. Substituting the current allocator for some other arbitrary one is **not supported**. + .. versionchanged:: 3.12 + All allocators must be thread-safe. .. c:function:: void PyMem_SetupDebugHooks(void) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index cbf47e4ad835d..39e702bb012fb 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -1922,6 +1922,13 @@ Porting to Python 3.12 * :c:func:`PyUnstable_Long_IsCompact` * :c:func:`PyUnstable_Long_CompactValue` +* Custom allocators, set via :c:func:`PyMem_SetAllocator`, are now + required to be thread-safe, regardless of memory domain. Allocators + that don't have their own state, including "hooks", are not affected. + If your custom allocator is not already thread-safe and you need + guidance then please create a new GitHub issue + and CC ``@ericsnowcurrently``. + Deprecated ----------