[New-bugs-announce] [issue42006] Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId

[Serhiy Storchaka]

New submission from Serhiy Storchaka <storchaka+cpython at gmail.com>:

There are design flaws in PyDict_GetItem(), PyDict_GetItemString() and _PyDict_GetItemId().

1. They suppress all exceptions raised internally. Most common of are MemoryError, KeyboardInterrupt and RecursionError whose raising is out of control of the programmer. When an exception is suppressed the function returns incorrect result. For example PyDict_GetItem() can return NULL for existing key if the user press Ctrl-C.

This situation seems impossible if the key and all keys in the dict are exact strings, but we cannot be sure.

2. PyDict_GetItem() preserves the current exception, and therefore can be used in handling exception but PyDict_GetItemString() and _PyDict_GetItemId() clear it if the creation of string object is failed (due to MemoryError or, less likely, UnicodeDecodeError). It can leads to very unexpected result.

Most of uses of PyDict_GetItem() etc were fixed in issue35459 and other issues. But some occurrences were left, in cases when there is no any error handling, either errors are not expected or all exceptions are suppressed in any case. It is mainly in the compiler and symtable (where we look up in just created dicts), in structseq.c and in the sys module.

The proposed PR (it can be split on several smaller PRs if needed, see also issue41993 and issue42002) removes all calls of PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId, replacing them with calls of PyDict_GetItemWithError, _PyDict_GetItemStringWithError and _PyDict_GetItemIdWithError or other functions if appropriate.

_PyDict_GetItemId is no longer used and can be removed.

----------
components: Extension Modules, Interpreter Core
messages: 378437
nosy: serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42006>
_______________________________________