[New-bugs-announce] [issue46399] Addition of `mapping` attribute to dict views classes has inadvertently broken type-checkers
Alex Waygood
report at bugs.python.org
Sun Jan 16 08:55:19 EST 2022
New submission from Alex Waygood <Alex.Waygood at Gmail.com>:
Issue40890 added a new `.mapping` attribute to dict_keys, dict_values and dict_items in 3.10. This addition is great for introspection. However, it has inadvertently caused some unexpected problems for type-checkers.
Prior to Issue40890, the typeshed stub for the three `dict` methods was something like this:
```
from typing import MutableMapping, KeysView, ItemsView, ValuesView, TypeVar
_KT = TypeVar("_KT")
_VT = TypeVar("_VT")
class dict(MutableMapping[_KT, _VT]):
def keys(self) -> KeysView[_KT]: ...
def values(self) -> ValuesView[_VT]: ...
def items(self) -> ItemsView[_KT, _VT]: ...
```
In other words, typeshed did not acknowledge the existence of a "dict_keys" class at all. Instead, it viewed the precise class as an implementation detail, and merely stated in the stub that `dict.keys()` would return some class that implemented the `KeysView` interface.
After Issue40890, however, it was clear that this approach would no longer suffice, as mypy (and other type-checkers) would yield false-positive errors for the following code:
```
m = dict().keys().mapping
```
Following several PRs, the typeshed stub for these `dict` methods now looks something like this:
```
# _collections_abc.pyi
import sys
from types import MappingProxyType
from typing import Generic, KeysView, ValuesView, ItemsView, TypeVar, final
_KT_co = TypeVar("_KT_co", covariant=True)
_VT_co = TypeVar("_VT_co", covariant=True)
@final
class dict_keys(KeysView[_KT_co], Generic[_KT_co, _VT_co]):
if sys.version_info >= (3, 10):
mapping: MappingProxyType[_KT_co, _VT_co]
@final
class dict_values(ValuesView[_VT_co], Generic[_KT_co, _VT_co]):
if sys.version_info >= (3, 10):
mapping: MappingProxyType[_KT_co, _VT_co]
@final
class dict_items(ItemsView[_KT_co, _VT_co], Generic[_KT_co, _VT_co]):
if sys.version_info >= (3, 10):
mapping: MappingProxyType[_KT_co, _VT_co]
# builtins.pyi
from _collections_abc import dict_keys, dict_views, dict_items
from typing import MutableMapping, TypeVar
_KT = TypeVar("_KT")
_VT = TypeVar("_VT")
class dict(MutableMapping[_KT, _VT]):
def keys(self) -> dict_keys[KT, VT]: ...
def values(self) -> dict_values[_KT, _VT]: ...
def items(self) -> dict_items[_KT, _VT]: ...
```
The alteration to the typeshed stub means that mypy will no longer give false-positive errors for code in which a user attempts to access the `mapping` attribute. However, it has serious downsides. Users wanting to create typed subclasses of `dict` have found that they are no longer able to annotate `.keys()`, `.values()` and `.items()` as returning `KeysView`, `ValuesView` and `ItemsView`, as mypy now flags this as an incompatible override. Instead, they now have to import `dict_keys`, `dict_values` and `dict_items` from `_collections_abc`, a private module. Moreover, they are unable to parameterise these classes at runtime.
In other words, you used to be able to do this:
```
from typing import KeysView, TypeVar
K = TypeVar("K")
V = TypeVar("V")
class DictSubclass(dict[K, V]):
def keys(self) -> KeysView[K]:
return super().keys()
```
But now, you have to do this:
```
from _collections_abc import dict_keys
from typing import TypeVar
K = TypeVar("K")
V = TypeVar("V")
class DictSubclass(dict[K, V]):
def keys(self) -> "dict_keys[K, V]":
return super().keys()
```
References:
* PR where `.mapping` attribute was added to the typeshed stubs: https://github.com/python/typeshed/pull/6039
* typeshed issue where this was recently raised: https://github.com/python/typeshed/issues/6837
* typeshed PR where this was further discussed: https://github.com/python/typeshed/pull/6888
----------
components: Library (Lib)
keywords: 3.10regression
messages: 410695
nosy: AlexWaygood, Dennis Sweeney, Jelle Zijlstra, gvanrossum, kj, rhettinger, serhiy.storchaka, sobolevn
priority: normal
severity: normal
status: open
title: Addition of `mapping` attribute to dict views classes has inadvertently broken type-checkers
type: behavior
versions: Python 3.10, Python 3.11
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46399>
_______________________________________
More information about the New-bugs-announce
mailing list