[New-bugs-announce] [issue35806] typing module adds objects to sys.modules that don't look like modules

[Eric Snow]

New submission from Eric Snow <ericsnowcurrently at gmail.com>:

tl;dr Should all objects in sys.modules look like module objects?

In #35791 Ronald described a problem with a "module" added to sys.modules that does not have all the attributes a module should have.  He also mentioned a similar problem with typing.io [1]:

    BTW. Typing.io is a namespace added to sys.modules by
    the typing module that also does not have __spec__, and
    causes similar problems. I have an simple workaround for
    that on my side.

I've verified the missing module attributes (using 3.8):

    >>> old = sorted(sys.modules)
    >>> import typing
    >>> new = sorted(sys.modules)
    >>> assert sorted(set(old) - set(new)) == []
    >>> sorted(set(new) - set(old))
    ['_collections', '_functools', '_heapq', '_locale',
     '_operator', '_sre', 'collections', 'collections.abc',
     'contextlib', 'copyreg', 'enum', 'functools', 'heapq',
     'itertools', 'keyword', 'operator', 're', 'reprlib',
     'sre_compile', 'sre_constants', 'sre_parse', 'types',
     'typing', 'typing.io', 'typing.re']
    >>> [name for name in vars(sys.modules['typing.io']) if name.startswith('__')]
    ['__module__', '__doc__', '__all__', '__dict__', '__weakref__']
    >>> [name for name in vars(sys.modules['typing.re']) if name.startswith('__')]
    ['__module__', '__doc__', '__all__', '__dict__', '__weakref__']

Per the language reference [2], modules should have the following attributes:

__name__
__loader__
__package__
__spec__

Modules imported from files also should have __file__ and __cached__.  (For the sake of completeness, packages also should have a __path__ attribute.)

As seen above, typing.io and typing.re don't have any of the import-related attributes.

So, should those two "modules" have all those attributes added?  I'm in favor of saying that every sys.modules entry must have all the appropriate import-related attributes (but doesn't have to be an actual module object).  Otherwise tools (e.g. importlib.reload(), Ronald's) making that (arguably valid) assumption break.  The right place for the change in the language reference is probably the "module cache" section. [3]  The actual entry for sys.modules [4] is probably fine as-is.


[1] https://bugs.python.org/issue35791#msg334212
[2] https://docs.python.org/3/reference/import.html#module-spec
[3] https://docs.python.org/3/reference/import.html#the-module-cache
[4] https://docs.python.org/3/library/sys.html#sys.modules

----------
components: Library (Lib)
messages: 334222
nosy: barry, brett.cannon, eric.snow, gvanrossum, ncoghlan, ronaldoussoren
priority: normal
severity: normal
stage: test needed
status: open
title: typing module adds objects to sys.modules that don't look like modules
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35806>
_______________________________________