dynamic import of dynamically created modules failes

[Stephan Lukits]

On 3/31/20 9:01 PM, Pieter van Oostrum wrote:
> "Dieter Maurer"<dieter at handshake.de>  writes:
>
>> Stephan Lukits wrote at 2020-3-31 17:44 +0300:
>>> background:
>>>
>>> - a daemon creates package p1 (e.g. directory with __init__.py-file) and
>>> in p1 a module m1 is created.
>>>
>>> - Then the daemon wants to import from m1, which functions (so far all
>>> the time).
>>>
>>> - Then a module m2 is created in p1 and the daemon wants to import from
>>> m2 which fails (most of the time but *not* always) with ModuleNotFoundError.
>>>
>>> See the little test script at the end which reproduces the problem.
>>> ...
>> I remember a similar report (some time ago). There, caching has
>> been responsible. I have forgotten how to invalidate the caches.
>> But, you could try logic found in
>> `importlib._bootstrap_external.PathFinder.invalidate_caches`.
> The first import creates a file __pycache__ in the directory p1.
> To remove it use rmtree(path.join(P1,'__pycache__'))
> Then the second import will succeed.


Thank you, but the behavior doesn't change.  I added


rmtree(path.join(P1, '__pycache__'))


Before the import:


try:
     from p1.m2 import hello


and still get (please note that there isn't a __pycache__ directory 
anymore):

answer=42
wd-content: __init__.py, p1
p1-content: __init__.py, __pycache__, m1.py, m2.py
p1-modlues: p1, p1.m1
p1-content after rmtree: __init__.py, m1.py, m2.py
Traceback (most recent call last):
   File "dynamic_modules/__init__.py", line 68, in <module>
     from p1.m2 import hello
ModuleNotFoundError: No module named 'p1.m2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File "dynamic_modules/__init__.py", line 71, in <module>
     hello = getattr(import_module('p1.m2'), 'hello')
   File 
"/home/goedel/.pyenv/versions/3.8.2/lib/python3.8/importlib/__init__.py", 
line 127, in import_module
     return _bootstrap._gcd_import(name[level:], package, level)
   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
   File "<frozen importlib._bootstrap>", line 973, in 
_find_and_load_unlocked
ModuleNotFoundError: No module named 'p1.m2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File "dynamic_modules/__init__.py", line 74, in <module>
     hello = getattr(__import__('p1.m2', fromlist=[None]), 'hello')
ModuleNotFoundError: No module named 'p1.m2'