dynamic import of dynamically created modules failes

[Stephan Lukits]

Hello,

background:

- a daemon creates package p1 (e.g. directory with __init__.py-file) and 
in p1 a module m1 is created.

- Then the daemon wants to import from m1, which functions (so far all 
the time).

- Then a module m2 is created in p1 and the daemon wants to import from 
m2 which fails (most of the time but *not* always) with ModuleNotFoundError.

See the little test script at the end which reproduces the problem.

The very strange thing for me is, that the import of m2 in the test 
script sometimes works.  I wonder if there is some 
package-content-caching.  And if so how to prevent or re-trigger it.

(neither a reload of p1 nor a deletion of p1 from sys.modules does the 
trick)


-----------------test-script-------------------

"""

(python --version: Python 3.8.2)

Dynamic import of dynamically created modules right after creation.
"""

from shutil import rmtree
from os import path, chdir, mkdir, listdir
from importlib import import_module, reload
import sys

# for test-package creation
P1 = 'p1'
INIT = '__init__.py'
INIT_CONTENT = """\
# -*- coding: utf-8 -*-
"""

# for first test-module creation
M1 = 'm1'
M1_CONTENT = """\
# -*- coding: utf-8 -*-

answer = 42
"""

# for second test-module creation
M2 = 'm2'
M2_CONTENT = """\
# -*- coding: utf-8 -*-

hello = 'world'
"""

chdir(path.dirname(__file__))    # make sure we are in the right directory

if path.isdir(P1):
     rmtree(P1)                   # always start off under the same 
conditions


mkdir(P1)                        # create test-package and first test-module
with open(path.join(P1, INIT), 'w') as f:
     f.write(INIT_CONTENT)
with open(path.join(P1, M1+'.py'), 'w') as f:
     f.write(M1_CONTENT)

# import from the just created module; this worked always so far
from p1.m1 import answer

print(f'{answer=}')

with open(path.join(P1, M2+'.py'), 'w') as f:
     f.write(M2_CONTENT)          # create the second test-module

# check current directory, file and module structure
print('wd-content:', ', '.join(listdir()))
print('p1-content:', ', '.join(listdir(P1)))
print('p1-modlues:', ', '.join([m for m in sys.modules if 
m.startswith('p1')]))
# reload(sys.modules['p1'])  # neither a reload
# del sys.modules['p1']      # nor a deletion of p1 does the trick

# here it most of the time fails (but NOT all the time)
# so far if it fails it fails in all three variants
# so far if it works the 'from ...'-import works already
try:
     from p1.m2 import hello
except ModuleNotFoundError:
     try:
         hello = getattr(import_module('p1.m2'), 'hello')
     except ModuleNotFoundError:
         try:
             hello = getattr(__import__('p1.m2', fromlist=[None]), 'hello')
         except ModuleNotFoundError:
             raise
         else:
             print("__import__-import worked")
     else:
         print("import_module-import worked")
else:
     print("'from ... '-import worked")

print(f'{hello=}')