[New-bugs-announce] [issue45506] Out of source tree builds failing on main - test_importlib others unreliable

[Gregory P. Smith]

New submission from Gregory P. Smith <greg at krypto.org>:

The behavior is odd, I don't really know how to characterize it other than something serious has gone wrong.  Memory corruption thus differing failures perhaps?  it seems like maybe a race condition?

on the main branch (i cannot reproduce this on 3.10 thankfully):

1) git checkout https://github.com/python/cpython.git upstream
2) mkdir b-u && cd b-u
3) ../upstream/configure --with-pydebug && make -j3 && ./python -m test.regrtest test_importlib

- sometimes it hangs forever.
- sometimes it crashes with a long list of error messages coming out of regrtest itself.  The crashes appear to happen in a child process so regrtest can continue and run other tests if you tell it to run more.  Sometimes it'll be an error about "import _frozenimport_lib as bootstrap" failing with sys.modules containing None. (Linux does that for me)  On macOS when it doesn't hang, I get a blowup from test_importlib/test_threaded_import.py.

I've reproduced this on Linux and macOS.  macOS alternates between a traceback blowup and hang with an occasional pass.  On Linux it is quite consistently a huge chain of stacktraces.

Linux:

```
0:00:00 load avg: 2.48 Run tests sequentially
0:00:00 load avg: 2.48 [1/1] test_importlib
Failed to import test module: test.test_importlib.builtin.test_finder
Traceback (most recent call last):
  File "/home/greg/oss/python/cpython/gpshead/Lib/importlib/__init__.py", line 16, in <module>
    import _frozen_importlib as _bootstrap
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: import of _frozen_importlib halted; None in sys.modules

During handling of the above exception, another exception occurred:

...  tons more  ...
```

Our buildbot fleet is not bloody looking.  Which is why I tried it on two different systems and OSes before reporting.  When I do a ./configure and run everything within the source tree it does not fail.  This is only for proper out-of-tree builds (which are what I always use - as should everybody).

Do our buildbots only do in-tree builds?

----------
components: Build
messages: 404159
nosy: gregory.p.smith
priority: high
severity: normal
stage: needs patch
status: open
title: Out of source tree builds failing on main - test_importlib others unreliable
type: behavior
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45506>
_______________________________________