[issue46110] compile("-"*3000000 + "4", '', mode) causes hard crash

[Guido van Rossum]

Guido van Rossum <guido at python.org> added the comment:

I'm still wondering why speed.python.org showed such a slowdown, and how we can revert that. Here's a new theory.

Thanks to an investigation I did together with Eric, I now suspect that the release of setuptools 60.0.0 on Dec 19 is a smoking gun. PyPerformance (re)installs the latest versions of pip and setuptools.

Setuptools 60.0 makes a change in the file distutils-precedence.pth that causes it (by default) to import something called _distutils_hack and to call its add_shim() function. In previous setuptools this was off by default, but in 60.0 it switched to on. See https://github.com/pypa/setuptools/blob/main/CHANGES.rst#v6000

That add_shim() call in turn installs an extra entry in front of sys.meta_path, which does a bunch of extra work. See https://github.com/pypa/setuptools/blob/main/_distutils_hack/__init__.py

Pablo, can we change the PyPerformance configuration or the script that runs it to set and export SETUPTOOLS_USE_DISTUTILS=stdlib, to see whether that affects perf at all?

(Note that the code in distutils-precedence.pth is executed by site.py in addpackage().)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46110>
_______________________________________