[Python-Dev] Python startup time

[Nick Coghlan]

On 20 July 2017 at 23:32, Stefan Behnel <stefan_ml at behnel.de> wrote:
> So, before considering to write an accelerator module in C that replaces
> some existing Python module, and thus duplicating its entire source code
> with highly increased complexity, I'd like to remind you that simply
> compiling the Python module itself to C should give at least reasonable
> speed-ups *without* adding to the maintenance burden, and can be done
> optionally as part of the build process. We do that for Cython itself
> during its installation, for example.

And if folks are concerned about the potential bootstrapping issues
with this approach, the gist is that it would have to look something
like this:

Phase 0: freeze importlib
- build a CPython with only builtin and frozen module support
- use it to freeze importlib

Phase 1: traditional CPython
- build the traditional Python interpreter with no Cython accelerated modules

Phase 2: accelerated CPython
- if not otherwise available, use the traditional Python interpreter
to download & install Cython in a virtual environment
- run Cython to selectively precompile key modules (such as those
implicitly imported at startup)

Technically, phase 2 doesn't actually *change* CPython itself, since
the import system is already setup such that if an extension module
and a source module are side-by-side in the same directory, then the
extension module will take precedence. As a result, precompiling with
Cython is similar in many ways to precompiling to bytecode, its just
that the result is native machine code with Python C API calls, rather
than CPython bytecode.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia