[Python-Dev] Python startup time

[Neil Schemenauer]

On 2018-05-03, Lukasz Langa wrote:
> > On May 2, 2018, at 8:57 PM, INADA Naoki <songofacandy at gmail.com> wrote:
> > * Add lazy compiling API or flag in `re` module.  The pattern is compiled
> > when first used.
> 
> How about go the other way and allow compiling at Python
> *compile*-time? That would actually make things faster instead of
> just moving the time spent around.

Lisp has a special form 'eval-when'.  It can be used to cause
evaluation of the body expression at compile time.

In Carl's "A fast startup patch" post, he talks about getting rid of
the unmarshal step and storing objects in the heap segment of the
executable.  Those would be the objects necessary to evaluate code.
The marshal module has a limited number of types that it handle.
I believe they are: bool, bytes, code objects, complex, Ellipsis
float, frozenset, int, None, tuple and str.

If the same mechanism could handle more types, rather than storing
the code to be evaluated, we could store the objects created after
evaluation of the top-level module body.  Or, have a mechanism to
mark which code should be evaluated at compile time (much like the
eval-when form).

For the re.compile example, the compiled regex could be what is
stored after compiling the Python module (i.e. the re.compile gets
run at compile time).  The objects created by re.compile (e.g.
SRE_Pattern) would have to be something that the heap dumper could
handle.

Traditionally, Python has had the model "there is only runtime".
So, starting to do things at compile time complicates that model.

Regards,

  Neil