[Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

[Stefan Behnel]

Carl Shapiro schrieb am 20.09.2018 um 20:21:
> On Wed, Sep 19, 2018 at 12:32 AM, Stefan Behnel wrote:
> 
>> Also, one thing that would be interesting to find out is whether constant
>> Python data structures can actually be pre-allocated in the data segment
>> (and I mean their object structs) . Then things like tuples of strings
>> (argument lists and what not) could be loaded and the objects quickly
>> initialised (although, is that even necessary?), rather than having to heap
>> allocate and create them. Probably something that we should try out in
>> Cython.
> 
> I might not be fully understanding the scope of your question but this
> patch does allocate constant data structures in the data segment.  We could
> be more aggressive with that but we limit our scope to what is presented to
> the un-marshaling code.

Ah, thanks, yes, it works recursively, also for tuples and code objects.
Took me a while to figure out how to open the "frozemodules.c" file, but
looking at that makes it clear. Yes, that's what I meant.


> This may be relevant to Cython, as well.

Totally. This might actually be more relevant for Cython than for CPython
in the end, because it wouldn't be limited to the stdlib and its core modules.

It's a bit more difficult for us, because this probably won't work easily
across Python releases (2.[67] and 3.[45678] for now) and also definitely
not for PyPy, but that just means some multiplication of the generated
code, and we have the dynamic part of it already. Supporting that for
Unicode strings will be fun, I'm sure. :)

Stefan