[Python-Dev] A fast startup patch (was: Python startup time)

[Nathaniel Smith]

On Sat, May 5, 2018, 11:34 Toshio Kuratomi <a.badger at gmail.com> wrote:

>
>
> On Sat, May 5, 2018, 10:40 AM Eric Fahlgren <ericfahlgren at gmail.com>
> wrote:
>
>> On Sat, May 5, 2018 at 10:30 AM, Toshio Kuratomi <a.badger at gmail.com>
>> wrote:
>>
>>> On Fri, May 4, 2018, 7:00 PM Nathaniel Smith <njs at pobox.com> wrote:
>>>
>>>> What are the obstacles to including "preloaded" objects in regular .pyc
>>>> files, so that everyone can take advantage of this without rebuilding the
>>>> interpreter?
>>>>
>>>
>>> Would this make .pyc files arch specific?
>>>
>>
>> Or have parallel "pyh" (Python "heap") files, that are architecture
>> specific... (But that would cost more stat calls.)
>>
>
> I ask because arch specific byte code files are a big change in consumers
> expectations.  It's not necessarily a bad change but it should be
> communicated to downstreams so they can decide how to adjust to it.
>
> Linux distros which ship byte code files will need to build them for each
> arch, for instance.  People who ship just the byte code as an obfuscation
> of the source code will need to decide whether to ship packages for each
> arch they care about or change how they distribute.
>

That's a good point.

One way to minimize the disruption would be to include both the old and new
info in the .pyc files, so at load time if the new version is incompatible
then you can fall back on the old way, even if it's a bit slower.

I think in the vast majority of cases currently .pyc files are built on the
same architecture where they're used? Pip and Debian/Ubuntu and the
interpreter's automatic compilation-on-import all build .pyc files on the
computer where they'll be run.

It might also be worth double checking much the memory layout of these
objects even varies. Obviously it'll be different for 32- and 64-bit
systems, but beyond that, most ISAs and OSes and compilers use pretty
similar struct layout rules AFAIK... we're not talking about actual machine
code.

-n

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180505/af1e4dda/attachment.html>