[Python-Dev] performance delta with .py presence v.s. only pyc on python 2.7.x?

[Brett Cannon]

On Tue Oct 07 2014 at 2:24:52 PM Skip Montanaro <skip.montanaro at gmail.com>
wrote:

> On Tue, Oct 7, 2014 at 12:46 PM, John Smith <sansprivacy at gmail.com> wrote:
> > pyc-only install sees mediocre performance. (pyc's are built using
> > compileall.py, then source .py's removed before packaging)
>
> (Warning: it's been probably a decade since I looked at any of this
> stuff, so treat this response as mere conjecture.)
>
> I'd take a look at startup time and things like stat(2) calls in the
> presence or absence of .py files. It's possible that it tries all
> other possible file extensions before considering .pyc. If you have a
> long sys.path, it would then run through all the other file extension
> possibilities before trying the .pyc. OTOH, if the .py is present, it
> might be found early in the search, then as an optimization, look for
> a .pyc file it can use rather than compiling the .py file. How long is
> sys.path?
>

The extension check is per sys.path entry so sys.path is the outer loop,
not the file extension list. The relevant code is all in
https://hg.python.org/cpython/file/05f70805f37f/Python/import.c for Python
2.7  and the search code is
https://hg.python.org/cpython/file/05f70805f37f/Python/import.c#l1291 .

But with the code being a black box there is no good way to answer this
question. E.g. if they have a custom finder that is very costly when there
is no source then that could explain this. But you're talking **app**
performance and not import performance, so either something on your system
or in that code is very quirky that is leading to an actual performance
loss to that level (import costs are usually washed out and bytecode is
literally just the internal representation of source after compilation so
there is no semantic difference at execution time if the same source is
used).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141007/039c3521/attachment.html>