[Python-Dev] Startup time

[Just van Rossum]

Neal Norwitz wrote:

> I think amny of the extra stat/open calls are due to zipimports.
> 
> I don't have python23.zip, but it's still looking for a bunch
> of extra files that can't exist (in python23.zip).  Perhaps
> if the zip file doesn't exist, we can short circuit the remaining
> calls to open()?

I think we should, although I wouldn't know off hand how to do that.

There's still some nice-to-have PEP302 stuff that remains to be
implemented, that could actually help solve this problem. Currently
there are no real importer objects for the builting import mechanisms: a
value of None for a path item in sys.path_importer_cache means: use the
builtin importer. If there _was_ a true builtin importer object, None
could mean: no importer can handle this path item, skip it. See also
python.org/sf/692884. I hope to be able to work on this before 2.3b2.

> stat64("/home/neal/local/lib/python23.zip/warnings", 0xbfffebc0) = -1
ENOENT (No such file or 
> directory)
> open("/home/neal/local/lib/python23.zip/warnings.so",
O_RDONLY|O_LARGEFILE) = -1 ENOENT (No 
> such file or directory)
> open("/home/neal/local/lib/python23.zip/warningsmodule.so",
O_RDONLY|O_LARGEFILE) = -1 ENOENT 
> (No such file or directory)
> open("/home/neal/local/lib/python23.zip/warnings.py",
O_RDONLY|O_LARGEFILE) = -1 ENOENT (No 
> such file or directory)
> open("/home/neal/local/lib/python23.zip/warnings.pyc",
O_RDONLY|O_LARGEFILE) = -1 ENOENT (No 
> such file or directory)

You could try editing site.py so it (as it used to) removes path items
that don't exist on the file system. Except this probably only helps if
you'd do this _before_ os.py is imported, as os.py pulls in quite a few
modules. Hm, chicken and egg... Or disable to the code that adds the
zipfile to sys.path in Modules/getpath.c, and compare the number of stat
calls.

Just