[Python-Dev] shared data (was: Some thoughts on the codecs...)

[Guido van Rossum]

> >>>>> "AMK" == Andrew M Kuchling <akuchlin@mems-exchange.org> writes:
> 
>     AMK> I don't see how this is fixable, unless we strip down
>     AMK> site.py, which drags in os, which drags in os.path and stat
>     AMK> and UserDict.
> 
> One approach might be to support loading modules out of jar files (or
> whatever) using Greg imputils.  We could put the bootstrap .pyc files
> in this jar and teach Python to import from it first.  Python
> installations could even craft their own modules.jar file to include
> whatever modules they are willing to "hard code".  This, with -S might
> make Python start up much faster, at the small cost of some
> flexibility (which could be regained with a c.l. switch or other
> mechanism to bypass modules.jar).

A completely different approach (which, incidentally, HP has lobbied
for before; and which has been implemented by Sjoerd Mullender for one
particular application) would be to cache a mapping from module names
to filenames in a dbm file.  For Sjoerd's app (which imported hundreds
of modules) this made a huge difference.  The problem is that it's
hard to deal with issues like updating the cache while sharing it with
other processes and even other users...  But if those can be solved,
this could greatly reduce the number of stats and unsuccessful opens,
without having to resort to jar files.

--Guido van Rossum (home page: http://www.python.org/~guido/)