Python does not play well with others

[Graham Dumpleton]

On Feb 6, 9:15 am, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
> "Graham Dumpleton" <grah... at dscpl.com.au> writes:
> > Yes, these per VirtualHost interpreter instances will only be created
> > on demand in the child process when a request arrives which
> > necessitates it be created and so there is some first time setup for
> > that specific interpreter instance at that point, but the main Python
> > initialisation has already occurred so this is minor.
>
> Well ok, but what if each of those interpreters wants to load, say,
> the cookie module?  Do you have separate copies of the cookie module
> in each interpreter?  Does each one take the overhead of loading the
> cookie module?

Each interpreter instance will have its own copy of any Python based
code modules. You can't avoid this as Python code is so modifiable
that they have to be separate else you would be modifying the same
instance as used by a different interpreter which could screw up the
other applications view of the world. The whole point of having
separate interpreters is to avoid applications trampling on each
other. If you really are concerned about multiple loading, use the
PythonInterpreter directive to specifically say that applications
running under different VirtualHost containers should use the same
interpreter.

Note though, that although you can run multiple applications in one
interpreter in many cases, it may not be able to be done in others.
For example, it is not possible to run two instances of Django within
the one interpreter instance. The first reason as to why this can't be
done is that Django expects certain information about its
configuration to come from os.environ. Since there is only one
os.environ it can't have two different values for each application at
the same time. Some may argue that in 'prefork' you could just change
os.environ to be correct for the application for the current request
and this effectively is what the mod_python adapter for Django does,
but this will fail when 'worker' MPM or Windows is used. I suspect
this is the where the idea that Django can't be run on 'worker' MPM
came from. Although the documentation for Django suggests it is a
mod_python problem, it is actually a Django problem. This use of
os.environ by Django also means that Django isn't a well behaved WSGI
application component. :-(

> It would be neat if there was a way of including
> frequently used modules in the shared text segment of the
> interpreters, as created during the initial build process.  GNU Emacs
> used to do something like that with a contraption called "unexec" (it
> could dump out parts of its data segment into a pure (shared)
> executable that you could then run without the overhead of loading all
> those modules) but the capability went away as computers got faster
> and it became less common to have a lot of Emacs instances weighing
> down timesharing systems.  Maybe it's time for a revival of those
> techniques.

I don't see it as being applicable. Do note that provided there are
precompiled byte code files for .py files then load time is at least
reduced because Python doesn't have to recompile the code. This
actually can be quite significant.

Graham