Keeping Python loaded

[Mats Wichmann]

On 25 Nov 2001 20:39:10 -0500, Kragen Sitaker <kragen at canonical.org>
wrote:

:> If you merely want to spare disk acceses under Unix, a possible approach
:> might be to set the sticky bit on the Python executable.  You might need
:> to be super-user to do that:
:> 
:>         chmod 1755 /usr/bin/python
:> 
:> On the other hand, on Linux at least, the disk cache maintained by
:> the system might be helpful to the point of not even having to
:> resort to the above trick.

:If you're using a version of Unix that runs on hardware that doesn't
:support paging, such as a PDP-11, then this might help.  But I don't
:think Python runs on any such versions of Unix, so the sticky bit on
:ordinary files is purely an anachronism.

Right. The sticky bit on programs doesn't do anything any longer.

The UNIX VM system needs to be left to its own devices most of the
time for best performance, /generally/ tricks to bind some pages
permanently into memory will make performance worse rather than
better.  Text (code) pages of programs are shareable so one might
think that leaving one copy of Python running, but not doing anything,
would have sort of the same effect as the old sticky stuff.  But since
the VM system knows it can fetch those pages back from the disk if it
needs them, if that Python process is inactive it will probably have
its' pages thrown out of memory over time anyway.  Also, when you
launch a larger program, it doesn't all come into memory at once, so
any benefits from "sticky"-like behavior are likely to be small.  I
believe most of the startup time is in work the Python executable does
when it launches, not in the system reading some code off the disk.

On the other hand, some more mature UNIX VM implementations (I think
this does not include Linux, but it's hard to keep track) will hang
onto pages previously used by a particular program, and if another
copy of that program is started any of those pages that still apply
will be reclaimed.  Oops, I think I'm starting to wander...

If the problem is needing to spawning a large-ish number of Python
processes relatively rapidly then some tricks may indeed help, as
Martin mentions.  I suspect they help most because a copy of an
already-set-up process (via fork) doesn't have to go through Python's
setup time, but that's only a guess, I've never tried to time anything
like this and I bet someone has (and will probably shoot my theories
to h*ll).
Mats Wichmann