[capi-sig] Deploying embedded Python

[Jeff Rush]

Andreas Raab wrote:
> 
> 1) How to define a useful subset of the stdlib that can serve as an 
> initial basis for the installation but later allows upgrade to the 
> "full" library if desirable.

There is no formal way of doing this, although you could at least start with
the full library and remove the really exotic stuff, like that for other
platforms and audio/video/crypto packages.  One man's worthless package is
another's absolute requirement though.


> 2) How to isolate the embedded interpreter from environmental effects. I 
> have found that on occasion, the interpreter would pick up "stray" 
> installations which can cause weird problems. Which environmental 
> settings affect the startup of an embedded Python interpreter? How does 
> one work around/remove those dependencies? Is there any information 
> available about how exactly the startup works? What is being read/loaded 
> in which order etc?

If you are picking up stray installations it is probably through the
PYTHONPATH environment variable.  For a brief understanding of these, run the
command "python --help".  For your embedded world, you ought to change your
distributed code to use a different environment variable, maybe a prefix or
suffix.

A useful tool in understanding the startup sequence of Python is the "-v"
option, which will display the various imports that occur.  It stays in
effect, so it is also useful to watch the secondary imports that occur as your
program executes.

There have been several efforts in the Python community to isolate specific
interpreter instances from each other, although not in the sense of embedded
work.  Two different approaches have been taken, that of SetupTools (the base
of Python Eggs) and that of zc.buildout which is a way of bringing together
specific versions of packages into a runtime environment.  The VirtualEnv
program for Python is also good for ideas on how to do this, and is really
cool too.

The SetupTools approach is to sprinkle .pth files in the
lib/python/site-packages/ directory, thereby enabling specific packages
globally across an installation.

The zc.buildout approach is less global but gives more rigorous control, by
letting individual scripts insert specific package references during startup.
 So you end up with startup scripts like:

  import sys

  sys.path[0:0] = [
    '/home/jeff/Clients/Johns/buildout/parts/zope2/lib/python',
    '/home/jeff/Clients/Johns/buildout/parts/thirdparty-checkouts',
    ]

  _interactive = True
  if len(sys.argv) > 1:
      import getopt
      _options, _args = getopt.getopt(sys.argv[1:], 'ic:')
      _interactive = False
      for (_opt, _val) in _options:
          if _opt == '-i':
              _interactive = True
          elif _opt == '-c':
              exec _val

      if _args:
          sys.argv[:] = _args
          execfile(sys.argv[0])

  if _interactive:
      import code
      code.interact(banner="", local=globals())

that customize the environment and then run the interpreter on your choice of
Python programs.  Note that zc.buildout leaves the existing search path
intact, so that you still have the problem of things leaking in from
PYTHONPATH though.

Hope this gives you some ideas,

-Jeff