Multiple interpreters retaining huge amounts of memory

[Graham Dumpleton]

On Feb 4, 10:03 am, "Martin v. Löwis" <mar... at v.loewis.de> wrote:
> >>> It means that
> >>> environment variable separation for changes made unique to a sub
> >>> interpreter is impossible.
> >> That's not really true. You can't use os.environ for that, yes.
>
> > Which bit isn't really true?
>
> The last sentence ("It means that...").
>
> > When you do:
>
> >   os.environ['XYZ'] = 'ABC'
>
> > this results in a corresponding call to:
>
> >   putenv('XYZ=ABC')
>
> Generally true, but not when you did
>
>    os.environ=dict(os.environ)
>
> Furthermore, you can make changes to environment variables
> without changing os.environ, which does allow for environment
> variable separation across subinterpreters.
>
> > As a platform provider and not the person writing the application I
> > can't really do it that way and effectively force people to change
> > there code to make it work. It also isn't just exec that is the issue,
> > as there are other system calls which can rely on the environment
> > variables.
>
> Which system calls specifically?

For a start os.system(). The call itself may not rely on environment
variables, but users can expect environment variables they set in
os.environ to be inherited by the program then are running.

There would similarly be issues with use of popen2 module
functionality because it doesn't provide means of specifying a user
specific environment and just inherits current process.

Yes you could rewrite all these with execve in some way, but as I said
it isn't something you can really enforce on someone, especially when
they might be using a third party package which is doing it and it
isn't even their own code.

> > It is also always hard when you aren't yourself having the problem and
> > you are relying on others to try and debug their problem for you. More
> > often than not the amount of information they provide isn't that good
> > and even when you ask them to try specific things for you to test out
> > ideas, they don't. So often one can never uncover the true problem,
> > and it has thus become simpler to limit the source of potential
> > problems and just tell them to avoid doing it. :-)
>
> You do notice that my comment in that direction (avoid using multiple
> interpreters) started that subthread, right :-?

I was talking about avoiding use of different versions of a C
extension module in different sub interpreters, not multiple sub
interpreters as a whole.

Graham