Question on multiple Python users in one application

[Chris Angelico]

On Fri, Oct 7, 2016 at 11:22 AM, Loren Wilton <myspamacct at earthlink.net> wrote:
>> Ah, that probably means you want separate interpreters, then.
>
>
> I hope not, unless I can get separate simultaneous interpreters out of one
> CPython.dll in one Windows process space. I'm guessing that I can't, but
> since I really don't know what I'm doing with Python yet, maybe I'm wrong
> there.

It is possible to do that, but based on the info you give in this
post, I withdraw the recommendation for separate interpreters.

>> (for instance, the interactive
>> interpreter uses "_" in the builtins to store your last result, and if
>> you have multiple users sharing the builtins, they'd be trampling over
>> each other's underscores).
>
>
> I'm very new to Python and I suspect I don't completely understand what you
> mean by 'builtins' in this instance. Does Python have a non-module-scope
> namespace that contains things like _ ? Could you give me a pointer to any
> docs on this?

In Python, there are three [1] namespaces: function locals, module
globals, and built-ins. Function locals are exactly what you'd expect:
one invocation of a function has one set of locals, and another
invocation (incl recursion) has a completely separate set.
Module-level names are called "globals", but they're not process-wide
like in C; code in a different module has separate globals. Built-ins
are for "standard" names, like range(), str(), len(), and so on. They
are shared among all modules in a process.

[1] There's also the class namespace, used while you're building a
class. Amongst our namespaces are such diverse elements as.......

> If it does, do you have an opinion on how hard it might be to wrap it in
> some form of module (or "super module"?) scope?

I wouldn't. It's generally immutable. What I'd advise is having some
way to terminate Python and restart it (which will take care of any
serious messes), and for the rest, just leave it shared.

>> But if you accept that this is a shared
>> environment, with the consequences thereof, you could have easy and
>> convenient sharing - and might even be able to have a tiny hack around
>> the interactive interpreter to make it use a different module name,
>> instead of __main__. So you could do something like this:
>>
>> #### User "Fred" ####
>>    etc.
>>
>> So they would be *visibly* sharing state. This might be acceptable to
>> you and your users. It would be pretty easy to code up (with the
>> possible exception of the interactive mode).
>
>
> What you show in example is just about exactly what I want. Each user by
> default will have his own auto-generated (or maybe overridable by the user)
> namespace, so normally can work independently. But given that he can get the
> namespace handle for some other user (which might not be an interactive
> user, it might be a background process) then they can share data.

That would align very nicely with modules in Python.

> The worry I have is any implicitly global variables, like underscore, and
> whatever else may exist in this global namespace. I'd really want some way
> to wrap those and make them unique for each user. I'm guessing that may mean
> hacking the code for CPython. I'd rather not do that, unless people would
> like the code back so I don't have my own unique branch that would break on
> each new release. But if that is what is needed, it is certainly something I
> could do.

It's probably only the underscore, and that's only for interactive
work. Seems to me it's not going to be too big a deal.

> One concern I have is if two users both "import widget". Are they now
> sharing the widget namespace? I suspect they are, and that is probably
> undesirable. Or does Python have a hierarchical namespace concept, so that
> we would have fred.widget.ham and joe.widget.ham after fred and joe both
> import widget?

Yes, they would. An import would be used in two ways:

1) Standard library modules, or third-party extension modules. It's
99.999% likely that sharing these won't be a problem.
2) Accessing another user's namespace in order to share data. Sharing
these modules is the very mechanic of data sharing.

Your users would have to be aware that they're sharing state. Off the
top of my head, I can think of very few consequences of this, most of
them having workarounds (for instance, they'd share PRNG state, but
you can explicitly construct an RNG and have your own dedicated state;
also, the decimal module has shared contexts, so you'd have to be
careful if you ever want to change its parameters). The only real
requirement would be a Python that doesn't do the usual thing with
__main__, but instead has a special module name. This would be easy to
do for script running, but might require a little more work for
interactive mode. Worst case, you roll your own REPL using exec(). It
wouldn't be too hard.

I think this is doable.

ChrisA