Question on multiple Python users in one application

[Loren Wilton]

> No idea as it's still not clear what you want to accomplish exactly.
> You are providing a lot of details, just not concrete ones that show how
> things are currently done and how you want to change that by adding
> Python to the mix.

Hum, this could take a lot of pages of text to describe how the existing 
environment works in detail, and some of it is considered proprietary so I 
couldn't say anyway. But I can try to give you some general concepts. Then 
once I have a description of the existing environment, I can say how I would 
like to extend it to handle Python. Also see my reply a few moments ago to 
Chris, where he siggested how to users might interact if they were sitting 
at terminals.

Viruses are common today on commodity machines and operating systems. 
Mainframes in general avoid this by obscurity. Most people don't believe 
that mainframes still exist (this is what they have been taught in school) 
so they don't think of hacking them. Also, most people have probably never 
heard of a mainframe other than an IBM 370 with AIX, so have no idea what 
the environment looks like, so they can't hack it. Just to make life harder 
for the hackers, mainframes these days usually don't have anything 
resembling interactive shell access that you can physically reach without a 
badge to open a locked door or two. They typically don't access the internet 
directly. They are a backend database machine with a frontend web server and 
a firewall between it and the internet.

The general way that banking and airline networks work is that they are 
private networks. They may be phone lines carrying RS-232 or similar 
traffic. They may be some sort of packet network that may or may not be IP 
based, and quite possibly does not use public network wires at all. When it 
comes to how terminals access the mainframes, there is a routing program on 
the mainframe that knows the name and address of every possible terminal 
that can access the system. A transaction is usually something like a 6 or 8 
character uppercase word followed by positional character data. The routing 
program gets this message, looks up the terminal name, makes sure it is 
allwed to do this trnasaction, looks up th eprogram that will handle the 
transaction, and sends the termianl name and input data to that program. The 
program does whatever it needs to do, and sends the terminal name and a 
screenload of data back to the routing program. The routing program forwards 
the response data to the terminal.

Web sites like Orbitz have web forms that make search or transaction 
requests. The web server takes the information from the web page, transforms 
it into one of these six character (or so) transaction requests, and 
pretends it is a terminal, and sends the 'screen data' to the mainframe. The 
mainframe processes the request from the "terminal" and sends back a 
screenload of data. The reply may be more than one screen, so there may be 
more transactions back and forth to get all the data. Then the CGI program 
running on the web server formats this into the web page response.

Some mainframes may also be web servers. In this case the web server program 
on the mainframe will interact using the same text commands with the backend 
programs that handle the terminals, in many cases.

Mainframes can have interactive terminals, possibly thousands of them. This 
is rare these days except for the programming machines. But the transaction 
termainls could be used as interactive terminals, by running a transaction 
that connects them to a PYTHON application. So there could be many users 
(thousands of them) all doing independent things in Python at once. There 
could be web page transaction servers that are talking to back-end programs 
that invoke Python commands or scripts. In fact the web page handler could 
appear to be a Python interpreter running a different script for each web 
page request. There are also nightly, weekly, monthly, year-end, and ad-hoc 
reports that are run on mainframes, typically to gether from the databases. 
Any or all of these reports could be Python scripts, or could be programs 
that somehow invoked snippits of Python code.

Ok, that is a user-level operational view of a typical mainframe. Now for 
some lower level details of the particular mainframe of interest.

Our mainframe is emulated in a single Windows program. Any given instance 
can have from 1 to 32 processors, and up to about 100 GW of RAM, and tens of 
thousands of disk drives online. It has an operating system of its own that 
can run tens of thousands of independent processoes ("stacks") at the same 
time. The OS does all of the sort of scheduling, IO, and communications 
things that an OS normally does. There are hundreds of system libraries 
available that programs can use to perform common functions.

User programs don't have separate address spaces in the sense that you 
understand. Memory isn't paged. Effectively memory consists of objects that 
have handles and offsets within that handle. There are descriptors that have 
a handle and an offset and a character size, and operators that take 
descriptors and possibly lengths, and do things like add or multiply or more 
or compare data. Users create data by requesting handles from the OS, which 
allocates the memory and returns a handle. The handle is tracked with the 
owning stack, and will be deleted when the stack exits, if not depeted 
before then for some reason. Users can share data by passing descriptors 
between them. This is very similar to passing references in Python.

If I implemented Python netively in this system, it would read source from a 
file (a terminal is a file) and compile bytecode, and put it in a code file, 
marked as PythonCode, as opposed to say CobolCode. The compiled code might 
then be executed automatically, or the code file might be made permanent on 
disk, and can be run some time later. When it is run, the byte code would be 
read into memory by the OS, and the OS would call the initial instruciton in 
the program. Of course the OS and all the other programs are being 
interpreted by the VM.

When the VM sees the PythonCode, it will switch to the Python interpreter, 
and user execution will continue on the same stack, but now in Python. When 
the Pyton code exits back to the OS, we will switch to the OS code file, and 
also switch abck to the Newp interpreter. Since we exited the stack that had 
the Python local data, it will be deallocated. Since allocated arrays will 
have their "mom descriptors" in that same deallocated stack, they will also 
be deallocated, unless they are shared by some other stack that still 
exists.

But the drawback with that, other than having to write a Python interpreter 
an dget it working and support it, is that I cna't call any existing 
libraries, since none of them are compiled for my mainframe architecture. 
And likely few of them would ever be. So that isn't a good approach. But I 
can use parts of the idea.

The VM is a program running on Windows. The mainframe object code is just 
byte code. The VM has routines to handle every operator and each kind of 
Descriptor that they might validly use. One form of Descriptor is only used 
by the CALL instruction. It allows an "internal call" to a procedure that is 
part of the VM.  Assuming that the VM linked to CPython, I could use one 
form of this to pass a text line to PyRun_SimpleString. Then I cna write a 
trivial program that will read from a terminal file and pass any received 
text to SimpleString, and somehow route any response back to the terminal 
file.

If I knew I was only ever going to have one Python user at a time, this 
would be sufficient. But if I assume that I'm going to have two independent 
users calling SimpleSting (or one of the other invocation interfaces, like 
InteractiveLoop), then I need to start thinking about sandboxing, so that 
they don't accidentally trash each other's data.

I assume that it is possible to compile Python down to bytecode and save 
that in a "code file" so that it can be executed later without having to be 
re-translated. Possibly if the user had a Python script on the mainframe 
that he wanted to compile, I could pass the text to CPython, and capture the 
generated bytecode and store it as a mainframe code file, rather than on the 
PC side. Then this could be executed just like any other mainframe program. 
When the VM saw a call into a PythonCode code file, it would in some way 
pass the byte code as a string to CPython and have it interpret it. Again, 
may PythonCode files might be running at once, and unless they deliberately 
wanted to share data, data sharing would have to be avoided.

Well, that was very long winded, but hopefuly it gives you some slightidea 
of the environment and desired results.

        Loren