Inception

[Christian Gollwitzer]

Hi Denis,

Am 03.03.16 um 06:01 schrieb Denis Akhiyarov:
> Is it possible to embed CPython in CPython, e.g. using ctypes, cffi, or cython?
>

since you titled your question with a famous movie title, I take it more 
as a "philosophical" question (musing about the CPython world) than an 
actual demand. Technically, I think it cannot work with the current 
state of CPython. Embedding CPython into another application requires 
you to load libpython3.5.so (or similar depending on OS and version) and 
to call the functino sfrom within there. When a Python program runs in 
the standalone interpreter, then this DLL is already loaded, obviously, 
so the question is if the Python C API functions can be called from a 
Python script.

Possibly  ctypes or cffi can do that for you, but you would end up with 
the same interpreter that executes your main script. A DLL is loaded by 
the OS only once, which makes it possible to create plugin interfaces 
etc. using shared symbols in C.

Now, the CPython interpreter saves its state in global variables. 
Therefore only one interpreter can exist in a single process, and this 
is also the reason that multithreading is complicated and does not 
really work (GIL &c).

How useful could it be to have more than a single interpreter? For 
instance, Tcl does it in a different way. All of the interpreter state 
is stored within a Tcl_Interp struct which is passed around to every API 
function. You can create more than a single interpreter, and that even 
works from the script level using the "interp create" command. This is 
useful to shield user script evaluation from the main script. Such 
interpreters can be made "safe", i.e. to not allow file operations, or 
to limit the CPU time for script evaluation, which has its primary use 
case in things like web browser plugins or simple servers. A pythonized 
version of the API would look like this:

from pyembedding import Interp

i=Interp()
i2=Interp(safe=True)

i.eval('print("Hello world")')
# prints Hello world

i2.eval('print("Hello world")')
# raises a SafeInterpException
# I/O  is unsafe and disabled here

def greetme():
	print("Hello world")

i2.alias(greet=greetme)
i2.eval('greet')
# calls back to greetme in main interpreter

This mechanism would also allow true multithreading. If you run another 
interpreter in the second thread, it will have it's own GIL.

I'm not familiar with the internals of other Python implementation, but 
I can imagine that maybe in Jython such an extension could be written.

	Christian