[Python-ideas] Python probe: execute code in isolation (subinterpreter?) and get results

[anatoly techtonik]

Hi,

Is there a standard way to execute a Python code and inspect the
results without spawning an external Python process? If there is no
such way, I'd like to propose the feature, and there are two user
stories. Both are about application environment probing.


Story #1: Choosing the best library in a safe manner

Probing environment is required for Python applications to make
component selection logic explicit and less error-prone. I can tell
from my experience with Spyder IDE that startup procedure is the most
fragile part for this cross-platform application, which makes use of
optionally installed components on user system. Implicit import nature
and inability to revert import operation makes situation complicated.
Below is an example. Take a note that this is not about packaging.

Spyder IDE is a Qt application that optionally embeds IPython console.
Qt has two bindings - PyQt4 and PySide. PyQt4 binding has two APIs -
#1 and #2. If PyQt4 is used and version of installed IPython >= 0.11,
the API #2 must be chosen. So, the IPython version probing should come
first. A standard way to detect IPython version is to import IPython
before the rest of the application, but IPython may detect PyQt4
itself and import it too for probing version. And if Spyder uses
PySide we now have a conflict with Qt libraries loaded. If there was a
way to execute Python script in subinterpreter to probe all installed
component versions and return results, the selection logic would be
much more readable and sane.


Story #2: Get settings from user script

Blender uses SCons to automate builds. SCons script is written in
Python and it uses execfile(filename, globals, locals) to read
platform specific user script with settings. Unfortunately, execfile()
is a hack that doesn't treat Python scripts the same way the
interpreter treats them - for example, globals access will throw
exceptions - http://bugs.python.org/issue14049  More important that
users won't be able to troubleshoot the exceptions, because standalone
script works as expected.

Executing user script code in a subprocess will most likely negatively
affect performance, which is rather critical for a build tool.
Pickling and unpickling objects with global state through
communication pipe may also be the source of bugs. So, having a cheap
way to communicate with Python subinterpreter and get a simple dict in
result will make Python more snappy.
--
anatoly t.