Embedded python 'scripting engine' inside Python app

[Chris Angelico]

On Sun, Nov 23, 2014 at 4:48 PM, Patrick Stinson <patrickkidd at gmail.com> wrote:
> I am writing a python app (using PyQt, but that’s not important here), and want my users to be able to write their own scripts to automate the app’s functioning using an engine API hat I expose. I have extensive experience doing this in a C++ app with the CPython api, but have no idea how to do this outside of calling exec() from with in Python :)
>
> Ideally their script would compile when the source changes and retain it’s state and respond to callbacks from the api object. It appears this won’t work with exec() because the script’s definitions and state disappear as soon as the exec() call is complete, and the script doesn’t seem to be able to access it’s own defined functions and classes.
>
> Thoughts? Fun stuff!

First off, a cautionary note: Security-wise, this is absolutely
equivalent to your users editing your source code. Be aware that
you're giving them complete control.

What you should be able to do is exec the script in a specific global
dictionary. Here's some example code (Python 3.4):

>>> script = """
def init():
    print("Initializing")

def on_some_event(status):
    trigger_some_action("Status is now "+status)
"""
>>> def trigger_some_action(msg):
    print("Action triggered.",msg)
>>> globl = {"trigger_some_action":trigger_some_action}
>>> exec(script,globl)
>>> globl["init"]()
Initializing
>>> globl["on_some_event"]("Something happened")
Action triggered. Status is now Something happened

You can provide globals like this, or you can create an importable
module for the scripts to call on. (Or both. Create a module, and
pre-import it automatically.) The script defines functions with
specific names and/or calls your functions to register hooks; you can
reach into the globals to trigger functions.

One way to handle updates to the code would be to exec it in the same
globals dictionary. That has its complexities (for instance, if you
rename a function, the old version will still exist under the old name
unless you explicitly del it), but it can be very convenient.
Alternatively, you could have the new version run in a new dictionary,
but important state can be kept in separate dictionaries. That's how I
manage things with a Pike program - all code gets reloaded cleanly,
but retained state is stored separately in a mutable object that gets
passed around.

There are several ways this sort of thing can be done. It's reasonably
easy, as long as you take a bit of care across the reload boundaries.

ChrisA