[Python-ideas] Isolated (?transactional) exec (?subinterpreter) calls

[anatoly techtonik]

On Fri, Jun 8, 2012 at 4:00 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> 2012/6/8 anatoly techtonik <techtonik at gmail.com>
>>
>> Optionally isolated from parent environment:
>>  - a feature to execute user script in a snapshot of current
>> environment and have
>>    a choice whenever to merge its modifications back to environment or not
>
>
> It would be a really interesting feature, but seems very difficult to
> implement.
> Do you have the slightest idea how this would work?
> What about global state, environment variables, threads, and all kinds of
> side-effects?
>
> Or are you thinking about a solution based on the multiprocessing module?

For my original user story both approaches will suffice. I've never
used multiprocessing (mostly because 2.6 only compatibility) and it
looks like it is capable to do what I want with some tweaks. But first
approach with fine-grained environment control (object space, state,
memory) will be more beneficial for Python as it can bring a nice
research methodology for interpreter improvements (and hopefully some
pictures). Two things are required:

 1. Execution rollback
 2. Scope control

Execution rollback (or transaction) can be either "save the state and
restore" or "keep track of changes and discard". On a lowest possible
level it is something like using memory copy-on-write while Python
bytecode modifies it and discarding the copied stuff in the end. Like
in OSI model for networking, this low level memory and code
abstraction is the 1st layer.

But you're absolutely right about global state, environment variables,
threads and other stuff - when we jump to a higher layer - rolling
back execution pointer to a saved checkpoint and discarding memory
will not be enough. We need to ensure that reverted operation did not
affect state of the system outside the execution scope. "Scope
control" means that every pathway when execution can alter global
state outside needs to be carefully recorded and classified. It will
then be possible to detect "escaped" transactions automatically and
detect if the operation is safe to revert or not.