[Python-ideas] PEP 554: Stdlib Module to Support Multiple Interpreters in Python Code

[Eric Snow]

Hi all,

As part of the multi-core work I'm proposing the addition of the
"interpreters" module to the stdlib.  This will expose the existing
subinterpreters C-API to Python code.  I've purposefully kept the API
simple.  Please let me know what you think.

-eric

https://www.python.org/dev/peps/pep-0554/
https://github.com/python/peps/blob/master/pep-0554.rst
https://github.com/python/cpython/pull/1748
https://github.com/python/cpython/pull/1802
https://github.com/ericsnowcurrently/cpython/tree/high-level-interpreters-module

**********************************************

PEP: 554
Title: Multiple Interpreters in the Stdlib
Author: Eric Snow <ericsnowcurrently at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2017-09-05
Python-Version: 3.7
Post-History:

Abstract
========

This proposal introduces the stdlib "interpreters" module.  It exposes
the basic functionality of subinterpreters that exists in the C-API.

Rationale
=========

Running code in multiple interpreters provides a useful level of
isolation within the same process.  This can be leveraged in number
of ways.  Furthermore, subinterpreters provide a well-defined framework
in which such isolation may extended.

CPython has supported subinterpreters, with increasing levels of
support, since version 1.5.  While the feature has the potential
to be a powerful tool, subinterpreters have suffered from neglect
because they are not available directly from Python.  Exposing the
existing functionality in the stdlib will help reverse the situation.

Proposal
========

The "interpreters" module will be added to the stdlib.  It will
provide a high-level interface to subinterpreters and wrap the low-level
"_interpreters" module.  The proposed API is inspired by the
threading module.

The module provides the following functions:

enumerate():

   Return a list of all existing interpreters.

get_current():

   Return the currently running interpreter.

get_main():

   Return the main interpreter.

create():

   Initialize a new Python interpreter and return it.  The
   interpreter will be created in the current thread and will remain
   idle until something is run in it.

The module also provides the following class:

Interpreter(id):

   id:

      The interpreter's ID (read-only).

   is_running():

      Return whether or not the interpreter is currently running.

   destroy():

      Finalize and destroy the interpreter.

   run(code):

      Run the provided Python code in the interpreter, in the current
      OS thread.  Supported code: source text.

Copyright
=========

This document has been placed in the public domain.