[New-bugs-announce] [issue34309] Embedding Python; Py_Initialize / Py_Finalize cycles

[chris]

New submission from chris <christoph.wiedemann at hs-weingarten.de>:

I'm linking an issue from numpy here: https://github.com/numpy/numpy/issues/8097

Embedding python suffers from a possibility to reliably reset the state of the python interpreter. For my use case, I noticed that when using numpy with Py_Initialize() and Py_Finalize():

Py_Initialize()
// call code importing numpy
Py_Finalize()
Py_Initialize()
// call same code again

The above code will result in a crash.

One of the comments in the referenced issue is that Py_Finalize doesn't unload loaded DLL's or shared objects. Doing that would probably fix the issues.

As of now, embedding python is fundamentally broken for applications which want to embed non-trivial scientific python scripts involving user-adapted python code, because

a) Py_Finalize cannot be used reliably
b) There is no possibility to reliably reset the python interpreter otherwise (because the sub-interpreters are also not working reliably, which is stated in the documentation)
c) manually reloading changed modules via importlib.reload is not a feasible solution

The possibility to reset an embedded python interpreter to an initial state is a strong requirement for such applications.

----------
components: Extension Modules, Interpreter Core
messages: 322876
nosy: christoph.wiedemann at hs-weingarten.de
priority: normal
severity: normal
status: open
title: Embedding Python; Py_Initialize / Py_Finalize cycles
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34309>
_______________________________________