[issue32030] Rewrite Py_Main()

[STINNER Victor]

New submission from STINNER Victor <victor.stinner at gmail.com>:

Python has a lot of code for its initialization. It's very hard to touch this code without risking to break something. It's hard to move code since many parts of the code are interdepent. The code rely on global "Py_xxx" configuration variables like Py_IsolateFlag (set by -I command line option).

Moreover, currently Python uses the "Python runtime" early. For example, the code to parse the -W command line option uses PyUnicode_FromWideChar() and PyList_Append(). We need a stricter separation for the code before the "Python runtime" is initialized, at least partially initialized.

Nick Coghlan and Eric Snow are already working on all these issues as part of the implementation of PEP 432. They redesigned Py_Initialize() and Py_Finalize().

I would like to finish the work on the step before: the Py_Main() function.

Attached PR is a work-in-progress to rework deeply the Py_Main() function. I have different goals:

* Enhance error handling:

  * Avoid whenever possible calls to Py_FatalError() -- currently, Py_FatalError() is still called, but at a single place
  * My patch adds missing checks on PyDict_SetItem() or PyList_Append() calls, catch errors when adding warnings options and XOptions

* Reorder code to initialize: initialize Python in the "correct" order
* Better "finalization": pymain_free() is now responsible to free memory of all data used by Py_Main(). The ownership of strings is now better defined. For example, Py_SetProgramName() memory was not released before.
* pymain_init() is now the code which must not use the Python runtime
* pymain_core() uses the Python runtime. Its code to initialize the Python runtime should be easier to follow

 
Since pymain_free() now wants to release the memory, we need to force a memory allocator for PyMem_RawMalloc(), since pymain_core() changes the memory allocator. The main() already does something similar, but with simpler code since main() is a private function, whereas Py_Main() seems to be part of the public C API!

----------
messages: 306245
nosy: haypo
priority: normal
severity: normal
status: open
title: Rewrite Py_Main()
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32030>
_______________________________________