[Python-Dev] New _Py_InitializeFromConfig() function (PEP 432)

[Brett Cannon]

On Tue, 31 Jul 2018 at 15:16 Victor Stinner <vstinner at redhat.com> wrote:

> Hi,
>
> I finished my work on the _PyCoreConfig structure: it's a C structure
> in Include/pystate.h which has many fields used to configure Python
> initialization. In Python 3.6 and older, these parameters were scatted
> around the code, and it was hard to get an exhaustive list of it.
>

That's great! Thanks for doing this! I know I have always found it hard to
track down where stuff in the start-up process ultimately gets set.


>
> This work is linked to the Nick Coghlan's PEP 432 "Restructuring the
> CPython startup sequence":
> https://www.python.org/dev/peps/pep-0432/
>
> Right now, the new API is still private. Nick Coghlan splitted the
> initialization in two parts: "core" and "main". I'm not sure that this
> split is needed. We should see what to do, but it would be nice to
> make the _PyCoreConfig API public! IMHO it's way better than the old
> way to configuration Python initialization.
>
> --
>
> It is now possible to only use _PyCoreConfig to initialize Python: it
> overrides old ways to configure Python like environment variables (ex:
> PYTHONPATH), global configuration variables (ex: Py_BytesWarningFlag)
> and C functions (ex: Py_SetProgramName()).
>
> I added tests to test_embed on the different ways to configure Python
> initialization:
>
> * environment variables (ex: PYTHONPATH)
> * global configuration variables (ex: Py_BytesWarningFlag) and C
> functions (ex: Py_SetProgramName())
> * _PyCoreConfig
>
> I found and fixed many issues when writing these tests :-)
>
> Reading the current configuration, _PyCoreConfig_Read(), no longer
> changes the configuration. Now the code to read the configuration and
> the code to apply the configuration is properly separated.
>
> The work is not fully complete, there are a few remaining corner cases
> and some parameters (ex: Py_FrozenFlag) which cannot be set by
> _PyCoreConfig yet. My latest issue used to work on this API:
>
> https://bugs.python.org/issue34170
>
> I had to refactor a lot of code to implement all of that.
>
> --
>
> The problem is that Python 3.7 got the half-baked implementation, and
> it caused issues:
>
> * Calling Py_Main() after Py_Initialize() fails with a fatal error on
> Python 3.7.0
>   https://bugs.python.org/issue34008
> * PYTHONOPTIMIZE environment variable is ignored by Py_Initialize()
>    https://bugs.python.org/issue34247
>
> I fixed the first issue, I'm now working on the second one to see how
> it can be fixed. Other option would be to backport the code from
> master to the 3.7 branch, since the code in master has a way better
> design. But it requires to backport a lot of changes. I'm not sure yet
> what is the best option.
>

If it is not extremely painful to fix just the issue then I say don't
backport the whole thing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180801/a224c629/attachment.html>