[issue36136] Windows: python._pth sets isolated mode late during Python initialization

[STINNER Victor]

New submission from STINNER Victor <vstinner at redhat.com>:

The read_pth_file() of PC/getpathp.c sets Py_IsolatedFlag and Py_NoSiteFlag to 1 in Python 3.6. calculate_path() checked if a file with a "._pth" extension exists in "dllpath" or "progpath".

I refactored deeply the Python initialization in Python 3.7 and I'm not sure I introduced a regression or not.

In Python 3.7, _PyCoreConfig_Read() calls config_init_path_config() which indirectly calls read_pth_file(). pymain_read_conf_impl() detects if Py_IsolatedFlag and Py_NoSiteFlag have been modified and store the new value in cmdline->isolated and cmdline->no_site_import.

Later, cmdline_set_global_config() sets Py_IsolatedFlag and Py_NoSiteFlag; and _PyCoreConfig_SetGlobalConfig() sets Py_IgnoreEnvironmentFlag.

The problem is the relationship between isolated/cmdline.Py_IsolatedFlag, no_site_import/cmdline.Py_NoSiteFlag and Py_IgnoreEnvironmentFlag/config.ignore_environment. The isolated mode must set Py_NoSiteFlag to 1 and Py_IgnoreEnvironmentFlag to 1.

For example, pymain_read_conf_impl() uses:

    /* Set Py_IgnoreEnvironmentFlag for Py_GETENV() */
    Py_IgnoreEnvironmentFlag = config->ignore_environment || cmdline->isolated;

But it's done before calling _PyCoreConfig_Read().

Moreover, _PyCoreConfig_Read() reads PYTHONxxx environment variables before calling indirectly read_pth_file(), and so PYTHONxxx env vars are read whereas they are supposed to be ignored.

Calling read_pth_file() earlier is challenging since it depends on configuration parameters which are before calling it.

At the end, I'm not sure if it's a real issue. I'm not sure if there is a regression in Python 3.7.

--

But the code in Python 3.8 changed a lot again: _PyCoreConfig_Read() is now responsible to read all environment variables.

In Python 3.8, read_pth_file() uses a _PyPathConfig structure to set isolated and site_import parameters. These parameters are then copied to _PyCoreConfig in _PyCoreConfig_CalculatePathConfig().

Moreover, _PyCoreConfig_Read() is more explicit with the relationship between isolated, use_environment and user_site_directory. The function *starts* with:

    if (config->isolated > 0) {
        config->use_environment = 0;
        config->user_site_directory = 0;
    }

Problems (inconsistencies) arise if isolated is set from 0 to 1 after this code.

----------
components: Interpreter Core
messages: 336760
nosy: vstinner
priority: normal
severity: normal
status: open
title: Windows: python._pth sets isolated mode late during Python initialization
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36136>
_______________________________________