[Python-checkins] cpython: Changes pyvenv.cfg trick into an actual sys.path file.
steve.dower
python-checkins at python.org
Fri Sep 9 12:23:11 EDT 2016
https://hg.python.org/cpython/rev/03517dd54977
changeset: 103408:03517dd54977
parent: 103406:fdb3537e633a
user: Steve Dower <steve.dower at microsoft.com>
date: Fri Sep 09 09:17:35 2016 -0700
summary:
Changes pyvenv.cfg trick into an actual sys.path file.
files:
Doc/using/windows.rst | 53 ++++---
Lib/site.py | 6 -
PC/getpathp.c | 181 +++++++++++++++---------
PCbuild/pythoncore.vcxproj | 2 +-
Tools/msi/make_zip.py | 11 +-
5 files changed, 154 insertions(+), 99 deletions(-)
diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst
--- a/Doc/using/windows.rst
+++ b/Doc/using/windows.rst
@@ -29,13 +29,13 @@
As specified in :pep:`11`, a Python release only supports a Windows platform
while Microsoft considers the platform under extended support. This means that
-Python 3.5 supports Windows Vista and newer. If you require Windows XP support
+Python 3.6 supports Windows Vista and newer. If you require Windows XP support
then please install Python 3.4.
Installation Steps
------------------
-Four Python 3.5 installers are available for download - two each for the 32-bit
+Four Python 3.6 installers are available for download - two each for the 32-bit
and 64-bit versions of the interpreter. The *web installer* is a small initial
download, and it will automatically download the required components as
necessary. The *offline installer* includes the components necessary for a
@@ -193,13 +193,13 @@
For example, to silently install a default, system-wide Python installation,
you could use the following command (from an elevated command prompt)::
- python-3.5.0.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0
+ python-3.6.0.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0
To allow users to easily install a personal copy of Python without the test
suite, you could provide a shortcut with the following command. This will
display a simplified initial page and disallow customization::
- python-3.5.0.exe InstallAllUsers=0 Include_launcher=0 Include_test=0
+ python-3.6.0.exe InstallAllUsers=0 Include_launcher=0 Include_test=0
SimpleInstall=1 SimpleInstallDescription="Just for me, no test suite."
(Note that omitting the launcher also omits file associations, and is only
@@ -234,13 +234,13 @@
useful to have a locally cached copy.
Execute the following command from Command Prompt to download all possible
-required files. Remember to substitute ``python-3.5.0.exe`` for the actual
+required files. Remember to substitute ``python-3.6.0.exe`` for the actual
name of your installer, and to create layouts in their own directories to
avoid collisions between files with the same name.
::
- python-3.5.0.exe /layout [optional target directory]
+ python-3.6.0.exe /layout [optional target directory]
You may also specify the ``/quiet`` option to hide the progress display.
@@ -345,7 +345,7 @@
To temporarily set environment variables, open Command Prompt and use the
:command:`set` command::
- C:\>set PATH=C:\Program Files\Python 3.5;%PATH%
+ C:\>set PATH=C:\Program Files\Python 3.6;%PATH%
C:\>set PYTHONPATH=%PYTHONPATH%;C:\My_python_lib
C:\>python
@@ -401,10 +401,10 @@
Besides using the automatically created start menu entry for the Python
interpreter, you might want to start Python in the command prompt. The
-installer for Python 3.5 and later has an option to set that up for you.
+installer for Python 3.6 has an option to set that up for you.
-On the first page of the installer, an option labelled "Add Python 3.5 to
-PATH" can be selected to have the installer add the install location into the
+On the first page of the installer, an option labelled "Add Python to PATH"
+may be selected to have the installer add the install location into the
:envvar:`PATH`. The location of the :file:`Scripts\\` folder is also added.
This allows you to type :command:`python` to run the interpreter, and
:command:`pip` for the package installer. Thus, you can also execute your
@@ -418,7 +418,7 @@
example variable could look like this (assuming the first two entries already
existed)::
- C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files\Python 3.5
+ C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files\Python 3.6
.. _launcher:
@@ -720,7 +720,15 @@
:file:`C:\\Python\\Lib\\` and third-party modules should be stored in
:file:`C:\\Python\\Lib\\site-packages\\`.
-This is how :data:`sys.path` is populated on Windows:
+To completely override :data:`sys.path`, create a text file named ``'sys.path'``
+containing a list of paths alongside the Python executable. This will ignore all
+registry settings and environment variables, enable isolated mode, disable
+importing :mod:`site`, and fill :data:`sys.path` with exactly the paths listed
+in the file. Paths may be absolute or relative to the directory containing the
+file.
+
+When the ``'sys.path'`` file is missing, this is how :data:`sys.path` is
+populated on Windows:
* An empty entry is added at the start, which corresponds to the current
directory.
@@ -755,10 +763,6 @@
path is used instead of the path to the main executable when deducing the
home location.
-* If ``applocal`` is set to true, the ``home`` property or the main executable
- is always used as the home path, and all environment variables or registry
- values affecting the path are ignored. The landmark file is not checked.
-
The end result of all this is:
* When running :file:`python.exe`, or any other .exe in the main Python
@@ -777,13 +781,11 @@
For those who want to bundle Python into their application or distribution, the
following advice will prevent conflicts with other installations:
-* Include a ``pyvenv.cfg`` file alongside your executable containing
- ``applocal = true``. This will ensure that your own directory will be used to
- resolve paths even if you have included the standard library in a ZIP file.
- It will also ignore user site-packages and other paths listed in the
- registry.
+* Include a ``sys.path`` file alongside your executable containing the
+ directories to include. This will ignore user site-packages and other paths
+ listed in the registry or in environment variables.
-* If you are loading :file:`python3.dll` or :file:`python35.dll` in your own
+* If you are loading :file:`python3.dll` or :file:`python36.dll` in your own
executable, explicitly call :c:func:`Py_SetPath` or (at least)
:c:func:`Py_SetProgramName` before :c:func:`Py_Initialize`.
@@ -801,6 +803,11 @@
the first suggestion is the best, as the other may still be susceptible to
non-standard paths in the registry and user site-packages.
+.. versionchanged:: 3.6
+
+ Adds ``sys.path`` file support and removes ``applocal`` option from
+ ``pyvenv.cfg``.
+
Additional modules
==================
@@ -900,7 +907,7 @@
When extracted, the embedded distribution is (almost) fully isolated from the
user's system, including environment variables, system registry settings, and
installed packages. The standard library is included as pre-compiled and
-optimized ``.pyc`` files in a ZIP, and ``python3.dll``, ``python35.dll``,
+optimized ``.pyc`` files in a ZIP, and ``python3.dll``, ``python36.dll``,
``python.exe`` and ``pythonw.exe`` are all provided. Tcl/tk (including all
dependants, such as Idle), pip and the Python documentation are not included.
diff --git a/Lib/site.py b/Lib/site.py
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -463,12 +463,6 @@
system_site = value.lower()
elif key == 'home':
sys._home = value
- elif key == 'applocal' and value.lower() == 'true':
- # App-local installs use the exe_dir as prefix,
- # not one level higher, and do not use system
- # site packages.
- site_prefix = exe_dir
- system_site = 'false'
sys.prefix = sys.exec_prefix = site_prefix
diff --git a/PC/getpathp.c b/PC/getpathp.c
--- a/PC/getpathp.c
+++ b/PC/getpathp.c
@@ -6,7 +6,8 @@
PATH RULES FOR WINDOWS:
This describes how sys.path is formed on Windows. It describes the
functionality, not the implementation (ie, the order in which these
- are actually fetched is different)
+ are actually fetched is different). The presence of a sys.path file
+ alongside the program overrides these rules - see below.
* Python always adds an empty entry at the start, which corresponds
to the current directory.
@@ -36,6 +37,12 @@
used (eg. .\Lib;.\plat-win, etc)
+ If a sys.path file exists adjacent to python.exe, it must contain a
+ list of paths to add to sys.path, one per line (like a .pth file but without
+ the ability to execute arbitrary code). Each path is relative to the
+ directory containing the file. No other paths are added to the search path,
+ and the registry finder is not enabled.
+
The end result of all this is:
* When running python.exe, or any other .exe in the main Python directory
(either an installed version, or directly from the PCbuild directory),
@@ -52,7 +59,10 @@
some default, but relative, paths.
* An embedding application can use Py_SetPath() to override all of
- these authomatic path computations.
+ these automatic path computations.
+
+ * An isolation install of Python can disable all implicit paths by
+ providing a sys.path file.
---------------------------------------------------------------- */
@@ -61,9 +71,12 @@
#include "osdefs.h"
#include <wchar.h>
-#ifdef MS_WINDOWS
+#ifndef MS_WINDOWS
+#error getpathp.c should only be built on Windows
+#endif
+
#include <windows.h>
-#endif
+#include <Shlwapi.h>
#ifdef HAVE_SYS_TYPES_H
#include <sys/types.h>
@@ -163,24 +176,30 @@
than MAXPATHLEN characters at exit. If stuff is too long, only as much of
stuff as fits will be appended.
*/
+
+static int _PathCchCombineEx_Initialized = 0;
+typedef HRESULT(__stdcall *PPathCchCombineEx)(PWSTR pszPathOut, size_t cchPathOut, PCWSTR pszPathIn, PCWSTR pszMore, unsigned long dwFlags);
+static PPathCchCombineEx _PathCchCombineEx;
+
static void
join(wchar_t *buffer, const wchar_t *stuff)
{
- size_t n;
- if (is_sep(stuff[0]) ||
- (wcsnlen_s(stuff, 4) >= 3 && stuff[1] == ':' && is_sep(stuff[2]))) {
- if (wcscpy_s(buffer, MAXPATHLEN+1, stuff) != 0)
- Py_FatalError("buffer overflow in getpathp.c's join()");
- return;
+ if (_PathCchCombineEx_Initialized == 0) {
+ HMODULE pathapi = LoadLibraryW(L"api-ms-win-core-path-l1-1-0.dll");
+ if (pathapi)
+ _PathCchCombineEx = (PPathCchCombineEx)GetProcAddress(pathapi, "PathCchCombineEx");
+ else
+ _PathCchCombineEx = NULL;
+ _PathCchCombineEx_Initialized = 1;
}
- n = wcsnlen_s(buffer, MAXPATHLEN+1);
- if (n > 0 && !is_sep(buffer[n - 1]) && n < MAXPATHLEN) {
- buffer[n] = SEP;
- buffer[n + 1] = '\0';
+ if (_PathCchCombineEx) {
+ if (FAILED(_PathCchCombineEx(buffer, MAXPATHLEN+1, buffer, stuff, 0)))
+ Py_FatalError("buffer overflow in getpathp.c's join()");
+ } else {
+ if (!PathCombineW(buffer, NULL, stuff))
+ Py_FatalError("buffer overflow in getpathp.c's join()");
}
- if (wcscat_s(buffer, MAXPATHLEN+1, stuff) != 0)
- Py_FatalError("buffer overflow in getpathp.c's join()");
}
/* gotlandmark only called by search_for_prefix, which ensures
@@ -214,7 +233,6 @@
return 0;
}
-#ifdef MS_WINDOWS
#ifdef Py_ENABLE_SHARED
/* a string loaded from the DLL at startup.*/
@@ -369,7 +387,6 @@
return retval;
}
#endif /* Py_ENABLE_SHARED */
-#endif /* MS_WINDOWS */
static void
get_progpath(void)
@@ -378,7 +395,6 @@
wchar_t *path = _wgetenv(L"PATH");
wchar_t *prog = Py_GetProgramName();
-#ifdef MS_WINDOWS
#ifdef Py_ENABLE_SHARED
extern HANDLE PyWin_DLLhModule;
/* static init of progpath ensures final char remains \0 */
@@ -390,7 +406,6 @@
#endif
if (GetModuleFileNameW(NULL, progpath, MAXPATHLEN))
return;
-#endif
if (prog == NULL || *prog == '\0')
prog = L"python";
@@ -483,6 +498,67 @@
return result;
}
+static int
+read_sys_path_file(const wchar_t *path, const wchar_t *prefix)
+{
+ FILE *sp_file = _Py_wfopen(path, L"r");
+ if (sp_file == NULL)
+ return -1;
+
+ size_t bufsiz = MAXPATHLEN;
+ size_t prefixlen = wcslen(prefix);
+
+ wchar_t *buf = (wchar_t*)PyMem_RawMalloc(bufsiz * sizeof(wchar_t));
+ buf[0] = '\0';
+
+ while (!feof(sp_file)) {
+ char line[MAXPATHLEN + 1];
+ char *p = fgets(line, MAXPATHLEN + 1, sp_file);
+ if (!p)
+ break;
+
+ DWORD n = strlen(line);
+ if (n == 0 || p[n - 1] != '\n')
+ break;
+ if (n > 2 && p[n - 1] == '\r')
+ --n;
+
+ DWORD wn = MultiByteToWideChar(CP_UTF8, 0, line, n - 1, NULL, 0);
+ wchar_t *wline = (wchar_t*)PyMem_RawMalloc((wn + 1) * sizeof(wchar_t));
+ wn = MultiByteToWideChar(CP_UTF8, 0, line, n - 1, wline, wn);
+ wline[wn] = '\0';
+
+ while (wn + prefixlen + 4 > bufsiz) {
+ bufsiz += MAXPATHLEN;
+ buf = (wchar_t*)PyMem_RawRealloc(buf, (bufsiz + 1) * sizeof(wchar_t));
+ if (!buf) {
+ PyMem_RawFree(wline);
+ goto error;
+ }
+ }
+
+ if (buf[0])
+ wcscat_s(buf, bufsiz, L";");
+ wchar_t *b = &buf[wcslen(buf)];
+
+ wcscat_s(buf, bufsiz, prefix);
+ join(b, wline);
+
+ PyMem_RawFree(wline);
+ }
+
+ module_search_path = buf;
+
+ fclose(sp_file);
+ return 0;
+
+error:
+ PyMem_RawFree(buf);
+ fclose(sp_file);
+ return -1;
+}
+
+
static void
calculate_path(void)
{
@@ -492,32 +568,34 @@
wchar_t *pythonhome = Py_GetPythonHome();
wchar_t *envpath = NULL;
-#ifdef MS_WINDOWS
int skiphome, skipdefault;
wchar_t *machinepath = NULL;
wchar_t *userpath = NULL;
wchar_t zip_path[MAXPATHLEN+1];
- int applocal = 0;
if (!Py_IgnoreEnvironmentFlag) {
envpath = _wgetenv(L"PYTHONPATH");
}
-#else
- char *_envpath = Py_GETENV("PYTHONPATH");
- wchar_t wenvpath[MAXPATHLEN+1];
- if (_envpath) {
- size_t r = mbstowcs(wenvpath, _envpath, MAXPATHLEN+1);
- envpath = wenvpath;
- if (r == (size_t)-1 || r >= MAXPATHLEN)
- envpath = NULL;
- }
-#endif
get_progpath();
/* progpath guaranteed \0 terminated in MAXPATH+1 bytes. */
wcscpy_s(argv0_path, MAXPATHLEN+1, progpath);
reduce(argv0_path);
+ /* Search for a sys.path file */
+ {
+ wchar_t spbuffer[MAXPATHLEN+1];
+
+ wcscpy_s(spbuffer, MAXPATHLEN+1, argv0_path);
+ join(spbuffer, L"sys.path");
+ if (exists(spbuffer) && read_sys_path_file(spbuffer, argv0_path) == 0) {
+ wcscpy_s(prefix, MAXPATHLEN + 1, argv0_path);
+ Py_IsolatedFlag = 1;
+ Py_NoSiteFlag = 1;
+ return;
+ }
+ }
+
/* Search for an environment configuration file, first in the
executable's directory and then in the parent directory.
If found, open it for use when searching for prefixes.
@@ -543,17 +621,6 @@
}
}
if (env_file != NULL) {
- /* Look for an 'applocal' variable and, if true, ignore all registry
- * keys and environment variables, but retain the default paths
- * (DLLs, Lib) and the zip file. Setting pythonhome here suppresses
- * the search for LANDMARK below and overrides %PYTHONHOME%.
- */
- if (find_env_config_value(env_file, L"applocal", tmpbuffer) &&
- (applocal = (wcsicmp(tmpbuffer, L"true") == 0))) {
- envpath = NULL;
- pythonhome = argv0_path;
- }
-
/* Look for a 'home' variable and set argv0_path to it, if found */
if (find_env_config_value(env_file, L"home", tmpbuffer)) {
wcscpy_s(argv0_path, MAXPATHLEN+1, tmpbuffer);
@@ -576,7 +643,6 @@
envpath = NULL;
-#ifdef MS_WINDOWS
/* Calculate zip archive path from DLL or exe path */
if (wcscpy_s(zip_path, MAXPATHLEN+1, dllpath[0] ? dllpath : progpath))
/* exceeded buffer length - ignore zip_path */
@@ -590,16 +656,13 @@
skiphome = pythonhome==NULL ? 0 : 1;
#ifdef Py_ENABLE_SHARED
- if (!applocal) {
- machinepath = getpythonregpath(HKEY_LOCAL_MACHINE, skiphome);
- userpath = getpythonregpath(HKEY_CURRENT_USER, skiphome);
- }
+ machinepath = getpythonregpath(HKEY_LOCAL_MACHINE, skiphome);
+ userpath = getpythonregpath(HKEY_CURRENT_USER, skiphome);
#endif
/* We only use the default relative PYTHONPATH if we havent
anything better to use! */
skipdefault = envpath!=NULL || pythonhome!=NULL || \
machinepath!=NULL || userpath!=NULL;
-#endif
/* We need to construct a path from the following parts.
(1) the PYTHONPATH environment variable, if set;
@@ -612,7 +675,6 @@
Extra rules:
- If PYTHONHOME is set (in any way) item (3) is ignored.
- If registry values are used, (4) and (5) are ignored.
- - If applocal is set, (1), (3), and registry values are ignored
*/
/* Calculate size of return buffer */
@@ -629,13 +691,11 @@
bufsz = 0;
bufsz += wcslen(PYTHONPATH) + 1;
bufsz += wcslen(argv0_path) + 1;
-#ifdef MS_WINDOWS
- if (!applocal && userpath)
+ if (userpath)
bufsz += wcslen(userpath) + 1;
- if (!applocal && machinepath)
+ if (machinepath)
bufsz += wcslen(machinepath) + 1;
bufsz += wcslen(zip_path) + 1;
-#endif
if (envpath != NULL)
bufsz += wcslen(envpath) + 1;
@@ -651,10 +711,8 @@
fprintf(stderr, "Using default static path.\n");
module_search_path = PYTHONPATH;
}
-#ifdef MS_WINDOWS
PyMem_RawFree(machinepath);
PyMem_RawFree(userpath);
-#endif /* MS_WINDOWS */
return;
}
@@ -664,7 +722,6 @@
buf = wcschr(buf, L'\0');
*buf++ = DELIM;
}
-#ifdef MS_WINDOWS
if (zip_path[0]) {
if (wcscpy_s(buf, bufsz - (buf - module_search_path), zip_path))
Py_FatalError("buffer overflow in getpathp.c's calculate_path()");
@@ -692,15 +749,7 @@
buf = wcschr(buf, L'\0');
*buf++ = DELIM;
}
- }
-#else
- if (pythonhome == NULL) {
- wcscpy(buf, PYTHONPATH);
- buf = wcschr(buf, L'\0');
- *buf++ = DELIM;
- }
-#endif /* MS_WINDOWS */
- else {
+ } else {
wchar_t *p = PYTHONPATH;
wchar_t *q;
size_t n;
diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj
--- a/PCbuild/pythoncore.vcxproj
+++ b/PCbuild/pythoncore.vcxproj
@@ -69,7 +69,7 @@
<PreprocessorDefinitions>_USRDLL;Py_BUILD_CORE;Py_ENABLE_SHARED;MS_DLL_ID="$(SysWinVer)";%(PreprocessorDefinitions)</PreprocessorDefinitions>
</ClCompile>
<Link>
- <AdditionalDependencies>ws2_32.lib;%(AdditionalDependencies)</AdditionalDependencies>
+ <AdditionalDependencies>shlwapi.lib;ws2_32.lib;%(AdditionalDependencies)</AdditionalDependencies>
<BaseAddress>0x1e000000</BaseAddress>
</Link>
</ItemDefinitionGroup>
diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py
--- a/Tools/msi/make_zip.py
+++ b/Tools/msi/make_zip.py
@@ -46,6 +46,10 @@
'python3stub',
}
+EXCLUDED_FILES = {
+ 'pyshellext',
+}
+
def is_not_debug(p):
if DEBUG_RE.search(p.name):
return False
@@ -53,7 +57,7 @@
if TKTCL_RE.search(p.name):
return False
- return p.stem.lower() not in DEBUG_FILES
+ return p.stem.lower() not in DEBUG_FILES and p.stem.lower() not in EXCLUDED_FILES
def is_not_debug_or_python(p):
return is_not_debug(p) and not PYTHON_DLL_RE.search(p.name)
@@ -209,8 +213,9 @@
copied = copy_to_layout(temp / t.rstrip('/'), rglob(s, p, c))
print('Copied {} files'.format(copied))
- with open(str(temp / 'pyvenv.cfg'), 'w') as f:
- print('applocal = true', file=f)
+ with open(str(temp / 'sys.path'), 'w') as f:
+ print('python{0.major}{0.minor}.zip'.format(sys.version_info), file=f)
+ print('.', file=f)
if out:
total = copy_to_layout(out, rglob(temp, '**/*', None))
--
Repository URL: https://hg.python.org/cpython
More information about the Python-checkins
mailing list